CheapFlights

From FarmShare

Revision as of 14:08, 5 January 2012 by Bishopj (Talk | contribs)
Jump to: navigation, search

Contents

cheap flights

Introduction

Povray, the Persistent of Vision raytracer provides a rather intuitive example of what can be done with the barley cluster. Povray, as of version 3.6 is still single threaded, just as it was in the very early '90's when the project began. Povray also provides a graphical representation of cpu cycles to help us learn how to think about breaking problems up into pieces which can be run on a cluster. This particular aspect will be a recurring theme in the next installment in this series.

Povray, as it is run normally, will ingest a scene file and crunch on it for some minutes or hours and produce a .png image file. For the scene files in this example, this takes approx 15 minutes. What is interesting is that if you run povray with the "camera" in different locations you can generate a series of .png files, a "fly through", in what is called "embarrassingly parallel" in HPC. Embarrassingly parallel means that each job that is run shares no data with any other job. If a sufficient number of cores are available, they can all be executed at the same time.

Executive summary

Running povray to generate a single png file of a raytraced scene takes 15 minutes. Running 240 jobs to generate 10 seconds of high definition video of a "fly through" of the raytraced scene takes 15 minutes. Thats what you call a cheap flight.


Methodology

For this example there is really no need to assess cardinality, as we will decide on how many individual raytrace jobs to run by how smooth we want the final video. The way the povray scene file is constructed, the +K parameter moves the camera 180 degrees over the interval of 0 to 1. All we need to do then is choose how many frames we want and submit the jobs, making sure to increment +K by the same value, to ensure smooth camera motion.

Run the Jobs

Run following python script to submit 240 jobs - 10 seconds of video at 24 frames per second.

#!/usr/bin/python

import os
import datetime 

howmany = 240

startjob = datetime.datetime.now() 

for i in xrange(1, howmany+1): 
  num = i/float( howmany)
  job = 'qsub -N poviogen-%03d ~/poviogen.submit %.3f %03d' % (i, num, i)
  os.system(job)
  print job

endjob = datetime.datetime.now()

print 'time to submit all jobs: ', endjob-startjob


poviogen.submit

#$ -cwd 
#$ -j y 
#$ -S /bin/bash

echo "Got $NSLOTS slots"
cat $PE_HOSTFILE

echo "clock: $1"

povray Display=false Width=1900 Height=1080 +K$1 +A0.01 Output_File_Name=/mnt/glusterfs/$USER/poviogen.$2.png /mnt/glusterfs/bishopj/barley-examples/povexample1.pov

Feel the Power

The povray jobs should all complete in about 15 minutes as long as there are enough available cores to run all of the individual jobs. Running povray once also takes 15 minutes.

To see the flight in action, run this command on one of the corns to take the pile of .png's produced by povray and create a HD mpeg4 video:

mencoder mf://*.png -mf w=1900:h=1080:fps=24:type=png -ovc lavc -lavcopts vcodec=mpeg4:mbd=2:trell:aspect=16/9:vbitrate=16000 -oac copy -o cheapflights.avi


click here for animated gif: http://stanford.edu/~bishopj/poviogenmovie.gif

click here for single frame: http://stanford.edu/~bishopj/poviogen.120.png


Poviogenmovie.gif

https://web.stanford.edu/group/farmshare/wiki/uploads/f/f7/Poviogenmovie.gif Media:Poviogenmovie.gif

Poviogen.120.png

Poviogen.120.jpg

Personal tools
Toolbox
LANGUAGES