CheapFlights

From FarmShare

Revision as of 14:08, 14 September 2013 by Bishopj (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

cheap flights

Introduction

Povray, the Persistence of Vision raytracer provides a rather intuitive example of what can be done with the barley cluster. Povray provides a graphical representation of cpu cycles because the more complex the scene, the more cpu cycles it takes to render. We will explore this fact to help us learn how to think about breaking problems up into pieces which can be run on the barley cluster.

Poviogen.120.jpg

Povray, as of version 3.6 is still single threaded, just as it was in the very early '90's when the project began. Because of this, there are no specific technology requirements we are going rely on. HPC specific technologies do exist however, so be sure to watch for the next installment in this series where we will experiment with a parallel raytracer.

Povray, as it is run normally, will ingest a scene file and crunch on it for some minutes or hours and produce a .png image file. For the scene file in this example, this takes approx 15 minutes. What is interesting is that if you run povray with the "camera" in different locations you can generate a series of .png files, a "fly through", in what is called "embarrassingly parallel" in HPC. Embarrassingly parallel means that each povray job that is run shares no data with any other povray job. If a sufficient number of cores are available, all of these povray jobs can be executed at the same time.

Executive summary

Running povray to generate a single png file of a raytraced scene takes 15 minutes. Running 240 jobs to generate 10 seconds of high definition video of a "fly through" of the raytraced scene takes 15 minutes. Thats what you call a cheap flight.


Methodology

For this example there is really no need to assess cardinality, as we will decide on how many individual raytrace jobs to run by how smooth we want the final video. The way the povray scene file is constructed, the +K parameter moves the camera 180 degrees over the interval of 0 to 1. All we need to do then is choose how many frames we want and submit the jobs, making sure to increment +K by the same value, to ensure smooth camera motion.

Run the Jobs

Run following python script to submit 240 jobs - 10 seconds of video at 24 frames per second.

#!/usr/bin/python

import os
import datetime 

howmany = 240

startjob = datetime.datetime.now() 

for i in xrange(1, howmany+1): 
  num = i/float( howmany)
  job = 'qsub -N poviogen-%03d ~/poviogen.submit %.3f %03d' % (i, num, i)
  os.system(job)
  print job

endjob = datetime.datetime.now()

print 'time to submit all jobs: ', endjob-startjob


poviogen.submit

#!/bin/bash
#$ -cwd 
#$ -N povray
#$ -S /bin/bash

echo "Got $NSLOTS slots"
cat $PE_HOSTFILE

echo "clock: $1"

module load povray
povray Display=false Width=1900 Height=1080 +K$1 +A0.01 Output_File_Name=poviogen.$2.png \
/farmshare/examples/povray/povexample1.pov

Feel the Power

Poviogenmovie.gif

The povray jobs should all complete in about 15 minutes as long as there are enough available cores to run all of the individual jobs.

To see the flight in action, run this command on one of the corns to take the pile of .png's produced by povray and create a HD mpeg4 video:

mencoder mf://*.png -mf w=1900:h=1080:fps=24:type=png \
-ovc lavc -lavcopts vcodec=mpeg4:mbd=2:trell:aspect=16/9:vbitrate=16000 \
-oac copy -o cheapflights.avi
Personal tools
Toolbox
LANGUAGES