# CheapFlights

(Difference between revisions)
 Revision as of 00:11, 5 January 2012 (view source)Bishopj (Talk | contribs)← Older edit Revision as of 00:13, 5 January 2012 (view source)Bishopj (Talk | contribs) Newer edit → Line 11: Line 11: For this example there is really no need to assess cardinality, as we will decide on how many individual raytrace jobs to run by how smooth we want the final video. For this example there is really no need to assess cardinality, as we will decide on how many individual raytrace jobs to run by how smooth we want the final video. - +
-                                                                                                                                                                                                                                                                                                        +
-                                                                                                                                                                                                                                                                                                  +
#!/usr/bin/python                                                                                                                                                                                                                                                                                         #!/usr/bin/python

import os import datetime                                                                                                                                                                                                                                                                                 import os import datetime

-                                                                                         howmany = 24                                                                                                                                                                                                   +                                                                                          howmany = 240

startjob = datetime.datetime.now()                                                                                                                                                                                                                                                                        startjob = datetime.datetime.now()

for i in xrange(1, howmany+1):                                                                                                                                                                                                                                                                            for i in xrange(1, howmany+1):
+                                                                                          num = i/float( howmany)
+                                                                                          job = 'qsub -N poviogen-%03d ~/poviogen.submit %.3f %03d' % (i, num, i)
+                                                                                          os.system(job)
+                                                                                          print job

-                                                                                         num = i/float( howmany)                                                                                                                                                                                        +                                                                                          endjob = datetime.datetime.now()
-                                                                                         job = 'qsub -N poviogen-%03d ~/poviogen.submit %.3f %03d' % (i, num, i)                                                                                                                         +
-                                                                                         os.system(job)                                                                                                                                                                                                 +
-                                                                                         print job                                                                                                                                                                                                      +
-                                                                                                                                                                                                                                                                                                        +
-                                                                                         endjob = datetime.datetime.now()                                                                                                                                                                               +

print 'time to submit all jobs: ', endjob-startjob                                                                                                                                                                                                                                                        print 'time to submit all jobs: ', endjob-startjob
-                                                                                                                                                                                                                                                                                                 +
- - -
- +
#\$ -cwd                                                                                                                                                                                                                                                                                                   #\$ -cwd
#\$ -j y                                                                                                                                                                                                                                                                                                   #\$ -j y
#\$ -S /bin/bash                                                                                                                                                                                                                                                                                           #\$ -S /bin/bash

-
echo "Got \$NSLOTS slots" cat \$PE_HOSTFILE + echo "Got \$NSLOTS slots" + cat \$PE_HOSTFILE - echo "clock: \$1" id klist povray Display=false Width=400 Height=300 +K\$1 +A0.01 Output_File_Name=/mnt/glusterfs/bishopj/poviogen/poviogen.\$2.png /mnt/glusterfs/bishopj/poviogen/jasonfoo-100read-0random.pov + echo "clock: \$1" -
+ + povray Display=false Width=400 Height=300 +K\$1 +A0.01 Output_File_Name=/mnt/glusterfs/bishopj/poviogen/poviogen.\$2.png /mnt/glusterfs/bishopj/poviogen/jasonfoo-100read-0random.pov - mencoder mf://*.png -mf w=1900:h=1080:fps=24:type=png -ovc lavc -lavcopts vcodec=mpeg4:mbd=2:trell:aspect=16/9:vbitrate=16000 -oac copy -o output.avi + mencoder mf://*.png -mf w=1900:h=1080:fps=24:type=png -ovc lavc -lavcopts vcodec=mpeg4:mbd=2:trell:aspect=16/9:vbitrate=16000 -oac copy -o output.avi + +
-
[[Image:Poviogenmovie.gif|frame|400px]] + [[Image:Poviogenmovie.gif|frame|400px|Poviogenmovie.gif]] {{FILEPATH:Poviogenmovie.gif}} [[Media:Poviogenmovie.gif]] {{FILEPATH:Poviogenmovie.gif}} [[Media:Poviogenmovie.gif]] - [[Image:Poviogen.120.png|frame|400px]] + [[Image:Poviogen.120.png|frame|400px|Poviogen.120.png]]

## cheap flights

### Introduction

Povray, the Persistent of Vision raytracer provides a quite interesting example of what can be done with the barley cluster. Povray, as of version 3.6 is still single threaded, just as it was in the very early '90's when the project began. Additionally, povray provides a graphical representation of cpu cycles to help us learn how to think about break problems up into pieces which can be run on a cluster.

Povray as it is run normally will ingest a scene file and crunch on it for some minutes or hours and produce a .png image file. For the scene files in this example, this takes approx XX minutes. What is interesting is that if you run povray with the camera in different locations you can generate a series of .png files - a "fly through" - in what is called "embarrassingly parallel" in HPC. Embarrassingly parallel means that each job that is run shares no data with any other job. They can all be executed at the same time.

### Methodology

For this example there is really no need to assess cardinality, as we will decide on how many individual raytrace jobs to run by how smooth we want the final video.

```#!/usr/bin/python

import os import datetime

howmany = 240

startjob = datetime.datetime.now()

for i in xrange(1, howmany+1):
num = i/float( howmany)
job = 'qsub -N poviogen-%03d ~/poviogen.submit %.3f %03d' % (i, num, i)
os.system(job)
print job

endjob = datetime.datetime.now()

print 'time to submit all jobs: ', endjob-startjob
```

```#\$ -cwd
#\$ -j y
#\$ -S /bin/bash

echo "Got \$NSLOTS slots"
cat \$PE_HOSTFILE

echo "clock: \$1"

povray Display=false Width=400 Height=300 +K\$1 +A0.01 Output_File_Name=/mnt/glusterfs/bishopj/poviogen/poviogen.\$2.png /mnt/glusterfs/bishopj/poviogen/jasonfoo-100read-0random.pov </code>

<code></code>

<code>mencoder mf://*.png -mf w=1900:h=1080:fps=24:type=png -ovc lavc -lavcopts vcodec=mpeg4:mbd=2:trell:aspect=16/9:vbitrate=16000 -oac copy -o output.avi

```
Poviogenmovie.gif
Poviogen.120.png