SFOtoHKGin5min

From FarmShare

(Difference between revisions)
Jump to: navigation, search
m (→Assessing Communication Overhead (was Assessing Cardinality))
 
(18 intermediate revisions not shown)
Line 3: Line 3:
=== Introduction  ===
=== Introduction  ===
-
This is a followon article to [[CheapFlights]], and while the metaphor may be showing stress cracks, please bear with me. In the previous article we made the most of a single threaded, and hence single core program, by taking advantage of the embarrassingly parallel nature of moving the camera. The time taken to render any given frame, however, was completely unchanged from running povray directly on a corn. To break the 15 minutes barrier (for this particular scene file) we need to employee an HPC specific technology - MPI.  
+
This is a followon article to [[CheapFlights]], and while the metaphor may be showing stress cracks, please bear with me. In the previous article we made the most of a single threaded, and hence single core program, by taking advantage of the embarrassingly parallel nature of moving the camera. The time taken to render any given frame, however, was completely unchanged from running povray directly on a corn. To break the 15 minute barrier (for this particular scene file) we need to employee an HPC specific technology - MPI.  
-
MPI stands for Message Passing Interface and is typically used as a library to a compiled language (C/C++/Fortran) or an interpreted (byte compiled actually) language such as Python. In this example we will explore a parallel raytracer called Tachyon which uses MPI as one of the available parallel options. MPI allows Tachyon access to the distributed compute and memory of all cores participating in the job. It is up to the code in Tachyon, then, to decide how to break up the task of rendering a frame such that all of the cores in the job can execute their portion and communicate their results back. This architectural decision has been made in the code of Tachyon, and we are free to run jobs on the barley cluster using as few or many cores as long as the number fits within those constraints. This number we choose becomes the size of our Tachyon job we submit to the scheduler.  
+
[[Image:Stmvao-white short.png|frame|Stmvao-white short.png]]
 +
 
 +
MPI stands for Message Passing Interface and is typically used as a library to a compiled language (C/C++/Fortran) or an interpreted (byte compiled actually) language such as Python. In this example we will explore a parallel raytracer called Tachyon which uses MPI as one of the available parallel options. MPI allows Tachyon access to the distributed compute and memory of all cores participating in the job. It is up to the code in Tachyon, then, to decide how to break up the task of rendering a frame such that all of the cores in the job can execute their portion and communicate their results back. This architectural decision has been made in the code of Tachyon, and we are free to run jobs on the barley cluster using as few or many cores as long as the number fits within those constraints. This number we choose becomes the size of our Tachyon job we submit to the scheduler.  
Let's explore what kind of speedup Tachyon can achieve on the barley cluster. This cluster is 10 gigabit ethernet connected, which plays an important part, as all of the intermediate processing steps are communicated within all nodes participating in the job.  
Let's explore what kind of speedup Tachyon can achieve on the barley cluster. This cluster is 10 gigabit ethernet connected, which plays an important part, as all of the intermediate processing steps are communicated within all nodes participating in the job.  
 +
<br>
=== Executive Summary  ===
=== Executive Summary  ===
-
We explore the possibilites presented with a parallel raytracer (Tachyon) using OpenMPI (an MPI library). Given a sample scene file, a singe core job (similar to the POVray example) takes 8.8 minutes to render a single frame. Rendering the same scene file using 208 cores as an MPI job takes 3.4 seconds. This is a speedup of 156 times. This is like reducing the flight to Hong Kong from 13hours to 5 minutes (also 156 times).  
+
We explore the possibilites presented by a parallel raytracer (Tachyon) using OpenMPI (an MPI library). Given a sample scene file, a singe core job takes 8.8 minutes to render a single frame. Rendering the same scene file as a 208 core MPI job on the barley cluster takes 3.4 seconds. This is a speedup of 156 times - the same as reducing the flight time from San Francisco to Hong Kong from 13 hours to 5 minutes.  
 +
 
 +
<br>
== Methodology  ==
== Methodology  ==
-
=== Assessing Network Overhead (was Assessing Cardinality) ===
+
=== Assessing Communication Overhead (was Assessing Cardinality) ===
 +
 
 +
For embarrassingly parallel problems, we considered the cardinality of the problem in order to chop the problem up into pieces which can be run on a cluster of finite size. When MPI is used, one needs to consider the constraints of the communication between the nodes participating in the job.
 +
 
 +
'''Running on a Single core'''
 +
 
 +
There is no particular requirement in the MPI library to use more than one core. So let us get a baseline time measurement by running Tachyon on a single core. Looking at the qsub script below there is now an additional parameter to qsub (-pe orte 1) and the use of mpirun to launch tachyon.
 +
<source lang="sh">#
 +
#$ -cwd
 +
#$ -j y
 +
#$ -S /bin/bash
 +
#$ -N tachyon1
 +
#$ -pe orte 1
 +
 
 +
tmphosts=`mktemp`
 +
awk '{ for (i=0; i < $2; ++i) { print $1} }' $PE_HOSTFILE > $tmphosts
 +
 
 +
echo "Got $NSLOTS slots"
 +
echo "jobid $JOB_ID"
 +
 
 +
mpirun -np $NSLOTS -machinefile $tmphosts /mnt/glusterfs/bishopj/tachyon/tachyon/compile/linux-mpi/tachyon +V -aasamples 4 \
 +
-trans_vmd /mnt/glusterfs/bishopj/tachyon/stmvao-white.dat -o /mnt/glusterfs/$USER/stmvao-white_short.$JOB_ID.bmp \
 +
-format BMP -rescale_lights 0.4 -add_skylight 0.9 -skylight_samples 32 -res 1900 1080
 +
 
 +
</source>
 +
 
 +
<br>
 +
<pre>$ qsub tachyon.submit
 +
</pre> <pre>
 +
1 core:
 +
CPU Information:
 +
Node    0:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
Total CPUs: 1
 +
Total Speed: 1.000000
 +
Preprocessing Time:  1.0531 seconds
 +
Rendering Progress:  100% complete
 +
Ray Tracing Time:  533.7124 seconds
 +
Image I/O Time:    0.0754 seconds
 +
 
 +
</pre>
 +
<br> '''Scaling the job up'''
 +
 
 +
Change the -pe orte 1 to -pe orte 8 and launch the job again.
 +
<pre>$ qsub tachyon.submit
 +
</pre> <pre>CPU Information:
 +
 
 +
Node    0:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
Node    1:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
Node    2:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
Node    3:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
Node    4:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
Node    5:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
Node    6:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
Node    7:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
Total CPUs: 8
 +
Total Speed: 8.000000
 +
Preprocessing Time:    1.2570 seconds
 +
Rendering Progress:      100% complete
 +
Ray Tracing Time:    76.1758 seconds
 +
Image I/O Time:    0.0695 seconds
 +
 
 +
</pre>
 +
If we take the 76 second result and calculate how much speedup we observe per core we get 87% (533/76.17)/8 = .87. All of the cores were on the same system system, so 87% would indicate the inter-core protocol efficiency of MPI. It is not 100%, but 87% is not a bad number.
 +
 
 +
Let's examine what happens when we cross a physical system boundary and need to communicate over ethernet network:
 +
 
 +
<br>
 +
<pre>CPU Information:
 +
  Node    0:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    1:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    2:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    3:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    4:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    5:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    6:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    7:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    8:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    9:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node  10:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node  11:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node  12:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node  13:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node  14:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node  15:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node  16:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  17:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  18:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  19:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  20:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  21:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  22:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  23:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  24:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  25:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  26:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  27:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  28:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  29:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  30:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Node  31:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley03.stanford.edu
 +
  Total CPUs: 32
 +
  Total Speed: 32.000000
 +
Allocating Image Buffer.
 +
Preprocessing Time:    1.3596 seconds
 +
Rendering Progress:      100% complete
 +
  Ray Tracing Time:    21.1898 seconds
 +
    Image I/O Time:    0.0761 seconds
 +
</pre>
 +
 
 +
As you can see in the Node list, barley01 and barley03 are being used. In this case, our efficiency per core is: (533/21.19)/32 = .78 or 78%. Approximately a 10% hit by going over the 10 gigabit ethernet. However, from a speedup per added core standpoint we are still definitely in the money. Every added core results in a significant speedup.
 +
 
 +
Let us scale up further, to 208 cores:
 +
<source lang="sh">#
 +
#$ -cwd
 +
#$ -j y
 +
#$ -S /bin/bash
 +
#$ -N tachyon208
 +
#$ -pe orte 208
 +
 
 +
tmphosts=`mktemp`
 +
awk '{ for (i=0; i < $2; ++i) { print $1} }' $PE_HOSTFILE > $tmphosts
 +
 
 +
echo "Got $NSLOTS slots"
 +
echo "jobid $JOB_ID"
 +
 
 +
mpirun -np $NSLOTS -machinefile $tmphosts /mnt/glusterfs/bishopj/tachyon/tachyon/compile/linux-mpi/tachyon +V \
 +
-aasamples 4 -trans_vmd /mnt/glusterfs/bishopj/tachyon/stmvao-white.dat \
 +
-o /mnt/glusterfs/$USER/stmvao-white_short.$JOB_ID.bmp -format BMP -rescale_lights 0.4 -add_skylight 0.9 \
 +
-skylight_samples 32 -res 1900 1080
 +
 
 +
</source>
 +
<br>
 +
 
 +
<br>
 +
<pre>CPU Information:
 +
  Node    0:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    1:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    2:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    3:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    4:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    5:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    6:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    7:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    8:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node    9:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
  Node  10:  1 CPUs, CPU Speed 1.00, Node Speed  1.00 Name: barley01.stanford.edu
 +
.
 +
.
 +
.
 +
  Total CPUs: 208
 +
  Total Speed: 208.000000
 +
 
 +
Preprocessing Time:    1.5401 seconds
 +
Rendering Progress:      100% complete           
 +
  Ray Tracing Time:    3.4819 seconds
 +
    Image I/O Time:    0.0680 seconds
 +
</pre>
 +
Calculating the efficiency, we get (533/3.4)/208 = .75 or 75% speedup per added core. Observe that at 208 cores and 3.4 seconds, the standup and tear down time of the processes themselves is starting to play a large role. I would say this scalability experiment is very successful, having reduced the time to render a single frame from 8.8 minutes to 3.4 seconds, a factor of 156 times.
 +
 
 +
== building Tachyon ==
 +
 
 +
Let's see what our options are:
 +
 
 +
<source lang="sh">
 +
bishopj@corn11:/mnt/glusterfs/bishopj/tachyon/tachyon/unix$ make
 +
  Choose one of the architectures specified below.
 +
--------------------------------------------------------------
 +
              Parallel Versions       
 +
 
 +
            aix-thr - IBM AIX 5.x      POSIX Threads       
 +
          aix-64-thr - IBM AIX 5.x      POSIX Threads, 64-bit
 +
            aix-mpi - IBM AIX 5.x (SP) MPI                 
 +
        asci-red-mpi - Intel ASCI Red  MPI                 
 +
        bluegene-mpi - IBM Blue Gene    MPI                 
 +
            cray-thr - Cray J90,C90...  POSIX Threads       
 +
        cray-t3e-mpi - Cray T3E        MPI                 
 +
        cray-xt3-mpi - Cray XT3        MPI                 
 +
  compaq-alphasc-mpi - Lemieux at PSC  MPI                 
 +
            hpux-thr - HP/UX 11.x      POSIX Threads       
 +
      hpux-ia64-thr - HP-UX IA-64, HP C, POSIX Threads     
 +
          irix6-thr - SGI IRIX 6.5.x  POSIX Threads       
 +
        irix6-64-thr - SGI IRIX 6.5.x  POSIX Threads, 64-bit
 +
  linux-beowulf-mpi - Scyld Linux MPI                       
 +
linux-alpha-ccc-qsw - Linux Alpha, Compaq C, MPI, QSWnet   
 +
          linux-lam - Linux            MPI (OSC LAM)       
 +
        linux-lam-64 - Linux AMD64/EM64T, MPI, 64-bit       
 +
        linux-64-thr - Linux AMD64/EM64T, POSIX Threads, 64-bit
 +
          linux-mpi - Linux            Generic MPI         
 +
          linux-thr - Linux            POSIX Threads       
 +
    linux-p4-icc-thr - Linux, Intel C compilers, P4-optimized
 +
      linux-ia64-thr - Linux IA-64, GCC, POSIX Threads       
 +
linux-ia64-thr-sgicc - Linux IA-64, SGI Pro64 Compilers     
 +
          macosx-thr - MacOS X PowerPC, POSIX Threads       
 +
      macosx-x86-thr - MacOS X Intel x86, POSIX Threads     
 +
  solaris-ultra-hpc - Sun Solaris 9/10 ClusterTools 4.0 MPI 
 +
        solaris-mpi - Sun Solaris 9/10 MPI (MPICH)         
 +
        solaris-lam - Sun Solaris 9/10 MPI (OSC LAM)       
 +
        solaris-thr - Sun Solaris 9/10 UI/Sun Threads       
 +
  solaris-ultra-thr - Sun Solaris 9/10 UI/Sun Threads, US-1/2
 +
  solaris-ultra3-thr - Sun Solaris 9/10 UI/Sun Threads, US-III
 +
      solaris-64-thr - Sun Solaris 9/10 UI/Sun Threads, 64-bit
 +
--------------------------------------------------------------
 +
            Hybrid Parallel Versions                         
 +
 
 +
      linux-mpi-thr - Linux    Generic MPI+POSIX Threads   
 +
 
 +
--------------------------------------------------------------
 +
              Sequential Versions                           
 +
 
 +
              aix - IBM AIX 5.x                           
 +
              bsd - OpenBSD/FreeBSD/NetBSD                 
 +
            hpux - HP/UX 11.x                             
 +
            irix6 - SGI Irix 6.x                           
 +
            linux - Linux                                 
 +
        linux-64 - Linux, AMD64/EM64T, GCC 3.x, 64-bit   
 +
    linux-p4-icc - Linux, Intel C compilers, P4-optimized 
 +
    linux-athlon - Linux using GCC 3.x Athlon optimizations
 +
        linux-p4 - Linux using GCC 3.x P4 optimizations   
 +
      linux-pgcc - Linux, Portland Group Compilers, Pentium
 +
linux-athlon-pgcc - Linux, Portland Group Compilers, Athlon
 +
        linux-ppc - Linux PowerPC                         
 +
        linux-ps2 - Linux Sony Playstation 2               
 +
      linux-alpha - Linux Alpha                           
 +
  linux-alpha-ccc - Linux Alpha, Compaq Compilers         
 +
      linux-ia64 - Linux IA-64, GCC                       
 +
linux-ia64-sgicc - Linux IA-64, SGI Pro64 Compilers       
 +
          macosx - MacOS X (aka Darwin, Rhapsody)         
 +
      tru64-alpha - Tru64 Unix, binary can run on AlphaLinux
 +
          solaris - Sun Solaris 8/9/10                     
 +
            win32 - Windows 95/NT with Cygnus EGCS/CygWin32
 +
 
 +
--------------------------------------------------------------
 +
              OpenGL Runtime Display Versions               
 +
 
 +
        irix6-thr-ogl - SGI IRIX 6.5.x, POSIX Threads, OpenGL
 +
solaris-ultra-thr-ogl - Sun Solaris 9/10 UI/Sun Threads, OpenGL
 +
solaris-ultra-hpc-ogl - Sun Solaris 9/10 Sun MPI, OpenGL     
 +
  solaris-64-thr-ogl - Sun Solaris 9/10 UI/Sun Threads, 64-bit
 +
        linux-thr-ogl - Linux POSIX Threads, OpenGL         
 +
    linux-64-thr-ogl - Linux POSIX Threads, OpenGL         
 +
    linux-lam-64-ogl - Linux, 64-bit AMD64/EM64T, LAM MPI, OpenGL
 +
linux-beowulf-mpi-ogl - Scyld Linux with MPI, OpenGL         
 +
  macosx-x86-thr-ogl - MacOS X Intel x86, POSIX Threads, OpenGL
 +
  tru64-alpha-thr-ogl - Tru64/OSF1/DUnix POSIX Threads, OpenGL
 +
 
 +
--------------------------------------------------------------
 +
Type: 'make arch' to build for an architecture listed above.
 +
</source>
 +
 
 +
We will choose linux-mpi.
 +
 
 +
<source lang="sh">
 +
bishopj@corn11:/mnt/glusterfs/bishopj/tachyon/tachyon/unix$ make linux-mpi
 +
make all \
 +
"ARCH = linux-mpi" \
 +
"CC = mpicc" \
 +
"CFLAGS = -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI      " \
 +
"AR = ar" \
 +
"ARFLAGS = r" \
 +
"STRIP = strip" \
 +
"RANLIB = ranlib" \
 +
"LIBS = -L. -ltachyon    -lm"
 +
make[1]: Entering directory `/mnt/glusterfs/bishopj/tachyon/tachyon/unix'
 +
Building Tachyon Parallel Ray Tracing library
 +
Copyright 1994-2007, John E. Stone
 +
All Rights Reserveed
 +
Making architecture directory ../compile/linux-mpi
 +
Making library directory ../compile/linux-mpi/libtachyon
 +
make ../compile ../compile/linux-mpi ../compile/linux-mpi/libtachyon  ../compile/linux-mpi/libtachyon.a  ../compile/linux-mpi/tachyon
 +
make[2]: Entering directory `/mnt/glusterfs/bishopj/tachyon/tachyon/unix'
 +
make[2]: Nothing to be done for `../compile'.
 +
make[2]: Nothing to be done for `../compile/linux-mpi'.
 +
make[2]: Nothing to be done for `../compile/linux-mpi/libtachyon'.
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/api.c -o ../compile/linux-mpi/libtachyon/api.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/apigeom.c -o ../compile/linux-mpi/libtachyon/apigeom.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/box.c -o ../compile/linux-mpi/libtachyon/box.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/global.c -o ../compile/linux-mpi/libtachyon/global.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/hash.c -o ../compile/linux-mpi/libtachyon/hash.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/parallel.c -o ../compile/linux-mpi/libtachyon/parallel.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/threads.c -o ../compile/linux-mpi/libtachyon/threads.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/camera.c -o ../compile/linux-mpi/libtachyon/camera.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/render.c -o ../compile/linux-mpi/libtachyon/render.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/trace.c -o ../compile/linux-mpi/libtachyon/trace.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/grid.c -o ../compile/linux-mpi/libtachyon/grid.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/intersect.c -o ../compile/linux-mpi/libtachyon/intersect.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/sphere.c -o ../compile/linux-mpi/libtachyon/sphere.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/plane.c -o ../compile/linux-mpi/libtachyon/plane.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/ring.c -o ../compile/linux-mpi/libtachyon/ring.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/triangle.c -o ../compile/linux-mpi/libtachyon/triangle.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/cylinder.c -o ../compile/linux-mpi/libtachyon/cylinder.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/quadric.c -o ../compile/linux-mpi/libtachyon/quadric.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/extvol.c -o ../compile/linux-mpi/libtachyon/extvol.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/vol.c -o ../compile/linux-mpi/libtachyon/vol.o
 +
../src/vol.c: In function 'LoadVol':
 +
../src/vol.c:294:8: warning: ignoring return value of 'fread', declared with attribute warn_unused_result [-Wunused-result]
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/ui.c -o ../compile/linux-mpi/libtachyon/ui.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/util.c -o ../compile/linux-mpi/libtachyon/util.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/vector.c -o ../compile/linux-mpi/libtachyon/vector.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/shade.c -o ../compile/linux-mpi/libtachyon/shade.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/texture.c -o ../compile/linux-mpi/libtachyon/texture.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/coordsys.c -o ../compile/linux-mpi/libtachyon/coordsys.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/imap.c -o ../compile/linux-mpi/libtachyon/imap.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/light.c -o ../compile/linux-mpi/libtachyon/light.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/imageio.c -o ../compile/linux-mpi/libtachyon/imageio.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/jpeg.c -o ../compile/linux-mpi/libtachyon/jpeg.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/pngfile.c -o ../compile/linux-mpi/libtachyon/pngfile.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/ppm.c -o ../compile/linux-mpi/libtachyon/ppm.o
 +
../src/ppm.c: In function 'readppm':
 +
../src/ppm.c:47:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../src/ppm.c:57:8: warning: ignoring return value of 'fread', declared with attribute warn_unused_result [-Wunused-result]
 +
../src/ppm.c: In function 'getint':
 +
../src/ppm.c:28:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../src/ppm.c:30:14: warning: ignoring return value of 'fgets', declared with attribute warn_unused_result [-Wunused-result]
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/psd.c -o ../compile/linux-mpi/libtachyon/psd.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/sgirgb.c -o ../compile/linux-mpi/libtachyon/sgirgb.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/tgafile.c -o ../compile/linux-mpi/libtachyon/tgafile.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/winbmp.c -o ../compile/linux-mpi/libtachyon/winbmp.o
 +
rm -f ../compile/linux-mpi/libtachyon.a
 +
ar r ../compile/linux-mpi/libtachyon.a ../compile/linux-mpi/libtachyon/api.o ../compile/linux-mpi/libtachyon/apigeom.o ../compile/linux-mpi/libtachyon/box.o ../compile/linux-mpi/libtachyon/global.o ../compile/linux-mpi/libtachyon/hash.o ../compile/linux-mpi/libtachyon/parallel.o ../compile/linux-mpi/libtachyon/threads.o ../compile/linux-mpi/libtachyon/camera.o ../compile/linux-mpi/libtachyon/render.o ../compile/linux-mpi/libtachyon/trace.o ../compile/linux-mpi/libtachyon/grid.o ../compile/linux-mpi/libtachyon/intersect.o ../compile/linux-mpi/libtachyon/sphere.o ../compile/linux-mpi/libtachyon/plane.o ../compile/linux-mpi/libtachyon/ring.o ../compile/linux-mpi/libtachyon/triangle.o ../compile/linux-mpi/libtachyon/cylinder.o ../compile/linux-mpi/libtachyon/quadric.o ../compile/linux-mpi/libtachyon/extvol.o ../compile/linux-mpi/libtachyon/vol.o ../compile/linux-mpi/libtachyon/ui.o ../compile/linux-mpi/libtachyon/util.o ../compile/linux-mpi/libtachyon/vector.o ../compile/linux-mpi/libtachyon/shade.o ../compile/linux-mpi/libtachyon/texture.o ../compile/linux-mpi/libtachyon/coordsys.o ../compile/linux-mpi/libtachyon/imap.o ../compile/linux-mpi/libtachyon/light.o ../compile/linux-mpi/libtachyon/imageio.o ../compile/linux-mpi/libtachyon/jpeg.o ../compile/linux-mpi/libtachyon/pngfile.o ../compile/linux-mpi/libtachyon/ppm.o ../compile/linux-mpi/libtachyon/psd.o ../compile/linux-mpi/libtachyon/sgirgb.o ../compile/linux-mpi/libtachyon/tgafile.o ../compile/linux-mpi/libtachyon/winbmp.o
 +
ar: creating ../compile/linux-mpi/libtachyon.a
 +
ranlib ../compile/linux-mpi/libtachyon.a
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/main.c -o ../compile/linux-mpi/libtachyon/main.o
 +
../demosrc/main.c: In function 'animate_scene':
 +
../demosrc/main.c:392:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/getargs.c -o ../compile/linux-mpi/libtachyon/getargs.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/parse.c -o ../compile/linux-mpi/libtachyon/parse.o
 +
../demosrc/parse.c:1468:16: warning: 'GetSphereArray' defined but not used [-Wunused-function]
 +
../demosrc/parse.c: In function 'GetTexBody':
 +
../demosrc/parse.c:794:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:798:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:802:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:806:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:809:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:811:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:815:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:819:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:821:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:822:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:826:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:837:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:839:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:840:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:854:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:862:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:885:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:895:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:909:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetString':
 +
../demosrc/parse.c:212:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'PrintSyntaxError':
 +
../demosrc/parse.c:189:10: warning: ignoring return value of 'fread', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'readmodel':
 +
../demosrc/parse.c:242:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetScenedefs':
 +
../demosrc/parse.c:321:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetObject':
 +
../demosrc/parse.c:592:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetTexture':
 +
../demosrc/parse.c:765:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetVCSTri':
 +
../demosrc/parse.c:1444:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1450:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1456:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetVertexArray':
 +
../demosrc/parse.c:1560:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1566:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1568:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1578:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1580:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1598:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1638:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1705:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1766:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetSphere':
 +
../demosrc/parse.c:1279:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetFCylinder':
 +
../demosrc/parse.c:1230:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetRing':
 +
../demosrc/parse.c:1356:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1358:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetPolyCylinder':
 +
../demosrc/parse.c:1248:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1257:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetCylinder':
 +
../demosrc/parse.c:1201:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetVol':
 +
../demosrc/parse.c:1318:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1320:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetImageDef':
 +
../demosrc/parse.c:665:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:689:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetTexDef':
 +
../demosrc/parse.c:743:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetTexAlias':
 +
../demosrc/parse.c:753:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:754:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetLight':
 +
../demosrc/parse.c:983:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:986:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:988:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1000:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1003:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1006:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetColor':
 +
../demosrc/parse.c:650:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetDirLight':
 +
../demosrc/parse.c:956:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:958:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetSkyLight':
 +
../demosrc/parse.c:1028:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1031:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetSpotLight':
 +
../demosrc/parse.c:1054:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1060:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1063:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1066:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1068:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1079:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1082:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1085:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetLandScape':
 +
../demosrc/parse.c:1790:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1793:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetShaderMode':
 +
../demosrc/parse.c:334:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:349:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:358:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:377:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:380:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:383:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetCamera':
 +
../demosrc/parse.c:404:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:406:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:415:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:418:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:426:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:429:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:437:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:441:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:444:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:447:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:453:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:459:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:467:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:469:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:471:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetTPolyFile':
 +
../demosrc/parse.c:1835:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1845:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetBackGnd':
 +
../demosrc/parse.c:1135:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetBackGndGradient':
 +
../demosrc/parse.c:1152:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1166:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1169:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1172:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1175:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1178:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c: In function 'GetFog':
 +
../demosrc/parse.c:1103:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1115:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1118:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/parse.c:1121:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/nffparse.c -o ../compile/linux-mpi/libtachyon/nffparse.o
 +
../demosrc/nffparse.c: In function 'NFFGetPatch':
 +
../demosrc/nffparse.c:246:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c: In function 'NFFGetPolygon':
 +
../demosrc/nffparse.c:227:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c: In function 'NFFGetSphere':
 +
../demosrc/nffparse.c:215:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c: In function 'NFFGetCylCone':
 +
../demosrc/nffparse.c:193:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c:196:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c: In function 'NFFGetTexture':
 +
../demosrc/nffparse.c:171:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c: In function 'NFFGetLight':
 +
../demosrc/nffparse.c:157:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c: In function 'NFFGetBackground':
 +
../demosrc/nffparse.c:138:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c: In function 'NFFGetScenedefs':
 +
../demosrc/nffparse.c:109:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c:114:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c:117:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c: In function 'NFFGetVector':
 +
../demosrc/nffparse.c:59:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/nffparse.c: In function 'NFFGetString':
 +
../demosrc/nffparse.c:47:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/glwin.c -o ../compile/linux-mpi/libtachyon/glwin.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/spaceball.c -o ../compile/linux-mpi/libtachyon/spaceball.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/trackball.c -o ../compile/linux-mpi/libtachyon/trackball.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc  -c ../demosrc/mgfparse.c -o ../compile/linux-mpi/libtachyon/mgfparse.o
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/ac3dparse.c -o ../compile/linux-mpi/libtachyon/ac3dparse.o
 +
../demosrc/ac3dparse.c: In function 'ParseAC3D':
 +
../demosrc/ac3dparse.c:105:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c: In function 'GetMaterial':
 +
../demosrc/ac3dparse.c:207:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:210:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:217:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:221:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:224:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:228:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:231:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c: In function 'GetString':
 +
../demosrc/ac3dparse.c:82:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c: In function 'GetRecurseObject':
 +
../demosrc/ac3dparse.c:398:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:399:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:402:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:405:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:406:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:409:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:410:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:413:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:414:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:419:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:422:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:426:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:428:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:433:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:436:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:447:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:448:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:452:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:457:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:467:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:475:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:478:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:479:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:492:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:497:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:500:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:501:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:504:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:510:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:542:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
../demosrc/ac3dparse.c:566:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
 +
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -o ../compile/linux-mpi/tachyon ../compile/linux-mpi/libtachyon/main.o ../compile/linux-mpi/libtachyon/getargs.o ../compile/linux-mpi/libtachyon/parse.o ../compile/linux-mpi/libtachyon/nffparse.o ../compile/linux-mpi/libtachyon/glwin.o ../compile/linux-mpi/libtachyon/spaceball.o ../compile/linux-mpi/libtachyon/trackball.o ../compile/linux-mpi/libtachyon/mgfparse.o ../compile/linux-mpi/libtachyon/ac3dparse.o  -L../compile/linux-mpi  -L. -ltachyon    -lm
 +
strip ../compile/linux-mpi/tachyon
 +
make[2]: Leaving directory `/mnt/glusterfs/bishopj/tachyon/tachyon/unix'
 +
 
 +
Build Complete
 +
make[1]: Leaving directory `/mnt/glusterfs/bishopj/tachyon/tachyon/unix'
 +
</source>
 +
 
 +
 
 +
The important thing to look for is if the build is using mpicc (or mpif90 for fortran90).  It did, and the build was successful

Latest revision as of 15:57, 10 February 2016

Contents

San Francisco to Hong Kong in 5 minutes

Introduction

This is a followon article to CheapFlights, and while the metaphor may be showing stress cracks, please bear with me. In the previous article we made the most of a single threaded, and hence single core program, by taking advantage of the embarrassingly parallel nature of moving the camera. The time taken to render any given frame, however, was completely unchanged from running povray directly on a corn. To break the 15 minute barrier (for this particular scene file) we need to employee an HPC specific technology - MPI.

Stmvao-white short.png

MPI stands for Message Passing Interface and is typically used as a library to a compiled language (C/C++/Fortran) or an interpreted (byte compiled actually) language such as Python. In this example we will explore a parallel raytracer called Tachyon which uses MPI as one of the available parallel options. MPI allows Tachyon access to the distributed compute and memory of all cores participating in the job. It is up to the code in Tachyon, then, to decide how to break up the task of rendering a frame such that all of the cores in the job can execute their portion and communicate their results back. This architectural decision has been made in the code of Tachyon, and we are free to run jobs on the barley cluster using as few or many cores as long as the number fits within those constraints. This number we choose becomes the size of our Tachyon job we submit to the scheduler.

Let's explore what kind of speedup Tachyon can achieve on the barley cluster. This cluster is 10 gigabit ethernet connected, which plays an important part, as all of the intermediate processing steps are communicated within all nodes participating in the job.


Executive Summary

We explore the possibilites presented by a parallel raytracer (Tachyon) using OpenMPI (an MPI library). Given a sample scene file, a singe core job takes 8.8 minutes to render a single frame. Rendering the same scene file as a 208 core MPI job on the barley cluster takes 3.4 seconds. This is a speedup of 156 times - the same as reducing the flight time from San Francisco to Hong Kong from 13 hours to 5 minutes.


Methodology

Assessing Communication Overhead (was Assessing Cardinality)

For embarrassingly parallel problems, we considered the cardinality of the problem in order to chop the problem up into pieces which can be run on a cluster of finite size. When MPI is used, one needs to consider the constraints of the communication between the nodes participating in the job.

Running on a Single core

There is no particular requirement in the MPI library to use more than one core. So let us get a baseline time measurement by running Tachyon on a single core. Looking at the qsub script below there is now an additional parameter to qsub (-pe orte 1) and the use of mpirun to launch tachyon. 

#
#$ -cwd
#$ -j y
#$ -S /bin/bash
#$ -N tachyon1
#$ -pe orte 1

tmphosts=`mktemp`
awk '{ for (i=0; i < $2; ++i) { print $1} }' $PE_HOSTFILE > $tmphosts

echo "Got $NSLOTS slots"
echo "jobid $JOB_ID"

mpirun -np $NSLOTS -machinefile $tmphosts /mnt/glusterfs/bishopj/tachyon/tachyon/compile/linux-mpi/tachyon +V -aasamples 4 \
 -trans_vmd /mnt/glusterfs/bishopj/tachyon/stmvao-white.dat -o /mnt/glusterfs/$USER/stmvao-white_short.$JOB_ID.bmp \
-format BMP -rescale_lights 0.4 -add_skylight 0.9 -skylight_samples 32 -res 1900 1080



$ qsub tachyon.submit

1 core:
CPU Information:



Node    0:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
Total CPUs: 1
Total Speed: 1.000000
Preprocessing Time:   1.0531 seconds
Rendering Progress:   100% complete
Ray Tracing Time:   533.7124 seconds
Image I/O Time:     0.0754 seconds


Scaling the job up

Change the -pe orte 1 to -pe orte 8 and launch the job again. 

$ qsub tachyon.submit

CPU Information:

Node    0:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
Node    1:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
Node    2:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
Node    3:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
Node    4:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
Node    5:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
Node    6:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
Node    7:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
Total CPUs: 8
Total Speed: 8.000000
Preprocessing Time:     1.2570 seconds
Rendering Progress:       100% complete
Ray Tracing Time:    76.1758 seconds
Image I/O Time:     0.0695 seconds

If we take the 76 second result and calculate how much speedup we observe per core we get 87% (533/76.17)/8 = .87. All of the cores were on the same system system, so 87% would indicate the inter-core protocol efficiency of MPI. It is not 100%, but 87% is not a bad number.

Let's examine what happens when we cross a physical system boundary and need to communicate over ethernet network: 


CPU Information:
  Node    0:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    1:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    2:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    3:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    4:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    5:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    6:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    7:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    8:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    9:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node   10:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node   11:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node   12:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node   13:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node   14:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node   15:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node   16:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   17:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   18:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   19:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   20:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   21:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   22:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   23:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   24:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   25:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   26:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   27:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   28:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   29:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   30:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Node   31:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley03.stanford.edu
  Total CPUs: 32
  Total Speed: 32.000000
Allocating Image Buffer.
Preprocessing Time:     1.3596 seconds
Rendering Progress:       100% complete
  Ray Tracing Time:    21.1898 seconds
    Image I/O Time:     0.0761 seconds

As you can see in the Node list, barley01 and barley03 are being used. In this case, our efficiency per core is: (533/21.19)/32 = .78 or 78%. Approximately a 10% hit by going over the 10 gigabit ethernet. However, from a speedup per added core standpoint we are still definitely in the money. Every added core results in a significant speedup.

Let us scale up further, to 208 cores: 

#
#$ -cwd
#$ -j y
#$ -S /bin/bash
#$ -N tachyon208
#$ -pe orte 208

tmphosts=`mktemp`
awk '{ for (i=0; i < $2; ++i) { print $1} }' $PE_HOSTFILE > $tmphosts

echo "Got $NSLOTS slots"
echo "jobid $JOB_ID"

mpirun -np $NSLOTS -machinefile $tmphosts /mnt/glusterfs/bishopj/tachyon/tachyon/compile/linux-mpi/tachyon +V \
-aasamples 4 -trans_vmd /mnt/glusterfs/bishopj/tachyon/stmvao-white.dat \
-o /mnt/glusterfs/$USER/stmvao-white_short.$JOB_ID.bmp -format BMP -rescale_lights 0.4 -add_skylight 0.9 \
-skylight_samples 32 -res 1900 1080




CPU Information:
  Node    0:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    1:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    2:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    3:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    4:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    5:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    6:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    7:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    8:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node    9:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
  Node   10:  1 CPUs, CPU Speed 1.00, Node Speed   1.00 Name: barley01.stanford.edu
.
.
.
  Total CPUs: 208
  Total Speed: 208.000000

Preprocessing Time:     1.5401 seconds
Rendering Progress:       100% complete            
  Ray Tracing Time:     3.4819 seconds
    Image I/O Time:     0.0680 seconds

Calculating the efficiency, we get (533/3.4)/208 = .75 or 75% speedup per added core. Observe that at 208 cores and 3.4 seconds, the standup and tear down time of the processes themselves is starting to play a large role. I would say this scalability experiment is very successful, having reduced the time to render a single frame from 8.8 minutes to 3.4 seconds, a factor of 156 times.

building Tachyon

Let's see what our options are: 

bishopj@corn11:/mnt/glusterfs/bishopj/tachyon/tachyon/unix$ make
  Choose one of the architectures specified below.
--------------------------------------------------------------
              Parallel Versions        

             aix-thr - IBM AIX 5.x      POSIX Threads         
          aix-64-thr - IBM AIX 5.x      POSIX Threads, 64-bit 
             aix-mpi - IBM AIX 5.x (SP) MPI                   
        asci-red-mpi - Intel ASCI Red   MPI                   
        bluegene-mpi - IBM Blue Gene    MPI                   
            cray-thr - Cray J90,C90...  POSIX Threads         
        cray-t3e-mpi - Cray T3E         MPI                   
        cray-xt3-mpi - Cray XT3         MPI                   
  compaq-alphasc-mpi - Lemieux at PSC   MPI                   
            hpux-thr - HP/UX 11.x       POSIX Threads         
       hpux-ia64-thr - HP-UX IA-64, HP C, POSIX Threads       
           irix6-thr - SGI IRIX 6.5.x   POSIX Threads         
        irix6-64-thr - SGI IRIX 6.5.x   POSIX Threads, 64-bit 
   linux-beowulf-mpi - Scyld Linux MPI                        
 linux-alpha-ccc-qsw - Linux Alpha, Compaq C, MPI, QSWnet     
           linux-lam - Linux            MPI (OSC LAM)         
        linux-lam-64 - Linux AMD64/EM64T, MPI, 64-bit         
        linux-64-thr - Linux AMD64/EM64T, POSIX Threads, 64-bit
           linux-mpi - Linux            Generic MPI           
           linux-thr - Linux            POSIX Threads         
    linux-p4-icc-thr - Linux, Intel C compilers, P4-optimized 
      linux-ia64-thr - Linux IA-64, GCC, POSIX Threads        
linux-ia64-thr-sgicc - Linux IA-64, SGI Pro64 Compilers       
          macosx-thr - MacOS X PowerPC, POSIX Threads         
      macosx-x86-thr - MacOS X Intel x86, POSIX Threads       
   solaris-ultra-hpc - Sun Solaris 9/10 ClusterTools 4.0 MPI  
         solaris-mpi - Sun Solaris 9/10 MPI (MPICH)           
         solaris-lam - Sun Solaris 9/10 MPI (OSC LAM)         
         solaris-thr - Sun Solaris 9/10 UI/Sun Threads        
   solaris-ultra-thr - Sun Solaris 9/10 UI/Sun Threads, US-1/2
  solaris-ultra3-thr - Sun Solaris 9/10 UI/Sun Threads, US-III
      solaris-64-thr - Sun Solaris 9/10 UI/Sun Threads, 64-bit
--------------------------------------------------------------
            Hybrid Parallel Versions                          

       linux-mpi-thr - Linux    Generic MPI+POSIX Threads     

--------------------------------------------------------------
              Sequential Versions                             

              aix - IBM AIX 5.x                             
              bsd - OpenBSD/FreeBSD/NetBSD                  
             hpux - HP/UX 11.x                              
            irix6 - SGI Irix 6.x                            
            linux - Linux                                   
         linux-64 - Linux, AMD64/EM64T, GCC 3.x, 64-bit     
     linux-p4-icc - Linux, Intel C compilers, P4-optimized  
     linux-athlon - Linux using GCC 3.x Athlon optimizations
         linux-p4 - Linux using GCC 3.x P4 optimizations    
       linux-pgcc - Linux, Portland Group Compilers, Pentium
linux-athlon-pgcc - Linux, Portland Group Compilers, Athlon 
        linux-ppc - Linux PowerPC                           
        linux-ps2 - Linux Sony Playstation 2                
      linux-alpha - Linux Alpha                             
  linux-alpha-ccc - Linux Alpha, Compaq Compilers           
       linux-ia64 - Linux IA-64, GCC                        
 linux-ia64-sgicc - Linux IA-64, SGI Pro64 Compilers        
           macosx - MacOS X (aka Darwin, Rhapsody)          
      tru64-alpha - Tru64 Unix, binary can run on AlphaLinux
          solaris - Sun Solaris 8/9/10                      
            win32 - Windows 95/NT with Cygnus EGCS/CygWin32 

--------------------------------------------------------------
              OpenGL Runtime Display Versions                 

        irix6-thr-ogl - SGI IRIX 6.5.x, POSIX Threads, OpenGL 
solaris-ultra-thr-ogl - Sun Solaris 9/10 UI/Sun Threads, OpenGL
solaris-ultra-hpc-ogl - Sun Solaris 9/10 Sun MPI, OpenGL       
   solaris-64-thr-ogl - Sun Solaris 9/10 UI/Sun Threads, 64-bit
        linux-thr-ogl - Linux POSIX Threads, OpenGL           
     linux-64-thr-ogl - Linux POSIX Threads, OpenGL           
     linux-lam-64-ogl - Linux, 64-bit AMD64/EM64T, LAM MPI, OpenGL
linux-beowulf-mpi-ogl - Scyld Linux with MPI, OpenGL          
   macosx-x86-thr-ogl - MacOS X Intel x86, POSIX Threads, OpenGL
  tru64-alpha-thr-ogl - Tru64/OSF1/DUnix POSIX Threads, OpenGL

--------------------------------------------------------------
Type: 'make arch' to build for an architecture listed above.

We will choose linux-mpi. 

bishopj@corn11:/mnt/glusterfs/bishopj/tachyon/tachyon/unix$ make linux-mpi
make all \
	"ARCH = linux-mpi" \
	"CC = mpicc" \
	"CFLAGS = -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI       " \
	"AR = ar" \
	"ARFLAGS = r" \
	"STRIP = strip" \
	"RANLIB = ranlib" \
	"LIBS = -L. -ltachyon     -lm"
make[1]: Entering directory `/mnt/glusterfs/bishopj/tachyon/tachyon/unix'
Building Tachyon Parallel Ray Tracing library
Copyright 1994-2007, John E. Stone
All Rights Reserveed
Making architecture directory ../compile/linux-mpi
Making library directory ../compile/linux-mpi/libtachyon
make ../compile ../compile/linux-mpi ../compile/linux-mpi/libtachyon  ../compile/linux-mpi/libtachyon.a  ../compile/linux-mpi/tachyon
make[2]: Entering directory `/mnt/glusterfs/bishopj/tachyon/tachyon/unix'
make[2]: Nothing to be done for `../compile'.
make[2]: Nothing to be done for `../compile/linux-mpi'.
make[2]: Nothing to be done for `../compile/linux-mpi/libtachyon'.
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/api.c -o ../compile/linux-mpi/libtachyon/api.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/apigeom.c -o ../compile/linux-mpi/libtachyon/apigeom.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/box.c -o ../compile/linux-mpi/libtachyon/box.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/global.c -o ../compile/linux-mpi/libtachyon/global.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/hash.c -o ../compile/linux-mpi/libtachyon/hash.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/parallel.c -o ../compile/linux-mpi/libtachyon/parallel.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/threads.c -o ../compile/linux-mpi/libtachyon/threads.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/camera.c -o ../compile/linux-mpi/libtachyon/camera.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/render.c -o ../compile/linux-mpi/libtachyon/render.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/trace.c -o ../compile/linux-mpi/libtachyon/trace.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/grid.c -o ../compile/linux-mpi/libtachyon/grid.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/intersect.c -o ../compile/linux-mpi/libtachyon/intersect.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/sphere.c -o ../compile/linux-mpi/libtachyon/sphere.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/plane.c -o ../compile/linux-mpi/libtachyon/plane.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/ring.c -o ../compile/linux-mpi/libtachyon/ring.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/triangle.c -o ../compile/linux-mpi/libtachyon/triangle.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/cylinder.c -o ../compile/linux-mpi/libtachyon/cylinder.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/quadric.c -o ../compile/linux-mpi/libtachyon/quadric.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/extvol.c -o ../compile/linux-mpi/libtachyon/extvol.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/vol.c -o ../compile/linux-mpi/libtachyon/vol.o
../src/vol.c: In function 'LoadVol':
../src/vol.c:294:8: warning: ignoring return value of 'fread', declared with attribute warn_unused_result [-Wunused-result]
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/ui.c -o ../compile/linux-mpi/libtachyon/ui.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/util.c -o ../compile/linux-mpi/libtachyon/util.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/vector.c -o ../compile/linux-mpi/libtachyon/vector.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/shade.c -o ../compile/linux-mpi/libtachyon/shade.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/texture.c -o ../compile/linux-mpi/libtachyon/texture.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/coordsys.c -o ../compile/linux-mpi/libtachyon/coordsys.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/imap.c -o ../compile/linux-mpi/libtachyon/imap.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/light.c -o ../compile/linux-mpi/libtachyon/light.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/imageio.c -o ../compile/linux-mpi/libtachyon/imageio.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/jpeg.c -o ../compile/linux-mpi/libtachyon/jpeg.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/pngfile.c -o ../compile/linux-mpi/libtachyon/pngfile.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/ppm.c -o ../compile/linux-mpi/libtachyon/ppm.o
../src/ppm.c: In function 'readppm':
../src/ppm.c:47:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../src/ppm.c:57:8: warning: ignoring return value of 'fread', declared with attribute warn_unused_result [-Wunused-result]
../src/ppm.c: In function 'getint':
../src/ppm.c:28:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../src/ppm.c:30:14: warning: ignoring return value of 'fgets', declared with attribute warn_unused_result [-Wunused-result]
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/psd.c -o ../compile/linux-mpi/libtachyon/psd.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/sgirgb.c -o ../compile/linux-mpi/libtachyon/sgirgb.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/tgafile.c -o ../compile/linux-mpi/libtachyon/tgafile.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -c ../src/winbmp.c -o ../compile/linux-mpi/libtachyon/winbmp.o
rm -f ../compile/linux-mpi/libtachyon.a
ar r ../compile/linux-mpi/libtachyon.a ../compile/linux-mpi/libtachyon/api.o ../compile/linux-mpi/libtachyon/apigeom.o ../compile/linux-mpi/libtachyon/box.o ../compile/linux-mpi/libtachyon/global.o ../compile/linux-mpi/libtachyon/hash.o ../compile/linux-mpi/libtachyon/parallel.o ../compile/linux-mpi/libtachyon/threads.o ../compile/linux-mpi/libtachyon/camera.o ../compile/linux-mpi/libtachyon/render.o ../compile/linux-mpi/libtachyon/trace.o ../compile/linux-mpi/libtachyon/grid.o ../compile/linux-mpi/libtachyon/intersect.o ../compile/linux-mpi/libtachyon/sphere.o ../compile/linux-mpi/libtachyon/plane.o ../compile/linux-mpi/libtachyon/ring.o ../compile/linux-mpi/libtachyon/triangle.o ../compile/linux-mpi/libtachyon/cylinder.o ../compile/linux-mpi/libtachyon/quadric.o ../compile/linux-mpi/libtachyon/extvol.o ../compile/linux-mpi/libtachyon/vol.o ../compile/linux-mpi/libtachyon/ui.o ../compile/linux-mpi/libtachyon/util.o ../compile/linux-mpi/libtachyon/vector.o ../compile/linux-mpi/libtachyon/shade.o ../compile/linux-mpi/libtachyon/texture.o ../compile/linux-mpi/libtachyon/coordsys.o ../compile/linux-mpi/libtachyon/imap.o ../compile/linux-mpi/libtachyon/light.o ../compile/linux-mpi/libtachyon/imageio.o ../compile/linux-mpi/libtachyon/jpeg.o ../compile/linux-mpi/libtachyon/pngfile.o ../compile/linux-mpi/libtachyon/ppm.o ../compile/linux-mpi/libtachyon/psd.o ../compile/linux-mpi/libtachyon/sgirgb.o ../compile/linux-mpi/libtachyon/tgafile.o ../compile/linux-mpi/libtachyon/winbmp.o
ar: creating ../compile/linux-mpi/libtachyon.a
ranlib ../compile/linux-mpi/libtachyon.a
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/main.c -o ../compile/linux-mpi/libtachyon/main.o
../demosrc/main.c: In function 'animate_scene':
../demosrc/main.c:392:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/getargs.c -o ../compile/linux-mpi/libtachyon/getargs.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI         -I../src -I../demosrc -c ../demosrc/parse.c -o ../compile/linux-mpi/libtachyon/parse.o
../demosrc/parse.c:1468:16: warning: 'GetSphereArray' defined but not used [-Wunused-function]
../demosrc/parse.c: In function 'GetTexBody':
../demosrc/parse.c:794:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:798:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:802:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:806:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:809:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:811:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:815:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:819:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:821:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:822:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:826:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:837:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:839:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:840:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:854:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:862:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:885:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:895:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:909:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetString':
../demosrc/parse.c:212:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'PrintSyntaxError':
../demosrc/parse.c:189:10: warning: ignoring return value of 'fread', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'readmodel':
../demosrc/parse.c:242:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetScenedefs':
../demosrc/parse.c:321:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetObject':
../demosrc/parse.c:592:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetTexture':
../demosrc/parse.c:765:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetVCSTri':
../demosrc/parse.c:1444:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1450:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1456:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetVertexArray':
../demosrc/parse.c:1560:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1566:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1568:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1578:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1580:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1598:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1638:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1705:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1766:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetSphere':
../demosrc/parse.c:1279:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetFCylinder':
../demosrc/parse.c:1230:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetRing':
../demosrc/parse.c:1356:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1358:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetPolyCylinder':
../demosrc/parse.c:1248:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1257:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetCylinder':
../demosrc/parse.c:1201:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetVol':
../demosrc/parse.c:1318:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1320:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetImageDef':
../demosrc/parse.c:665:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:689:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetTexDef':
../demosrc/parse.c:743:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetTexAlias':
../demosrc/parse.c:753:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:754:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetLight':
../demosrc/parse.c:983:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:986:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:988:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1000:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1003:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1006:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetColor':
../demosrc/parse.c:650:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetDirLight':
../demosrc/parse.c:956:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:958:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetSkyLight':
../demosrc/parse.c:1028:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1031:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetSpotLight':
../demosrc/parse.c:1054:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1060:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1063:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1066:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1068:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1079:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1082:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1085:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetLandScape':
../demosrc/parse.c:1790:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1793:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetShaderMode':
../demosrc/parse.c:334:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:349:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:358:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:377:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:380:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:383:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetCamera':
../demosrc/parse.c:404:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:406:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:415:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:418:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:426:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:429:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:437:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:441:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:444:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:447:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:453:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:459:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:467:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:469:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:471:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetTPolyFile':
../demosrc/parse.c:1835:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1845:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetBackGnd':
../demosrc/parse.c:1135:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetBackGndGradient':
../demosrc/parse.c:1152:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1166:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1169:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1172:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1175:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1178:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c: In function 'GetFog':
../demosrc/parse.c:1103:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1115:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1118:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/parse.c:1121:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI         -I../src -I../demosrc -c ../demosrc/nffparse.c -o ../compile/linux-mpi/libtachyon/nffparse.o
../demosrc/nffparse.c: In function 'NFFGetPatch':
../demosrc/nffparse.c:246:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c: In function 'NFFGetPolygon':
../demosrc/nffparse.c:227:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c: In function 'NFFGetSphere':
../demosrc/nffparse.c:215:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c: In function 'NFFGetCylCone':
../demosrc/nffparse.c:193:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c:196:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c: In function 'NFFGetTexture':
../demosrc/nffparse.c:171:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c: In function 'NFFGetLight':
../demosrc/nffparse.c:157:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c: In function 'NFFGetBackground':
../demosrc/nffparse.c:138:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c: In function 'NFFGetScenedefs':
../demosrc/nffparse.c:109:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c:114:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c:117:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c: In function 'NFFGetVector':
../demosrc/nffparse.c:59:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/nffparse.c: In function 'NFFGetString':
../demosrc/nffparse.c:47:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/glwin.c -o ../compile/linux-mpi/libtachyon/glwin.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/spaceball.c -o ../compile/linux-mpi/libtachyon/spaceball.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -c ../demosrc/trackball.c -o ../compile/linux-mpi/libtachyon/trackball.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI         -I../src -I../demosrc  -c ../demosrc/mgfparse.c -o ../compile/linux-mpi/libtachyon/mgfparse.o
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI         -I../src -I../demosrc -c ../demosrc/ac3dparse.c -o ../compile/linux-mpi/libtachyon/ac3dparse.o
../demosrc/ac3dparse.c: In function 'ParseAC3D':
../demosrc/ac3dparse.c:105:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c: In function 'GetMaterial':
../demosrc/ac3dparse.c:207:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:210:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:217:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:221:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:224:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:228:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:231:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c: In function 'GetString':
../demosrc/ac3dparse.c:82:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c: In function 'GetRecurseObject':
../demosrc/ac3dparse.c:398:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:399:9: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:402:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:405:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:406:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:409:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:410:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:413:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:414:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:419:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:422:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:426:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:428:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:433:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:436:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:447:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:448:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:452:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:457:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:467:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:475:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:478:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:479:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:492:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:497:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:500:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:501:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:504:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:510:15: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:542:13: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
../demosrc/ac3dparse.c:566:11: warning: ignoring return value of 'fscanf', declared with attribute warn_unused_result [-Wunused-result]
mpicc -Wall -O3 -fomit-frame-pointer -ffast-math -DLinux -DMPI        -I../src -I../demosrc -o ../compile/linux-mpi/tachyon ../compile/linux-mpi/libtachyon/main.o ../compile/linux-mpi/libtachyon/getargs.o ../compile/linux-mpi/libtachyon/parse.o ../compile/linux-mpi/libtachyon/nffparse.o ../compile/linux-mpi/libtachyon/glwin.o ../compile/linux-mpi/libtachyon/spaceball.o ../compile/linux-mpi/libtachyon/trackball.o ../compile/linux-mpi/libtachyon/mgfparse.o ../compile/linux-mpi/libtachyon/ac3dparse.o  -L../compile/linux-mpi   -L. -ltachyon     -lm
strip ../compile/linux-mpi/tachyon
make[2]: Leaving directory `/mnt/glusterfs/bishopj/tachyon/tachyon/unix'

Build Complete
make[1]: Leaving directory `/mnt/glusterfs/bishopj/tachyon/tachyon/unix'


The important thing to look for is if the build is using mpicc (or mpif90 for fortran90). It did, and the build was successful

Retrieved from "https://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php/SFOtoHKGin5min"
Personal tools
Toolbox
LANGUAGES