Running OxCal on an Xgrid

OxCal is a freeware program, developed by the Oxford Radiocarbon Accelerator Unit (ORAU), University of Oxford. It applies a Bayesian approach for the analysis of archaeological and environmental chronological information. Binaries are distributed for the current version (v.4) for Mac and Win platforms.

The application consists of three main modules. The project manager and output module are web based, but the core analysis module is an executable, compiled from C++ source code and can be launched from the command line. It takes a plain text input file *.oxcal which contains the instructions for OxCal.
On a Mac this would look like, for example:

OxCalMac 6000_5950_SEQ.oxcal

To execute multiple runs of this, each one with a different (*.oxcal) input file - in our case a total of 1200 - I set OxCal up to run on an Xgrid.

In this particular case I needed to submit the job using my MacBook Pro as a client to submit to a Mac OsX server, the controller. This scenario is different from Charles Parnot's tutorial, which assumes that you run the Xgrid controller on the same machine that you use to submit your job.

This was particularly relevant, since I needed to run an application, which I knew was not available on every agent in the cluster. So I needed the also submit the program and all required input files to the Xgrid.

Thus, in order to run OxCal I needed to use the "-in" switch, which specifies a directory, and its entire content is then sent to every machine on which the job runs.

Here is what worked for me.

1. Create a directory, called oxcal for example. It needs to be accessible for the controller, for that reason it is convenient to put it under /Users/Shared. From the downloaded package copy the contents of the OxCal/bin directory into /Users/Shared/oxcal/bin

2. Create another subdirectory, input for example, and copy the *.oxcal files there.

With this directory structure you would submit a single job like:

xgrid -h [yourcontrollerhost] -p [yourpassword] -job submit -in /Users/Shared/oxcal/ bin/OxCalMac input/6000_5950_SEQ.oxcal

which returns your job ID.

To check if it's running:

xgrid -h [yourcontrollerhost] -p [yourpassword] -job attributes -id [yourjobID]

To get results create a local a directory, for example oxcal_results and retrieve like this:

xgrid -h [yourcontrollerhost] -p [yourpassword] -job results -id [yourjobID] -out /Users/Shared/oxcal_results

3. To submit multiple jobs I used GridStuffer. Gridstuffer takes an input file, which describes your "meta-job" where you specify your runs following a format, just slightly different from xgrid syntax.
Create a textfile oxcal_metajob.txt, for example (*). The first line contains the 'template' for all the following instructions. So if you had 10 different jobs it would look something like:

-in /Users/Shared/oxcal/bin -dirs /Users/Shared/oxcal/input OxCalMac 6000_5950_SEQ.oxcal
OxCalMac 6000_5950_SEQ.oxcal
OxCalMac 6000_5951_SEQ.oxcal
OxCalMac 6000_5952_SEQ.oxcal
OxCalMac 6000_5953_SEQ.oxcal
OxCalMac 6000_5954_SEQ.oxcal
OxCalMac 6000_5955_SEQ.oxcal
OxCalMac 6000_5956_SEQ.oxcal
OxCalMac 6000_5957_SEQ.oxcal
OxCalMac 6000_5958_SEQ.oxcal
OxCalMac 6000_5959_SEQ.oxcal

Then submit via GridStuffer like this:


(*) To generate this file more easily, change into the directory where your input files are and say:

ls *.oxcal | awk '{if (NR!=1) {sub(/^/, "OxCalMac ")}else {sub(/^/, "-in \/Users\/Shared\/oxcal\/bin -dirs \/Users\/Shared\/oxcal\/input OxCalMac ")};print}' > oxcal_metajob.txt

No TweetBacks yet. (Be the first to Tweet this post)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>