From FarmShare

Example of using python to submit jobs

For python, (or anything really) it's best to use getopt to handle command line arguments. But I don't do it here:


[chekh@scorn.stanford.edu] ~ [0]
$ cat test.py
import sys

print ("received this many arguments: ", len(sys.argv))

# argument 0 is the command itself usually

for i in xrange(len(sys.argv)):
print("Argument # %d : %s" % (i, str(sys.argv[i])))


[chekh@scorn.stanford.edu] ~ [0]
$ python test.py -c one -b two
('received this many arguments: ', 5)
Argument # 0 : test.py
Argument # 1 : -c
Argument # 2 : one
Argument # 3 : -b
Argument # 4 : two
[chekh@scorn.stanford.edu] ~ [0]
$ cat submit_test_jobs.py
import os
import sys


if 2 != len(sys.argv):
print "only want one argument, exiting"

# argument 0 is the command itself usually

for i in range(0, int(sys.argv[1])):
#print out a job file
jobfilename = "job" + str(i) + ".job"
f = open(jobfilename, 'w')
s = """
#$ -S /bin/bash

#
# set the name of the job; this will appear in the job listing
#$ -N python_job
#

#
# set the maximum memory usage (per slot)
#$ -l mem_free=2G
#
# on other clusters this memory resource may have a different name

#
# set the number of slots, replace '1' with a larger number if needed
#$ -pe shm 1
#
# on other clusters this pe may have a different name

#
# set the maximum run time, hh:mm:ss, default is 48hrs on FarmShare
#$ -l h_rt=1:00:00
#

# check for errors in the job submission options
#$ -w e
#

# join the stdout and stderr streams into one file
#$ -j y
#

python test.py hello
"""
f.write(s)
#submit the job file
commandstring = 'qsub -cwd ' + jobfilename
print commandstring
os.system(commandstring)
[chekh@scorn.stanford.edu] ~ [0]
$ python submit_test_jobs.py 5
qsub -cwd job0.job
sh: qsub: command not found
qsub -cwd job1.job
sh: qsub: command not found
qsub -cwd job2.job
sh: qsub: command not found
qsub -cwd job3.job
sh: qsub: command not found
qsub -cwd job4.job
sh: qsub: command not found
[chekh@scorn.stanford.edu] ~ [0]
$