Main Page

From FarmShare

(Difference between revisions)
Jump to: navigation, search
(Mailing Lists)
m (Mailing Lists)
Line 115: Line 115:
*[https://mailman.stanford.edu/mailman/listinfo/farmshare-announce farmshare-announce] - announcements list - [https://mailman.stanford.edu/mailman/suprivate/farmshare-announce/ Archives]
*[https://mailman.stanford.edu/mailman/listinfo/farmshare-announce farmshare-announce] - announcements list - [https://mailman.stanford.edu/mailman/suprivate/farmshare-announce/ Archives]
-
*[https://mailman.stanford.edu/mailman/listinfo/farmshare-announce farmshare-discuss] - user discussion - [https://mailman.stanford.edu/mailman/suprivate/farmshare-discuss/ Archives]
+
*[https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss farmshare-discuss] - user discussion - [https://mailman.stanford.edu/mailman/suprivate/farmshare-discuss/ Archives]
= Links  =
= Links  =

Revision as of 16:21, 27 January 2012

This wiki is intended for the users of FarmShare, the Stanford shared research computing environment: the "cardinal" and "corn" and "barley" machines. For a general description of this service, and Stanford's shared computing policies, see the main service catalog page.

Most useful pages: Special:AllPages and Special:RecentChanges

Contents

How to connect

The machines are available for anyone with a SUNetID. Simply "ssh corn.stanford.edu" with your SUNetID credentials. The DNS name "corn.stanford.edu" actually goes to a load balancer and it will connect you to a particular corn machine (e.g. corn21) that has relatively low load.

The "barley" machines are only accessible via a resource manager (currently Open Grid Engine). You cannot log in directly, but you can submit jobs from any corn.  Storage dedicated for jobs running on the barley cluster is available via /mnt/glusterfs on all corn and barley nodes.  Login to corn-image-new.stanford.edu and a directory will be created for you as /mnt/glusterfs/<your user name> (can take up to 5 minutes).  Sign up and email the farmshare-discuss mailing list if you have any questions or would like any info not listed here.

cardinal info

The "cardinal" machines are small VMs intended for long-running processes (on the order of days) that are not resource intensive, e.g. mail/chat clients. You could log in to a cardinal and run a screen/tmux session there to do things on other machines...

There are currently 3 cardinal machines: cardinal1, cardinal2 and cardinal3, load-balanced via cardinal.stanford.edu.

corn info

The "corn" machines are general-purpose Ubuntu boxes and you can run whatever you want on them (so long as you don't negatively impact other users). Please read the policies and the motd first.

Each of the 30 corn machines has 8 cores, 32GB RAM and ~70GB of local disk in /tmp.

barley info

The "barley" machines are general-purpose newer Ubuntu boxes that can run jobs that you submit via the resource manager software.

current barley policies

  • 3000 max jobs per user (look for max_ujobs in output of 'qconf -sconf')
  • 3000 max jobs in the system (look for max_jobs in output of 'qconf -sconf')
  • 48hr max runtime for any job in regular queue (look for h_rt in output of 'qconf -sq main.q')
  • one week max runtime for the long queue (look for h_rt in output of 'qconf -sq long.q')
  • no memory/CPU limits yet

Technical details

  • 19 new machines, 24 cores each, 96GB RAM
  • 1 new machine, 24 cores, 192GB RAM
  • ~450GB local scratch on each
  • ~3TB in /mnt/glusterfs
  • Grid Engine v6.2u5 (via standard Debian package)
  • 10GbE interconnect (Juniper QFX3500 switch)

how to use the barley machines

To start using these new machines, you can check out the man page for 'sge_intro' or the 'qhost', 'qstat', 'qsub' and 'qdel' commands.

Initial issues:

  • You are limited in space to your AFS homedir ($HOME) and local scratch disk on each node ($TMPDIR)
  • The execution hosts don't accept interactive jobs, only batch jobs for now.
  • You'll want to make sure you have your Kerberos TGT and your AFS token.

If you want to use the newer bigger storage:

  1. log into corn-image-new: "ssh sunetid@corn-image-new.stanford.edu"
  2. cd to /mnt/glusterfs/<your username> (or wait 5mins if it doesn't exist yet)
  3. write a job script: "$EDITOR test_job.script"
    1. see 'man qsub' for more info
    2. use env var $TMPDIR for local scratch
    3. use /mnt/glusterfs/<your username> for shared data directory
  4. submit the job for processing: "qsub -cwd test_job.script"
  5. monitor the jobs with "qstat -f -j JOBID"
    1. see 'man qstat' for more info
  6. check the output files that you specified in your job script (the input and output files must be in /mnt/glusterfs/)

Any questions, please email 'farmshare-discuss@lists.stanford.edu'

We plan to roll out to the full Stanford community on Jan 20.

Examples of using the barley cluster

  1. Examples Ready to Make
  2. R
  3. MATLAB

barley software

stock software

The barley machines are running Ubuntu 11.04, and the software is from the Ubuntu repositories, e.g. do 'dpkg -l' to see the list of installed packages.

licensed software

There is a group on campus called "SSDS" that can provide support for R, SAS and stata: http://www.stanford.edu/group/ssds/cgi-bin/drupal/content/who-we-are-what-we-do

Monitoring / Status

For important announcements, we plan to:

  • add it to this wiki
  • modify /etc/motd on the corn machines
  • send a mail to farmshare-announce

Mailing Lists

We have mailing lists, @lists.stanford.edu - https://itservices.stanford.edu/service/mailinglists/tools

Links

Want to learn HPC? Free education materials available:

All about GPUs

We don't have any GPUs as part of FarmShare, but there is a CUDA development cluster on campus available for your use:

Getting started with MediaWiki

Consult the User's Guide for information on using the wiki software. MediaWiki FAQ

Personal tools
Toolbox
LANGUAGES