Main Page

From FarmShare

(Difference between revisions)
Jump to: navigation, search
m ("barleys" -> "barley machines", similarly "corns" -> "corn machines")
(Monitoring / Status)
Line 94: Line 94:
For important announcements, we plan to:
For important announcements, we plan to:
 +
* add it to this wiki
* modify /etc/motd on the corn machines
* modify /etc/motd on the corn machines
* send a mail to farmshare-announce
* send a mail to farmshare-announce

Revision as of 16:23, 15 December 2011

+-+-+-+-+-+-+-+-+-+
|F|a|r|m|S|h|a|r|e|
+-+-+-+-+-+-+-+-+-+

This wiki is intended for the users of the Stanford shared research computing resources. E.g. the "cardinal" and "corn" and "barley" machines.

Most useful pages: Special:AllPages and Special:RecentChanges

Contents

How to connect

The machines are available for anyone with a SUNetID. Simply "ssh corn.stanford.edu" with your SUNetID credentials. The DNS name "corn.stanford.edu" actually goes to a load balancer and it will connect you to a particular corn machine (e.g. corn21) that has relatively low load.

The "barley" machines are only accessible via a resource manager (currently Open Grid Engine). You can submit jobs from any corn. You'll need to ssh to corn-image-new.stanford.edu and a directory on /mnt/glusterfs will be created for you on non-AFS storage. E-mail the barley-alpha mailing list for more info.

cardinal info

The "cardinal" machines are older and slower and smaller and intended for long-running processes (on the order of days) that are not resource intensive. E.g. mail/chat clients. You could log in to a cardinal and run a screen/tmux session there to do things on other machines...

corn info

The "corn" machines are general-purpose Ubuntu boxes and you can run whatever you want on them (so long as you don't negatively impact other users). Please read the policies and the motd first.

Each of the 30 corn machines has 8 cores, 32GB RAM and ~70GB of local disk in /tmp.

barley info

The "barley" machines are general-purpose newer Ubuntu boxes that can run jobs that you submit via the resource manager software.

current barley policies

  • 100 max jobs per user
  • 3000 max jobs in the system
  • 48hr max runtime for any job in regular queue
  • one week max runtime for the long queue
  • no memory/CPU limits yet

Technical details

  • 19 new machines, 24 cores each, 96GB RAM
  • 1 new machine, 24 cores, 192GB RAM
  • ~450GB local scratch on each
  • ~3TB in /mnt/glusterfs
  • Grid Engine v6.2u5 (via standard Debian package)
  • 10GbE interconnect (Juniper QFX3500 switch)

how to use the barley machines

To start using these new machines, you can check out the man page for 'sge_intro' or the 'qhost', 'qstat', 'qsub' and 'qdel' commands.

Initial issues:

  • You are limited in space to your AFS homedir ($HOME) and local scratch disk on each node ($TMPDIR)
  • The execution hosts don't accept interactive jobs, only batch jobs for now.
  • You'll want to make sure you have your Kerberos TGT and your AFS token.

If you want to use the newer bigger storage:

  1. log into corn-image-new: "ssh sunetid@corn-image-new.stanford.edu"
  2. cd to /mnt/glusterfs/<your username> (or wait 5mins if it doesn't exist yet)
  3. write a job script: "$EDITOR test_job.script"
    1. see 'man qsub' for more info
    2. use env var $TMPDIR for local scratch
    3. use /mnt/glusterfs/<your username> for shared data directory
  4. submit the job for processing: "qsub -cwd test_job.script"
  5. monitor the jobs with "qstat -f -j JOBID"
    1. see 'man qstat' for more info
  6. check the output files that you specified in your job script (the input and output files must be in /mnt/glusterfs/)

Any questions, please email 'barley-alpha@lists.stanford.edu'

We plan to roll out to the full Stanford community on Jan 1.

barley software

stock software

The barley machines are running Ubuntu 11.04, and the software is from the Ubuntu repositories, e.g. do 'dpkg -l' to see the list of installed packages.

licensed software

There is a group on campus called "SSDS" that can provide support for SAS and stata: http://www.stanford.edu/group/ssds/cgi-bin/drupal/content/who-we-are-what-we-do

Monitoring / Status

Current status of farmshare machines: http://barley-monitor.stanford.edu/ganglia/ More detailed graphs: http://barley-monitor.stanford.edu/munin/

For important announcements, we plan to:

  • add it to this wiki
  • modify /etc/motd on the corn machines
  • send a mail to farmshare-announce
  • duplicate that mail also to the farmshare "blog"

Mailing Lists

We have several mailing lists, all @lists.stanford.edu, most are not used.

  • barley-alpha - temp list till end of Oct/Nov 2011, for discussion around testing the barley machines
  • farmshare-announce - announce list (new service name)
  • farmshare-discuss - users discussion (new service name)
  • stanford-timeshare-users - users discussion list for the corn users, list to be retired


Links

Want to learn HPC? Free education materials available:

Getting started with MediaWiki

Consult the User's Guide for information on using the wiki software.

Personal tools
Toolbox
LANGUAGES