Main Page

From FarmShare

(Difference between revisions)
Jump to: navigation, search
Line 100: Line 100:
* FarmShare mail list.  FarmShare maillist has hundreds of other FarmShare users.  It is quite likely somebody will be able to help.  Please [mailto:farmshare-discuss@lists.stanford.edu e-mail] the [https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss FarmShare user community]
* FarmShare mail list.  FarmShare maillist has hundreds of other FarmShare users.  It is quite likely somebody will be able to help.  Please [mailto:farmshare-discuss@lists.stanford.edu e-mail] the [https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss FarmShare user community]
-
*SMACC.  SMACC office hours is an excellent place to start if you would like to discuss or ask a question in-person.  see
+
*SMACC.  SMACC office hours is an excellent place to start if you would like to discuss or ask a question in-person.  see [[Main_Page#SMACC_office_hours]]
== stock software  ==
== stock software  ==
-
The FarmShare machines are running Ubuntu, and the software is from the Ubuntu repositories, e.g. run '''dpkg -l | grep ^i''' to see the list of installed packages.
+
The FarmShare machines are running Ubuntu 13.04, and the software is from the Ubuntu repositories, e.g. run '''dpkg -l | grep ^i''' to see the list of installed packages.
*[[R]]  
*[[R]]  
Line 114: Line 114:
== licensed software  ==
== licensed software  ==
-
As of April 2012, we're transitioning the location of licensed software into /mnt/glusterfs/software/ and using [[FarmShare software | modules]].  As of Feb 13, 2013, the software that is available is:
+
In addition to Ubuntu packages, the following packages are installed and are available via the "module" command:
-
<pre>
+
<source lang="sh">
-
---------------------------------------------- /mnt/glusterfs/software/free/modules/tcl/modulefiles -----------------------------------------------
+
$ module avail
-
AMPL                  GAMS-23.8.1-AFS      MATLAB-R2012a        R-2.15.0              abinit-6.12.3       
+
-
ANSYS                GAMS-23.8.2-GlusterFS MATLAB-R2012b        R-2.15.1-precise      abinit-7.0.5         
+
-
ATK                  IMSL                  Mathematica-8.0.4    StatTransfer-11.2    abinit-7.0.5-barley 
+
-
CMTK-2.2.4            MATLAB-R2010b        NAG-C-23              StataMP-12.1          gephi-0.8.1-beta     
+
-
CPLEX_Studio-12.4    MATLAB-R2011b        NAG-gfortran-23      StataSE-12.1          opensim-3.0         
+
-
</pre>
+
-
Licensed software that hasn't been transitioned yet is still available in /usr/sweet/bin.  Older versions of some of the same programs as above may also be available here.
+
--------------------- /farmshare/software/free/lmod-5.0-install/lmod/lmod/modulefiles/Core ---------------------
 +
  lmod/lmod    settarg/settarg
-
<pre>
+
 
-
# ls /usr/sweet/bin/
+
---------------------------------------- /farmshare/software/mf/raring -----------------------------------------
-
MathKernel@   cplexamp@      g03@        hlm2@      lmutil@      mex@        sicstus@  stata@
+
  cplex/12.4    gaussian/g09gview50    matlab/r2013a   (D)    statase/12.1
-
Mathematica@  cplexconvert@  g09@        hlm3@     maple@       mint@      spdet@    stata-se@
+
  cuda/5.5     mathematica/9.0       sas/9.2                stattransfer/12
-
Splus@        dbmscopy@      gams@      hmlm@      math@        rats@      spld@     tracker@
+
  gams/24.1     matlab/r2012b          statamp/12.1
-
ampl@        dbmsnox@      gamsbatch@  hmlm2@    mathematica@  ratsgraph@  splfr@    xdisplay@
+
 
-
anshelp@      eqs@          gamslib@    launcher@  matlab@      rcomp@      splm@    xmaple@
+
------------------------------------- /farmshare/software/mf/raring-compat -------------------------------------
-
ansys@        f95@          gview@      limdep@   mbuild@      rgf2pst@   splus@   xstata@
+
   CPLEX-12.4   MATLAB-R2012b   Mathematica-9.0    StatTransfer-v12    StataSE-12.1
-
cplex@        f95mcheck@     hcm2@      lmgrd@    mcc@          sas@        spxref@  xstata-se@
+
  GAMS-24.1     MATLAB-R2013a    SAS-v9.2          StataMP-12.1
-
</pre>
+
 
 +
  Where:
 +
  (D):  Default Module
 +
 
 +
</source>
= Monitoring / Status  =
= Monitoring / Status  =

Revision as of 10:12, 14 September 2013

This wiki is intended for the users of FarmShare, the Stanford shared research computing environment: the "cardinal", "corn", and "barley" machines. For a general description of this service, and Stanford's shared computing policies, see the main service catalog page.


Most useful pages: Special:AllPages and Special:RecentChanges and User Guide and FAQ and FarmShare tutorial



Last 10 messages on Farmshare-Discuss mail list (this month)

Contents

    How to connect

    The machines are available for anyone with a SUNetID. Simply "ssh corn.stanford.edu" with your SUNetID credentials. The DNS name "corn.stanford.edu" actually goes to a load balancer and it will connect you to a particular corn machine (e.g. corn21) that has relatively low load.

    The "barley" machines are designed to be used for high performance computing (HPC) and only accessible via a resource manager (currently Open Grid Scheduler). You cannot log in directly, but you can submit jobs from any corn. Storage dedicated for jobs running on the barley cluster is available via /farmshare/user_data/ on all corn and barley nodes. Sign up and email the farmshare-discuss mailing list if you have any questions or would like any info not listed here.

    corn SSH fingerprint is:

     RSA key fingerprint is 0b:e7:b4:95:03:c1:1e:07:df:04:ca:a2:3d:8e:e3:37.
    

    How to get help

    email support

    Huangsmacc.png
    • You can e-mail research-computing-support@stanford.edu
      • If you're e-mailing about a barley job, please mention that it's on barley and the job number.


    SMACC office hours

    Smacctable.jpg
    • You can come to office hours. Every Wednesday from 1-3PM in Huang basement in front of ICME door:
      • Have a computational or statistical problem that you need help with? Or maybe you have an account on Farmshare or Proclus, and so now what? You have a boatload of data to make sense of, but how? Wonder where you can do your research project, and who can help you? You know what you want to do – but how best to do it, you just aren’t sure. Help is here in the form of SMACC – Stat, Math, Algorithmic and Computational Consulting! Technical consultants from ICME, Research Computing, Statistics and IRiSS will be available to work with you each Wednesday, from 1-3 pm, in the basement of Huang (in front of ICME). One stop shopping for your scientific computing needs. Rather than poke around web sites and send mail to multiple groups, drop by to catch all of us at once.
      • Details are at SMACC


    cardinal info

    The "cardinal" machines are small VMs intended for long-running processes (on the order of days) that are not resource intensive, e.g. mail/chat clients. You could log in to a cardinal and run a screen/tmux session there to do things on other machines.

    Simply "ssh cardinal.stanford.edu" with your SUNetID credentials.

    There are currently 3 cardinal machines: cardinal1, cardinal2 and cardinal3, load-balanced via cardinal.stanford.edu.

    corn info

    The "corn" machines are general-purpose Ubuntu boxes and you can run whatever you want on them (so long as you don't negatively impact other users). Please read the policies and the motd first.

    Each of the 30 corn machines has 8 cores, 32GB RAM and ~70GB of local disk in /tmp.

    barley info

    The "barley" machines are general-purpose newer Ubuntu boxes that can run jobs that you submit via the resource manager software. You should not log in to any barley directly, but can do so to troubleshoot your jobs.

    • Q? E-mail research-computing-support@stanford.edu

    barley info

    Examples of using the barley cluster

    1. Introductory examples:
      1. Flac Like a Boss
      2. Cheap Flights
      3. San Francisco to Hong Kong in 5 minutes
      4. Monte Carlo Simulations in Matlab
    1. R
    2. MATLAB
    3. Access Mysql from Matlab
    4. Rmpi
    5. Gaussian
    6. Ipython
    7. ANSYS
    1. Gaussview: Automated Submission Script Creation & Submission

    FarmShare software

    questions or requests about installed software

    If you have a question or request about the installed software on FarmShare please email us: research-computing-support@stanford.edu

    questions about how to use installed software

    If you need help on usage we would suggest:

    • FarmShare mail list. FarmShare maillist has hundreds of other FarmShare users. It is quite likely somebody will be able to help. Please e-mail the FarmShare user community
    • SMACC. SMACC office hours is an excellent place to start if you would like to discuss or ask a question in-person. see Main_Page#SMACC_office_hours

    stock software

    The FarmShare machines are running Ubuntu 13.04, and the software is from the Ubuntu repositories, e.g. run dpkg -l | grep ^i to see the list of installed packages.

    If the package you're looking for isn't installed, search the Ubuntu Packages page and submit a HelpSU with the package name(s) you want.

    licensed software

    In addition to Ubuntu packages, the following packages are installed and are available via the "module" command:

    $ module avail
    
    --------------------- /farmshare/software/free/lmod-5.0-install/lmod/lmod/modulefiles/Core ---------------------
       lmod/lmod    settarg/settarg
    
    
    ---------------------------------------- /farmshare/software/mf/raring -----------------------------------------
       cplex/12.4    gaussian/g09gview50    matlab/r2013a   (D)    statase/12.1
       cuda/5.5      mathematica/9.0        sas/9.2                stattransfer/12
       gams/24.1     matlab/r2012b          statamp/12.1
    
    ------------------------------------- /farmshare/software/mf/raring-compat -------------------------------------
       CPLEX-12.4    MATLAB-R2012b    Mathematica-9.0    StatTransfer-v12    StataSE-12.1
       GAMS-24.1     MATLAB-R2013a    SAS-v9.2           StataMP-12.1
    
      Where:
       (D):  Default Module
    

    Monitoring / Status

    You probably want to try these commands:

     qstat -g c #cluster slots summary by queue
     qhost -F mem_free #show available memory on each host
     qstat -f -u \* #show all jobs in the system
    

    For important announcements, we plan to:

    • add it to this wiki
    • modify /etc/motd on the corn machines
    • send a mail to farmshare-discuss

    Mailing Lists

    We have mailing lists, @lists.stanford.edu - https://itservices.stanford.edu/service/mailinglists/tools

    Links

    Want to learn HPC? Free education materials available:


    GPUs! We don't have any GPUs as part of FarmShare, but there are other campus resources available:


    Other similar wikis/clusters on campus (you might not have access to these):

    Vision

    The Farmshare resources are being made available to students, faculty and staff with fully sponsored SunetIDs to facilitate research at Stanford University.  This resource is designed so that those doing research will have a place to experiment and learn about technical solutions to assist in reaching their research goals without needing to write a grant for a cluster.  The Farmshare resources are focused on making it easier to learn how to parallelize research computing tasks and use research software including a "scheduler" or "distributed resource management system" to submit compute jobs.

    By using Farmshare, new researchers can more easily adapt to using larger clusters when they have big projects that involve using federally funded resources, shared Stanford clusters, or on a small grant funded cluster.

    Personal tools
    Toolbox
    LANGUAGES