https://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&feed=atom&action=historyUser Guide - Revision history2024-03-29T08:21:04ZRevision history for this page on the wikiMediaWiki 1.16.5https://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4903&oldid=prevEpoch: /* Scratch */2020-08-26T22:05:16Z<p><span class="autocomment">Scratch</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 22:05, 26 August 2020</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 21:</td>
<td colspan="2" class="diff-lineno">Line 21:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Scratch ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Scratch ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Scratch storage is available in <code>/farmshare/user_data</code>, and each user is provided with a personal scratch directory, <code>/farmshare/user_data/$USER</code>. The total volume size is currently 126 TB; quotas are not currently enforced, but <del class="diffchange diffchange-inline">old </del>files <del class="diffchange diffchange-inline">may be </del>purged without warning. The scratch volume is ''not'' backed up, and is ''not'' suitable for long-term storage, but can be used as working storage for batch jobs, and as a short-term staging area for data waiting to be archived to permanent storage.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Scratch storage is available in <code>/farmshare/user_data</code>, and each user is provided with a personal scratch directory, <code>/farmshare/user_data/$USER</code>. The total volume size is currently 126 TB; quotas are not currently enforced, but <ins class="diffchange diffchange-inline">''</ins>files <ins class="diffchange diffchange-inline">not modified in the last 90 days are regularly </ins>purged<ins class="diffchange diffchange-inline">, </ins>without warning<ins class="diffchange diffchange-inline">''</ins>. The scratch volume is ''not'' backed up, and is ''not'' suitable for long-term storage, but can be used as working storage for batch jobs, and as a short-term staging area for data waiting to be archived to permanent storage.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Temp ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Temp ===</div></td></tr>
</table>Epochhttps://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4818&oldid=prevEpoch: /* Installed Software */2018-09-04T17:47:35Z<p><span class="autocomment">Installed Software</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 17:47, 4 September 2018</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 43:</td>
<td colspan="2" class="diff-lineno">Line 43:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Installed Software ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Installed Software ==</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>FarmShare systems run Ubuntu 16.04 LTS, and most software is sourced from standard repositories. Additional software, including licensed software, is organized using environment modules and can be accessed using the <code>module</code> command. Users can build and/or install their own software in their home directories, either manually, or using a local package manager. FarmShare supports running software packaged as Singularity containers.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>FarmShare systems run Ubuntu 16.04 LTS, and most software is sourced from standard repositories. Additional software, including licensed software, is organized using environment modules and can be accessed using the <code>module</code> command<ins class="diffchange diffchange-inline">. '''Please note''' that commercial software is licensed for use on FarmShare ''only'' in coursework, and ''not'' in research</ins>. Users can <ins class="diffchange diffchange-inline">also </ins>build and/or install their own software in their home directories, either manually, or using a local package manager. FarmShare supports running software packaged as Singularity containers.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Running Jobs ==</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>== Running Jobs ==</div></td></tr>
</table>Epochhttps://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4743&oldid=prevEpoch: /* Limits */2018-04-12T23:05:09Z<p><span class="autocomment">Limits</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:05, 12 April 2018</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 79:</td>
<td colspan="2" class="diff-lineno">Line 79:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The <code>gpu</code> quality-of-service has a minimum GPU requirement (1), so you must request access to a GPU explicitly when submitting a job.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The <code>gpu</code> quality-of-service has a minimum GPU requirement (1), so you must request access to a GPU explicitly when submitting a job.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><source lang="sh">sbatch --partition=gpu --gres=gpu:1</source></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><source lang="sh">sbatch --partition<ins class="diffchange diffchange-inline">=gpu --qos</ins>=gpu --gres=gpu:1</source></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The <code>bigmem</code> quality-of-service has a minimum memory requirement; you must request at least 96GB when submitting a job.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The <code>bigmem</code> quality-of-service has a minimum memory requirement; you must request at least 96GB when submitting a job.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div><source lang="sh">sbatch --partition=bigmem --mem=96G</source></div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><source lang="sh">sbatch --partition<ins class="diffchange diffchange-inline">=bigmem --qos</ins>=bigmem --mem=96G</source></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Monitoring your Jobs ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Monitoring your Jobs ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>You can use the <code>squeue</code> and <code>sacct</code> commands to monitor the current state of the scheduler and of your jobs. The <code>sprio</code> command can provide some information on how priority was determined for particular jobs, and the <code>sshare</code> command on how current fair-share was calculated. Use the <code>scontrol</code> and <code>sacctmgr</code> commands to examine the configuration of hosts, partitions, and qualities-of-service.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>You can use the <code>squeue</code> and <code>sacct</code> commands to monitor the current state of the scheduler and of your jobs. The <code>sprio</code> command can provide some information on how priority was determined for particular jobs, and the <code>sshare</code> command on how current fair-share was calculated. Use the <code>scontrol</code> and <code>sacctmgr</code> commands to examine the configuration of hosts, partitions, and qualities-of-service.</div></td></tr>
</table>Epochhttps://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4651&oldid=prevEpoch: /* Interactive Jobs */2017-12-12T22:14:05Z<p><span class="autocomment">Interactive Jobs</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 22:14, 12 December 2017</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 57:</td>
<td colspan="2" class="diff-lineno">Line 57:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><source lang="sh">srun --pty --qos=interactive $SHELL -l</source></div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><source lang="sh">srun --pty --qos=interactive $SHELL -l</source></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Interactive jobs <del class="diffchange diffchange-inline">are not scheduled any differently than </del>batch jobs<del class="diffchange diffchange-inline">: </del>when contention for resources is high<del class="diffchange diffchange-inline">, </del>interactive jobs may wait a long time before starting.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Interactive jobs <ins class="diffchange diffchange-inline">receive a modest priority boost compared to </ins>batch jobs<ins class="diffchange diffchange-inline">, but </ins>when contention for resources is high interactive jobs may wait a long time before starting<ins class="diffchange diffchange-inline">. Each user is allowed one interactive job, which may run for at most one day</ins>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Batch Jobs ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Batch Jobs ===</div></td></tr>
</table>Epochhttps://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4635&oldid=prevEpoch: /* Using SSH */2017-12-04T20:31:52Z<p><span class="autocomment">Using SSH</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:31, 4 December 2017</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 33:</td>
<td colspan="2" class="diff-lineno">Line 33:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>FarmShare supports any file-transfer method using SSH as a transport, including standard tools like <code>scp</code>, <code>sftp</code>, and <code>rsync</code> on Linux and macOS systems, and SFTP clients like [https://uit.stanford.edu/software/fetch Fetch] for macOS and [https://uit.stanford.edu/software/scrt_sfx SecureFX] for Windows. Because 2-step authentication is required you may need enable keep-alive in your preferred SFTP client to avoid repeated authentications. For Fetch, in the Preferences dialog, select <code>General</code> → <code>FTP compatibility</code> → <code>Keep connections alive</code>; for SecureFX, in Global Options, select <code>File Transfer</code> → <code>Options</code> → <code>Advanced</code> → <code>Options</code> → <code>Keep connections alive</code>.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>FarmShare supports any file-transfer method using SSH as a transport, including standard tools like <code>scp</code>, <code>sftp</code>, and <code>rsync</code> on Linux and macOS systems, and SFTP clients like [https://uit.stanford.edu/software/fetch Fetch] for macOS and [https://uit.stanford.edu/software/scrt_sfx SecureFX] for Windows. Because 2-step authentication is required you may need enable keep-alive in your preferred SFTP client to avoid repeated authentications. For Fetch, in the Preferences dialog, select <code>General</code> → <code>FTP compatibility</code> → <code>Keep connections alive</code>; for SecureFX, in Global Options, select <code>File Transfer</code> → <code>Options</code> → <code>Advanced</code> → <code>Options</code> → <code>Keep connections alive</code>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>You can also use FUSE and SSHFS to mount your FarmShare home and scratch directories. Most Linux distributions provide a standard <code>sshfs</code> package. On macOS you can use [https://brew.sh Homebrew] to install the <code>osxfuse</code> and <code>sshfs</code> packages, or download FUSE and SSHFS installers from the [https://osxfuse.github.io FUSE for macOS] project. Support for this option on Windows typically requires commercial software (like [https://www.expandrive.com ExpanDrive]).</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>You can also use FUSE and <ins class="diffchange diffchange-inline">[[</ins>SSHFS<ins class="diffchange diffchange-inline">]] </ins>to mount your FarmShare home and scratch directories. Most Linux distributions provide a standard <code>sshfs</code> package. On macOS you can use [https://brew.sh Homebrew] to install the <code>osxfuse</code> and <code>sshfs</code> packages, or download FUSE and SSHFS installers from the [https://osxfuse.github.io FUSE for macOS] project. Support for this option on Windows typically requires commercial software (like [https://www.expandrive.com ExpanDrive]).</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Using AFS ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Using AFS ===</div></td></tr>
</table>Epochhttps://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4633&oldid=prevEpoch: /* Using SSH */2017-12-04T20:29:56Z<p><span class="autocomment">Using SSH</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 20:29, 4 December 2017</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 33:</td>
<td colspan="2" class="diff-lineno">Line 33:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>FarmShare supports any file-transfer method using SSH as a transport, including standard tools like <code>scp</code>, <code>sftp</code>, and <code>rsync</code> on Linux and macOS systems, and SFTP clients like [https://uit.stanford.edu/software/fetch Fetch] for macOS and [https://uit.stanford.edu/software/scrt_sfx SecureFX] for Windows. Because 2-step authentication is required you may need enable keep-alive in your preferred SFTP client to avoid repeated authentications. For Fetch, in the Preferences dialog, select <code>General</code> → <code>FTP compatibility</code> → <code>Keep connections alive</code>; for SecureFX, in Global Options, select <code>File Transfer</code> → <code>Options</code> → <code>Advanced</code> → <code>Options</code> → <code>Keep connections alive</code>.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>FarmShare supports any file-transfer method using SSH as a transport, including standard tools like <code>scp</code>, <code>sftp</code>, and <code>rsync</code> on Linux and macOS systems, and SFTP clients like [https://uit.stanford.edu/software/fetch Fetch] for macOS and [https://uit.stanford.edu/software/scrt_sfx SecureFX] for Windows. Because 2-step authentication is required you may need enable keep-alive in your preferred SFTP client to avoid repeated authentications. For Fetch, in the Preferences dialog, select <code>General</code> → <code>FTP compatibility</code> → <code>Keep connections alive</code>; for SecureFX, in Global Options, select <code>File Transfer</code> → <code>Options</code> → <code>Advanced</code> → <code>Options</code> → <code>Keep connections alive</code>.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>You can also use FUSE and SSHFS to mount your FarmShare home and scratch directories. Most Linux distributions provide a standard <code>sshfs</code> package. On macOS you can use [https://brew.sh Homebrew] to install the <code>sshfs</code> <del class="diffchange diffchange-inline">package</del>, or download FUSE and SSHFS installers from the [https://osxfuse.github.io FUSE for macOS] project. Support for this option on Windows typically requires commercial software (like [https://www.expandrive.com ExpanDrive]).</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>You can also use FUSE and SSHFS to mount your FarmShare home and scratch directories. Most Linux distributions provide a standard <code>sshfs</code> package. On macOS you can use [https://brew.sh Homebrew] to install the <ins class="diffchange diffchange-inline"><code>osxfuse</code> and </ins><code>sshfs</code> <ins class="diffchange diffchange-inline">packages</ins>, or download FUSE and SSHFS installers from the [https://osxfuse.github.io FUSE for macOS] project. Support for this option on Windows typically requires commercial software (like [https://www.expandrive.com ExpanDrive]).</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Using AFS ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Using AFS ===</div></td></tr>
</table>Epochhttps://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4511&oldid=prevEpoch: /* Scratch */2017-10-09T23:23:40Z<p><span class="autocomment">Scratch</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:23, 9 October 2017</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 21:</td>
<td colspan="2" class="diff-lineno">Line 21:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Scratch ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Scratch ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>Scratch storage is available in <code>/farmshare/user_data</code>, and each user is provided with a personal scratch directory, <code>/farmshare/user_data/<del class="diffchange diffchange-inline">sunetid</del></code>. The total volume size is currently 126 TB; quotas are not currently enforced, but old files may be purged without warning. The scratch volume is ''not'' backed up, and is ''not'' suitable for long-term storage, but can be used as working storage for batch jobs, and as a short-term staging area for data waiting to be archived to permanent storage.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>Scratch storage is available in <code>/farmshare/user_data</code>, and each user is provided with a personal scratch directory, <code>/farmshare/user_data/<ins class="diffchange diffchange-inline">$USER</ins></code>. The total volume size is currently 126 TB; quotas are not currently enforced, but old files may be purged without warning. The scratch volume is ''not'' backed up, and is ''not'' suitable for long-term storage, but can be used as working storage for batch jobs, and as a short-term staging area for data waiting to be archived to permanent storage.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Temp ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Temp ===</div></td></tr>
</table>Epochhttps://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4509&oldid=prevEpoch: /* AFS */2017-10-09T23:22:49Z<p><span class="autocomment">AFS</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 23:22, 9 October 2017</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 15:</td>
<td colspan="2" class="diff-lineno">Line 15:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== AFS ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== AFS ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'>-</td><td style="background: #ffa; color:black; font-size: smaller;"><div>[https://uit.stanford.edu/service/afs AFS] is accessible from <code>rice</code> systems ''only''. A link to each user's AFS home directory, <code>~/afs-home</code> is provided as a convenience, but should only be used to access files in the legacy environment, and for transferring data. It should ''not'' be used as a working directory when submitting batch jobs, as AFS is not accessible from compute nodes. Please note that a valid Kerberos ticket and an [[AFS#Authentication|AFS token]] are required to access locations in AFS; run <code>kinit && aklog</code> to re-authenticate if you have trouble accessing any AFS directory.</div></td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div>[https://uit.stanford.edu/service/afs AFS] is accessible from <code>rice</code> systems ''only''. A link to each user's AFS home directory, <code>~/afs-home</code><ins class="diffchange diffchange-inline">, </ins>is provided as a convenience, but should only be used to access files in the legacy environment, and for transferring data. It should ''not'' be used as a working directory when submitting batch jobs, as AFS is not accessible from compute nodes. Please note that a valid Kerberos ticket and an [[AFS#Authentication|AFS token]] are required to access locations in AFS; run <code>kinit && aklog</code> to re-authenticate if you have trouble accessing any AFS directory.</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The default, per-user quota for AFS home directories is 5 GB, but you may have additional quota due to your enrollment in certain courses, and you can [https://tools.stanford.edu/cgi-bin/afs-request request] additional quota (up to 20 GB total) with faculty sponsorship. AFS is backed up every night, and backups are kept for 30 days. The most recent snapshot of your AFS home directory is available in the <code>.backup</code> subdirectory, and you can request recovery from older backups by submitting a [https://helpsu.stanford.edu HelpSU] ticket.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>The default, per-user quota for AFS home directories is 5 GB, but you may have additional quota due to your enrollment in certain courses, and you can [https://tools.stanford.edu/cgi-bin/afs-request request] additional quota (up to 20 GB total) with faculty sponsorship. AFS is backed up every night, and backups are kept for 30 days. The most recent snapshot of your AFS home directory is available in the <code>.backup</code> subdirectory, and you can request recovery from older backups by submitting a [https://helpsu.stanford.edu HelpSU] ticket.</div></td></tr>
</table>Epochhttps://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4333&oldid=prevEpoch: /* Limits */2017-09-22T01:14:42Z<p><span class="autocomment">Limits</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 01:14, 22 September 2017</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 80:</td>
<td colspan="2" class="diff-lineno">Line 80:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><source lang="sh">sbatch --partition=gpu --gres=gpu:1</source></div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><source lang="sh">sbatch --partition=gpu --gres=gpu:1</source></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">The <code>bigmem</code> quality-of-service has a minimum memory requirement; you must request at least 96GB when submitting a job.</ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"><source lang="sh">sbatch --partition=bigmem --mem=96G</source></ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Monitoring your Jobs ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Monitoring your Jobs ===</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>You can use the <code>squeue</code> and <code>sacct</code> commands to monitor the current state of the scheduler and of your jobs. The <code>sprio</code> command can provide some information on how priority was determined for particular jobs, and the <code>sshare</code> command on how current fair-share was calculated. Use the <code>scontrol</code> and <code>sacctmgr</code> commands to examine the configuration of hosts, partitions, and qualities-of-service.</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>You can use the <code>squeue</code> and <code>sacct</code> commands to monitor the current state of the scheduler and of your jobs. The <code>sprio</code> command can provide some information on how priority was determined for particular jobs, and the <code>sshare</code> command on how current fair-share was calculated. Use the <code>scontrol</code> and <code>sacctmgr</code> commands to examine the configuration of hosts, partitions, and qualities-of-service.</div></td></tr>
</table>Epochhttps://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php?title=User_Guide&diff=4319&oldid=prevEpoch: /* Interactive Jobs */2017-09-18T19:19:48Z<p><span class="autocomment">Interactive Jobs</span></p>
<table style="background-color: white; color:black;">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr valign='top'>
<td colspan='2' style="background-color: white; color:black;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black;">Revision as of 19:19, 18 September 2017</td>
</tr><tr><td colspan="2" class="diff-lineno">Line 56:</td>
<td colspan="2" class="diff-lineno">Line 56:</td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><source lang="sh">srun --pty --qos=interactive $SHELL -l</source></div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div><source lang="sh">srun --pty --qos=interactive $SHELL -l</source></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;"></ins></div></td></tr>
<tr><td colspan="2"> </td><td class='diff-marker'>+</td><td style="background: #cfc; color:black; font-size: smaller;"><div><ins style="color: red; font-weight: bold; text-decoration: none;">Interactive jobs are not scheduled any differently than batch jobs: when contention for resources is high, interactive jobs may wait a long time before starting.</ins></div></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Batch Jobs ===</div></td><td class='diff-marker'> </td><td style="background: #eee; color:black; font-size: smaller;"><div>=== Batch Jobs ===</div></td></tr>
</table>Epoch