SCIEN data

SCIEN data is a collection of directories that contain relatively large files needed for different [http://scien.stanford.edu SCIEN] projects. Here, we will describe how to download these data sets from a web-browser or from within Matlab.

There is a parallel MRVISTA data collection for MR data sets.

= Placing data on the server =

The data are stored on a disk with 30TB of space that can be accessed from wandell-lab machines such as black.stanford.edu.

The disk is mounted as /wandellfs/data/validation/SCIEN/, and you can browse that partition from [http://scarlet.stanford.edu/validation/SCIEN this link].  You will see that various open data sets related to  ISET, ISETBIO, L3, and LIGHTFIELDS are contained there.  (Note:  The name of the directory may change in the next month or two.)  

You can browse the data on the web at [http://scarlet.stanford.edu/validation/SCIEN this site].

To copy data to the storage area, you will need sudo permission (ask Michael).  You can then login to black, sftp the files to black and move them into the appropriate directory in the partition.  For example, to add files for yourPROJECT, you would 

 * place yourFILES on black
 * make sure there is an appropriate directory for yourPROJECT
 * sudo cp yourFILES /wandellfs/data/validation/SCIEN/yourPROJECT/

= Description and index files =

For the files to appear properly in the browser, you must do two more things.  First, create a soft link named index.php in the yourPROJECT directory.  This link should be 
 
 ln -s /wandellfs/data/validation/category_index.php* index.php

Then, within the yourPROJECT directory create a description.txt file.  This file describes the directory contents.   

= Data organization =

You may wish to include helpful additional files in the directories. For example, you might want to include a Matlab m-file that can be executed and provide you a description of the files and their organization or commands that automate downloading the entire set of files.

Also, be aware the the ISETBIO team is creating a [https://github.com/isetbio/RemoteDataToolbox RemoteDataToolbox] that we hope will simplify the process of downloading (and possibly uploading) data.