SCIEN Software

From VISTA LAB WIKI

(Difference between revisions)
Jump to: navigation, search
(Data)
(Project on scientific transparency (POST))
 
(6 intermediate revisions not shown)
Line 1: Line 1:
-
We are part of [http://scien.stanford.edu Stanford's Center for Image Systems Engineering].  The software we develop for this work is distributed through our [http://github.com github] repositories. These are links to those repositories
+
The software we develop for this work is distributed through our [http://github.com github] repositories.  
-
 
+
-
* [https://github.com/scienstanford SCIENSTANFORD]
+
-
* [https://github.com/isetbio ISETBIO]
+
-
 
+
-
We also maintain some additional github repositories that are described in specific portions of the documentation.
+
Here is a local page [http://white.stanford.edu/newlm/index.php/GIT describing the basic GIT operations].
Here is a local page [http://white.stanford.edu/newlm/index.php/GIT describing the basic GIT operations].
-
= Scientific methods =
+
= Software =
-
== Software revisions ==
+
We are part of [http://scien.stanford.edu Stanford's Center for Image Systems Engineering].  Several projects related to image systems engineering are stored at these accounts.
-
== Data ==
+
* [https://github.com/scienstanford SCIENSTANFORD]
 +
* [https://github.com/isetbio ISETBIO]
 +
* [http://imageval.com ImageVal Web site for ISET]
-
We are storing critical data sets for [http://52.32.77.154/#browse~isetbio ISETBIO] and [http://52.32.77.154/#browse~scien SCIEN] and [http://52.32.77.154/#browse~vistasoft  VISTASOFT] on the Archiva server.
 
-
We interact with these databases using the [https://github.com/isetbio/RemoteDataToolbox RemoteDataToolbox].
+
The github repositories maintain a series of programs related to our database and computational work in our  [http://post.stanford.edu project on scientific transparency].
-
== [[PDCDATA]] ==
+
* [https://github.com/orgs/scitran/dashboard SCITRAN]
 +
* [https://github.com/scitran/core/wiki SCITRAN WIKI]
-
Needs updating. We will probably start maintaining these data sets elsewhere.
+
See also:
-
We maintain a repository for data files, [[PDCDATA]]. The data sets can be checked out from the ee svn repository in the usual way, just substituting pdcdata for pdcsoft. There exist folders for reflectances, illuminants, RGB images, multispectral scenes, sensors, etc.
+
* [https://github.com/flywheel-io Flywheel Exchange, LLC]
 +
* [https://flywheel.io Flywheel Exchange, LLC]
-
If you wish to add data to this directory, please follow the guidelines outlined in the README files in the directories.  All data of a particular type should have a consistent format (outlined in the README files) to ensure interoperability of code and data.
+
= Data =
-
There is a separate repository for multispectral data for scenes because these files tend to be very large.  The repository is called "scenedata"  (https://white.stanford.edu/ee/scenedata).
+
We are storing critical data sets for  
-
== Project on scientific transparency (POST) ==
+
* [http://52.32.77.154/#browse~isetbio ISETBIO]
 +
* [http://52.32.77.154/#browse~scien SCIEN]
 +
* [http://52.32.77.154/#browse~vistasoft  VISTASOFT]
-
= Running Matlab Remotely =
+
on the Archiva server.  We interact with these databases using the [https://github.com/isetbio/RemoteDataToolbox RemoteDataToolbox].
-
If you have an account on white, you can run Matlab remotely on other computers in the lab. This is a rather simple process.
 
-
== Cygwin ==
 
-
If you are on Windows, you can use Cygwin/X [http://x.cygwin.com/] to have a graphical interface with Matlab running on a server.  When installing the software, make sure to choose to install X as described here [http://x.cygwin.com/docs/ug/setup.html] and OpenSSH (under net category).  Once installed and running XWin Server on your machine, you can login to a machine such as peach by [ssh -X2 username@peach].
 
-
== Xming ==
+
= Running Matlab Remotely =
-
If you are on a Windows machine, you can also use Xming to open xterminals [http://www.straightrunning.com/XmingNotes] on the linux machines and run Matlab remotely.
+
We use Microsoft Remote Desktop to login to servers and run remotely.
-
 
+
-
From off-campus do this
+
-
 
+
-
1. Download and install Xming [http://sourceforge.net/projects/xming/files/Xming/6.9.0.31/Xming-6-9-0-31-setup.exe/download]
+
-
2. Use SecureCRT to login to white.stanford.edu
+
-
3. Type "ssh azure" to open a window on azure
+
-
4. Type "xterm &" to open an X window
+
-
5. Type  "matlabr2010aREAL &" to open Matlab in a separate window.
+
-
 
+
-
From on-campus you can log into azure or viridian or other machines directly, without logging into white first.
+
-
 
+
-
== No graphics ==
+
-
 
+
-
If you don't need the graphical interface, you have another option.  You need an SSH client such as [http://www.stanford.edu/services/ess/pc/securecrt.html SecureCRT] if you are on Windows.
+
-
 
+
-
  1. Start your client
+
-
  2. SSH into one of the machines, e.g., azure.stanford.edu
+
-
  3. Type:  'matlab' and this will open up a version of Matlab in text only mode.
+
-
 
+
-
If you want to run a Matlab process that will continue in the background even if you close your SSH session.  Execute the following command:
+
-
 
+
-
  nohup matlab < to_do.m > results.txt &
+
-
 
+
-
Here Matlab will execute the file to_do.m in the background and will save any text displayed in the Command Window to results.txt.  The above command will also show also return the ID number for this task.  If you execute the command 'top' you can see all the processes currently running on the machine.  You can use this to check if your program is still running.  If for some reason you would like to terminate your program, execute 'kill #####' where the number is the ID number of the process you want to kill.  You can execute the following command to view the contents of results.txt:
+
-
 
+
-
  less results.txt
+
-
 
+
-
If you need to execute a file that is not in your home directory, you can change directories using:
+
-
 
+
-
  cd directory_name
+
-
 
+
-
This method will result in only the current folder being on Matlab's path.  If you want to add all of the subdirectories of the current folder to Matlab's path, add the following command to your .m file:
+
-
 
+
-
  addpath(genpath(pwd))
+
-
 
+
-
The 'azure' computer can run multiple scripts at the same time.  You can view the currently running processes by executing the command 'top'.
+
-
 
+
-
In order to transfer files to or from white, use a SFTP client such as [http://winscp.net/eng/index.php WinSCP] if you are on Windows and connect to white.stanford.edu.
+
= Virtual Box =
= Virtual Box =
Line 131: Line 90:
Additional comments or notes can be added after the description of the output(s).
Additional comments or notes can be added after the description of the output(s).
-
 
+
= [http://white.stanford.edu/newlm/index.php/SVN SVN] (DEPRECATED) =  
-
= [[ISET]] =
+
-
 
+
-
The see the [http://imageval.com ImageVal Web site] for information about this too.
+
-
 
+
-
= [http://white.stanford.edu/newlm/index.php/SVN SVN] (TO BE DEPRECATED) =  
+
== Repository Browsing ==
== Repository Browsing ==
As far as I know, everything is now on github or the RDT (Maven/Archiva) repository.
As far as I know, everything is now on github or the RDT (Maven/Archiva) repository.
 +
 +
I leave these links here just in case we forgot something.
* [http://white.stanford.edu/trac/pdcprojects/browser PDCProjects]
* [http://white.stanford.edu/trac/pdcprojects/browser PDCProjects]
* [http://white.stanford.edu/trac/pdcsoft/browser/trunk PDCSoft]
* [http://white.stanford.edu/trac/pdcsoft/browser/trunk PDCSoft]
* [http://white.stanford.edu/trac/iset-4.0/browser ISET-4.0]
* [http://white.stanford.edu/trac/iset-4.0/browser ISET-4.0]
-
 
-
These links show you the repository directories as well as the history of the different files and directories in the repositories.
 

Latest revision as of 08:45, 23 April 2016

The software we develop for this work is distributed through our github repositories.

Here is a local page describing the basic GIT operations.

Contents

[edit] Software

We are part of Stanford's Center for Image Systems Engineering. Several projects related to image systems engineering are stored at these accounts.


The github repositories maintain a series of programs related to our database and computational work in our project on scientific transparency.

See also:

[edit] Data

We are storing critical data sets for

on the Archiva server. We interact with these databases using the RemoteDataToolbox.


[edit] Running Matlab Remotely

We use Microsoft Remote Desktop to login to servers and run remotely.

[edit] Virtual Box

Andy Lin has us running Ubuntu on Windows machines using VirtualBox. This is a free virtual machine system from Oracle.

Install Virtual Box on a Windows machine. Then, download a version of Ubuntu.

You can then create a virtual instance of Ubuntu that will run on the Windows machine. Some suggestions:

  1. Set up the Machine properties for more than 1 processor.
  2. Set an execution cap that is less than 100 percent so the Windows side of your machine can continue to run
  3. Use more RAM (e.g., 4 or 5 GB).
  4. Use a virtual disk of 15-20GB at least.

We will try to make the notes here a little better as more people go through the process. Though it may be that only Andy and Brian have Windows machines.

[edit] Writing Matlab Files

When you are writing Matlab scripts or functions, please follow some guidelines in order to allow others to be able to use and understand your code and its outputs. It is important that the code is written in a readable modular fashion so that other people in the lab (now and in the future) may benefit from and work with your code.

At the top of each of your files, please add a header describing what it does. The suggested format for a simple example function appears below:

 function ycbcr = myrgb2ycbcr(rgb)
 % Converts an image using the color conversion used in JPEG
 % 
 %     ycbcr=myrgb2ycbcr(rgb)
 %
 % A few sentences about the function there ...
 %
 % ...
 % 
 %
 %INPUT:
 %   rgb:        Double array giving an RGB image with values between 0 and 255
 %               size(rgb)=m x n x 3
 %
 %OUTPUT:
 %   ycbcr:      Double array giving a YCbCr image with values between 0 and 255
 %               size(ycbcr)=m x n x 3
 %
 % See also:  Related functions
 %
 % Examples:
 %    Examples go here
 %
 %
 % (c) Copyright notice


Additional comments or notes can be added after the description of the output(s).

[edit] SVN (DEPRECATED)

[edit] Repository Browsing

As far as I know, everything is now on github or the RDT (Maven/Archiva) repository.

I leave these links here just in case we forgot something.

Personal tools