Getting started with Git

If you have a large shared software project and want an easy way to manage collaboration, branch off different versions of the project and generally keep things organized, you need a version control system. Git is the one I know (Mercurial and Subversion are also popular choices) and the one other people in the group are using, so it's the one I'll go over here.

Download git here: http://git-scm.com/download

Set up an ssh public key and register an account at github.com: http://help.github.com/mac-key-setup/ (apologies to non-Mac users)

Enter the following at the terminal (with obvious substitutions):

PATH=$PATH:/usr/local/git/bin/

export PATH

git config --global user.name "Your Name"

git config --global user.email your.email@something.com

To pull a repository called "repo" from user "person":

mkdir repo (or what have you)

cd repo

git init

git remote add origin git@github.com:person/repo.git

git pull origin master

In particular, every call of "git pull origin master" pulls the currest master version off github. Be sure you're working with the current version before trying to push local changes, or you might get conflicts. To commit changes, either call

git add files_that_changed

git commit

or

git commit -a

which commits all changed files. Committing brings up a text editor where you describe any changes made. Be aware, if you don't write anything the commit will be aborted. Then to push your changes from the local machine to github just type

git push

and that's it! Things get messier when working with branches, checkouts and merges, but for 90% of what I do the above suffices. The web abounds with tutorials and a quick reference sheet can be found here.

Parallel computing : matlab on the HPC cluster

I've improved the codes for the parallel computing, which I talked about at the seminar a month ago - it should be really simple to use now :). Also, the problem with the atomic operation, at least under Linux, is solved as well now. I've written up a description of the codes, commented them and compiled two examples: one to be run on a single machine with several copies of Matlab running in parallel and another is for the HPC cluster. Everything can be found here: http://neurotheory.columbia.edu/~max/codes/ParallelComputation.zip

If you have comments, suggestions - will be happy to hear! Will also be glad to help resolving problems, if they arise, or to explain the code, if needed. Also, if you start using the code, please let me know - it's always encouraging to know that the work goes to masses :).

Submit jobs to the HPC cluster from matlab

While we are talking about tools for using the HPC cluster, here's an ad for a tool of my own.

I have been using agricola to submit jobs to the HPC cluster from within matlab.  It is a very simple tool:  Instead of launching a calculation on your local machine by typing in the matlab prompt:

my_result = my_function( some_parameters ) ;

one types:

sow( 'my_result' , @()my_function( some_parameters ) ) ;

This will copy all the .m files in your current directory into a folder on the HPC submit machine, generate a submit file there, and launch the calculation on the cluster. Then some time later, when you suspect the job is done, you type:

reap

which makes the variable  my_result  appear in your matlab workspace.  reap itself returns all the .out, .log, and .err files for you to look at from within matlab.

Unlike Max's code, agricola does not aim to parallelize your code; it just handles sending files back and forth with ssh and job submission.