Walk through for creating a python environment

Hopefully a self contained tutorial on how to create your own python environment which to a great extent does not at all rely on the sys-admins on the cluster.

First thing we're going to do is download the miniconda installer from the miniconda website. Go here and pick your preferred flavor of python (I recommend 3.6), right click on the 64-bit bash installer for linux, and click on copy link address.

Then ssh into spock: ssh your_username@spock.princeton.edu, now you want to paste the link after wget, so the command should look something like:

wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh # ubuntu, debian, ect.
wget https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh # mac

that will put the file Miniconda3-latest-Linux-x86_64.sh in the directory that you ran wget from. Once you've done that you can do: bash Miniconda3-latest-Linux-x86_64.sh this will install miniconda just for you, remember to say yes when it asks to add miniconda to your path.

Great, now do: source ~/.bashrc, and try typing in the command conda, if all is good you'll see some help text. If there are no hickups up until this point, lets create a python environment. The following command will create it in a "base" directory in your ~/user directory on spock, and it will give it a name that you specify:

conda create -n your_env_name python=3.6

About the above, your_env_name is the name you specific you can call this whatever you might like, I call it py36 to remind myself it's python 3.6, then the part python=3.6 tells anaconda what version of python you want installed. You can replace 3.6 for 2.7 or any supported version number. Doing this will prompt you to say yes about installing python and some required packages, say yes. At the end it will plop out some text on the screen that look similar to:

#
# To activate this environment, use:
# > source activate your_env_name
#
# To deactivate this environment, use:
# > source deactivate your_env_name
#

what this means is, typing source activate your_env_name into the terminal and hitting enter will get you into your custom python environment. However, we're going to make it so that you don't need to that this, ever.

The following is optional but recommend.

First, open up your ~/.bashrc file with vim. To do that type:

vi ~/.bashrc

into the terminal and hit enter. Now you want to hit the key i on your keyboard to get into edit mode. Now in a new line on the file add the following:

alias your_env_name="source activate your_env_name"

remember, that your_env_name will be the variable name you create and should be replaced everywhere accordingly. After you do that hit the key esc on your keyboard to get out of edit mode. Now hit the keys shift+; to make a :, then hit the key x and press enter, this will save the changes to your ~/.bashrc file and get you out of vim. Again, now we'll do source ~/.bashrc, which will cause the changes we've made to the ~/.bashrc file to take effect. You should now be able to type your_env_name and as a result, get "into" your custom python environment.

Optional section done, below is required

OK so you're in your environment, what you want to do now is install all the packages you might need. This is simple, however it should be noted that you should only do this after you've gotten into your environment via either source activate your_env_name or the alias command that you created in the optional section above. Only after you've done that you should begin to install packages, and you'll do that like so:

conda install numpy pandas scipy scikit-learn

the above will install multiple packages at a time, but you can do it one at a time if you like. This means you'll need to do multiple conda install commands so it'll look something like:

conda install numpy
conda install pandas
conda install scipy
conda install scikit-learn

you can use the patterns above to install any package you like.

Conclusion

So that's it for the most part. One thing to note is, any time you ssh into spock you will need to perform one of:

source activate your_env_name or just your_env_name, depending on the route you took above, (you can think of this just as when you have to start up matlab or julia, you type "matlab" or "julia" in the terminal to get into a session.