Working on a *nix Cluster
By now we know how access a remote cluster. How to navigate a Linux environment. How to create and destroy files and directories. And how to transfer files from one machine to another. This constitutes the bulk of what you need to know to use a *nix cluster effectively, but not necessarily efficiently. There are a few more topics that can make your day to day life on the cluster much more pleasant.
Controlling the bash Environment
Recall that our default command line interpreter on Cypress is bash (the Bourne Again SHell).
Environment Variables
There are certain variables whose values determine the behavior of the bash environment. These are called environment variables and are traditionally denoted by ALL CAPS. To see the values of all our environment variables we can invoke the env command
[tuhpc002@cypress1 ~]$ env
MODULE_VERSION_STACK=3.2.10
MANPATH=/cm/shared/apps/slurm/14.03.0/man:/usr/local/share/man:/usr/share/man/overrides:/usr/share/man/en:/usr/share/man:/cm/local/apps/environment-modules/current/share/man
HOSTNAME=cypress1
TERM=xterm-256color
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=72.211.65.92 58652 22
LIBRARY_PATH=/cm/shared/apps/slurm/14.03.0/lib64/slurm:/cm/shared/apps/slurm/14.03.0/lib64
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/0
USER=tuhpc002
LD_LIBRARY_PATH=/cm/shared/apps/slurm/14.03.0/lib64/slurm:/cm/shared/apps/slurm/14.03.0/lib64
LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lz=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.bz=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.rar=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
CPATH=/cm/shared/apps/slurm/14.03.0/include
MODULE_VERSION=3.2.10
MAIL=/var/spool/mail/tuhpc002
PATH=/usr/lib64/qt-3.3/bin:/share/apps/bbcp/amd64_rhel60/bin:/share/apps/idev/bin:/cm/shared/apps/slurm/14.03.0/sbin:/cm/shared/apps/slurm/14.03.0/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/opt/dell/srvadmin/bin:/home/workshop/tuhpc002/bin
PWD=/home/workshop/tuhpc002
_LMFILES_=/cm/shared/modulefiles/slurm/14.03.0:/share/apps/modulefiles/idev:/share/apps/modulefiles/bbcp/amd64_rhel60
LANG=en_US.UTF-8
MODULEPATH=/cm/local/modulefiles:/cm/shared/modulefiles:/share/apps/modulefiles
LOADEDMODULES=slurm/14.03.0:idev:bbcp/amd64_rhel60
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/workshop/tuhpc002
LOGNAME=tuhpc002
QTLIB=/usr/lib64/qt-3.3/lib
CVS_RSH=ssh
SSH_CONNECTION=72.211.65.92 58652 76.165.13.3 22
MODULESHOME=/cm/local/apps/environment-modules/3.2.10/Modules/3.2.10
LESSOPEN=|/usr/bin/lesspipe.sh %s
G_BROKEN_FILENAMES=1
module=() { eval `/cm/local/apps/environment-modules/3.2.10/Modules/$MODULE_VERSION/bin/modulecmd bash $*`
}
_=/bin/env
OLDPWD=/home/workshop/tuhpc002/examples
[tuhpc002@cypress1 ~]$
This list can easily be searched by "piping" (|)it's output to grep
[tuhpc002@cypress1 ~]$ env | grep PATH MANPATH=/cm/shared/apps/slurm/14.03.0/man:/usr/local/share/man:/usr/share/man/overrides:/usr/share/man/en:/usr/share/man:/cm/local/apps/environment-modules/current/share/man LIBRARY_PATH=/cm/shared/apps/slurm/14.03.0/lib64/slurm:/cm/shared/apps/slurm/14.03.0/lib64 LD_LIBRARY_PATH=/cm/shared/apps/slurm/14.03.0/lib64/slurm:/cm/shared/apps/slurm/14.03.0/lib64 CPATH=/cm/shared/apps/slurm/14.03.0/include PATH=/usr/lib64/qt-3.3/bin:/share/apps/bbcp/amd64_rhel60/bin:/share/apps/idev/bin:/cm/shared/apps/slurm/14.03.0/sbin:/cm/shared/apps/slurm/14.03.0/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/opt/dell/srvadmin/bin:/home/workshop/tuhpc002/bin MODULEPATH=/cm/local/modulefiles:/cm/shared/modulefiles:/share/apps/modulefiles [tuhpc002@cypress1 ~]$
To use the information stored in the environment variable, we invoke it with the prefix $.
[tuhpc002@cypress1 examples]$ pwd /home/workshop/tuhpc002/examples [tuhpc002@cypress1 examples]$ cd $HOME [tuhpc002@cypress1 ~]$ pwd /home/workshop/tuhpc002 [tuhpc002@cypress1 ~]$
The value of a single variable can be seen by using the echo command
[tuhpc002@cypress1 ~]$ echo $HOME /home/workshop/tuhpc002 [tuhpc002@cypress1 ~]$
Also the value of an environment variable can be changed (or a new variable created) by invoking export. In general such changes will remain in effect throughout the remainder of your session - as well as for the commands and programs you execute therein.
[tuhpc002@cypress1 ~]$ export EXAMPLEDIR=$HOME/examples/ [tuhpc002@cypress1 ~]$ echo $EXAMPLEDIR /home/workshop/tuhpc002/examples/ [tuhpc002@cypress1 ~]$
The PATH variable
The PATH environment variable establishes a search path for executables. That is, when you type a command, the OS searches for that executable starting with the first directory listed in the PATH variable and moving along until it finds a match. Note that order matters, if multiple instances of an executable exist in the path, the one contained in the directory listed first will be used.
[tuhpc002@cypress1 ~]$ echo $PATH /usr/lib64/qt-3.3/bin:/share/apps/bbcp/amd64_rhel60/bin:/share/apps/idev/bin:/cm/shared/apps/slurm/14.03.0/sbin:/cm/shared/apps/slurm/14.03.0/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/opt/dell/srvadmin/bin:/home/workshop/tuhpc002/bin [tuhpc002@cypress1 ~]$
bash_profile and bashrc
We can control the behavior of bash at login by editing the hidden configuration files in our home directory.
[tuhpc002@cypress1 ~]$ ls -al .bash* -rw------- 1 tuhpc002 workshop 1259 Aug 18 22:34 .bash_history -rw-r--r-- 1 tuhpc002 workshop 18 Jul 18 2013 .bash_logout -rw-r--r-- 1 tuhpc002 workshop 176 Jul 18 2013 .bash_profile -rw-r--r-- 1 tuhpc002 workshop 124 Sep 30 2014 .bashrc [tuhpc002@cypress1 ~]$
The files we wish to concentrate on are ".bash_profile" and ".bashrc". Let's examine their contents
[tuhpc002@cypress1 ~]$ cat .bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then . ~/.bashrc fi # User specific environment and startup programs PATH=$PATH:$HOME/bin export PATH [tuhpc002@cypress1 ~]$ cat .bashrc # .bashrc # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi # User specific aliases and functions [tuhpc002@cypress1 ~]$
You can see that the two are nearly identical. Aside from their reference to different directories (/etc vs ~), the only other difference is that our .bash_profile modifies and exports the PATH environment variable.
There is much confusion over the role that each of these files play. When you login to a system, the .bash_profile file is executed UPON LOGIN. The .bashrc file modifies the default environment when connected without login, for example when executing commands remotely using SSH. Note that changes made in the .bashrc file will be reflected in the .bash_profile file but not vise versa.
SSH config (Linux and Mac)
It can be cumbersome to type your username and host address every time you login to a machine. The .ssh/config file offers the ability to create aliases to speed up this process. Add the following to your .ssh/config file to create a shortcut
Host cypress1 Hostname cypress1.tulane.edu User <your user ID on cypress>
Now from your local machine you can simply type ssh cypress1 and the username and hostname will be set automatically.
