Working on a *nix Cluster

By now we know how access a remote cluster. How to navigate a Linux environment. How to create and destroy files and directories. And how to transfer files from one machine to another. This constitutes the bulk of what you need to know to use a *nix cluster effectively, but not necessarily efficiently. There are a few more topics that can make your day to day life on the cluster much more pleasant.

Controlling the bash Environment

Recall that our default command line interpreter on Cypress is bash (the Bourne Again SHell).

Environment Variables

There are certain variables whose values determine the behavior of the bash environment. These are called environment variables and are traditionally denoted by ALL CAPS. To see the values of all our environment variables we can invoke the env command

[tuhpc002@cypress1 ~]$ env
MODULE_VERSION_STACK=3.2.10
MANPATH=/cm/shared/apps/slurm/14.03.0/man:/usr/local/share/man:/usr/share/man/overrides:/usr/share/man/en:/usr/share/man:/cm/local/apps/environment-modules/current/share/man
HOSTNAME=cypress1
TERM=xterm-256color
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=72.211.65.92 58652 22
LIBRARY_PATH=/cm/shared/apps/slurm/14.03.0/lib64/slurm:/cm/shared/apps/slurm/14.03.0/lib64
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/0
USER=tuhpc002
LD_LIBRARY_PATH=/cm/shared/apps/slurm/14.03.0/lib64/slurm:/cm/shared/apps/slurm/14.03.0/lib64
LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lz=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.bz=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.rar=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
CPATH=/cm/shared/apps/slurm/14.03.0/include
MODULE_VERSION=3.2.10
MAIL=/var/spool/mail/tuhpc002
PATH=/usr/lib64/qt-3.3/bin:/share/apps/bbcp/amd64_rhel60/bin:/share/apps/idev/bin:/cm/shared/apps/slurm/14.03.0/sbin:/cm/shared/apps/slurm/14.03.0/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/opt/dell/srvadmin/bin:/home/workshop/tuhpc002/bin
PWD=/home/workshop/tuhpc002
_LMFILES_=/cm/shared/modulefiles/slurm/14.03.0:/share/apps/modulefiles/idev:/share/apps/modulefiles/bbcp/amd64_rhel60
LANG=en_US.UTF-8
MODULEPATH=/cm/local/modulefiles:/cm/shared/modulefiles:/share/apps/modulefiles
LOADEDMODULES=slurm/14.03.0:idev:bbcp/amd64_rhel60
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/workshop/tuhpc002
LOGNAME=tuhpc002
QTLIB=/usr/lib64/qt-3.3/lib
CVS_RSH=ssh
SSH_CONNECTION=72.211.65.92 58652 76.165.13.3 22
MODULESHOME=/cm/local/apps/environment-modules/3.2.10/Modules/3.2.10
LESSOPEN=|/usr/bin/lesspipe.sh %s
G_BROKEN_FILENAMES=1
module=() {  eval `/cm/local/apps/environment-modules/3.2.10/Modules/$MODULE_VERSION/bin/modulecmd bash $*`
}
_=/bin/env
OLDPWD=/home/workshop/tuhpc002/examples
[tuhpc002@cypress1 ~]$ 

This list can easily be searched by "piping" (|)it's output to grep

[tuhpc002@cypress1 ~]$ env | grep PATH
MANPATH=/cm/shared/apps/slurm/14.03.0/man:/usr/local/share/man:/usr/share/man/overrides:/usr/share/man/en:/usr/share/man:/cm/local/apps/environment-modules/current/share/man
LIBRARY_PATH=/cm/shared/apps/slurm/14.03.0/lib64/slurm:/cm/shared/apps/slurm/14.03.0/lib64
LD_LIBRARY_PATH=/cm/shared/apps/slurm/14.03.0/lib64/slurm:/cm/shared/apps/slurm/14.03.0/lib64
CPATH=/cm/shared/apps/slurm/14.03.0/include
PATH=/usr/lib64/qt-3.3/bin:/share/apps/bbcp/amd64_rhel60/bin:/share/apps/idev/bin:/cm/shared/apps/slurm/14.03.0/sbin:/cm/shared/apps/slurm/14.03.0/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/opt/dell/srvadmin/bin:/home/workshop/tuhpc002/bin
MODULEPATH=/cm/local/modulefiles:/cm/shared/modulefiles:/share/apps/modulefiles
[tuhpc002@cypress1 ~]$ 

To use the information stored in the environment variable, we invoke it with the prefix $.

[tuhpc002@cypress1 examples]$ pwd
/home/workshop/tuhpc002/examples
[tuhpc002@cypress1 examples]$ cd $HOME
[tuhpc002@cypress1 ~]$ pwd
/home/workshop/tuhpc002
[tuhpc002@cypress1 ~]$ 

The value of a single variable can be seen by using the echo command

[tuhpc002@cypress1 ~]$ echo $HOME
/home/workshop/tuhpc002
[tuhpc002@cypress1 ~]$ 

An the value environment variable can be changed (or a new variable created) by invoking export.

[tuhpc002@cypress1 ~]$ export EXAMPLEDIR=$HOME/examples/
[tuhpc002@cypress1 ~]$ echo $EXAMPLEDIR
/home/workshop/tuhpc002/examples/
[tuhpc002@cypress1 ~]$ 

The PATH variable

The PATH environment variable establishes a search path for executables. That is, when you type a command, the OS searches for that executable starting with the first directory listed in the PATH variable and moving along until it finds a match. Note that order matters, if multiple instances of an executable exist in the path, the one contained in the directory listed first will be used.

[tuhpc002@cypress1 ~]$ echo $PATH
/usr/lib64/qt-3.3/bin:/share/apps/bbcp/amd64_rhel60/bin:/share/apps/idev/bin:/cm/shared/apps/slurm/14.03.0/sbin:/cm/shared/apps/slurm/14.03.0/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/opt/dell/srvadmin/bin:/home/workshop/tuhpc002/bin
[tuhpc002@cypress1 ~]$ 

bash_profile and bashrc

We can control the behavior of bash at login by editing the hidden configuration files in our home directory.

[tuhpc002@cypress1 ~]$ ls -al .bash*
-rw------- 1 tuhpc002 workshop 1259 Aug 18 22:34 .bash_history
-rw-r--r-- 1 tuhpc002 workshop   18 Jul 18  2013 .bash_logout
-rw-r--r-- 1 tuhpc002 workshop  176 Jul 18  2013 .bash_profile
-rw-r--r-- 1 tuhpc002 workshop  124 Sep 30  2014 .bashrc
[tuhpc002@cypress1 ~]$ 

The files we wish to concentrate on are ".bash_profile" and ".bashrc". Let's examine their contents

[tuhpc002@cypress1 ~]$ cat .bash_profile 
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
	. ~/.bashrc
fi

# User specific environment and startup programs

PATH=$PATH:$HOME/bin

export PATH

[tuhpc002@cypress1 ~]$ cat .bashrc 
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

# User specific aliases and functions
[tuhpc002@cypress1 ~]$

You can see that the two are nearly identical. The only difference being that our .bash_profile modifies the PATH environment variable.

There is much confusion over the role that each of these files play. When you login to a system, the .bash_profile file is executed UPON LOGIN. The .bashrc file modifies the default environment when connected without login, for example when executing commands remotely using SSH. Note that changes made in the .bashrc file will be reflected in the .bash_profile file but not vise versa.

SSH config (Linux and Mac)

It can be cumbersome to type your username and host address every time you login to a machine. The .ssh/config file offers the ability to create aliases to speed up this process. Add the following to your .ssh/config file to create a shortcut

  Host cypress1
  Hostname cypress1.tulane.edu
  User <your user ID on cypress>

Now from your local machine you can simply type ssh cypres1 and the username and hostname will be set automatically.

Next Section

Modules