Install Packages to Anaconda Python

For this example, we assume you have loaded the "anaconda3/2019.03", but you can use another version if you want.

module load anaconda3/2019.03

'conda' is the command to add packages to Anaconda python distribution. However, since you don't have permission in /share/apps/anaconda, it won't work. There is a workaround. Here, for example, we want to add 'vtk' python wrapper library. You try:

conda install vtk

But you will see an error message with an instruction for the workaround. Following the instruction, you have to clone the python environment. By default, your conda environments are located in your home directory. But, we recommend you to use your group project directory. Setting the CONDA_ENVS_PATH environment variable will tell Anaconda where to place and look for your conda environments. For example:

export CONDA_ENVS_PATH=/lustre/project/mygroup/myuser/conda-envs

will put your conda environments at the folder /lustre/project/mygroup/myuser/conda-envs (you may have to create this folder if it does not exist).

By default, the package cache directory is created in your home directory, which will take a large space as you use 'conda'. To change the package cache directory, edit ~/.condarc file and add

pkgs_dirs:
    - /path/to/package_cache_directory

(If no ~/.condarc exists, try 'conda config'.)

Then, you can create a new personal conda environment:

conda create -n my_root python

'my_root' is the name of the environment that you decide.

Also, in general in conda environments, python should not be using PYTHONPATH. (For more information, see ​here.)

In particular, if you encounter errors during python module import, then you should unset PYTHONPATH prior to activating your conda environment.

Hence,

unset PYTHONPATH
source activate my_root

(When you get an error "CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'", see ​here)

Finally, you can install the package,

conda install vtk

In case you encounter "Error: 'conda' can only be installed into the root environment", try

conda remove conda-build
conda remove conda-env
conda install vtk

Next time, to use Anaconda python with your environment,

module load anaconda3/2019.03
export CONDA_ENVS_PATH=/lustre/project/mygroup/myuser/conda-envs
unset PYTHONPATH
source activate my_root

You have to state the above commands in your script when you run python on batch jobs.