Install Packages to Anaconda Python
For this example, we assume you have loaded the "anaconda3/2019.03", but you can use another version if you want.
module load anaconda3/2019.03
'conda' is the command to add packages to Anaconda python distribution. However, since you don't have permission in /share/apps/anaconda, it won't work. There is a workaround. Here, for example, we want to add 'vtk' python wrapper library. You try:
conda install vtk
But you will see an error message with an instruction for the workaround. Following the instruction, you have to clone the python environment. By default, your conda environments are located in your home directory. But, we recommend you to use your group project directory. Setting the CONDA_ENVS_PATH environment variable will tell Anaconda where to place and look for your conda environments. For example:
will put your conda environments at the folder /lustre/project/mygroup/myuser/conda-envs (you may have to create this folder if it does not exist).
By default, the package cache directory is created in your home directory, which will take a large space as you use 'conda'. To change the package cache directory, edit ~/.condarc file and add
pkgs_dirs: - /path/to/package_cache_directory
(If no ~/.condarc exists, try 'conda config'.)
Then, you can create a new personal conda environment:
conda create -n my_root python
'my_root' is the name of the environment that you decide.
Also, in general in conda environments, python should not be using PYTHONPATH. (For more information, see https://conda.io/projects/conda/en/latest/user-guide/troubleshooting.html#conda-reports-that-a-package-is-installed-but-it-appears-not-to-be.)
In particular, if you encounter errors during python module import, then you should unset PYTHONPATH prior to activating your conda environment.
unset PYTHONPATH source activate my_root
Finally, you can install the package,
conda install vtk
In case you encounter "Error: 'conda' can only be installed into the root environment", try
conda remove conda-build conda remove conda-env conda install vtk
Next time, to use Anaconda python with your environment,
module load anaconda3/2019.03 export CONDA_ENVS_PATH=/lustre/project/mygroup/myuser/conda-envs unset PYTHONPATH source activate my_root
You have to state the above commands in your script when you run python on batch jobs.