[[PageOutline]] = Singularity = '''The current installation of Singularity on Cypress works on CentOS 7 nodes only.''' == About Singularity == [https://sylabs.io/ Singularity] allows you to create and run containers that package up pieces of software in a way that is portable and reproducible. === Advantages For Using Containers * '''Platform Mobility''' - You can build a container using Singularity - or [https://apptainer.org/docs/user/1.1/index.html Apptainer] - on your laptop, and then run it on many of the largest HPC clusters like Cypress. * '''Convenient Usage''' - Your container is a single file - normally with file extension '''.sif''', and you don’t have to worry about how to install all the software you need on each different operating system and system. * '''Bypass Host OS Dependencies''' - See [https://wiki.hpc.tulane.edu/trac/wiki/cypress/glibc-version-problems glibc version problems]. === Caveats For Using Containers === * '''SIF Compatibility''' - The target machine must be running a linux operating system (OS) and have Singularity or Apptainer installed - using the same version of the '''.sif''' image file format. (See [https://apptainer.org/docs/user/main/singularity_compatibility.html#sif-image-compatibility| SIF Image Compatibility] in Apptainer documentation.) * '''Machine Architecture''' - Your image file may not run on the target machine's architecture, x86/amd64 (for Intel/AMD) or arm64 (for Apple), if it is not the same as that of the machine on which the image file was constructed. * For linux platform architecture, use the command '''uname -m'''. * For linux kernel version use, use the command '''uname -r'''. * '''Container Security''' - Because Singularity runs entirely inside your Linux user space (see [https://www.linfo.org/user_space.html| User Space Definition]), the container instantly inherits your identity, your permissions, and your network access. This includes your Cypress account's access to files and internet. * '''Avoid using the :latest for production or research''' - The images whose name ends with the tag ''':latest''' can change over time, reducing reproducibility and making it difficult to know exactly what software was reviewed, tested, or approved. Whenever possible, use an image tagged with a specific version, such as '''ubuntu:14.04''' rather than '''ubuntu:latest'''. See [SingularityImageInspection Inspecting a Singularity .sif file]. === More about containers === * SingularityCE Tutorial https://sylabs.github.io/singularity101/intro.html * Introduction to SingularityCE https://sylabs.github.io/singularity101/introduction.html == Possible uses for Singularity on Cypress == * Run an application built for a different distribution or '''flavor''' (for example [SingularityDockerhub#Example1:BuildingacontainerforUbuntu Ubuntu]) of Linux than Cypress OS (Centos). * Reproduce an environment to run a workflow created by someone else. * Run a series of applications (a 'pipeline') that includes applications built on different platforms. * Run an application from [https://singularityhub.github.io/singularityhub-docs/ Singularity Hub] or [https://hub.docker.com/ Docker Hub] or [https://biocontainers.pro/ BioContainers] without actually installing anything. == Build Singularity Container on Cypress == See [SingularityDockerhub Build Singularity Containers from Dockerhub]. == Build Singularity Container elsewhere to run on Cypress == See [SingularityRecipes Build Singularity Containers from Recipes] If you need a singularity image with specific software installed, please request a [https://tulane.service-now.com/tulaneit?id=sc_cat_item&sys_id=dc086d4ddb004c10e8ce5bd2ca9619d1&sysparm_category=8d648d93dbc44c10f460562bdc9619ad&catalog_id=-1 Cypress HPC Software Installation] or [https://tulane.service-now.com/tulaneit?id=sc_cat_item&sys_id=277b5905db004c10e8ce5bd2ca961945&sysparm_category=8d648d93dbc44c10f460562bdc9619ad&catalog_id=-1 Cypress HPC Consultation]. == Run Singularity Container on Cypress == For example, the container used in the following was built using [SingularityDockerhub this page] in an interactive session using the '''idev''' command. {{{ [fuji@cypress01-013 SingularityTest]$ singularity exec ubuntu_14.04.sif cat /etc/os-release NAME="Ubuntu" VERSION="14.04.6 LTS, Trusty Tahr" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 14.04.6 LTS" VERSION_ID="14.04" HOME_URL="http://www.ubuntu.com/" SUPPORT_URL="http://help.ubuntu.com/" BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" }}} The user in the container is the same as in the host machine. {{{ [fuji@cypress01-013 SingularityTest]$ singularity exec ubuntu_14.04.sif whoami fuji }}} Singularity has a dedicated syntax to open an interactive shell prompt in a container: {{{ [fuji@cypress01-013 SingularityTest]$ singularity shell ubuntu_14.04.sif Singularity> pwd /home/fuji Singularity> whoami fuji Singularity> cat /etc/os-release NAME="Ubuntu" VERSION="14.04.6 LTS, Trusty Tahr" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 14.04.6 LTS" VERSION_ID="14.04" HOME_URL="http://www.ubuntu.com/" SUPPORT_URL="http://help.ubuntu.com/" BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" Singularity> exit exit }}} === Use host directories === The container filesystem is read-only, so if you want to write output files you must do it in a bind-mounted host directory. '/home/userid' is mounted by default. Following environmental variable sets '/lustre/project/hpcstaff/fuji' as '/home/fuji' in the container. {{{ export SINGULARITY_BINDPATH=/lustre/project/hpcstaff/fuji:/home/fuji }}} === Run containers in a batch job === For example, {{{ #!/bin/bash #SBATCH --job-name=Singularity # Job Name #SBATCH --partition=centos7 # Partition #SBATCH --qos=normal # Quality of Service #SBATCH --time=0-00:10:00 # Wall clock time limit in Days-HH:MM:SS #SBATCH --nodes=1 # Node count required for the job #SBATCH --ntasks-per-node=1 # Number of tasks to be launched per Node #SBATCH --cpus-per-task=1 # Number of threads per task (OMP threads) # Load Singularity module module load singularity/3.9.0 # Set $TMPDIR in containar to /tmp, keeping $TMPDIR in host (/local/tmp/...) export SINGULARITYENV_TMPDIR=/tmp # Mount the lustre directory to home, $TMPDIR to /tmp export SINGULARITY_BINDPATH=/lustre/project/hpcstaff/fuji:/home/fuji,$TMPDIR:/tmp # Run container singularity exec ubuntu_14.04.sif ls }}} By default, the cache is stored in ~/.singularity; this location can be customized using the environmental variable '''SINGULARITY_CACHEDIR'''. A subcommand, ''singularity cache'', can be used to manage the cache. == Running RStudio using Singularity on Cypress == See [wiki:RunningRStudioWithSingularity here].