Build Singularity Containers from Dockerhub

​Docker is a container virtualization environment that can establish development or runtime environments without modifying the environment of the base operating system. It runs on Mac and Windows.

Use CentOS 7

To use Singularity on Cypress, you have to use one of the computing nodes running version CentOS 7 of the operating system.

For Workshop

idev --partition=workshop7

Non Workshop

idev --partition=centos7

Load the module,

module load singularity/3.9.0

Build from Docker Hub, for example ubuntu 14.04

singularity pull docker://ubuntu:14.04

Singularity uses /tmp as a default temporary directory for building images. /tmp is just 2Gbyte so you might encounter errors because of no space on disk. In such case, you should set SINGULARITY_TMPDIR as, for example,

export SINGULARITY_TMPDIR=$TMPDIR

By default, the cache is stored in ~/.singularity; this location can be customized using the environmental variable SINGULARITY_CACHEDIR. A subcommand, singularity cache, can be used to manage the cache.

Example : ​FastDTLmapper

This tool depends on many python packages and requires both python2 and python3. Fortunately, the developer provides Docker Images so we can use it to generate a Singularity image.

On Centos7 computing node, (need 'idev'),

module load singularity/3.9.0
singularity pull docker://ghcr.io/moshi4/fastdtlmapper:latest

This command creates ‘fastdtlmapper_latest.sif’, which is the Singularity Image. You can rename/transfer it as you like.

To run it to show ‘help’,

singularity exec fastdtlmapper_latest.sif FastDTLmapper -h

To run it with the minimum test dataset, Assuming the example dataset is extracted under /lustre/project/group/user/fastdtlmapper, where 'group' is the group name and 'user' is the user name,

export SINGULARITYENV_TMPDIR=/tmp
export SINGULARITY_BINDPATH=/lustre/project/group/user:/home/user,$TMPDIR:/tmp
singularity exec fastdtlmapper_latest.sif FastDTLmapper -i fastdtlmapper/example/minimum_dataset/fasta/ -t fastdtlmapper/example/minimum_dataset/species_tree.nwk -o fastdtlmapper/output_minimum

For Slurm batch jobs,

#!/bin/bash
#SBATCH --job-name=fastDLmapper # Job Name
#SBATCH --partition=centos7    # Partition
#SBATCH --qos=normal           # Quality of Service
#SBATCH --time=0-24:00:00      # Wall clock time limit in Days-HH:MM:SS
#SBATCH --nodes=1              # Node count required for the job
#SBATCH --ntasks-per-node=1    # Number of tasks to be launched per Node
#SBATCH --cpus-per-task=20     # Number of threads per task (OMP threads)
#SBATCH --output=try_fastDLmapper.out       ### File in which to store job output
#SBATCH --error=try_fastDLmapper.err        ### File in which to store job error messages
 
# Load Singularity module
module load singularity/3.9.0
 
# Set $TMPDIR in containar to /tmp, keeping $TMPDIR in host (/local/tmp/...)
export SINGULARITYENV_TMPDIR=/tmp
 
# Mount the lustre directory to home, $TMPDIR to /tmp
export SINGULARITY_BINDPATH=/lustre/project/group/user:/home/user,$TMPDIR:/tmp
 
# Run container
singularity exec fastdtlmapper_latest.sif FastDTLmapper -i fastdtlmapper/example/minimum_dataset/fasta/ -t fastdtlmapper/example/minimum_dataset/species_tree.nwk -o fastdtlmapper/output_minimum