Skip to main content

singularityce

Container technology is a virtualization technology centered on application software. It packages software and all its dependencies into a container image, which can be directly copied to different Linux hosts for execution. Container technology effectively resolves issues related to dependency installation, software environment isolation, and software environment portability.

Singularity is a container technology developed by Lawrence Berkeley National Laboratory specifically for high-performance computing scenarios. Singularity is fully based on portability for virtualization, is more lightweight, and deploys faster. Singularity is currently widely used in various high-performance computing centers.

Singularity container technology is functionally similar to Docker, but differs slightly in usage. Docker users can easily get started. Due to inherent and unfixable flaws in Docker regarding security, permissions, high-speed network support, and MPI parallel support, Docker is unsuitable for use in supercomputing. To adapt to the unique platform environment of supercomputing, container software specifically developed for supercomputing environments has emerged. Among the container software currently suitable for supercomputing, Singularity has the best compatibility, the most complete support for supercomputing features, and the best performance.

1. Script Template​

 #!/bin/bash
 #SBATCH --nodes=1                   # Number of nodes
 #SBATCH --ntasks-per-node=56        # Number of cores per node
 #SBATCH --ntasks=56                 # Total number of cores
 #SBATCH --partition=g1_share        # Queue partition; must specify the correct partition
 #SBATCH --job-name=singularity             # Job name
 #SBATCH --output=sin.%j.out        # Standard output log (%j parameter value is jobId)
 #SBATCH --error=sin.%j.err         # Error log output (%j parameter value is jobId)

 ##############################################
 #          Software Envrironment             #
 ##############################################
 unset I_MPI_PMI_LIBRARY                        # Disable default MPI library, use Intel's built-in
 export I_MPI_JOB_RESPECT_PROCESS_PLACEMENT=0   # Parameter modification required for Intel multi-node jobs
 module load singularity/3.10.0        # Load intel environment

 ##############################################
 #               Run job                      #
 ##############################################
 singularity exec --bind /home/cloudam/singularity/lammps:/home/cloudam/singularity/lammps  lammps.sif mpirun  lmp_mpi -in M-1.in
 ###--bind parameter same as -B maps directory host_directory:container_directory

2. Local Installation of Singularity​

# Install dependencies
yum install -y gcc libuuid-devel squashfs-tools openssl-devel make

# Install go
export VERSION=1.17.2 OS=linux ARCH=amd64   
wget https://dl.google.com/go/go$VERSION.$OS-$ARCH.tar.gz 
tar -C /usr/local -xzvf go$VERSION.$OS-$ARCH.tar.gz
rm -f go$VERSION.$OS-$ARCH.tar.gz 
echo 'export PATH=/usr/local/go/bin:$PATH' >> /etc/profile
source /etc/profile

# Install singularity
export VERSION=3.9.2
wget https://github.com/sylabs/singularity/releases/download/v${VERSION}/singularity-ce-${VERSION}.tar.gz 
tar -xzf singularity-ce-${VERSION}.tar.gz
cd singularity-ce-${VERSION}
./mconfig --prefix=/opt/singularity/${VERSION}
cd builddir/
make && make install
echo "export PATH=/opt/singularity/${VERSION}/bin:\$PATH" >> /etc/profile

3. Building Containers​

Building from an Existing Image​

singularity build --sandbox ansys2022R1 docker://centos:7.6.1810
 #build build command; --sandbox writable container image, without this suffix defaults to creating a read-only sif image; molspin container name 
#docker:// ubuntu:18.04  

##Can search for corresponding version images in docker image repository   https://hub.docker.com/search?image_filter=official&q=

Definition File -.def​

#Definition container build file
Bootstrap: docker #Select the proxy for building the image docker library shub
From: tensorflow/tensorflow:latest-gpu #Select the image to build
Stage: build

%files #Copy files from host to inside the container
    /var/py/train.py /opt/train.py
%environment #Set environment variables
    export LISTEN_PORT=12345
    export LC_ALL=C
%runscript #Execute when container runs
    echo "Container was created $NOW"
    echo "Arguments received: $*"
    exec echo "$@"
%labels #Set labels on the image
    Author jason.zhang@xtaotech.com
    Version v0.0.1

%help #Set help documentation
    This is a demo container used to illustrate a def file that uses all
    supported sections.

Build build

singularity build jason-tf.sif jason_tf.def

4. Common Singularity Commands​

  1. Run singularity
singularity shell molspin.sif #Interactive execution
singularity exec molspin.sif which mpirun #Background execution of application commands in singularity environment

Other Commands

singularity inspect molspin.sif #View labels
singularity run-help molspin.sif #View help documentation

5. Setting Up the Software Environment​

singularity shell -w molspin #Enter the image named molspin, -w writable mode
singularity shell molspin.sif # Enter the read-only mode image

6. Using Singularity in the Cluster​

# Create a sandbox;
# Here, the created sandbox is named ubuntu20_lammps, using the existing container ubuntu:20.04 from Docker Hub as the base image.
singularity build --sandbox ./ubuntu20_lammps docker://ubuntu:20.04

# Enter the created sandbox and make modifications;
# The -w flag indicates writable mode. Upon entry, Singularity automatically mounts the HOME directory; if entering as root, the /root directory is mounted.
singularity shell -w ./ubuntu20_lammps

# Installing the parallel version of LAMMPS on Ubuntu requires installing necessary dependency packages.
apt update && apt upgrade -y
apt install openmpi-bin openmpi-doc libopenmpi-dev -y
apt domain name

# Install fftw
wget http://www.fftw.org/fftw-3.3.9.tar.gz
tar zxvf fftw-3.3.9.tar.gz
cd fftw-3.3.9
./configure --prefix=/opt/software/fftw_3.3.9 --enable-shared --enable-static --enable-fma
make -j && make install

# Set temporary fftw environment variables
export PATH=/opt/software/fftw_3.3.9/bin:$PATH
export LD_LIBRARY_PATH=/opt/software/fftw_3.3.9/lib:$LD_LIBRARY_PATH

# Install lammps
wget https://lammps.sandia.gov/tars/lammps-10Feb21.tar.gz
tar zxvf lammps-10Feb21.tar.gz
cd lammps-10Feb21
cd src
vim MAKE/OPTIONS/Makefile.g++_openmpi # Modify the following lines
  FFT_INC = -DFFT_FFTW -I/opt/software/fftw_3.3.9/include
  FFT_PATH = -L/opt/software/fftw_3.3.9/lib
  FFT_LIB = -lfftw3

make yes-std
make no-lib
make -j g++_openmpi

mkdir /opt/software/lammps
cp ./lmp_g++_openmpi /opt/software/lammps/

# Set temporary lammps environment variables
export PATH=/opt/software/lammps:$PATH

# Verify (inside container)
cd /root
cp /opt/lammps-10Feb21/bench/in.lj .
mpirun --allow-run-as-root -np 2 --mca btl ^openib lmp_g++_openmpi -in in.lj


# Exit the container and set permanent environment variables (on the host)
vim ./ubuntu20_lammps/environment
# Add the following two lines
export PATH=/opt/software/fftw_3.3.9/bin:/opt/software/lammps:$PATH
export LD_LIBRARY_PATH=/opt/software/fftw_3.3.9/lib:$LD_LIBRARY_PATH

# Verify (on the host)
singularity exec ./ubuntu20_lammps mpirun --allow-run-as-root -np 2 --mca btl ^openib lmp_g++_openmpi -in in.lj

# Package the modified sandbox into sif format;
# Delete unnecessary installation packages, such as fftw-3.3.9.tar.gz and lammps-10Feb21.tar.gz
# Use the previously created sandbox directory to generate a Singularity image file.
singularity build ubuntu20_lammps.sif ./ubuntu20_lammps

Upload ubuntu20_lammps.sif to the cluster environment.

Use the submission script to submit the job.

7. Deleting Built Containers

# Assume the sandbox image to be deleted is named molspin

# First, enter the image to be deleted in read-only mode
singularity shell --fakeroot -w molspin

# Delete all files created based on fakeroot within the container
rm -rf /* 1>/dev/null 2>&1

# Exit the image
exit

# Upload the created software image to the high-performance computing cluster and load the Singularity software environment
# Delete the remaining files
rm -rf molspin