Skip to main content

Job Submission and Execution

Job Submission and Execution (Recommended for background job execution) sbatch writes the entire computation process into a script, which is then submitted to compute nodes for execution via the sbatch command.

Note:​

  • Do not execute computational programs directly on the login node after logging into the cluster.

  • You must submit or run jobs interactively using a job submission script.

Demo Video URL https://www.bilibili.com/video/BV1114y1o7zB​

1. Preparation​

Preparation: You need to upload your example files to your cluster account. Assume you have uploaded a binary program to the 'test' directory in your home directory.​

Refer to the 'Uploading Files' section for upload instructions.

2. Script Template​

First, prepare a script template. The following is a reference example that executes a binary program using 56 cores on a single node.​

Refer to the 'Software Usage' section for software templates.

If you need to specify nodes or core counts, modify the corresponding node count, node cores (max 56), and total core count in the example below.

submit.sh
#!/bin/bash
#SBATCH --nodes=1                   # Number of machine nodes
#SBATCH --ntasks-per-node=56   	 # Number of cores per machine
#SBATCH --ntasks=56                 # Total number of cores (total running cores)
### #SBATCH --partition=g3_share   	   # Queue partition; must specify the correct partition (will be informed)
#SBATCH --job-name=hello    	    # Job name (recommended to modify for distinction)
#SBATCH --output=hello.%j.out       # Normal log output (%j parameter value is jobId)
#SBATCH --error=hello.%j.err        # Error log output (%j parameter value is jobId)

##############################################
#          Software Environment              #
##############################################
unset I_MPI_PMI_LIBRARY                        # Unset default MPI library, use Intel's built-in
export I_MPI_JOB_RESPECT_PROCESS_PLACEMENT=0   # Parameter modification required for Intel multi-node jobs
module load intel/2022 intelmpi/2022           # Load Intel environment

##############################################
#               Run job                      #
##############################################
mpirun -np $SLURM_NPROCS ./binary              # The -np option specifies $SLURM_NPROCS as the total number of cores, i.e., nodes*tasks

3. Modify the Submission Script​

Upload the template to the cluster via upload or online editing.​

Assume the edited script file is named 'run.slurm'. Place the example files in the example directory, using the 'test' directory in the home directory as an example (the directory containing the binary file).

4. Submit the Job​

Submit the job for background execution using the following command:​
sbatch run.slurm

Appendix: Script Template for Monitoring Job Memory and CPU Usage​

submit.sh
#!/bin/bash
#SBATCH --nodes=1                   # Number of nodes
#SBATCH --ntasks-per-node=56   	 # Number of cores per node
#SBATCH --ntasks=56                 # Total number of cores
#SBATCH --partition=g3_share   	   # Queue partition; must specify the correct partition
#SBATCH --job-name=hello    	    # Job name
#SBATCH --output=hello.%j.out       # Normal log output (%j parameter value is jobId)
#SBATCH --error=hello.%j.err        # Error log output (%j parameter value is jobId)

##############################################
# memeory total calculate #
##############################################
mkdir ./memory_total                # ben
nameflag=`scontrol show hostname $SLURM_JOB_NODELIST`
jobflag=`squeue | grep $SLURM_JOB_ID | wc -l`

function gather_job(){
while [[ "$jobflag" -eq 1 ]]
do
  for line in $nameflag
  do
    meminfo=`scontrol show node $line | grep Real | sed 's/.\{3\}//' | sed 's/.\{17\}.$//'`
    hostinfo=`scontrol show node $line | grep NodeName | awk '{print $1}'`
    cpuinfo=`scontrol show node $line | grep CPULoad | awk '{print $1}'`
    echo -e "$hostinfo\n$meminfo\n$cpuinfo\n" >> ./memory_total/`date '+%Y%m%d-%T' | sed 's/.\{2\}.$//'`
  done
  sleep 1m
done
}
gather_job &

##############################################
#          Software Envrironment             #
##############################################
unset I_MPI_PMI_LIBRARY                        # Unset default MPI library, use Intel's built-in
export I_MPI_JOB_RESPECT_PROCESS_PLACEMENT=0   # Parameter modification required for Intel multi-node jobs
module load intel/2022 intelmpi/2022           # Load Intel environment

##############################################
#               Run job                      #
##############################################
mpirun -np $SLURM_NPROCS ./binary              # The -np option specifies $SLURM_NPROCS as the total number of cores, i.e., nodes*tasks