HPC Workshop Spring 2026

Module 7 of 8 - Job Parallelism (Under construction)

(Content subject to change prior to the workshop)

What is Job Parallelism?

Job parallelism refers to jobs for which 2 or more processing units (or processors or cores) execute the given job's code instructions simultaneously - rather than sequentially.

Your job may need more than one processor to finish in a timely manner - or possibly to finish at all.

Why use a HPC Cluster?

• tasks take too long
  • When the task to solve becomes heavy on computations, the operations are typically outsourced from the local laptop or desktop to elsewhere. 
  • Your computation may execute more efficiently if the code supports multithreading or multiprocessing.

• one server is not enough
  • When a single computer can’t handle the required computation or analysis, the work is carried out on larger groups of servers.

Tools for implementing various levels of job parallelism on Cypress

See Module 2 of 8 - Introduction to Multi-processing for more information on tools available on Cypress for creating and preparing your jobs at the various levels of parallelism - programming, single job, and multi-job.

Before running your job

Before you run your job you should consider the following in order for your job to run most efficiently on Cypress.

Review your software provider's information

See Codes for Multi-Cores, Multi-Nodes. Offloading.

Choices for application programming tools

Refer to the following table to determine what programming model to use based on the type of algorithm your job requires.

Algorithm Type	Programming Model	Hardware Used	Examples
Single Instruction Multiple data (SIMD)	Compiler vectorization	Intel Advanced Vector Extensions (AVX), 256-bit vector processor	See ​Math Libraries
Multithreaded (shared memory)	OpenMP	1 Node, ≥2 cores	See OpenMP
Problem domain decomposition	MPI	≥2 Nodes	See MPI
Massively Parallel, Single Instruction Multiple Threads (SIMT)	#pragma offload (GPU kernels not available on Cypress)	Coprocessors - XeonPhi (GPUs not available on Cypress)	See XeonPhi, Offloading to Accelerator
Hybrid Parallel	MPI + OpenMP	≥2 Nodes	See Hybrid Jobs job script

Choices for Job scripting

• Many independent tasks

See Running Many Serial/Parallel Jobs if your computational workload can be split easily - or perhaps with some minimal or one-time effort - into many independent tasks, requiring minimal communication. For more information.

• Many dependent tasks

Otherwise, see MPI if your computational workload includes too many tasks to run on a single node and the tasks require a significant level of inter - communication during the computation.