Message Passing Interface; MPI

• A standardized and portable message-passing system
• Not a new language —— Library of functions
• Scales to over 100k processors
• Runs on distributed memory cluster systems.
• Portable
• Explicit parallel programming (unlike OpenMP)
• No shared variables (unlike OpenMP)

MPI’s Execution model

• All processors execute same program, but with difference data. (no fork & join)
• Variable on each are all private.
• Data that must be shared is explicitly send between processors.

Distributed Memory Cluster

Data sent over Network:

• Ethernet
  • 10M~1GBit/Sec
• InfiniBand
  • 100GBit~
• MPI on SMP machine
  • You can run MPI program on SMP (Symmetric Multiprocessing) machine.
  • Variables are not shared (all private).

Software Distribution

• ​OpenMPI
• ​Intel MPI Library
• ​LAM/MPI
• ​MPICH
• ​MVAPICH
• ​MS MPI

Brief Tutorials for Programming Languages

• C
• C++
• Fortran