If you have ever run a job on an HPC cluster, chances are you have used MPI without fully knowing what’s happening behind the scenes. And that’s completely normal. MPI often feels like a black box that just “makes parallel jobs work.”

Let’s open that box a bit, without diving into heavy theory or academic jargon.

⸻

The Basic Idea

MPI (Message Passing Interface) is simply a way for multiple processes to talk to each other while running a program.

Think of it like this:

Instead of one program doing all the work, MPI lets you run many copies of the same program. Each copy handles a portion of the task and communicates with others when needed.

⸻

What Actually Happens When You Run an MPI Job?

When you launch an MPI job using something like:

mpirun -np 4 ./my_app

Enter fullscreen mode

Exit fullscreen mode

Here’s what’s going on under the hood:

1. Multiple Processes Are Started

MPI doesn’t create threads.…