Lesson 2: Loosely vs Tightly Coupled Workloads

One question decides the network

In Level 1 we chose to carry one benchmark from your laptop to a written decision. Before you compare a single VM type, ask one question about that benchmark: how often do its parallel parts need to talk to each other while they run?

Your answer tells you whether you need a low-latency fabric such as InfiniBand or Cloud RDMA, whether the physical placement of nodes matters, and whether cheap, scattered capacity like Spot VMs across a region is good enough. Classify the workload wrongly and you can do everything else right and still choose the wrong platform.

A useful comparison is office work:

• Embarrassingly parallel is like 1,000 people each marking their own exam paper. Nobody talks to anyone. You can seat them in different buildings.
• Loosely coupled is like those markers handing their totals to a coordinator at the end of each hour. There is some talking, but it is rare and not urgent.
• Tightly coupled is like an orchestra. Each player has to hear the others every bar, and one late player holds up everyone.

The three classes, with real examples

Google Cloud's own CAE guidance makes the same split: jobs submitted many times with different input parameters are loosely coupled and do not need a low-latency interconnect, while jobs where many systems work together need one.

Walkthrough: read the code for evidence

You don't need to run anything to make a first classification. Look for communication calls in the code. Here is a quick scan of a source tree:

# Count MPI communication calls by type in a C/C++/Fortran code base
# -r recursive, -o print only the match, -h hide file names, -i ignore case (Fortran)
grep -rohiE 'MPI_(Allreduce|Alltoall|Bcast|Sendrecv|Isend|Irecv|Send|Recv|Barrier)' src/ \
  | tr '[:lower:]' '[:upper:]' | sort | uniq -c | sort -rn

This is realistic output for a CFD-style solver:

     42 MPI_ISEND        # non-blocking sends: halo exchange with neighbours
     42 MPI_IRECV
     18 MPI_ALLREDUCE    # global sums: every rank waits for every other rank
      3 MPI_BARRIER

If the scan finds no MPI and the job is launched as N separate copies with different input files, it is almost certainly embarrassingly parallel. A collective like MPI_Allreduce inside the timestep loop is the strongest sign of tight coupling, because every process stops until the slowest one arrives.

Gotcha: finding MPI in the code doesn't settle it. Some codes use MPI only to hand out independent work and gather results once at the end, which is loosely coupled. Check where the calls sit: inside the hot loop, or only at the start and finish?

Why latency hurts tightly coupled jobs and not sweeps

This toy model uses made-up but plausible numbers. Suppose each timestep does 2 ms of computing per process and then 30 small message exchanges.

# toy_step_time.py: how per-message latency inflates each timestep
compute_ms = 2.0          # useful work per step, per process
messages_per_step = 30    # halo exchanges plus allreduces

# Illustrative per-message latencies, not vendor figures
for name, latency_us in [("low-latency fabric", 3), ("ordinary network", 30)]:
    comm_ms = messages_per_step * latency_us / 1000
    step_ms = compute_ms + comm_ms
    print(f"{name:20s} step={step_ms:.2f} ms  comm share={comm_ms/step_ms:.0%}")

low-latency fabric   step=2.09 ms  comm share=4%
ordinary network     step=2.90 ms  comm share=31%

Over a million timesteps, that 31% is paid on every step. It also grows as you add nodes: each process gets less work, so the 2 ms shrinks, while the number of messages stays the same or rises. Now apply the same model to a parameter sweep. There is one message per task, at the end, so latency rounds to zero. A sweep cares about cost per task and how fast capacity arrives, not about microseconds.

Placement matters for the same reason. Every extra network hop between racks adds latency to each of those small messages. Cloud providers let you ask for nodes to be placed close together:

• GCP: compact placement policies place VMs physically close within a zone, and Google recommends them for tightly coupled jobs such as MPI.
• Azure: RDMA-capable HPC VMs must be in the same VM scale set or availability set to talk over InfiniBand. Scale sets also need singlePlacementGroup set to true.

You'll configure these in later levels. For now, just note that placement is only a decision for tightly coupled jobs.

Trade-off: placing nodes close together makes them harder to get. Asking for 64 nodes in one rack is a stricter request than asking for 64 nodes anywhere in a region, so start-up can be slower or fail. A sweep should never pay that price.

Product context (as of October 2026)

The fabrics you'll test later exist for tightly coupled work. GCP's H4D machine series supports Cloud RDMA at up to 200 Gbps between nodes. Azure HBv4 and HX VMs carry 400 Gb/s NDR InfiniBand. If your workload is embarrassingly parallel, you may not need either one, and that can remove whole options from your shortlist.

Write your coupling statement

Before going further, write three or four sentences into the project's decision notes:

COUPLING STATEMENT: OSU-style halo solver (draft, 2026-10-31)
Class:     Tightly coupled.
Evidence:  MPI_Isend/Irecv and MPI_Allreduce inside the timestep loop
           (grep of src/solver/, 18 allreduce call sites).
Implies:   Needs low-latency fabric and compact placement; candidate
           for H4D + Cloud RDMA and HB/HX + InfiniBand.
Open:      Confirm with a multi-node scaling run (later level).

Each line has a job: the class, the evidence behind it, what it means for platforms, and what is still unproven. A statement with no evidence is a guess, and reviewers will treat it as one.

Key takeaways

• Classify by communication pattern: none means embarrassingly parallel, occasional means loosely coupled, every step means tightly coupled.
• Collectives like MPI_Allreduce inside the main loop are the strongest sign of tight coupling, because everyone waits for the slowest process.
• Latency is paid on every message in every step, so it dominates tightly coupled jobs and barely affects sweeps.
• Placing nodes close together (GCP compact placement, or one scale set or availability set on Azure) only matters for tightly coupled jobs, and it can make capacity harder to get.
• Write a coupling statement that gives the class, the evidence, what it implies and what is still open.