One walk, every acquisition
An intermediate representation, like a compiler's
A compiler never runs your C directly, and it never re-parses it for every optimisation. It lowers the source into an intermediate representation — LLVM IR — one well-specified middle form that every language front-end targets and every hardware back-end consumes. The front-ends and back-ends never have to know about each other; they only have to agree on the IR.
The replay pack (.rpk) is exactly that, for Monte-Carlo diffusion simulation.
| A compiler | Substrate Commons |
|---|---|
| source — C, Rust, Swift … | a realistic substrate — a .ply mesh |
| a front-end lowers it to LLVM IR | a generator lowers the walk to .rpk |
| back-ends consume the IR to emit machine code | acquisitions consume .rpk to emit a signal |
| add a language or a CPU — the IR is unchanged | add a mesher or a sequence — .rpk is unchanged |
Because everyone agrees on the middle form, a pack from dmipy-sim and a pack from MC-DC are the same kind of object — poolable, comparable, and readable by tools that have no idea how the walk was produced.
Not a trajectory dump
Here is what makes it a real IR and not just a recording. The .rpk does not store walker positions
that you replay step by step. The walk is lowered into a compact basis — a compressed representation
of how each walker's phase and field evolve along its path. To get a signal you contract against that
basis directly; the original trajectories are never reconstructed.
And within the pack's declared envelope, that contraction is cheaper than working from the raw walk. That's the whole trick — the IR is not a smaller copy of the simulation, it is a form that is faster to produce a signal from than the simulation ever was. It is the reason a solved substrate evaluates at analytical-model speed while being a real mesh underneath.
In one line
.rpk is a lossless intermediate representation of the Monte-Carlo process — a compressed,
signal-ready basis that every simulator lowers into and every acquisition reads out of, faster than the
raw walk it came from.
The narrow waist
An IR earns its keep by being a narrow waist: many producers above, many consumers below, one stable contract in the middle (the shape you see in the diagram on the home page). Add a new mesher or a new simulator on top, or a new sequence underneath, and nothing else has to change. The capability tiers say what a given pack's waist can carry; the replay invariant says why one walk can serve them all.