Robustness testing works best before it matters, and stops working almost as well the moment it doesn't. Once real capital is already committed to a strategy, checking whether it should have been is a very different exercise than checking beforehand — not because the math changes, but because the person doing the checking has changed.
Why the order matters
Before capital is at risk, a weak Monte Carlo result is just information — mildly disappointing, easy to act on, no different in kind from any other test that came back unfavorable. After capital is at risk, the same result is now a reason to reconsider a decision that's already been made, with money already moving, and a natural, human resistance to concluding that the decision was premature. The test itself doesn't get any less accurate after the fact. The willingness to accept an unfavorable answer does.
What "before" actually requires
This means the robustness check has to be a gate, not a formality performed after the fact to justify a decision already made informally. A concrete bar decided in advance — a risk-of-ruin threshold below 5%, a probability-of-loss below 20%, whatever the specific numbers are for the situation — needs to be checked and cleared before capital moves, not glanced at afterward as reassurance. If the bar is only ever consulted after the decision, it isn't really a bar. It's a formality that was never going to change the outcome.
The cost of getting the order backwards
A strategy that fails robustness after capital is already committed doesn't just cost the money eventually lost to a bad idea — it costs the clear-headedness that made the test meaningful in the first place. Deciding the threshold, and actually checking against it, before any capital is at stake is what keeps the test doing its real job: telling you something true, rather than something you're already inclined to accept.
Full reference: docs · The workflow this maps onto: reamerlabs.com