Knowing how to write a specific hypothesis, design a fair experiment, and extract the right knowledge from a result doesn't, by itself, make a research process repeatable. Repeatability is a separate property — it comes from applying the same standards to every idea without re-deciding them each time, not from knowing what good practice looks like in principle.
Repeatable doesn't mean deterministic
This isn't about a strategy producing the same fills on the same data every time — that's determinism, a property of the execution engine, not of a research practice. A repeatable process is one where the researcher doesn't have to reinvent, for every new idea, how rigorous to be this time: whether to start with realistic costs or add them later, how much data is enough, when iteration stops and validation begins. Re-deciding those questions from scratch for each idea is where a lot of process inconsistency quietly creeps in — not from any single bad decision, but from every idea getting slightly different treatment depending on the mood of the day it was tested.
Turning good practice into defaults
Each piece of good practice covered elsewhere in this tier — specific hypotheses, honest experiment design, a real discard bar, a clean separation between iteration and validation, deliberately extracting the reason behind a result — is only actually repeatable once it stops being a decision made per idea and becomes a default applied to all of them. Always start with realistic costs. Always decide the acceptance bar before seeing the result. Always name the specific reason before moving to the next idea. None of these need to be reconsidered every time; deciding them once, and then simply applying them, is what repeatability actually looks like in practice.
Why consistency across ideas is the actual payoff
An idea tested against a strict standard and an idea tested against a looser one aren't really comparable, even if both produced a similar-looking number — the difference in how carefully each was checked is invisible in the result itself. A repeatable process is what keeps that comparison fair: when every idea gets held to the same standard, differences between results actually mean something about the ideas, not about how much scrutiny each one happened to receive.
Repeatable isn't rigid
None of this means the standards themselves are fixed forever — the specific cost assumptions, discard thresholds, or validation criteria can be deliberately revised as understanding improves. What stays fixed is the discipline of applying whatever the current standard is consistently, rather than quietly loosening it for an idea that feels promising or tightening it for one that doesn't. That consistency, more than any single technique, is what turns a series of one-off experiments into a practice that actually accumulates.
Full reference: docs · The workflow this maps onto: reamerlabs.com