A backtest can be extremely convincing and still not be trustworthy. A smooth equity curve, a confident explanation of why the strategy works, someone who clearly knows what they're talking about — none of that is actually evidence. It's just what evidence tends to look like when it's real, which is exactly why it's also what a flattering, badly-tested result looks like too. The two are easy to confuse from the outside. Telling them apart requires something more specific than confidence.
Trustworthy has to mean checkable
A result earns trust by being verifiable, not by being persuasive. Four properties, specifically:
Reproducible. The same inputs have to produce the same output — not something in the same ballpark, the same fills, every time, on any machine. If a result can't be regenerated exactly, there's no way to tell whether what you're looking at is the strategy's real behavior or an artifact of whatever happened to run that one time.
Specified. The rules that produced the result need to be written down somewhere checkable, not just embodied in whatever the code happens to do. "The execution model behaves however the implementation behaves" isn't a specification — it's an invitation to find out the hard way when the implementation has a bug. A published spec is something a claim can be checked against.
Inspectable. A summary statistic is a claim, not a demonstration. Being able to find the exact fill, at the exact tick, that explains why a specific trade happened is what turns "trust me, it made money" into something an outside observer — or a future, more skeptical version of yourself — can actually verify.
Tested against failure. A result that's only ever been asked "how did this do" hasn't been tested — it's been described. The real question is whether it survives being asked "would this hold up against data it hasn't seen, or plausible variations of what actually happened." A strategy that's never had to answer that question hasn't earned trust yet, however good the headline number is.
What doesn't count, even though it gets mistaken for it
A high return number, on its own, says nothing about whether it will repeat. A compelling story about why a pattern "should" exist is not the same thing as the pattern surviving contact with realistic costs. And a strategy that's been adjusted repeatedly until it finally looks good on the data in front of it is the single easiest way to manufacture a convincing result that means nothing — the fit improves, but that's a description of the tuning process, not evidence the underlying idea is real.
None of these are always wrong. A real edge usually does come with a plausible story and a decent-looking return. The problem is that a fake one can produce exactly the same surface impression, which is why the surface impression is never the actual test.
Why this is worth being precise about
The whole point of testing a trading idea is to replace a guess with something more reliable. A result that's convincing but not checkable doesn't do that — it just replaces an obvious guess with a more expensive-feeling one. Reproducibility, a real specification, inspectability, and an honest test against failure aren't formalities layered on top of research. They're what separates research from a well-told story.
Full reference: docs · The execution model this rests on: execution specification · The workflow this maps onto: reamerlabs.com