If most ideas that get tested end up surviving, that's not evidence of a good ideas pipeline. It's usually evidence the test isn't strict enough. A healthy research process discards most of what it tests, and that's the correct outcome, not a disappointing one.
Why the discard rate should be high
Look at enough charts, enough indicators, enough parameter combinations over a finite dataset, and some of them will appear to work — not because they capture something real, but because a finite amount of historical data contains a finite amount of noise, and noise occasionally lines up in a way that looks like a pattern. There are vastly more ways to notice an apparent regularity than there are actual, persistent edges underneath the data. A rigorous test exists specifically to catch this, which means its job is to reject most candidates — because most candidates, honestly, are exactly that: noise that happened to resemble signal closely enough to get noticed.
An idea surviving contact with realistic costs and a robustness check is supposed to be the exception. If it isn't — if nearly everything that gets tested comes out looking validated — the test is the thing worth questioning, not the unusual run of good ideas.
The trap is specifically psychological
Discarding an idea that took real time to implement and run is uncomfortable in a way that's easy to underestimate. That discomfort creates quiet pressure to read an ambiguous or marginal result as "good enough" rather than as the rejection it actually is — not because of any conscious dishonesty, just because a clean rejection feels like wasted effort and a generous reading doesn't. A process that keeps an unusually high fraction of what it tests is often not finding better ideas; it's just under more of that pressure, and giving in to it more often.
What a high discard rate actually signals
Rejecting most tested ideas is evidence the test is actually discriminating between something real and something that only looked real — which is exactly the job. A process that never discards anything is either applying too generous a bar, or isn't really testing new, independent ideas at all — just re-confirming small variations of one idea, which survive together for the same reason and don't represent separate evidence of anything.
The discard is not the failure
None of this makes a discarded idea a wasted afternoon — that's a separate point, covered elsewhere in this loop. What matters here is narrower: expecting most ideas to fail is the correct expectation to walk in with, not a low bar or a sign of pessimism. The actual skill in this stage isn't protecting ideas from that outcome. It's being willing to reach it quickly, honestly, and without resistance, as often as the evidence actually says to.
Full reference: docs · The workflow this maps onto: reamerlabs.com