Benchmarks — reamer_py vs. Backtrader vs. QuantConnect LEAN

Test system

reamer_py's execution core is single-threaded by design — these results reflect one CPU core.

Why Backtrader & QuantConnect LEAN

There is no certified, industry-standard benchmark suite for backtesting engines. Backtrader is the long-standing standard for retail Python backtesting — the tool an entire generation of retail/indie quants learned on, and still one of the most widely-used. It's been in long-term maintenance mode since 2023 (no major new features, last major commit in 2020), and current community consensus doesn't recommend it for brand-new projects in 2026 — but that doesn't change what it does when it runs.

Maintenance status affects future feature velocity, not the runtime behavior of the version actually benchmarked here, and it remains — like reamer_py — an event-driven engine: both step through the timeline one bar at a time via a per-bar Python callback (next() vs. on_bar()), rather than processing the entire history as a single batch array operation the way a purely vectorized backtester (e.g. VectorBT) does.

What reamer_py and Backtrader share, and VectorBT doesn't, is the per-bar event loop — the thing that makes path-dependent state, realistic order management, and tick-level fills possible in the first place.

QuantConnect LEAN is included because it's the most widely recognized name in this space with an actual open-source, freely runnable core — unlike most research-infrastructure competitors, its engine can be built from source and run head-to-head without a subscription or an account. QCAlgorithm.OnData(slice) is the same per-bar event-driven shape as next() and on_bar(), called once per aligned timestep, so it's a fair architectural match for this comparison.

NautilusTrader was evaluated for this comparison and deliberately excluded from the published numbers — not for performance reasons alone, but because it sits at a different point in the trading pipeline than reamer_py targets. NautilusTrader (like QuantConnect/LEAN) is built around backtest/live parity as a first-class goal — the same order, portfolio, and risk code paths run in both a backtest and a live deployment. reamer_py is deliberately research-only, with no live trading connectivity at all, which is exactly why it doesn't carry that infrastructure's overhead. LEAN carries the same live-parity architecture and the same overhead — it's included anyway because, unlike NautilusTrader, it's already a named, same-tier pricing comparison for reamer_py's positioning, so a real, disclosed head-to-head is more useful here than a clean architectural exclusion. That's a real difference in what each tool is for, not just a speed gap — the numbers below are what each engine actually does today, not a claim about what either is built for.

What "minimal" and "realistic" mean

Two workloads were run against both engines, to separate two different questions.

minimal — pure per-bar callback cost

The strategy reads one price and does nothing else — never submits an order:

# reamer_py
def on_bar(self, data):
    _ = data["TICKER"].close[-1]
    return None

# Backtrader
def next(self):
    _ = self.data.close[0]

# QuantConnect LEAN
def OnData(self, data):
    _ = data[self.symbol].Close

This isolates the fixed overhead of crossing into the strategy once per bar, with nothing else in the way — the fastest either engine could possibly go for the given bar count, upper-bounded only by interpreter/FFI overhead.

realistic — 20-bar Donchian channel breakout with real order flow

Enter long on a new 20-bar high, enter short on a new 20-bar low, submitting real market orders through each engine's own order management and fill-matching pipeline — the same entry rule used in eval-kit/benchmark_your_machine.py:

# reamer_py
def on_bar(self, data):
    tv = data["TICKER"]
    channel_high, channel_low = tv.high[-21:-1].max(), tv.low[-21:-1].min()
    last_close, side = tv.close[-1], tv.position.side
    if last_close > channel_high and side <= 0:
        return buy_market(1000.0 + tv.position.qty, ticker="TICKER")
    if last_close < channel_low and side >= 0:
        return sell_market(1000.0 + tv.position.qty, ticker="TICKER")

# Backtrader
def next(self):
    pos, close = self.position.size, self.data.close[0]
    if close > self.channel_high[0] and pos <= 0:
        self.buy(size=1000.0 - pos)
    elif close < self.channel_low[0] and pos >= 0:
        self.sell(size=1000.0 + pos)

# QuantConnect LEAN
def OnData(self, data):
    pos, close = self.Portfolio[self.symbol].Quantity, data[self.symbol].Close
    if close > channel_high and pos <= 0:
        self.MarketOrder(self.symbol, 1000.0 - pos)
    elif close < channel_low and pos >= 0:
        self.MarketOrder(self.symbol, -(1000.0 + pos))

A position reversal is submitted as a single order sized to flip the net position directly on both sides (not close-then-reopen as two separate orders), so trade counts across engines land within a fraction of a percent of each other rather than differing by a fixed 2x purely from a counting-convention mismatch. reamer_py's own reversal quantity — 1000.0 + tv.position.qty — isn't cosmetic: reamer's netting rule requires qty to exceed the current position size for the excess to open the opposite side (position.qty is 0 when flat, so the formula is a no-op there and only adds the existing opposite position's size when actually reversing) — a flat 1000.0 would just close an opposite position to flat, never reverse it. This workload exercises indicator computation, order submission, and fill matching — the actual cost profile of a strategy that trades, not just reads data. Trade size is 1,000 units, not 1 — a realistic minimum lot size for the FX data used here, not a single share/coin.

Data & methodology

A synthetic 15-minute-bar OHLCV dataset built from a real historical GBP/USD price series, extended to arbitrary length by tiling it end-to-end with continuous re-stamped timestamps — real price action, not fabricated from scratch. For the 50-ticker portfolio test, each ticker is an independently price-scaled, time-offset slice of that same base series, all sharing one timeline so each engine's alignment treats them as fully overlapping — approximating 50 different instruments traded over the same 1-year window. reamer_py reads a memory-mapped .bin format directly; Backtrader reads pickled pandas.DataFrames; LEAN reads the same series as CSV through a custom PythonData subscription, since its own historical data provider can't serve arbitrarily-sized synthetic series.

• Each engine run happens in its own isolated process, so peak-memory measurements are clean per engine, not inflated by whatever the other engine already had loaded.
• Timing wraps only the actual backtest call, excluding one-time setup (data loading into the engine, strategy/indicator object construction) — for LEAN, this means CLR + Python.NET startup cost is included in the timed figure, since there's no clean way to separate engine bootstrap from execution when driving the launcher DLL directly. LEAN's numbers carry a fixed startup tax the other two engines don't.
• Peak memory measured via each process's own resident-set-size high-water mark (RUSAGE_CHILDREN for LEAN, since dotnet runs as a direct child process, not containerized).
• All engines' full test suite ran sequentially, never in parallel — running single-threaded throughput comparisons concurrently would contend for the same CPU cores and contaminate all measurements.

Running LEAN's open-source engine (rather than lean-cli) also required a real OpenSSL 1.1 compatibility fix on a modern (OpenSSL 3.x) system — Python.NET's native interop needs the actual historical libssl.so.1.1/libcrypto.so.1.1 binaries, not a same-named symlink to the 3.x libraries, which fails ABI/symbol validation at load time.

Single ticker, 500,000 bars

~14.25 years of continuous 15-min bars.

Single ticker, 5,000,000 bars

~142.5 years of continuous 15-min bars. QuantConnect LEAN is excluded from this tier — see the wall-clock clamp callout above; a 142.5-year backtest cannot fit before "now" under any anchor.

50-ticker portfolio

35,064 bars/ticker (1 year of continuous 15-min bars each, ~1.75M total ticker-bar-instances).

"steps/s" = aligned timesteps/second, not ticker-bars/second — reamer_py's on_bar and LEAN's OnData both fire once per aligned step across all 50 tickers simultaneously, not once per ticker-bar, so this isn't directly comparable to the single-ticker bars/s figures above.

Peak RSS, for reference

LEAN's RSS is consistently the highest of the three by a wide margin — expected, given it's running a CLR host with an embedded Python.NET interpreter bridging every call, on top of its own framework object graph (Portfolio, Securities, TradeBuilder, order/fill event machinery) built for backtest/live parity, not throughput.

Reading these numbers honestly

• The gap is consistently far larger on minimal (49-212x) than realistic (18-35x) — expected, since minimal isolates pure per-bar interpreter/FFI overhead (where a compiled C++ core vs. pure-Python-or-CLR-bridge architecture is starkest), while realistic adds order-matching and bookkeeping work every engine has to do regardless of implementation language.
• The portfolio test shows the largest gaps of all (212.0x vs. Backtrader, 198.3x vs. LEAN on minimal) — both other engines' per-bar synchronization cost across 50 simultaneous data feeds appears to scale worse than linearly with ticker count, not just proportionally.
• Trade counts on realistic land within a fraction of a percent of each other across all three engines at every tier tested (e.g. 41,465 / 41,415 / 41,373 at the 50-ticker tier) — confirming the three implementations are running functionally equivalent logic, not just similarly fast, unrelated ones. Getting this tight required reamer_py's reversal order to explicitly account for the existing opposite position's size, not a flat order quantity.
• LEAN's numbers include CLR + Python.NET startup cost that the other two engines' numbers don't carry (see Methodology) — a real cost of running LEAN's engine this way, not an artifact to subtract out, but worth keeping in mind when comparing LEAN's minimal figure specifically, where fixed startup cost is a proportionally larger share of the total.
• These are throughput numbers on zero-cost execution config, run on an ultrabook laptop CPU — not a benchmark of P&L realism, and not a general claim about every backtesting tool on the market. Backtrader and QuantConnect LEAN were chosen because they're the two most widely-recognized, freely-runnable names in this space, not because every alternative was surveyed.