Reamer Research Benchmarks

Test system

Constrained hardware (8-year-old ultrabook) used deliberately for reproducibility. Production deployments on newer server CPUs will see lower absolute latencies due to higher clock speeds and larger caches. The architecture and relative costs remain identical. Only absolute numbers scale.

What's measured: three integration paths

Reamer Research can be integrated three ways. Each is measured separately.

Pure C ABI floor

A customer links libreamer_research.a or libreamer_research.so directly into their C/C++/Rust process and calls the backtest entry point through a vtable callback. This measurement isolates the cost of the ABI boundary itself (indirect function calls and struct marshaling) with no wrapper overhead.

C++ reference wrapper

The reference-cpp wrapper (in the release tarball) uses std::span ergonomics and RAII handles to wrap the raw ABI. This adds convenience over raw ABI calls at the cost of a thin wrapper layer. Measures the real cost that a C++ or Rust customer sees when using the reference implementation as provided.

Python ctypes wrapper

The reference-python binding uses Python's ctypes module to call the C ABI and reconstructs Python method ergonomics. This is the highest-level integration path, trading Python simplicity for absolute per-call latency. The binding is explicitly unsupported (a demonstration only), but the performance measurements show what customers who choose Python will experience.

Minimal vs. realistic workloads

Two workloads isolate two different questions: callback-dispatch overhead versus real strategy execution cost.

Minimal: pure callback dispatch

The strategy reads one price field and submits no orders (measures wrapper overhead in isolation). For the ABI, this means the on_bar callback receives a window of OHLCV bars, reads one close price, and returns zero orders. This is the upper bound of callback overhead.

Realistic: 20-bar Donchian breakout with order flow

The strategy detects new 20-bar channel highs and lows, submitting market orders through the ABI's vtable when breakouts occur. This includes real computation (indicator calculation, threshold comparison), order construction, and order submission. For the ABI, the callback receives the window, computes the channel, constructs orders in the caller-allocated buffer, and returns the order count. This workload exercises the real cost profile of a strategy that trades.

Dataset

Minimal workload: 1,000 bars, 50 independent runs. Realistic workload: 884,130 bars (the same as the old reamer_py benchmark for historical comparison), 3 independent runs. Both use synthetic 15-minute-bar OHLCV data derived from a real GBP/USD price series, extended by tiling to arbitrary length with continuous timestamps.

Minimal workload results

How much overhead does each layer add to a trivial callback? Headline: pure ABI, 514,186 bars/sec. C++, 440,083 bars/sec. Python, 18,313 bars/sec. ABI floor: 1.94µs mean

Wrapper overhead is a fixed per-call cost. The C++ wrapper adds roughly 330 nanoseconds per callback. The Python wrapper adds 52.67 microseconds. At trivial callback cost (callback dispatch only, zero strategy work), this overhead becomes the dominant part of the bill.

Realistic workload results

How much overhead do wrappers add when strategy work dominates? C++ overhead nearly disappears. Python still pays the fixed cost, but it amortizes into real strategy work. ABI: 664,585 bars/sec (1.50µs) · C++: 664,842 bars/sec (1.51µs) · Python: 16,502 bars/sec (60.59µs)

On realistic workload with real order flow, the strategy logic and order matching dominate the per-bar cost. The C++ wrapper's fixed callback-dispatch overhead becomes immeasurable (only +0.01µs, within noise). The Python wrapper's 60µs cost remains fixed, but now the delta versus minimal workload is only 6µs (60.59 minus 54.61), showing that Python's fixed boundary-crossing cost becomes less visible as real work grows.

What this means

A C++ or Rust customer using the C++ reference wrapper pays zero overhead for integration convenience. A Python customer pays 60 microseconds per bar, which is interactive for research and strategy development (1 million bars on this machine takes about 60 seconds, single-threaded), a reasonable tradeoff for zero build complexity and Python's language flexibility. The ABI floor itself (1.5 microseconds) is the cost when the customer writes pure C/C++ and links the library directly.

Reading these numbers honestly

• This is not a competitive claim. The numbers measure wrapper cost, not performance ranking. Compare to your own integration path (C++? Python?) and your own machine (production EPYC? ultrabook laptop?), not to other products.
• The ABI floor (1.5 microseconds per bar on realistic workload) is what you get when you write C++ and link the library directly, with zero wrapper overhead. This is production-grade performance.
• The C++ wrapper adds negligible overhead (+0.01µs) on real strategy work, making it a free convenience layer for safe C++ integration.
• The Python wrapper is explicitly unsupported in production (it is a reference demo, not a claim surface). But for research and iterative strategy development, 60 microseconds per bar is interactive.
• Both workloads use zero-cost execution config (no commission, no slippage, no spread). These are throughput numbers on clean execution, not P&L simulation.
• Hardware is constrained (8th-gen ultrabook CPU). Production deployments on newer server CPUs will see lower absolute latencies but identical relative costs between the three layers.
• The historical comparison: the old pybind11 monolithic Python build achieved 7.5 microseconds per bar on this same machine, same Donchian workload. The new C ABI achieves 1.5µs (C++) and 60.6µs (Python). The C++ path is 5x faster than the old build. The Python path is 8x slower than the old build but gains language independence, stable ABI, and deployment flexibility in exchange.

Methodology and reproducibility

Measurement methodology

• Each tier (ABI / C++ / Python) runs in its own isolated process, so peak-memory measurements are clean.
• Timing wraps only the actual backtest execution, excluding one-time setup (data ingestion, strategy object construction).
• Wall-clock latency is measured per on_bar() callback invocation.
• All tiers ran sequentially, never in parallel. Concurrent runs would contend for the same CPU core and contaminate single-core measurements.
• No governor tuning or CPU pinning; measurements on standard system defaults.

Running the benchmarks

Pure C ABI (vtable calls, no wrapper):

./build/strategy/abi/benchmarks/native_research_benchmark
./build/strategy/abi/benchmarks/native_research_benchmark 2000 100  # custom bar count and runs

C++ reference wrapper:

./build/reference-cpp/benchmarks/bench_reference_cpp
./build/reference-cpp/benchmarks/bench_reference_cpp 2000 100

Python ctypes wrapper:

PYTHONPATH=reference-python REAMER_RESEARCH_LIB=build/libreamer_research.so \
  python3 reference-python/benchmarks/bench_research_binding.py

# Custom parameters:
PYTHONPATH=reference-python REAMER_RESEARCH_LIB=build/libreamer_research.so \
  python3 reference-python/benchmarks/bench_research_binding.py 2000 100

All three benchmarks self-activate a test license internally (no manual activation needed to run them). Full build and run instructions in the main repository: see BENCHMARK.md for complete methodology, raw numbers, and histogram percentiles.