Conformance Test Suite

The execution specification defines what correct behavior is. This suite is what actually checks the engine against it — not a description of testing, the real test code, downloadable below.

"282 checks pass" is otherwise just a claim. This page exists so it doesn't have to be one: read the assertions yourself, see exactly what's being verified and against what threshold, and if you have an evaluation license, run it yourself against your own build.

What it runs

One script, tests/test_execution_engine.py, drives real OHLCV data through reamer_py.run_backtest() across eight configurations and checks the result against explicit invariants — not snapshot comparisons, not mocked engine internals, the actual compiled binary running actual bars:

1. Single-asset — all order types, all TIF variants, bracket exits
2. Multi-asset (2 tickers) — concurrent positions, shared equity, per-ticker independence
3. Multi-asset (4 tickers) — four tickers simultaneously
4. High leverage — 10x, position sizing and margin checks
5. Commission modes — RoundTrip / OpenOnly / CloseOnly
6. Spread / OhlcvType — Bid / Ask / Midpoint fill-price adjustments
7. Overnight swap — equity reduced by swap charges across date boundaries
8. Alignment union vs. intersection — mixed-schedule assets

What a "check" actually is

Each of the 282 is a real assertion against the result of an actual backtest run — things like: total slippage equals the sum of per-trade slippage, cancelled orders carry fill_price=0, IOC orders never remain Pending past the bar they were submitted on, margin rejections fire exactly when the spec says they should, roll events land only on roll-configured tickers. Every one of these ties back to a rule in the execution specification — the suite doesn't define behavior, it verifies conformance to behavior already defined there.

Running it yourself

The two strategy files below (execution_test_suite.py, multi_asset_test_suite.py) are what actually generates the order flow the suite checks — real strategies submitting real orders, not fixture data. All OHLCV data is generated locally by the suite itself, from a fixed seed — no external files, no market data license, nothing beyond these three files. Reading them doesn't require anything installed. Running the full suite does: a compiled reamer_py and an active license (a free evaluation license is enough — run_backtest() has no free tier, by design).

Download

• test_execution_engine.py Main suite runner and invariant checker — the 282 checks live here.
  Download
• execution_test_suite.py Single/multi-asset strategy exercising every order type and TIF variant.
  Download
• multi_asset_test_suite.py Concurrent multi-ticker strategy used for the 2- and 4-asset configurations.
  Download

To run: place the three files in the same relative layout as the Reamer repo (tests/test_execution_engine.py, strategy/examples/execution_test_suite.py, strategy/examples/multi_asset_test_suite.py), then python tests/test_execution_engine.py against a licensed reamer_py build — no data download, no setup beyond that. Pass condition is printed at the end: N/282 checks passed — ALL PASS.