All docs pages

Docs · Integration

Live trading with Reamer Research

Drawn from include/reamer_research_abi.h, python/README.md, cpp/README.md in the 4.11.3 Reamer Research kit.

Reamer Research trades live on its own, with your own data feed and your own broker. The engine makes no network calls, so nothing is tied to one vendor, and going live is three parts, two of which you run:

  1. Your bar feed and loop. Your process receives each closed bar from your broker or data vendor, keeps a rolling window per ticker, and calls your strategy's decision logic, the same logic your backtest ran.
  2. rr.Relay, in the library. It sends the orders your strategy returns, checked by the same submission rules as a backtest, over a Unix domain socket, and keeps your positions from the fills sent back.
  3. Your receiver. A process listening on that socket. It places each order with your broker's API and writes each fill back to the relay.
bar feed ──> your loop ──> on_bar() ──> rr.Relay ══ Unix socket ══> receiver ──> broker API
                 ^                         │                            │
                 └─── relay.positions() <──┴──────── fills <────────────┘

A working version of all three, standard library only, ships in the kit under python/live/, and is public under the MIT License in github.com/reamerlabs/reamer-research-examples:

  • relay_receiver.py is a complete receiver: the wire codec, a socket server, and PaperBroker, which fills orders against the bars you give it. Replace PaperBroker with your broker's API to trade for real.
  • live_paper.py runs the quickstart's on_bar, unchanged, live against PaperBroker. It replays a CSV bar by bar, or follows Alpaca's free 1-minute crypto bars, which need no key.
  • cpp/live/live_paper.cpp runs the same rule from C++ through rr::Relay, against the same receiver. See From C or C++.

Before you start

  • A licence. Evaluation Mode has no live trading. Without a licence, opening a relay raises rr.EvaluationLimit (rr::EvaluationLimit in C++): the live relay needs a Reamer Research licence. Everything else on this page can be read and built first.
  • Linux or macOS (WSL2 counts as Linux). The relay speaks over a Unix domain socket.
  • A backtested strategy. Live trading reuses its decision logic, not its data files or engine_config.json. The fees, slippage and spread you configured are a simulation; live, your broker charges what it charges.

1. Run the paper example

pip install research-kit-*/python/reamer_research-*.whl
cd research-kit-*/python
python3 live/live_paper.py                   # replay sample_data/SAMPLE_5min.csv
python3 live/live_paper.py --alpaca BTC/USD  # live 1-minute bars, Ctrl-C to stop

The replay sends each order through the relay to the receiver, and prints each order as it goes and the fill count at the end. On the sample data, the quickstart strategy sends 87 orders and gets 173 fills back, with 86 positions closed by their brackets and the last one still open. The same on_bar backtested over the same file closes 88 trades. The counts differ because PaperBroker fills at a bar's open or at the order's price, while the backtest simulates fills inside each bar (EXECUTION_SPEC.md). Trust the backtest for results; the paper broker tests the plumbing.

2. The live loop

Nothing calls on_bar live: your loop does, once per closed bar. In a backtest on_bar receives a window of the last lookback bars; live, your loop keeps that window itself.

import reamer_research as rr

with rr.Relay('/tmp/reamer-live.sock', ['AAPL']) as relay:
    data = LiveData(['AAPL'], lookback=20, relay=relay)   # from live_paper.py
    for bar in your_feed():              # each closed bar: ts, open, high, low, close, volume
        relay.poll(0)                    # read fills that arrived since the last bar
        data.push('AAPL', bar)
        orders = on_bar(data)            # your backtest's on_bar
        if orders is not None:
            relay.send(orders)

LiveData in live_paper.py provides the parts of a backtest's data most strategies read: close, open, high, low, volume, notional and ts as numpy windows, oldest first and zero-padded while they fill, and position(), position_state(), ts and tickers. With several tickers, data['AAPL'] gives one ticker's window, and valid is True only for the ticker whose bar just closed. A strategy that reads only these runs live unchanged.

What LiveData does not provide, and what to do instead:

Backtest call Live
data.open_orders() relay.send() returns the IDs of the orders it sent; your receiver's records say which still rest. The relay keeps its own record for cancel_all and modify_order.
data.action_results() relay.action_results gives one result per action of the last send(), from the relay's own checks. Your broker's accept or reject reaches your receiver, not the relay; log it there.
data.exogenous(ticker) Read the live value from its own source.
position_state() P&L qty, avg_entry_price, take_profit and stop_loss come from the relay. unrealized_pnl and entry_ts are 0: the relay has no market price.

An indicator that you kept as running state, rather than recomputing from the window, needs the same warm-up live: push enough history through the loop before you let it trade. live_paper.py does this from Alpaca's recent bars.

3. What the relay sends

Order On the relay
buy_market, sell_market, buy_limit, sell_limit, buy_stop, sell_stop Sent, with tif (GTC, IOC or GTD and its expiry)
take_profit=, stop_loss= on an entry Sent with the entry. Your receiver places them and reports their fills (below)
close_position(qty=0) Sent as a market order on the side that reduces the position; 0 means all of it
cancel_order(order_id) Sent for an order this relay sent earlier
modify_order(order_id, ...) Sent as a Replace of a resting order, with any new take profit or stop loss in it, and a Cancel of each exit it clears
modify_position(ticker=, ...) A Replace of each live exit it moves, a Cancel of each it clears, and a new exit order for each level the position had none for
close_all() One close per open position, each taking an order ID
cancel_all() One Cancel per order resting before the call

relay.send() takes the same list on_bar returns and applies it in the backtest's order: the orders, then close_all's closes, then the other actions. It checks everything before it sends anything, using the backtest's rules. An entry needs qty > 0, a limit or stop needs its price, a close needs a position, modify_order needs an order still resting and modify_position an open position. If one fails, the call raises with that item's index and the rule it broke, and nothing is sent. The n-th order on a relay is order ID n, counting from 1 and not counting cancels, as in a backtest; relay.send() returns the IDs it took.

The relay has no market price, so it does not check which side of the market a new exit level is on. A backtest rejects a stop loss above the price of a long position; live, your receiver or your broker has to.

relay.positions() is built from the fills this relay sent and received. After a reconnect, fills for orders sent before it do not arrive, so check positions against your broker's records when the connection drops.

4. Writing your receiver

The receiver is a Unix socket server. The relay connects to it at rr.Relay(socket_path, ...), sends one frame per order, and reads fills back on the same connection.

Every frame is a version byte (1), the payload length as a big-endian u32 (at most 1 MiB), then the payload. The payload's first byte is its type. Integers are big-endian, a double is its 8 IEEE-754 bytes in big-endian order, and a string is a big-endian u16 byte length followed by that many UTF-8 bytes.

Relay to receiver

Type Message Fields, in order
0 NewOrder strategy_id str, intent_id str, instrument str, side u8, order_type u8, quantity f64, limit_price f64, stop_price f64, position_effect u8, time_in_force u8, expire_time str, then the bracket block when the order has a take profit or stop loss
1 Cancel strategy_id str, request_id str, target_order_id str
2 Replace strategy_id str, request_id str, target_order_id str, new_quantity f64, new_limit_price f64, new_stop_price f64, then, when it moves a take profit or stop loss, ext_version u8 (1), new_take_profit f64, new_stop_loss f64, new_trailing_offset f64

The bracket block is ext_version u8 (1), take_profit f64 and stop_loss f64, then three f64, a u8, an f64, a u8, two f64 and a u32 the relay sends as zero, then contingency_type u8 (1 one-cancels-other) and linked_intent_id str, then two str, a u32 and a u32 payload length of 0. relay_receiver.py reads them in order. In a Replace, a 0 leaves that value as it is.

Receiver to relay

Type Message Fields, in order
1 OrderUpdate intent_id str, instrument str, side u8, status u8, filled_quantity f64, last_price f64, venue_order_id str, commission f64, reason str, timestamp str

Values: side 0 buy, 1 sell. order_type 0 market, 1 limit, 2 stop. position_effect 0 open, 1 close. time_in_force 1 GTC, 2 IOC, 4 GTD. status 1 new, 2 partially filled, 3 filled, 4 cancelled, 5 rejected.

The rules that matter:

  • intent_id is <strategy_id>-<session nonce>-<n>, unique across sessions, and the key for everything that follows. A cancel's target_order_id is one of these.
  • filled_quantity is cumulative for that intent_id. The relay moves the position by the increase since the last update, at last_price. Send the total filled so far, not the size of the latest fill.
  • Exits fill under their own IDs. Report an entry's take-profit fill as <intent_id>:tp and its stop-loss fill as <intent_id>:sl. They count against the entry. An exit modify_position adds arrives as a NewOrder of its own, <strategy_id>-<nonce>-p<k>:tp or :sl, a close linked one-cancels-other to its partner through linked_intent_id. A Cancel or Replace of <intent_id>:tp targets that exit.
  • Exits cancel each other. When one of a pair fills, cancel the other at your broker. The relay counts the partner done as soon as the fill arrives, and a fill that leaves a position flat ends all its exits.
  • The relay reads intent_id, status, filled_quantity and last_price. An order ends when its status is filled, cancelled or rejected, and cancel_all and modify_order then leave it alone. Report a broker reject as status 5 so the relay stops counting the order as resting. It sends no acknowledgement of its own.
  • A malformed frame should close the connection. The relay reconnects on its next send() or poll(), backing off from 1 s to 30 s.

The skeleton of a real receiver, on top of relay_receiver.py:

from relay_receiver import Receiver, FILLED

class MyBroker:
    def on_order(self, order, report):
        # order: intent_id, instrument, side, order_type, quantity,
        # limit_price, stop_price, is_close, time_in_force, expire_time,
        # take_profit, stop_loss, linked_intent_id
        broker_id = place_with_your_broker(order)       # your broker's API
        remember(broker_id, order, report)

    def on_cancel(self, cancel, report):
        cancel_with_your_broker(cancel['target_order_id'])

    def on_replace(self, replace, report):
        # target_order_id, quantity, limit_price, stop_price, take_profit,
        # stop_loss: 0 leaves a value as it is
        modify_with_your_broker(replace)

def on_broker_fill(broker_id, total_filled, price):     # from your broker's fill stream
    order, report = recall(broker_id)
    report(order['intent_id'], order['instrument'], order['side'],
           FILLED, total_filled, price)

with Receiver('/tmp/reamer-live.sock', MyBroker()):
    run_your_broker_stream(on_broker_fill)

Run the receiver as its own process, or as a thread beside your loop, as live_paper.py does. The socket is created with mode 0600, so only your user can connect to it.

From C or C++

The relay is the same in every language: reamer_relay_open(), reamer_relay_send_batch() for orders and actions, reamer_relay_poll() and reamer_relay_get_positions() in include/reamer_research_abi.h, and rr::Relay in the C++ header. The C++ calls look like this:

rr::Relay relay("/tmp/reamer-live.sock", {"AAPL"});
auto ids = relay.send(rr::buy_market(10).stop_loss(180));   // {1}
relay.poll(500);                     // read fills for up to 500 ms
double qty = relay.positions()[0].qty;
relay.send(rr::modify_position(0, 185));   // take profit 0 clears, stop loss 185

rr::modify_order, rr::modify_position, rr::close_all and rr::cancel_all build the actions, and relay.send(orders, actions) sends both in one call, as an on_bar returns them.

The bytes on the wire are the same, so the Python receiver above serves a C++ strategy too. As in Python, your C++ loop keeps the bar window and calls your strategy's logic once per closed bar.

cpp/live/live_paper.cpp in the kit is the paper example in C++. It starts python/live/relay_receiver.py itself, marks each bar on the receiver's bar channel so PaperBroker can fill what rests, and runs the quickstart's rule on a 20-bar window. From the kit root:

c++ -std=c++20 -O2 cpp/live/live_paper.cpp -Iinclude \
    -Llib -lreamer_research -Wl,-rpath,"$PWD/lib" -o live_paper
./live_paper                       # replays python/sample_data/SAMPLE_5min.csv

On the sample data it sends the same 87 orders and gets the same 173 fills back as live_paper.py. To run the receiver on its own, for a loop of your own, start python3 python/live/relay_receiver.py /tmp/reamer-live.sock.

Buy a licenceKey and kit by email on payment.