Stop orders
How conditional orders rest outside the matching engine in their own Scylla table, why they reserve customer funds before they trigger, where the monitor runs, and the orphan failure to watch for.
Stop-limit and stop-market orders are accepted on the same endpoint as ordinary orders and share almost nothing with them. They live in a different table, are watched by a different process, reserve funds by a different rule, and fail in ways ordinary orders cannot. This page is the operator-side of all four.
The single fact that generates the most support traffic is at the top of it: a stop order takes the customer's money at placement, not at trigger. If somebody reports a locked balance with no open order, this is almost always why.
They rest outside the matching engine, deliberately
A stop_limit or stop_market body is diverted before any matching code runs
and written into trading.stop_orders — a table the matching engine never
reads. The engine loads open orders from trading.orders and
trading.open_orders_by_market only, so a resting stop can never reach a
matcher.
That separation is not tidiness. The engine has no trigger mechanism, and its
match loop only branches on LIMIT and MARKET; an unsupported type fell
through to a branch where neither side of the loop could advance, spinning the
while forever and pinning the event loop at 100% CPU. Keeping conditional
orders in their own table means that cannot recur.
When a stop fires, the monitor materialises it: it releases the reservation
and places a real LIMIT or MARKET order through the same placement path a
user would hit. From that moment it is an ordinary order and
the order lifecycle applies.
The row an operator will be asked about
trading.stop_orders is partitioned by ("userId") and clustered
("createdAt" DESC, id ASC), the same shape as orders.
| Column | What it holds |
|---|---|
stopPrice |
The trigger price. Crossing this arms the order |
limitPrice |
The price the resulting order rests at. Set for STOP_LIMIT, null for STOP_MARKET |
triggerDirection |
UP or DOWN — which way price must move to fire it |
referencePrice |
The market price at placement, from which triggerDirection was derived |
orderType |
LIMIT or MARKET — what the stop becomes when it fires |
status |
See the table below |
triggeredOrderId |
The id of the real order this stop created. Null until it fires |
failReason |
Why a trigger failed. Only set on FAILED |
reservedCurrency / reservedAmount |
The hold taken at placement. See below |
triggerDirection is decided once, at placement, from a reference price taken in
this order: the engine ticker's last, then the mid of the real order book,
then zero. A stop priced above the reference is UP, below it is DOWN, and a
stop priced exactly at the reference is refused with 422 — place a market or
limit order instead. If no price is available at all, the direction is inferred
from the side: SELL means DOWN, BUY means UP.
The book used for that mid is the real one, never the display book. AI market-maker levels carry a TTL and no backing order, so a mid derived from them is a price at which no trade can occur — and getting it wrong arms the stop the wrong way, so it triggers immediately or never.
Statuses
| Status | Meaning |
|---|---|
PENDING |
Resting, funds reserved, watched by the monitor |
TRIGGERING |
Transient. Claimed by the monitor; the real order is being placed |
CANCELLING |
Transient. Claimed by a cancel; the release is in flight |
TRIGGERED |
Fired successfully. triggeredOrderId names the real order |
CANCELLED |
Cancelled; the reservation was released |
FAILED |
The trigger could not place an order. failReason says why; the reservation was released |
Both spellings here use two Ls, unlike orders.status, which stores
CANCELED with one. They are different tables and different vocabularies.
To the customer, only PENDING stops reach the Open Orders tab, where they are
surfaced as OPEN. TRIGGERING is deliberately left out of that list: it is a
sub-second transient during which the real order is already being placed, and
listing it would show the resting stop and the freshly-placed live order in the
Open tab at once. (A PENDING row that already carries a triggeredOrderId is
dropped from the list for the same reason.) Terminal statuses keep their real
value and land in History — except TRIGGERED, which is omitted, because the
real order it created represents it there.
The reservation, and why funds are held with no open order
Funds are held before the stop row is written, and long before any trigger. The order matters: an active, claimable stop must never exist without its funds already locked, or a concurrent cancel could act on a stop whose funds are not held and release nothing.
| Side and type | Reserved | In |
|---|---|---|
| SELL | amount |
Base currency — exact; it matches the eventual order's hold |
BUY STOP_LIMIT |
amount x limitPrice + fee |
Quote currency — exact |
BUY STOP_MARKET |
amount x stopPrice x 1.1 + fee |
Quote currency — an estimate |
The fee component is estimated at the market's taker rate, because a triggered stop crosses.
The 1.1 is a 10% slippage buffer (STOP_MARKET_BUY_SLIPPAGE_BUFFER). A
stop-market BUY's real fill cost is unknown until it triggers, so the reservation
holds headroom for a modest gap through the stop price. It is a soft lock, not a
charge: at trigger the reservation is released in full and the exact hold is
recomputed against a fresh view of the book. A customer who sees "10% more than I
expected" locked against a stop-market BUY is seeing this, and it comes back.
The reservation is held under the key eco_stop_place_<stopId>. Cancel and
trigger both release with that same key, release-only and idempotent, so a race
between the two can never double-release — and the LWT claim below guarantees
only one of them acts at all.
Every limit the live placement path enforces is enforced here too: minimum and
maximum amount, price band, and cost band for a BUY. That is on purpose. A stop
is validated once, at placement; anything let through now is discovered at
trigger time, where placement rejects it and the stop is marked FAILED — so the
protection the customer was relying on does not fire, at exactly the moment it was
supposed to.
Where the monitor runs
StopOrderMonitor is an in-memory singleton with no election of its own. It
is started by the matching engine's init, on the leaseholder only, and stopped
on stand-down. So the placement rule is the engine's placement rule, unchanged:
- Exactly one process runs it, the one holding the
ecosystem-matchinglease. - A process that loses the lease tears the monitor down and clears its index — triggering a stop materialises a real, funded order, and a demoted process must not keep doing that beside the new leader.
- Read Operations for which process that is in your deployment. A cron-only process is refused the lease before any store is consulted.
Placement registers the stop with the monitor in the process that served the HTTP request. On a process that does not hold the lease, that monitor was never started and nothing sweeps its index — so the registration does nothing. The stop is picked up by the leaseholder's 60-second reconcile instead. The row is correct and the funds are held throughout; only the arming is delayed.
Two trigger paths
| Path | Cadence | What it catches |
|---|---|---|
onPriceUpdate(symbol, last) |
After every matching cycle | The fast path. A stop fires within milliseconds of the fill that crossed it |
sweep() |
Every 2s | Re-checks every indexed symbol against the engine ticker |
The sweep exists because not every price move comes from a user fill. AI market-maker activity in particular moves the ticker without a matching cycle on the fast path, and anything the fast path missed converges here.
Both paths are safe to fire at the same stop. materializeStopOrder opens with a
lightweight-transaction claim — UPDATE ... SET status = 'TRIGGERING' ... IF status = 'PENDING' — and only the caller for whom that applies proceeds. A
repeated tick, an overlapping sweep, or a cancel racing a trigger all resolve to
exactly one winner. The monitor also keeps an in-flight set so a stop already
being materialised is skipped rather than re-attempted.
The 60-second reconcile, and the orphan it exists for
Every 60 seconds the monitor re-reads the active stops from Scylla and diffs them
against its in-memory index: PENDING rows the index is missing are added,
indexed entries whose row is no longer PENDING are dropped.
Without it, two things orphan a stop forever:
- A failed boot load.
getAllActiveStopsreturns an empty array on error, so a Scylla blip at start-up would leave the index empty while everyPENDINGrow still shows as open to its customer. - A failed registration at placement. The register call is wrapped and logged rather than allowed to fail the placement, so a stop can be persisted and funded without ever entering the index.
Either way the row reads PENDING, the customer sees an open order, the funds are
held — and nothing is watching the price. The reconcile heals both directions and
logs when it does:
Reconciled stop index with DB: +2 missing PENDING stop(s), -1 stale entr(y/ies)Seeing that line occasionally is normal. Seeing it repeatedly with a growing +
count means registrations are failing at placement, and the STOP_ORDER log
lines around each placement will say why.
Crash recovery
The two transient statuses are finalised at the next boot, before the index is seeded:
CANCELLING— a cancel was interrupted after its claim. The reservation is released (idempotent) and the stop is markedCANCELLED.TRIGGERING— a trigger was interrupted mid-materialisation. Placement is not re-run, because it may already have created the real order and re-running would duplicate it. The reservation is released and the stop is markedFAILEDwithfailReason: "Server restarted while triggering; not retried to avoid a duplicate order."Any order that was placed stays live and healthy.
That is the correct trade: a FAILED stop beside a live order is a labelling
problem, a duplicated funded order is a money problem.
Cancelling a stop
The cancel endpoint checks the stop table first and falls through to the ordinary
order path only when the id is not a stop for that user. A timestamp query
parameter is still required by the handler, though the stop path does not use it.
Cancellation claims PENDING → CANCELLING with the same LWT the monitor uses,
so a cancel that arrives during a trigger loses cleanly and the customer is told
"Stop order has already triggered (or is triggering) and can no longer be
cancelled". Only PENDING is cancellable.
What happens when a market is deleted
Deleting a market at Admin → Ecosystem → Trading → Markets cancels and refunds the ordinary orders on that symbol and removes its candles, aggregated book and index rows.
It does not touch trading.stop_orders. A resting stop survives the removal
of the market it belonged to: the row stays PENDING, the reservation stays held
against the customer's wallet, and the monitor's reconcile keeps re-indexing it.
With the market gone the engine has no ticker to move, so in practice it simply
rests. If it does fire, placement fails at the market lookup and the stop is
marked FAILED — and because the reservation is released before placement is
attempted, the customer gets their funds back on that path.
There is no screen that lists stop orders, so an orphaned reservation is invisible from the admin panel. Ask affected customers to cancel, or retire the pair by disabling the market instead of deleting it — a disabled market refuses new stops while leaving cancellation open.
Two things to verify after any market deletion, because the Scylla cleanup is
matched on the full symbol (BTC/USDT) while the delete routes pass the market's
currency (BTC):
pnpm eco:index:check # open orders still indexed for the dead symbol?
pnpm rebuild:eco-orderbook # aggregated levels left behind?Both are read-only in this form. If either reports rows for a market you removed, the corresponding repair command is in Operations.
Diagnosing a stop that did not fire
Work down this list.
-
Confirm the row exists and is
PENDING. Everything else is a different problem. There is no admin screen forstop_orders; querytrading.stop_orderswithcqlshdirectly, keyed by the customer'suserId. -
Check
triggerDirectionagainstreferencePrice. A stop armed the wrong way never fires. That happens when the reference price at placement was wrong or absent — the log around placement will show which source it came from. -
Confirm the monitor is running in the process you think it is. Its start line is
StopOrderMonitor started: N pending stop(s) across M market(s). If it is absent, the engine in that process is a follower and the monitor lives elsewhere. -
Check the market's ticker is moving. The 2-second sweep compares against
engine.getTicker(symbol)?.last, and a market with no trades has no last price. A pair quoted only by AI display levels still has a ticker; a pair with no activity at all does not. -
Look for a
FAILEDrow with afailReason. A stop that fired and could not place tells you exactly what it hit — a limit it now breaches, a disabled market, an insufficient balance after other trading.
Related
- The order lifecycle — what a triggered stop becomes
- Tokens and markets — the limits and fee rates a stop is validated against
- ScyllaDB schema and engine tuning — the
stop_orderstable and its view - Operations — which process holds the matching lease