Binary orders that did not settle

Why a binary contract expires and stays PENDING — the health endpoint, the in-process expiry timer, the 15-second backstop cron, the per-process settings cache, and the port-4000 socket.

9 min readUpdated 6 August 2026binary, settlement, cron, health, websocket

"A binary trade expired and nothing happened" is the ticket this page answers. It has a small number of causes, and they are quick to separate once you know which of the two settlement paths is supposed to have run.

Work through it in this order. Each step rules something out cheaply before the next one costs you time.

0. Is binary trading switched on at all

Admin → Finance → Binary Options → Binary Settings, the Binary Trading master switch under Master Controls. It writes the binaryStatus setting.

With it off, POST /api/exchange/binary/order refuses every order with a 403, "Binary trading is currently disabled", and the /binary link disappears from the main navigation, the footer and the home page — all three read binaryStatus. The backend also skips its boot sweep entirely: the "Binary Orders" startup task that re-arms expiry timers runs only when binaryStatus is "true".

So an install with the switch off looks, to a customer, exactly like binary options were never installed — and a contract that was open when you turned it off has no timer behind it any more. Check this before debugging anything else.

Practice mode has a second switch, binaryPracticeStatus. Demo orders are refused with "Binary practice mode is currently disabled" when it is off, independently of the master switch.

1. Ask the health endpoint

The product ships a health check for exactly this and no screen links to it:

GET /api/exchange/binary/health

It requires a signed-in session and no permission key, so the simplest way to read it is to open the URL in a browser tab while signed in as an admin. It runs five checks and folds them into one verdict.

Check	What it actually does
`system`	reads the `binaryStatus` setting — `down` when binary trading is disabled
`database`	`sequelize.authenticate()` — a plain connectivity probe
`durations`	counts active binary durations
`markets`	counts binary markets it considers active
`orders`	counts `PENDING` binary orders, and those pending more than 24 hours

Each check reports up, warning or down. The overall status is down if any check is down, degraded if any is a warning, and healthy only when all five are up.

Read the individual checks, not the headline verdict.

durations warns on every install. It queries a binaryDuration model that no longer exists — durations moved into the binarySettings JSON blob, which is what GET /api/exchange/binary/duration actually reads. The lookup yields nothing, so this check always reports "No active binary durations found". Confirm your durations on the Binary Settings screen instead.
markets counts the wrong side of the toggle. It filters binaryMarket on status: "ACTIVE" while status is a boolean column, so MySQL compares the string as zero and the number returned is the count of disabled markets. Confirm enabled markets from Binary markets instead.

system, database and orders are correct and are the three worth acting on. A permanently degraded verdict on an otherwise healthy install is expected.

The one number to take seriously is orders.stuck — contracts still PENDING more than 24 hours after they were created. Anything above zero means neither settlement path finished, and that is the rest of this page.

2. Understand which path was supposed to settle it

Binary settlement has a primary path and a backstop, and they fail for different reasons.

The primary path is an in-process timer. When an order is placed the backend arms a setTimeout for its expiry and holds the handle in a per-process map. When it fires, the order is settled immediately. This is what makes a normal contract resolve the instant it expires.

That timer is memory in one Node process. It does not survive:

a deploy or a pnpm restart,
a crash or an OOM kill,
a settlement that outlives the process that owned it.

On boot the backend replays the gap: the Binary Orders startup task loads every PENDING order, settles the ones already expired in batches of ten, and re-arms a timer for the rest. It runs on the main thread only, so under pnpm start:thread one realm owns the timers rather than every worker arming its own duplicate set.

The backstop is a cron job. processPendingOrders runs every 15 seconds and settles any expired order the timers did not.

Lateness is not cosmetic here. Once the expiry minute has rolled over, the AI engine can no longer publish into the candle an order settles against, so a settlement that lands a minute late is silently converted from a steered settlement to a fair one — and the operator sees a refusal reason with no way to tell a deliberate decision from a slow backstop.

The tight cadence is safe because the handler skips orders an in-process timer still owns and takes a Redis lock per order before it settles: it costs one cheap indexed query per tick and cannot double-settle.

The two paths hand over cleanly. The backstop defers to a live timer for the first two minutes past expiry; beyond that it settles the order regardless, because past that window a map entry can only be a leak. A timer that fired and failed drops its entry immediately, so the backstop picks the order up on the very next tick.

Finding the job

Admin → System → System Monitoring → Scheduled Tasks (/admin/system/cron). The job is Process Pending Orders, and it is filed under the normal category — there is no binary category, so do not go looking for one. Opening the screen needs access.cron; the job list and scheduler status behind it are checked against view.cron; running a job by hand needs manage.cron. A role holding only view.cron cannot open the page at all — grant access.cron with it.

If the whole scheduler is not running, nothing on that page is running either — the cron app can be down while the site serves pages perfectly. See Processes and ports for CRON_MODE and how to confirm the scheduler is alive.

pm2 list                                   # is `cron` up?
pm2 logs cron --lines 200 | grep -i "pending orders"
pm2 logs backend --lines 200 | grep -i BINARY

The handler retries three times, five seconds apart, before it reports a failed run — so a single failure in the log is not necessarily an incident, and three in a row is.

What a stuck order usually turns out to be

In the log	What it means
`Binary market {symbol} is disabled`	somebody turned the market off with contracts still open. Turn it back on until they settle — see Binary markets
`Binary market {symbol} not found`	the market row was deleted. Deletion is permanent; the orders have no feed left
`Price unavailable for {symbol}. The market maker for this market is not running.`	an `ECOSYSTEM` market whose AI Market Maker has stopped. Start it; the backstop will settle on the next tick
`No ecosystem price for order {id}`	same cause, seen from the cron side
`Order {id} is being processed by another instance. Skipping.`	normal. Two paths raced and the lock did its job

Settlement never guesses a price. If the feed cannot answer, the order is left PENDING and retried rather than resolved against a made-up number — which is why "nothing happened" is the correct symptom for a dead feed.

PUT /api/admin/finance/order/binary/{id}/status (edit.binary.order) writes the status column and nothing else. It does not compute a profit, does not credit the customer's wallet, and does not release the stake. Flipping a stuck PENDING order to WIN marks it won and pays out nothing, which turns a settlement delay into a support case about missing money.

The admin Binary Orders table (/admin/finance/order/binary, access.binary.order) is deliberately read-only for this reason — no create, no edit, no delete. Fix the feed and let the backstop settle the order.

3. Settings changed and only some customers saw it

Binary settings are cached per process — but in two different caches, on different terms.

Payout percentages, durations and order types are one binarySettings JSON row, held by a dedicated binary cache with a 60-second TTL.
The master switches binaryStatus and binaryPracticeStatus are ordinary settings rows, read through the platform-wide settings cache, which has no TTL at all. That cache reloads only when something empties it, so a master switch that misses its invalidation is stale until the process restarts — not for a minute.

A single-process install never notices either one. Anything with more than one JS realm does: pnpm start:thread, a PM2 cluster, the separate cron process, or a rolling restart whose old worker has not exited. Each realm holds its own copies.

Saving on the Binary Settings screen therefore clears both: it drops the copies in the process that served the write, then announces the change on two Redis channels — cache:invalidate:binary-settings for the payout blob, and cache:invalidate for the settings rows, the master switches among them. That is what makes a payout change platform-wide at once instead of drifting in over a minute, and what makes a master switch reach the other workers at all.

Redis pub/sub has no store-and-forward: a message published into a dead socket is simply gone. The bus notices, logs once under SETTINGS_BUS, and both caches fall back to the clock: the binary cache shortens its TTL from 60 seconds to 5, and the settings cache — which has no TTL to shorten — instead re-reads a version stamp the writer leaves in the settings table, at most once every 5 seconds, and reloads from the database when that stamp has moved. So convergence becomes seconds rather than never, but it is polling, not push.

If a payout change appeared to take effect for some customers and not others, check pm2 logs backend | grep SETTINGS_BUS for that warning around the time you saved. Redis is a hard boot requirement, so this is only reachable when Redis dies after the platform started.

4. Prices are frozen on the trading page

If the chart and the price are stuck but orders do settle, this is not a settlement fault at all — it is the socket.

The binary trading page is the platform's one exception to same-origin WebSockets. Every other feed connects to the page's own origin; the binary order socket builds its URL as:

wss://<hostname>:4000/api/exchange/binary/order

unless NEXT_PUBLIC_WS_URL is set. A reverse proxy that only fronts 3000 and 4000-via-/api never sees that connection, and port 4000 should not be publicly reachable anyway, so the feed simply never opens — on an install where everything else works.

NEXT_PUBLIC_WS_URL="wss://yourdomain.com"

NEXT_PUBLIC_* values are compiled into the browser bundle, so this takes effect only after pnpm build:frontend. Editing .env and restarting PM2 changes nothing. The market and ticker feeds read a different variable, NEXT_PUBLIC_WEBSOCKET_URL — leave that one alone unless you are deliberately terminating sockets elsewhere. Full detail in Nginx and reverse proxy.

A frozen price does not stop settlement: the backend prices the contract from its own feed at expiry, not from anything the browser saw.

The order of operations, condensed

Check the master switch. Binary Settings → Binary Trading. Off means no orders, no navigation entry and no boot sweep.
Open /api/exchange/binary/health in an admin tab. Read system, database and orders; ignore the durations and markets checks and the headline verdict.
Confirm the scheduler is alive. pm2 list, then Admin → System → Scheduled Tasks → Process Pending Orders, in the normal category. A dead cron app removes the only backstop the timers have.
Read the reason. pm2 logs backend --lines 200 | grep -i BINARY. Almost every stuck order names its own cause: a disabled market, a deleted market, or a market maker that is not running.
Fix the feed, do not edit the row. Re-enable the market or restart the market maker, then wait 15 seconds. Settling by hand from the admin pays nobody.
If the page is frozen but orders settle, set NEXT_PUBLIC_WS_URL and rebuild the frontend.

Related: Binary markets and where their prices come from for the two price feeds, Processes and ports for the scheduler, and Troubleshooting for faults that are not specific to binary.