Bicrypto 6.7.6
Latest4 September 2026
This release has upgrade notes. Read them before updating — they describe behaviour changes that need your attention.
Core v6.7.6
Release Date: September 4, 2026 Tags: WALLETS, DEPOSITS, WITHDRAWALS, TRANSFERS, FIAT, SPOT, SECURITY, AUTHENTICATION, DATABASE, PERMISSIONS, ECOSYSTEM, STAKING, RECONCILIATION, BLOG, CONTENT, MOBILE, I18N, PERFORMANCE
Overview
A money-correctness release. An audit read every deposit, withdrawal and transfer path across the four wallet types and found defects that lose customer money, pay the platform's own out twice, or credit a stranger: four fiat gateways that could never credit a deposit, refunds and chargebacks that never reached a wallet on ten gateways, Eco to Futures transfers that stranded it, a reject door that refunded payouts already in flight, and an exchange deposit any observer could claim before the customer who sent it. Ecosystem 6.5.0, Futures 6.2.3, Staking 6.2.3 and the chain addons carry their halves alongside.
Three new bodies of work land with it: pool backing, the ledger, reconciliation and settlement engine for the coins behind a Spot balance; Spot deposit modes, three ways a deposit finds its owner, each requiring an intent the customer declared before the coins moved; and ScyllaDB backup, restore and a record browser for the half of your platform that does not live in MySQL.
It also repairs two everyday doors: signing in and registering with Google — sign-in could not complete on a phone at all, and registration failed on every device — and publishing a blog post or adding a page, which failed outright on installs without the FAQ add-on.
Figures move on their own, records need checking, and two things want a decision — the gap your first reconciliation reports, and which deposit mode you run. It is all in Upgrade Notes; read it before you update.
Update Instructions
pnpm updatorBoth halves matter: the money paths, the schema changes and the boot steps are server-side and need the backend restart; the deposit and withdrawal screens and the transfer form's idempotency nonce live in the browser bundle and need the frontend rebuild.
Restart the backend before you deploy the new web build or ship the app update, not after. The new screens call routes only this backend answers, and the app reads the spot deposit mode from a settings blob only this backend writes. A new build against the old backend fails those calls; the old build against the new backend keeps working.
- Update Ecosystem to 6.5.0, Futures to 6.2.3 and Staking to 6.2.3 in the same window if you run them, along with the Monero, Solana, TON and Tron addons, which each have a release of their own. This release carries the withdrawal guard, the index relaxations, the scheduled jobs and the permission keys those versions depend on.
- Every schema change applies itself on the first boot — six new tables, four
new columns, two ENUM members, two relaxed unique indexes and one new index
(
idx_pool_backing_reconciliation_at, on the reconciliation table'satcolumn, which the stale-exchange check and the console's newest-first page now read over instead of scanning a table nothing prunes) — andinitial.sqlcarries them all for fresh installs. An install that boots withDB_SYNC=noneneeds one boot allowed to sync — that boot creates the new index along with everything else — plus one index step run by hand; see Upgrade Notes. - Twenty-six permission keys are seeded and most ship strict. Two seeders derive four of them from keys your roles already hold; the other twenty-two you grant by hand. Upgrade Notes says which is which.
Upgrade Notes
Refunds and chargebacks your vendors reported before this release were not reversed
Every gateway on the shared reversal engine (Stripe, PayPal, Adyen, Paystack, Mollie, Authorize.Net, eWAY, PayU, Paytm, Paysafe) answered a vendor refund or chargeback with "no wallet" and reversed nothing, silently, since the engine shipped. Reconcile the refunds and chargebacks on your vendor statements against Finance → Transactions and correct any that still matter by hand. dLocal reversals were applied and are unaffected.
Eco to Futures transfers made before this release are stranded, not pending
Look for OUTGOING_TRANSFER rows whose metadata carries
"transferStatus":"PENDING" with status COMPLETED and no matching
INCOMING_TRANSFER row. The customer was debited and nothing was credited.
Credit the destination wallet by hand from Finance → Wallets, or refund the
source. The approval screen cannot settle them, because it accepts PENDING
rows only. (A row that is PENDING with that metadata came from the
documentation seeder, not from a customer.)
Manual spot withdrawal approval works now, so check your queue
Approve on a SPOT withdrawal answered This withdrawal has no currency recorded on it on every row, because it read a field the request never wrote. Operators had only Reject. If your withdraw queue holds spot rows that were parked for that reason, approving them now sends real funds through the exchange; read the row before you click.
Three admin figures were zero on every install and will now show real numbers
The dashboard's daily-trades chart, its period-over-period trade trend and its top-assets table all filtered spot orders on a status that column has never held, so each matched zero rows on every install since they shipped — a flat line, a trend computed against a permanent zero, and an empty table, with no error anywhere. They now read the status the reconciler actually writes. Nothing about your order history has changed; only what the dashboard was asking for.
The ECO transaction migration reported success without ever moving a row
POST /api/admin/system/upgrade/migrate-eco-transactions selected on a
transaction type of ECO. ECO is a WALLET type; the transaction type column
is a forty-four member list that has never contained it. The count was therefore
always zero, so the route answered success: true, updated: 0 with "No ECO
transactions to migrate (already migrated)" while the update it exists to run
stayed unreachable.
If you called that migration and were told there was nothing to do, it did not run. It now selects transactions on ECO wallets. There is no admin screen for it: call the route directly as a Super Admin. It is idempotent, so calling it once after updating is safe, and it reports the real count.
The wallet address index is no longer unique, and the backend relaxes it itself
wallet.addressLookupKey — the hash the withdrawal path uses to find out
whether an address belongs to one of your own customers — was indexed as
UNIQUE. Under the Ecosystem extension's new per-user address model every
currency wallet a customer holds carries the same address on every
Ethereum-style chain, so the same key on several rows is the designed state,
and a unique index refuses the customer's second wallet outright.
The schema sync now relaxes that index to a plain index on the first boot after the update; a database whose index is already plain costs one read. Nothing to run, unless:
- You start the backend with
DB_SYNC=none, which skips the schema sync. Runnode backend/scripts/relax-wallet-address-lookup.mjs(a dry run that reports the index state), then again with--apply. - Fresh installs import a regenerated
initial.sql, which already carries the plain index.
The futures insurance ledger's unique index is relaxed the same way
The Futures extension's ledger gains a sliceKey column and a new unique index
on (positionId, type, sliceKey); the old index on (positionId, type) dropped
every clearing slice after the first of a multi-slice close. The schema sync
relaxes the old index to a plain index on the first boot after the update, by
the same mechanism as the wallet index, and creates the new one. Installs that
boot with DB_SYNC=none must boot once with DB_SYNC=always (or unset, which
syncs when the models changed), or apply the column and index change by hand.
Fresh installs import a regenerated initial.sql.
The engine lease's epoch column must exist before the matching engine will arm
The ledger batcher fences its commits on a new epoch column of
engine_lease, and the Ecosystem matching engine bumps that column under
lock every time it takes the lease, before it accepts anything, whether or
not the batcher is on. A bump that fails refuses the arm, deliberately: the
one case the fence exists for is a bump that waited and lost. On a database
that does not have the column yet, every bump fails, so an install that
boots with DB_SYNC=none has no matching engine after this update until the
column is added: the trading process logs Failed to bump the engine lease epoch; refusing to arm as leader (the lease will retry) on every attempt,
and orders placed in the meantime rest unmatched. The schema sync adds the
column on the first boot after the update, and initial.sql carries it for
fresh installs; nothing to run, unless:
- You start the backend with
DB_SYNC=none. Runnode backend/scripts/add-engine-lease-epoch-column.mjs(a dry run that reports whether the column exists), then again with--apply(INT NOT NULL DEFAULT 0), before you start the trading process. - You run the ledger archive job. Its two tables,
transaction_archiveandwallet_audit_log_archive, are created by the same schema sync; underDB_SYNC=none,node backend/scripts/ledger-archive.mjs --ensure-tablescreates them from the models. Until they exist the job fails its first run and moves nothing, which is the safe direction.
Twenty-six new permission keys, and which of them you grant by hand
The Ecosystem extension gains a settings screen. Its keys —
access.ecosystem.settings, view.ecosystem.settings,
edit.ecosystem.settings — are seeded by this release, and the gates ship
strict: until a role holds them, only a Super Admin can open
Admin → Ecosystem → Settings (/admin/ecosystem/settings). Grant them
under Admin → Users → Roles & Permissions → User Roles to any admin role
that manages the extension.
The Staking extension's on-chain consoles — Chains, Wallets, Validators,
Batches and Incidents — carry eighteen keys of their own (access, view,
edit for each, and create for chains, validators and wallets), seeded
here for the same reason and shipped just as strict. Grant them to the roles
that run that extension; the Staking 6.2.3 notes say what each screen does.
The Pool Backing console (Admin → Finance → Pool Backing) carries three
more, and these are the exception: a seeder derives two of them from keys
you already granted, on every seed run. view.pool.backing (see the page)
goes to every role holding view.deposit. The reconciliation alerts are
addressed more narrowly: to users of the Admin and Super Admin roles
who hold view.pool.backing (a Super Admin always; on a demo install, Super
Admins only) — a custom role that holds the key sees the page but is not
alerted.
edit.pool.backing (run a reconciliation, record an external movement,
settle or convert now, acknowledge drift, attach a hash, mark arrived, set a
per-currency cap) goes to every role holding both edit.deposit and
edit.withdraw — a role that decides customer money in one direction only
does not acquire the other sideways from a console it has never seen.
manage.pool.backing (waive an obligation as recognised loss, mark a
settlement failed) is granted to nobody: a Super Admin holds it by name and
grants it by hand. The seeder only ever inserts — it never revokes, and a
grant you made by hand stays. A role holding neither witness is still
granted by hand; a Super Admin sees the page without any of them.
The Spot Deposit Intents console (Admin → Finance → Spot Deposit
Intents) carries the last two, view.spot.deposit.intent and
edit.spot.deposit.intent, and its seeder derives both the same way and on
the same terms: the first goes to every role holding view.deposit, the
second to every role holding edit.deposit. A role that holds neither
witness is granted by hand.
Anyone watching your exchange deposit address could claim another customer's deposit
The claim route bound a pasted hash to whoever posted it first and checked nothing about ownership; the verifiers then credited that account once the exchange listed the deposit as accepted. Because the deposit address is the same for every customer and public, an observer could paste an incoming deposit's hash ahead of its sender, and the sender was then refused with "Transaction already exists". This was live on every install with a Spot provider.
Every mode now requires an intent that predates the deposit, and a deposit that does not satisfy the intent it was claimed against — wrong amount after the exchange's fee, older than the intent, two intents claiming it — is held for review with the reason, never credited and never silently dropped. The admin deposit queue gained a console for exactly those rows.
Old app builds keep working: a claim that arrives with no intent is accepted, confirmed against the exchange, and then left pending for an admin instead of credited — on the console's Pre-intent claims tab, since it has no intent row. The customer's screen shows "This deposit is being reviewed". Ship the mobile update that reads the mode from the settings blob in the same cycle and this path is never taken.
The first pool-backing reconciliation shows the gap your install has accumulated
The reconciliation compares every Spot wallet's balance with what your
exchange account holds, and until this release nothing did. The first run's
unexplained figure per currency is the sum of every ledger-only Funding →
Spot transfer, every admin credit and every treasury payout since the install
began, less whatever you moved to the exchange by hand. It is a measurement,
not a demand: review it against your exchange statements, then acknowledge it
on the page. Nothing is refused and nothing moves until you set a cap and
switch the mode to manual; the mode after this update is monitor. The Pool
backing page in the operator guide walks through the first run.
Auto mode moves real coins, and refuses to until four things are true
With poolBackingMode on auto the Pool Backing Settlement job nets each
currency's open transfer obligations every five minutes and, when the net is
above the settle threshold, makes one movement: custody coins to the exchange's
deposit address when Spot is short, an exchange withdrawal to the platform's
treasury wallet when it holds more than it owes. It will not plan a movement
for a currency until the latest reconciliation is under an hour old and read
the exchange, the currency's drift is acknowledged — and while a residual sits
outside tolerance but has not yet survived enough runs to be drift, the cron
waits too ("drift not yet confirmed"; Settle now is not held by that gate) —
the chain is mapped to the exchange's network id on the Pool Backing page,
and the pause switch is off.
The platform can send EVM tokens (from its treasury address, any customer
address holding surplus, or the master wallet — the same movers a customer
withdrawal uses), UTXO coins (from the pooled inputs), and — on the chains
whose addon you have updated — native coin, SOL and SPL tokens, TRX and
TRC20, TON (with the memo the exchange asks for) and Monero, from the
treasury's own address first and, for native coin, SOL, TRX and Monero, from
the master wallet as a fallback that never takes it below the reserve you set
per chain in poolBackingMasterReserve. A customer's address is never spent
on those chains. A movement is
marked settled only on the receiving side's own evidence — the exchange
listing the deposit, or the treasury wallet's credited deposit row — never at
broadcast; one it cannot prove either way after seventy-two hours is parked
for review with three doors (attach the hash, mark arrived with proof, mark
failed). Start on manual: the Settle now button runs the same engine for one
currency with you watching, and every movement the engine plans — the cron's
in auto, Settle now and Convert now in manual — is bounded by
poolBackingMaxSettlementUsd (Max Settlement (USD), 10,000 USD until you
change it); a bigger net goes in instalments, one per cycle. Only Record
external and Mark arrived, which record a movement you made by hand, are
unbounded.
The treasury is an account nobody can sign into (pool-backing@treasury.invalid,
no password, the User role). It appears in the user list, holds Funding
wallets only, and its coins are working capital: a customer withdrawal may be
paid from its address like from any other custody address, and the ledger
records that it did.
Switching an addon off no longer abandons what it owes
Disabling an extension in Admin → System → Extensions stops every scheduled job that addon owns, within a minute. For most addons that is exactly what you meant. For eight of them it also stopped the only machinery that gives customers their money back — releasing P2P escrow, refunding ICO contributors when an offering fails, returning a losing NFT bidder's hold, paying a matured investment, claiming unbonded stake and sending it home, re-enqueueing a withdrawal whose row had already debited the wallet. Nothing warned you and nothing errored: the jobs simply disappeared from the cron console, and in six of those addons the same pass removed the alarm that would have reported it.
Two changes, and the first is the one that matters:
-
Eleven jobs now keep running when their addon is switched off, because they settle money that is already owed. Their addon's screens still close — nothing new can start, which is what disabling should mean — and they are registered again after a restart, which is where this was worst: Ecosystem's recovery sweep was gated on the same switch as its routes, so restarting did not pick up a debited withdrawal either.
-
Disabling an addon that still owes customers money now asks first, saying how many and what it means, for Staking, Ecosystem, P2P and Forex Trading. It is a warning, not a lock: repeating the request with
acknowledgeInFlightalways goes through, whatever the number. -
If you genuinely need everything down — the addon's own settlement code is the fault — send
stopEverythingwith the disable. That stops the settlement jobs too, and is cleared automatically when you enable the addon again. -
Nothing changes for an addon you leave enabled, and enabling is never refused.
-
The staking specifics are in Staking v6.2.3.
Fixed
Money doors found by the new build checks
- Fixed — deleting an investment from the admin history screen could pay its
principal back a second time. The door read the row with no lock, graded it,
then credited the principal and wrote
CANCELLEDwith no status predicate. The settlement cron pays principal and return together under one idempotency key and one reference; this door paid the principal again under a different key and a different reference, so neither unique index deduped it. A matured 1,000 deleted at the wrong moment paid out 2,050, and a bulk delete multiplied that per row. It now re-reads the row under a lock and re-grades it there, the way its sibling in the same file already did, so whichever door takes the lock second sees what the first one wrote and pays nothing. - Fixed — the Paystack status poll committed the deposit as final and then credited the wallet, in two separate writes. A credit that threw left the row in a final state with no money behind it, and every other Paystack door refuses a final row — the webhook answers "already processed" and verify answers "already completed" — so nothing could ever retry it. The customer had paid, the balance never moved, and the only trace was one warning line.
- Fixed — the iPay88 and Mollie status polls answered
successto the caller after their own write had failed, reporting the stale local row as a settled status check. Neither could double-credit — the Mollie poll touches no wallet at all — but both told a customer their deposit had been checked when nothing had been recorded. Both now surface the failure. - Fixed — the admin deposit-log approve and reject settled the row with an
instance save and no predicate, and the
PENDINGcheck that admits the request runs outside the transaction. Two operators, or one operator and the bulk-decision route, could both pass it: the approval credited the wallet and booked the platform fee, then the rejection overwrote the status. The customer kept the money, the ledger showed a rejected deposit, and revenue held a fee booked against it. It is now a compare-and-set with a 409, matching its sibling on the withdrawal side. - Fixed — the spot withdrawal rollback flipped a failed payout to
CANCELLEDwith no status predicate while the reconciler settles the same rows against what the exchange actually did. It now compare-and-sets on thePROCESSINGclaim the request itself wrote and, when the reconciler got there first, refuses rather than refunding on top of a settlement. - Fixed — the legacy funding lookup for a binary order filtered on a
transaction type of
BINARY; the value written isBINARY_ORDER. For orders predating the reference fix, cancel and settle could never find the funding row: the money still moved through a direct wallet lookup, but the transaction was never marked cancelled and the link between the order and its ledger row was lost. - Fixed —
updateUserWalletBalancedefaulted its idempotency key to a per-wallet string when a caller omitted one — the exact shape the same file documents at length as a shipped money-loss bug, because the duplicate check matches globally and the second refund on a wallet then becomes a silent no-op after the status has already flipped. Unreached today; removed before it was. - Fixed — two refund helpers with no callers took no transaction and never asked whether a payout had been dispatched, so a future caller would have inherited a refund that autocommits separately from their status flip and pays out a withdrawal an exchange may already have broadcast. Both now refuse and name the hardened door.
- Fixed — the spot approval door and the Authorize.Net verify door each read a metadata key no producer has ever written — one as a fallback behind a working value, one as the first choice with a working fallback behind it — so neither was broken. They are removed because they advertise a shape the platform does not record.
Screens that were reading the wrong thing
- Fixed — the landing page's featured staking pools selected a
currencycolumn that pool table does not have; it issymbol. The select raised "Unknown column" behind a handler that answers an empty list on error, so the landing page has never shown a featured pool. - Fixed — the professional trading terminal resolved its chart provider from a settings key spelled nowhere else in the product. The admin trading screen writes a different key, so selecting Chart Engine under Admin → Trading → Settings changed nothing and the terminal always rendered TradingView.
- Fixed — the tickers feed sent per-symbol unsubscribe frames on sockets that carry one bulk stream with no per-symbol subscription to cancel. The frames did nothing; the local cache drop, which is what callers actually needed, is unchanged and now says so.
Fiat deposits
- Fixed — Authorize.Net, iPay88 and Mollie deposits credit. Each credited
under the exact reference the pending row already held, and
referenceIdis unique, so the ledger row could never be inserted; the error escaped as a 500 (Mollie's webhook then answered OK and Mollie stopped retrying). All three now credit under a derived reference, the same shape Paystack, PayPal and TransFi already used. The iPay88 status poll also looked its row up by a metadata field on a TEXT column and never found it; it now credits through the same path. - Fixed — Paytm deposits credit. The pending row stored the token as its reference, the verify door looked the row up by order id as a primary key and the webhook by order id as the reference; both answered 404. The status poll stamped rows complete without crediting at all, and the verify door credited Paytm's reported amount with no fee while the webhook credited net of it. All three doors now find the row by its order id and credit the recorded amount once.
- Fixed — vendor refunds and chargebacks reverse. The engine resolved the
wallet by
transaction.currency, a column that does not exist, so every reversal ended at "no wallet". It now resolves the wallet the deposit was credited to. A replayed reversal against an emptied wallet no longer records a second phantom receivable, and the status stamp no longer overwrites the row's metadata with an empty document. - Fixed — a Stripe full refund or dispute with a gateway fee configured was treated as partial and refused. Stripe reports the gross charge; the credit row holds the net. The comparison now uses the deposit's gross on either row shape, and the reversal still debits the net and books the fee as a loss.
- Fixed — the Stripe PaymentIntent verification door credited whoever
called it for any succeeded intent. Intents are now created with the
customer's id in their metadata and the door refuses a mismatch; the
Checkout
verifieddoor gained the same ownership check its sibling already had. - Fixed — dLocal refunds and chargebacks go through the shared engine.
With an empty wallet the hand-rolled path stamped the deposit
REFUNDED, booked the fee as a loss and recorded no receivable; the engine takes what is there, records the remainder as a pending receivable and suspends the wallet. A partial refund now debits the portion dLocal states under its own key, so the later full refund is no longer skipped as a duplicate, and the payment amount is never mistaken for the refunded portion. - Fixed — the dLocal verify door filters by the caller, so one customer can no longer force-verify or read another customer's order; the webhook answers an unknown order with an ignored 200 and preserves 4xx codes instead of turning a forged delivery into a retryable 500.
- Fixed — the Authorize.Net, Klarna and Paysafe webhooks verify their signature over the raw request bytes. Signing a re-serialised body failed closed, so those webhooks never verified in production and the verify door was the only working path.
- Fixed — the admin deposit-log update route only accepts deposit rows. A pending withdrawal id sent to it was credited back and stamped complete.
Spot withdrawals and deposits
- Fixed — rejecting a withdrawal from the withdraw-log screen refunded a
payout the exchange was executing. The route checked only
PENDING, and the spot route stored the exchange's own pending asPENDINGwith the exchange's id attached. Reject now refuses, with a 409, any row that carries a dispatch signal, and the status flip is a compare-and-set inside the refund's own transaction. - Fixed — a spot withdrawal accepted by the exchange is stored as
PROCESSINGuntil the reconciler confirms it, neverPENDING. Exchange statuses are normalised through one table: ccxt'scanceled(one L) failed theCANCELLEDcheck, tripped the status validation, and parked the row inPROCESSINGwith the provider id in a metadata field the reconciler never read, leaving the customer debited with no admin exit. - Fixed — the reconciler reads a provider id recorded in metadata as it
reads
referenceId, converges the row, and promotes the id. The rule that a referenced payout the exchange does not list is never auto-refunded is unchanged. - Fixed — manual approval of a spot withdrawal. The route read
currencyandaddressfrom metadata the request never wrote, so every approval answered 400. It now reads the wallet's currency and the recorded destination, network and memo; refuses a row that carries a dispatch signal before claiming it; sends the amount the customer asked for under the chain-fee policy the request was quoted with; refunds exactly the recorded debit if the exchange refuses; leaves a timed-out exchange callPROCESSINGrather than refunding; records the exchange id and on-chain hash; and books the platform fee on success. Fiat approval gained the same compare-and-set claim, so a concurrent approve and reject can no longer both succeed. - Fixed — the
withdrawChainFeesetting was wrong in both positions on exchanges that take their fee out of the submitted amount (Binance, KuCoin, OKX, Bybit). With the platform absorbing the fee, the bare amount was submitted and the customer was short by the fee; with the customer paying, the quoted fee was subtracted and the exchange took its own, so the customer paid it twice. The exchange is now asked for amount plus fee in the first case and the bare amount in the second. - Fixed — an exchange call that times out no longer refunds the customer.
The coins may have left; the row stays
PROCESSINGfor the reconciler. - Fixed — a cancelled spot deposit, or one whose coin did not match, released its transaction hash. The mismatch path soft-deleted the row, which kept the unique hash reserved, so the correct re-submission failed forever; cancel changed the status while the verification kept running and could still credit. The hash is now moved into metadata, the watcher stops, and the credit path re-reads the row under a lock and credits only a row still pending.
Transfers
- Fixed — Eco to Futures and Futures to Eco transfers settle immediately. They were flagged pending, the debit verb wrote the row as completed, the credit was skipped, and the approval screen refused the row. Both routes now do their ledger bookkeeping inline and complete on the spot.
- Fixed — rejecting a legacy pending transfer that carried a platform fee minted the fee: the treasury had been credited at creation and the sender was refunded in full. The reject path now debits the treasury the recorded fee and writes a matching negative revenue row, after the refund and without ever failing it.
- Fixed — account status is enforced on the money path. A suspended or banned account with a live session or an API key could keep transferring, and money could be pushed into a banned account. Sender and recipient must both be active.
Wallet core and admin doors
- Fixed — a duplicate idempotency key on a real race or a replay surfaced as a raw 500. The dialect keys the constraint error by index name and the service matched on the column name, so the duplicate handler never fired; it now recognises the index, and the pre-flight check sees soft-deleted keys. No double-apply was possible either way; the response is now a 409.
- Fixed — the admin wallet Reject route does the status flip and the refund
in one database transaction, with a compare-and-set on the status the screen
showed, and caps the refund at the recorded debit for ECO rows as well. It
refuses rows in
PROCESSINGorTIMEOUTand rows carrying an on-chain hash, a pending hash or a provider reference;TIMEOUTis written by the Solana, TRON and Monero senders precisely when the transaction was broadcast. - Fixed — the admin transaction editor refuses
amountandfeeedits on withdrawals and on deposits whose money has moved. Combined with a refund door that read the edited figure, an inflated amount minted money. - Fixed — the ecosystem chain-transfer verb (used by P2P escrow) locks both tracker rows, creates the receiver's tracker when missing, conserves the per-chain trackers instead of inflating the receiver's, writes the audit row with the ledger row's id rather than the idempotency key, and requires a key.
- Fixed — the ecosystem precision cache no longer marks itself initialised when a loader failed, which pinned every token to eight decimals until a restart; it retries on the next lookup. The private-ledger writer throws instead of reporting success when the ecosystem models are absent.
- Fixed — the AI market maker's pool account (the user row that holds the
maker's futures margin,
ai-market-maker@pool.invalid) was reachable through ordinary admin and customer doors before this release: the CRM user editor could re-home its email, after which the standard password reset set a password and returned a live session; a Super Admin could soft- or hard-delete it, and any admin withedit.usercould ban or block it; an anonymous caller could keep a live email-verification code minted for it (the mail was dropped as undeliverable, the code was not) and the verify route answered a correct guess with its session; any customer could transfer into it by its fixed id;edit.walletcould freeze or delete its wallet at zero balance; and the daily wallet PnL job valued its margin as a customer's equity. Every one of those doors now refuses it by name — see "The platform's own accounts cannot be signed into, taken over or deleted" under Changed.
A bot's orders made every page slow
A wallet's ledger writes serialise on that wallet's row, and every hold, release, credit and debit opened its database transaction — taking a pool connection — before it reached the row. A market maker re-quoting several orders a second on two wallets therefore held a connection for every write waiting its turn: most of the pool, doing nothing. From then on every other query in the process queued behind them — the bot's own authentication, a balance read, the roles list behind every page, the FAQ — and the whole site answered in seconds while the log showed wallet holds taking twenty seconds and "before the first step" taking ten on routes that never touch a wallet. Reproduced by holding two bot wallets' rows for twenty-three seconds under a bot-shaped burst: sixteen connections sat idle inside queued transactions and public routes went from five milliseconds to three hundred, with four-second spikes.
- Fixed — ledger writes wait for their wallet's turn in memory and take a
connection only when they are next. Writes to different wallets still run in
parallel, capped at
WALLET_TX_CONCURRENCYconnections (half the pool by default, never fewer than four) so a burst across many wallets cannot take the pool either; the row lock stays as the guarantee between processes. An operation handed an outer transaction is untouched — the caller owns the ordering, as before. - Fixed — a write that waits longer than
WALLET_QUEUE_TIMEOUT_MS(thirty seconds, the pool's own acquire timeout) is refused withWALLET_BUSY(503) instead of hanging. A bot retries a 5xx; a person sees a message. - Applies to holds, releases, credits, debits, execute-from-hold, transfers and
the ecosystem credit, debit, refund and chain-transfer verbs — and to the
admin balance adjustment, which opens its own transaction around a credit or
debit and now takes the wallet's turn before it takes a connection; an
adjustment that waits longer than
WALLET_QUEUE_TIMEOUT_MSis refused withWALLET_BUSY(503) like any other ledger write. The ecosystem cancel refund, which opens its own transaction around a release, queues the same way — see the Ecosystem 6.5.0 notes.
A burst of orders grew every queue without limit
The serialisation above fixed what a bot's orders did to everyone else. It did not fix what a burst did to the process itself. Every request that arrived was read, authenticated, rate-limited and handed to the wallet gate, where each waiter armed its own thirty-second timer; a hundred thousand orders in one second were a hundred thousand timers that all failed together, thirty seconds late, after each had cost the process the 8 to 9 milliseconds of CPU an accepted order costs. Measured on the reference box: an order that reaches the balance check and fails there costs 670 to 900 microseconds, and the process accepts 50 to 62 place-and-cancel requests per second from bots, so a burst of any size was served at that rate and queued the rest. Nothing refused early, and nothing refused cheaply.
- Fixed with a shed gate in front of both trade doors, the trading screen's
and the Hummingbot bot door's, that decides before the body is read, before
authentication and before any database or Redis call. It reads three things:
the process's own event-loop delay, how many admitted requests each door is
still answering, and a global budget of admissions per second. A refusal is
429 or 503 with
Retry-After, rendered exactly as each door renders a refusal today (the session door keeps HTTP 200 with the code in the body; the bot door answers a real status as-1003or-1001with theX-RateLimit-*headers on the budget refusal), and costs 30 to 40 microseconds. A burst of refused connections yields the event loop every 256 refusals so unrelated routes keep answering through it. Off by default;ECO_ADMISSION_MODE=logcounts and prints without refusing,enforcerefuses. - Fixed with a per-caller cap on open requests, checked before the Trade
budget is charged: a bot with
ECO_INFLIGHT_PER_KEYorders already in flight is refused with 429, its budget untouched and itsX-RateLimit-*headers read without spending, and a signed-in account gets the same cap on the session door. Off by default. - Fixed in the wallet gate: a hold arriving at a wallet that already has
WALLET_QUEUE_MAX_PER_KEYwriters waiting, or at a connection queue alreadyWALLET_QUEUE_MAX_SLOTdeep, is refused withWALLET_BUSY(503) at once, before it queues or arms a timer. Only holds are refused: a release, a cancel refund, a fee credit, a transfer or an admin adjustment keeps the deadline behaviour, so a refund is never turned away and the ledger rows are the ones this release already writes. Both unbounded by default. The refusals are counted on the engine health route (refusedImmediateForKey,refusedImmediateForSlot). - Every bound is off unless set, and with all of them unset the order path is byte for byte the previous release's. The Ecosystem 6.5.0 notes carry the bounds inside the engine (the cancel claim's timeout and the pre-hold check); the Hummingbot Connector's own limiter changes ride on the bot door and are described with the environment variables.
Refused requests were logged as though the server had broken
Every install on the open internet is swept by scanners, and their probe URLs
are exactly what the platform's input filter exists to reject. Those refusals
were being written to the log as server faults — repeating blocks of
Server error : CustomError: Invalid parameter: path with a stack trace
through the framework, most visibly right after a restart, when a sweep already
in progress met a backend that had just come back up. Nothing was wrong with
the server, and nothing was at risk: the requests were being refused, which is
the filter working. What was wrong was that the log said otherwise, and named
neither the URL that had been refused nor the value, so there was no way to
tell a scanner apart from a genuine caller being turned away by mistake.
- Fixed — a refused path or query parameter is answered with its own status
(
400, "Invalid parameter") instead of500 Internal Server Error, and is no longer logged as a server fault with a stack trace. It now records one line naming the method and the URL that was refused, visible whenLOG_LEVELis set todebug. Genuine server faults still log in full. - Fixed — the page-level geographic gate asked the backend about the page the visitor had requested, passing the URL on verbatim, so a probe was refused by that same filter. A refusal reads to the gate as "no answer", and the shortcut that skips the check entirely on an install with geographic restrictions switched off — the shipped default — is only taken after a successful answer. So a sweep put the frontend back to the backend on every page request instead of once every five minutes, and each of those requests was refused in turn and logged. The gate now asks about the part of the URL a country rule can actually match, and caches the answer under that same part, so probe traffic can no longer push real visitors' verdicts out of the cache either.
A testnet Bitcoin install had no usable Bitcoin at all
The platform decides whether a token can be offered by comparing the network
your server is configured for against the network stamped on the token's row.
It normalised one side of that comparison and not the other. Bitcoin's test
network answers to several names — testnet, testnet3, testnet4, signet
and test — and the platform's own chain diagnostics publish all of them as
valid values for BTC_NETWORK, stating plainly that testnet is another name
for testnet4. The configured value was folded down to testnet before the
comparison; the token's own label was compared exactly as written.
So an operator running BTC_NETWORK="testnet4" who imported the Bitcoin token
as testnet4 — a server that agreed with itself, spelled the way the console
told them to spell it — had every Bitcoin row rejected. And because that
label is checked before the exemption that spares chains with their own
service, it was the whole test: Bitcoin vanished from the deposit currency
list with no message, existing wallets stopped showing their Bitcoin address,
and, through Ecosystem, the currency could not be traded either. There was no
value of BTC_NETWORK that could rescue it. Only the row could change, and the
admin token editor cannot change a token's network. Litecoin, Dogecoin and Dash
share the rule and the same fate.
- Fixed — the token's own network label is now resolved the same way the configured value is, so the alternative spellings of one network are treated as one network on both sides. If you have been running a testnet Bitcoin install and wondering where the currency went, it comes back on update with no data change.
- Fixed — a label the chain does not recognise still matches nothing. The
rule that answers "which network does this row claim to describe" resolves
alternative names only; it never falls back to a default the way the reading
of your
.envmust. A Solana row labelledmainnetis still refused whileSOL_NETWORKnames a value Solana's own client lands on devnet, which is the refusal that stops real coins being sent somewhere nothing is watching. - Fixed — a token row carrying no network at all was accepted on any chain the platform has no default mapping for, because two absent values compared equal. Such rows exist: the ecosystem token table on this platform carries rows with empty chain, currency and network fields. They are now refused like any other unrecognised label.
The explanation for a hidden currency named the wrong cause
When the platform refuses a currency it writes a sentence explaining why, and
that sentence is the only evidence an operator gets. It was reading the raw
.env value while the rule it was explaining compared the normalised one — so
on the failure above it checked the labels, found them identical, concluded
they agreed, and went looking for another culprit. It then applied two tests
that the rule itself exempts Bitcoin-style chains from, and blamed both: it
told the operator to set a BTC_TESTNET4_RPC key that nothing in the platform
reads, and then that the chain "defines no testnet4 network", inviting them to
pin a chain identifier that Bitcoin does not have and never will. Both halves
sent them somewhere there was nothing to find.
- Fixed — the explanation now resolves the configured network exactly as the rule does, and skips the two provider tests for the chains that are exempt from them. When the labels genuinely disagree it prints both the value you wrote and the network this server resolves it to, and names the label the row needs — including that the token editor cannot set it, so the change is a re-import or a direct row edit.
Signing in with Google did nothing on a phone
On a mobile browser the "Continue with Google" button sat on "Connecting…" for two minutes and then quietly reset, with no error and no sign-in. A phone only lets a site open a sign-in window while the customer's tap is still live, and this flow asked for that window several steps later — after fetching Google's script over the network and consulting its one-tap prompt first. Safari on iPhone could not use that prompt at all, because it blocks the cookies the prompt depends on. The customer was never told: the message the flow produced when it gave up contained the word "cancelled", and both the sign-in and signup forms treat a cancellation as something not worth interrupting anyone about.
- Fixed — the Google window is now requested on the tap itself, with Google's script loaded in advance, so it opens inside the moment the browser allows it to. The one-tap prompt is no longer part of the button's path at all, which is what made the flow depend on cookies Safari does not allow.
- Fixed — a sign-in that actually succeeded is no longer thrown away. The path a phone took returned its result in a shape the form refused to read, so even a completed sign-in ended as "No credential received from Google".
- Fixed — a blocked or failed attempt now says what happened instead of leaving a dead button. Only a real cancellation stays silent, and the two are no longer told apart by looking for a word in the message.
- Fixed — one failed download of Google's script no longer disables the button for the rest of the visit. It used to leave the page unable to try again until it was reloaded.
- Changed — inside an in-app browser such as Instagram's or Facebook's, which Google refuses to serve at all, the customer is now told to open the page in Safari or Chrome instead of watching a window that cannot work.
- Changed — the Google button is hidden when no Google client id is configured for the site, rather than being offered and failing on every tap.
Two-factor on a Google sign-in announced success and signed nobody in
A customer with two-factor enabled who signed in with Google was shown "Login successful — welcome back", the page reloaded, and they were still signed out. The one-time code had already been sent and was spent by the attempt. The Google door asks for a second factor exactly as the password door does, and the Google forms had no step for it, so they read the challenge as a finished sign-in.
- Fixed — a Google sign-in that needs a second factor now shows the code step, the same one the password sign-in uses, and the code that was sent is the code it accepts.
- Fixed — registering with Google using an address that already has an account with two-factor now hands the customer to the sign-in form instead of announcing a registration that did not happen.
Registering with Google failed on every device
Creating an account with Google answered "Security verification failed. Please try again." and made no account, on desktop as well as mobile. The Google signup door was given the same bot protection the email signup door has, but the Google form never solved the challenge, so there was nothing for the door to check and it refused every attempt.
- Fixed — the Google signup form now solves the security challenge before it submits, so registering with Google completes. It is solved after the Google window closes rather than before it, so it cannot cost the sign-in window the moment it needs to open.
Google sign-in was refused for an account with no surname
Signing in with Google required Google to supply both a first and a last name, and answered "Incomplete user information from Google" without them — for names the sign-in never went on to use. Google omits the family name for accounts whose owner has a single name, which is ordinary in much of the world, so those customers could not get in at all even with an account already linked.
- Fixed — signing in with Google now needs only the identity and the email address Google vouches for. Every other check is unchanged: the account must already be linked, the address must still match, and a banned, suspended or inactive account is refused as before.
- Changed — registering with Google still requires both names, because a customer record without a surname would print half-empty everywhere a name appears, including in mail sent to investors. The refusal now says so and names the two ways forward — add the name to the Google profile, or sign up with an email address — instead of the blanket "Incomplete user information".
Names were altered or rejected when registering with Google
The Google signup door had its own idea of what a name may contain, and it disagreed with the rest of the platform. It deleted hyphens and periods that the customer record keeps, so "Anne-Marie" was stored as "AnneMarie" and "J.R." as "JR". It accepted tabs and line breaks that the record itself rejects, so a name carrying one was approved by the door and then refused at the row, which the customer saw as a server error rather than as anything they could correct. It also capped nothing.
- Fixed — both signup doors now apply one rule, the email door's, which was the correct one: hyphens and periods survive, tabs and line breaks are removed rather than passed on, and an over-long name is shortened instead of being rejected by the database.
- Fixed — a name that is nothing but spaces is refused at the door with a message, rather than stored as a blank.
Publishing a blog post or adding a page failed on installs without the FAQ add-on
Creating or editing a blog post, and adding or editing a page, failed with
(0 , faq_validation_1.sanitizeHTML) is not a function and saved nothing. The
routine these screens use to clean submitted HTML lived inside the FAQ add-on,
which is sold separately and is absent from any install that did not buy it —
so on those platforms six doors could not save at all: creating and editing a
blog post from the admin panel, the same two from the author's own screen, and
creating and editing a page.
- Fixed — the HTML cleaner is now part of the platform itself, so the blog and page screens no longer depend on an add-on being present. The FAQ add-on uses the very same routine, so answers are cleaned exactly as before.
- Fixed — nothing else in the platform's own screens reaches into an add-on this way. A build check now refuses any new one, so this class of failure cannot come back through another screen.
Disabling an addon silently stopped the jobs that pay people
The extension switch performed no check of any kind, and the screen is a plain toggle with no confirmation. Every scheduled job the addon owns stops within the minute — including, for eight addons, the only path that returns customer money. The deregistration is silent by design: the job leaves the cron console rather than reporting an error, so nothing on the platform noticed that an obligation had stopped being settled.
- Fixed the extension switch to leave running the eleven scheduled jobs that settle money customers are already owed, and to register them after a restart even while their addon is off. Disabling now closes the addon's screens without abandoning what it owes.
- Fixed the switch to report what is outstanding before it disables an addon that still owes customers money, rather than doing it silently.
Changed
The Ecosystem deposit screen shows a permanent address
For an Ecosystem token without permit support (NO_PERMIT) on an EVM chain,
the deposit screen used to show a shared platform contract address with a
thirty-minute countdown, and released it when the timer ran out, when the page
was left, or when the customer switched currency or network. Every Ecosystem
token now deposits to the customer's own address, which never changes, so the
screen has no countdown, no "Address valid" tile, and no release call.
- Changed — the countdown panel and the expiry tile are removed from the Ecosystem deposit page; the store no longer starts a countdown for an Ecosystem deposit and its unlock call is gone. The Spot countdown is unchanged.
- Removed — the "All custodial wallets are currently in use. Please try again in a few minutes." message. A failed address fetch now shows the server's own reason.
- Changed — the "Try again" button under an Ecosystem address error used to appear only when that message matched; it now appears for any failed Ecosystem address fetch once a network is chosen.
The withdrawal form shows the network fee it is about to charge
The Ecosystem withdrawal route now charges a token withdrawal on an EVM chain its network fee in the withdrawn token (see the Ecosystem v6.5.0 notes). The form used to show the platform fee and nothing else for those withdrawals.
- Changed — for an Ecosystem wallet the form asks the maximum-withdrawable
endpoint on every network, not only the UTXO chains, and renders its
estimatedNetworkFeeas a "Network fee (estimated, surplus refunded)" row that counts towards the total debited. The "Max" button subtracts the same quote.
The boot sequence prepares the Ecosystem custody settings
When the Ecosystem extension's backend files are present (the step does not consult the extension's enabled flag), the backend materialises its two install-dependent settings on the main thread during boot, right after custom chains load, so the admin menu and the deposit path agree from the first request. See the Ecosystem v6.5.0 notes for what the two settings mean.
- Changed —
spotDepositModeis written ashash_claimon the first boot after the update, on every install, as its own main-thread boot step ahead of the custom chains and outside the Ecosystem step, whose first import fails on an install without the extension — so a Core-only install gets the write too. Until now nothing wrote the key at boot: the public settings blob lacked it until the first customer posted a Spot deposit intent, and the web and mobile deposit screens fell back tohash_claimmeanwhile. The public settings blob therefore carries the mode before the first customer opens the deposit screen; a settings table that cannot be written logs a warning and never stops the boot. - Changed —
wallet.addressLookupKeyis a plain index; the schema sync relaxes a live UNIQUE index automatically (see Upgrade Notes), and the older backfill scriptadd-wallet-address-lookup.mjscreates a plain index too. - Changed — the token form for Ecosystem tokens carries an optional fallback network fee, and the form's type definitions with it.
Two settings become reachable, and thirteen dead keys are gone
- Added — Platform Absorbs the Network Fee (
withdrawChainFee), under Admin → System → Settings → Wallet → Fees. It decides who bears the blockchain network fee on a spot withdrawal, and it has been read by the withdrawal path since it shipped with no control on any screen — so every install has run as "the customer pays" whether or not that was intended, and no admin of any role could change it from any page. The default is unchanged. - Added — Require a signature to unlink a wallet
(
walletDisconnectRequiresSignature), under Security → Wallet Linking. Unlinking a linked address is what frees it to be claimed by someone else, and the route has been able to demand a fresh signature since 6.7.4. The key had no control anywhere, so the protection could only be switched on with a database write. Still off by default. - Removed — twelve shipped defaults that no control could set and nothing read, and one control that wrote a key nothing consulted. The blog's Blog Post Layout select saved a value and the blog rendered identically, which is the worse of the two failures: an operator chose Modern and was shown no reason to doubt it.
- Changed — three rows the pool-backing engine writes for itself in the
settings table —
poolBackingEngineState(the last cycle's memo: what it planned, what it refused and why),poolBackingAttributedThroughandpoolBackingVenueFeesThrough(the nightly jobs' high-water marks) — are withheld from the publicGET /api/settingsblob, as the captcha secret already was, and the settings editor refuses a write to any of them with a 400 that names what the row is, as it already refused the cache version stamp. A hand write to the memo would put words in the engine's mouth on the console; one to a marker would make the next run skip days of attribution or walk them again.
Transfers between Funding and Spot are recorded, and can be capped
- Changed —
POST /api/finance/transferrecords a pool-backing obligation for every ECO ↔ Spot, FIAT ↔ Spot and FIAT ↔ ECO transfer, in the same database transaction as the balance change: what the receiving wallet was credited, signed by direction, attributed to the chains the Funding debit came from. Inmanualandautomode a transfer that would push a currency's open obligations past its cap is refused with a 409 — before any wallet row is locked, so nothing moves and nothing is recorded. The refusal names the currency and what is waiting to be settled, and points the customer at the route they can take — a crypto holder at the Spot deposit flow, a FIAT → Spot customer at the fiat deposit flow (Finance > Deposit > Fiat), since a fiat balance has nothing to deposit to Spot; a transfer refused at the cap loses nothing. The default mode,monitor, refuses nothing. - Changed — an admin balance adjustment on a Spot or Funding wallet records
an
adminobligation keyed to its ledger row, in the same database transaction as the balance change: a ledger row that cannot be written rolls the adjustment back, and a retry lands cleanly. The confirmation email is sent only once both have committed. Approving a Spot deposit by hand asks the exchange first, outside the row locks: a deposit the exchange lists as accepted is recorded settled with that evidence; one it does not list is recorded open. - Changed —
POST /api/finance/deposit/spotrefuses a transaction hash that belongs to a platform settlement, and both spot verifiers (the live socket and the fifteen-minute job) check again after the exchange matches the deposit and before any credit: a settlement's hash, or a sender the exchange reports as a platform-controlled address, releases the claim as FAILED with the reason instead of crediting the claimant — unless the claimant's own Funding withdrawal produced that hash, which is a customer moving their own coins to Spot and is credited as before. The sender check runs only on the tick that would credit, so a pending deposit no longer costs a custody scan every fifteen seconds, and on MariaDB the scan carries its own two-second statement limit. The Funding deposit path refuses to credit a settlement's own change output as a fresh deposit. The exchange deposit address lookup the deposit screen uses was moved, unchanged, intoapi/finance/currency/deposit-address.tsso the engine shares it.
A Spot deposit is claimed against an intent, never against a hash alone
- Changed —
POST /api/finance/deposit/spotbinds the hash to the caller's open intent for that currency and network and refuses more than ten submissions per customer per hour; a claim with no intent is recorded as under review. The wire contract is unchanged. - Changed — the live verification socket and the fifteen-minute job both check the matched deposit against the intent before crediting: exchange status accepted, timestamp no earlier than two minutes before the intent, amount equal to the declared amount net of the exchange's deposit fee. A mismatch holds the row and tells the customer; a match credits as before and closes the intent. The socket now refuses a subscription for a hash that is not the caller's own, so a deposit's owner, email and balance are no longer readable by anyone who knows a public hash; and the thirty-minute stop timer it left behind after every credit is now cleared.
- Changed — the fifteen-minute deposit job also matches open exact-amount intents, expires intents past their window, and runs the Ecosystem sweep housekeeping (the one automatic resweep, adopting a sweep that broadcast without its call-back, replaying a deposit the hook missed).
Spot orders keep the venue's own fee
- Changed — the spot order route and the pending-order job record the
exchange's own fee (
metadata.venueFee: cost and currency) on the order before the platform's fee replaces it in the order's fee fields. Nothing else about the order changes; the console can now show what the venue charged beside what the platform charged.
Extension menus honour their own gates
An extension's navigation tree could declare that an entry needs an extension, a setting, or a setting with a particular value — and the header rendered every entry regardless, because those three gates ran for the core menus only. The Ecosystem console's Custodial Wallets entry, conditioned on the custodial mode, showed in every mode; the Staking extension's two product entries would have shown together.
- Changed the site header to run an extension menu through the same extension, setting and setting-condition gates the core menus always did, before the operator's menu override, which keeps its place as the last transform and can still only remove entries.
The scheduled jobs carried for the Staking extension
- Changed — six scheduled jobs are registered for the Staking extension's on-chain product: signing and broadcasting its batches every two minutes, observing what the network paid every fifteen, reconciling the book against the chain hourly, re-screening validator sets every six hours, raising its time-based alerts every ten minutes, and writing monthly statements daily. Each is a no-op on an install without the extension or with no chain activated. The behaviour is in the Staking 6.2.3 notes.
Transfers and the Bitcoin scanner job
- Changed —
POST /api/finance/transferaccepts an optionalnonce. When present the transfer's idempotency keys derive from it, so a double-submit lands on the same key and is refused as a duplicate instead of moving the funds twice; the transfer form mints one per submission. - Changed — the Bitcoin deposit scanner job carried by Core aggregates a transaction's entries and fetches its full detail in node mode, so every output paying an address is credited and the UTXO rows are recorded. The behaviour and its upgrade note are in the Ecosystem v6.5.0 notes.
The trading screen says which liquidity is the platform's own
The platform's public market-data feed has always excluded the AI market maker's own prints from the trade tape. The trading screen showed them, unlabelled and indistinguishable from a customer's trade, to the people pricing orders against them — so the aggregators and the venue's own customers were being given different answers about what had traded.
- Changed — the order book panel marks a market-maker print on its trade tape, and states what share of the depth on display was posted by the platform's own market maker rather than by customers. The flag on the tape has been on the wire since the feature shipped and nothing read it; the depth figure is new and arrives from the Ecosystem extension. The behaviour and the data behind it are in the Ecosystem v6.5.0 notes, and the market maker itself in the AI Market Maker v6.2.5 notes.
The platform's own accounts cannot be signed into, taken over or deleted
Two user rows belong to the platform, not to a customer: the pool-backing
treasury (pool-backing@treasury.invalid), which owns the settlement reserve's
Funding wallets and their keys, and the AI market maker's pool account
(ai-market-maker@pool.invalid), which holds the maker's futures margin. Both
were built to be un-sign-in-able by construction — no password, an address no
mail can reach, the plain User role — and those properties closed the front
door only. Everything that acted on a row by id treated them as customers.
One shared registry (backend/src/utils/system-accounts.ts) now names them,
and every door asks it first.
- Changed — no authentication door issues them a session: password login, the mobile login, Google, wallet (SIWE), one-time-code completion, the email-verification code, the password-reset confirmation and the token refresh all refuse, with the same 401 a wrong password gets, before any hash is touched. Session minting refuses them as a backstop, and the request middleware refuses a session or API key that names them.
- Changed — password reset answers neutrally and mints no token for them;
email verification mints no code for them or for any
.invalidaddress (the code used to be written before the mailer dropped the message); the verify route answers "Token not found or expired" if a code ever resolves to one. - Changed — the CRM user editor, delete (single and bulk, soft, hard and
restore), status (single and bulk), block, unblock, reset-2FA and
reset-Transfer-PIN doors all refuse them, Super Admins included, with a 403
that names the account. The user model refuses the same writes and deletes
at the row, so a door that forgets is still refused. The CRM list and detail
mark the row (
system: true, a "System account" badge, one line of explanation) and disable the block and reset buttons. - Changed — their wallets cannot be deleted at any balance, frozen or edited by hand; withdrawal approve/reject, the withdraw-log editor and the transfer settlement door refuse rows naming them. Balance adjustment stays open — the pool-backing ledger records it — and the response says so.
- Changed — a customer transfer to them, by the internal transfer route or the ecosystem transfer route, is refused: "it is funded only by the settlement engine, and a balance sent to it by hand would be a liability nothing tracks". The recipient check on the transfer form says the same.
- Changed — registration, Google registration, the admin user editor, the
admin user creator and the CSV import refuse any
.invalidaddress, so a customer cannot hold the row's place before the engine creates it; the two usernames are reserved for the same reason. When a row from before this release already holds the address or handle, the engine answers a 409 naming it and the hard-delete to run, instead of a 500 on every settlement. - Changed — nothing automatic touches them: the mailer drops any recipient
that is a system account or an RFC 2606 domain at the lowest send (the
Mailwizard cron, the FAQ answer mailer and the CSV import used to bypass the
queue-side guard and open real SMTP sessions to
.invalid); the notification service delivers nothing to them on any channel; the daily wallet PnL job skips them; a KYC decision cannot name them; Mailwizard refuses them as campaign targets. A seeder deletes the notification rows the two accounts collected before these guards existed — every fill the AI pool made, every settlement the treasury received — and touches no other table; a second run finds nothing to delete.
Added
Eleven build checks, and the reason they exist
Every defect this release fixes sat on a line the test suite already executed.
Coverage was not the problem and would not have moved. Each check below asks a
question that spans doors rather than describing one, runs a self-test first so
a check that stops finding things fails rather than passes, and reports what it
examined on every run so it cannot go quiet unnoticed. All eleven run in
pnpm gate.
- Added —
check-model-columns: every column named in a query exists on that model. The deposit reversal engine resolved its wallet by a column the transaction table does not have, so no refund or chargeback on any gateway that used the engine ever reached a wallet. - Added —
check-sql-tables: every table named in raw SQL exists. A mocked query never runs the string, so the SQL can say anything. - Added —
check-enum-values: every literal written to or filtered on an enum column is a value that column can hold. A value outside the list makes the query match nothing, silently. - Added —
check-metadata-contract: a transaction metadata key that is read is written by some producer. - Added —
check-settings-keys: a key the backend reads is writable by some screen, and a control an operator can set is read by something. - Added —
check-idempotency-keys: a money key is scoped to its own operation, never to a wallet, a user or an operation type. - Added —
check-refund-atomicity: every door that returns money does it in one transaction, compare-and-sets where it can be raced, and consults the dispatch guard where the payout may already have left. - Added —
check-swallowed-errors: no money path discards its own failure. - Added —
check-ws-contract: every client subscription can be matched by a broadcast on its route. - Added —
check-i18n-keys: every rendered translation key exists, in every catalogue. A missing key renders a machine-made label built from the key — "Set pin" where a sentence belongs. - Added —
check-route-response: the frontend only reads fields its route returns. A missing field isundefined, which renders as blank or zero and never as an error. - Added — a money-invariant suite that drives the real wallet service and asserts three things across every door at once: money is neither created nor destroyed, a replayed operation moves money once, and no signal that a payout left may be followed by a refund. Five of its tests exist to prove the others can fail, by breaking the store on purpose.
- Added —
pnpm mutate, a mutation runner that breaks a file deliberately and reports which breaks the suite does not notice. It proves a green baseline before mutating anything and treats a run with no test summary as a runner fault rather than a kill, because a sweep over a runner that never launched reports every mutant killed and proves nothing. Not a build step: it rewrites source while it runs.
Pool backing: a ledger for every transfer that moved a number and not the coins
- Added — the obligation ledger (
pool_backing_obligation): one signed row per balance change with no physical counterpart — its source (transfer,conversion,fiat_transfer,admin), which pool it describes, and whether a settlement may ever net it. A same-currency transfer is one row; a conversion (Funding USDT → Spot BTC) is two, one per asset, because shipping USDT cannot close a BTC hole. Drift is deliberately not a row: it is a measurement, and a settleable drift would let the platform ship customers' custody coins to back balances it minted itself. - Added — the scheduled job Pool Backing Reconciliation
(
poolBackingReconcile, every fifteen minutes): liabilities per currency — every Spot wallet, the Super Admin's shown separately, plus withdrawals debited but not yet sent — against the exchange's holdings summed over every account type the provider splits its balance across (KuCoin main and trade, Binance spot and funding, OKX trading and funding). One unreadable account makes holdings unknown for the run: no gap, no drift, no alert. A residual that stays outside the currency's precision for three consecutive runs, whichever sign it shows on each run, is recorded as drift, unacknowledged, and the admins are notified. The residual is the gap less the open obligations and less the waived ones, so a waived obligation keeps explaining the gap as recognised loss. Alerts — new drift per currency, a USD gap abovepoolBackingAlertUsdper currency, a settlement parked for review, and the exchange unreadable for more than three runs in a row (the fourth) — go to the Admin and Super Admin users holdingview.pool.backing(a Super Admin always), once per subject per day; new drift once per time it is recorded, so a drift that clears and is recorded again the same day is notified again. - Added — Admin → Finance → Pool Backing: per-currency liabilities,
holdings, gap, open obligations by source, unexplained residual and drift,
with the doors to run a reconciliation, record a movement made outside the
platform (which settles the currency's open obligations of that direction
oldest first and splits the row it only partly covers), acknowledge drift,
and waive an obligation as recognised loss — a reason is required, the
admin who caused an admin credit cannot waive it, and a fresh two-factor
code is always required, whatever the withdrawal 2FA settings say: the
dialog reads the policy, sends or asks for the code, verifies it, and only
then waives (
GET/POST …/obligation/{id}/waive/verification,POST …/waive/verification/verify, thenPOST …/waivewith the token, allmanage.pool.backing). The requirement fails closed: an admin with no enabled second factor, and every admin while two-factor authentication is switched off platform-wide, is refused with a 403 that says so — a write-off of money is never waved through because 2FA happened to be off. A waive posted without the code's token, or with one already used, expired or minted for a different obligation, answers 409; after five wrong codes in ten minutes the next answers 429; a recovery code is accepted in place of the one-time code. The code goes by email or SMS to the admin's own enrolled channel; an authenticator app is asked, not sent. The page shows the waived sum beside the open obligations and the residual's run count toward the drift threshold. - Added — six settings under Wallet → Pool backing:
poolBackingMode(off,monitor,manual,auto),poolBackingCapUsd,poolBackingThresholdUsd,poolBackingAlertUsd,poolBackingDriftRunsandpoolBackingPause. The mode, the cap and the pause are Super Admin only. The mode is materialised asmonitoron first read so the public settings blob carries it. A seventh,poolBackingMaxSettlementUsd(Max Settlement (USD), Super Admin only), bounds every movement the engine plans, whoever pressed the button — the cron inauto, Settle now and Convert now inmanual(default 10,000 USD; empty removes the bound; a bigger net is settled in instalments, one per cycle). Only Record external and Mark arrived, which record movements made by hand, are unbounded.
Pool backing: the settlement engine
- Added — the treasury: a system account (
pool-backing@treasury.invalid, fixed id, no password, the User role) with a Funding wallet per currency the engine touches, created on first need through the same path a customer's Funding wallet takes, so it has real per-chain addresses the deposit monitors, the UTXO pool and the withdrawal movers all see. The master wallet stays the gas payer; the Super Admin's wallets stay the fee revenue. - Added — the scheduled job Pool Backing Settlement
(
poolBackingSettle, every five minutes). It first verifies every movement in flight: an outbound one against the exchange's deposit history by hash (and by address and tag when the exchange reports them); an inbound one against the exchange's withdrawal status and then the treasury wallet's own credited deposit row, registering the treasury address with the background deposit scanner and scanning it itself each tick. Then, inautomode only, it nets each currency's open nettable obligations and dispatches one movement per currency and direction: EVM tokens through the EIP-7702, permit and just-in-time-gas movers from the treasury's address, a customer address holding surplus, or the master wallet; UTXO coins from the pooled inputs with the expected hash reserved before broadcast; exchange withdrawals to the treasury address on the mapped network, moving funds to the account the provider withdraws from first (KuCoin main, OKX funding). A settlement is claimed under the currency's anchor row lock, and one UNIQUE column is the only in-flight lock, so the web process and the cron cannot both send. Fees are booked as recognised loss under the newadminProfittypePOOL_BACKING, on the pool that received less than was sent: the exchange's deposit fee against the Super Admin's Spot wallet, an exchange withdrawal's fee against its Funding wallet on that chain, gas against its Funding wallet — inside the settlement's own transaction, which rolls back when a booking cannot be written, so a movement is never marked settled with an unbooked fee. - Added — the movement's state machine and its crash rule: PLANNED → DISPATCHED → CONFIRMED → SETTLED, or NEEDS_REVIEW with the reason, or FAILED when nothing was ever broadcast (the obligations reopen). A row that had started dispatching when the process died is parked, never resent; an exchange withdrawal the provider answered indeterminately is never re-issued — the engine looks for it by address, amount and time instead. While the pause switch is on nothing is sent and nothing is marked settled; the ledger, the reconciliation and the verification keep running.
- Added — the scheduled job Pool Backing Attribution
(
poolBackingAttribute, daily): for every finished UTC day it derives the Spot balance changes the platform minted itself — investment and AI returns, staking and referral rewards, binary outcomes, commerce, moves into and out of copy-trading and futures, platform fees — asmintedobligation rows per currency and family, signed by the ledger row's own direction, so the reconciliation's residual is explained by category rather than discovered. Rows are informational: never netted, never settled by the engine, excluded from the cap. The exchange's own trading fees are the one figure the ledger cannot derive; they remain in the residual. - Added — on the Pool Backing page: the engine banner (mode, pause, the
last cycle's refusals by currency), the treasury card (addresses per chain,
what each can send or receive on the mapped network, and what is wrong when
it cannot), the settlements table with the hash and proof of each movement
and the review doors, and Settle now per currency in
manualandauto. Routes under/api/admin/finance/pool-backing/:settlement(list),settlement/{id}/attach-txid(which refuses a conversion with a 400: a market order on the venue has no chain hash, and attaching one re-took the currency's in-flight key for the conversion direction behind a row the verifier could never confirm — mark it arrived with the order's fill, or mark it failed),settlement/{id}/mark-arrived,settlement/{id}/mark-failed(manage.pool.backing; a row carrying a hash, an exchange withdrawal the provider has queued, or a dispatch that had started, needs an explicit confirmation that nothing reached the chain or the exchange, because FAILED reopens the obligations and releases the hash from the guard; a row that is mid-dispatch cannot be failed at all),settle,convert(Convert now,edit.pool.backing) andtreasury, which also lists anypoolBackingMasterReservekey that names no ecosystem chain so the card can flag it. No new permission keys.
Pool backing: the native chains, the ecosystem side, parked principals, conversions and venue fees
- Added — native-chain settlements: the engine now sends native EVM coin,
SOL and SPL tokens, TRX and TRC20, TON with a memo, and Monero, through
row-free cores the chain addons carve out of their withdrawal handlers
(the handlers' behaviour is unchanged and pinned). Source order per chain:
the treasury's own address, then the master wallet where it can hold the
asset (native coin, SOL, TRX, Monero) — never below the per-chain reserve
poolBackingMasterReserve— and never a customer's address. A TON hash is found by its payload after the send; a TRON hash is reserved from the signed transaction; Monero relays once and confirms on the exchange side. - Added — the ecosystem side of the reconciliation: for every currency
and chain with a treasury address or an open Funding-side row, what the
Funding wallets say sits at customers' addresses (Le; the treasury's own
figure and the unattributed remainder are shown apart and are not in it, so
a treasury reserve reads as a surplus, never as a hole) against what the
chain says
(He): the treasury, the masters and the custodial contracts read every
run, customer addresses in a rotating slice of
poolBackingCustodyReadsPerRunper run under the deposit scanner's own per-chain rate limits, every figure cached inpool_backing_custody_readwith its read time and error. He is a number only when every address has a figure this run or from the cache (ok),partialwhen some never have,unknownwhen the treasury or a master could not be read; an RPC failure is an error, never a zero. UTXO customers are read from the platform's own unspent pool; Monero customers from the wallet mirror, labelled so. Stored per chain inecosystemSplit, shown on the page as the Ecosystem side card. The Funding-side residual is shown, not persisted as drift; open and waived Funding-side obligations explain it as they do on the exchange side, andecosystemSplitcarries the waived sum beside the open one per chain. - Added — principals parked in parallel stores count as Spot
liabilities: copy-trading binary allocations, active investments and AI
investments, synthetic staking positions, live forex accounts and their
investments, and fx-trading balances derived from the ledger — each in the
Spot currency it left, each 0 with a note when its extension is not
installed, shown as the Parked principals line with its breakdown. The
nightly attribution no longer writes
mintedrows for those moves and cancels the ones it wrote, so the same money is not explained twice. - Added — conversion legs, behind the switch
poolBackingAutoConvert(off by default; Super Admin): once the Funding leg of a Funding USDT → Spot BTC transfer is settled, the exchange owes BTC it never received; the engine buys it on the venue — theC2/C1market when it exists, elseC2/USDT— sized in the base asset to the venue's own precision and minimums, capped by the max automatic settlement, funds moved into the trading account first on KuCoin and OKX, one order per currency in flight (exchange_convert), the fill read back and the venue's fee booked as recognised loss; an order not filled within thirty minutes is cancelled and parked for review with what filled. Convert now on the page runs it for one currency inmanual. - Added — venue-fee attribution in the daily job: the exchange's own
trading fees are read from its trade history for every symbol the platform
traded since the last run and written as
exchange_feerows, one per trade, each booked as recognised loss against the Super Admin's Spot wallet in the fee's asset — settled for the part the wallet covered, open for the remainder — so the last unexplained figure of a healthy install is explained too. - Added — three settings under Wallet → Pool backing:
poolBackingAutoConvert,poolBackingCustodyReadsPerRunandpoolBackingMasterReserve(a JSON map of chain to native units); the first and the last are Super Admin only.
Spot deposit modes: three ways a deposit finds its owner
- Added — the setting
spotDepositMode(hash_claim,amount_match,ecosystem_custody; Super Admin only;hash_claimafter this update) and the tablespot_deposit_intent: one row per declared deposit with the mode it runs under, the declared and the exact amount, the address shown, a sixty-minute window and a seven-day matching tail. At most three open intents per customer, one per currency and network. - Added —
POST /api/finance/deposit/spot/intent(declare; returns the mode, the address, the tag, the exact amount to send and the send-by time),GET …/intent(the caller's intents in flight — OPEN, MATCHED and SWEEPING — with the current mode),POST …/intent/{id}/cancel, and the intent stream/api/finance/deposit/spot/intent(subscribe with{ intentId }; frames carry the stage — waiting, received, moving, on the exchange, credited, review, failed, expired). - Added — the exact-amount matcher: the declared amount plus a nudge in the last decimals of the network's precision, unique for the currency across every network while its window can still deliver, never handed to a second customer until then; one exchange poll per currency per tick shared by every open intent; a deposit that would satisfy two intents credits nobody and holds both.
- Added — the customer's-own-address mode, in the Ecosystem extension (its notes describe the sweep). Core's part: the platform claims the sweep's hash on the exchange for the customer, the verifiers credit it as any deposit, the pool-backing guard lets it through because the customer's own withdrawal row vouches for the hash, and the deposit-expiry setting does not time a sweep out while the exchange is still confirming it. A sweep the chain mined but the exchange has not listed by the end of the seven-day matching window is parked for review naming the hash and where the coins are (on the exchange, under the platform's account); the customer is told once ("Your … deposit was moved to the exchange (transaction …) but the exchange has not listed it after N days. Our team is reviewing it; nothing has been lost."), the claim row in Deposit Records carries the same reason, and the fifteen-minute job's housekeeping line counts them as unlisted sweeps parked for review.
- Added — Admin → Finance → Spot Deposit Intents: the queue of
intents in review or failed, what is in flight, the mode and stage of each,
the exchange's evidence for the hash it carries, where the customer's money
is, and the doors approve (credits through the deposit log's locked path
and records the pool-backing evidence), reject (releases the hash; refused
while a sweep is debited but not yet on chain) and resweep (refused when
the previous sweep reached the chain); and a Pre-intent claims tab
(
GET …/spot-deposit-intent/pre-intent) for the claims pasted the old way with no intent behind them, each with the exchange's own answer for its hash — approve and reject act on the claim itself — reject releases the hash and tells the customer why ("Deposit not credited", with your reason), refuses a claim already credited or released, and sends a deposit row that carries no pre-intent flag back to Deposit Records — and the badge counts both queues. Permissionsview.spot.deposit.intentandedit.spot.deposit.intent. - Added — the web deposit flow's declare step, the exact-amount and own- address screens, and the stage panel driven by the intent stream; the hash field stays for the hash-claim mode and as the fallback under exact-amount match.
- Added — the mobile app's three flows read the mode from the settings
blob; its verification socket now sends the
SUBSCRIBEaction the broker requires, which it never had, so the app receives the credit it used to poll for.
Translations
-
Fixed — forty-six interface strings existed in English only and in none of the other eighty-nine catalogues, so those customers saw a machine-made label built from the key where a sentence belongs. The worst of them sat on the wallet transfer confirmation step: "Set pin", "Current pin", "Or", in English, mid page, while approving a money movement. All forty-six are seeded into all catalogues in English until translated, which is this platform's convention.
-
Added —
backend/scripts/relax-wallet-address-lookup.mjs: the standalone, idempotent form of the wallet index relaxation, dry-run by default. -
Added — permission keys
access.ecosystem.settings,view.ecosystem.settingsandedit.ecosystem.settings; the route manifest gates/admin/ecosystem/settingson the first. -
Added — permission keys
view.pool.backing,edit.pool.backingandmanage.pool.backing; the route manifest gates/admin/finance/pool-backingon the first. -
Added — permission keys
view.spot.deposit.intentandedit.spot.deposit.intent; the route manifest gates/admin/finance/spot-deposit-intenton the first. -
Added — eighteen permission keys for the Staking extension's on-chain consoles (
staking.chain,staking.wallet,staking.validator,staking.batch,staking.incident, each withaccess,viewandedit, andcreatefor chains, validators and wallets); the route manifest gates each console on itsaccesskey and the new Compliance console on the existingaccess.staking.settings. -
Added — the module manifest the mobile app reads now reports which staking product the platform sells and requires the Ecosystem extension only for the on-chain one.
-
Added —
backend/src/api/finance/withdraw/exchange-status.ts: the exchange status table and chain-fee arithmetic the spot route, the approve route and the reconciler now share. -
Added — thirty-four interface strings: twenty for the new Ecosystem Settings screen, thirteen for the sweep action and drain notice on the existing Custodial Wallets page and the network-fee fallback field on the token forms, and one for the withdrawal form's network fee row. All are seeded into all ninety language catalogues in English until translated. The Custodial Wallets menu description was reworded in the English catalogue only.
-
Added — sixty-nine interface strings for the Pool Backing console and its menu entry, seventy-two more for its settlement engine, treasury card and review doors, seventy-two more for the ecosystem side, the parked principals, conversions, the obligation source filter and the review doors' conversion confirmation, and nineteen more for the waive's two-factor step, the waived line, the residual's run count and the master-reserve notice, seeded into all ninety catalogues in English until translated.
-
Added — one hundred and twenty-seven interface strings for the Spot deposit modes: thirty-nine for the customer's deposit flow and eighty-eight for the Spot Deposit Intents console, eight of those for its pre-intent claims tab, seeded into all ninety catalogues in English until translated.
-
Added — sixteen interface strings: fourteen for the AI Market Maker's reference-venue picker and its new Mirror price mode, and two for the trade tape's market-maker marker and the order book's depth notice. All are seeded into all ninety catalogues in English until translated.
-
Added — seven interface strings for the AI Market Maker's pool funding screens, which now name the platform's wallet (the Super Admin's) as where every pool deposit is taken from and every withdrawal and market deletion pays back into, whoever operates the market; the behaviour is in the AI Market Maker v6.2.5 notes. Seeded into all ninety catalogues in English until translated.
-
Changed — the Staking extension's interface strings that described the fixed-rate product were rewritten in English to say who pays its rate, and the rewritten values were seeded into every catalogue in English until translated, replacing translations of the old sentences. Which strings, and why, is in the Staking 6.2.3 notes.
A script that names the stop orders still at risk
Ecosystem 6.5.0 fixes a stop order whose table was written by two clocks. With
the database's clock ahead, a stop could be left reading CANCELLING or
TRIGGERING with its reservation already released and the money right. With the
backend's clock ahead it went the other way: a cancel could be told it had
applied while the row still read PENDING, releasing the reservation and
leaving the stop armed against money nobody holds. Rows written before the fix
keep the stamp they were written with, and it is the second case they can still
reproduce. This script is the aftermath.
- Added
node backend/scripts/stop-order-doctor.mjs: reads every non-terminal stop, compares each row's write stamp with the database's clock rather than this machine's, and prints which stops can still refuse a terminal write and when each stops being at risk. It writes nothing; the Ecosystem note says what to do about each one.
The slow-request line says what the process was doing
The four slowest steps of a slow request could not tell a slow step from a slow process: "Updating wallet balance 18s" reads as a wallet problem when the whole process was starved. The report now ends with three figures read at the moment it prints, and the troubleshooting guide explains each.
- Added —
event loop stalled Nms of this request: how long the process was frozen while the request was open, from a 50ms heartbeat kept for the life of the process. Large means nothing this request did was the cause. - Added —
db pool U/S in use of max M, W waiting:waitingabove zero onceShas reachedMmeans the pool is the bottleneck, not the query. - Added —
wallet queue A active (cap C), Q queued: ledger writes waiting for their wallet's turn. A long queue is a bot re-quoting faster than its wallet row can commit. - Added —
WALLET_TX_CONCURRENCYandWALLET_QUEUE_TIMEOUT_MS, in the environment reference; neither needs a value.
ScyllaDB can be backed up, restored and inspected from the admin
If you run the Ecosystem extension, ScyllaDB holds your order books, open and historical orders, stop orders, trades, candles, price history and the AI market maker's own tables — sixteen tables across two keyspaces. The Database screen backed up MySQL and nothing else, so a "database backup" covered half the platform, and there was no way to look at a single ScyllaDB row without a CQL client and a shell on the node.
A new System → Database → ScyllaDB screen covers both. It only appears
where the Ecosystem extension is installed, and it carries the same
access.database permission the MySQL backup screen does — a role that can
back up one can back up the other, and no new key is needed.
- Added — Snapshots. One button writes every row of every base table,
plus the schema, to a single compressed file under
backup/scylla. The snapshot is written under a temporary name and only appears in the list once it is complete, so a backup interrupted by a dropped connection or a full disk can never be restored from. Ten are kept;SCYLLA_BACKUP_RETAINchanges that. Snapshots are listed with their size, their table count and the keyspaces they cover, and one that cannot be read is marked rather than offered. - Added — Restore, in two modes, because only one of them is safe to offer by default. Merge writes the snapshot's rows over what is there now and deletes nothing, so anything created since the snapshot survives. Replace empties every table in the snapshot first and reproduces it exactly, losing everything written since. Merge is preselected; Replace has to be chosen deliberately and is coloured as the destructive action it is.
- Added — The restore tells you when it is not finished. The matching engine keeps the order book in memory. Restoring orders or order books underneath a running engine leaves it serving — and writing back — the book it had before, which would quietly undo the restore. The screen says so before you confirm, and the result says so again when the restore touched those tables: restart the backend before trading resumes.
- Added — A record browser. Pick a table, page through it, and inspect, edit or delete a row. Rows are paged by ScyllaDB's own cursor rather than by an offset, so the tenth page costs what the first one did.
- Added — Filtering is restricted to the partition key, and says why.
A filter on any other column would need
ALLOW FILTERING, which on a table the size of your orders table is a scan of the whole cluster and can take the node down. The screen names the columns it will accept instead of offering a filter that works on a quiet install and stalls a busy one. - Added — The editor refuses the two edits that corrupt data rather than change it. In CQL an update is an upsert: updating three columns of a row that is not there does not fail, it creates the row with those three columns and nothing else — a half-record the matching engine will later reject. Every edit therefore reads the row back under its full primary key first and refuses if it is absent. A primary-key column cannot be edited at all, and a delete requires the complete primary key: a delete with only the partition key is valid CQL and would remove every order a customer has ever placed.
- Added — Materialized views are left to rebuild themselves. The twelve views over these tables are derived, CQL forbids writing to one, and a restore of the base tables regenerates all of them. They are not stored in the snapshot and are not listed in the browser.
- Added — Backups and restores are recorded in the audit trail, with
the row an edit or delete touched, under
ADMIN_SYS. - Added — thirty interface strings for the new screen, and six for the new System → Database menu group and its MySQL Backup and ScyllaDB entries, seeded into all ninety catalogues in English until translated.
The snapshot is a logical export taken over the normal database connection, not a filesystem snapshot, so it works whether ScyllaDB runs on the same machine as the platform, on another host, or as a managed cluster — none of which give the platform a shell on the node. Measured on a development cluster, just under 60,000 rows exported in four seconds to a 2.7 MiB file.
The MySQL backup screen is now in the menu
The Database Backup screen existed but appeared nowhere in the navigation — it
was reachable only by typing its URL. System → Database is a new group
holding both it and the ScyllaDB screen, gated on the same access.database
key both pages already required.
Admission control on the trade doors
The bounds above are all environment variables, none of them needs a value, and each is described with the number it was set from in the environment reference. The monitoring guide gained the probe that reads them and the troubleshooting guide two entries, "Clients see 503 with Retry-After on the trade doors" and "Hummingbot bots see -1001".
- Added
ECO_ADMISSION_MODE,ECO_ADMIT_PER_SEC,ECO_ADMIT_REFUSALS_PER_TURN,ECO_ADMIT_LOOP_DELAY_MSandECO_ADMIT_INFLIGHT_PER_DOOR: the shed gate. Inlogmode the backend prints one[ECO_ADMISSION]line per second, only in a second where something would have been refused, with the accepted rate, the open requests per door, the budget's remaining tokens and the event-loop delay of that second; that line is where the budget for your box comes from. - Added
ECO_INFLIGHT_PER_KEY, the per-caller cap on open requests, andWALLET_QUEUE_MAX_PER_KEYandWALLET_QUEUE_MAX_SLOT, the wallet gate's immediate refusals, with two matching counters onGET /api/admin/ecosystem/engine/health. - Added
GET /api/admin/ecosystem/engine/healthto the monitoring guide: the wallet gate, the event-loop delay histogram, the pool and the engine's resident orders for the one process that answered, behindaccess.ecosystem. - Changed — the shed gate is part of the Ecosystem add-on, and the two
doors it guards are Ecosystem's own trade door and the Hummingbot Connector's
bot door. A platform running without the Ecosystem add-on admits every request
at both as it always did — including one that owns the Hummingbot Connector on
its own — and the shed gate's five variables and the wallet gate's two bounds
have no effect there.
ECO_INFLIGHT_PER_KEYis the exception: the per-caller cap is part of the platform rather than the add-on and fronts every door the platform throttles as a trading door, the DEX swap and forex order doors among them, so an install without Ecosystem that sets it does refuse a caller's extra open requests with 429. Like the rest, it stays off until you set it.
The engine lease knows a door, and the maker follows it
Ecosystem 6.5.0's shard tier runs the engine as shard processes behind door processes that forward by symbol. Two rules for that live in the core and are inert without it.
- Changed — a process with
ECO_DOORon is refused the single-processecosystem-matchinglease outright when the tier owns the whole venue, and stays a candidate whenECO_SHARD_SYMBOLS, a shadow list or the signed shard map leaves some markets its own, so a per-market canary still has a matcher for everything the shard does not hold.ECO_DOOR=0is not a door and stays a candidate, which is the rollback path. Read from the environment, never from the runtime flag, so a settings blip cannot let a door arm a second engine. - Changed — the AI Market Maker's supervisor arms on a door as it does on
the matching leaseholder, and the maker's own lease candidacy no longer
derives from the matcher's. With AI Market Maker 6.2.5, which carries the
maker's own half of this rule, a door quotes; on an earlier maker a door
stays idle. Futures and forex leases are untouched by
ECO_DOOR.
A dedicated trading process, as an opt-in
The bounds above keep a burst from growing the queues of the process that serves it. They do not change the fact that the process serving the burst is the one serving the login form, the admin screens and every unrelated socket: measured on the reference box, a ten-times trade burst moved an unrelated GET's p99 by 5.8x to 32.7x on the single process. The engine can now have a process of its own. Nothing moves unless you ask for it.
- Added
ECO_TRADING_ENABLED, the one variable you set in.env. With it on,production.config.jsstarts a fourth PM2 app,trading, the same backend entry point onECO_TRADING_PORT(4010), and assigns every app the role it runs with — there is nothing else for you to set, and the scheduler goes on running every job exactly as it does today.production.backend.config.jsbecomesbackend(web),tradingandcronunder the same opt-in, because an inline process would race the trading app for the engine.production.thread.config.jsrefuses it: worker threads run the matcher unarbitrated. Unset, all three configs produce exactly the apps they produced before. - Added
ECO_PROCESS_ROLE, the engine-hosting axis of a backend process, set per app by the PM2 configs and never in.env.tradingis the only lease candidate for the Ecosystem matching engine once the split is on;webis never one, even with every trading process down. A process that finds the variable withCRON_MODEnotoff, or with a value it does not know, refuses to boot with one stderr line and exit code 78, the code PM2 stops on rather than restart-looping, because a scheduling process finding it means it leaked, and a misspelttradingthat booted as a web process would leave nothing hosting the matcher. Unset keeps today's rule, so the default layout is unchanged and unsetting the opt-in is the whole rollback. - Added
ECO_TRADING_PORT, pinned to the trading app'sPORTandNEXT_PUBLIC_BACKEND_PORTby both configs and refused when it collides with the web or cron port. The trading app is givenDB_POOL_MAX=40,WALLET_TX_CONCURRENCY=16andSCYLLA_LOCAL_CONNECTIONS=8as defaults a value in.envstill overrides. - Changed the engine health route reports
tradingas arole, and the log prefix of a trading process readsTRAD. - The split is a proxy property, not a code path: four prefixes
(
/api/ecosystem/order,/api/ecosystem/market,/api/ecosystem/tickerand/api/hb/) must be sent to the trading port by the reverse proxy. A trading path that reaches the web process by mistake still places and cancels through the addon's cross-process path, but its order-book, ticker and order streams stay silent, because the engine pushes those frames to the sockets of its own process. The nginx and Apache guides carry the blocks; the Ecosystem 6.5.0 notes and the Ecosystem operations page carry the lease rule and the failover expectation.pnpm stopandpnpm restartdo not yet own thetradingapp; stop it withpm2 stop tradingalongside.
WebSocket ingress limits
Nothing bounded what a client could send the WebSocket server or how much the server would buffer for a client that had stopped reading: the socket server's own defaults applied, the market feed answered every SUBSCRIBE with a database read, and a slow consumer had every book snapshot built and queued for it regardless.
- Added
WS_MAX_PAYLOAD_BYTES,WS_IDLE_TIMEOUT_S,WS_MAX_BACKPRESSURE_BYTESandWS_CLOSE_ON_BACKPRESSURE, the socket server's four limits, now set explicitly on every route from the environment. The defaults are the server's own, so nothing changes until you set one. - Added
WS_MSGS_PER_SEC, a per-socket inbound budget on every route, counted before the frame is decoded; the socket past it is closed with code 1008.WS_MAX_SUBSCRIPTIONScaps the subscriptions one socket may hold, refusing a new one with the subscription error frame the platform already sends. Both unlimited by default; the Hummingbot stream keeps its own two. - Added a slow-consumer skip in the broker: a socket whose buffered bytes
are past
WS_MAX_BACKPRESSURE_BYTESis skipped for snapshot frames (the market feed's book, ticker, candles and recent trades) before the frame is built for it, while event frames, which carry a fact the client must not miss, are still handed to the server. The market feed is the only route that marks its frames as snapshots; every other route's frames are events. Sends the server reports as dropped are counted. - Added
WS_MARKET_CACHE_MS, so the market feed answers "is this an enabled market" from one list refreshed on that interval instead of one read per SUBSCRIBE. Off by default; see the Ecosystem 6.5.0 notes.
The ledger batcher: one commit per tick
Every hold, release and fill leg opened a MySQL transaction of its own and
paid a commit for it, and on the order path that commit was the floor under
every placement. A batcher now exists in the wallet service that gathers
ledger operations for a few milliseconds and commits them together, writing
exactly the rows the verbs write, with the same keys, amounts, types and
descriptions; the Ecosystem extension is what routes holds and fills through
it, behind ECO_LEDGER_BATCHER, and its notes carry the modes. Off, nothing
in this release touches the verbs.
- Added
backend/src/services/wallet/batcher.ts: a queue of ledger operations grouped by the request that produced them, an adaptive tick (ECO_LEDGER_TICK_MS, the minimum, 20 ms; a tick lasts as long as its commit), and a fixed shape per tick: one bulk idempotency pre-check over every key in the batch, the wallet rows locked in id order, the lease row, oneCASEupdate over the wallets, one multi-row insert of the transaction rows and the audit rows. A group is applied whole or not at all; a batch that hits a deadlock, a lock-wait timeout or a lost connection is retried three times with the same keys, so a retry after a commit that did land writes nothing. A suite compares every row the batcher writes against the rows the verbs write for the same operations, byte for byte. - Added an
epochcolumn onengine_lease, the fence the batcher checks under lock inside every tick. The Ecosystem leader bumps it by one each time it takes the lease, before it accepts anything, so a deposed leader's next tick aborts withEPOCH_MISMATCHand its batcher refuses every operation until the process re-arms; a tick that already held the row lock finishes with the epoch it read, which is the old leader's legitimate last work. The cancel-all route reads the same epoch once per run and compares it inside each of its batches. The column is added by the schema sync; installs that boot withDB_SYNC=nonerunnode backend/scripts/add-engine-lease-epoch-column.mjs --apply(see Upgrade Notes). - Changed
GET /api/admin/ecosystem/engine/healthto report the batcher (ledgerBatcher: the configured and effective mode, pending operations, whether it is fenced and at which epoch, and its tick metrics), the fee journal when the leader has one open, and every engine flag as configured and as effective (scaleFlags). The monitoring guide describes the three blocks and which of the figures are alerts.
Ledger archiving
The two ledger tables gain a row for every hold, release, fill leg and fee and nothing ever removed one, so on a busy exchange it is their size, not their write rate, that eventually limits the database. Off by default.
- Added
ledgerArchive, an hourly core job on the cron process, and the four variables that drive it:ECO_LEDGER_ARCHIVE_ENABLED(off),ECO_LEDGER_ARCHIVE_AFTER_DAYS(400),ECO_LEDGER_ARCHIVE_BATCH(1,000) andECO_LEDGER_ARCHIVE_MAX_PER_RUN(100,000). Atransactionrow older than the window whose status isCOMPLETED,CANCELLEDorFAILEDand that no foreign key points at is copied totransaction_archivewith itswallet_audit_logrows towallet_audit_log_archive, then hard-deleted, one MySQL transaction per batch and the live row deleted only after the archive has been read back holding it.PENDING,PROCESSINGandFROZENrows never move; neither does a fee credit, becauseadmin_profitreferences it with a cascading key, so the archive alone does not bound the table's growth until that table has a retention rule of its own. - Added the two archive tables, the same columns as the live ones with a
primary key and a
createdAtindex, created by the schema sync on the first boot after the update or by the manual script's--ensure-tables. - Added
backend/scripts/ledger-archive.mjs, the same job body by hand: a dry run by default that reports what would move and why the rest would not,--applyto move it,--after-days,--batch,--maxand--json. - Changed
scripts/ledger-conservation.mjsto read the live and archive tables as one ledger whenever both archive tables exist, so its verdict is the same before and after an archive run; a lone archive table is reported and the live tables are read alone. - The backup guide gained a section on what moves, when, how to run it by hand and how to put a row back; the scheduled-jobs reference lists the job.