Troubleshooting
The staking failures that look like nothing is wrong — silent withdrawal queues, rewards that never accrue, fees that vanish, cron rows that read refused, and kill switches nobody remembers closing.
Most staking problems are silent. Nothing errors, no screen turns red, and the first signal is a support ticket. This page is ordered by how often that happens.
The two products fail differently. Everything down to the on-chain sections
below is the fixed-rate product — a typed APR, a term, an operator
treasury — and none of it applies to a pool whose mode is REAL: an
on-chain pool has no APR, no capacity, no lock period and no withdrawal
approval queue. The on-chain sections are the second half of this page. Which
product a pool belongs to is on the row, not on the global switch; see
Staking settings.
Backend log modules to filter on: STAKING for the user endpoints and
every staking cron, on-chain included, ADMIN_STAKE for the admin routes.
First checks — fixed-rate
- Awaiting your decision - count and oldest request age
- The withdrawal queue, oldest first
Before anything specific, confirm these four:
- The Staking extension row is active at
/admin/system/extension - Process Staking Logs appears at
/admin/system/cronand has a recent run - A Super Admin user exists, and it is not the account you are testing with
- The Overview screen at
/admin/stakingloads with real figures rather than the "figures unavailable" state
If the Overview says figures are unavailable, the aggregation query failed. That
is deliberately not a 500 — the page keeps its header and Retry button — but the
underlying error is in the log under ADMIN_STAKE.
First checks — on-chain
Process Staking Logs is the fixed-rate job and it is the wrong one to look at here. It never touches an on-chain position. On-chain staking runs on six separate tasks, and the one that moves coins is On-chain staking: batches, every two minutes.
-
/admin/staking/settingsreports on-chain staking as available. Five things must hold — the Ecosystem extension and the Web3 Trading addon each installed, enabled and licensed, the ecosystem key vault unlocked, and at least one chainACTIVE. The screen names whichever one is missing rather than saying "not available" -
stakingModeisREAL, or the platform is refusing new stakes on every on-chain pool with "This pool belongs to on-chain staking, which is not enabled on this platform" - All six rows titled On-chain staking: … appear at
/admin/system/cron— batches (2 min), observe rewards (15 min), reconcile (1h), validator policy (6h), alerts (10 min), monthly statements (24h) — and none of them carries a refusal line under its name -
SOL_NETWORK(andETH_NETWORK, for a Lido pool) is set, and is the same network the ecosystem addon takes deposits on. There is only one pair of variables, so they cannot disagree - The staking wallet's gas reserve floor is above zero. A floor of zero switches the low-gas alarm off entirely
Their registered names, which is what a log line and an API response use, are
processRealStakingBatches, observeRealStaking, reconcileRealStaking,
refreshStakingValidatorSets, alertRealStaking and
generateRealStakingStatements. What each one does, and the consoles it feeds,
is in How an on-chain stake moves and
Operating on-chain staking.
Switching the addon off does not stop the batch job. It settles money the
platform already owes, so the scheduler registers it on the disabled branch
too, logging "Extension … is off — scheduled cron … anyway". Disabling an
extension closes its routes; it does not abandon a return a customer is
waiting for. The only thing that stops such a job is stopEverything on the
extension toggle, which records the addon in cronHardStoppedExtensions — and
that is a last resort, not a pause. To stop new stakes, pause the chain or the
pool.
Symptoms — fixed-rate
Users say withdrawals "do nothing"
The most common one, and it is a configuration trap.
Almost always this is approval doing exactly what it is configured to do.
stakingRequireWithdrawalApproval defaults to on, so an exit requested
before endDate becomes a PENDING_WITHDRAWAL position and waits for you.
Users see the request accepted and then apparently nothing happen.
Confirm it on the Overview screen: the awaiting your decision tile will have
a non-zero count and an oldest-request age. Then open
/admin/staking/settings → Earnings and decide: turn approval off so early
exits settle immediately, or leave it on and set an auto-approve threshold so
the small ones release themselves. Either way, work the queue that has built up.
Before August 2026 this had a second cause worth knowing if you are reading an older install. The screen used to draw this switch as off while the endpoint treated an absent key as on, and because the settings form saves only the keys you change, an operator who agreed with the screen and saved wrote no row — so the platform queued exits the operator believed were settling. The screen now states what the platform actually does. If your queue predates that fix, the backlog is real and still needs working.
Note the difference between the two positions. Approval on means every early exit needs an admin. Approval off means an early exit settles immediately and pays the pool's early-withdrawal fee. A withdrawal requested after the lock has expired settles immediately either way and never enters the queue.
No rewards are accruing
Work down this list:
- Is the cron running? Check
/admin/system/cron. It is hourly. Nothing accrues without it. - Is the pool
END_OF_TERMor auto-compound? Neither produces claimable rewards during the term — by design.END_OF_TERMsettles the whole reward at maturity; auto-compound folds it into the principal. - Has a full interval elapsed? A
DAILYpool credits nothing until 24 hours after the position started,WEEKLYnothing until 7 days,MONTHLYnothing until 30 days — not a calendar month. - Is
stakingAutomaticEarningsDistributionoff? With it off, matured positions still settle but nothing accrues in between until you run the accrual catch-up by hand. - Is
stakingEarningsDistributionTimeset? Only the hour matters. Periodic accrual runs only on the cron pass inside that hour, server-local. Check the cron output for "Outside the configured earnings distribution hour". - Is the position very small? Rewards are rounded to the currency's precision. A tiny stake can round to zero for several periods. Nothing is lost — the accrual is a running total, so it catches up as soon as the outstanding amount clears one unit of precision.
To settle it definitively, run the catch-up with dryRun: true against the
pool. It reports exactly what each position is owed without writing anything.
Matured positions are not returning principal
Look at the settlement has stalled alert on the Overview screen. It counts
positions that matured more than 24 hours ago and are still ACTIVE.
There is exactly one cause: the cron is not running, or it is failing. Maturity settlement is never gated by the distribution settings — holding capital past an agreed term is not a configuration option, and the code goes out of its way to make sure a toggle cannot cause it.
Check the cron process on port 4001, then the STAKING log. Failed positions
are retried three times with a five-second gap; if they still fail, a
high-priority in-app notification listing the position IDs is sent to a Super
Admin.
Users cannot stake
On-chain pools refuse for different reasons and answer 503 rather than 400 — see the on-chain symptoms below. On a fixed-rate pool the refusal message names the reason. In order of likelihood:
- "You don't have a USDT wallet. Please create one first." — staking does not create the source wallet. The user needs a wallet in the pool's exact symbol and wallet type.
- "Insufficient available amount to stake in this pool" — the pool's capacity is exhausted. See the capacity item below.
- "Staking pool is not active" — the pool is
INACTIVEorCOMING_SOON. - "Amount must be at least ..." / "must not exceed ..." — the pool's minimum or maximum stake.
- "Amount exceeds the allowed precision" — too many decimal places for that currency.
- A KYC refusal — the
invest_stakingfeature gate.
Also check the rate limit: five stake attempts a minute per user.
I cannot change a pool's available capacity
You cannot, from the admin form. The update endpoint deliberately strips
availableToStake out of the request body, because writing back the value the
form loaded would revert every stake and settlement that happened since the page
opened. The input on the edit screen is inert and nothing on screen says so.
Capacity changes only through an explicit signed delta:
curl -X PUT "https://your-host/api/admin/staking/pool/POOL_ID" \
-H "Content-Type: application/json" \
-H "Cookie: accessToken=..." \
-d '{"capacityDelta": 50000}'A negative delta is refused if the pool has less than that amount available, so you cannot drive capacity negative. Set capacity generously at creation time to avoid needing this.
Platform fees are zero
Two innocent explanations before you suspect a bug:
- You are testing as the Super Admin. The fee collector skips collection entirely when the actor is the Super Admin, rather than crediting them from themselves. Test with a normal user.
- The pool's admin fee is 0. No fee, no row.
The one that is not innocent: if there is no Super Admin role, or the role has
no users, every fee is dropped and logged as
[CRITICAL] ... platform fees are being dropped. No bookkeeping row is written
in that case — deliberately, so the earnings screen never shows revenue the
platform did not receive. There is no retry and no queue. Fix the Super Admin,
and treat the fees earned in the meantime as lost.
A pool shows as underpaying
The Overview flags a pool when its realised APR is more than 5% below its net promised APR.
First confirm you are reading the right comparison. A pool advertising 12% with a 20% admin fee is supposed to pay the staker 9.6%. The console already accounts for that — it measures against net promised, not advertised — so a flag is not just the fee showing up.
Then check:
- Has distribution been off or restricted to an hour? Accrual lag shows up as a shortfall until the next run catches up.
- Was the APR raised recently? Existing positions keep the APR they snapshotted at stake time, so the pool's realised rate lags its new advertised rate until the old positions roll off. This is correct behaviour.
- Pools that show "not measurable yet" are
END_OF_TERMor auto-compound. They never advance a watermark during the term, so there is no denominator. That is not a fault.
Distribution says earnings were already distributed this cycle
The one-off bonus endpoint allows one run per pool per cycle, where the cycle
length comes from the pool's earning frequency — 24 hours for DAILY, 7 days
for WEEKLY, 30 days for MONTHLY, and daily for END_OF_TERM. The window
slides from the epoch rather than resetting at midnight, so two requests either
side of midnight fall in the same bucket.
If you genuinely need a second payout in the same cycle, run it with the other
distributionType label — regular and bonus occupy separate buckets, and
both write BONUS earning rows.
If what you actually wanted was to catch up on APR that should already have
accrued, you want the other endpoint: /api/admin/staking/earnings/distribute
(plural). It has no cycle limit because it only ever credits the outstanding
delta.
Claiming returns "No earnings to claim"
- Nothing has accrued yet — see the accrual checklist above.
- Everything already claimed.
- The pool is auto-compound, in which case the message is different and explicit: rewards come back with the principal at maturity and are not separately claimable.
- The position was
CANCELLED, which forfeits unclaimed rewards permanently.
A pool or position will not delete
Both are guarded, and the guard applies to forced deletion too.
A pool cannot be deleted while it holds ACTIVE or PENDING_WITHDRAWAL
positions. A position cannot be deleted while it is itself ACTIVE or
PENDING_WITHDRAWAL. In both cases the principal has not been returned, and
deleting the row would strand it.
Settle first — complete, cancel, or let the term run — then delete. Note that cancelling forfeits the staker's unclaimed rewards; completing does not.
A second pool for the same token is rejected
Only one ACTIVE pool is allowed per (symbol, walletType) pair. Multiple
INACTIVE or COMING_SOON tiers on the same asset are fine, so you can stage a
replacement — but switching over is two steps: deactivate the old pool, then
activate the new one.
Existing positions in the deactivated pool keep accruing and still settle normally.
The staking menu does not appear
Both navigation trees are gated on the extension flag. Check
/admin/system/extension and confirm the Staking Crypto row (product ID
37434481) is active, then reload.
If the toggle animates on and reverts on reload, the write failed — look for
EXTENSION in the backend log.
The Overview shows no currency on its totals
Deliberate. A staking book holding BTC and USDT positions has no single denomination, so no unit is printed on any rolled-up figure. Money appears only inside rows that carry exactly one symbol: the pool table, the asset table and the withdrawal queue. Book-level rollups are counts.
If your entire live book is one asset, the unit appears automatically.
A withdrawal in the queue shows "term complete" but still deducts a fee
Correct, and intentional. An early exit is priced from the moment the user asked to leave, not from the moment you decide. A request filed inside the lock period keeps its early-exit price no matter how long it waits.
The alternative — pricing from settlement — meant the longer an admin took the
cheaper the exit became, and once endDate passed in the queue it became free.
The queue labels these rows "term ended while waiting" so the situation is
visible.
A settlement log mentions a failed final accrual
The message reads "Final reward accrual failed for position X; continuing with principal return".
Settlement tries to credit the last outstanding reward before the position goes terminal. If that step fails, the failure is logged and the principal is returned anyway — stranding someone's capital to protect a reward calculation would be strictly worse.
The consequence is a small unpaid reward on that one position. Investigate the logged cause; you can pay the shortfall with a one-off bonus distribution.
Symptoms — on-chain
None of the following exists in the fixed-rate product. The consoles these refer to are described in Chains, wallets and validators and Operating on-chain staking; this section is only what to do when one of them is telling you something you did not expect.
The on-chain cron rows read REFUSED
All six on-chain tasks run out of one module,
@b/api/(ext)/staking/utils/real/cron. If that module does not import, none of
them runs, and each of the six rows carries a standing refusal naming the
loader's own message. It is a state rather than a run result, so the rows stay
refused between ticks instead of turning green — a job whose module had never
loaded used to report as completed with a 100% success rate, which is the
failure this replaced.
In the backend log:
CRON [IMPORT] Failed to load module @b/api/(ext)/staking/utils/real/cron:
Cannot find module '.../backend/dist/src/blockchains/network-fee-tracker'One URGENT in-app notification and email goes to every Admin and Super Admin, at most once every fifteen minutes, and it covers the module rather than the job — six identical alerts about one broken import is a mail flood nobody reads. The per-job refusal state is separate and is not throttled, which is why all six rows still show refused between announcements.
An install that never bought the addon says nothing at all. The loader records only the installed-but-broken case, so no refusal and no rows means the module is absent, which is a deployment rather than a fault.
The refusal quotes the loader instead of diagnosing, and you should read it the same way. A missing dependency usually means an addon built against a newer core than the one installed, and updating core fixes it; a syntax error, or a throw while the module initialises, is a defect in that addon's build. Restart the scheduler once the module loads.
While the module is refused, nothing in the engine runs: no batch is signed or broadcast — gathers, delegations, exits, claims, returns and refunds all stop — no rewards are observed, the share price never moves, no commission is minted, no statement is issued, and neither the reconciler nor the alerting that would catch any of it is running. Principal a user has already unstaked is not being returned to them.
Users cannot stake into an on-chain pool
Two doors run the same check — the quote and the stake — so a closed switch refuses before the customer is ever shown a figure.
POST /api/staking/position refuses in this order:
- the native-app residence attestation, on native callers
- the
invest_stakingKYC feature. The quote asserts it too, which makes it the first door an on-chain customer hits.view_stakingis not a gate — it was removed rather than wired, and no route reads it - the pool exists and its
statusisACTIVE - the global
stakingModematches the pool's mode, or "This pool belongs to on-chain staking, which is not enabled on this platform" - the amount's precision, and a consent carrying the disclosure version and every acknowledgement
Then the engine's own intake check, which is the part people forget. Each of these answers 503 and names itself:
- on-chain staking is not ready — Ecosystem or Web3 Trading not installed, enabled or licensed, or the ecosystem key vault locked
- the pool's
intakeStatusisPAUSED - the chain activation is missing or not
ACTIVE, quoting the pause reason when one was given - the staking wallet is missing, or
FROZEN - on Solana only: the pool has no validator set, or the set does not currently pass its policy
- a
CRITICALincident is open on the pool or on its chain
Last comes the territory check, whose blocked list is derived from that pool's activation and is not the same list on every chain, and then the customer's own ecosystem wallet: an on-chain stake is gathered from the holder's own deposit address, so "You need a SOL ecosystem wallet with a deposit address first" is a refusal you will see on new accounts.
None of them holds an exit. Every one of the intake refusals says so in its own message. A regulator's "no new funds" order has to be one click, and a customer's exit must never be the thing that click holds — so the engines consult the intake check only on the way in, never on an exit, a claim, a return or a refund, and a source-scan test fails if that stops being true. Pausing a chain, a pool or a wallet is safe to do immediately.
No LOW_GAS incident has ever opened
The reconciler compares a staking wallet's native balance with its gas reserve floor only when the floor is above zero. The column shipped defaulting to zero, so on any install predating the current default the check is switched off and the incident cannot open — and the first sign of trouble is an exit that stops moving because the wallet can no longer pay the fee.
A wallet created by the current version starts at 0.05 SOL or 0.05 ETH. Check
the floor on every wallet older than that. It is the only editable field on a
wallet: PUT /api/admin/staking/wallet/:id with gasReserveFloor, permission
edit.staking.wallet. Size it to cover a week of exits.
Activating a chain today refuses to complete against a wallet whose floor is zero — "Set a gas reserve floor on the staking wallet above zero, or the low-gas alert cannot fire" — so a chain activated since that check landed cannot be in this state. One activated before it can.
The floor is load-bearing twice. The reconciler also uses it as the tolerance on the wallet's drift check, because a wallet legitimately spends fees between an exit settling and the transfer landing. A floor of zero removes that slack as well.
An exit is past the date the pool promised
Every ten minutes the alerts job counts the positions in UNSTAKE_REQUESTED or
UNBONDING whose unbondingBoundAt is in the past, and opens one
UNBONDING_OVERDUE warning per pool carrying that count. It resolves itself
when nothing is overdue.
That is the whole of it. The job does not ask the chain, does not re-broadcast anything and does not move the exit — it counts rows against a clock. What moves an exit is On-chain staking: batches, every two minutes. So an overdue alert is a symptom with three ordinary causes:
- the batches job is refused or not running — see the first checks above
- the claim or return batch is stuck. The batch row carries the hash and the
last error, and the reconciler opens
BATCH_STUCKonce a batch has been in flight longer than the chain's finality window - the protocol is genuinely slower than the bound, which is a per-chain constant the pool snapshotted rather than a measurement
The clock starts at the unstake request, not at delegation: an on-chain
position has no term, so its endDate is null and unbondingEndsAt carries
the only clock on it.
Do not quote a customer an hour figure from here — the pool's own "Exit takes
about / up to" tile is what they should be reading.
A batch has been retrying for hours
By design, and the alternative is worse. Once a batch has been broadcast it is
never marked FAILED: a send that returned an error may still have reached the
chain, and failing the row would invite the platform to make the same transfer
twice. It stays RETRYING until it lands. Only a batch that was never
broadcast can fail — and a failed gather is the event that makes a position
refundable, which is why that one distinction is enforced in code rather than
left to a caller.
Retries back off exponentially from 30 seconds and cap at one hour, so a batch with a dozen attempts behind it is genuinely only being tried hourly.
Retry now does not sign anything. It sets the row's clock back so the runner
picks the batch up on its next tick, re-approves the intent through the
transaction firewall and broadcasts from there — an admin button never bypasses
the firewall. POST /api/admin/staking/batch/:id/retry, permission
edit.staking.batch. It is refused on a CONFIRMED batch, "there is nothing
to retry", and on a BROADCAST one, which the runner is already polling.
So the question to answer is not why the batch has not failed. It is whether the transaction landed. The hash is written to the batch row before anything polls for it, so take it to a block explorer.
Solana reads are being rate limited
With nothing configured, the Solana path falls back to the public cluster endpoint. It is rate limited and was never meant to carry a platform.
Set SOL_MAINNET_RPC to your own endpoint — or SOL_DEVNET_RPC /
SOL_TESTNET_RPC, matching whatever SOL_NETWORK says. Several endpoints can
go in one value separated by commas, or in SOL_MAINNET_RPC_FALLBACK;
duplicates are dropped and the pool then chooses between them on health and
latency, trying an endpoint it has never used before it trusts a latency
figure.
Two traps:
- HTTP(S) only. A
ws://orwss://url is filtered out of the pool. If every url you configured is a websocket, the pool is empty and you are back on the public endpoint with nothing on screen saying so. - Reads fail over; broadcasts do not. A broadcast is made once, to the endpoint the preceding read just proved alive, because re-sending a transaction to a second endpoint is how one exit becomes two. And when every endpoint has failed, a read throws rather than returning an empty answer — a balance must never read zero because the nodes were down.
Ethereum does not use this pool at all. A Lido pool reaches the network through
the Ecosystem addon's own provider, on ETH_NETWORK with ETH_MAINNET_RPC or
ETH_HOODI_RPC, which is one of the reasons that addon is a requirement rather
than a convenience.
Escalation
If none of the above fits, collect before you ask for help:
- The position or pool ID, and the pool's mode.
- The
STAKINGorADMIN_STAKElog lines around the timestamp. - Whether the cron has run since the problem started — and for an on-chain problem, whether any of the six on-chain rows is refused.
- Which of the eleven staking settings actually have saved values, rather than what the screen displays.
That last one resolves more staking tickets than anything else on this page. The settings console writes eleven keys and the form saves only the ones you changed, so a screen that agrees with you can still be sitting on no row at all. Staking settings lists every key with its absent-key behaviour.
For a fixed-rate problem, add the output of the accrual catch-up with
dryRun: true for that pool — it reports what each position is owed without
writing anything. For an on-chain problem, add instead:
- The batch ID and its transaction hash, from
/admin/staking/batch. - The open incidents on that pool and chain, and when each was opened.
- The wallet's last read balance and its gas reserve floor.
- Which chain and network the activation is on, and its status.
The engine's own behaviour is documented in How an on-chain stake moves; the doors and drills are in Operating on-chain staking; the consoles and their permissions are in Chains, wallets and validators.