MMashDiv

Troubleshooting

The staking failures that look like nothing is wrong — silent withdrawal queues, rewards that never accrue, fees that vanish, cron rows that read refused, and kill switches nobody remembers closing.

20 min readUpdated 15 September 2026troubleshooting, support, diagnostics, on-chain

Most staking problems are silent. Nothing errors, no screen turns red, and the first signal is a support ticket. This page is ordered by how often that happens.

The two products fail differently. Everything down to the on-chain sections below is the fixed-rate product — a typed APR, a term, an operator treasury — and none of it applies to a pool whose mode is REAL: an on-chain pool has no APR, no capacity, no lock period and no withdrawal approval queue. The on-chain sections are the second half of this page. Which product a pool belongs to is on the row, not on the global switch; see Staking settings.

Backend log modules to filter on: STAKING for the user endpoints and every staking cron, on-chain included, ADMIN_STAKE for the admin routes.

First checks — fixed-rate

  1. Awaiting your decision - count and oldest request age
  2. The withdrawal queue, oldest first

Before anything specific, confirm these four:

  • The Staking extension row is active at /admin/system/extension
  • Process Staking Logs appears at /admin/system/cron and has a recent run
  • A Super Admin user exists, and it is not the account you are testing with
  • The Overview screen at /admin/staking loads with real figures rather than the "figures unavailable" state

If the Overview says figures are unavailable, the aggregation query failed. That is deliberately not a 500 — the page keeps its header and Retry button — but the underlying error is in the log under ADMIN_STAKE.

First checks — on-chain

Process Staking Logs is the fixed-rate job and it is the wrong one to look at here. It never touches an on-chain position. On-chain staking runs on six separate tasks, and the one that moves coins is On-chain staking: batches, every two minutes.

  • /admin/staking/settings reports on-chain staking as available. Five things must hold — the Ecosystem extension and the Web3 Trading addon each installed, enabled and licensed, the ecosystem key vault unlocked, and at least one chain ACTIVE. The screen names whichever one is missing rather than saying "not available"
  • stakingMode is REAL, or the platform is refusing new stakes on every on-chain pool with "This pool belongs to on-chain staking, which is not enabled on this platform"
  • All six rows titled On-chain staking: … appear at /admin/system/cron — batches (2 min), observe rewards (15 min), reconcile (1h), validator policy (6h), alerts (10 min), monthly statements (24h) — and none of them carries a refusal line under its name
  • SOL_NETWORK (and ETH_NETWORK, for a Lido pool) is set, and is the same network the ecosystem addon takes deposits on. There is only one pair of variables, so they cannot disagree
  • The staking wallet's gas reserve floor is above zero. A floor of zero switches the low-gas alarm off entirely

Their registered names, which is what a log line and an API response use, are processRealStakingBatches, observeRealStaking, reconcileRealStaking, refreshStakingValidatorSets, alertRealStaking and generateRealStakingStatements. What each one does, and the consoles it feeds, is in How an on-chain stake moves and Operating on-chain staking.

Switching the addon off does not stop the batch job. It settles money the platform already owes, so the scheduler registers it on the disabled branch too, logging "Extension … is off — scheduled cron … anyway". Disabling an extension closes its routes; it does not abandon a return a customer is waiting for. The only thing that stops such a job is stopEverything on the extension toggle, which records the addon in cronHardStoppedExtensions — and that is a last resort, not a pause. To stop new stakes, pause the chain or the pool.

Symptoms — fixed-rate

Users say withdrawals "do nothing"

The most common one, and it is a configuration trap.

Almost always this is approval doing exactly what it is configured to do. stakingRequireWithdrawalApproval defaults to on, so an exit requested before endDate becomes a PENDING_WITHDRAWAL position and waits for you. Users see the request accepted and then apparently nothing happen.

Confirm it on the Overview screen: the awaiting your decision tile will have a non-zero count and an oldest-request age. Then open /admin/staking/settings → Earnings and decide: turn approval off so early exits settle immediately, or leave it on and set an auto-approve threshold so the small ones release themselves. Either way, work the queue that has built up.

Before August 2026 this had a second cause worth knowing if you are reading an older install. The screen used to draw this switch as off while the endpoint treated an absent key as on, and because the settings form saves only the keys you change, an operator who agreed with the screen and saved wrote no row — so the platform queued exits the operator believed were settling. The screen now states what the platform actually does. If your queue predates that fix, the backlog is real and still needs working.

Note the difference between the two positions. Approval on means every early exit needs an admin. Approval off means an early exit settles immediately and pays the pool's early-withdrawal fee. A withdrawal requested after the lock has expired settles immediately either way and never enters the queue.

No rewards are accruing

Work down this list:

  1. Is the cron running? Check /admin/system/cron. It is hourly. Nothing accrues without it.
  2. Is the pool END_OF_TERM or auto-compound? Neither produces claimable rewards during the term — by design. END_OF_TERM settles the whole reward at maturity; auto-compound folds it into the principal.
  3. Has a full interval elapsed? A DAILY pool credits nothing until 24 hours after the position started, WEEKLY nothing until 7 days, MONTHLY nothing until 30 days — not a calendar month.
  4. Is stakingAutomaticEarningsDistribution off? With it off, matured positions still settle but nothing accrues in between until you run the accrual catch-up by hand.
  5. Is stakingEarningsDistributionTime set? Only the hour matters. Periodic accrual runs only on the cron pass inside that hour, server-local. Check the cron output for "Outside the configured earnings distribution hour".
  6. Is the position very small? Rewards are rounded to the currency's precision. A tiny stake can round to zero for several periods. Nothing is lost — the accrual is a running total, so it catches up as soon as the outstanding amount clears one unit of precision.

To settle it definitively, run the catch-up with dryRun: true against the pool. It reports exactly what each position is owed without writing anything.

Matured positions are not returning principal

Look at the settlement has stalled alert on the Overview screen. It counts positions that matured more than 24 hours ago and are still ACTIVE.

There is exactly one cause: the cron is not running, or it is failing. Maturity settlement is never gated by the distribution settings — holding capital past an agreed term is not a configuration option, and the code goes out of its way to make sure a toggle cannot cause it.

Check the cron process on port 4001, then the STAKING log. Failed positions are retried three times with a five-second gap; if they still fail, a high-priority in-app notification listing the position IDs is sent to a Super Admin.

Users cannot stake

On-chain pools refuse for different reasons and answer 503 rather than 400 — see the on-chain symptoms below. On a fixed-rate pool the refusal message names the reason. In order of likelihood:

  • "You don't have a USDT wallet. Please create one first." — staking does not create the source wallet. The user needs a wallet in the pool's exact symbol and wallet type.
  • "Insufficient available amount to stake in this pool" — the pool's capacity is exhausted. See the capacity item below.
  • "Staking pool is not active" — the pool is INACTIVE or COMING_SOON.
  • "Amount must be at least ..." / "must not exceed ..." — the pool's minimum or maximum stake.
  • "Amount exceeds the allowed precision" — too many decimal places for that currency.
  • A KYC refusal — the invest_staking feature gate.

Also check the rate limit: five stake attempts a minute per user.

I cannot change a pool's available capacity

You cannot, from the admin form. The update endpoint deliberately strips availableToStake out of the request body, because writing back the value the form loaded would revert every stake and settlement that happened since the page opened. The input on the edit screen is inert and nothing on screen says so.

Capacity changes only through an explicit signed delta:

curl -X PUT "https://your-host/api/admin/staking/pool/POOL_ID" \
  -H "Content-Type: application/json" \
  -H "Cookie: accessToken=..." \
  -d '{"capacityDelta": 50000}'

A negative delta is refused if the pool has less than that amount available, so you cannot drive capacity negative. Set capacity generously at creation time to avoid needing this.

Platform fees are zero

Two innocent explanations before you suspect a bug:

  • You are testing as the Super Admin. The fee collector skips collection entirely when the actor is the Super Admin, rather than crediting them from themselves. Test with a normal user.
  • The pool's admin fee is 0. No fee, no row.

The one that is not innocent: if there is no Super Admin role, or the role has no users, every fee is dropped and logged as [CRITICAL] ... platform fees are being dropped. No bookkeeping row is written in that case — deliberately, so the earnings screen never shows revenue the platform did not receive. There is no retry and no queue. Fix the Super Admin, and treat the fees earned in the meantime as lost.

A pool shows as underpaying

The Overview flags a pool when its realised APR is more than 5% below its net promised APR.

First confirm you are reading the right comparison. A pool advertising 12% with a 20% admin fee is supposed to pay the staker 9.6%. The console already accounts for that — it measures against net promised, not advertised — so a flag is not just the fee showing up.

Then check:

  • Has distribution been off or restricted to an hour? Accrual lag shows up as a shortfall until the next run catches up.
  • Was the APR raised recently? Existing positions keep the APR they snapshotted at stake time, so the pool's realised rate lags its new advertised rate until the old positions roll off. This is correct behaviour.
  • Pools that show "not measurable yet" are END_OF_TERM or auto-compound. They never advance a watermark during the term, so there is no denominator. That is not a fault.
Distribution says earnings were already distributed this cycle

The one-off bonus endpoint allows one run per pool per cycle, where the cycle length comes from the pool's earning frequency — 24 hours for DAILY, 7 days for WEEKLY, 30 days for MONTHLY, and daily for END_OF_TERM. The window slides from the epoch rather than resetting at midnight, so two requests either side of midnight fall in the same bucket.

If you genuinely need a second payout in the same cycle, run it with the other distributionType label — regular and bonus occupy separate buckets, and both write BONUS earning rows.

If what you actually wanted was to catch up on APR that should already have accrued, you want the other endpoint: /api/admin/staking/earnings/distribute (plural). It has no cycle limit because it only ever credits the outstanding delta.

Claiming returns "No earnings to claim"
  • Nothing has accrued yet — see the accrual checklist above.
  • Everything already claimed.
  • The pool is auto-compound, in which case the message is different and explicit: rewards come back with the principal at maturity and are not separately claimable.
  • The position was CANCELLED, which forfeits unclaimed rewards permanently.
A pool or position will not delete

Both are guarded, and the guard applies to forced deletion too.

A pool cannot be deleted while it holds ACTIVE or PENDING_WITHDRAWAL positions. A position cannot be deleted while it is itself ACTIVE or PENDING_WITHDRAWAL. In both cases the principal has not been returned, and deleting the row would strand it.

Settle first — complete, cancel, or let the term run — then delete. Note that cancelling forfeits the staker's unclaimed rewards; completing does not.

A second pool for the same token is rejected

Only one ACTIVE pool is allowed per (symbol, walletType) pair. Multiple INACTIVE or COMING_SOON tiers on the same asset are fine, so you can stage a replacement — but switching over is two steps: deactivate the old pool, then activate the new one.

Existing positions in the deactivated pool keep accruing and still settle normally.

The staking menu does not appear

Both navigation trees are gated on the extension flag. Check /admin/system/extension and confirm the Staking Crypto row (product ID 37434481) is active, then reload.

If the toggle animates on and reverts on reload, the write failed — look for EXTENSION in the backend log.

The Overview shows no currency on its totals

Deliberate. A staking book holding BTC and USDT positions has no single denomination, so no unit is printed on any rolled-up figure. Money appears only inside rows that carry exactly one symbol: the pool table, the asset table and the withdrawal queue. Book-level rollups are counts.

If your entire live book is one asset, the unit appears automatically.

A withdrawal in the queue shows "term complete" but still deducts a fee

Correct, and intentional. An early exit is priced from the moment the user asked to leave, not from the moment you decide. A request filed inside the lock period keeps its early-exit price no matter how long it waits.

The alternative — pricing from settlement — meant the longer an admin took the cheaper the exit became, and once endDate passed in the queue it became free. The queue labels these rows "term ended while waiting" so the situation is visible.

A settlement log mentions a failed final accrual

The message reads "Final reward accrual failed for position X; continuing with principal return".

Settlement tries to credit the last outstanding reward before the position goes terminal. If that step fails, the failure is logged and the principal is returned anyway — stranding someone's capital to protect a reward calculation would be strictly worse.

The consequence is a small unpaid reward on that one position. Investigate the logged cause; you can pay the shortfall with a one-off bonus distribution.

Symptoms — on-chain

None of the following exists in the fixed-rate product. The consoles these refer to are described in Chains, wallets and validators and Operating on-chain staking; this section is only what to do when one of them is telling you something you did not expect.

The on-chain cron rows read REFUSED

All six on-chain tasks run out of one module, @b/api/(ext)/staking/utils/real/cron. If that module does not import, none of them runs, and each of the six rows carries a standing refusal naming the loader's own message. It is a state rather than a run result, so the rows stay refused between ticks instead of turning green — a job whose module had never loaded used to report as completed with a 100% success rate, which is the failure this replaced.

In the backend log:

CRON [IMPORT] Failed to load module @b/api/(ext)/staking/utils/real/cron:
  Cannot find module '.../backend/dist/src/blockchains/network-fee-tracker'

One URGENT in-app notification and email goes to every Admin and Super Admin, at most once every fifteen minutes, and it covers the module rather than the job — six identical alerts about one broken import is a mail flood nobody reads. The per-job refusal state is separate and is not throttled, which is why all six rows still show refused between announcements.

An install that never bought the addon says nothing at all. The loader records only the installed-but-broken case, so no refusal and no rows means the module is absent, which is a deployment rather than a fault.

The refusal quotes the loader instead of diagnosing, and you should read it the same way. A missing dependency usually means an addon built against a newer core than the one installed, and updating core fixes it; a syntax error, or a throw while the module initialises, is a defect in that addon's build. Restart the scheduler once the module loads.

While the module is refused, nothing in the engine runs: no batch is signed or broadcast — gathers, delegations, exits, claims, returns and refunds all stop — no rewards are observed, the share price never moves, no commission is minted, no statement is issued, and neither the reconciler nor the alerting that would catch any of it is running. Principal a user has already unstaked is not being returned to them.

Users cannot stake into an on-chain pool

Two doors run the same check — the quote and the stake — so a closed switch refuses before the customer is ever shown a figure.

POST /api/staking/position refuses in this order:

  1. the native-app residence attestation, on native callers
  2. the invest_staking KYC feature. The quote asserts it too, which makes it the first door an on-chain customer hits. view_staking is not a gate — it was removed rather than wired, and no route reads it
  3. the pool exists and its status is ACTIVE
  4. the global stakingMode matches the pool's mode, or "This pool belongs to on-chain staking, which is not enabled on this platform"
  5. the amount's precision, and a consent carrying the disclosure version and every acknowledgement

Then the engine's own intake check, which is the part people forget. Each of these answers 503 and names itself:

  1. on-chain staking is not ready — Ecosystem or Web3 Trading not installed, enabled or licensed, or the ecosystem key vault locked
  2. the pool's intakeStatus is PAUSED
  3. the chain activation is missing or not ACTIVE, quoting the pause reason when one was given
  4. the staking wallet is missing, or FROZEN
  5. on Solana only: the pool has no validator set, or the set does not currently pass its policy
  6. a CRITICAL incident is open on the pool or on its chain

Last comes the territory check, whose blocked list is derived from that pool's activation and is not the same list on every chain, and then the customer's own ecosystem wallet: an on-chain stake is gathered from the holder's own deposit address, so "You need a SOL ecosystem wallet with a deposit address first" is a refusal you will see on new accounts.

None of them holds an exit. Every one of the intake refusals says so in its own message. A regulator's "no new funds" order has to be one click, and a customer's exit must never be the thing that click holds — so the engines consult the intake check only on the way in, never on an exit, a claim, a return or a refund, and a source-scan test fails if that stops being true. Pausing a chain, a pool or a wallet is safe to do immediately.

No LOW_GAS incident has ever opened

The reconciler compares a staking wallet's native balance with its gas reserve floor only when the floor is above zero. The column shipped defaulting to zero, so on any install predating the current default the check is switched off and the incident cannot open — and the first sign of trouble is an exit that stops moving because the wallet can no longer pay the fee.

A wallet created by the current version starts at 0.05 SOL or 0.05 ETH. Check the floor on every wallet older than that. It is the only editable field on a wallet: PUT /api/admin/staking/wallet/:id with gasReserveFloor, permission edit.staking.wallet. Size it to cover a week of exits.

Activating a chain today refuses to complete against a wallet whose floor is zero — "Set a gas reserve floor on the staking wallet above zero, or the low-gas alert cannot fire" — so a chain activated since that check landed cannot be in this state. One activated before it can.

The floor is load-bearing twice. The reconciler also uses it as the tolerance on the wallet's drift check, because a wallet legitimately spends fees between an exit settling and the transfer landing. A floor of zero removes that slack as well.

An exit is past the date the pool promised

Every ten minutes the alerts job counts the positions in UNSTAKE_REQUESTED or UNBONDING whose unbondingBoundAt is in the past, and opens one UNBONDING_OVERDUE warning per pool carrying that count. It resolves itself when nothing is overdue.

That is the whole of it. The job does not ask the chain, does not re-broadcast anything and does not move the exit — it counts rows against a clock. What moves an exit is On-chain staking: batches, every two minutes. So an overdue alert is a symptom with three ordinary causes:

  • the batches job is refused or not running — see the first checks above
  • the claim or return batch is stuck. The batch row carries the hash and the last error, and the reconciler opens BATCH_STUCK once a batch has been in flight longer than the chain's finality window
  • the protocol is genuinely slower than the bound, which is a per-chain constant the pool snapshotted rather than a measurement

The clock starts at the unstake request, not at delegation: an on-chain position has no term, so its endDate is null and unbondingEndsAt carries the only clock on it. Do not quote a customer an hour figure from here — the pool's own "Exit takes about / up to" tile is what they should be reading.

A batch has been retrying for hours

By design, and the alternative is worse. Once a batch has been broadcast it is never marked FAILED: a send that returned an error may still have reached the chain, and failing the row would invite the platform to make the same transfer twice. It stays RETRYING until it lands. Only a batch that was never broadcast can fail — and a failed gather is the event that makes a position refundable, which is why that one distinction is enforced in code rather than left to a caller.

Retries back off exponentially from 30 seconds and cap at one hour, so a batch with a dozen attempts behind it is genuinely only being tried hourly.

Retry now does not sign anything. It sets the row's clock back so the runner picks the batch up on its next tick, re-approves the intent through the transaction firewall and broadcasts from there — an admin button never bypasses the firewall. POST /api/admin/staking/batch/:id/retry, permission edit.staking.batch. It is refused on a CONFIRMED batch, "there is nothing to retry", and on a BROADCAST one, which the runner is already polling.

So the question to answer is not why the batch has not failed. It is whether the transaction landed. The hash is written to the batch row before anything polls for it, so take it to a block explorer.

Solana reads are being rate limited

With nothing configured, the Solana path falls back to the public cluster endpoint. It is rate limited and was never meant to carry a platform.

Set SOL_MAINNET_RPC to your own endpoint — or SOL_DEVNET_RPC / SOL_TESTNET_RPC, matching whatever SOL_NETWORK says. Several endpoints can go in one value separated by commas, or in SOL_MAINNET_RPC_FALLBACK; duplicates are dropped and the pool then chooses between them on health and latency, trying an endpoint it has never used before it trusts a latency figure.

Two traps:

  • HTTP(S) only. A ws:// or wss:// url is filtered out of the pool. If every url you configured is a websocket, the pool is empty and you are back on the public endpoint with nothing on screen saying so.
  • Reads fail over; broadcasts do not. A broadcast is made once, to the endpoint the preceding read just proved alive, because re-sending a transaction to a second endpoint is how one exit becomes two. And when every endpoint has failed, a read throws rather than returning an empty answer — a balance must never read zero because the nodes were down.

Ethereum does not use this pool at all. A Lido pool reaches the network through the Ecosystem addon's own provider, on ETH_NETWORK with ETH_MAINNET_RPC or ETH_HOODI_RPC, which is one of the reasons that addon is a requirement rather than a convenience.

Escalation

If none of the above fits, collect before you ask for help:

  • The position or pool ID, and the pool's mode.
  • The STAKING or ADMIN_STAKE log lines around the timestamp.
  • Whether the cron has run since the problem started — and for an on-chain problem, whether any of the six on-chain rows is refused.
  • Which of the eleven staking settings actually have saved values, rather than what the screen displays.

That last one resolves more staking tickets than anything else on this page. The settings console writes eleven keys and the form saves only the ones you changed, so a screen that agrees with you can still be sitting on no row at all. Staking settings lists every key with its absent-key behaviour.

For a fixed-rate problem, add the output of the accrual catch-up with dryRun: true for that pool — it reports what each position is owed without writing anything. For an on-chain problem, add instead:

  • The batch ID and its transaction hash, from /admin/staking/batch.
  • The open incidents on that pool and chain, and when each was opened.
  • The wallet's last read balance and its gas reserve floor.
  • Which chain and network the activation is on, and its status.

The engine's own behaviour is documented in How an on-chain stake moves; the doors and drills are in Operating on-chain staking; the consoles and their permissions are in Chains, wallets and validators.