The health screen and what each probe means

"Platform health: degraded" is not an answer on its own. This page says what was measured, which threshold tripped, and which of the eleven probes is worth acting on.

Where it lives

There is no standalone health screen. The surface is the Platform Health card on the admin dashboard, /admin, which polls GET /api/admin/system/health/batch every five minutes and draws:

• a score ring, 0-100 — green at 80 and above, amber at 60-79, red below 60;
• the overall status word — healthy, warning or critical;
• counts of services up, in warning and down;
• a Scheduler line of its own, above the service list, fed by a separate call to GET /api/admin/system/cron/scheduler;
• Show N services, which expands every probe with its status, its latency in milliseconds where it has one, a Critical tag where the probe is weighted as critical, and its message — the message is printed only for services in warning or down.

A failed refresh leaves the previous figures on screen rather than blanking the card, so a frozen score can mean the API is unreachable rather than that nothing changed. Use the refresh button in the card header to re-run the checks now.

Both the batch endpoint and its single-service sibling require authentication and the catch-all access.admin. The scheduler line needs view.cron as well, and is dropped — rather than taking the whole card down — for an operator who does not hold it.

Two endpoints, and they are not the same thing

Returns overall (score and status), a services array of {name, status, message, latency, critical}, and a timestamp. Service status is one of up, warning, down or unconfigured. This is what the dashboard card draws.

If the whole endpoint throws, it does not error — it returns a score of 50, status warning, and a single service called Health Check carrying the exception message. A card showing exactly one service is that fallback.

A different, older endpoint driven by ?service=. Nothing in the admin panel calls it; it is a manual tool. Valid names are email, stripe, transfi, sms, openexchangerates, googletranslate, ethereum, bsc, polygon, ftm, optimism, arbitrum, celo, and scylla — the last only when the ecosystem extension is enabled. An unrecognised name returns an empty object rather than an error.

The eleven probes

Blockchain RPC is omitted entirely unless the ecosystem or wallet_connect extension is enabled, so most installs see ten services rather than eleven. That is why the card says "Show 10 services" on a plain trading install and there is nothing missing.

Database

Times a single COUNT against the users table.

• warning above 1000 ms — connected but slow. The latency is in the message and in the card's own latency column.
• down — the query threw; the message carries the driver's error.

Critical, so a down here forces the overall status to critical and takes 40 points off the score on its own.

Cache (Redis)

Sends PING and expects PONG, giving up after 3 seconds.

• warning above 500 ms, or on an unexpected reply.
• down when the ping fails, with the message "Scheduled jobs and cross-process cache invalidation are stopped until it returns."

Also critical. Redis is a hard dependency, not a cache — sessions, rate limits, distributed locks, the job queue and cross-process settings invalidation all live in it, and the in-memory fallback was removed. A down here on a running backend means Redis went away after boot, because an unreachable Redis at boot stops the process with exit 78 instead. See Troubleshooting.

Scheduler (Cron)

The highest-consequence, lowest-visibility failure in the product, which is why it gets its own line rather than a row eight deep in a collapsed list. When the cron app dies every other probe stays green — the database answers, Redis answers, the API answers — while nothing scheduled runs.

It is scored from three inputs: readSchedulerHeartbeat(), the staleness threshold SCHEDULER_STALE_MS (60 seconds), and cronMode() — this process's own CRON_MODE.

Email Service

A configuration check, not a delivery check. It reads APP_EMAILER — defaulting to nodemailer-service when unset — and asks whether that transport's variables are present:

A missing configuration reports unconfigured, which costs the score nothing — the platform does not assume you want mail. A failure of the check itself reports warning. Neither state means a send failed, and this probe will happily read up while every message bounces.

To test delivery you need the notification console, not this card. See Notification delivery operations.

Exchange Provider

Looks for an exchange row with status: true, then for APP_<PROVIDER>_API_KEY and APP_<PROVIDER>_API_SECRET in the environment.

• No active provider → unconfigured, no score penalty.
• Active but credentials missing → warning, naming the provider.
• Both present → up.

It is a presence check. It does not call the exchange, so a revoked key, an IP ban or a rate limit all show up. For the real thing, see The exchange provider is degraded.

The four queue probes

These measure backlog age against the SLA cutoff in backend/src/utils/sla.ts — the same numbers the header inbox uses. (The KYC probe writes its seven days out as a literal rather than reading the file, but it is the same seven days.) A warning means old unactioned rows, not a broken queue and not a stopped worker.

The message always names the count, so "3 withdrawals pending >7 days" is telling you to open the withdrawal queue, not to restart anything.

Three of the four queue probes are written to fail soft: if the model is missing or the query throws, Transaction Queue, Withdrawal Queue and Support Queue all report up rather than manufacturing an outage — "Transaction system ready", "Withdrawal system ready", "Support system ready", and "No open tickets" where the support model itself is absent. So those three never being red is not proof the queue is healthy.

KYC Queue is the exception. Only its missing-model branch reports up ("KYC system ready"); a query that throws reports warning with the message "Could not check KYC queue", which costs the score 5 points. That amber means the probe could not run, not that applications are piling up — read the message before you open the KYC screen.

Error Rate

Failed transactions as a percentage of all transactions created in the last 24 hours.

• above 10% → warning, and flagged critical, which pulls 15 points off rather than 5.
• above 5% → warning.
• No transactions in the window → up, "No transactions in last 24h".

A spike here is usually one payment rail failing. Cross-check the Deposits queue and A customer paid and the balance did not move.

Blockchain RPC

Only present when ecosystem or wallet_connect is enabled. It walks the networks ETH, BSC, POLYGON, FTM, ARBITRUM, OPTIMISM, takes the first that has a <NETWORK>_MAINNET_RPC or <NETWORK>_TESTNET_RPC set, and issues a raw eth_blockNumber call with a 5-second timeout — probeEvmRpcBlockNumber, a plain fetch rather than an ethers provider, because a provider retries network detection every second forever when the node is down.

• No endpoint configured at all → unconfigured.
• First endpoint that answers → up, naming that network.
• Endpoints configured, none answered → warning, "RPC configured but connection failed".

How the score is arrived at

Start at 100 and deduct:

Then: critical if any critical service is down or the score is under 50; warning under 80; otherwise healthy.

Two consequences worth internalising. A single critical outage — Database, Redis or the scheduler — takes the card to critical on its own, however green everything else is. And a pile of queue warnings can push you to warning without anything being broken: five non-critical warnings is 25 points, and 75 is below the healthy line. Read the service list before you treat an amber card as an incident.

What to check first for each red service

/api/health is the probe; this card is the diagnosis

This card is an admin surface: it needs a session and the access.admin permission, and it runs eleven checks. That makes it the wrong thing for a load balancer to poll and the right thing for a human to read once something pages.

The probe is GET /api/health — unauthenticated, two checks, cheap enough to poll every few seconds forever. It answers 200 while the backend can serve and 503 when it cannot, so a monitor can act on the status line alone:

{"status":"ok","uptime":8241,"timestamp":"2026-08-12T12:25:47.738Z",
 "checks":{"database":{"status":"up","latency":1},"redis":{"status":"up","latency":1}}}

status is ok or degraded, and degraded still answers 200 — Redis being gone stops scheduled jobs and cache invalidation but does not stop this process serving requests. The database is the only check that turns into a 503. See Monitoring for what to alert on.

The maintenance server answers the same URL, and answers 503

While the platform is stopped, pnpm stop leaves a lightweight maintenance server holding port 3000 and the backend port. It answers /health and /api/health with HTTP 503 and the body:

{"status":"maintenance","message":"Site is under maintenance"}

Everything else under /api/ gets a 503 JSON body with Retry-After: 300, and every other path gets the 503 HTML maintenance page.

Related

• Working the Operations inbox — the queue budgets the four queue probes are measured against.
• The scheduler console — per-job state, which this card does not carry.
• Logs: locations, format and rotation — where the message behind a red probe was written.
• Troubleshooting — exit 78, Redis, ports and 502s.
• Processes and ports — what each PM2 app does and what stops when it is down.