Operate
Operations
Run metered inference with kill switches, circuit breakers, and auditable changes.
Mandatory controls
- Global inference, provider, model, and skill kill switches stop new upstream work.
- Account balance, API-key limit, request output, rate, token, and concurrency checks bound spend.
- Circuit breakers move from closed to open to half-open after configured failure thresholds.
- Payment and credit changes use idempotency and append-only ledger entries.
- Production OpenRouter routing is controlled by its separate commercial gate.
- Same-model failover runs only when the failed attempt is known not to have incurred upstream spend. Dispatched or partially metered attempts stop for settlement or reconciliation.
- Cross-model fallback remains disabled until each attempt has route-specific reservation and billing.
Observe and reconcile
- Correlate structured logs with request ID, account, key prefix, model, skill, and provider; never include secrets.
- Track success, provider errors, p50/p95/p99 latency, tokens, cost, revenue, spread, and stream disconnects.
- Compare upstream pricing metadata with stored cost and flag discrepancies for review.
- Reconcile completed usage records, reservations, and inference ledger charges.
- Run metering:reconcile before reservations:reconcile and alert on stale holds or inconsistent counters.
- Test timeout, provider outage, malformed stream, client abort, and insufficient-credit paths.