Documentation

Trust & Evidence Center

title: Trust Center description: Operator-facing documentation for the Trust Center reliability dashboard and the evidence spine behind it.

Trust Center

What it's for

The Trust Center (/trust in the authenticated app) answers six questions for the person who owns this trading system, without requiring them to know the internal evidence schema:

  1. Is the trading system trustworthy right now?
  2. What needs my attention?
  3. Which strategies are safe, blocked, stale, promoted, or reverted?
  4. Do broker reality and internal records agree?
  5. Does live performance still resemble tested (backtested) performance?
  6. Where can I inspect the evidence behind any of the above?

It is strictly read-only: every /api/trust/* endpoint is a GET request behind session authentication (get_current_user), and the page contains no button that places an order, promotes a strategy, or mutates trading state.

Overall status

The header shows one of three states, computed from the same data shown in the cards below it — never from "the API calls returned 200":

  • Healthy — the integrity audit last ran clean, there is no unresolved drift, and no strategy is blocked.
  • Needs attention — nothing is actively broken, but something needs review: the integrity audit has never run, or a strategy is stale, reverted, or has no validation on record yet.
  • Critical — the integrity audit found violations, drift is unresolved, or a strategy is blocked by a rejected validation.

The badge is always shown with a one-line reason (e.g. "2 subject(s) have unresolved drift between broker reality and internal records").

How unresolved drift is calculated

Every reconciliation cycle (holdings, fills, EOD, options orphan/mismatch) writes an evidence event: RECONCILED when the check passed, or DRIFT_DETECTED when it found a mismatch.

Unresolved drift = the number of subjects (subject_type/subject_id pairs, e.g. holding/RELIANCE) whose latest drift-relevant event is DRIFT_DETECTED. The moment a later RECONCILED event lands for that subject, it drops out of the unresolved count — regardless of how many DRIFT_DETECTED events preceded it.

This replaced an earlier, misleading open_drift_last_7d metric that simply counted DRIFT_DETECTED events emitted in the last 7 days — a drift that was reconciled minutes after being detected still inflated that count for a full week. The corrected calculation is exposed as unresolved_drift in EvidenceReader.overview() / /api/trust/overview, and as drift.unresolved_count in /api/trust/summary. The old, differently-named drift_detected_last_7d field keeps the raw emission count for historical context, clearly labeled as such — it is not used to decide overall status.

EvidenceReader.unresolved_drift_subjects() computes this with one SQL query (EvidenceStore.latest_per_subject, a DISTINCT ON (subject_type, subject_id) ... ORDER BY ts DESC query) instead of scanning the whole event table, so it stays cheap as evidence volume grows.

How strategy readiness is determined

arb_bot/evidence/readiness.py::classify_strategies() is a pure function that combines, per strategy code (GoLiveManager.STRATEGIES):

  • The latest VALIDATED/REJECTED event for subject_type="strategy" (the validation gate).
  • The latest PROMOTED/REVERTED event for subject_type="transition" (the go-live transition history).
  • The strategy's row in validation.calibration (live-vs-backtest error bars), if any live/backtest trade pairs have been matched.

into one of five labels:

| Label | Meaning | |---|---| | ready | Latest gate is VALIDATED and within the freshness window (VALIDATION_FRESHNESS_WINDOW_DAYS, default 30 days). | | blocked | Latest gate is REJECTED. The rejection reasons (from the validation report payload) are surfaced verbatim. | | needs_revalidation | Latest gate is VALIDATED but older than the freshness window. | | insufficient_evidence | No VALIDATED/REJECTED event exists for this strategy at all. | | reverted | The latest go-live transition is REVERTED — this takes priority over the gate state, since a revert usually means something went wrong live and is worth reviewing before re-promoting. |

The Trust Center never infers ready from the absence of a rejection; a strategy with no gate event is insufficient_evidence, not ready.

Research-run validation before promotion

Completing a research run records its result and provenance, but it does not by itself make the run eligible for policy promotion. An independent validation workflow must record a VALIDATED event for that backtest_run; a rejected or missing event keeps promotion disabled. The Research page links a not-yet-validated completed run to this read-only evidence surface so an operator can inspect its state before running the appropriate validation flow. This separation prevents a successful execution from being mistaken for validation evidence.

Health cards

The Overview tab's summary cards are computed from current state, not lifetime event volume:

  • Integrity audit — latest audit run's violation count.
  • Unresolved drift — see above.
  • Strategy validation — ready / total strategies, with blocked taking priority for the card's severity.
  • Live vs backtest calibration — how many strategies have at least one matched live/backtest trade pair (no pass/fail threshold is fabricated; the backend does not currently expose a calibration tolerance to the UI).
  • Last reconciliation — the most recent RECONCILED event across all subjects, when one exists.

Lifetime counts by event kind (CLAIMED, VALIDATED, PROMOTED, …) remain available under the collapsed Event diagnostics panel on the Overview tab.

Attention Required

A single prioritized list, computed client-side from /api/trust/summary, that never fabricates data the backend didn't provide:

  1. Integrity audit violations (critical) — with plain-language explanations for the two current codes.
  2. Unresolved drift (warning) — one row per still-drifting subject.
  3. Blocked strategies (critical), stale/needs_revalidation strategies (warning), and recently reverted strategies (warning).

Each item links straight into the relevant tab — audit violations and drift rows open the Evidence Explorer preselected on that subject; strategy issues open Strategy Readiness.

Tabs

  • Overview — health cards, Attention Required, the integrity audit detail panel, and the Event diagnostics expandable.
  • Issues & Drift — a filterable table (cards on mobile) of every DRIFT_DETECTED/RECONCILED event, with resolved/unresolved and subject type filters, a search box, and a time window. Clicking a row opens a drawer with the full event detail (payload, code SHA, config hash, data refs, previous-event link) and a link into the Evidence Explorer.
  • Strategy Readiness — one expandable row per strategy combining its gate, transition, and calibration state.
  • Evidence Explorer — the provenance walk, made discoverable: a subject-type selector backed by /api/trust/subject-types, an autocomplete subject picker backed by /api/trust/subjects, and a chronological timeline for the selected subject (/api/trust/provenance). Any drift row, strategy, or attention item can deep-link here with the subject preselected.

/provenance <subject_type> <subject_id> (Telegram)

Walk the immutable evidence chain for any subject from Telegram. Returns a chronological list of events (CLAIMED, VALIDATED, PROMOTED, RECONCILED, DRIFT_DETECTED, REVERTED) with their actor, git SHA, config hash, and prior event link — the same data the Evidence Explorer tab shows.

Examples:

  • /provenance transition DC — go-live transition history for the DC strategy
  • /provenance holding RELIANCE — holdings reconciliation history
  • /provenance signal DC:20260706103000 — the scan decision at that cycle

/explain_signal [strategy] [n]

  • /explain_signal (no args): the current-cycle in-memory last scan result for the active strategy.
  • /explain_signal DC 10: the last 10 persisted scanner decisions for DC, ordered newest first.

Integrity audit job

arb_bot.evidence.audit_job.run_integrity_audit() is an operator-scheduled (cron / systemd timer) job that verifies the spine:

  • BROKEN_CHAIN — an event whose prev_event_id points to an event_id that does not exist in evidence.events.
  • MISSING_PROVENANCE — an internal event (sig_type='internal') of kind CLAIMED/VALIDATED/PROMOTED/REVERTED with neither a code_sha nor a config_hash.

Each run writes one evidence.audit_runs row and one evidence.audit_violations row per violation. The latest run (with its violations) is queryable at /api/trust/audit and is embedded in /api/trust/summary. The job is best-effort: it logs and returns on any storage failure; it never raises into the bot loop.

API reference

All endpoints require authentication (Depends(get_current_user)) and are read-only.

| Endpoint | Purpose | |---|---| | GET /api/trust/summary | Consolidated snapshot driving the header, health cards, Attention Required, and Strategy Readiness — one SQL-aggregated round trip instead of six parallel requests. | | GET /api/trust/overview | Legacy lightweight counts endpoint. open_drift_last_7d was removed (see semantic correction above) and replaced with unresolved_drift and drift_detected_last_7d. | | GET /api/trust/drift | Filterable drift/reconciliation timeline. Params: limit (1–500), since (ISO) or since_days (1–3650, computed server-side), subject_type, unresolved (bool), q (subject search). Each row carries an unresolved flag. | | GET /api/trust/gates | Latest VALIDATED/REJECTED event per subject. Optional subject_type filter (Strategy Readiness uses subject_type=strategy). | | GET /api/trust/transitions | PROMOTED/REVERTED history, newest first (limit 1–500). | | GET /api/trust/calibration | validation.calibration mean/max |Δ₹| per strategy. | | GET /api/trust/audit | Latest completed integrity-audit run, with violations. | | GET /api/trust/provenance | Full prev_event_id chain for subject_type+subject_id (both required, non-empty). | | GET /api/trust/subject-types | Distinct subject_type values seen in evidence.events — powers the Evidence Explorer's type selector. | | GET /api/trust/subjects | Distinct subject_ids for a subject_type, optionally filtered by q, most-recently-active first (limit 1–50). |

overview(), unresolved_drift_subjects(), and gate_status() all use EvidenceStore.latest_per_subject() (DISTINCT ON) or count_by_kind() (GROUP BY) instead of pulling the full evidence.events table into Python — the previous overview() implementation read up to 10,000,000 rows per call. latest_audit() similarly paginates violations (violations_limit/ violations_offset, default page size 200; /api/trust/summary embeds a tighter 50-row page) instead of returning every violation for the latest run in one response — the run's violation_count field is always the true total, independent of the page size, and /api/trust/audit supports full pagination through the Audit panel.

Deployed git SHA (GIT_SHA)

EvidenceWriter.emit() auto-fills code_sha via git_sha() for every internal event. Production containers have neither a .git directory (scripts/docker_deploy.sh excludes it from the rsync payload) nor the git binary installed, so the subprocess-based fallback in git_sha() always silently returned None there. Combined with high-frequency internal CLAIMED events that never set config_hash either (e.g. signal scanner-decision claims, one per scan cycle), this produced a steady stream of false MISSING_PROVENANCE audit violations — because MISSING_PROVENANCE only fires when both code_sha and config_hash are missing.

scripts/docker_deploy.sh now resolves the SHA on the deploying machine (where .git and git are both available) and records it into the remote .env as GIT_SHA before restarting containers; git_sha() reads that env var first. This stops new violations of this kind from accumulating for every internal event.

Separately, signal (the scanner's per-scan-cycle decision claim, see arb_bot/scanners/persistence.py) is exempt from the MISSING_PROVENANCE check entirely (_PROVENANCE_EXEMPT_SUBJECT_TYPES in arb_bot/evidence/audit_job.py): it's high-frequency operational telemetry, not a go-live-affecting trust claim, so requiring code/config provenance on every scan cycle produced noise (thousands of violations) disproportionate to its value. VALIDATED/PROMOTED/REVERTED — the kinds that actually gate live trading decisions — are unaffected by this exemption and still require provenance regardless of subject_type. This exemption applies from the next audit run onward; evidence.events is append-only, so historical signal events already in the ledger keep whatever code_sha/config_hash they were written with, but they no longer count toward missing_provenance on any audit run after this change shipped.

What the Trust Center does not do

  • It does not place orders, promote or revert strategies, or delete/mutate any evidence or trading state — every action is a link to more evidence, never a mutation.
  • It does not fabricate a calibration pass/fail verdict — the backend does not currently expose a calibration tolerance, so the UI only reports how many strategies have matched trade pairs and their error bars.
  • It does not claim "no issues" when a request failed — loading, empty, and error states are visually and semantically distinct throughout, and a failed fetch never silently renders as a healthy or empty result.

Retention policy

evidence.events is append-only by default. The evidence.retention_policy table is the only opt-out: an operator inserts a row marking a specific (subject_type, kind) as prunable=true with a keep_days window, and the evidence.prune_expired() SQL function deletes only matching rows older than the window. The table ships empty — by default nothing is pruned.

Convention: the trust-critical kinds (VALIDATED, PROMOTED, REVERTED, REJECTED, DRIFT_DETECTED, RECONCILED) are never made prunable. Only high-volume snapshot-style claims (e.g. operator-selected holding CLAIMED) qualify for bounded retention, mirroring the existing iv_snapshots / battle_log / broker_health_samples TTLs.

Live math = research math, provably

PolicyLedger reuses research/engine/costs.py + tax.py, so a live policy and its source backtest share the same cost/tax math by construction. The spine makes this checkable: both emit evidence events pinned to the same config_hash (Phase 1 provenance). A parity test (tests/evidence/test_parity_spine.py) builds a synthetic backtest run and a policy ledger from the same resolved config and asserts byte-equal P&L. A drift between the two is a bug, not a regression.