Live Policies

Live Policies

arb_bot/policies/ promotes an arb_bot/research/ strategy from a one-off backtest into a running, persistent portfolio that trades on a schedule — first in PAPER mode (simulated fills, real signals, real costs/tax), then optionally in LIVE mode against a real broker connection once you trust the paper track record.

Open Live Policies

⚠️ Every LIVE order requires a human approval. The daily cycle for a LIVE policy only ever produces a PENDING_APPROVAL order plan — no broker order is placed until an operator approves it from the dashboard or via /approve_plan <plan_id> in Telegram. The single exception is the opt-in overnight liquid-fund sweep (see below), which is capped, restricted to one configured symbol, and always notified after the fact. arb_bot/policies/ never imports arb_bot.bot, execution.engine, or command_handler, and never touches option-strategy state — it is a fully separate execution path that happens to reuse the research engine's signal generation and cost/tax modelling.

Concept: Policy → Cycle → Plan

A policy (policies table) is a named, persistent instance of a research strategy config — tag_code (an 8-character alphanumeric code sent to the broker on every order so you can identify a policy's orders in your own orderbook), mode (PAPER or LIVE), status (ACTIVE / PAUSED / RETIRED), the strategy's config JSONB (the same shape as a research RunConfig, plus policy-only extras — see Risk controls below), and (once live) a broker_connection_id pointing at one row in brokers.broker_connections.

Each trading day the scheduler runs one cycle (policy_cycles) per active policy: generate signals → build an order plan (policy_order_plans + policy_orders) → either simulate the fills (PAPER) or park the plan for approval (LIVE). A cycle is keyed by (policy_id, cycle_date, mode) and its status machine — SIGNALS → PLANNED → PENDING_APPROVAL → APPROVED → EXECUTING → EXECUTED | PARTIAL | FAILED | EXPIRED | REJECTED | NOOP — is the source of truth for "has this policy already run today," so the scheduler, a manual "run cycle now" API call, and a process restart can never double-execute or double-notify for the same day. The dashboard now also surfaces the policy's next scheduled run time so operators can see when the next cycle is due without opening the scheduler logs.

Ledger tables (policy_positions, policy_trades, policy_holdings, policy_equity, policy_tax_lots, policy_cash_flows, policy_state) mirror the shape of the equivalent research_* tables and are keyed by (policy_id, mode) — so a policy's PAPER history is preserved untouched when it goes LIVE; LIVE simply starts its own parallel ledger under the same policy_id. policy_signals and policy_universe_events record the raw signal pass/fail and universe entrant/leaver events each cycle, the same way the research Universe tab does for a backtest run.

PAPER vs LIVE lifecycle

PAPERLIVE
Order plan outcome Auto-claimed and simulated in the same cycle call — fills booked at a fresh current provider quote when available, falling back to the planner's reference price only if the quote source does not return one, subject to the same transaction-cost model the research engine uses. Left in PENDING_APPROVAL. Nothing is placed until a human calls /approve_plan or the dashboard Approve action.
Broker interaction None — no broker_connection_id is required. Real orders via EquityOrderRequest / place_equity_order against the bound broker connection.
Fill confirmation Immediate — simulated at plan time. Polled from the broker by the reconciler (poll_order_fills); positions/cash are only booked once the broker reports a fill, never from the plan's estimate.
EOD valuation eod_valuation marks the book from provider price history. Same eod_valuation, plus a cross-policy broker holdings audit (audit_holdings). PAPER cycles also run a valuation snapshot immediately after execution so holdings, cash, and gain/loss update as soon as the paper trade completes.

Going live is a one-way, explicit action: POST /api/policies/{pid}/go-live with a broker_connection_id and capital. This sets mode=LIVE, records live_started_at, and binds the policy to that broker connection — from the next cycle onward its plans stop auto-executing and start requiring approval. The PAPER ledger rows already written under this policy_id are left exactly as they were, so you can always compare PAPER-projected vs LIVE-actual performance for the same strategy (see GET /api/policies/{pid}/tracking below).

The detail page now pages the larger policy tables so long histories stay usable. Cycle history, signals, universe events, order plans, trades, and holdings all accept page / page_size on the API and render page controls in the UI when a table has multiple pages. The policy header also shows the active market data source so PAPER policies make it obvious whether they are using the broker-backed read-only feed or a configured fallback.

Approval flow

A LIVE cycle that produces orders ends in PENDING_APPROVAL and sends a notification (notify_plan_pending) summarizing the plan's buy/sell value and estimated costs. An operator approves or rejects it from either surface:

  • Dashboard — the Policies page lists every pending plan (GET /api/policies/pending-approvals); the Approve action calls POST /api/policies/plans/{plan_id}/approve, optionally excluding specific orders via excluded_order_ids.
  • Telegram/approve_plan <plan_id> and /reject_plan <plan_id>; /policies lists every policy (name · mode · status · tag_code).

approve_plan claims the plan atomically (a plan can only ever be approved once, even under concurrent calls) and re-validates before anything is placed: the policy must still be ACTIVE/ LIVE, every non-excluded order's estimated value must be under max_order_value, and — if the plan needs net new cash after sells fund whatever they can — the broker's available funds must cover the shortfall (a lookup failure here is logged and traded through, since it is a best-effort check, not the exchange's own margin gate).

If the market is open, approval places orders immediately: SELLs first (to raise cash), then a liquid-fund redemption if the projected cash still falls short of the BUY total, then BUYs — each order separated by POLICY_ORDER_DELAY_SEC. If the market is closed, the plan is left APPROVED and the scheduler's next market-open tick places it via the same place_plan_orders path, which is idempotent (only orders still in status APPROVED are ever placed, so a scheduler retry after a partial failure never double-places an order). If the broker connection cannot be built before any order is submitted, the plan stays APPROVED so the next market-open scheduler tick can retry it.

Broker tagging & multi-connection binding

Every order a policy places carries its tag_code as the broker's own order tag — Zerodha's Kite tag parameter and Dhan's equivalent tag/correlation field, both truncated to the broker's 20-character limit (well above the 8-character tag_code). This means orders are attributable per policy directly in the broker's own orderbook UI — an operator can open Kite or the Dhan app and see which live orders came from which policy.

⚠️ Holdings are NOT grouped or tagged per policy broker-side. get_equity_holdings() on both the Zerodha and Dhan adapters returns one aggregate {symbol: quantity} map for the whole account — a snapshot of everything currently held on that connection, with no way to ask the broker "how much of this holding came from policy X." If two LIVE policies on the same broker_connection_id both hold the same symbol, the broker view only ever shows the combined total. This is exactly why reconciler.audit_holdings exists: it sums each LIVE policy's own tracked policy_positions rows across every policy sharing a connection and compares that sum against the broker's aggregate — the system has to reconstruct per-policy attribution itself, because the broker never provides it.

Multiple policies can share one broker_connection_id (e.g. two strategies both trading through the same Zerodha account) or use separate connections entirely. GET /api/policies/broker-connections lists the available connections (id, broker name, account id, role) for the go-live picker.

Overnight fund sweep

A policy can opt in (liquid_sweep_enabled, default false) to automatically buy a configured liquid/overnight fund symbol (liquid_symbol, default LIQUIDCASE) with idle cash above a floor (liquid_sweep_min), the same way the research engine parks idle cash between rebalances. This is the one exception to "every order needs human approval": run_liquid_sweep fires automatically at the end of a PAPER cycle, or from the LIVE reconciler after fill polling, without a PENDING_APPROVAL plan — but it is opt-in per policy, hard-capped at the ledger's own cash (it will shrink the buy quantity until the total spend, including estimated costs, fits inside available cash), restricted to a single symbol, and always sends a notification after it executes (or after a broker rejection, in which case the ledger is left untouched).

On the LIVE side, a rejected sweep or redemption order is handled conservatively: the in-memory ledger mutation is only flushed to policy_positions/policy_state after the broker confirms the order, so a rejected sweep never corrupts tracked cash or position state. The reconciler also uses the same liquid-fund mechanics in reverse (redeem_liquid_for) to raise cash for an approved BUY plan when broker funds fall short — selling just enough whole units to cover the shortfall, never more than currently held.

Risk controls

KeyDefaultMeaning
max_order_value200000.0Per-order cap enforced at approval time; a plan containing any order above this is rejected outright rather than partially approved.
plan_expiry_hours30How long a PENDING_APPROVAL plan stays actionable before expire_stale_plans flips it to EXPIRED — long enough to span an overnight gap, short enough that a stale plan can't suddenly execute against a market that has moved on.
liquid_sweep_enabledfalseOpt-in for the overnight auto-sweep described above.
liquid_symbolLIQUIDCASEThe single symbol the sweep/redemption logic is allowed to trade.
liquid_sweep_min5000.0Cash floor below which the sweep does not fire; the sweep also keeps back half of this floor as a costs buffer.

These live inside a policy's config JSONB alongside its research RunConfig fields (top_n, rebalance_frequency, position_sizing, rebalance_policy, etc. — see the Equity Research docs for those). No-shorts and whole-share sizing are inherited directly from the research planner — build_order_plan never emits a SELL beyond a currently-held quantity.

Config keys (POLICY_*)

KeyDefaultMeaning
POLICY_WORKER_ENABLEDtrueMaster switch for the PolicyWorker background daemon thread started on dashboard API startup. Never starts under pytest.
POLICY_CYCLE_TIME_IST16:30Earliest time of day the daily signals → plan cycle is allowed to fire (post-market close, once EOD prices are settled).
POLICY_EOD_TIME_IST17:30Earliest time of day the EOD holdings/equity valuation snapshot runs.
POLICY_MARKET_OPEN09:15Start of the window in which an approved LIVE plan is placed immediately rather than queued.
POLICY_MARKET_CLOSE15:30End of that window.
POLICY_ORDER_DELAY_SEC0.4Delay between consecutive live policy orders placed in the same plan (SELLs, then BUYs).

API endpoints

EndpointPurpose
POST /api/policiesCreate a policy (PAPER by default) — validates tag_code shape, top_n > 0, positive initial_capital.
GET /api/policiesList every policy with its latest equity row and pending-plan count.
GET /api/policies/pending-approvalsEvery PENDING_APPROVAL plan across all policies, for the dashboard approval queue.
GET /api/policies/broker-connectionsBroker connections available for the go-live picker.
GET /api/policies/{pid}Policy detail + current policy_state + latest cycle.
POST /api/policies/{pid}/go-livePromote PAPER → LIVE: binds a broker_connection_id and starting capital.
POST /api/policies/{pid}/pauseSet status PAUSED — the scheduler skips paused policies entirely.
POST /api/policies/{pid}/resumeSet status back to ACTIVE.
POST /api/policies/{pid}/retireSet status RETIRED — permanent, not resumable via the API.
POST /api/policies/{pid}/run-cycleManually trigger today's cycle immediately instead of waiting for the scheduler.
POST /api/policies/{pid}/snapshotTake a one-time valuation snapshot for the policy's current holdings and equity, without starting a new cycle.
POST /api/policies/{pid}/retry-latest-cycleRetry the latest failed cycle in place when it has not produced plans or trades yet.
GET /api/policies/{pid}/equityDaily equity curve for a given mode (default PAPER).
GET /api/policies/{pid}/holdingsHoldings snapshot; latest by default, or nearest on/before an optional date. Supports page / page_size for long histories.
GET /api/policies/{pid}/tradesTrade ledger for a given mode, with pagination support.
GET /api/policies/{pid}/signalsRaw per-cycle signal rows (Universe-tab equivalent), with pagination support.
GET /api/policies/{pid}/cyclesCycle history with status, error summary, and retry note; now paginated.
GET /api/policies/{pid}/ordersEvery plan for this policy (joined with its cycle) plus each plan's orders, paginated for large histories.
GET /api/policies/{pid}/universe-eventsENTERED/LEFT universe membership events per cycle, paginated like the other detail tables.
GET /api/policies/{pid}/trackingPAPER equity, LIVE equity, the source research run's backtest equity (if promoted from a run), and per-order fill slippage — for comparing backtest-projected vs paper-simulated vs live-actual performance.
POST /api/policies/plans/{plan_id}/approveHuman-approve a PENDING_APPROVAL plan (optionally excluding specific orders).
POST /api/policies/plans/{plan_id}/rejectReject a PENDING_APPROVAL plan outright.

Telegram commands

CommandDescription
/policiesList every policy: name · mode · status · tag_code.
/approve_plan <plan_id>Human-approve a pending LIVE order plan.
/reject_plan <plan_id>Reject a pending LIVE order plan.

Troubleshooting

  • Plan expired without approval. A PENDING_APPROVAL plan left untouched past plan_expiry_hours is atomically flipped to EXPIRED by expire_stale_plans (run once per scheduler sweep, not per policy) and its cycle moves to EXPIRED too, with a Telegram notification. This is expected behavior for a plan nobody acted on — the next scheduled cycle will generate a fresh plan from current signals rather than executing stale ones. If plans are expiring faster than you can review them, raise plan_expiry_hours for that policy's config.
  • Need a fresh holdings date for a test policy. Use the policy detail page's Snapshot holdings action (or POST /api/policies/{pid}/snapshot) to force a one-time valuation using the latest available market prices. If a symbol is missing from the historical price window, the snapshot now falls back to current LTP before giving up. It writes a fresh holdings and equity snapshot for the policy without creating a new cycle, which is useful when you are validating a policy for the first time and want the table to reflect the current price instead of the last scheduled valuation date.
  • Paper trades but no visible gain/loss. PAPER execution now writes a fresh valuation snapshot immediately after the cycle completes, so the holdings table and equity curve update without waiting for the overnight scheduler. The reconciler also retries any symbols that the first price-history fetch missed, so a partial provider response no longer leaves a holding stuck at its avg_cost. If you still see stale values, check whether the latest cycle actually completed or whether the UI is showing an older page in a paginated table.
  • Signals/trades but empty universe events. policy_universe_events only records membership changes (ENTERED/LEFT), not every signal row or every trade. A retry that reuses the same cycle date can also legitimately produce no new universe delta if the set of members did not change.
  • Latest cycle failed. The Policies page and policy detail view surface the latest cycle's error_summary when a cycle fails, and the policy detail page now includes a cycle-history table with the retry note for each recent cycle. If that failure did not create any plans or trades, the UI exposes a Retry action that reruns the failed cycle in place without changing the policy's PAPER/LIVE mode. If the failure came from a single stale Dhan instrument lookup, the retry path now skips that unresolved symbol and proceeds with the rest of the universe instead of failing the entire cycle again. If the failed cycle already spawned plan rows or trades, retry stays blocked so the audit trail and broker state cannot drift silently.
  • "Holdings shortfall" audit warning. audit_holdings runs once per day (LIVE policies only, at EOD) and warns when the broker's actual holding for a symbol is less than the sum of every LIVE policy's tracked position for that symbol on the same connection — a real shortfall, not a benign surplus (extra broker-side shares from a source outside the policies system are not flagged). This usually means a fill was booked into a policy's ledger that the broker doesn't actually reflect — check recent policy_orders rows for that symbol/connection against the broker's own orderbook and trade confirmations before trusting the ledger further.
  • Partial fills. When the broker reports a PARTIAL status, the reconciler does not book anything immediately — it waits a 15-minute grace period (_PARTIAL_GRACE) in case the rest of the order fills shortly after. Only once that grace period has elapsed does it book whatever quantity has filled so far and mark the order PARTIAL (notifying the operator), leaving the unfilled remainder alone. A plan is only finalized to EXECUTED/PARTIAL once every one of its non-excluded orders has left the PLACED/ APPROVED states.

See docs/research_backtest.md for how a research run is promoted into a Live Policy, and the equity-research pipeline that feeds every policy's daily signals.

Research-to-PAPER promotion

A completed research backtest is promoted to a PAPER Policy through a dedicated backend-owned endpoint that clones the persisted research configuration exactly, applies only a whitelist of Policy-only overrides, and writes an immutable provenance record. The generic POST /api/policies endpoint issues a typed SOURCE_RUN_REQUIRES_PROMOTION_ENDPOINT migration pointer (HTTP 422) when a request includes source_run_id; the dedicated endpoint is the only way to create a source-linked Policy.

EndpointPurpose
POST /api/research/runs/{run_id}/promote-to-paperAtomically promote a completed research run to a PAPER Policy. Requires an Idempotency-Key header (16–128 chars). Body accepts tag_code, name, optional initial_capital (defaults to source), policy_overrides (whitelist: liquid_sweep_enabled, liquid_symbol, liquid_sweep_min, max_order_value, plan_expiry_hours), optional allocation_mode (omitted → source sip_allocation_mode), confirm_additional_policy, and rejected_confirmation. Returns 201 on first creation, 200 on idempotent replay, 404 / 409 / 422 on typed failure.
GET /api/research/runs/{run_id}/promotionsList all PAPER Policies linked to a research run, with audit fields (promoted_by, base/effective validation state, current LIVE eligibility) and latest_complete_authoritative_valuation: null until PR 2B.

Validation is metadata, not a gate. PAPER creation is allowed for all validation states (VALIDATED, NOT_VALIDATED, UNKNOWN, REJECTED). A run whose base validation state is REJECTED requires an explicit confirmation (acknowledged=true + a reason of at least 20 non-whitespace characters), even when the effective validation state has been overridden to OVERRIDDEN. PAPER creation never implies LIVE eligibility — /go-live still performs a fresh fail-closed validation read and requires validation.promotion_allowed == true as a hard, non-overridable invariant.

HTTP status mapping. Resource conflicts return 409 (RUN_NOT_COMPLETED, IDEMPOTENCY_CONFLICT, ADDITIONAL_POLICY_CONFIRMATION_REQUIRED, TAG_CODE_CONFLICT); not-found returns 404 (RUN_NOT_FOUND); request-shape / validation failures return 422.

Idempotency. The same (run_id, Idempotency-Key) pair with the same request body returns the original result with replay=true. A different body with the same key returns 409 IDEMPOTENCY_CONFLICT. A different key for an already-linked run requires confirm_additional_policy=true to create another Policy. Concurrent identical requests converge on one Policy via a per-run advisory lock.

Atomicity. The entire promotion — policy row, PAPER state, optional initial DEPOSIT cash flow, contribution schedule, PROMOTED evidence event, and immutable provenance row — is written in a single database transaction. The source research_runs row is read FOR SHARE inside the same transaction. If any step fails, nothing is left behind.

Authoritative config source. The persisted research_run_configs.config JSONB is the source of truth. Known fields are validated against the current RunConfig; unknown / future fields survive verbatim into the policy config. The browser sends only Policy-only overrides and capital; the authoritative strategy configuration never leaves the backend.

Linked Policy visibility. The linked-Policy list includes latest_complete_authoritative_valuation: null intentionally until PR 2B introduces explicit complete authoritative valuation attempts. PR 2A never uses the current non-strict eod_valuation() as an authoritative result.

Combined-strategy policy configuration

combined is a virtual strategy, not a registry strategy that can run alone. A policy using it must persist a non-offcombine_mode plus its member strategies and weights. The manual Policy form selects an executable composite default whencombined is chosen, and the API rejects an incomplete combined configuration before it creates a policy. Existing invalid policies fail without placing orders and report the configuration problem; recreate them with the combination settings rather than repeatedly retrying the failed cycle.

Policy unit and NAV accounting

Each PAPER and LIVE ledger persists its own units_outstanding,nav_gross, nav_net_of_costs, andnav_net_of_tax in policy_state. NAV starts at ₹10. PolicyLedger.flush() writes these values with cash, cumulative costs, accrued tax, and paid tax, so restarts retain the same accounting basis. Policy creation and go-live initialize the deposit's units at ₹10 NAV; zero units are only valid when the ledger has no assets. Repeating go-live after promotion is a no-op: it does not replace the broker binding or configuration, reset evolved cash/units/NAV, or insert another initial deposit.

Asset totals use one cash-plus-holdings valuation at all three levels: gross adds cumulative trading costs back, net-of-costs uses actual assets, and net-of-tax subtracts only outstanding tax (tax_accrued - tax_paid). Equity and liquid-fund positions use supplied market prices. Contribution unit issuance never falls back to average cost: a priced book requires a same-day market valuation, otherwise application fails closed without a cash or unit write.

Contribution state machine and funding separation

Policy contributions use explicit, transition-checked operations instead of a generic status update. LIVE contributions followSCHEDULED → FUNDING_DUE → CONFIRMED → APPLIED; PAPER contributions may move directly from SCHEDULED toAPPLIED. APPLIED, SKIPPED, andCANCELLED are terminal. Deferred and failed LIVE funding can return to FUNDING_DUE for an operator-controlled retry. Every transition locks the contribution row, and schedule/date uniqueness makes creation safe to retry after a worker restart.

Funding confirmation and investment application are deliberately separate.confirm_funding() records exactly one allocation inpolicy_funding_allocations and marks the contributionCONFIRMED; it does not credit policy cash, approve an order plan, or place an order. apply_live_contribution() is the later ledger action that issues units at the pre-flow net-of-tax NAV, credits cash, records policy_cash_flows, persists all three NAV levels, and marks the contribution APPLIED in one transaction. Application persists the valuation source and reserves the broker-account allocation. It records BROKER_VERIFIEDonly when a broker funds snapshot is available; otherwise an explicitOPERATOR_ATTESTED record is required. A LIVE order plan remains PENDING_APPROVAL until a human separately approves it through the existing approval flow.

Before each normal policy scheduler tick, the contribution worker evaluates the active schedule using IST and the policy subsystem's eligible-date convention (the NSE trading calendar, including weekday holidays as well as weekends). Due rows are created with database-backed uniqueness on schedule and scheduled date, so restarts and concurrent retries cannot duplicate a contribution. The locked LIVE transition reports which caller actually changed SCHEDULED to FUNDING_DUE, and only that winner sends the notification. PAPER rows are applied immediately and their cash is available to that day's subsequent rebalance. LIVE rows stop atFUNDING_DUE and notify the operator exactly once; the worker never confirms funding, applies LIVE cash, approves plans, constructs a broker, or places orders. The next scheduled date is persisted for operator visibility.

The authenticated apply-funding route applies a PAPER contribution directly. For LIVE, confirm-funding first records funding evidence and apply-funding later applies the contribution as two explicit operations. LIVE requests require a positiveconfirmed_amount plus either anexternal_reference or operator_attested=true; they may also identify the broker_connection_id. The funding allocation records the authenticated dashboard identity, not a client-supplied operator name. Deferral and cancellation routes likewise use their explicit, transition-checked state-machine operations. Route dispatch uses the mode persisted on the contribution row, so a historical PAPER contribution remains a PAPER ledger action even after its policy is promoted to LIVE.

Contribution application fails closed if persisted assets are positive while units are zero, or if the pre-flow NAV is non-finite or not positive. No cash, unit, cash-flow, or status write survives that failed transaction. Recovery is mode-aware: PAPER failures return toSCHEDULED; an unconfirmed LIVE failure returns toFUNDING_DUE; and a confirmed LIVE failure returns directly to CONFIRMED while reusing its existing unique funding allocation. Recovery never confirms the same funding twice. Before restoring a funded LIVE contribution toCONFIRMED, recovery locks the allocation and requires its policy and amount to exactly match the contribution row; mismatches fail atomically and leave the contribution FAILED.

Same-day contribution batching

When two or more contributions land on the same effective date (a base mandate and a step-up mandate, or two colliding schedules), the worker gathers every due contribution for that day first, obtains one pre-flow valuation, and applies each contribution's cash sequentially against that single running valuation — it never calls the daily valuation routine between individual contributions. Each contribution is still recorded and auditable on its own row, but the day's single equity row reflects the combined cash exactly once, never double-counted. Each same-day group is also recorded as one policy_contribution_application_batchkeyed by (policy_id, mode, effective_date, requested_allocation_mode), uniquely enforced only while the batch is still open (NOT_REQUIRED,PENDING, or WAITING_FOR_REBALANCE) — a later contribution never reuses an already-planning/executed batch and instead gets its own fresh one. Lookup-or-create is concurrency-safe: the insert uses ON CONFLICT ... DO NOTHING against that same open-only key and re-selects the winner's row on a losing race, so two simultaneous requests never raise an unhandled constraint error mid-flow. If the winner's batch is itself claimed past open (a separate, concurrent planning transition) in the gap between that losing insert and the re-select, the re-select can come up empty — rather than surface that as an error, the whole select-or-insert retries fresh (up to 5 attempts), which now sees no open batch and creates its own. It is durable and idempotent, so a retry after a partial failure only applies whatever has not yet been credited.

Getting/creating the open batch and attaching a cash-applied contribution to it are never two separate calls — a caller that did those as independent transactions (get/create batch → record item, as two calls) would release the batch row's lock in between, leaving a real window where a concurrent claim for planning could read and plan from the batch's item list before this caller's own item ever landed in it, stranding that contribution's already-applied cash outside any plan.attach_applied_contribution_to_open_batch (LIVE) and the equivalent single-transaction path inapply_paper_contribution_batch (PAPER) instead hold the batch row locked across BOTH the lookup/create step and the item-attach step, so a concurrent claim can never land in that gap; if the batch is claimed away before this call reaches the attach step, it re-validates and opens a fresh batch generation instead of silently attaching to one that is no longer open.

apply_live_contribution is itself idempotent (an already-APPLIED contribution safely skips cash movement and returns normally), so the dashboard's apply-funding endpoint always continues into the attach step on every call for that contribution — a double-click, a network retry, or a next-day re-submission included.attach_applied_contribution_to_open_batch is written to be safe against exactly that, including SIMULTANEOUS retries of the same contribution: before checking for an existing item it acquires a PostgreSQL transaction advisory lock derived from a stable, dedicated contribution-attachment namespace plus thecontribution_id. That lock is held through the complete validate → check → select/create batch → insert item → update total transaction, so it serializes separate application workers and processes, not merely threads in one Python process. A second concurrent call for the same contribution blocks until the first commits, then reads back the item the first call created instead of racing to attach its own. The locked contribution must match the supplied Policy and PAPER/LIVE mode, be APPLIED, and carry a finite positive confirmed amount; mismatches fail closed before any batch mutation. If an item already exists, that item's own batch is returned outright — the call never re-searches using this retry's own (possibly different-day) effective_date, never inserts a second item, and never re-pointsapplication_batch_id at an unrelated batch that never actually received the item. If the pointer ever differs, the item-to-batch relationship is authoritative and the pointer is repaired without changing the batch total. If cash was applied but no item exists yet (a prior request crashed between crediting cash and attaching), the item is attached using the consumed valuation's persisted as_of timestamp through its uniqueconsumed_by_contribution_id link. That is the durable economic date also used for the contribution'spolicy_cash_flows entry; the cash-flow table itself does not currently carry contribution_id, so it cannot be matched safely when equal-value contributions exist. Only legacy rows without a consumed valuation fall back toapplied_at, and the retry request's fresh date is never treated as recovery evidence.

Duplicate apply-funding requests therefore converge on one cash application, one item, and one authoritative batch. Buy-only mode still uses the batch's atomic planning claim, so only one caller can create its contribution cycle and order plan; a retry reads the already-dispatched state. Next-rebalance mode only marks an open batch as waiting, so it cannot downgrade a batch that another caller has already advanced to planning, approval, execution, or a terminal state. Duplicate requests cannot create an empty replacement batch or duplicate allocation activity.

For LIVE, the batch is dated by the real application date — whenapply_live_contribution actually credits cash and issues units — never the contribution's original mandateeffective_date/scheduled_date, which stay untouched for audit. Confirmation and application are separate human actions that can land a day or more later (deferral, next-morning confirmation, a retry); every downstream buy-only step — current prices, target-age check, the cycle it creates — is dated off that same real application date so it never mixes a fresh cash credit against stale, days-old market prices. That date is read from the SAME fresh valuation's persistedas_of timestamp used to credit the cash, not a separate now-timestamp call made a few lines later — around midnight those two "now"s could otherwise land on different calendar days and silently disagree.

Flow-safe performance and drawdown

Alongside the existing corpus-based drawdown_net, every policy equity row now also carries a flow-safeperformance_index_gross /performance_index_net_of_costs /performance_index_net_of_tax — a daily-compounding index (starting at 10.0, mirroring the research engine's own index) that isolates the day's real market return from any same-day contribution, plus a drawdown_net_of_taxcomputed from that index's running peak. A contribution can never create investment return, reset drawdown, or reduce it — the index only moves with price. The index is deliberately anchored to the prior day's persisted row rather than a same-day-mutable field, so it stays correct even when the contribution worker calls the valuation routine more than once for the same date (pre-flow, then post-flow).

Allocation state machine and deployment modes

Once cash is credited, a contribution's allocation is tracked independently of its (now-terminal) APPLIED cash status: NOT_REQUIRED → PENDING → WAITING_FOR_REBALANCE → PLANNING → PENDING_APPROVAL/EXECUTING → EXECUTED, orFALLBACK_NEXT_REBALANCE / FAILED. A contribution never returns to CONFIRMED once its cash is credited, and an allocation retry never re-issues units or re-credits cash — only the allocation step itself is retried.

next_rebalance (the default) leaves the contribution as Policy cash and waits: the batch sits atWAITING_FOR_REBALANCE until the next normal daily cycle, which links every eligible waiting batch into whatever plan it produces — PAPER executes that plan automatically, LIVE stops at the same human-approval gate every other LIVE plan uses. A cycle that produces no orders (NOOP) leaves the batch waiting rather than losing the link, and a cycle that fails after moving a batch into planning reverts it back toWAITING_FOR_REBALANCE rather than stranding it.

buy_only_current_targets deploys the contribution immediately into the latest valid target portfolio (same strategy/config version, not older than a configurable maximum age, with complete current market prices) — buy-only, sized off current market value (never average cost), respecting whole-share sizing, stock/sector caps, and the batch's own contribution amount. If no valid target source exists, the batch falls back tonext_rebalance (allocation_status = FALLBACK_NEXT_REBALANCE, with a fallback_codeand human-readable fallback_reason) rather than failing outright. As with next_rebalance, PAPER executes automatically and LIVE only ever produces aPENDING_APPROVAL plan. The portfolio-value denominator used to scale each target weight prices every current pick andevery currently-held equity or liquid-fund position — never falling back to a held position's average cost just because it isn't one of today's picks — and fails closed (FALLBACK_NEXT_REBALANCE) if any required price is unavailable rather than sizing against a stale value.

The cycle this creates is tagged cycle_type = CONTRIBUTION_ALLOCATION and identified by the application batch that owns it (contribution_batch_id), never by calendar date — so it can never collide with the normal daily scheduler's NORMAL_REBALANCE cycle for the same(policy_id, cycle_date, mode) (whichever runs first would otherwise attach a second plan to an already-owned cycle, or have its terminal status clobbered by the other side), and two same-day contribution batches for the same policy each get their own cycle too, never sharing one just because they land on the same date. Retry routing respects this split: "retry latest cycle" only ever looks at the latest NORMAL_REBALANCEcycle (both the dashboard's retryable-state annotation and the retry action itself), since that path re-runs the normal daily scheduler workflow (signals, universe diff, planner) — a failedCONTRIBUTION_ALLOCATION cycle is never retried through it, even if it is the most recent cycle row overall.

LIVE final post-flow snapshot

Applying LIVE funding now always ends with a final post-flow equity snapshot: fresh valuation → capture valuation evidence → reserve funding → consume the valuation → issue units and credit cash → transition the reservation toAPPLIED_TO_POLICY → run the allocation dispatch → write the day's final equity row. If the allocation step itself throws, the cash and units already applied are not rolled back — the batch is left in a recoverable, non-terminal allocation state and the final equity row still reflects the real, credited cash position; only the allocation is ever retried afterward.

Funding reservation consumption

A funding reservation now has an evidence-backed path fromAPPLIED_TO_POLICY to CONSUMED:ORDER_EXECUTION (a linked plan's order actually filled — evaluated from real order-level fill status, never from a plan-level label a fully-rejected plan could also carry — and only once the plan's actual net cash deployment covers the linked batch items' full reserved amount; whole-share rounding, minimum-trade-value filtering, and stock/sector caps can all legitimately leave some contribution cash undeployed, and a plan that only partly deploys its reservation is left atAPPLIED_TO_POLICY for manual reconciliation rather than auto-consumed),BROKER_BALANCE_SNAPSHOT, orMANUAL_RECONCILIATION, each recordingconsumed_at, the evidence type, and a reference.

Net cash deployment is read back from policy_trades' own reconciled trade_value (fill_qty × avg_fill_price) per broker_order_id — the broker-reported fill, never the planner's est_value reference price, since a lower actual fill price can leave real cash undeployed even though every order FILLED — and nets any SELL proceeds in the same plan out of the BUY spend, since a normal next_rebalanceplan can contain both and gross BUY value alone would overstate how much of the reserved cash actually left the broker balance. This is deliberately GROSS fill value only, neverpolicy_trades' own transaction_cost/slippage_cost columns added on top — those are this codebase's own MODELLED cost estimate (the same formula used to project costs before a fill even happens), not broker-confirmed charges, and the broker's actual fill price already embodies real market slippage; adding a second modelled figure could push a gross deployment that is genuinely below the reserved amount over the auto-consume threshold.ACTIVE and APPLIED_TO_POLICY reservations reduce unallocated broker capacity; CONSUMED ones do not, since the evidence itself proves the broker-side movement already happened and would otherwise be subtracted twice. A plan rejection never releases or consumes an APPLIED_TO_POLICYreservation — the money stays allocated as Policy cash until real evidence says otherwise.

SIP & Contributions dashboard tab

The Policy detail page has a dedicated, mobile-safeSIP & Contributions tab: the schedule (with pause/resume), the contribution list with allocation status and fallback reason, and — for LIVE policies — funding evidence (broker-verified vs. operator-attested vs. unverified, external reference, manual funding source, valuation freshness, reservation lifecycle). Confirm funding,Apply to Policy, and order-planApprove are three separate, independently confirmed actions and are never combined into one button — each uses its own confirmation dialog with money-specific impact copy. Defer, retry, cancel, and opening a contribution's linked plan are available per contribution. The Policies list page shows concise badges (funding due, pending allocation, schedule needing review, next SIP due date) that link straight into this tab.