Portfolio

Consolidated Portfolio Runtime

arb_bot/portfolio_app/ is a FastAPI sub-application that owns the consolidated portfolio runtime: broker connections, holdings, transactions, CSV import/export, and market-data refresh. It is mounted into the authenticated dashboard at dashboard_server.py under /api/portfolio/* behind the same JWT get_current_user dependency used by the rest of the app.

ℹ️ Scope. The backend runtime (routes, services, data_fetcher adapters, DB layer) shipped in Phase B1. Phase B2 added the React frontend under /portfolio/*, and Phase B3 added the optional portfolio Telegram bot and APScheduler sync loop, documented in the sections below.

See also: Connections & Sync Reliability documents the auto-refresh internals — the token-refresher loop, intraday throttle, escalating backoff, the STALLED terminal state, the per-connection advisory lock, and the atomic holdings persist.

Strict isolation from live trading

arb_bot/portfolio_app must never import the live-trading modules arb_bot.bot, arb_bot.execution.engine, arb_bot.command_handler, or arb_bot.client. Importing the shared broker platform (arb_bot.brokers.config_repo / arb_bot.brokers.credentials) is allowed — that is the intentional sharing point for the account/credential store.

The boundary is enforced by tests/portfolio_app/test_import_isolation.py, which (a) statically scans every .py under the package for forbidden imports, and (b) dynamically imports each B1 route, data_fetcher adapter, and the api.services layer and asserts no live-trading module lands in sys.modules transitively.

Routes under /api/portfolio/*

All routers are composed in arb_bot/portfolio_app/api/router.py (prefix /api/portfolio) and require an authenticated session.

Cross-cutting serialization/mapping helpers live in api/services.py, which is also where broker sync is wired to the shared brokers.config_repo account store.

Broker sync adapters (data_fetcher/)

Each broker has a thin adapter that returns normalized holdings. Adapters read credentials through brokers.credentials / brokers.config_repo and never touch the live-trading client.

Shared broker platform

Broker accounts and credentials are not duplicated into a portfolio-private store. Connection reads/writes go through arb_bot.brokers.config_repo and encryption/decryption through arb_bot.brokers.credentials, so the trading bot and the portfolio runtime share one account store (see the Brokers section). The role column (TRADING | SYNC | TRADING+SYNC) distinguishes accounts usable for holdings sync from live-trading accounts. The dashboard Connections tab, portfolio Telegram broker commands, trading startup, token refresh, and holdings sync all use brokers.broker_connections. Dhan sync creates/updates the default portfolio row for the connection and records current-day Dhan tradebook rows as transactions without letting partial-day trades overwrite broker-sourced holdings.

CSV import/export

/api/portfolio/csv/upload accepts an uploaded holdings or transactions CSV, detects the file type, maps canonical fields with case-insensitive aliases, validates every row, and returns an import_id plus a bounded preview. The browser commits by import_id, so the full uploaded file is imported rather than only the displayed preview rows.

Imports default to merge: holdings are upserted by symbol and portfolio, while transactions are inserted idempotently using the existing transaction de-duplication rules. The optional replace portfolio mode requires an explicit target portfolio and deletes only that portfolio's rows for the selected import type before importing valid selected rows. There is no replace-all operation.

When a target portfolio is supplied, every imported row is assigned to that portfolio regardless of the CSV's Portfolio column. Without a target portfolio, row-level portfolio values are preserved. Invalid rows are reported with row number, field, and message; valid selected rows can still be imported and the commit response reports imported, skipped, and replaced counts.

Persistence

The portfolio runtime stores its own tables (portfolios, holdings, transactions) in the shared Postgres instance via arb_bot/portfolio_app/db/ (SQLModel models + repository). It does not read or write the live-trading SQLite store (arb_bot_records.sqlite3) or any trade/go-live state.

Watchlist

The portfolio watchlist is DB-backed: items are stored in the Postgres table portfolio.watchlist_items, modeled by WatchlistItem (arb_bot/portfolio_app/db/models.py) with persistence handled by WatchlistItemRepository (arb_bot/portfolio_app/db/repository.py). The legacy watchlist.json file and the DEFAULT_WATCHLIST constant have been removed, so the watchlist starts empty on a fresh database; load_watchlist_items() in arb_bot/portfolio_app/watchlist.py simply returns an empty list when there are no rows.

The list is editable at runtime through three independent surfaces, all backed by the same repository:

• REST API — GET, POST, and DELETE on /api/portfolio/watchlist (arb_bot/portfolio_app/api/routes/watchlist.py), protected by the same JWT dependency as the rest of the API.
• Dashboard — the Watchlist page at /portfolio/watchlist supports search, add (via a drawer with all fields), and remove.
• Telegram — /watchlist_add (guided add) and /watchlist_remove (remove by symbol); the existing /watchlist command stays read-only.

The scheduler job morning_watchlist_intelligence and the reports read these rows, so with an empty watchlist they simply produce an empty digest until items are added through any of the surfaces above.

Config

Runtime knobs live in arb_bot/portfolio_app/config.py (PortfolioRuntimeConfig) and are injected into the route handlers. See the Config Reference for the full list.

Portfolio Telegram bot (Phase B3)

arb_bot/portfolio_app/telegram_app.py runs a second, independent python-telegram-bot Application — separate from the trading bot's one-way send_telegram sender. It is strictly opt-in: it only starts when PORTFOLIO_TELEGRAM_ENABLED=1 and a bot token is configured. start_portfolio_telegram_in_thread() launches it in a daemon thread with its own asyncio loop, so a portfolio-runtime crash never takes down trading.

The launch is wired into ArbitrageBot.run() immediately after _setup_schedules(), behind a lazy import (there is no top-level portfolio_app import in bot.py) wrapped in try/except. If it is disabled or fails, the trading bot continues unaffected.

Env vars (read by PortfolioRuntimeConfig.from_env()):

Commands (registered in build_application(); the menu is published via set_my_commands): portfolio overview (/start, /help, /complete_status, /live_status, /chart); quick wins (/top_gainers, /top_losers, /dividends, /holding, /sectors); analytics (/benchmark, /allocation, /currency, /risk, /rebalance, /sip, /bonds, /health, /report, /watchlist, /stock_news); price alerts (/alert, /alerts, /delete_alert); broker connections (/add_connection, /sync_connection, /edit_connection, /list_connections, /delete_connection); transactions (/add_transaction, /remove_symbol); and data management (/clear, /format, plus document upload). Broker sync through these commands is read-only — it never places orders.

⚠️ Authorization. The portfolio bot has no command-level allowlist in source: anyone who reaches the bot can run its commands. Treat the token as a secret. Chat-id lockdown is a net-new feature, out of scope for this consolidation.

Scheduler (Phase B3)

arb_bot/portfolio_app/scheduler.py (PortfolioScheduler) is an APScheduler AsyncIOScheduler pinned to Asia/Kolkata. It runs inside the portfolio runtime's own asyncio loop (the daemon thread above) and pushes proactive reports and alerts.

It coexists with the trading bot's separate schedule loop in ArbitrageBot.run(): the two never share a scheduler, thread, or event loop, and the portfolio scheduler is only created when the Telegram runtime starts.

Frontend (Phase B2)

The React dashboard ships a portfolio surface under frontend/src/portfolio/, rendered inside the authenticated app shell (PortfolioLayout) behind the same cookie session used by the rest of the dashboard. The standalone OTP login is gone; the portfolio pages call /api/portfolio/* with the session cookie.

Routes (nested under /portfolio):

Portfolio Workspace Foundation

The authenticated /portfolio route now uses a dedicated nested application shell instead of rendering the trading, research, strategy-docs, and operations navigation while a user is managing wealth data. Its primary information architecture is Overview, Holdings, Activity, Goals, Insights, Accounts, and Settings. A deliberate link returns to Trading & Research. Existing portfolio URLs remain available, including the legacy report redirects.

The Overview reads one coherent response fromGET /api/portfolio/v2/dashboard. The response has onecalculated_at timestamp and includes totals by original currency, an estimated converted total, per-portfolio summaries, allocation, snapshot history, goal highlights, sync state, freshness, warnings, and disclosures. Missing prices and previous closes are represented as unavailable or partial; they are not converted to believable zero gains or zero daily movement.

Holdings responses now include a backend-calculated reconciliation contract: broker quantity, ledger-calculated quantity, difference, status, reason, and calculation time. The dashboard only explains discrepancies; reconciliation never mutates holdings or transaction records automatically.

Frontend calls flow through frontend/src/portfolio/api/client.ts, which normalizes errors, applies timeouts and abort signals, emits request IDs, retries only safe transient reads, and never retries destructive writes.queryClient.ts adds a small cache and explicit invalidation for read models. The critical dashboard response is runtime validated.

Known financial-data limits. This foundation does not yet replace the legacy transaction uniqueness constraint, symbol/provider heuristics, binary-float storage, or the lack of transaction-level historical FX. Converted multi-currency totals are therefore labeled estimated. Seedocs/portfolio-product-audit.md anddocs/portfolio-product-roadmap.md for the evidence and additive migration sequence.

Dhan trade history

Dhan portfolio sync uses the paginated Statement Trade History API, GET /v2/trades/{from-date}/{to-date}/{page}, to import historical equity transactions for the synced portfolio. Imported trade symbols are normalized back to holding symbols by ISIN/security id when Dhan returns display names in trade history. Dhan can mark some equity rows as INTRADAY even when they are needed to reconcile the current demat holding quantity; sync keeps those rows only when they close a holding quantity gap, while still excluding derivatives and unrelated intraday rows. Sync uses idempotent inserts and does not delete existing transaction rows, so manual correction entries for broker gaps are preserved on later syncs. Dhan holdings do not include LTP, so connection sync preserves an existing refreshed market price instead of overwriting it with average-cost fallback. The older order tradebook endpoint, GET /v2/trades, only covers the current trading day and is not used for portfolio transaction backfill.

Zerodha transaction sync

Zerodha sync imports holdings from Kite Connect and stores Kite's latest price fields (last_price / close_price) as portfolio LTP / previous close so a connection resync does not zero out Portfolio P&L before the separate market-data refresh runs. It also imports the transaction rows Kite Connect exposes: current-day NSE/BSE equity executions from GET /trades and completed mutual-fund orders from GET /mf/orders for the API's seven-day MF order window. Kite Connect does not expose the full historical Console transaction ledger, so older Zerodha transactions still require CSV/statement import if they were not captured by a prior sync.

IBKR Flex sync

IBKR sync uses Activity Flex XML only. Save the Activity Flex query_id and token on the IBKR connection; the Open Positions section imports holdings, preferring explicit per-share fields such as markPrice, closePrice, costBasisPrice, and averageCost before falling back to aggregate position values. When the Flex query includes the Trade section, stock trades are imported into the transaction ledger as BUY/SELL rows with USD currency and IBKR commission as Fee_Tax. Options are skipped until multiplier-aware option holdings are modeled. IBKR Activity Flex data is statement data, so the maintenance loop treats IBKR as a once-daily broker sync and skips repeated syncs after a successful sync on the same date. If the daily sync fails, the next automatic retry is scheduled one hour later. IBKR can return broker error 1001 while a statement is not yet generated; the hourly retry handles that without continuous polling.

MOSL Rise sync

MO_RISE is a sync-only connection type for Motilal Oswal accounts whose demat sits with a third-party DP (for example an IDFC First-linked account): the official MO OpenAPI broker connection only exposes holdings held in MOSL's own DP and returns NO RECORDS FOUND for these accounts. MO_RISE talks to MOSL's "Rise" web-app backend (api.riseapp.in / invest.motilaloswal.com) instead, and takes user_id + pin as credentials — there is no API key/secret pair, unlike the official MO broker.

Sync first attempts a PIN-only login using a device_id stored on the connection from a prior successful login. If MOSL rejects that login because device trust has lapsed, the adapter raises NeedsOtpError, which ConnectionSyncService turns into a NEEDS_OTP sync-state reason code instead of a hard failure and persists the exception's session cookies, anti-forgery token, and MOSL's otpToken into the connection's (encrypted) credentials under _otp_session — confirmed necessary because MOSL's OTP-validation endpoint correlates purely via session cookies, with nothing in its own request tying it back to a specific login attempt, so verify_otp must resume the exact session login was using, potentially minutes later. A live sync attempt that comes back needs_otp opens a modal directly (no separate error banner) prompting for the code, with resend; a connection that already carries NEEDS_OTP state from a prior session shows a "Verify OTP" row action that reopens the same modal. On successful verification the routes clear _otp_session and persist the new device_id, and the Connections page immediately re-triggers a sync for that connection rather than waiting for the operator to click Sync again.

mo_rise_crypto.py ports MOSL's own client-side encryption (recovered from the app's public, unminified BindEncryptor.js): a static-key AES scheme (encrypt_static_value/decrypt_static_value) used for risebe_pilot's enc= route parameters and, applied directly to a JWT string, for the reftkn/Authorization header — both confirmed byte-for-byte against real captured traffic — and an RSA+AES hybrid used for the Home/GetLoginUserType/Home/RiseAuthorise/Home/ValidateOTPRise endpoints. get_holdings, login, and verify_otp are all fully implemented and confirmed live against the real server: login's identify call goes to Home/GetLoginUserType (an earlier version used Home/RiseAuthorise for identify, matching one real capture, but a live test showed RiseAuthorise returns a generic data=null/"Something went wrong in method." error for identify — GetLoginUserType, from a separate real OTP-flow capture, is the one that actually works), while PIN submission still goes to Home/RiseAuthorise (confirmed correct by the same live test, which got past identify and triggered a real OTP send). Both steps are correlated purely via ASP.NET session cookies plus a __RequestVerificationToken anti-forgery header, not a bearer token — confirmed from a captured curl of a real PIN-submission request, and _bootstrap_session's ASP.NET MVC convention (GET the homepage, read the antiforgery value out of a hidden input) is now confirmed working live as well. When OTP is required, a confirmed real capture showed the full chain: identify (returns an otpResponse with an otpToken) → submit OTP to Home/ValidateOTPRise → submit PIN to Home/RiseAuthorise (PIN is required after OTP, not instead of it) → the same session-bundle redirect the PIN-only path reaches. The PIN-submission response's tokenInfo.accessToken/refreshToken JWT bundle (confirmed via a decrypted client-side redirect URL) is used to build the session: reftkn/Authorization (for holdings) are accessToken re-encrypted with the static-AES scheme, and appid is read from the token's own claim. get_transactions needs a completely different pair of headers — a static, app-wide xapikey (TRANSACTIONS_XAPIKEY, confirmed both as a literal constant in the Rise app's own client-side JS and matching a live captured request) plus Authorization: Bearer <accessToken> using the raw (unencrypted) JWT, not the encrypted reftkn. Both are populated on the session automatically by the same login/verify-OTP code path. Missing the Authorization header here was a real production gap discovered after login/holdings started working live: transactions had never actually been fetched successfully. services.py's transactions fetch calls get_all_transactions, not get_transactions directly — a single call only returns the first page (default 20, matching a real captured curl's own pagesize), silently truncating any account with more history; get_all_transactions loops pages while the previous one came back full, stopping on the first short/empty page (no captured response has ever carried a total-count field to key off instead), capped by a max_pages safety limit. Missing this loop was a real production gap too: an account's holdings quantity and its transaction-derived quantity didn't reconcile because older transactions were silently missing. A third real production gap, reported by a user even after the pagination fix: MOSL Rise uses genuinely different symbol naming across its own endpoints for the same security — holdings' tradeSymbol is the compact NSE ticker (e.g. ATHERENERG), transactions' shortname is a display name plus segment suffix (e.g. ATHER ENERGY-EQ), with no simple string transformation between them. ISIN and MOSL's internal scripCode/scriptcode (populated on both parsers as ISIN/Security_ID) were both tried as a join key through the existing, broker-agnostic ConnectionSyncService._align_transaction_symbols_to_holdings (already used by DHAN) but confirmed, against a real live account (via a diagnostic log added specifically to get this evidence), to be a dead end for MOSL Rise: holdings never carries ISIN at all, and holdings' scripCode/transactions' scriptcode come from two entirely unrelated MOSL-internal numbering systems. mo_rise_sync.realign_mo_rise_transaction_symbols replaces that approach for MO_RISE specifically, with two tiers: prefix-matching the normalized (letters-only) transaction name against current holdings' symbols (auto-handles tickers that are a straight truncation of the company name, e.g. ATHERENERG/"ATHER ENERGY-EQ", with no manual entry needed), falling back to a small, explicitly-maintained KNOWN_SYMBOL_ALIASES dict for tickers no algorithm can derive — acronym-style tickers (Computer Age Management Services → CAMS, Tamilnad Mercantile Bank → TMB) or corporate-action renames (HBL Power Systems → HBL Engineering, confirmed by the user) — add a new entry there whenever a newly-bought security doesn't auto-align. Fail-safe either way: a transaction symbol matching neither tier (most commonly a fully-exited position with no current holding) is left as-is rather than being guessed at. _extract_data_field unwraps MOSL's occasional double-JSON-encoded responses and surfaces MOSL's own generic error object (allowlisted fields only — never the full response, which can carry PII or session tokens) instead of a bare crash. request_otp (resend) remains a stub — the one captured OTP flow only validated the originally-sent code and never triggered a resend, so that endpoint is still unconfirmed. In the data_fetcher/ adapter set, mo_rise_client.py is the raw HTTP client for the Rise backend and mo_rise_sync.py parses its holdings/transactions responses into the unified DataFrame shapes HoldingRepository and TransactionRepository expect, alongside the existing mo_sync.py + mo_api.py pair used by the official MO broker.

Automatic broker sync

The deployed token-refresher service is also the portfolio maintenance loop. It wakes every five minutes, refreshes due broker tokens, then runs due portfolio syncs. Dhan and Zerodha holdings sync automatically every 30 minutes for SYNC and TRADING+SYNC rows. IBKR Activity Flex sync runs once per day after a success and retries every hour after a failure until the daily statement imports successfully. Dhan token refresh keeps the existing 23-hour maximum token age and 4-hour-before-expiry renewal guard; Zerodha token refresh continues to use the backend TOTP OAuth lifecycle before token expiry.

Automatic price refresh

The Portfolio Overview, Portfolio list, and Portfolio Detail pages call /api/portfolio/market-data/refresh when opened and then every five minutes while the page remains visible. The endpoint first asks the active TRADING+SYNC broker to quote the holding when that broker can resolve the symbol, then falls back to the public PriceEngine. That keeps portfolio cards on broker-fed current prices when a trading connection is available, without requiring a manual button click.

The public yfinance fallback tries NSE before BSE for INR equities and includes conservative aliases for renamed exchange symbols, such as pricing legacy TATAMOTORS rows from TMPV after the 2025 Tata Motors demerger.

CSV samples and inline fixes

The CSV import wizard offers downloadable sample files for transactions and holdings. Transaction CSVs are the preferred import path because TransactionRepository.upsert_from_df(..., sync_holdings=True) recalculates holdings after accepted BUY, SELL, DIVIDEND, and SPLIT rows are imported. Direct holdings CSV import remains available for manual holdings maintenance.

Transaction imports support asset-class classification before commit. The mapping step includes an optional Asset_Class transaction field for row-level classification, and a default asset-class selector fills rows where the CSV has no mapped asset-class value. This is how mutual-fund transaction CSVs can be imported as transactions while the replayed holdings are created or updated as Mutual Fund instead of the default Indian Equity.

Transaction CSVs can also map ISIN. When imported rows replay into derived holdings, the holding stores that ISIN, allowing mutual-fund NAV refresh to resolve the scheme through the ISIN map before falling back to fund-name search.

Unlisted holdings use manual valuation rather than public market-data lookup. The Holding Detail page shows an Update LTP action for Unlisted holdings; each update writes the new current_price, moves the prior value into previous_close, and records a row in portfolio.holding_price_updates with valuation date, prior price, source, and note. Automatic market-data refresh skips unlisted holdings so manual valuations are not overwritten by ticker collisions or unavailable public prices.

Income ledger

The portfolio runtime tracks dividends and bond coupons in portfolio_income_events, exposed through /api/portfolio/income/events, /api/portfolio/income/summary, and /api/portfolio/income/recompute. Events store symbol, portfolio, asset class, currency, ISIN, record/ex/payment dates, amount per unit, entitled quantity, gross amount, withheld tax, net amount, status, source, confidence, and source reference. The dashboard adds /portfolio/income for a consolidated one-year paid and next-twelve-month expected view, and Holding Detail shows the same symbol-scoped income rows next to transactions.

Entitlements are replayed from the transaction ledger at the record date, with ex-date as a fallback when no record date exists. Listed stocks and ETFs use T+1 settlement before entitlement: a BUY counts only when its settlement date is on or before the record/ex entitlement date, and a SELL before that date reduces entitlement only when it has settled by that date. Same-day record-date sells do not remove entitlement. This preserves cases such as holding 2 shares on a 20 June record date and buying 4 more on 26 June: the 28 June payment is counted on 2 shares, not 6.

Broker dividend cashflows imported as DIVIDEND transactions reconcile into paid income events with BROKER source and preserve explicit broker entitlement when the event has no corporate-action dates. /api/portfolio/income/recompute also infers bond coupon schedules from ISIN metadata, face value, coupon rate, payment frequency, and payment anchors, producing source-ref keyed FORECAST / BSE_ISIN / INFERRED events. Missing or zero coupon metadata is ignored rather than creating misleading income rows.

The Income page exposes a Backfill action that calls POST /api/portfolio/income/recompute with declared-dividend refresh enabled. It reconciles existing broker dividend transactions, refreshes declared stock/ETF dividend metadata, converts holding-level next-dividend fields into future income events, cancels stale holding-derived forecasts whose payment dates have passed, enriches bond-like broker symbols from BSE/BondsIndia when a high-confidence ISIN match exists, refreshes entitlement quantities, and rebuilds inferred bond coupon rows without duplicating source-ref keyed events. Ambiguous or unresolved bond symbols are left unchanged for manual review rather than guessed. The page also groups all non-cancelled income rows by payment month in a monthly calendar, keeping separate INR/USD totals when a month contains multiple currencies.

CSV imports also ensure a matching portfolio definition exists for every imported portfolio name. The v2 portfolio list repairs older holdings-only or transactions-only custom portfolios on read, so imported custom portfolios appear on the Portfolio tab even when they were created outside broker sync.

Deleting a portfolio from the v2 Portfolio tab deletes that portfolio definition plus all mapped holdings, transactions, and snapshots for the same portfolio name. The name-based Networth aggregate remains protected.

/api/portfolio/csv/upload now returns the first 25 preview rows plus issue_rows for every invalid row in the uploaded file. Step 3 groups errors by original CSV row and lets the user edit mapped fields inline. The browser sends those corrections as row_overrides to /api/portfolio/csv/commit; the backend applies the overrides to the cached upload before normalization and validation, so server-side validation remains authoritative.

Snowball exports are treated as transaction-ledger imports: BUY, SELL, and SPLIT rows are imported, split rows that store the ratio in Price are normalized to the internal split quantity, and zero-price BUY rows are accepted for bonus shares. Cash-only and custom metadata rows such as CASH_IN, CASH_OUT, CUSTOM_HOLDING_PRICE, and CUSTOM_HOLDING_SETTINGS are ignored because they do not represent held units. Replaying transactions removes existing holding rows when the reconciled quantity reaches zero.

Transaction replay uses a stable same-day ordering for end-of-day portfolio reconciliation: splits and buys are applied before sells, then dividends. This handles Snowball rows where a same-day sell can appear before its earlier buy in CSV order.

Snowball adjustment rows with negative quantities are replayed after normal trades so imported balancing entries can close out stale positions. Identical execution rows in one CSV batch are aggregated before insert, preserving cases where Snowball exports repeated same-symbol/date/price sell rows.

Step 3 also supports bulk selection and import-only ignores: select all preview rows, select all error rows after fixing them, clear the current selection, ignore one error row, or ignore every error row. Ignore only excludes rows from the current import; it does not mutate the source CSV or delete persisted portfolio data.

Goals & SIP Plans

A Goal is a target the user is tracking progress toward, stored in the Goal table (arb_bot/portfolio_app/db/models.py). There are two goal types: CORPUS goals target a lump-sum corpus amount by atarget_date, inflation-adjusted via inflation_pct and grown atexpected_return_pct; INCOME goals target atarget_monthly_income figure with no date, measured against holdings' projected monthly income (dividends/coupons). Progress calculations live inarb_bot/portfolio_app/goals.py (compute_corpus_progress,compute_income_progress), which is pure calculation code with no DB or network access, mirroring sip.py / rebalancer.py / risk.py.

SIP Plans (SipPlan table) are explicit, forward-looking recurring investment commitments — a monthly (or quarterly) contribution amount, astart_date, and an optional step-up schedule (step_up_pct applied every step_up_frequency period). A SIP plan whose start_date is in the future only starts contributing to a goal's projected corpus from that date onward;sip_future_value() skips any projected monthly contribution date that falls before start_date. SIP Plans are independent of the existing transaction-history detect_sips() heuristic in sip.py, which infers likely recurring purchases from past transactions rather than recording an explicit forward-looking commitment.

Goals and SIP Plans link to portfolios, holdings, and each other through apercentage-split linking model: the GoalLink junction table lets a Portfolio, Holding, or SipPlan link to multiple goals at once, each link carrying its own percentage (0–100) of that resource's value counted toward the linked goal. Linking is capped so one resource's percentages across all of its goal links sum to no more than 100% —GoalLinkRepository.link() raises (surfaced as a 409) when a new or updated link would push a resource over that cap. When a holding is linked both directly (aHOLDING-type link) and indirectly through a PORTFOLIO-type link to the portfolio it belongs to, the holding is only counted once toward progress — the direct link wins and the portfolio-derived copy of that same holding is skipped (linked_holdings_with_pct() in api/routes/goals.py, shared by the REST progress calculation and the /goals Telegram command).

Routes (arb_bot/portfolio_app/api/routes/goals.py, mounted under /api/portfolio/goals): GET/POST /goals list and create goals; GET/PUT/DELETE /goals/{id} read, update, and delete a single goal (every read response includes a computedprogress block); GET/POST /goals/{id}/linkslist and create percentage-split links for a goal; andDELETE /goals/{id}/links/{link_id} removes one link. Creating a link validates that link_id resolves to a real holding/portfolio/SIP-plan row of the claimed link_type, returning 404 rather than silently succeeding when it does not. SIP Plans have their own CRUD surface atGET/POST /api/portfolio/sip-plans andGET/PUT/DELETE /api/portfolio/sip-plans/{id}.

Telegram: /goals lists active goals with each one's progress percentage (and projected corpus or projected monthly income, depending on goal type);/sips lists active SIP plans with their current monthly contribution amount and next due date. A daily scheduler job, daily_sip_reminders, runs at 08:00 IST and sends two independent kinds of reminders: a due-date reminder as a plan's next contribution date approaches, and a separate step-up reminder when a plan's contribution amount is about to increase per its step-up schedule.

Frontend: the dashboard ships three pages underfrontend/src/portfolio/ — /portfolio/goals (goal list with progress bars), /portfolio/goals/:id (goal detail, including its linked resources and percentage splits), and /portfolio/sip-plans (SIP plan list with next due dates and step-up schedules).