mirror of
https://github.com/we-promise/sure.git
synced 2026-05-07 21:04:12 +00:00
d74e9caf7b
* Performance and bug fixes in provider price fetches
Three distinct bugs caused the price provider API to be called unnecessarily
on every investment account sync.
1. Sold securities triggered a provider call on every sync forever
import_security_prices passed end_date: Date.current for every security
ever traded. Security::Price::Importer short-circuits via all_prices_exist?
only when persisted_count == expected_count, where:
expected_count = (clamped_start_date..Date.current).count
This range increases daily, so a security closed two years ago would have
all historical prices in the DB unnecessarily. This also causes any closed
securities to fetch prices daily, forever.
Fix: separate currently-held securities (end_date: Date.current) from
historical-only securities (end_date: last holding date for that security).
Once a closed position's price range is complete through its last holding
date, all_prices_exist? becomes permanently stable and no further provider
calls occur for that security.
"Currently held" is defined as appearing in account.current_holdings, which
returns the most recent holding per security with qty != 0. On the first
sync after a sell, the pre-sale holding is still the most recent, so the
security correctly receives end_date: Date.current for one final sync before
the new qty=0 holding is materialised.
2. Offline securities were not filtered
account.trades.map(&:security) returned all securities regardless of the
offline flag. This results in fetching of securities even if the provider
cannot serve them, or if the user don't want them served for some reason
(eg when there are symbol collisions that causes the wrong prices to be
returned) The global MarketDataImporter correctly uses Security.online;
the account-scoped importer did not.
Fix: Security.online.where(id: all_security_ids) matches the established
contract. Offline IDs still pass through the pluck step but resolve to nil
in the securities hash and are skipped by the existing `next unless security`
guard.
3. N+1 queries for security loading and per-security start dates
- account.trades.map(&:security): triggered one SQL query per trade to load
the security association (N+1).
- first_required_price_date(security): issued 2 DB queries per security -
one MIN(entries.date) and one EXISTS - so S securities = 2S queries.
Fix: replace with batch queries totalling 4 regardless of security count:
- account.current_holdings.pluck(:security_id) - current security IDs
- account.trades.pluck(:security_id).uniq - traded security IDs
- Security.online.where(id: ...) - load all security records at once
- batch_first_required_price_dates: one GROUP BY security_id MIN(entries.date)
over trades, one pluck for provider-holding security IDs, one GROUP BY
security_id MAX(date) over holdings for historical end dates
* fix(market-data-importer): fetch prices through today for reopened positions
Account::Syncer runs import_market_data before materialize_balances, so
current_holdings reflects the last materialized snapshot rather than the
current trade state. If a security was previously sold (stale holdings show
qty=0) and then repurchased in the same sync cycle, it landed in
historical_ids and had its end_date capped at the old last_holding_date.
This caused all_prices_exist? to short-circuit, skipping the price fetch
through today, and leaving the forthcoming holding materialization without
a price for the repurchase period.
Fix: compare the latest trade date against the last holding date for each
historical security. If the trade is newer, the position was reopened before
holdings were rematerialized; treat end_date as Date.current for that sync.
The cap still applies on subsequent syncs once materialize_balances has
updated the holdings table.
Adds a regression test covering the repurchase scenario.
* hoist account.start_date out of per-security loop
Account#start_date issues SELECT MIN(date) FROM entries on every call.
Inside batch_first_required_price_dates it was called up to twice per
security (holding_date assignment + fallback), producing up to 2N extra
queries for an account with N provider-held securities.
Cache the result in account_start_date before the loop.
* assert offline securities are skipped
Adds a regression test verifying that Account::MarketDataImporter never
calls fetch_security_prices for a security with offline: true, covering
the Security.online filter on line 54 of the importer.