* Performance improvements in holding calculation pipeline
Investment accounts with large histories were pegging CPU at 100% during
sync. Root cause was a cluster of quadratic and superlinear algorithms in
the inner holding calculation loop. All are replaced with O(1) hash lookups
built from single-pass indexes over the already-loaded data.
Holding::PortfolioCache - load_prices:
Three O(SxN) patterns inside the per-security loop:
1. DB prices: `security.prices.where(...)` fired one SQL query per
security (N+1). Replaced with a single bulk query before the loop:
Security::Price.where(security_id: ..., date: ...).group_by(&:security_id)
70 securities -> 70 queries becomes 1.
2. Trade prices: `trades.select { |t| t.entryable.security_id == id }`
scanned the full trades array for every security - O(SxT). Replaced
with trades_by_security_id, pre-indexed once from the loaded array.
3. Holding prices: `holdings.select { |h| h.security_id == id }` - same
O(SxH) pattern. Replaced with holdings_by_security_id.
Prices are now indexed into prices_by_date and prices_by_date_and_source
hashes during load_prices, making get_price O(1) instead of scanning the
flat prices array on every lookup.
Holding::PortfolioCache - get_trades / get_price:
- get_trades(date:): `trades.select { |t| t.date == date }` (O(T) scan)
replaced with trades_by_date hash (O(1)).
- get_price: two `prices.select { p.date == date ... }.min_by` linear
scans replaced with direct hash lookups into prices_by_date and
prices_by_date_and_source.
Holding::PortfolioCache - collect_unique_securities:
`holdings.map(&:security)` traversed the security association on every
holding record (N+1 if not preloaded). Replaced with a pluck of
security_ids followed by a single Security.where(id: ...) batch load.
Holding::ForwardCalculator / ReverseCalculator:
`holdings += build_holdings(...)` allocated a new array copy on every
iteration - O(N) per day x thousands of days = O(D^2) total allocations.
Replaced with holdings.concat(...) which appends in place, O(1).
Holding::ReverseCalculator - precompute_cost_basis:
Old: walked every date from account.start_date to Date.current (O(D)),
writing a cost_basis entry for every security on every date. For an
account with 2 trades over 9,250 days this wrote ~18,500 hash entries
and consumed the full date range in the outer loop regardless of trade
density.
New: walks only buy trades (O(T)), appending one [date, avg_cost]
snapshot per trade. cost_basis_for binary-searches the sparse snapshot
array - O(log T) per lookup. Memory drops from O(DxS) to O(T).
Holding::Gapfillable:
`security_holdings.find { |h| h.date == date }` was called on every
date in the gapfill range - O(H) per date, O(HxD) total. Replaced with
security_holdings.index_by(:date) built once before the loop, making
each date lookup O(1).
Holding::Materializer - purge_stale_holdings:
`account.entries.trades.map { |entry| entry.entryable.security_id }.uniq`
loaded all trade entry records into Ruby then traversed the entryable
association on each (N+1). Replaced with account.trades.pluck(:security_id).uniq
(single SQL query returning only the IDs).
In testing, these changes were able to reduce sync time of an account with
25 years of history and 70 securities from about 90 minutes down to under
3 minutes.
* Lint fix
* Lint fix
* addressing the open review nits I agreed with:
* return dup'd arrays from PortfolioCache#get_trades so callers can't mutate memoized cache state
* use the precomputed security-id indexes in collect_unique_securities
* keep security-id dedupe in SQL via distinct.pluck(:security_id)
* tighten the DB price preload to select only needed columns
* harden cost-basis assertions with assert_in_delta
* Back out unnecessary AI slop
* Add back dup to trades array returned from memoized hash
trades_by_date[date] returns a live reference into the memoized hash.
Any caller that mutates the result would silently corrupt the cache for
subsequent calls on the same date within the same sync run. Add .dup to
return a shallow copy, matching the safety of the original select path.
* Feat: Implement manual sync prices functionality and enhance holdings display
* Feat: Enhance sync prices functionality with error handling and update UI components
* Feat: Update sync prices error handling and enhance Spanish locale messages
* Fix: Address CodeRabbit review feedback
- Set fallback @provider_error when prices_updated == 0 so turbo stream
never fails silently without a visible error message
- Move attr_reader :provider_error to class header in Price::Importer
for conventional placement alongside other attribute declarations
- Precompute @last_price_updated in controller (show + sync_prices)
instead of running a DB query directly inside ERB templates
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix: Replace bare rescue with explicit exception handling in turbo stream view
Bare `rescue` silently swallows all exceptions, making debugging impossible.
Match the pattern already used in show.html.erb: rescue ActiveRecord::RecordInvalid
explicitly, then catch StandardError with logging (message + backtrace) before
falling back to the unknown label.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix: Update test assertion to expect actual provider error message
The stub returns "Yahoo Finance rate limit exceeded" as the provider error.
After the @provider_error fallback fix, the controller now correctly surfaces
the real provider error when present (using .presence || fallback), so the
flash[:alert] is the actual error string, not the generic fallback.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix: Assert scoped security_ids in sync_prices materializer test
Replace loose stub with constructor expectation to verify that
Balance::Materializer is instantiated with the single-security scope.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: Assert holding remap in remap_security test
Add assertion that @holding.security_id is updated to the target
security after remap, covering the core command outcome.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix: CI test failure - Update disconnect external assistant test to use env overrides
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* **Add Coinbase integration with item and account management**
- Creates migrations for `coinbase_items` and `coinbase_accounts`.
- Adds models, controllers, views, and background tasks to support account linking, syncing, and transaction handling.
- Implements Coinbase API client and adapter for seamless integration.
- Supports ActiveRecord encryption for secure credential storage.
- Adds UI components for provider setup, account management, and synchronization.
* Localize Coinbase-related UI strings, refine account linking for security, and add timeouts to Coinbase API requests.
* Localize Coinbase account handling to support native currencies (USD, EUR, GBP, etc.) across balances, trades, holdings, and transactions.
* Improve Coinbase processing with timezone-safe parsing, native currency support, and immediate holdings updates.
* Improve trend percentage formatting and enhance race condition handling for Coinbase account linking.
* Fix log message wording for orphan cleanup
* Ensure `selected_accounts` parameter is sanitized by rejecting blank entries.
* Add tests for Coinbase integration: account, item, and controller coverage
- Adds unit tests for `CoinbaseAccount` and `CoinbaseItem` models.
- Adds integration tests for `CoinbaseItemsController`.
- Introduces Stimulus `select-all` controller for UI checkbox handling.
- Localizes UI strings and logging for Coinbase integration.
* Update test fixtures to use consistent placeholder API keys and secrets
* Refine `coinbase_item` tests to ensure deterministic ordering and improve scope assertions.
* Integrate `SyncStats::Collector` into Coinbase syncer to streamline statistics collection and enhance consistency.
* Localize Coinbase sync status messages and improve sync summary test coverage.
* Update `CoinbaseItem` encryption: use deterministic encryption for `api_key` and standard for `api_secret`.
* fix schema drift
* Beta labels to lower expectations
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Refactor: Enhance cash balance calculation and holdings management with money market classification and provider-sourced data handling
* Fix: Clear fixture holdings in test to ensure clean creation and update raw_holdings_payload format
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* Add cost basis tracking and management to holdings
- Added migration to introduce `cost_basis_source` and `cost_basis_locked` fields to `holdings`.
- Implemented backfill for existing holdings to set `cost_basis_source` based on heuristics.
- Introduced `Holding::CostBasisReconciler` to manage cost basis resolution logic.
- Added user interface components for editing and locking cost basis in holdings.
- Updated `materializer` to integrate reconciliation logic and respect locked holdings.
- Extended tests for cost basis-related workflows to ensure accuracy and reliability.
* Fix cost basis calculation in holdings controller
- Ensure `cost_basis` is converted to decimal for accurate arithmetic.
- Fix conditional check to properly validate positive `cost_basis`.
* Improve cost basis validation and error handling in holdings controller
- Allow zero as a valid cost basis for gifted/inherited shares.
- Add error handling with user feedback for invalid cost basis values.
---------
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
- Refactored `persist_holdings` to separate and conditionally upsert holdings with and without cost_basis.
- Updated `avg_cost` logic to treat 0 cost_basis as unknown and return nil when cost_basis cannot be determined.
- Modified trend and investment calculation to exclude holdings with unknown cost_basis.
- Adjusted `average_cost` formatting to handle nil values in API responses and views.
- Added comprehensive tests to ensure cost_basis preservation and fallback behavior.
- Localized `unknown` label for display when cost_basis is unavailable.
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
* Handle reverse calculator starting portfolio generation correctly
* Fix current_holdings to handle different dates and hide zero quantities
- Use DISTINCT ON to get most recent holding per security instead of assuming same date
- Filter out zero quantity holdings from UI display
- Maintain cash display regardless of zero balance
- Use single efficient query with proper Rails syntax
* Continue to process holdings even if one is not resolvable
* Lint fixes
