* feat(splits): add excluded attribute support for split children and improve rendering of split transactions
* address coderabbitai suggestions to improve code quality
* Fix split excluded coercion, DRY helpers, and clean up view partials
Fix boolean coercion bug where string "false" from form params was
truthy in Ruby, causing all split children to be marked excluded.
Use ActiveModel::Type::Boolean for explicit casting in Entry#split!.
Additional changes addressing code review feedback:
- Extract duplicated in_split_group logic from TransactionsController
and TransactionCategoriesController into TransactionsHelper
- Remove redundant local_assigns.fetch calls in partials that already
declare defaults via the Rails 7.1 locals: magic comment
- Simplify ternary in _transaction.html.erb to pass grouped directly
- Guard hidden_field_tag :grouped to only emit when value is "true"
- Add model tests for excluded on split children (boolean and string)
- Add controller test for excluded param through full HTTP stack
- Add test confirming excluded children are dropped from balance queries
* fix(splits): simplify excluded attribute boolean check
* refactor(splits): extract truthy values constant for excluded check
Extract the array of truthy values used for excluded attribute check
into a private constant to improve code maintainability and avoid
duplication of the magic array.
* refactor: simplify split grouping link generation and add test coverage for excluded split parameters
* feat(enable-banking): safe pending transaction merge with sync re-import prevention
* preserve all merged pending IDs across syncs
* fix(enable-banking): harden merge locking, safe logging, and non-blocking index
* fix(enable-banking): use safe external ID in invalid currency log
* refactor(models): centralize pending transaction SQL logic
Move the SQL fragment used to identify pending transactions from the `Entry` model to a constant in the `Transaction` model. This improves maintainability and ensures that the logic for determining if a transaction is pending is defined in a single location.
* fix(enable-banking): drop dead manual_merge index, use lateral join for excluded IDs
* No net schema changes
---------
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* feat(exports): preserve transfer decisions
* fix(api): apply transfer date filters to both sides
* fix(api): refine transfer decision handling
* fix(api): align transfer decision schemas
* fix(api): use current context for transfer filters
* fix(api): include either side in transfer date filters
* fix(api): deduplicate transfer decision filters
* fix(api): guard transfer decision exports
* perf(accounts): kill sidebar/sparkline N+1s and cache the sidebar
The dashboard was issuing hundreds of per-account `SELECT 1` and
polymorphic `accountable` lookups on every page load. Sidebar render
alone hit the DB ~50–100× and ran twice per request (mobile + desktop).
Changes:
- AccountableSparklinesController: short-circuit
`requires_normalized_aggregation?` to Investment/Crypto only and
collapse the per-account `linked?` loop into a single `EXISTS`. Kills
the N+1 `AccountProvider Exists?` queries on every sparkline endpoint.
- BalanceSheet::AccountTotals#visible_accounts: preload `:accountable`,
`:plaid_account`, `:simplefin_account`, and
`account_providers: :provider` so the sidebar's
`account.subtype` / `account.linked?` / `account.provider` calls don't
trigger per-row polymorphic loads.
- AccountsController#index: same preloads on `@manual_accounts`.
- accounts/index/_account_groups.erb: extend the existing `Preloader`
call to batch-load accountable + provider associations so the
per-provider-item partials (Plaid, SimpleFIN, Coinbase, etc.) stop
re-issuing N+1s when rendering account rows on /accounts.
- accounts/_account_sidebar_tabs.html.erb: wrap the partial in a
`cache` block keyed on the family's data-version, the current user,
shares fingerprint, locale, mobile flag, active tab, and a
path-derived "current account" component (`sidebar_active_account_id`
helper). The sidebar is rendered on every page in the layout
(twice — mobile + desktop drawers), so most navigations now serve
the cached fragment instead of re-walking accounts/balances.
Local impact (DZG family, 23 accounts, 6.1k transactions):
- Dashboard `/`: ~6.5s → ~1.95s
- /accounts: ~2.7s → ~0.85s on warm cache
- /accountable_sparklines/*: per-request N+1s eliminated; remaining
cost is request boilerplate which can be addressed by bumping
`RAILS_MAX_THREADS` (the dashboard fans out 5 sparkline turbo frames
in parallel and Puma's default 3 threads serialize them).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(perf): address PR review on sidebar/sparkline perf changes
- AccountableSparklinesController#requires_normalized_aggregation?
also matches legacy plaid_account_id / simplefin_account_id links,
not just new-style account_providers, so investment/crypto accounts
in the legacy linking state still get LinkedInvestmentSeriesNormalizer
applied (Codex P1 / CodeRabbit major).
- Sidebar share fingerprint includes both `count` and `max(updated_at)`
so deleting a non-most-recent AccountShare invalidates the cached
fragment for users who lost access (Codex P1).
- Move the sidebar cache-key construction (incl. the AccountShare
query) from the ERB into a new `account_sidebar_tabs_cache_key`
helper, per the project's "no heavy logic in ERB" rule (CodeRabbit).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(perf): address human review on perf PR
- Account.linked: new SQL-level scope mirroring `Account#linked?` so
the controller and per-instance method share one definition. Removes
the duplicated raw SQL string in
`AccountableSparklinesController#requires_normalized_aggregation?`,
which now reads `accounts.linked.exists?` (jjmata, sure-design).
- AccountsHelper: move `sidebar_active_account_id` and
`account_sidebar_tabs_cache_key` out of `ApplicationHelper`. The
cache-key helper also collapses the AccountShare `count` + `max(updated_at)`
fingerprint into a single `pick` query so we don't pay two round-trips
on every render (jjmata, sure-design).
- test/models/account/linkable_test.rb: pin the `Account.linked` scope
against all three link types (account_providers, legacy plaid_account,
legacy simplefin_account) so any future schema change that diverges
the SQL definition from `linked?` breaks a test instead of silently
serving wrong sparkline aggregations (sure-design).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(perf): correct shares cache fingerprint on raw-SQL pick
`pick(Arel.sql("count(*), max(updated_at)"))` passes a single comma-
separated fragment, which Rails returns as a String (per the documented
behavior of `pluck` with SQL fragments). The previous `max_at&.to_i`
silently truncated `"2025-05-06 12:34:56.789 UTC"` to `2025`, so the
sidebar cache key would not change for share `updated_at` movements
within the same calendar year — including share deletions — leaving
revoked users with a stale sidebar until the 12h expiry.
Pass the aggregates as two separate `Arel.sql` args and just concatenate
the raw String values into the cache key. The values only need to be
stable for a given DB state, not numerically meaningful.
Caught by CodeRabbit on PR #1683.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(api): allow creating categories via API
Adds POST /api/v1/categories so external integrations (e.g. bulk
classification scripts that import already-categorized data from
another system) can create categories without going through the web UI.
Mirrors the existing tags create endpoint: requires the read_write
scope, accepts name/color/icon/parent_id, auto-suggests an icon when
omitted, and rejects parent_ids from other families.
Also adds Minitest behavioural coverage, an rswag docs spec, a
CategoryCreateRequest schema, and regenerates docs/api/openapi.yaml.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(api): address review feedback on POST /api/v1/categories
- Re-raise ActionController::ParameterMissing in #create so the
BaseController rescue_from handles it as a 400 instead of the
generic 500 from the broad rescue inside the action.
- Add a 403 'insufficient scope' response block to the rswag POST
example so the generated OpenAPI documents read-only key rejection.
- Switch the new create-action Minitest cases to API key auth via
X-Api-Key + api_headers (using the existing api_keys fixtures),
matching the project's API endpoint consistency rule.
- Add Minitest coverage for two more 4xx paths: rejecting third-level
nesting (parent_id pointing at a depth-2 subcategory) and rejecting
requests without the category payload (400).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(test): migrate categories API index/show tests to X-Api-Key
The pre-existing index and show tests in this file authenticated via
Doorkeeper bearer tokens. Per the project's API endpoint consistency
rule (CLAUDE.md, .cursor/rules/api-endpoint-consistency.mdc) Minitest
controller tests under test/controllers/api/v1/ must use ApiKey +
X-Api-Key auth. Drops the Doorkeeper application/access-token setup
and routes every request through the existing api_keys fixtures and
the api_headers helper, matching the create-action tests already in
this file (and the pattern used in sync/users/family_settings tests).
No behavioural change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(api): address second-round review on POST /api/v1/categories
- Add a 400 response block to the POST rswag example so the generated
OpenAPI documents the missing-category-payload contract that
BaseController#handle_bad_request already returns. Regenerate
docs/api/openapi.yaml.
- Replace fixture-backed read_write_api_key / read_only_api_key
helpers with explicit ApiKey.create! calls (matching the pattern in
sync_controller_test, users_controller_test, and
family_settings_controller_test). Setup now destroys active keys for
the test user so the one-active-key-per-source validation does not
collide with fixtures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(api): tighten 422 create-category cases
- Pass color and icon explicitly in the duplicate-name and
third-level-nesting tests so each case is self-documenting about
which validation it isolates (the model's color presence check is
satisfied by the column default today, but reviewers — human and
bot — flagged the implicit reliance).
- Assert the JSON error envelope (error key + present message) on every
422 path so the response shape stays consistent and a regression in
the rendered error body is caught uniformly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(api): tighten POST /api/v1/categories per review
- Drop the no-op `rescue ActionController::ParameterMissing; raise` and
the broad `rescue => e` from the create action. The BaseController
already has rescue_from ActionController::ParameterMissing → 400, and
unexpected exceptions are best left to Rails' default 500 handling
(which logs identically). Keeps the action focused on its happy path
and the two real error branches.
- Stop accepting `lucide_icon` as a request key. The OpenAPI schema
documents only `icon`; the dual permit was undocumented and pointless.
`icon` is now the single canonical request key, mapped to
`lucide_icon` on the model in category_params.
- Migrate the Minitest helpers to the project's documented API key
pattern: ApiKey.generate_secure_key + api_key.plain_key in the
X-Api-Key header (matching the rswag spec in this PR and the rule in
.cursor/rules/api-endpoint-consistency.mdc), instead of hand-built
display_key strings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Botched conflict merge
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* Performance improvements in holding calculation pipeline
Investment accounts with large histories were pegging CPU at 100% during
sync. Root cause was a cluster of quadratic and superlinear algorithms in
the inner holding calculation loop. All are replaced with O(1) hash lookups
built from single-pass indexes over the already-loaded data.
Holding::PortfolioCache - load_prices:
Three O(SxN) patterns inside the per-security loop:
1. DB prices: `security.prices.where(...)` fired one SQL query per
security (N+1). Replaced with a single bulk query before the loop:
Security::Price.where(security_id: ..., date: ...).group_by(&:security_id)
70 securities -> 70 queries becomes 1.
2. Trade prices: `trades.select { |t| t.entryable.security_id == id }`
scanned the full trades array for every security - O(SxT). Replaced
with trades_by_security_id, pre-indexed once from the loaded array.
3. Holding prices: `holdings.select { |h| h.security_id == id }` - same
O(SxH) pattern. Replaced with holdings_by_security_id.
Prices are now indexed into prices_by_date and prices_by_date_and_source
hashes during load_prices, making get_price O(1) instead of scanning the
flat prices array on every lookup.
Holding::PortfolioCache - get_trades / get_price:
- get_trades(date:): `trades.select { |t| t.date == date }` (O(T) scan)
replaced with trades_by_date hash (O(1)).
- get_price: two `prices.select { p.date == date ... }.min_by` linear
scans replaced with direct hash lookups into prices_by_date and
prices_by_date_and_source.
Holding::PortfolioCache - collect_unique_securities:
`holdings.map(&:security)` traversed the security association on every
holding record (N+1 if not preloaded). Replaced with a pluck of
security_ids followed by a single Security.where(id: ...) batch load.
Holding::ForwardCalculator / ReverseCalculator:
`holdings += build_holdings(...)` allocated a new array copy on every
iteration - O(N) per day x thousands of days = O(D^2) total allocations.
Replaced with holdings.concat(...) which appends in place, O(1).
Holding::ReverseCalculator - precompute_cost_basis:
Old: walked every date from account.start_date to Date.current (O(D)),
writing a cost_basis entry for every security on every date. For an
account with 2 trades over 9,250 days this wrote ~18,500 hash entries
and consumed the full date range in the outer loop regardless of trade
density.
New: walks only buy trades (O(T)), appending one [date, avg_cost]
snapshot per trade. cost_basis_for binary-searches the sparse snapshot
array - O(log T) per lookup. Memory drops from O(DxS) to O(T).
Holding::Gapfillable:
`security_holdings.find { |h| h.date == date }` was called on every
date in the gapfill range - O(H) per date, O(HxD) total. Replaced with
security_holdings.index_by(:date) built once before the loop, making
each date lookup O(1).
Holding::Materializer - purge_stale_holdings:
`account.entries.trades.map { |entry| entry.entryable.security_id }.uniq`
loaded all trade entry records into Ruby then traversed the entryable
association on each (N+1). Replaced with account.trades.pluck(:security_id).uniq
(single SQL query returning only the IDs).
In testing, these changes were able to reduce sync time of an account with
25 years of history and 70 securities from about 90 minutes down to under
3 minutes.
* Lint fix
* Lint fix
* addressing the open review nits I agreed with:
* return dup'd arrays from PortfolioCache#get_trades so callers can't mutate memoized cache state
* use the precomputed security-id indexes in collect_unique_securities
* keep security-id dedupe in SQL via distinct.pluck(:security_id)
* tighten the DB price preload to select only needed columns
* harden cost-basis assertions with assert_in_delta
* Back out unnecessary AI slop
* Add back dup to trades array returned from memoized hash
trades_by_date[date] returns a live reference into the memoized hash.
Any caller that mutates the result would silently corrupt the cache for
subsequent calls on the same date within the same sync run. Add .dup to
return a shallow copy, matching the safety of the original select path.
The streaming code assumed every stream produced a `response.completed`
event and dereferenced its data unconditionally, causing
`undefined method 'data' for nil` whenever OpenAI emitted
`response.failed`, `response.incomplete`, or a top-level `error` event
(e.g. expired `previous_response_id`, context-window overflow,
transient upstream failures). Surface a descriptive `Provider::Error`
instead.
- Extend `ChatStreamParser` to recognise `response.failed`,
`response.incomplete`, and `error` events and emit an `error` chunk
with a `StreamErrorData` payload (event, message, code, details).
- In `Provider::Openai#native_chat_response`, detect the missing
`response` chunk, build a user-facing error message from the
collected error chunk, and raise `Provider::Error`.
- Add unit tests for the parser (8 cases) and integration tests for
the error path in the chat response flow.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(investments): add India investment subtypes and exchange support
* fix(yahoo-finance): scope Indian exchange de-duplication per company instead of globally
Resolves feedback from Codex and CodeRabbit on #1413.
prefer_indian_exchange previously collapsed all Indian securities into a
single entry, silently dropping unrelated tickers. Now groups Indian
listings by name and only de-duplicates within each group, so distinct
companies (e.g. Reliance and Infosys) are preserved while NSE/BSE
dual-listings still prefer NSE.
- Derive India subtype keys dynamically from Investment::SUBTYPES in tests
- Fix missing keyword arguments in Security.new test calls
* refactor(yahoo-finance): generalize exchange config and dual-listing de-duplication
Replaces hardcoded Indian exchange logic with a declarative EXCHANGE_CONFIG
hash that maps ISO MIC codes to Yahoo-specific settings (symbol suffix,
default currency, dual-listing group, and preference rank). This makes
adding new markets a one-line hash entry instead of scattered conditionals.
* fix(yahoo-finance): normalize security names for dual-listing de-duplication
* fix(yahoo-finance): skip dual-listing de-duplication when filtering by exchange
* fix: address PR review feedback for India market support
- fix cache key mismatch in fetch_security_price by normalizing symbol before building cache key
- remove dead YAHOO_EXCHANGE_CURRENCY constant
- tighten normalize_symbol guard to use end_with?(suffix) instead of include?('.')
- remove misleading '# India' comment from Property::SUBTYPES
- remove 'rented' property subtype in favor of 'investment_property'
- rename 'demat' to 'indian_stocks' for clarity
- add INR to CURRENCY_REGION_MAP so India appears first for INR users
- add dotted-symbol regression test for normalize_symbol
* fix(investments): rename 'demat' subtype to 'indian_stocks' and remove trailing comma
* refactor(design-system): migrate fg-* utilities to text-* and remove namespace
The design system carried two parallel namespaces for foreground colors:
text-* (canonical, ~2,000 uses) and fg-* (32 uses). Most fg-* tokens
were 1:1 duplicates of a text-* counterpart. fg-gray was nearly
identical to text-secondary, with a one-step shade difference in dark
mode.
This PR migrates all 32 usages to their text-* equivalents and removes
the fg-* block from the design tokens. Closes#1606.
Mapping:
- fg-inverse -> text-inverse (20 usages, identical light/dark values)
- fg-gray -> text-secondary (7 usages; light values match, dark is
one step lighter: gray-300 vs gray-400)
- fg-primary -> text-primary (3 usages, identical values)
- fg-subdued -> text-subdued (2 usages, identical values)
The four other fg-* tokens (fg-contrast, fg-primary-variant,
fg-secondary, fg-secondary-variant) had zero usages despite being
defined; they are removed without replacement.
JSON / build:
- design/tokens/sure.tokens.json: $version 1.0.0 -> 2.0.0 (breaking
schema change per the policy added in #1620). 8 fg-* token
definitions removed.
- button-bg-ghost-hover's dark value still references "fg-inverse"
internally; rewritten to "bg-gray-800 text-inverse" so the cleanup
doesn't break that utility.
- _generated.css regenerated. 42 utility blocks now (was 50).
Lookbook tokens preview:
- The Text & foregrounds section dropped its split between text-*
(canonical) and fg-* (legacy). Now a single section listing the
five text-* utilities. The "(legacy)" framing is gone since there's
no legacy left.
README:
- design/tokens/README.md's button-bg-ghost-hover edge-case example
updated to reflect the new "bg-gray-800 text-inverse" dark value.
Visual review needed in dark mode:
- Anywhere icons use the application_helper#icon helper with
color: "default" (most icons in the app). The default class moved
from fg-gray (gray-400 dark) to text-secondary (gray-300 dark), so
default-color icons render slightly lighter in dark mode.
- DS::Buttonish icons in secondary buttons (same shade shift).
- DS::Link icons (same).
- Time series chart axes (same).
- All tooltips, account add flow, settings hostings buttons,
invitations, AI consent, family export, danger-zone buttons --
these used fg-inverse, which is identical to text-inverse, so no
visual change expected.
* fix(design-system): use inverse pair on tooltips for readable dark mode
* fix(lookbook): use semantic tokens in menu preview header text
* fix(lookbook): set text-primary on layout body so previews inherit theme
* fix(design-system): keep shadows dark-toned in dark mode
Inverting shadows to white|8% on dark surfaces produces a halo
effect rather than an elevation cue, and stacks redundantly with
the alpha-white 1px ring already in shadow-border-*.
Switch dark-mode shadows to black at progressively higher alpha
(25%/30%/35%/40%/50% for xs..xl) so they read as actual cast
shadows on near-black surfaces. Surface-tint differences and the
existing alpha-white border ring continue to handle elevation
hierarchy and edge definition.
Approach matches Material 3, Apple HIG, IBM Carbon, Refactoring UI,
and the dark-mode shadows used in Linear/Vercel/Stripe.
* fix(design-system): set text-primary on DS::Dialog element
Browser UA stylesheets apply color: black directly to <dialog>,
which overrides ancestor inheritance even when a body or html
ancestor sets a theme-aware color. Unstyled child content then
renders black regardless of theme.
Setting text-primary on the dialog element itself defeats the UA
override and lets descendants inherit the semantic token.
* fix(lookbook): use shadow css vars in effects preview so dark theme renders
* Revert "fix(design-system): keep shadows dark-toned in dark mode"
This reverts commit 3e9d76ed0b.
* fix(design-system): use opacity-70 instead of text-inverse/70 in value tooltip
The custom @utility text-inverse expands to @apply text-white and
isn't modifier-aware, so text-inverse/70 produced no CSS at all and
the muted labels fell through to inherited color (invisible on the
white pill in dark mode).
Replace with text-inverse + opacity-70. Same visual effect, works
with the existing utility definition.
* feat(api): expose rule run history
* fix(api): address rule run review
* fix(api): complete rule run review
* test(api): cover unauthenticated rule run show
* test(api): align rule run api key helper
* Small Sonnet nit-pick
---------
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* feat(api): expose family settings
* test(api): assert family settings moniker
* test(api): align family settings api key helper
* fix(api): tighten family settings schema
* fix(chat): persist eager pending assistant message to fix subscribe race
When the LLM replies in ~1-2s the assistant message broadcast could
fire before the client's Turbo stream subscription was established,
leaving the UI stuck on the thinking indicator while the response was
already persisted.
Create the AssistantMessage as `pending` synchronously in
`Chat#ask_assistant_later`, so it is rendered server-side on the chat
show page with a "Thinking ..." inline placeholder. The worker then
finds and updates the existing row via `append_text!`, which flips the
status to `complete` and broadcasts updates against a DOM id that is
already in the page — no race possible. On error, the placeholder is
destroyed if no content streamed, otherwise demoted to `failed`.
Replaces the standalone thinking indicator partial and the
`Assistant::Broadcastable` thinking helpers, both now redundant.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(chat): bind each assistant job to its specific pending placeholder
Addressing review feedback on #1658:
1. The pending placeholder lookup based on `last pending` was racy —
back-to-back user messages would let one job fill another job's
placeholder. Pass the placeholder through the job arguments
(`AssistantResponseJob.perform_later(user_message, pending)`) so
each turn is bound to its own row.
2. In `Assistant::External#respond_to`, the configured/authorized
guards raise before the local was bound, leaving rescue cleanup
with `nil` and the placeholder visible forever. Bind the parameter
first so cleanup can destroy it on the misconfigured path.
The kwarg defaults to nil so the API#retry path
(`AssistantResponseJob.perform_later(new_message)`) and the model-level
test calls continue to work — they fall back to an in-memory new
message, restoring the original test count assertions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(chat): i18n the pending assistant placeholder string
Move the hardcoded "Thinking ..." indicator into the locale file per
CLAUDE.md i18n guidelines. With i18n.fallbacks enabled, non-en locales
fall back to English until translated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add thinking label translations
* Fix chat pending assistant expectations
* Fix external assistant pending test lookup
* Scope chat stream targets per chat
* Update message broadcast target tests
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Performance and bug fixes in provider price fetches
Three distinct bugs caused the price provider API to be called unnecessarily
on every investment account sync.
1. Sold securities triggered a provider call on every sync forever
import_security_prices passed end_date: Date.current for every security
ever traded. Security::Price::Importer short-circuits via all_prices_exist?
only when persisted_count == expected_count, where:
expected_count = (clamped_start_date..Date.current).count
This range increases daily, so a security closed two years ago would have
all historical prices in the DB unnecessarily. This also causes any closed
securities to fetch prices daily, forever.
Fix: separate currently-held securities (end_date: Date.current) from
historical-only securities (end_date: last holding date for that security).
Once a closed position's price range is complete through its last holding
date, all_prices_exist? becomes permanently stable and no further provider
calls occur for that security.
"Currently held" is defined as appearing in account.current_holdings, which
returns the most recent holding per security with qty != 0. On the first
sync after a sell, the pre-sale holding is still the most recent, so the
security correctly receives end_date: Date.current for one final sync before
the new qty=0 holding is materialised.
2. Offline securities were not filtered
account.trades.map(&:security) returned all securities regardless of the
offline flag. This results in fetching of securities even if the provider
cannot serve them, or if the user don't want them served for some reason
(eg when there are symbol collisions that causes the wrong prices to be
returned) The global MarketDataImporter correctly uses Security.online;
the account-scoped importer did not.
Fix: Security.online.where(id: all_security_ids) matches the established
contract. Offline IDs still pass through the pluck step but resolve to nil
in the securities hash and are skipped by the existing `next unless security`
guard.
3. N+1 queries for security loading and per-security start dates
- account.trades.map(&:security): triggered one SQL query per trade to load
the security association (N+1).
- first_required_price_date(security): issued 2 DB queries per security -
one MIN(entries.date) and one EXISTS - so S securities = 2S queries.
Fix: replace with batch queries totalling 4 regardless of security count:
- account.current_holdings.pluck(:security_id) - current security IDs
- account.trades.pluck(:security_id).uniq - traded security IDs
- Security.online.where(id: ...) - load all security records at once
- batch_first_required_price_dates: one GROUP BY security_id MIN(entries.date)
over trades, one pluck for provider-holding security IDs, one GROUP BY
security_id MAX(date) over holdings for historical end dates
* fix(market-data-importer): fetch prices through today for reopened positions
Account::Syncer runs import_market_data before materialize_balances, so
current_holdings reflects the last materialized snapshot rather than the
current trade state. If a security was previously sold (stale holdings show
qty=0) and then repurchased in the same sync cycle, it landed in
historical_ids and had its end_date capped at the old last_holding_date.
This caused all_prices_exist? to short-circuit, skipping the price fetch
through today, and leaving the forthcoming holding materialization without
a price for the repurchase period.
Fix: compare the latest trade date against the last holding date for each
historical security. If the trade is newer, the position was reopened before
holdings were rematerialized; treat end_date as Date.current for that sync.
The cap still applies on subsequent syncs once materialize_balances has
updated the holdings table.
Adds a regression test covering the repurchase scenario.
* hoist account.start_date out of per-security loop
Account#start_date issues SELECT MIN(date) FROM entries on every call.
Inside batch_first_required_price_dates it was called up to twice per
security (holding_date assignment + fallback), producing up to 2N extra
queries for an account with N provider-held securities.
Cache the result in account_start_date before the loop.
* assert offline securities are skipped
Adds a regression test verifying that Account::MarketDataImporter never
calls fetch_security_prices for a security with offline: true, covering
the Security.online filter on line 54 of the importer.
* feat(api): expose rule export endpoints
* fix(api): tighten rule export contracts
* fix(api): document balance sheet auth errors
* test(api): align rule API key fixtures
* Update docs/api/openapi.yaml
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
* Quick win
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* feat(api): expose valuation history index
* fix(api): hide valuation exception details
* fix(api): reuse eager-loaded valuation entries
* fix(api): tighten valuation index contracts
* fix(api): scope valuation filter errors
* docs(api): nest valuation account filter format
* Fix merge conflict mistakes
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* feat(design-system): live tokens reference page in Lookbook
Adds `DesignTokensPreview` at `/design-system/inspect/design_tokens/*`,
split into seven sub-pages (typography, palette, surfaces, text,
borders, controls, effects). Each reads `design/tokens/sure.tokens.json`
at request time and renders the corresponding slice with values
pre-resolved to literal hex / rgba in Ruby — Tailwind doesn't need to
keep every CSS variable alive for the swatches to render.
Also drops the `@source not "../../../design/tokens"` directive added
in #1604. Excluding the JSON tree-shook ten or so design system
utilities that aren't yet used in app views (`shadow-border-md/sm/xl`,
`button-bg-ghost-hover`, etc.). The preview references each utility
through dynamic ERB, which Tailwind's scanner can't follow, so those
swatches were rendering blank. Letting Tailwind scan the JSON keeps
every declared utility available, which matches the intent of a design
system. Compiled CSS grows by about 3 KB.
Stacked previously on the `refactor/design-system-tokens` branch behind
#1604; rebased onto `main` once that landed.
* style(design-system): apply rubocop indented_internal_methods to preview
CI lint flagged the private helpers in DesignTokensPreview because the
project's RuboCop config uses `indented_internal_methods` style (methods
after `private`/`protected` get an extra 2-space indent). Auto-fixed
with `bin/rubocop -A`.
* fix(design-system): pre-resolve utility token values for the preview
CodeRabbit caught: collect_utilities was passing raw `{ref}` strings
(e.g. `{color.gray.50}`) as light_value/dark_value, while the rest of
the class pre-resolves to literal hex / rgba. The four templates that
display them (surfaces, text, borders, controls) showed the unresolved
template strings to users.
Adds `light_resolved` / `dark_resolved` fields to each utility entry,
populated via the same `resolve_template` helper the other collectors
use. Templates display `:light_resolved || :light_value` so plain class
strings (e.g. `border-tertiary`, `bg-gray-800 fg-inverse`) and compose
cases still fall through correctly.
Refs #1059.
When you auto-match a $500 expense from your checking account against
the matching deposit on your credit card, the resulting transfer pair
was leaving traces in the per-card "Recent transactions" list under
each budget category card, even though the aggregate
`Budget#actual_spending` (via `IncomeStatement`) already excluded
`BUDGET_EXCLUDED_KINDS` (funds_movement / one_time / cc_payment) from
the totals. The user saw $X under the card while the totals showed
$X less.
Fix: extend the same exclusion to the drilldown list. The aggregate
and the list now agree.
```ruby
# app/controllers/budget_categories_controller.rb
@recent_transactions = @budget.transactions
.where.not(transactions: { kind: Transaction::BUDGET_EXCLUDED_KINDS })
```
`loan_payment` and `investment_contribution` are intentionally NOT in
`BUDGET_EXCLUDED_KINDS`, so those transfers still appear (they are
budget-tracked).
What this PR does NOT do:
- It does not clear the matched transactions' `category_id` in the
matcher itself. An earlier draft of this PR did, but codex
correctly flagged that doing so causes data loss when a user
rejects an incorrect auto-match: `Transfer#reject!` resets `kind`
to `standard` but does not restore the previously-cleared
category, permanently dropping the user's original
categorisation. The controller filter alone is sufficient to fix
the user-visible bug, and the inconsistency between
`kind = funds_movement` and a retained category is harmless because
every relevant view filters one or the other.
- The mortgage scenario in #1059 (a `loan_payment` match showing as
"Uncategorised" in the budget) isn't a leak; it is a missing
feature. The matcher doesn't auto-assign a category to
`loan_payment` rows the way #924 does for
`investment_contribution`. The natural follow-up is a parallel
`loan_payments_category` plus matcher / import-adapter
auto-assignment, which deserves a maintainer signoff first.
Tests:
- `BudgetCategoriesControllerTest#show drilldown excludes
BUDGET_EXCLUDED_KINDS transfers from recent transactions`: a
matched depository <-> CC pair does not appear in the
Uncategorised drilldown after the matcher runs.
- `BudgetCategoriesControllerTest#show drilldown still lists
loan_payment transfers (intentionally budget-tracked)`: a matched
depository <-> loan pair stays visible in the drilldown.
Suite: 3239 / 0 / 0 / 24 on the latest upstream/main. Lint clean.
* Performance improvements in balance sync cache
Balance::SyncCache#converted_holdings called account.holdings.map { |h| h.dup }
which duplicated every holding record into a new ActiveRecord object, converted
its currency, and stored the full object in a holdings_by_date array hash.
For an investment account with years of history this allocates 100,000+
AR objects on every sync - one per holding row - creating proportional GC
pressure that scaled with account age.
The only consumer of get_holdings(date) was BaseCalculator#holdings_value_for_date,
which immediately discarded the objects after calling .sum(&:amount). The
individual holding objects were never accessed for any other attribute.
Replace the dup-and-group approach with a single aggregation pass that stores
only the per-date sum:
holdings_value_by_date: account.holdings.each_with_object(Hash.new(0)) do |h, totals|
converted = Money.new(h.amount, h.currency).exchange_to(account.currency, date: h.date).amount
totals[h.date] += converted
end
Interface change: get_holdings(date) -> get_holdings_value(date) returns a
Numeric directly rather than an Array. BaseCalculator#holdings_value_for_date
is updated accordingly, and its own per-date memoization layer is removed
since holdings_value_by_date is already fully memoized at the SyncCache level.
* fall back to 1:1 rate in SyncCache when holding exchange rate is missing; update tests to use investment class
* Improve chat LLM error messages
* Fix chat visibility regression in tests
* Harden chat error handling for review feedback
* Fix rubocop private method indentation
* Fix nil presentable_error_message, i18n strings, bare rescue
- Guard `presentable_error_message` with `return nil if error.blank?` so
chats with no error return nil instead of the fallback string; this
prevents the API serialisers from emitting a spurious error message and
stops the mobile polling guard from firing on every successful chat
- Move all hardcoded user-facing error strings into
config/locales/models/chat/en.yml and reference them via I18n.t()
- Replace bare `rescue` in `error_message_for` with `rescue StandardError`
to avoid swallowing system-level exceptions
- Update tests to reference I18n keys instead of raw strings, and add
tests for the nil-error case and the unrecognized-error fallback
https://claude.ai/code/session_01YFMjEds5WVyKPL42xBqMCX
---------
Co-authored-by: SureBot <sure-bot@we-promise.com>
Co-authored-by: Claude <noreply@anthropic.com>
