mirror of
https://github.com/we-promise/sure.git
synced 2026-09-05 14:51:15 +00:00
6d6dac44f3d7139a7bcdb9dcc0f0d2fc62da32bb
665
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
4e010493c7 |
feat(up): map Up category slugs to Sure categories on import (#2487)
* feat(up): map Up category slugs to Sure categories on import UpEntry::Processor captured Up's category slug into extra but never applied it, so Up transactions imported uncategorised even though the user had already tagged them in the Up app. Add UpAccount::Transactions::CategoryTaxonomy + CategoryMatcher, mirroring PlaidAccount::Transactions::CategoryMatcher: map Up's child category slugs onto the family's existing/default Sure categories by alias, and wire the matcher through UpAccount::Transactions::Processor into UpEntry::Processor. The category is applied via the adapter's enrich_attribute, so a category the user has set or locked is preserved on re-sync. High-confidence mappings only. Up-specific categories with no honest Sure default (Booze, Pets, Apps & Games, Life Admin, Technology, ...) intentionally stay uncategorised for the user's own rules / AI, since a wrong auto-category is worse than none. Adds a matcher unit test and processor wiring tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(up): match category slugs as strings in CategoryMatcher Up category ids are string slugs; compare them against the taxonomy keys as strings so the lookup does not depend on the keys being symbols. No behaviour change (the "slug": hash syntax already produces symbol keys that matched the symbolized input, covered by the matcher unit test), but it removes a subtle footgun and reads clearer. Flagged by the Codex review on the PR. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(up): make category import non-destructive; word-boundary the alias match Per review feedback: do not bootstrap Sure's default categories during a sync. family_categories now returns the family's existing categories without creating defaults, so a family that has none (deliberately cleared, or pre-onboarding) gets uncategorised transactions rather than having the full default set silently created. Matching resumes once the user sets up categories through the normal UI flow. Also word-boundary the "and" stripping in the matcher normalization so it strips only the standalone conjunction, not "and" inside a word (e.g. errand). Adds a processor test for the non-destructive guarantee. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Gavin Matthews <matthews.gav@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
super
|
1a5d04d527 |
feat(rules): add a Transaction tag condition filter (#2558)
* feat(rules): add a Transaction tag condition filter Rules could already set tags via the set_transaction_tags action but had no way to match transactions by an existing tag. Add a select-type transaction_tag condition filter (mirroring transaction_category) with the standard "Equal to" and "Is empty" operators, registered on the transaction rule resource so it surfaces in the rule builder automatically. Also remap the tag UUID<->name in family data export/import for the new condition, matching how the transaction_category/transaction_merchant conditions and the set_transaction_tags action are already handled, so tag-based rules survive an export/import round-trip. Closes #2557 * fix(rules): match transaction_tag via EXISTS so compound tag conditions work Addresses review feedback on #2558: the tag filter joined transactions.tags and predicated on tags.id, which broke two compound cases: - two ANDed tag conditions collapsed to `tags.id = a AND tags.id = b` on the same joined alias and could never match, even when the transaction had both tags; - OR / multi-tag matches returned a transaction once per tagging row, inflating counts and making rule actions iterate duplicate transactions. Use a correlated EXISTS subquery per condition instead. Each condition is independent (fixes the AND case) and no join is added, so rows are never multiplied (fixes the OR duplication) and prepare adds nothing, keeping branches structurally compatible inside a compound OR. Add tests for both. |
||
|
|
6439a731ab |
feat(self-hosting): surface Sidekiq-unhealthy nudge + admin system health page (#1906)
* feat(self-hosting): surface Sidekiq-unhealthy nudge + admin system health page (#1481) When the Sidekiq worker container isn't running — the most common Docker Compose misconfiguration in self-hosted setups — every background job silently never executes. Balance calculations, net-worth updates, and account syncs stall. The UI shows zeros and "No balance data available for this date" without explaining why (#1481, #1047). Per jjmata's resolution on the issue, this PR ships both halves of the fix in one pass: 1. A user-facing nudge banner that appears on every authenticated page when Sidekiq isn't processing jobs. Tells the user their data may be stale; doesn't pretend zeros are real. 2. An admin-only deep link from that banner into a new `/settings/admin/system_health` page (super-admin gated, matching the existing admin namespace contract) showing live Sidekiq state: process count, last heartbeat, max queue latency, job counters, and per-queue depth. ## What changed - New `SidekiqHealth` PORO (`app/models/sidekiq_health.rb`) eagerly loads ProcessSet + Queue + Stats in one pass and exposes `healthy?` plus a stable `reason` symbol (`:redis_unreachable`, `:no_worker_processes`, `:stale_heartbeat`, `:queue_backed_up`). Any Redis/Sidekiq failure during the eager load is caught and surfaced as `:redis_unreachable` so a degraded broker never crashes the layout. - `ApplicationController#current_sidekiq_health` memoizes a single instance per request via `helper_method` so the layout, banner partial, and any controller checks share one Redis round-trip. - New `app/views/shared/_sidekiq_health_banner.html.erb` rendered from `_htmldoc.html.erb` when `Current.user` is present and the health check is failing. Banner shows the user-facing message to everyone; the "View system health" CTA + reason detail are gated on `Current.user&.super_admin?`. - New `Admin::SystemHealthController#show` (inherits the existing `Admin::BaseController`, so super-admin gating is enforced for free) + view rendering status, counters, and per-queue breakdown. - Routes: `resource :system_health, only: :show` inside the existing `namespace :admin`. - Settings nav: new "System health" entry under the Advanced section, gated on `super_admin?` to match `sso_providers_label` and `users_label`. - i18n: new `shared.sidekiq_health_banner.*` keys (title, body, CTA, per-reason explanations) and a full `admin.system_health.show.*` namespace for the new admin page. English-only, matching how `ds.pill.*` and other DS keys are scoped. ## Why - jjmata: "Let's take both approaches ... a nudge about 'data unavailable' which hyperlinks to the admin UI if you are an admin only (not for other types of users) sounds like the best path forward. **Any takers for the PR?**" (#1481) - smurfpandey: "We can add a section in Settings for superadmins to see 'health' of the application/host." - The detection signal is conservative on purpose: - `PROCESS_HEARTBEAT_TIMEOUT = 2.minutes` tolerates deploy restarts and brief Redis blips without flapping. - `LATENCY_THRESHOLD = 5.minutes` is well above the sync-job tail under default `config/sidekiq.yml` concurrency. ## Validation This worktree runs on Windows without a local Ruby toolchain, so I could not run `bin/rubocop`, `bundle exec erb_lint`, `bin/brakeman`, or `bin/rails test` locally. CI will run the full matrix on the PR: - `lint` — `bin/rubocop -f github` - `lint_js` — `npm run lint` (no JS touched, should be green) - `scan_ruby` — `bin/brakeman --no-pager` - `scan_js` — `bin/importmap audit` - `test_unit` — `bin/rails test` (includes 7 new tests under `test/models/sidekiq_health_test.rb` and 4 new under `test/controllers/admin/system_health_controller_test.rb`) - `test_system` — `DISABLE_PARALLELIZATION=true bin/rails test:system` - `pipelock` — secret + agent-security diff scan Manual checks done in this worktree: - Re-read `CONTRIBUTING.md` and `.cursor/rules/project-conventions.mdc`. PORO under `app/models/` per Convention 2. No new gem dependency per Convention 1. Banner uses semantic tokens (`bg-warning/10`, `text-warning`) per the design-system rules. No `lucide_icon` direct call — uses the `icon` helper per CLAUDE.md. - Confirmed `Sidekiq::ProcessSet` / `Sidekiq::Queue` / `Sidekiq::Stats` are the same APIs Sidekiq 7+ exposes (we're on Sidekiq 8.x per the `Gemfile.lock` comment in `config/initializers/sidekiq.rb`). - Tests stub `Sidekiq::ProcessSet.new` / `Sidekiq::Queue.all` / `Sidekiq::Stats.new` so the suite doesn't need Redis populated. - The admin route lives inside the existing `namespace :admin` so `Admin::BaseController#require_super_admin!` enforces auth — no new authorization surface added. ## Notes - No public API endpoints, no rswag specs, no OpenAPI changes. - No migrations, no model changes outside the new PORO. - No background jobs touched. - English-only locale entry, mirroring the `ds.*` / `admin.invitations.*` precedent in this repo. Other locales fall back to English. - Detection thresholds are constants on `SidekiqHealth` so they're easy to tune from a follow-up PR if the defaults turn out to flap on any real-world deployment. - The banner positions itself at `top-20` (below the impersonation / super-admin bars) and uses `z-40` (below the `z-50` notification tray). Single-screen overlap with mobile flash toasts is acceptable for V1. Refs: #1481, #1047 * fix(self-hosting): address review on Sidekiq health PR (#1481) - `Admin::SystemHealthController#show` now reads from the request-memoized `current_sidekiq_health` instead of building a fresh `SidekiqHealth.new`, so the controller and the layout banner share one Redis round-trip. - `SidekiqHealth#reason` now treats `last_heartbeat_at.nil?` the same as a stale beat: a registered process that hasn't published a heartbeat is not "healthy". Previously the check short-circuited on the nil guard and silently fell through to the queue-latency branch. Added a unit test covering the `ProcessSet` entry with `"beat" => nil` case. - Settings nav: switched the "System health" entry's icon from `activity` to `heart-pulse` so it no longer duplicates the LLM Usage icon. - Routes: dropped the redundant `controller: "system_health"` option from the `resource :system_health` declaration — Rails infers `Admin::SystemHealthController` from the namespace, matching the style of the sibling `:sso_providers`, `:users`, `:invitations`, and `:families` admin resources. * fix(self-hosting): scope + cache Sidekiq health, admin-only banner (#1481) Addresses the second round of maintainer review on the Sidekiq health PR. - Skip the check entirely in managed mode. `current_sidekiq_health` returns `nil` unless `Rails.application.config.app_mode.self_hosted?`, so authenticated requests in managed deployments add zero Redis round-trips for this feature. - Cache the snapshot across requests via `SidekiqHealth.current` (Rails.cache, TTL `CACHE_TTL` = 60s default, env-overridable). The per-request memoization on `ApplicationController` is preserved on top, so even back-to-back self-hosted pages share one fetch. - Make thresholds operator-tunable. `PROCESS_HEARTBEAT_TIMEOUT`, `LATENCY_THRESHOLD`, and the new `CACHE_TTL` read from `SIDEKIQ_HEALTH_HEARTBEAT_TIMEOUT`, `SIDEKIQ_HEALTH_LATENCY_THRESHOLD`, and `SIDEKIQ_HEALTH_CACHE_TTL` env vars (seconds), with the previous values as defaults. Comments now explain the tuning rationale. - Gate the banner on `Current.user&.super_admin?` at the layout level rather than rendering a vague warning to family members who can't act on it. The partial no longer carries an internal admin check since the call site does it; non-admins see nothing. - Replace the hard-coded `top-20` offset with a computed offset based on which impersonation bars are visible (`top-4` / `top-20` / `top-36`) so the banner doesn't collide with the super-admin or approval bars when both are stacked above it. - `Admin::SystemHealthController#show` now bypasses the cache (`SidekiqHealth.expire_cache!` + `SidekiqHealth.new`) so an operator who just restarted the worker sees fresh state instead of a stale 60-second snapshot. Also lets the page render in managed mode where `current_sidekiq_health` is nil. - Tests: add coverage for `.current` cache reuse and `.expire_cache!` forcing a re-query, swapping `Rails.cache` to a MemoryStore since the test env defaults to `:null_store`. * fix(self-hosting): route singular resource + drop assert_same on cached snapshot (#1481) Two CI failures surfaced once the full pipeline ran on this branch for the first time (it was gated on contributor approval until d04b78e): - Admin system-health controller tests returned 404. Singular `resource :system_health` in `config/routes.rb` makes Rails infer `Admin::SystemHealthsController` (it pluralizes the controller name even for singular resources), but the controller file is named `system_health_controller.rb` / `Admin::SystemHealthController`. Restore the explicit `controller: "system_health"` override that the previous "address review" commit dropped on the (mistaken) premise that Rails would infer it from the namespace — the sibling admin routes all use plural `resources` so they round-trip cleanly, this one doesn't. Comment now spells the gotcha out so the next reviewer doesn't try to "simplify" it again. - `SidekiqHealthTest#test_current_memoizes_across_calls_inside_the_cache_TTL` used `assert_same` on the two returns from `SidekiqHealth.current`. `ActiveSupport::Cache::MemoryStore` defaults to `dup_values: true` and Marshals on read, so a cache hit returns an `==`-equal but `equal?`-different instance. Replace the identity check with the behavioral assertion we actually care about: re-stub `ProcessSet` to raise on the second call, then assert the second `current` return is still healthy (proving Redis was not re-queried). * fix(i18n): drop redundant inline default on system_health nav label (#1481) `system_health_label` is already defined in config/locales/views/settings/en.yml, so the inline `default: "System health"` was a hard-coded English string in the template (DS Drift Patrol Rule 5). Use the bare locale lookup like the sibling nav entries. --------- Co-authored-by: John Baillie <johnbaillie2007@gmail.com> Co-authored-by: Khaostica <256858950+Khaostica@users.noreply.github.com> |
||
|
|
5a2bf02b13 |
fix(simplefin): stop repair_stale_linkages from hijacking a live linkage on a same-name twin (#3116)
* fix(simplefin): stop repair_stale_linkages from hijacking a live linkage on a same-name twin repair_stale_linkages matched purely on case-insensitive display name, so two distinct upstream accounts sharing a name (e.g. two "CHECKING (0001)" accounts at the same institution) caused the unlinked twin to silently steal the linked account's AccountProvider, merge in its transactions, and overwrite its balance on every subsequent sync. Thread the upstream account_id set already computed during account discovery through to repair_stale_linkages so it only treats a linked account as stale when its account_id is actually absent upstream, and skip ambiguous multi-way name matches instead of picking the first one. Fixes #2852 * fix(simplefin): clear stale upstream_account_ids and log skipped repairs - Clear simplefin_item.upstream_account_ids at the start of each perform_account_discovery run so a later discovery that finds zero accounts can't reuse IDs from a prior run on the same SimplefinItem instance (CodeRabbit review finding). - Capture skipped stale-linkage repairs via DebugLogEntry so operators can see them in /settings/debug, not just the raw Rails log (Codex review finding). - Fix two pre-existing SimplefinAccount::Transactions::ProcessorInvestmentTest tests broken by the new upstream_account_ids nil-guard: they called process_accounts directly without going through the Importer, so they now set upstream_account_ids explicitly to simulate a legitimate "old account_id genuinely absent upstream" repair. - Add regression coverage for both fixes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
|
b0ecb919b0 |
fix(lunchflow): refresh stored transaction on pending to posted (#2778)
* fix(lunchflow): refresh stored transaction on pending to posted LunchflowItem::Importer keyed stored raw transactions but treated them as immutable snapshots. When Lunchflow flipped a transaction from pending to posted under a stable ID, fetch_and_store_transactions skipped it as a duplicate and kept the stale pending snapshot. The processor then re-imported it with isPending:true, so ProviderImportAdapter#import_transaction never reached its pending-clearing branch and the entry stayed stuck with a "Pending" badge. Index stored transactions by key (the Lunchflow ID, or a content hash for the blank IDs Lunchflow returns for some pendings) and refresh the stored snapshot in place when the upstream payload actually changed, while still deduplicating by key to prevent unbounded growth. Fixes #2735 * fix(lunchflow): preserve identical same-response blank-ID transactions The stored snapshot was keyed with a Hash of key -> transaction, so two rows in one sync response that share a content hash (Lunchflow returns blank IDs for some pending transactions, and two genuinely distinct identical purchases hash the same) collapsed into a single entry. The second row hit the existing-key branch as though it were a duplicate, dropping a real transaction before LunchflowEntry::Processor could apply its collision suffix. Pool the existing snapshot into one bucket per key and match incoming rows one-for-one (shift), so same-response collisions each claim their own slot or count as new. This preserves every real transaction while still deduplicating across syncs (re-syncing the same pair stays at two, not four) and refreshing pending -> posted transitions. Adds a regression test for two identical blank-ID rows in the same response. --------- Co-authored-by: agentloop <agentloop@localhost> Co-authored-by: pro3958 <pro3958@users.noreply.github.com> |
||
|
|
29c0a369d0 |
fix(transfers): isolate concurrent transfer match in a savepoint (#2769)
* fix(transfers): isolate concurrent transfer match in a savepoint auto_match_transfers! opens one Transfer.transaction and, per candidate, calls Transfer.find_or_create_by! while rescuing RecordNotUnique. The transfers table has a composite unique index on (inflow_transaction_id, outflow_transaction_id), so when two syncs of the same family run at once the losing insert raises the unique violation. On PostgreSQL a failed statement aborts the whole surrounding transaction. Rescuing the Ruby exception does not clear that state, so the next update! raises PG::InFailedSqlTransaction and every following candidate is dropped. Run the per-candidate insert in its own savepoint via Transfer.transaction(requires_new: true), extracted into a private find_or_create_transfer! helper. A lost race now rolls back only to the savepoint; the outer transaction stays healthy and the loop keeps matching. The same race surfacing through the uniqueness validation (RecordInvalid with :taken) is treated as already-created; any other validation failure is re-raised. Fixes #2471 * fix(transfers): only swallow the uniqueness race for the exact pair The rescue treated a :taken on inflow_transaction_id or outflow_transaction_id as proof that this candidate's transfer was created. But the uniqueness validations are per-column, so two same-amount candidates racing (one committing (inflow, outflow_a) while another tries (inflow, outflow_b)) raise :taken on inflow_transaction_id even though no Transfer exists for (inflow, outflow_b). The caller then marked outflow_b as matched with no Transfer behind it. Confirm the exact (inflow_transaction_id, outflow_transaction_id) row exists before accepting the race; otherwise return nil and skip the candidate. Non-:taken validation failures still re-raise. The RecordNotUnique path (composite index) already implies the exact pair — it now returns that row for the same reason. * test(transfers): assert matching continues past a skipped collision The concurrent-race test had no surviving candidate, so a regression that stopped processing after the skipped collision would still pass. Add a second, non-conflicting candidate and assert its transfer is created and both entries are marked. * test(transfers): pass insert! attributes as an explicit hash Ruby 3 treats insert!(inflow_transaction_id: ..., outflow_transaction_id: ...) as keyword arguments, so ActiveRecord's insert!(attributes) got zero positional args and raised ArgumentError (given 0, expected 1). Wrap the attributes in { } so they are the positional attributes hash. --------- Co-authored-by: agentloop <agentloop@localhost> Co-authored-by: pro3958 <pro3958@users.noreply.github.com> |
||
|
|
d57c4301f2 |
fix(import): tolerate null Rule names and orphaned rejected transfers (#2775)
* fix(import): tolerate null Rule names and orphaned rejected transfers Importing a full all.ndjson export aborted on data that is actually valid. The preflight listed name as a required field for Rule, but rules.name is nullable and the model allows it, so a single rule with "name": null blocked the entire import. Separately, a RejectedTransfer whose referenced transaction had been deleted raised a hard missing_reference error in preflight and a MissingReferenceError in strict mode, even though the importer already had a skip path for it. Require Rule.id instead of Rule.name in preflight, matching the field the importer actually needs. Treat RejectedTransfer references as advisory: a missing referenced transaction becomes a warning, and the importer resolves the references with required: false so the orphaned row is skipped and counted instead of raising. Fixes #2721 * fix(import): keep SureImport preflight warnings as strings at the API boundary The orphaned-RejectedTransfer path emits warnings as {code, message} hashes via add_warning, but the published OpenAPI contract documents /api/v1/imports/preflight warnings as strings. sure_import_preflight_payload copied them through unchanged, so the endpoint returned a heterogeneous array once that path was exercised and contract-generated clients could fail to deserialize. Map warnings to their human-readable message at the API boundary so the array stays homogeneous strings, matching the documented schema. The internal {code, message} shape is unchanged. Adds a regression test. * i18n(import): localize missing-reference preflight messages The missing-reference warning and error are user-facing (returned in Result#payload[:warnings]/[:errors]) but were hard-coded. Move the full templates to config/locales/models/sure_import/preflight/en.yml and interpolate line, type, field, and value, per the i18n coding guideline. Warning key and error behavior are unchanged, and the rendered text is identical. --------- Co-authored-by: agentloop <agentloop@localhost> Co-authored-by: pro3958 <pro3958@users.noreply.github.com> |
||
|
|
f0a0013da9 |
fix(charts): round series values to the currency's display precision (#3091)
* fix(charts): round chart values to the currency's display precision Charts are drawn from the serialized amounts, not from the formatted strings, so a sub-unit residue plotted as a visible move between two points that both read $0.00, and the trend between them reported a change. Round in `Series#as_json` so the series values stay exact for insights, goals and the assistant. Also hide the percentage when the previous value is zero, since that makes it infinite. * fix(charts): round the two payloads that bypass Series#as_json `NetWorthBreakdownSeriesBuilder` builds its payload by hand, so the reports chart never went through the rounding added in `Series#as_json` and still plotted raw amounts: adjacent points printing the same value rendered a visible move, and the tooltip it inherits reported a change between them. `Series#trend` had the same gap on the server side. It is rendered right above the chart by `UI::Account::Chart` and by the reports summary, so an account going from 0 to a sub-cent residue showed a coloured $0.00 change next to a flat line. Rounding the trend can make a previously finite percentage infinite, so guard the three views that render `percent_formatted` without checking, as `shared/_trend_change` already does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(charts): address net worth review comments * fix(charts): tighten rounded trend handling * fix(reports): restore positive sign in print trend --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: sure-admin <sure-admin@splashblot.com> |
||
|
GFRandGerald
|
fa8769f792 |
fix: support DD/MM/YY date format for CSV imports (#3110)
Adds "DD/MM/YY" as a CSV-only date format for transaction and account balance imports, addressing #1530. Kept out of Family::DATE_FORMATS (the global date preference) since a prior PR (#531) adding it there was rejected by a maintainer: 2-digit years are ambiguous (Ruby's %y assumes 1969-2068) and could silently misparse historical or future-dated transactions. Restricting it to Import::CSV_ONLY_DATE_FORMATS keeps it available where the user can see and verify a parsed preview against their own CSV data, per the maintainer's suggested approach in that PR's discussion. Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> |
||
|
Juan José MataandClaude Opus 5
|
8ca65ffd27 |
Reconcile PDF statement imports against transactions that already exist (#3105)
* Reset to main, keeping only the account matcher improvements Backs out the LLM-driven reconciliation work (PR #1382's approach and the two commits hardening it). That approach compares whole-statement aggregates, which is all-or-nothing: when 18 of 20 transactions have already synced the totals disagree, the import proceeds, and 18 duplicates are created. Reconciliation is a row-level problem and belongs in the import path, where TransactionImport already solves it via Account::ProviderImportAdapter. Kept from that work, because it stands on its own: - AccountMatcher gains a hint-based class-level entry point so callers without an AccountStatement row can score against the same rules. The instance path used by AccountStatement#assign_account_match is unchanged. - It also refuses to guess between equally-confident candidates rather than letting max_by take whichever the scan reached first. Account names are not unique within a family, so that tie was reachable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Reconcile PDF statement imports against transactions that already exist Reconciliation is a row-level problem. Comparing whole-statement totals is all-or-nothing: when 18 of 20 transactions have already synced the totals disagree, the import proceeds, and 18 duplicates are created. This matches each extracted transaction against what the account already holds, so only genuinely new transactions are ever offered for import and the rest are marked reconciled. TransactionImport#import! already does this for CSV via Account::ProviderImportAdapter. PdfImport#import! called it zero times and built one Transaction per row unconditionally -- the only import path in the repo with no duplicate protection. Reconciliation state follows Quicken's uncleared / cleared / reconciled, but only the last state is stored: - "Cleared" means the institution acknowledged the transaction, which is exactly what entries.source and entries.external_id already record. It stays accurate on its own, because the adapter stamps both onto a manual entry when a provider transaction claims it, so a hand-entered transaction that later appears in a download becomes cleared with no extra bookkeeping. Deriving it also keeps it non-editable, which is right: it is a fact about provenance, not an opinion. - "Reconciled" means a statement was matched against the transaction. Nothing can derive that, so entries gains reconciled_at and reconciled_by_statement_id. It is a judgement, so it can be set and unset, and it survives the statement being deleted (the FK nullifies, the timestamp stays). Matching: - find_duplicate_transaction grows include_provider_entries, which is what makes this work for Provider-backed accounts -- the existing where(external_id: nil) filter hid synced transactions from every import path, so this gap affected CSV imports equally. Default stays false: provider sync must not claim another provider's entry. - It also grows date_window, because a statement's posting date routinely differs by a day or two from the date a provider recorded. Nearest date wins. - Name is deliberately not matched on: statement descriptions and provider names for the same transaction rarely agree. The adapter makes the same choice for sync. - Candidates are built as real Import::Row objects so matching uses the same signed_amount and date_iso the import itself would write, rather than a second interpretation of signage that could drift. - Matching is per-account, so with no account assigned every row is offered and re-judged on assignment; reassigning also releases the previous account's reconciliations. - A row whose date or amount will not parse is offered for import rather than dropped, so nothing disappears silently. - import! re-checks at publish, since a sync can land between review and publish, and new transactions are born reconciled: the statement is their evidence. Provider-backed accounts are now offered in the import target picker. The manual-only restriction existed because importing into a synced account would duplicate what sync brought in, which is precisely what this removes. Also fixes a bug this uncovered on main: extract_transactions stored the extractor's symbol-keyed hash, while every reader digs with strings. jsonb keeps the hash as assigned until reload and ProcessPdfJob never reloads, so has_extracted_transactions? was false and PDF imports generated zero rows. The existing tests missed it -- one uses a YAML fixture, the other stubs the extractor with string keys. Supersedes #1382. Refs #1379. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Fix three review findings in statement reconciliation All three confirmed against the code before fixing. Publish-time recheck consumed the same entry twice. import! started its exclusion list empty, so a statement carrying two same-amount transactions against an account holding only one would re-match the surviving row against the entry row generation had already consumed -- silently dropping a genuinely new transaction instead of creating it. Seed the exclusions with the entries this statement already reconciled; newly synced entries are still caught, since only already-reconciled ones are excluded. A regeneration that emptied the row set left the import stranded. In the normal upload flow the account is assigned after extraction, so assign_account! regenerates -- and if everything then reconciled, rows_count went to zero while status stayed pending. _pdf_import.html.erb renders pending-with-no-rows as the processing screen, and process_with_ai_later cannot restart because ai_processed? is already true, so the import was stuck with no way forward. Status now follows the same rule ProcessPdfJob applies after initial processing, in both directions: no rows completes it, rows returning sends it back to pending. Guarded by data_committed? so a published import is never reopened. Unevaluatable rows went only to the Rails log. AGENTS.md asks for DebugLogEntry.capture on recoverable import failures so they surface in /settings/debug with structured context. Capture family, account, import, statement, row number and the raw date/amount that would not parse. Adds regression coverage for each. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Fix row regeneration collision and over-broad reconciliation release test_unit caught one error in 6462 tests, and it was real. Row regeneration collided on the second call. insert_all! bypasses ActiveRecord, so the rows association is never populated with what it wrote. Calling generate_rows_from_extracted_data twice on the same in-memory record -- which assign_account! now does after ProcessPdfJob has already generated once -- made rows.destroy_all clear a stale empty collection, delete nothing, and then collide on (import_id, source_row_number). Reload before destroying, and reset the association after inserting so sync_mappings and the view read what was actually written. Releasing reconciliations was scoped to the statement, not the account. A statement is evidence for exactly one account at a time but can back more than one import, so reassigning an account cleared reconciliations another account still relied on. Scoped to the account being moved away from; a blank scope releases nothing, which is correct because nothing is reconciled while no account is assigned. The second finding was raised by CodeRabbit. Its other flagged risk -- entries being marked reconciled before the import is published -- is deliberate and stays: the statement is the evidence, reconciling is reversible via unmark_reconciled!, and deferring it to publish would leave a fully reconciled statement with nothing to publish and therefore nothing ever marked. Adds regression coverage for both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Address review nitpicks: model validation, scope placement, mock style, lock-safe DDL All four checked against the repo's own conventions before applying. Mirror the check constraint as a model validation. Assigning reconciled_by_statement without reconciled_at raised StatementInvalid rather than a validation error. CLAUDE.md Convention 5 asks for exactly this pairing -- the constraint in the database, an ActiveRecord validation for form-friendly errors. Stop hijacking the pending-scope comment. The reconciliation scopes were inserted directly under "Pending transaction scopes", so that header read as documentation for them and the provider note below read as a continuation of reconciled_by. Given the reconciliation scopes their own header. Use OpenStruct for the provider response double, per "Always prefer OpenStruct when creating mock instances". Verified OpenStruct.new(success?: true) responds to success?, and ostruct is already a dependency used elsewhere in test/. Make the migration lock-safe. entries is the largest table in the app: both indexes now build concurrently, and the check constraint is added unvalidated then validated separately so VALIDATE takes only SHARE UPDATE EXCLUSIVE instead of holding ACCESS EXCLUSIVE for a full scan. This follows existing practice -- 13 migrations already use disable_ddl_transaction! and 11 use algorithm: :concurrently, with add_offline_reason_to_securities combining add_column and a concurrent index in one migration exactly like this. The suggested follow-up migration for validation was not needed: validating in the same non- transactional migration gets the same lock behavior without a second file, and the repo has no validate: false precedent in 400 migrations. schema.rb is unchanged: a validated constraint and a concurrently-built index dump identically. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q * Add the two review-requested regression tests Covers the invalid-amount half of "malformed rows are offered, not dropped" -- the existing test only exercised an unparseable date. Asserts the raw value is stored verbatim rather than coerced to 0, so the review step shows the user what the statement actually said. Also covers the Entry validation added in |
||
|
GFRandGerald
|
ecf00f8a0f |
fix(enable_banking): recognize N26's PERIOD_INVALID error shape as a retryable period rejection (#3112)
N26 (via Enable Banking) rejects an out-of-range transaction period with
{"code": "PERIOD_INVALID", "detail": "dateFrom=...,dateTo=..."} instead of
the {"error": "WRONG_TRANSACTIONS_PERIOD"} shape the retry ladder from
#2992 already handles. wrong_transactions_period? never matched, so the
sync failed outright instead of retrying with a shorter window. Also
guard corrected_date_from against a non-hash detail payload, which would
otherwise raise NoMethodError for this ASPSP's plain-string detail.
Fixes #1262
Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com>
|
||
|
|
9d225a21e6 |
fix: disable Mark as Recurring button when a manual recurring transaction already exists (#3103)
* fix: disable "Mark as Recurring" button when a manual recurring transaction already exists Previously the button was always clickable and only failed after a POST, showing "A manual recurring transaction already exists for this pattern". Extract the lookup into Transaction#existing_manual_recurring_transaction (reused by the controller guard) so the view can disable the button ahead of time and show the reason inline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: address CodeRabbit review feedback on PR #3103 Move the existing_manual_recurring lookup out of the show view and into the controller so rendering no longer runs an Active Record query in-template, and strengthen the "no match" model test with near-match recurring transactions that individually differ by account, merchant, amount, currency, and manual flag. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: move mark-recurring presentation state fully into controller, fix stale state on failed update Address CodeRabbit follow-up on PR #3103: - Compute the mark-recurring button's subtitle text/class, href, disabled state, title, and class entirely in TransactionsController (via a shared assign_mark_recurring_state helper) instead of deriving them with ternaries in the view. - Populate that state before TransactionsController#update re-renders :show on a failed entry update, so the button doesn't incorrectly appear enabled when a matching manual recurring transaction exists. - Add a controller test covering the failed-update render path. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: use blank name instead of blank date to trigger validation failure in mark-recurring test The CI test_unit run flagged a real bug in the test itself: TransactionsController#entry_params strips blank :date/:amount before update, so date: "" never reached model validation and the update succeeded (302) instead of failing (422) as the test expected. Use a blank :name instead, which isn't stripped, and add DOM assertions (disabled button, no mark_as_recurring form action) per CodeRabbit's follow-up review. Verified against a live NAS Rails console reproduction (bypassing the test stack's broken fixtures) that the failed-update render now correctly shows the button as disabled with the "already exists" message and no action link. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: use transaction id instead of entry id in mark-recurring route assertion mark_as_recurring is a member route on the transactions resource keyed by the Transaction's id, not the Entry's id (Entry uses delegated_type, so Entry and its Transaction entryable have distinct ids). The prior assertion built the path from `entry`, which could produce a different URL than the one actually rendered, so the "no href" check could pass even if the button leaked a link. Use entry.entryable so the assertion matches the real route. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: refresh mark-recurring state on turbo_stream update, avoid unconditional query, keep DS::Button href Addresses jjmata's review on PR #3103: - Extract the "Mark as Recurring" block into a dom_id-wrapped partial and replace it in the successful update turbo_stream response, so inline edits that change whether the transaction matches an existing manual recurring transaction are reflected immediately instead of only on the next full page render. - Skip the existing_manual_recurring_transaction lookup entirely when the block won't be rendered (no edit permission, or split-child entry), avoiding an unconditional extra query on every transaction show/failed update. - Keep href present on the DS::Button and only toggle disabled, matching the established pattern elsewhere in the app, instead of nulling href (which flips the component to a bare <button> and leaks a stray method="post" attribute). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Brandon
|
25a9011f14 |
feat(ai): analytical tool-set upgrade for the builtin assistant (#3064)
* fix(assistant): survive tool failures with error and hint results instead of aborting the turn
A tool exception used to raise FunctionExecutionError out of the responder
loop, turning the whole turn into a generic chat error banner. An unknown
tool name was worse: the rescue block itself crashed (fn.name on nil).
Tool failures now come back to the model as data ({error, hint}) so the
conversation survives and the model can retry once with corrected
arguments. The catch-all branch logs and tells the model not to retry.
FunctionExecutionError remains defined for API compatibility.
* fix(assistant): strict schemas declare every property as required
get_categories and get_tags declared an optional page property while
inheriting strict mode, which is invalid under strict function calling
(every property must be listed in required). Both now opt out of strict
mode like every other paginated tool, and gain a page_size param
(1..100) while being touched.
A registry-walking test asserts the invariant for every current and
future tool, preview tools included.
* fix(assistant): HistoryTrimmer always keeps the newest turn
Trimming iterates newest-first and stopped at the first group over
budget. When the newest group alone exceeded the budget, everything was
dropped, including the user message the model was being asked to
answer, and the provider received only the system prompt. The newest
group now always survives.
* perf(assistant): compact AI time series and make account history opt-in
get_accounts shipped a 5-year monthly series for every account on every
call, roughly 60 formatted money strings per account, which dominates
the tool payload for multi-account families and swamps small
self-hosted context windows. The series is now opt-in
(include_balance_series) and bounded by a named period (series_period,
default last_365_days).
to_ai_time_series states the currency once and emits numeric values
instead of formatting every point; the system prompt already tells the
model how to render currency.
get_accounts also now returns account ids (they are what other tools
accept as account_ids filters) and respects the visible scope, so
hidden accounts no longer leak into responses.
* refactor(assistant): id and name filters replace user-data enums in get_transactions
The schema inlined every account, category, merchant, and tag name as
enum values on every request. That grows without bound with family
data, defeats provider prompt caching (definitions change whenever a
name does), and is the pattern that made empty-enum pruning necessary
in the first place.
Filters are now plain string arrays documented as exact names from the
sibling get_* tools, which Transaction::Search already resolves
server-side, plus an account_ids UUID filter. New params: page_size
(1..100), sort_by amount, types (income/expense/transfer, the way to
exclude transfers), and statuses (pending/confirmed).
The three now-unused enum helpers are removed from the base class;
family_tag_names stays for update_tag, which still identifies tags by
name.
* feat(assistant): add get_merchants and get_recurring_transactions
Merchants were unreachable: names appeared nowhere and
update_transaction's merchant_id had no source of ids, making it
unusable. get_merchants lists id, exact name, and source, scoped
through available_merchants_for so merchants seen only in accounts
hidden from the user never leak.
Recurring transactions had a model, an Upcoming view, and no assistant
reach. get_recurring_transactions lists detected and manual recurring
items (status filter defaulting to active, optional
upcoming_within_days window) with per-currency totals of active
non-transfer items, answering subscription and upcoming-bill questions
directly instead of via transaction paging.
* feat(assistant): flexible periods on get_balance_sheet and trends on get_income_statement
get_balance_sheet was hard-wired to five years of monthly history with
no parameters, although Period supports arbitrary ranges and the chart
builder takes any interval. It now accepts a named period or custom
dates plus an interval, with a 400-point cap so a day-granularity
request over a decade returns an error instead of a giant series. The
default call is byte-compatible with the old shape. The balance sheet
object is also memoized; it was being constructed four times per call.
get_income_statement gains the analysis surface the assistant lacked:
group_by month for a monthly income/expenses/net series (capped at 36
buckets), compare_previous_period for an equal-length prior window
with absolute and percent deltas, and account_ids to scope totals to
specific accounts via IncomeStatement#totals_for. Category breakdowns
are family-wide by construction, so the account-filtered view omits
them and says why. Unknown or inaccessible account ids come back as a
soft failure naming the ids so the model can correct itself.
* feat(assistant): preview reads for insights and valuations
The Insights feed is generated nightly with pre-computed numbers, and
the chat assistant could not read a word of it. get_insights returns
the visible feed (type filter, acknowledged toggle, limit) without
marking anything read; an assistant read is not the user viewing the
feed. It sits in PREVIEW_FUNCTION_CLASSES because the feature itself is
preview-gated, which also keeps it off the default /mcp surface.
record_valuation was write-only: an agent recording provenance-cited
valuations had no way to audit what it wrote or find dates already
carrying a value. get_valuations lists valuation entries newest first
with kind and the citation notes, scoped to accessible visible
accounts.
* feat(assistant): cache-stable system prompt with session context
The prompt interpolated currency mid-text and the date near the end, so
no two requests shared a cacheable prefix, and it told the model
nothing about the family: not one account name, not a single category.
Models opened most conversations blind, either wandering through tools
or answering without data.
The prompt is now STATIC_INSTRUCTIONS, a frozen constant that is
byte-identical for every request (providers discount an
exactly-repeated prefix; tool definitions are also stable now that
schemas carry no user data), followed by a trailing Session context
block holding everything volatile: date, date format, currency details,
an account roster with balances, and category names.
The static half gains a request-classification rule (CHAT / LOOKUP /
ANALYSIS), a reuse-what-you-have rule with an explicit re-fetch
carve-out, specific-tool preference, and the error/hint retry-once
rule that pairs with the tool soft-fail contract.
Context stays cheap by construction: the roster collapses to per-type
counts beyond 25 accounts, categories to a count beyond 60 names, and
both collapse whenever the configured context window is under 4096
(the self-hosted default is 2048), via the new Assistant::TokenBudget
helper. Intro chats are untouched.
* feat(assistant): raise tool-round cap to 8 with a no-tools grace turn; instructions-aware history budget
Five rounds was tight for a tool surface that now supports real
analysis chains, and hitting the cap raised ToolCallLimitError, which
surfaced to the user as a dead chat with an error banner. The default
is now eight rounds (env override unchanged), and on the final
permitted round the follow-up request offers no tools, so the model
must answer in text with whatever it gathered. The limit error remains
as a defensive backstop.
The generic-path history budget reserved a flat 256 tokens for a
system prompt that already estimates well past that; the trimmer now
budgets against the actual instructions when available.
LLM_MAX_RESPONSE_TOKENS was reserved in budget math but never sent to
the provider. It is now sent (max_tokens on chat completions,
max_output_tokens on the Responses API) only when explicitly
configured via ENV or a stored Setting; stock installs keep today's
uncapped behavior.
* test(evals): chat golden v2 exercising the real prompt and registry
The eval runner scored a fiction: hardcoded instructions and four fake
permissive tool schemas, so a prompt or registry regression could
sail through green. It now runs STATIC_INSTRUCTIONS plus a fixed
synthetic session context and builds definitions from
Assistant.function_classes against a reference user (classes whose
schema cannot build are skipped with a log line, never faked).
chat_golden_v2 adds routing scenarios the upgrade cares about: CHAT
classification must use no tools, aggregates route to
get_income_statement / get_balance_sheet rather than transaction
paging, and the new analytical tools are selected with sensible
params. The dataset header documents the harness's single-shot
limitation.
* docs(ai,mcp): current tool tables, responder loop, prompt structure, timeout math
Both docs listed 7 tools against a registry of 19, in three separate
drift-prone copies. mcp.md now carries the canonical tables (default +
preview); ai.md links to them from the MCP section, keeps one grouped
functions list for the architecture chapter, and replaces its stale
hardcoded registry snippet with a pointer to assistant.rb.
The architecture section gains the contracts contributors need when
adding a function: the responder loop (rounds vs calls, cap 8, the
no-tools grace turn) and the error/hint soft-failure convention, plus
the prompt's static/session-context split and its collapse gates.
Timeout guidance is recomputed for the new default cap.
* fix(assistant): address automated review findings
Codex and CodeRabbit findings on the initial push, all verified before
changing anything:
- AI time series rounded every value to two decimals, which turns
0.001 BTC into 0.0; values now round to the currency's own precision
(BTC 8, CLF 4, OMR 3).
- get_income_statement validated account_ids against all visible
accounts, but totals_for excludes hidden, excluded-from-reports and
tax-advantaged accounts, so those ids produced silent zeros. Ids now
validate against income_statement.eligible_accounts and the soft
failure explains eligibility.
- get_recurring_transactions computed totals from the displayed rows,
so past the 200-row cap the value labeled a total was partial. Totals
now aggregate over the full filtered scope in SQL, and the response
carries total_results and a truncated flag. The upcoming_within_days
window also starts at today, matching its documentation; overdue
items appear in unwindowed calls.
- get_valuations silently dropped a malformed date filter and presented
unfiltered data as filtered; malformed dates now return invalid_date.
- get_balance_sheet returned a generic failure for a reversed custom
range because Period's own validation raises past the Date::Error
rescue; it now returns the structured invalid_date error.
- get_insights documents that its family-wide scope matches the web
feed exactly (InsightsController serves Current.family.insights to
every member), so the tool exposes nothing the /insights page does
not already show the same user.
- Tests: limit clamp proven against more insights than the cap,
Setting fallbacks stubbed in the provider budget tests, currency
precision and reversed-range regression tests added.
* refactor(assistant): apply reviewer nitpicks
- order declares type alongside its enum, matching sort_by
- page-size clamp deduplicated into the base class (MAX_PAGE_SIZE +
shared resolved_page_size); dead per-tool copies removed
- get_accounts preloads balance rows only when the series is requested
- get_income_statement validates the bucket count before running any
aggregation work
Deliberately unchanged: the balance sheet's monthly_history key. The
default response shape stays byte-compatible for existing MCP
consumers, and the nested series already states its interval.
* fix(assistant): second-round review findings on get_valuations
- A reversed date range (start after end) now returns the structured
invalid_date error instead of presenting an empty result as filtered
data, matching get_balance_sheet's handling.
- Page numbers are normalized before pagination: Pagy raises on zero,
negative or non-numeric pages. The fix lands as a shared
resolved_page helper on the base class and applies to every
paginated tool (categories, tags, merchants, transactions, holdings,
valuations), since all shared the same page-or-1 pattern; schemas
declare minimum: 1.
* fix(assistant): round series amounts as BigDecimal before Float conversion
Converting to Float first can perturb the value at the requested
precision; round the exact decimal, then convert for JSON.
* fix(assistant): address maintainer review findings
- get_accounts no longer fails the whole listing when one account's
start date lies beyond the requested period (start_date derives from
the first entry, which can be future-dated); that account simply has
no series. The unrescued Period.custom was reachable exactly there.
- The balances preload is gone: the series goes through
Balance::ChartSeriesBuilder, which runs its own query keyed by
account ids, so the eager-loaded rows were loaded and discarded.
- Provider::Openai#context_window now delegates to
Assistant::TokenBudget, removing the duplicated ENV > Setting >
default precedence so prompt assembly and the provider can never
disagree about the window.
* fix(ai): final no-tools round uses tool_choice none instead of dropping tools
Anthropic rejects requests whose messages contain tool_use blocks when
no tools are defined, so re-requesting with an empty tool list made the
final-round grace die in a provider 400 on Anthropic models. The final
round now sends the real tool definitions with tool_choice none, which
both providers accept, and the model answers in prose as intended.
* fix(assistant): scope every income statement read to the requesting user
get_income_statement validated account_ids against the user-scoped statement
but computed every total from an unscoped one. IncomeStatement falls back to
Current.user, which is nil in the assistant job and the MCP endpoint, so the
unscoped reads dropped the included_in_finances_for filter and reported
family-wide totals next to ids that had been checked against a narrower set.
Route all reads through one memoized user-scoped statement, the idiom
get_balance_sheet already uses. Also lets the per-instance memoization in
IncomeStatement apply across the eligibility check and the totals.
Adds a regression test that fails without the change, plus a companion test
asserting eligibility and totals agree on scope. Guard the strictness walk
against an empty registry so it cannot silently assert nothing.
|
||
|
Ellion BlessanandClaude Sonnet 5
|
e69894adb9 |
fix(providers): capture API error response body in PDF processor span output (#2937)
* fix(providers): capture API error response body in PDF processor span output Anthropic and OpenAI PDF processing errors only logged the exception message, dropping the parsed response body that usually explains the failure. Add safe_error_body to both providers' UsageRecorder concerns and include it in the langfuse span output on failure, with tests. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013SYp89xEYfTkrxw8HcCUQ8 * fix(providers): allowlist PDF processor error fields sent to Langfuse safe_error_body forwarded the entire upstream error body into the langfuse span output. For custom OpenAI-compatible providers/proxies (and the analogous Anthropic path), that body can echo request content from the financial document being processed. Replace it with safe_error_detail, which extracts only type/message/code/request_id instead of the raw body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013SYp89xEYfTkrxw8HcCUQ8 * test(openai): cover request_id extraction in PDF processor error_detail The safe_error_detail request_id path (error.response_headers) had no test coverage. Stub response_headers with x-request-id and assert it appears in error_detail. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013SYp89xEYfTkrxw8HcCUQ8 --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Abhinav Dhiman
|
e3d46021c2 |
fix: memory leak in sidekiq (#1940)
* perf(sync): reduce per-job memory peak in Balance/Holding materialization
Profiling of SyncJob (StackProf object mode + Sidekiq memory middleware)
showed peaks of 600k-1.1M live heap slots per job and ~196k retained
ActiveModel::Attribute::FromUser objects post-GC, driven by full-history
in-memory accumulation in the balance/holding sync pipeline.
Changes:
- Replace Holding.new / Balance.new in calculators with lightweight
Struct-based HoldingData / BalanceData. Skips AR attribute sets,
belongs_to proxies, dirty tracking, type casting, and callbacks
that were never used (upsert_all bypasses validations/callbacks
anyway). Eliminates ~30% of allocations and the bulk of retained
ActiveModel::Attribute::* instances.
- Build upsert payloads directly from struct fields instead of
Holding/Balance#attributes.slice(...).
- Batch upsert_all in PERSIST_BATCH_SIZE (2,000) slices in both
Balance::Materializer and Holding::Materializer so the intermediate
attribute-hash array is bounded instead of holding the full
multi-year history alongside the calculator output.
- Replace account.holdings.reload with account.holdings.reset in
Holding::Materializer. Same cache invalidation, no eager re-query;
the next consumer (Balance::SyncCache) loads on demand.
- Mutate entries in place in Balance::SyncCache#converted_entries
instead of Entry#dup. The instances are scoped to the throwaway
sync-cache and never persisted, so dup'ing was producing tens of
thousands of unused FromUser/FromDatabase attribute wrappers per
sync.
All persist paths run inside the existing Balance.transaction wrapper,
so batched upserts retain transactional atomicity. No production caller
of Balance::SyncCache or Holding::Materializer reuses the affected
instances outside the materializer's lifetime.
Test coverage: balance/{sync_cache,materializer,forward_calculator,
reverse_calculator} and holding/{materializer,forward_calculator,
reverse_calculator} plus account/syncer and sync (82 runs, 2,548
assertions, 0 failures).
* refactor(holding): stream materializer upserts to bound peak memory
Replace full-array accumulation + each_slice in Materializer#persist_holdings
with two flush-on-fill buffers (holdings_buffer_to_upsert_with_cost /
holdings_buffer_to_upsert_without_cost) that upsert and clear at
PERSIST_BATCH_SIZE, keeping peak RSS bounded to ~2x batch size.
Also add assert_not_nil guards in ReverseCalculatorTest before
dereferencing calculated.find results to surface clear failures
instead of NoMethodError.
* refactor(balance): promote BalanceData to Balance namespace and document sync mutation safety
- Extract Balance::BalanceData struct into its own file (app/models/balance/balance_data.rb)
so it is discoverable without knowing it lived inside BaseCalculator
- Remove inline Struct definition from Balance::BaseCalculator; update build_balance
to reference Balance::BalanceData explicitly (required because class Foo::Bar syntax
does not nest Foo in constant lookup)
- Add comment to SyncCache#converted_entries clarifying that to_a materialises
independent AR instances with no identity map active, making in-place mutation safe
- Update all Balance::BaseCalculator::BalanceData references in materializer_test
* test(balance): use to_h instead of attributes on BalanceData struct in waypoint test
* perf(balance,market_data): replace sort_by + first/last with minmax_by and push account-entry join to SQL
- Balance::Materializer#purge_stale_balances: replace sort_by(&:date) + first/last with minmax_by(&:date) to avoid full sort when only min/max are needed
- MarketDataImporter: replace Entry.group(:account_id).minimum(:date) (which loads all account IDs into a Hash) with a LEFT JOIN subquery that computes MIN(date) per account in SQL and exposes it as first_entry_date on the Account relation
* test(balance): use BalanceData struct in materializer purge test
* fix(holding): carry cost_basis forward onto gap-filled dates
|
||
|
Atlasandsure-admin
|
eb382b8899 |
Fix onboarding country and currency defaults (#3070)
* fix: derive onboarding currency from country * feat: use browser locale for onboarding defaults * Fix onboarding currency defaults * Preserve saved onboarding currency during hydration --------- Co-authored-by: sure-admin <sure-admin@splashblot.com> |
||
|
|
e5750a6c09 |
feat(budgets): add per-user personal budgets with strict isolation (#2891)
* feat(budgets): add per-user personal budgets with strict isolation Families can now opt into personal budgets (toggleable via family settings): each family member gets their own budget for a given period instead of sharing a single family-wide budget. - Add families.personal_budgets flag and budgets.user_id, with partial unique indexes so shared budgets (user_id IS NULL) and personal budgets (user_id IS NOT NULL) can't collide. - Budget.find_or_bootstrap scopes lookup/creation by user when the family has personal_budgets enabled. - Scope most_recent_initialized_budget (used to seed a new budget from the prior period) by user_id so one user's copy-forward never bleeds into another user's budget. - budgets.user_id cascades on user deletion so personal budgets don't outlive their owner. * feat(budgets): enforce user-specific budget ownership and cascade deletion * feat(budgets): display user name for personal budgets in budget card on the plan section * feat(budgets): enhance personal budgets display for admins with preview feature indication * feat(budgets): enforce user-specific budget and category visibility for personal budgets * feat(budgets): create budget section titles and add translations notice in preferences * feat(budgets): let household and personal budgets coexist with sharing Previously enabling personal_budgets made the shared household budget unreachable. Budget.find_or_bootstrap now takes an explicit household: flag so both can be resolved independently for the same period, with a new household_budget_enabled family setting to opt out of the household side and keep personal budgets only. Adds a BudgetShare model (read_only/read_write) so a member can grant another family member access to their personal budget, enforced via Budget#viewable_by?/editable_by? across BudgetsController, BudgetCategoriesController, PlansController, and the read-only API. Preferences gains a Budget sharing card (gated on preview access like the rest of the personal budgets UI) and an owner switcher pill ( Household / mine / shared-with-me) appears on the budget page and the Plan hub card. Also fixes personal budgets showing the same "actual spending" as the household budget: actual spending/income now scope to the budget owner's own accounts instead of the viewer's full accessible set, via a new accounts: override on IncomeStatement. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * feat(budgets): enhance budget switcher with icons and improved styling * feat(budgets): remove user name display from budget card and header * feat(budgets): remove unique index on taggable_type and taggable_id in taggings * feat(budgets): enhance budget sharing functionality and improve UI elements * Collapse personal budget migrations --------- Signed-off-by: JulienGourmet <69808509+jubbakka@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: sure-admin <sure-admin@splashblot.com> |
||
|
Scott HughesandClaude Opus 5
|
5bba880e66 |
Fix SnapTrade holdings by using the /positions/all endpoint (#3043)
* Fix SnapTrade holdings by using the /positions/all endpoint SnapTrade returns HTTP 410 Gone on /positions, /holdings and /options for apps registered after their 2026 cutoff, so newly connected accounts import their balance but never any holdings. The importer rescues the error and the sync still reports success, which makes it look like the brokerage simply holds nothing. Switch get_positions to the documented replacement, /positions/all, and teach the shared payload helpers to read its flat `instrument` object alongside the legacy nested `symbol.symbol` shape. Fixes #3029 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Filter unsupported instrument kinds before they reach the payload Derivatives skipped by HoldingsProcessor still landed in raw_holdings_payload, where calculate_holdings_value sums units * price over every entry. Any non-zero sum then displaces SnapTrade's own figure in calculate_total_balance, so an options-only account could report a balance derived from per-contract units against a per-share price. Move the denylist to Provider::Snaptrade#get_positions so unsupported kinds never enter the payload at all, keeping holdings, balance, currency detection and cash-equivalent handling consistent with one filter. Also raise when the positions response carries no results array, so a partial or schema-changed response leaves the previous snapshot in place instead of overwriting it with nothing. An empty results array remains a legitimately empty account. Note that tax_lots[].cost_basis is documented as a whole-lot total, unlike the per-share field this reads. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Sure Admin (bot)
|
b0edf99262 | Use moniker interpolation for user family labels (#3060) | ||
|
GFRandClaude Sonnet 5
|
d0bb1a31e8 |
fix(recurring): include amount in manual recurring duplicate check (#2972)
* fix(recurring): include amount in manual recurring duplicate check TransactionsController#mark_as_recurring blocked a second manual recurring transaction whenever an existing one shared the same account + payee name/merchant + currency, even when the amount differed -- stricter than the DB unique indexes (idx_recurring_txns_acct_name / idx_recurring_txns_acct_merchant), RecurringTransaction::Identifier's own grouping key, and the equivalent check already used in TransfersController#mark_as_recurring. Add amount to the duplicate lookup so two distinct recurring payments to the same payee at different amounts are both allowed, while an exact duplicate is still blocked. Also rescue ActiveRecord::RecordNotUnique around the create call so a race between the pre-check and the DB constraint (e.g. a double-submit) surfaces the same friendly "already exists" message instead of a generic error, mirroring the existing race-handling pattern in RecurringTransaction::Identifier. Fixes #2936 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(recurring): don't blend distinct charge amounts into variance band Once two manual recurring rows with the same payee/different amounts can coexist (this PR), RecurringTransaction.create_from_transaction's variance-band discovery still matched historical entries only by account/payee/currency/day-window -- never by amount -- so it could blend genuinely unrelated charges (e.g. a fee + a due from the same merchant, same day) into one row's expected_amount_min/max/avg. Flagged by Codex review on this PR. Confirmed this is not hypothetical: two real production transactions (3.00 and 19.68, same merchant, same day) got blended into a single recurring row showing a fabricated "11.34" projected amount that matches neither real transaction. The same unfiltered matching independently exists in RecurringTransaction::Identifier#manual_recurring_matches_entry?, which periodically re-derives every manual recurring row's variance after each sync (via IdentifyRecurringTransactionsJob). Both call sites needed the fix together, or the job would silently re-blend amounts on the next sync. Add RecurringTransaction.amount_within_variance_band?(candidate, anchor, ratio: 2) -- a candidate only counts as "the same fluctuating payment" if it's within 2x (double/half) of the anchor. Anchored on the target amount (not pairwise) so unrelated charges can't chain together; ratio-based (not %-of-target-with-floor) so it's scale-invariant and handles signed (expense) amounts correctly. Threshold checked against real data: existing variance test fixtures sit at ~1.2-1.3x (must stay included), the real corrupted case sits at ~6.6x (must be excluded) -- 2x leaves comfortable margin on both sides. Wire this into find_matching_transaction_entries/ find_matching_transaction_amounts (SQL-level filter, same pattern as the existing day-of-month bounds) and into manual_recurring_matches_entry?. amount_window_scope/ matching_transactions and create_from_transfer need no changes -- confirmed by reading: the former only consumes an already-computed band, the latter never does variance discovery at all. Does not touch any already-corrupted production data -- deliberately out of scope, discussed separately. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Guillem Arias FausteandJuan José Mata
|
acf4cb2010 |
fix(goals): allow deleting a goal without archiving it first (#2963)
* fix(goals): allow deleting a goal without archiving it first
Goals could only be deleted after being archived. `GoalsController#destroy`
redirected with "Archive the goal before deleting it." unless the goal was
already archived, and the Delete item in the show-page kebab was wrapped in
`if @goal.archived?`. Nothing in the archive confirm copy hinted that
archiving was the prerequisite, so in practice an active goal had no delete
affordance anywhere in the UI.
The gate bought no safety. Destroying a goal cascades only to its own
`goal_accounts` and `goal_pledges`, and `GoalPledge#clear_matched_transaction_extra`
unstamps `extra["goal"]["pledge_id"]` from any transaction a matched pledge
claimed. No account, balance, entry or transaction is touched. Every other
resource in Sure (accounts, categories, rules, family merchants) deletes in
one step.
Drop the gate, render Delete unconditionally, and shorten the label from
"Delete permanently" to "Delete" now that it no longer needs to contrast
with an archive-first step.
The confirm copy moves to `Goal#deletion_confirm` and spells out what
survives. The generic `CustomConfirm.for_resource_deletion` only says "This
is not reversible", which overstates it for a goal.
Index cards deliberately keep no actions — the card stays a single click
target, and the show-page kebab is one click away.
* fix(goals): escape the goal name in the delete confirmation
`confirm_dialog_controller` assigns the confirm `body` to `innerHTML` — bodies
such as the accounts' `confirm_body_html` legitimately carry markup — so a goal
named "<img src=x onerror=…>" ran as soon as a family member opened the delete
confirmation. Verified in a browser: parsing the rendered `data-turbo-confirm`
and assigning its body produced a live `<img>` element with a working `onerror`
handler.
Escape the interpolated name. Only `body` needs it; the dialog sets its title
and button label with `textContent`.
`CustomConfirm.for_resource_deletion` interpolates a record name into the same
HTML-rendered body and was already reachable from accounts, categories, rules
and family merchants, so it is escaped here too rather than left as a known
hole next to the fixed one.
Also add the three `confirm_delete_*` keys to every locale that ships goal
translations. Fallbacks meant these silently rendered English rather than
breaking, so this is untranslated copy rather than a fault — ru is included,
which the review list omitted.
* i18n(confirm): move the resource-deletion copy to locale keys
`for_resource_deletion` built its title, body and button label as English
string interpolation, against the project's rule that user-facing strings go
through `t()`. It backs ~39 call sites — accounts, rules, tags, chats, every
provider item — so all of them were English-only.
Moved to `shared.custom_confirm.resource_deletion_*`, alongside the
`default_*` keys the same class already used.
`titleize` / `downcase` stay applied to the record name so the English output
is byte-identical to what the hardcoded strings produced; a locale needing
different casing can absorb it in its own string. Pinned by a test, along with
the escaping of the one field the dialog renders as HTML.
* i18n(confirm): translate the resource-deletion copy
The keys added when this copy moved out of hardcoded English only landed in
en.yml, leaving ~40 call sites falling back to English in every other locale.
Added to the eight other shared locale files that already carry the sibling
`custom_confirm.default_*` strings: ca, fr, hu, it, ru, tr, vi, zh-CN. Each
body reuses that locale's own "this is not reversible" sentence, so the
generic and resource-specific confirmations read the same, and each follows
the register its `default_title` already set (vous / siz / Вы, tu for ca).
The remaining shared locale files (de, es, nb, nl, pl, pt-BR, ro, zh-TW) have
no `custom_confirm` block at all, so they are left alone — adding one would
invent structure they have not adopted, and fallbacks already cover them. The
test derives its locale list from which files define the sibling key rather
than hardcoding it, so it follows that set as it grows.
* test(goals): restore the active-goal destroy test lost in the merge
Merging main into this branch hit a conflict in
`test/controllers/goals_controller_test.rb`: main had added two tests
immediately above the destroy block, and the resolution took main's side
wholesale for that hunk. That resurrected `destroy on non-archived is
rejected` — the test this PR replaces — and dropped its replacement.
The resurrected test failed against the new controller, since destroy no
longer gates on `archived?`:
GoalsControllerTest#test_destroy_on_non-archived_is_rejected
`Goal.count` didn't change by 0, but by -1.
Swap it back for `destroy deletes an active goal and cascades to its
links and pledges`. Main's two new tests stay.
* i18n(goals): finish the delete copy in de and zh-TW
Nine locales ship goals translations, not seven. `de.yml` and `zh-TW.yml`
were left behind: both still carried the dead `goals.destroy.archive_first`
key, still labelled the kebab item "Delete permanently" (Endgültig löschen
/ 永久刪除) after it was shortened elsewhere, and had none of the
`confirm_delete_*` keys, so a German or Traditional Chinese family saw the
new delete dialog in English.
Add the three confirm keys using each file's existing vocabulary — Zusagen
for pledges in German (informal du, matching the rest of the file), 投入 in
Traditional Chinese — drop `archive_first`, and shorten the label.
`confirm_delete copy resolves in every locale that ships goal translations`
could not have caught this. It hardcoded the seven locales, and its
assertions went through plain `I18n.t`: the backend has
I18n::Backend::Fallbacks mixed in, so a missing German key resolved to the
English string and `.present?` passed anyway. Verified — deleting
`confirm_delete_title` from `de.yml` left the test green.
Derive the locale list from the goals YAMLs and look the keys up with
`fallback: false, default: nil`. The same deletion now fails with
"de is missing goals.show.confirm_delete_title".
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
|
||
|
Sure Admin (bot)
|
75aa16e4e2 |
Log async rule run failures to debug log (#3045)
* Log async rule run failures to debug log * Propagate auto-categorize provider failures |
||
|
Andrew B
|
c9fbfd9f71 |
fix(chat): make the assistant response timeout configurable (#2910)
* fix(chat): make the assistant response timeout configurable (#2893)
Self-hosted users running a local model report the chat failing with
"assistant not available" after 90 seconds even though the model
generates a reply and tokens are billed.
Three timeouts are involved and only one was configurable:
- OPENAI_REQUEST_TIMEOUT (60s) — already settable, not the blocker.
- The browser watchdog in chat_controller.js (90s) — hardcoded, and
this is what actually fires.
- Chat::UNDELIVERED_RESPONSE_TIMEOUT (60s) — a constant, so raising
the client value alone would not have helped.
The watchdog cannot be avoided by streaming here: custom
OpenAI-compatible providers route through generic_chat_response, which
forces synchronous calls, so nothing renders until the whole generation
finishes. Time-to-last-token has to beat the deadline.
Adds AI_RESPONSE_TIMEOUT (ENV > Setting > 90s default, floored at 30s),
exposed on the Self-Hosting settings page and passed to the Stimulus
controller at all three mount points — show, index and the sidebar in
the application layout, each of which declares data-controller="chat"
independently.
The server floor is derived from the same value but kept 10s below it.
report_timeout answers 200 whether or not it acted and the client only
retries on a non-ok response, so a floor at or above the client value
would let clock skew strand a pending bubble permanently.
Also guards AssistantMessage#append_text!. The watchdog runs in the web
process while the job holds its own copy of the message, so a job
finishing after the bubble was destroyed or demoted would silently
resurrect it alongside the error the user was already shown.
* fix(chat): let the watchdog retry when report_timeout declines
`report_timeout` answered 200 whether or not `handle_undelivered_response!`
acted. The Stimulus watchdog only stops retrying a URL once it sees a 2xx, so
a declined report was treated as final.
That stranded the bubble whenever the client's clock ran more than
SERVER_TIMEOUT_GRACE ahead of the server's: the watchdog posts at its own
timeout, the server sees a message younger than its floor and no-ops, and
nothing ever retries. The bubble spins forever with no error and no Retry.
Answering 409 instead lets the next 5s tick try again, so any amount of skew
costs retries rather than a stuck chat. The grace window stays as an
optimisation to keep those retries rare, not as the correctness mechanism.
* docs(chat): correct the AI_RESPONSE_TIMEOUT ordering guidance
The docs, locale string and examples all said to keep OPENAI_REQUEST_TIMEOUT at
or above AI_RESPONSE_TIMEOUT. That is backwards.
The two limits span different things. OPENAI_REQUEST_TIMEOUT bounds each HTTP
call to the model on its own; AI_RESPONSE_TIMEOUT covers the whole turn and its
clock starts when the message is queued, so it also absorbs Sidekiq queue time
and, for a tool-using turn, two model calls plus the tool run between them.
Keeping the chat timeout the larger of the two means a slow model surfaces the
specific HTTP timeout error rather than a generic "no response", and the job
stops instead of running on after the chat has given up. The shipped 60/90
defaults already had this ordering; only the guidance was wrong.
compose.example.ai.yml gets 300/660 so the Ollama example can actually complete
a tool-using turn.
* fix(chat): claim the pending bubble atomically before appending
append_text! read the row's status and then saved, leaving a window in which
the watchdog could demote the row to `failed` between the two. The late
content would then land on a bubble the user had already been told failed,
flipping it back to `complete`.
Replaces the read with a conditional UPDATE that only succeeds while the row is
still pending, so the check and the state change cannot be separated.
Uses a conditional UPDATE rather than with_lock because append_text! is called
once per chunk on the streaming path; a row lock and transaction per chunk would
be far more expensive. The claim only runs on the first append, since later ones
are no longer pending.
* test(chat): isolate AI_RESPONSE_TIMEOUT from the environment
Chat.response_timeout reads ENV ahead of Setting, so stubbing only the Setting
left the assertions at the mercy of the environment they run in. With
AI_RESPONSE_TIMEOUT=45 exported, five of these tests failed — the default,
floor and grace assertions were all silently measuring the env value.
Adds a with_setting_timeout helper that stubs the Setting and clears the
variable together, and switches the controller tests to stub
Chat.undelivered_response_timeout directly, since what they care about is the
resolved floor rather than how it was configured.
Both files now pass with or without AI_RESPONSE_TIMEOUT set.
* docs(chat): size AI_RESPONSE_TIMEOUT for chained tool calls
The guidance assumed a tool-using turn costs two model calls. #2767 landed
after this branch was opened and made tool calls iterative: `Assistant::Responder`
now loops until `iteration > max_tool_call_iterations`, so a turn runs to
1 + ASSISTANT_MAX_TOOL_CALL_ITERATIONS calls — six by default — with tool
execution in between. At the default 60s per-call timeout that is up to 360s of
model time against a 90s watchdog.
Streaming does not rescue this either. `emit(:output_text)` only fires for a
response that carries text, and tool-call-only rounds carry none, so the bubble
stays on "Thinking…" through every round regardless of provider.
Documents ASSISTANT_MAX_TOOL_CALL_ITERATIONS as the cheaper lever: dropping it to
2 halves the worst case instead of demanding a half-hour timeout, at the cost of
failing long tool chains earlier with a clear limit error. compose.example.ai.yml
now shows that combination rather than a timeout sized for six calls it never had.
* docs(chat): state the whole-turn timeout as a sum, not a maximum
The guidance said to keep AI_RESPONSE_TIMEOUT "above" or "the larger of"
OPENAI_REQUEST_TIMEOUT. That understates it: the watchdog covers the entire turn,
so the bound is
(1 + ASSISTANT_MAX_TOOL_CALL_ITERATIONS) * OPENAI_REQUEST_TIMEOUT
+ tool execution + queue wait
Merely exceeding the per-call limit can still leave the chat reporting failure
while the worker keeps going.
One phrasing was outright wrong: "keep AI_RESPONSE_TIMEOUT the largest of the
three" compared a duration against ASSISTANT_MAX_TOOL_CALL_ITERATIONS, which is a
count, not seconds.
Resizes the examples against the formula — compose.example.ai.yml 1000 -> 1200 and
the Ollama doc example 600 -> 720, both now showing the arithmetic — and states
plainly that the 90s default is sized for typical cloud latency rather than the
worst-case bound, with the formula being what matters once per-call latency
approaches the timeout.
* docs(chat): list the AI settings fields and tag the formula fence
The Settings UI walkthrough listed three of the eight fields on the AI Provider
form. JSON Mode, the three Token Budget fields and the new Chat Response Timeout
were all missing, so the timeout was only discoverable from the troubleshooting
section. Rewrites the list to follow the form's own grouping and uses the labels
the form actually renders.
Also tags the whole-turn formula fence as `text` (markdownlint MD040).
* fix(compose): forward ASSISTANT_MAX_TOOL_CALL_ITERATIONS in standard compose
This file enumerates container environment explicitly — there is no env_file — so
a variable absent from the x-rails-env anchor never reaches web or worker.
The tool-call cap was only named in a comment here, while the docs added in
|
||
|
Sure Admin (bot)
|
73d43bc1d0 |
Fix SSO JIT new family creator role (#3024)
* Fix SSO JIT new family creator role * Preserve super admin SSO creator defaults * Update new family creator role test |
||
|
Brandon
|
1973c557e5 |
fix(ai): drop empty data-driven enums from assistant function schemas (#3016)
* fix(ai): drop empty data-driven enums from assistant function schemas
Enum values in tool schemas are built from family data (account names,
categories, merchants, tags, tickers). A family with none of these gets
enum: [], which is invalid JSON Schema. OpenAI tolerates it, but strict
OpenAI-compatible providers reject the entire request, breaking chat for
fresh families until they create a tag or merchant.
Prune empty enums in build_schema, falling back to a plain string. One
choke point covers every function and both consumers: chat tool
definitions for all providers, and the /mcp endpoint's tools/list.
* fix(ai): address review feedback on enum pruning
Stop recursion at populated enum values: enum members are literal
values, not subschemas, so a literal like enum: [{ enum: [] }] must be
preserved verbatim rather than rewritten.
Also cover PREVIEW_FUNCTION_CLASSES in the registry regression test by
enabling the preview preference on the test user, with a guard assertion
so the test fails if preview functions ever silently drop out.
|
||
|
GFRandClaude Sonnet 5
|
746d56c4bd |
fix: gracefully handle invalid family timezone instead of crashing (#2821)
* fix: gracefully handle invalid family timezone instead of crashing Family#timezone is a free-text IANA zone name with no validation on write. If it becomes stale (e.g. tzdata renames a zone, like the historical Europe/Kiev -> Europe/Kyiv switch) or a migration meant to remap legacy names never ran, Localize#switch_timezone passed the raw string straight to Time.use_zone, which raises ArgumentError for any unrecognized zone. Since switch_timezone runs as an around_action on every request, this crashed the entire app for the affected family, including the login page. Now validates the zone via ActiveSupport::TimeZone[] first and falls back to the app default (logging a DebugLogEntry) instead of raising. The log write is debounced per (family, bad value) via Rails.cache (once per day) so an affected family doesn't write one DebugLogEntry row per page view indefinitely. Fixes #390 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: address review feedback on timezone fallback - Make the invalid-timezone debounce lease atomic. Rails.cache.fetch is read-then-write, not atomic, so two concurrent requests could both observe a cache miss and both log before either write landed. Rails.cache.write(unless_exist: true) maps to Redis's atomic SET NX in production, so only one request ever wins the lease. (via CodeRabbit) - Stop using "Europe/Kiev" as the invalid-timezone value in tests. Whether ActiveSupport::TimeZone still resolves that legacy alias depends on the host's installed tzdata version (tzinfo-data is Windows/JRuby-only per Gemfile), so the test's pass/fail behavior wasn't deterministic across machines/CI. Use a deliberately nonexistent name instead. (via Codex) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix: validate Family#timezone on write to address root cause of #390 The previous commit made the *crash* graceful, but left the actual defect in place: nothing stopped an unrecognized IANA zone name from being written to Family#timezone in the first place (direct DB/API access, an old dump predating a tzdata rename, or a future rename of a currently-valid zone). Add a Family-level validation using the same ActiveSupport::TimeZone[] lookup Localize#resolved_timezone uses at request time, so "valid at save" and "valid when rendering" can't drift apart. Deliberately not `inclusion: { in: ActiveSupport::TimeZone.all.map(&:name) }`, matching the neighboring locale/date_format validations: verified empirically that the settings form submits `tz.tzinfo.identifier` (e.g. "America/New_York"), not `tz.name` (e.g. "Eastern Time (US & Canada)"), and those differ for all 150 zones Rails ships. An inclusion check against `.name` would have rejected every legitimate value the form submits. The validation only runs when timezone is actually being changed (if: :timezone_changed?). A family with a pre-existing bad value (the exact #390 scenario) must still be able to save unrelated changes -- otherwise this would turn a previously-harmless bad value into a blocker for any other settings update or background job touching that family's record. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Guillem Arias Fauste
|
214a139a7e |
fix(goals): stop the days-left phrase splitting across lines (#2970)
* fix(goals): stop the days-left phrase splitting across lines The goal header read "Target 700 € by 08 de Febrer de 2027 · 184 days left" as one text node, so on a phone it wrapped wherever it ran out of room — "184 days" on the first line and "left" orphaned on the second. `header_summary_parts` now returns the dot-separated segments instead of a joined string, and the view renders each as its own span. Only the segments after the first are `whitespace-nowrap`: the first carries the target and a long-format date and has to stay free to wrap, or it would overflow a narrow screen. Checked at 320px, where the first segment takes two lines and "675 days left" still moves down whole. * test(goals): pin the clock for the days-left assertion The count is `target_date - Date.current`, and the target was set from `184.days.from_now` a moment earlier — a suite crossing midnight between the two would compute 183 and fail. Wrap it in travel_to, matching the idiom in budget_category_test and family_export_test. The neighbouring 'omits days left once reached' test asserts only the segment count, which 183 and 184 satisfy equally, so it is left alone. |
||
|
Blaž Dular
|
37301e15ae |
feat(mcp): add pagination to get_tags and get_categories tools (#2492)
* feat(mcp): add pagination to get_tags and get_categories tools Both tools now accept a `page` param and return `total_results`, `total_pages`, `page`, and `page_size` — matching the pattern used by get_transactions and get_holdings. Adds a migration for composite (family_id, name) indexes on tags and categories to support efficient paginated ordering within a family scope. * fix(mcp): make page param optional in get_tags/get_categories schema and fix migration version Page defaults to 1 in call() so marking it required in the JSON schema was incorrect. Also fixes migration to use ActiveRecord::Migration[7.2] instead of [8.0]. * fix: update schema.rb with family_id+name indexes for tags and categories * fix(mcp): stable pagination sort and unique family/name indexes on tags and categories - Make family_id+name indexes unique (matches existing AR uniqueness validations) - Add id tie-breaker to alphabetically/alphabetically_by_hierarchy scopes so paginated results are deterministic * refactor(mcp): replace Pagy::Backend mixin with Pagy.new in model layer Pagy::Backend is designed for controllers; using it in plain model classes is fragile. Switch all four assistant functions to call Pagy.new(count:, page:, limit:) directly and apply offset/limit on the scope, which is the correct approach for non-controller contexts. |
||
|
Faldy Ikhwan Fadila
|
792047b82e |
feat(yahoo_finance): add Indonesia Stock Exchange (XIDX) support (#3000)
Add JKT → XIDX exchange MIC mapping, .JK symbol suffix normalization, IDR default currency, and ID country code for Jakarta exchange. Yahoo Finance returns Indonesian stocks (e.g. BBCA.JK) with exchange code 'JKT'. Without this mapping, the provider cannot resolve the exchange to the XIDX MIC already defined in config/exchanges.yml, and normalize_symbol cannot append the .JK suffix for price lookups. Tested manually: Yahoo Finance search and chart endpoints return valid results for IDX tickers (BBCA.JK, currency=IDR, timezone=WIB). |
||
|
Pedro Santos
|
7c56c8e2e8 |
Enable Banking: progressive date_from fallback on WRONG_TRANSACTIONS_PERIOD (#2992)
* Enable Banking: progressive date_from fallback on WRONG_TRANSACTIONS_PERIOD Some ASPSPs (e.g. Santander Totta and Activo Bank in PT) reject the transactions window with 422 WRONG_TRANSACTIONS_PERIOD but do NOT return a corrected date_from in the payload. The existing single-shot retry only fires when the API supplies detail.date_from, so for these banks the retry was skipped and the error surfaced as the generic "communication error"; transactions never synced even though the connection and session were valid. This adds a bounded, progressive fallback: when the period is rejected and no corrected date is available, retry with progressively shorter windows (89 -> 60 -> 30 days). The ASPSP-suggested date is still preferred on the first retry, so existing behaviour is preserved. The step-down only moves the window forward, guaranteeing progress and avoiding an infinite loop. Verified on a live instance: Activo Bank went from 0 to 82 transactions imported once the fallback kicked in. Refs #2989 Signed-off-by: Pedro Santos <pedro_santos@outlook.pt> * Address review: forward-only progress across all fallback windows - Accept the ASPSP-suggested corrected date only when it moves the window forward (current is nil or corrected > current), preserving the forward-only retry bound (CodeRabbit). - Skip fallback windows that are not newer than the current date_from and pick the first that advances, instead of bailing out on a stale first candidate. Fixes the case where an initial/user lookback (e.g. 45d) is newer than the first window (89d) but the bank caps at 30d (Codex). Signed-off-by: Pedro Santos <pedro_santos@outlook.pt> * Add tests for progressive transactions date_from fallback Covers the two cases the previous single-shot retry missed: - WRONG_TRANSACTIONS_PERIOD without a corrected date_from -> falls back to the first shorter window (89 days). - An initial lookback newer than the leading windows (45d) -> skips the 89/60 windows and retries with the first that advances (30 days). Signed-off-by: Pedro Santos <pedro_santos@outlook.pt> --------- Signed-off-by: Pedro Santos <pedro_santos@outlook.pt> |
||
|
Brandon WolfandClaude Fable 5
|
00b7252fbf |
feat(mcp): Add MCP budget update tool (#2908)
* feat(mcp): Add MCP budget update tool Adds an update_budget assistant/MCP function so AI assistants can write monthly budgets: total budgeted spending, expected income, and per-category allocations in one transactional call. - Month resolution and slug format mirror get_budget (YYYY-MM or MMM-YYYY, custom month start respected); targeting a valid month with no budget row bootstraps it via Budget.find_or_bootstrap, same as the budgets UI. - Category allocations accept an exact (case-insensitive) name or id and go through BudgetCategory#update_budgeted_spending!, so subcategory writes keep the parent total in sync. - All writes in one call share a transaction: an invalid category rolls back a totals change from the same call. - Family-scoped like the budgets UI; amounts validated non-negative. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(mcp): harden update_budget per review feedback - Extract shared month resolution into Assistant::Function::MonthResolvable so get_budget and update_budget can't drift on custom month starts - Run budget bootstrap inside the update transaction so a failed entry no longer leaves a newly created budget behind - Apply explicit parent amounts after subcategory syncs so results don't depend on the caller's array order - Reject non-finite amounts (NaN/Infinity) - Explain the synthetic Uncategorized bucket instead of a generic category-not-found error Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
packetsnscripts
|
babb039ad1 |
Fix wise imports only 90 days history on initial setup (#2998)
* Wise connection get full transaction history and adjust for fees * undo devcontainers change * address comments in PR |
||
|
FalloutmanandJustin
|
fc130a1957 |
fix: add Schwab to SimpleFIN total-basis cost_basis allowlist (#2986)
Charles Schwab's SimpleFIN feed reports `cost_basis` as the total position cost rather than per-share, violating the spec in the same way Vanguard (#1182) and Fidelity (#1718) already do. Schwab wasn't on the TOTAL_BASIS_INSTITUTIONS allowlist, so the raw total was stored directly into holdings.cost_basis and treated as per-share downstream. Holding#calculate_trend multiplies avg_cost by qty again when reconstructing original cost, so an unadjusted total gets squared by share count — a $46,950 position with a true +55.7% gain rendered as -99.8% / -$19.6M "return" on the dashboard and per-account holdings views. Picks up where #2626 left off: adds the allowlist entry, fixes the now-outdated compliant-institution test case it broke, and adds a dedicated regression test using real observed Schwab payload values. Fixes #2626 Co-authored-by: Justin <justin@local> |
||
|
William Wei MingandCursor
|
62fd47def9 |
Add “Is not equal to” operator for transaction amount rules (#2922)
* Add not-equal operator for transaction amount rules Enable excluding a specific amount in rule conditions without needing paired greater/less than workarounds (#2882). Co-authored-by: Cursor <cursoragent@cursor.com> * Strengthen amount not-equal absolute-value coverage Include a -100 transaction so != 100 proves both signed amounts are excluded. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
William Wei MingandCursor
|
35c0b08f11 |
Add tags support for transfer transactions (#2921)
* Add tags support for transfer transactions Expose TagSelect on transfer create/edit so users can classify fund movements; apply the same family-scoped tags to both sides. Co-authored-by: Cursor <cursoragent@cursor.com> * Require annotate permission on both transfer sides for tags Prevent tagging a read-only destination transaction when the user only has write access on the outflow account. Co-authored-by: Cursor <cursoragent@cursor.com> * Restore transfer tag selections on create form errors * Localize transfer create validation error messages --------- Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
UnamedRusandClaude Opus 4.8
|
8c52e20906 |
fix(binance): correct base_url for futures api endpoint, start_time param (#2839)
* Update binance.rb Signed-off-by: UnamedRus <UnamedRus@users.noreply.github.com> * Update binance.rb Signed-off-by: UnamedRus <UnamedRus@users.noreply.github.com> * Update binance.rb Signed-off-by: UnamedRus <UnamedRus@users.noreply.github.com> * Fix parameter naming for get_spot_trades and get_futures_trades Signed-off-by: UnamedRus <UnamedRus@users.noreply.github.com> * Update processor_test.rb Signed-off-by: UnamedRus <UnamedRus@users.noreply.github.com> * reset startTime after fromId * Fix Binance trade sync param conflict with windowed initial fetch Binance rejects fromId combined with startTime/endTime, and an unbounded startTime (default sync ~1yr) exceeds the window cap (spot 24h, futures 7d), so the initial sync could fail or miss trades. Split fetch_new_trades into two non-mixing paths: - incremental (cached trades): fromId-only pagination - initial sync: walk forward in fixed windows (24h spot / 7d futures), clamped to the 6-month futures lookback Add endTime param to get_spot_trades/get_futures_trades. Add tests covering multi-window initial sync and multi-page fromId pagination. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Signed-off-by: UnamedRus <UnamedRus@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
efb7cc3935 |
Tooling for the wealth + tax agent harness (#2848)
* Expose the Statement Vault to external agents over MCP A user wants to manage patrimonial history — a document-backed record of a family's wealth where every figure traces back to the statement it came from — by pointing an external agent harness at Sure. That model belongs in the harness, not in Sure: it needs numbered build deltas, golden tests and closed periods that a mutable Postgres row cannot provide. What Sure was missing was the seam. The Statement Vault already does most of the work — original bytes retained, SHA-256 dedup, period detection, account matching with a confidence score, reconciliation against ledger balances, and a month-by-month coverage map — but it is reachable only from the web UI. An agent could not archive a document, cite one, or check for gaps. Adds five preview MCP tools over what already exists, plus a citation grammar for values the agent writes: - upload_account_statement, list_account_statements, get_account_statement, get_statement_coverage - record_valuation, whose source citation is parsed rather than trusted: ["estimated: "] citation [" (grade: A|B|C)"]. An uncited or free-styled value is rejected at the write boundary instead of landing in the ledger looking authoritative. link and reject are deliberately not exposed. Attaching a statement to an account is the human's decision, and the vault UI is where it is made; the agent reports the suggested match and stops there. Assistant.function_classes now takes a user so preview tools stay out of the default surface. They are hidden from tools/list and not callable by name without the preference enabled, and the vault tools re-check the manager role and per-account permissions, since MCP calls never pass through a controller. Docs: the blueprint this implements, and a guide covering which side owns which layer, the vocabulary map between the two, the monthly runbook, and the gaps (non-user holders, non-statement documents, one value per date). No migrations, no API endpoints, no UI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * Address review feedback on the vault MCP tools Two non-blocking items from the review pass: Document why get_statement_coverage reads through accessible_by rather than writable_by. It reports which documents exist and writes nothing, so read access is the right bar — and tightening it would hide coverage gaps from people who can already see the figures those gaps sit behind. The comment exists so a future refactor doesn't "fix" it. Close the acknowledged verification gap with tests rather than a one-off manual check. The review noted that nothing proved a real vault payload serializes cleanly out through tools/call — vault responses are richer than the other tools' output, with nested account hashes, decimal balances, dates and a compacted hash. Two integration tests now drive the real /mcp endpoint end to end against a real AccountStatement: one listing it, one uploading bytes and reading back the SHA-256. Permanent regression coverage instead of a smoke test someone has to remember to repeat. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * docs(llm-guides): replace patrimonial blueprint with its final revision Swap the embedded early draft for the authoritative final revision of the wealth + tax modelling blueprint (MIT © 2026 diegomarino): - rename the domain vocabulary: patrimonial -> wealth, fiscal -> tax (tax_data/, the tax layer, tax_runner) - add §9.5 (the intel file: shape, generation, and the capture loop) - tighten worked examples down to placeholders - add the MIT header; keep the in-repo NOTE block (adapted to the new vocabulary) and the filename untouched so cross-links don't break * docs(llm-guides): align agent-harness guide with blueprint + fix reconcile semantics Follow the blueprint rename (patrimonial -> wealth, fiscal -> tax, fiscal_data/ -> tax_data/, "Phase 7 (fiscal layer)" -> "(the tax layer)") so the two docs stop disagreeing on vocabulary. Correct the reconciliation mapping, which conflated two different invariants: - blueprint reconcile-or-abort (§7 pass 3) is parse-integrity (parsed parts == the document's own printed total); Sure's reconciliation_checks is ledger agreement (statement balances vs the ledger). Sure has no parse-integrity check and never aborts. - opening_balance / closing_balance are user-entered, not auto-extracted, so over MCP reconciliation is "unavailable" until a human fills them. - tolerance differs: blueprint 1.00/account-period vs Sure's fixed 0.01. State in the ownership table, the invariants section, the vocabulary map and the monthly runbook that parse-integrity and the abort belong to the harness extractor. * Correct the vault tools' reconciliation claims and citation parsing Review findings from @diegomarino, all verified against the code before changing anything. The reconciliation claim was the serious one. get_account_statement told agents the checks were "the trustworthy part" and returned "the balances read off it" — but nothing reads balances off a document. MetadataDetector never touches them and create_from_prepared_upload! never sets them; they are user-editable fields in the Statement Vault UI. So a statement archived over MCP always came back with an empty check list, which an agent could easily read as "the document agrees with the ledger" when it means "nobody has entered the figures". The description now says so, and the payload carries a reconciliation_note spelling it out for anything reading only the JSON. Also noted that these checks are ledger agreement, not parse integrity: nothing here verifies a document's parts sum to its printed total. Provenance::Citation had two patterns disagreeing about spacing. GRADE_SUFFIX allowed "(grade:A)" but FORMAT required exactly one space, so that citation passed the pre-check and then parsed as ungraded with the grade swallowed into the text — silently discarding the reliability the caller supplied, which is the one thing this parser exists to prevent. list_account_statements downcases content_sha256 before querying. The column is constrained to lowercase hex, so uppercase input could never match, and an agent would read the empty result as "not archived" and upload a duplicate. Its period filters are renamed overlapping_from / overlapping_until, since they match on overlap and the old names claimed otherwise to anyone reading the schema without the descriptions. has_more now explains that there is no cursor and the way forward is a bigger limit or narrower filters. record_valuation no longer overwrites the entry's notes. Re-recording a date would destroy a note a person had written there. Nothing is removed now: an identical citation is a no-op, a changed one is appended, and the trail of what was cited when survives. Detecting "did this tool write that line?" is not possible — almost any prose parses as a valid ungraded citation — so the code does not guess. Minor: accept urlsafe base64 on upload, and explain in the code why record_valuation checks the account ACL rather than the vault manager role, so nobody "tightens" it into the wrong permission later. Tests cover each: the grade-spacing cases both ways, uppercase SHA lookup, overlap window boundaries, note preservation and no-stacking, the unavailable reconciliation note appearing and disappearing, and — per the review — that the download URL's signed id actually expires, rather than trusting the description's claim. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * Repair a bad merge in the MCP controller test The merge of main spliced the incoming `tools/call executes update_transaction` test into the middle of the upload round-trip test, before its closing `end`. That left the file one `end` short, so it did not parse — taking out both `ci / lint` (Lint/Syntax) and `ci / test_unit` (the whole file failed to load). Restores the missing `end`. Both tests are kept as their authors wrote them; nothing else changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * docs: use wealth history wording (#2885) * Stop the vault tools promising verification they don't perform Three findings from the automated review passes, all confirmed against the code before changing anything. The download URL was dead on arrival for the caller it was built for. Sure serves stored files through Active Storage controllers that config/initializers/active_storage_authorization.rb gates on `viewable_by?(Current.user)` — a signed-in browser session. An MCP client has a bearer token and no session, so following the URL would have redirected to sign-in. Removed it rather than leaving a link that cannot work, and the description now points at search_family_files or the vault UI. Coverage called a month `covered` when a document merely existed. An unreconciled statement is not mismatched, so it took the `covered` branch, and the payload carried nothing to correct the reading — the same "advertised verification that never happened" bug fixed last round in get_account_statement, in a second place. Months now carry their own reconciliation_status, and the description says covered means presence, not agreement. Listing filtered visibility after limiting. Beyond underfilling a page, with no cursor and a 100-row cap an accessible statement behind enough newer invisible ones was unreachable. Visibility now lives in the query, mirroring viewable_by? for a statement manager. Also: rescue unexpected upload failures into a tool error instead of a raw exception string, derive the documented size limit from MAX_FILE_SIZE, list every coverage status in mcp.md, and cover the failed-reconciliation and base64-normalisation branches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * Keep storage exception detail out of the MCP response The upload_failed message interpolated the exception text, which crosses out to an external agent. A storage failure can carry bucket names, object keys, paths or request details, so the agent now gets a fixed message and the exception stays in the server log. The test asserts the absence of detail rather than pinning the leaked string into the contract. Also fixes a test that did not test what it claimed: the urlsafe-base64 case used a fixture encoding to plain base64, so it exercised the padding branch and never the "-_" translation. It now uses content whose encoding contains both characters and asserts that up front. Renames "rejects content that decodes to zero bytes" to "rejects blank content", which is what it actually covers — Base64.strict_encode64("") is "", which is blank and returns before the decoder runs, so invalid_content is correct and empty_file is not reachable from this path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * docs: align wealth blueprint review feedback * Correct the harness runbook: parse before publishing The guide told an implementer to archive each document to Sure first and work from there. That strands them. Sure never returns a document's bytes over MCP — Active Storage serves stored files only to a signed-in browser session — and there is no text fallback either, because statements archived through upload_account_statement never enter the vector store, so search_family_files cannot see them. A statement in Sure is metadata to an agent and nothing more. That blocks exactly three blueprint steps, all of them operating on bank and broker statements: the extractors, the parts-vs-printed-total check, and the glyph decoder. Everything else it parses — tax returns, capital accounts, annual accounts — the harness already holds locally. So the order inverts: the harness ingests into its own vault, extracts there with the whole file in reach, and publishes to Sure afterwards. This restores principle 8 rather than bending it — the recurring pipeline reads from the canonical store, and treating Sure as canonical forced a re-fetch the architecture never sanctioned. Both sides hash the same bytes, so the SHA-256 verifies Sure holds the identical document without moving it. Writes down the two consequences: a statement uploaded straight into Sure's UI can be known but never parsed (reliability C or PENDING until a copy reaches the harness), and neither vault backs up the other. Also drops a stale tools-table row still advertising the 15-minute download URL removed earlier, corrects get_account_statement's description where it suggested search_family_files as a fallback it cannot be, and disambiguates "the vault" in the MCP tool table, which is what misled me in the first place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: diegomarino <diegomarino@users.noreply.github.com> Co-authored-by: Sure Admin (bot) <sure-admin@splashblot.com> |
||
|
Maurício Pólvora
|
93ba2b8c08 |
Fix chained assistant tool calls (#2767)
* Fix chained assistant tool calls * Preserve chained tool context |
||
|
Guillem Arias Fauste
|
23e7db4f4d |
feat(transactions): surface recently-used categories in the category picker (#2829)
* fix(transactions): don't crash the rule-prompt flash when clearing a category needs_rule_notification? only checked saved_change_to_category_id? and eligible_for_category_rule?, neither of which accounts for category_id being nil. Clearing a category (Clear category / entryable_attributes category_id: nil) satisfies both, so the caller went on to read transaction.category.name against a nil category and crashed. A rule prompt only makes sense when a category was assigned, not cleared, so bail out early when there's no category to build a rule around. * feat(transactions): surface recently-used categories in the category picker Reframes the recency-vs-muscle-memory question as additive, not either/or: a small "Recent" section pinned above the existing alphabetical list, which stays exactly where it always was below it. Precedent for reordering the primary list by frequency (Office's old adaptive menus, browser-history-style resorting) is a well-known anti-pattern — position drifts under the user's hand. Every picker that does recency well (VS Code's command palette, Spotify, Slack's emoji picker) adds a small separate recent cluster instead. - Category#last_used_at, touched only in TransactionCategoriesController#update — the one place a category is actually hand-picked by a person, as opposed to a rule or import auto-assigning one. - Category.recently_used_for(family:, excluding:, limit:) batches the family-scoped query; dropdowns_controller excludes the already- selected category from the Recent section since it's already pinned to the top of the main list. - "Recent" hides itself the moment a search query is typed — it's a pre-search shortcut, not a second copy of search results. Its rows are force-hidden (not just filtered) so keyboard nav can't land on a row that's invisible only because its ancestor section is hidden. * fix(categories): address review feedback on recent-categories picker - Track last_used_at from every manual assignment path (transaction edit form, categorization wizard bulk-update, create-and-assign), not just the category-picker endpoint. Centralized as Transaction#record_category_usage!, called explicitly from each manual controller action rather than wired to a blanket after_save callback, since rule/import auto-assignment must not count as a "recent" pick. - Give recent-section rows a distinct DOM id (recent_category_option_<id>) from their canonical-list counterpart so aria-activedescendant can't resolve to a hidden duplicate during keyboard nav. - Fix migration to ActiveRecord::Migration[7.2] to match the rest of the repo. - Materialize @recent_categories with .to_a to avoid a redundant query. |
||
|
|
5f0f5ec89d |
feat(mcp): Add MCP transaction update tool (#2719)
* Add MCP transaction update tool * Fix MCP transaction authorization * Ignore Pipelock false positive on SnapTrade token lookup Pipelock scan-diff flags `token = oauth_refresh_token...` as "Credential in URL" even though these are ActiveRecord attribute names, not embedded secrets. Add the established inline ignore. Co-authored-by: Martin Molcrette <Mart1M@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Martin Molcrette <Mart1M@users.noreply.github.com> |
||
|
Guillem Arias Fauste
|
9d3879a859 |
feat(plan): unify Budgets and Goals under a single Plan tab (#2687)
* feat(plan): unify Budgets and Goals under a single Plan tab
Preview users get one "Plan" nav entry (compass icon) in place of the
separate Budgets and preview-gated Goals items. It fronts a new /plan
hub with two summary cards — this month's budget (spent vs budgeted,
days left, top categories) and active goals (total saved vs targets,
behind/pending counts, per-goal rows) — each drilling into the existing
/budgets and /goals pages, whose breadcrumbs now start Home > Plan.
The two features share a home, not a model: no schema changes, no URL
changes. Users without preview features keep exactly the pre-Plan nav
(Budgets entry, Goals hidden), and /plan falls through to /budgets for
them.
Supporting changes:
- Goal.active_prepared_for: index-style sorted active goals with the
family-wide pooled-allocations + market-flows injection reused
- Goal::FUNDABLE_ACCOUNT_TYPES and Goal::ACTIVE_DISPLAY_STATUS_RANK
extracted from GoalsController
- Budget#days_remaining (same day math as suggested_daily_spending)
- Breadcrumbable#plan_breadcrumb_prefix for the conditional Plan crumb
- New BudgetsController web tests (previously untested) + Plans tests
* fix(plan): address review feedback on the Plan hub
- Keep the "All goals" footer link rendered when the family has only
completed/archived goals — the hub is a preview user's only route to
the goals index now that the Goals nav entry is gone (Codex P2)
- Replace the two hand-rolled footer button-links with DS::Link
(variant secondary, full_width, right icon) per DS Drift Patrol
- Fix DS::Link template comparing icon_position against the string
"right" — the initializer symbolizes it, so right-positioned icons
never rendered on links (DS::Button already compared symbols);
existing callers passing icon_position now get the layout they asked for
- Clamp progress-bar percentages to 0..100 instead of capping only the
upper bound (CodeRabbit)
* refactor(plan): single source of truth for goal loading, sorting, and counts
Addresses jjmata's draft review notes:
- GoalsController#index now builds on the shared Goal loaders instead
of hand-rolling its own copy: Goal.prepared_for (preloads + family-
wide backing-math injection, scope-able) and Goal.active_display_sort
carry the algorithm once; active_prepared_for composes them for the
hub. The controller-side constant alias is gone
- One definition of "behind pace": Goal#behind_pace? (excludes paused —
pausing stops the pace clock on purpose). Both the Plan hub summary
and GoalsController#kpi_payload's behind/needs-this-month figures use
it, so adjacent pages can't disagree. While there, the kpi on-track
numerator also excludes paused goals — it was counted against a
paused-excluding denominator, so the "X of Y" fraction could exceed
its own total
- BudgetCategory#suggested_daily_spending calls Budget#days_remaining
instead of keeping an inline copy of the day math
- Per the fat-model convention, the hub's aggregation moved off the
controller: Budget#top_spending_categories(limit:) and
Goal.summary_for(goals, currency:)
* fix(plan): move Edit budget/New goal into their own cards
Both actions lived in the hub's shared page header, unlinked to either
card and, on mobile, wrapping above all content before any real data
appeared. Each now lives in its own card's header instead: Edit budget
as a compact icon-only control next to the status pill (only when a
budget exists — the uninitialized state already has its own "Set up"
CTA), New goal as a small outline button next to the goals count (only
once there's a goal to sit beside; the empty state keeps its own CTA).
Also swaps the edit icon from "pencil" to "square-pen" — at the sizes
these header controls render, lucide's plain pencil is a thin diagonal
stroke that reads noticeably smaller than a neighboring bold glyph like
"plus", even in the same size box. square-pen carries more visual mass
and reads clearly at the same footprint.
* fix(plan): match established DS precedent for the card header actions
Edit budget was a bare icon-only button; verified against the app's
own precedent for this exact action (app/views/budgets/_budget_donut.html.erb,
the budget card already shipped on /budgets) and it's a labeled
secondary link with a trailing pencil, not icon-only and not a
three-dot menu. Matched that: DS::Link, variant secondary, size sm,
icon right. New goal gets the same treatment for consistency between
the two cards' header actions, rather than the full-page-scoped
"primary" weight goals/index.html.erb uses for its own create button —
that's calibrated for a whole page's sole CTA, not a compact card.
Adding a labeled button (wider than the bare icon this replaces)
crowded the header row on mobile enough to wrap "This month" onto two
lines and truncate the "· July 2026" meta away entirely. Header rows
now wrap as a whole (flex-wrap) with the title pinned (shrink-0) so
the action cluster drops to its own line instead of squeezing the
title and meta text.
Also drops the hub's footer note ("Budgets cap your spend; goals track
what you're saving toward...") — redundant with the subtitle right
above the cards.
* fix(plan): lead the budget card header with status, not the edit action
On Track/Over/Warning is what a glance at the card wants first; Edit
budget is the secondary action. Swapped their order so status leads
and the edit control trails, gap-2 unchanged.
* fix(plan): put the status pill on the left, next to the title
Meant the left side of the card, not just left of the edit button. On
Track/Over/Warning now sits beside "This month · July 2026" in normal
flow; ml-auto carries only the Edit budget link, alone on the right —
matching the goals card's own left-meta/right-action split ("· 7
active" left, "New goal" right).
* fix(plan): lead Edit budget with its icon, matching same-shape precedent
Wrong axis on the earlier match: _budget_donut's trailing pencil labels
the VALUE itself ("$12,850 ✎"), not a static action. Our button's label
is a static "Edit budget", and that shape takes a leading icon
everywhere else it appears — the categories "Edit" on budgets/show.html.erb
(icon: settings-2) and "Edit split" in transactions/show.html.erb both
lead with their icon. Drops icon_position: :right so it defaults to
left, matching New goal's shape in the sibling card.
* fix(plan): use the divider token for row separators, not border-primary
Traced against the dashboard outflows list (pages/dashboard/_outflows_donut.html.erb),
which renders its row separators via shared/_ruler → border-divider
(border-tertiary: black/8%, white/10%). Our category and goal rows used
border-b border-primary instead (black/15%, white/30%) — 2-3x heavier
than the established row-separator weight elsewhere in the app. Swapped
both to border-divider.
* fix(plan): lift the duplicated card shell into DS::Card
Codex P1: _budget_card.html.erb and _goals_card.html.erb hand-rolled
the identical "bg-container rounded-xl shadow-border-xs p-5 flex
flex-col" shell twice, with no DS:: card primitive to reach for
instead. Extracted a minimal wrapper — content-only, no header/footer
slots — matching what both cards actually need right now; the roadmap
cards (envelopes #2153, retirement #2044) can adopt it too instead of
copying the class string a third time.
Verified pixel-identical in a browser: same classes, same DOM shape,
just rendered through the component.
* fix(plan): batch pace queries before sorting goals
Codex P2: active_display_sort calls goal.status per goal to build the
sort key; Goal#status reaches Goal#pace for any goal with a
target_date, which fired its own Entry.sum(:amount) query per goal.
The /plan hub renders only the first 5 of active_prepared_for's list,
but paid the full O(N) query cost sorting all of them.
Adds Goal.pace_for(family) (account_id => 90-day net inflow), grouped
in one query and injected via inject_backing_math! alongside the
existing pooled_allocations/market_flows pattern. #pace now sums from
that shared map instead of firing its own query — same math, same
90-day window, same exclusions, just computed once per family instead
of once per goal.
|
||
|
Guillem Arias Fauste
|
59852ca0f3 |
fix(insights): respect recurring_transactions_disabled in subscription_audit (#2831)
* fix(insights): respect recurring_transactions_disabled in subscription_audit SubscriptionAuditGenerator queried family.recurring_transactions directly, so disabling recurring-transaction detection (Settings -> Recurring Transactions) never stopped already-identified rows from surfacing "recurring charge overdue" insights on the Insights feed — the family-wide flag was already checked at both call sites in IdentifyRecurringTransactionsJob, just not here. Returning [] early is enough for existing insights to self-clean up: produced_types is a class-level declaration, so GenerateInsightsJob still counts subscription_audit as a succeeded type and expires any insight whose dedup_key wasn't regenerated on the next nightly run. * docs(insights): document why cash_flow_warning skips the recurring-disabled guard Answers jjmata's open review question. Unlike SubscriptionAuditGenerator, recurring transactions here are one input into a broader cash-flow projection, not the insight's entire subject — so it intentionally keeps using the last-known identified set rather than gating on family.recurring_transactions_disabled?. No behavior change. |
||
|
kai392andClaude Opus 4.8
|
73aac31f89 |
fix(exports): include merchants.csv in family data export (#2758)
* fix: include merchants.csv in family data export The family export ZIP contained CSVs for accounts, transactions, trades, categories, and rules — but merchants were only present inside the all.ndjson bulk file, never as a standalone merchants.csv. Meanwhile merchants can already be imported via CSV (MerchantImport), so backups were lossy and the import/export cycle was asymmetric. Add generate_merchants_csv to Family::DataExporter, wired into the export ZIP, with headers (name,color,website_url) matching exactly what MerchantImport expects so the exported file round-trips. Fixes #2736 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: fix merchants CSV round-trip assertion for family fixtures The round-trip test asserted the target family's total merchant count grew by exactly 1, but the export legitimately includes every family merchant — including dylan_family's fixtures — so the import created 4, failing CI. Scope the count assertion to the merchant under test. Also assert the imported color now that merchant colors survive a save (the set_default_color callback only backfills when no valid color is present), giving the round-trip full name/color/website coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Carlos Lindo
|
ef61466e87 |
fix(enable-banking): preserve manual card limit when provider reports zero (#2841)
* Preserve manual card limit when provider reports zero * Test non-positive provider credit limits * Address credit limit review feedback |
||
|
Guillem Arias Fauste
|
c40e7f4807 |
fix(insights): correct the budget card's figure, badge noise and toast a11y (#2799)
* fix(insights): correct the budget card's figure, badge noise and toast a11y
Four defects found while reviewing the insights surfaces for hierarchy.
**The budget_at_risk card's focal figure argued against its own headline.**
`insight_key_figure` returned `budget_spent_pct` for both budget cards, so
"2 categories need attention in your budget" displayed "14% / of budget" —
a reassuring number as the visual focus of a warning. It now leads with the
flagged count ("2 / need attention"); budget_on_track keeps the percentage,
where overall consumption genuinely is the subject.
**The "New" pill carried no information.** Visiting /insights marks every
insight read in one `update_all`, so at first paint the pill was on every
row. On the page it becomes a dot — same signal, without an uppercase
tracked chip stealing weight from the title beside it. In the dashboard
widget it goes entirely: the well's header already counts unread ("New · 3")
and, with three rows, the pill was usually on all of them.
**The undo toast was silent to screen readers.** A card leaves the page via
a Turbo `remove`, which announces nothing, and the toast that explains it
had no live region — unlike its neighbour `_sync_toast`, which sets
`role="status" aria-live="polite"`.
**The undo toast could only be closed with a mouse.** Its close affordance
was a bare `icon "x"` with a click action: not focusable, not named. Now a
real `DS::Button`, matching `_sync_toast`.
The controller test asserting a per-row badge is updated to assert the
header count that replaces it, and to lock in the pill's removal.
* feat(insights): acknowledge instead of dismiss, on both surfaces (#2800)
Two complaints about the insight feed: the close (×) control felt wrong,
and clearing an insight was only possible on /insights — not on the
dashboard widget, which is the surface people actually look at.
**The × was lying.** Dismissal has never been permanent. GenerateInsightsJob
resurfaces a row whose bucketed metadata changes materially "even if the user
had read or dismissed the stale version" (its own comment), and 6 of 8
generators scope dedup_key to a month token, so dismissing July's budget card
says nothing about August's. A destructive-looking control was performing a
non-destructive act. It is now "Got it", and the contract is statable:
acknowledgement covers the numbers you saw; new numbers are a new insight.
No migration. The DB value stays "dismissed" and dismissed_at keeps its name;
only the enum key and the vocabulary the code speaks change, so existing rows
stay hidden and become undoable under an honest label.
**The action pyramid was inverted.** The escape hatch was a chromed icon
button in the card's top-right — the strongest secondary scan position — while
the card's actual purpose ("View budget") was a borderless ghost link under
the body text. Both now sit in a footer strip: the subject action gets the
chrome, acknowledging is quiet labelled text beside it, and the key figure
gets the corner to itself instead of competing with a control.
**The widget can clear its own rows.** Each row gains an acknowledge control,
revealed on pointer hover, on keyboard focus, and shown unconditionally on
touch where there is no hover. No gesture, so the section's drag-to-reorder
handlers are untouched. The row becomes a stretched link plus a sibling
button, because button_to renders a <form> and a form cannot nest in an <a>.
The group is named (group/insight). The dashboard <section> is itself a
`.group` for its header controls, and a bare group-hover: matches any ancestor
group — hovering one row, or the section header, revealed every row's control.
Acknowledging re-renders the well rather than removing a row, so the next
insight is promoted into the freed slot; Insight::FEED_LIMIT is now shared
between the two controllers that render it so they cannot drift. Undo restores
the row on both surfaces, and carries autofocus so it is one keystroke away
after the acknowledged card leaves the DOM.
* fix(insights): guard unacknowledge! against non-acknowledged insights
CodeRabbit, Major: an arbitrary/stale PATCH /unacknowledge (e.g. an old
undo-toast link clicked after GenerateInsightsJob has since expired or
resurrected the insight) could force it back to :read regardless of
its actual current state — including pulling an :expired insight back
into visible view.
Guards the transition to only reverse an actual acknowledgement, per
CodeRabbit's suggested fix.
* test(insights): fix stale dismiss_insight_url route from main merge
main's preview-gate test used the pre-rename dismiss/undismiss route names;
this branch renamed those to acknowledge/unacknowledge earlier.
|
||
|
Gerald
|
9313ad4cba |
fix: tolerate Trade Republic Enable Banking pagination/PDNG errors (#392) (#2828)
* fix: tolerate Enable Banking pagination/PDNG errors for Trade Republic Trade Republic (available via Enable Banking since ~2026-07-22, see #392) fails to sync with two distinct errors on its own side: 1. The BOOK transaction fetch issues a continuation_key on page 1 that its own API then rejects on page 2 as mismatched with transaction_status (422 WRONG_REQUEST_PARAMETERS: "transactionStatus in request is not the same as in continuationKey"). This previously discarded every page already fetched. Once at least one page has succeeded, a validation error is now treated as pagination exhausted and the partial result is kept instead of raising. A validation error on the very first page still propagates as a real failure. 2. The PDNG (pending) fetch is rejected with a plain 400 (:bad_request) instead of the 422 (:validation_error) other ASPSPs use for the same "transaction status not supported" case. Both error types are now treated as "ASPSP doesn't support pending transactions". Verified against a live Trade Republic connection through Enable Banking. * fix: surface Enable Banking pagination truncation as a debug log entry Add a DebugLogEntry.capture call when a mid-pagination validation error truncates the transaction fetch (e.g. the Trade Republic continuation_key bug from the previous commit). This follows the project convention of using DebugLogEntry for support-relevant sync diagnostics rather than only Rails.logger, so a truncated sync is visible in /settings/debug instead of only in container logs. Currently harmless for narrow incremental sync windows (the account observed in production has under 100 transactions per window, fitting entirely on page 1), but a wider historical resync could otherwise lose data past page 1 with no visible indication. * fix: address CodeRabbit review feedback on PR #2828 - Add a DebugLogEntry when the PDNG fetch is skipped as unsupported, matching the pattern already used for pagination truncation — this was a partial-degradation case that was previously only visible via Rails.logger. - Replace the OpenStruct#define_singleton_method provider fakes in the three new pagination tests with sequenced Mocha stubs (expects(...).twice.returns(...).then.raises(...)), per the project's "use Mocha for stubs and mocks" guideline. * fix: don't swallow WRONG_TRANSACTIONS_PERIOD as pagination truncation The mid-pagination validation-error handling treated any 422 after page one as "ASPSP rejected the continuation key" and kept the partial result as a success. WRONG_TRANSACTIONS_PERIOD is a different, real failure (an invalid date range, already retried once with a corrected date_from at the provider level) and must still propagate instead of silently dropping the remaining pages. Addresses CodeRabbit review feedback on PR #2828. * fix: address jjmata's review feedback on partial-result asymmetry - fetch_paginated_transactions now tolerates :bad_request the same way it already tolerates :validation_error mid-pagination, matching the PDNG-unsupported rescue in fetch_and_store_transactions which already accepts both error types. Without this, a :bad_request on PDNG page 2+ would discard the already-fetched PDNG page 1 instead of keeping it like the BOOK path does. Trade Republic only 400s on PDNG page 1 today, so this was latent, not currently observed. - Bump the pagination-truncation log (Rails.logger + DebugLogEntry) from warn to error: this now discards data for any ASPSP/scenario matching the tolerated error types mid-pagination, not just the specific Trade Republic case it was written for, so it deserves higher visibility. - Add a regression test for :bad_request interrupting PDNG pagination on page 2+, pinning down the now-symmetric behavior with BOOK. |
||
|
Sure Admin (bot)
|
1e800e2f93 | Include uncategorized spending in budget UI (#2877) | ||
|
Sure Admin (bot)
|
078c49b35b | Stabilize net worth breakdown series test dates (#2863) | ||
|
Maximus Barbare
|
4c1bc774e5 |
Fix SnapTrade account setup and reconnection (#2858)
* Infer SnapTrade account types from categories * Allow choosing SnapTrade account types * Reauthorize SnapTrade connections needing attention * Clear stale SnapTrade reconnect state * Always schedule SnapTrade reconnect syncs * Recognize SnapTrade credit card accounts * Recognize SnapTrade crypto account types * Schedule SnapTrade syncs after active imports * Fix SnapTrade account type CI tests * Normalize SnapTrade card account types |
||
|
Guillem Arias Fauste
|
cedf28a5a9 |
fix(accounts): make the sync toolbar and toast agree, fix toast overlap (#2813)
* fix(accounts): make the sync toolbar and toast agree, fix toast overlap The Accounts page's own sync toolbar (refresh icon, "Cancel sync") and the global sync-complete toast were three inconsistently-styled, disconnected pieces of UI representing one action, and the toast overlapped the page's own header instead of sitting near it. - "Cancel sync" was hand-rolled markup instead of a DS::Button, unlike its sibling refresh icon right next to it — now both are DS::Button (:ghost). - The refresh icon just went `disabled` with no visible "working" state — now shows a spinning loader-circle while a family sync is in progress, matching the pattern already used in provider_sync_summary.html.erb. - The toolbar was plain server-rendered HTML with no way to know a sync finished, so it stayed stuck showing "still syncing" indefinitely next to a toast now saying otherwise. Family::SyncCompleteEvent now broadcasts a second replace target for the toolbar alongside the existing toast replace, so both resolve together. - The notification tray was a <body>-level fixed overlay centered on the full viewport, but every layout that renders it has a sidebar of some kind — so it never actually centered on the visible content pane, and landed on top of the settings-layout header. It now renders in-flow at the top of each layout's own content region (opt-in via notification_tray_inline, since the simpler single-column layouts don't have this mismatch and are unaffected). - Added the Catalan sync_toast/cancel_sync strings that were missing entirely, which is why the toast/toolbar showed English text on an otherwise-Catalan page. Verified live in a real browser via a new system test covering the idle, syncing, and cancel-flash states, plus a model test on the new broadcast target. * fix(accounts): keep the tray a floating overlay, sidebar-aware instead Codex on this PR: with the tray as first-child-of-scrollable-main, a notification delivered while scrolled down is inserted above the viewport and stays unseen — breaking the sync toast's manual-refresh path specifically, since sync_toast_controller.js suppresses auto-refresh while a form is focused and relies on the toast being visible to offer that manual refresh. Reverts the tray to a position: fixed overlay for every layout (so it can't be scrolled out of view), and fixes the actual bug that made it overlap the accounts toolbar in the first place — a ResizeObserver on <main> centers it on the real content pane instead of the viewport, for the two layouts with a sidebar (application, settings passed via sidebar_aware:). The five single-column layouts are untouched; for those, viewport-center already is content-pane-center. One trap worth flagging: this app renders turbo_refreshes_with method: :morph, and idiomorph resets any inline style a client script set that isn't in the freshly-fetched HTML — including the JS-set `left`. data-turbo-permanent looked like the fix but isn't: it invokes idiomorph's node-identity matching (same id preserved across ANY morphed page), which broke navigation once the id existed on structurally different layouts (app vs settings) — a real, reproduced bug, caught by the system test before it shipped. Went with the narrower turbo:before-morph-attribute event instead, which blocks only the `style` attribute on this one element, with no node-identity system involved. Rewrote the system test's positioning assertion to match: it now asserts the tray centers on <main> rather than sitting above the page header, since a fixed overlay was never going to satisfy the latter by construction. Verified: full bin/rails test (6023 runs, 0 failures), rubocop, erb_lint, brakeman (0 warnings) all clean. Live-verified in a browser across both sidebar-aware layouts and a simple layout, including the full cancel-sync -> morph -> re-render cycle. * fix(accounts): use declarative Stimulus action for morph-attribute guard Replace the manual addEventListener/removeEventListener pair for turbo:before-morph-attribute with a data-action, per the repo's declarative-actions convention. Same element, same listener — just no manual lifecycle management. |