* feat(insights): gate the insights feed behind preview features
Insights shipped to everyone in #2550. Make it opt-in via Settings →
Preferences until it's proven, so users who haven't enabled preview
features see nothing and cost nothing.
Entry points gated:
- InsightsController — require_preview_features! covers all four actions,
including the refresh action that enqueues the job
- Dashboard — the insights_feed section is omitted from the section list
rather than left in it hidden, so the saved-order lookup and the
insights_feed unshift special-case never fire; the feed query is skipped
- Top bar — the lightbulb entry and its unread COUNT, which previously ran
on every page render
The job is gated too, departing from the guide's default that background
jobs keep running. That default fits a job like SweepExpiredGoalPledgesJob,
which only walks records opted-in users created and is naturally inert.
GenerateInsightsJob instead manufactures data for every family nightly —
seven generators over the income statement and balance sheet, plus paid LLM
narration — so it would have kept spending on families who can't see the
result. The fan-out filters with Family.with_preview_features (one indexed
jsonb containment query, not load-and-iterate), and generate_for_family
re-checks above the advisory lock so a gated family skips the broadcast too.
Adds Family#preview_features_enabled? and the matching scopes, keeping the
predicate name identical on User and Family so the guide's GA-removal grep
finds every call site. Verified the SQL scope and the Ruby predicate agree
for true / false / "yes" / nil. Documents the job-gating pattern in
docs/llm-guides/gating-a-preview-feature.md, which previously said the gate
does nothing for jobs.
Existing insight rows are left alone: invisible without the flag, and the
next nightly run refreshes facts and expires anything stale if a family
opts in.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4
* perf(insights): use EXISTS for the family preview rollup
Family#preview_features_enabled? is asked once per family by the nightly
job; the block form loaded and instantiated every member to answer a
boolean. Delegate to the scope instead.
The predicate now shares an implementation with the scope, so the
truthy-non-boolean test asserts against User#preview_features_enabled? —
the predicate the UI actually gates on — to keep the cross-check
meaningful rather than tautological.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4
* docs(insights): fix guide/code drift and stale cron description
Review follow-ups from @gariasf:
- The guide's family-rollup snippet still showed the block form after the
EXISTS commit changed it. It mattered more than normal doc drift: the
paragraph below calls GenerateInsightsJob "the reference implementation",
so the next person writing a gated job would have copied the form
family.rb's comment explicitly rejects.
- schedule.yml still described the job as running for "all families" — the
string someone reads while debugging why a family got no insights.
- Document that the shared predicate name is per-user on User but
"anyone in the household" on Family, and prohibit gating UI on the family
form: Current.family.preview_features_enabled? reads naturally and would
show the feature to a user who explicitly opted out. Noted in both the
model and the guide.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Guillem Arias Fauste <accounts@gariasf.com>
* fix: prevent Anthropic chat crash on no-argument tool calls
When the model streams a tool_use block with no arguments (e.g.
get_categories), the accumulated input arrives as an empty string.
The Anthropic chat parser passed that empty string straight through as
function_args, and Assistant::FunctionToolCaller then called
JSON.parse("") — which raises "unexpected end of input", surfaces as a
Provider::Anthropic::Error, and kills the whole assistant turn.
Fix at the source by normalizing empty/nil tool input to an empty JSON
object in the parser, plus a defensive guard in FunctionToolCaller so
any provider that emits blank arguments cannot crash a turn.
Fixes#2722
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: assert nil-args result as Hash to match jsonb function_result
Addresses Codex P1 / CodeRabbit review: EchoFunction returns the parsed
params Hash and ToolCall::Function stores it in a jsonb column, so the
nil-arguments test must assert the Hash directly instead of JSON.parse.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SimpleFIN protocol only defines pending=1 (include pending); pending
transactions are excluded by default when the param is absent. Bridges
presence-check the param, so the pending=0 we sent when the 'Include
pending transactions' setting (or SIMPLEFIN_INCLUDE_PENDING=0) was
disabled behaved exactly like pending=1, making the setting a no-op —
pending transactions kept being downloaded, causing pending/posted
duplicates and churn.
Omit the pending query param entirely unless pending is enabled, per
the spec.
Fixes#2440
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fixes#2793.
## Problem
IndexaCapitalAccount::ActivitiesProcessor#normalize_cash_amount used the inverted amount-sign convention relative to Sure's core conventions.
- Outflows (WITHDRAWAL, TRANSFER_OUT, FEE, TAX) were stored as negative amounts.
- Inflows (CONTRIBUTION, TRANSFER_IN, DIVIDEND, DIV, INTEREST) were stored as positive amounts.
Sure requires asset account inflows to be stored as negative amounts (-amount.abs) and outflows as positive amounts (amount.abs).
## Fix
Flip signs in IndexaCapitalAccount::ActivitiesProcessor#normalize_cash_amount to align with Sure's sign convention and sibling processors (SnaptradeAccount::ActivitiesProcessor).
## Test
Added unit tests in test/models/indexa_capital_account/activities_processor_test.rb covering cash inflows, outflows, transfers, fee, label mappings, and empty payloads. All tests pass (39/39 for indexa_capital_account).
Co-authored-by: erkdgn <erkdgn@users.noreply.github.com>
* update transactions_controller
* fix FEEDBACK - Remove DISTINCT that breaks tag-filtered index queries
* fix FEEBACK from jjmata
* Ignore Brakeman EOLRails warning for Rails 7.2
Restore fingerprint-scoped ignore lost during merge from main.
* update transactions_controller
* fix FEEDBACK - Remove DISTINCT that breaks tag-filtered index queries
* fix FEEBACK from jjmata
* Ignore Brakeman EOLRails warning for Rails 7.2
Restore fingerprint-scoped ignore lost during merge from main.
* resolve review - Add .distinct to the tag filtering subquery
* fix(ci): skip scheduled preview cleanup on forks
Only run the hourly Cloudflare preview cleanup on we-promise/sure,
where the required secrets exist.
* Drop obsolete Rails EOL note and Brakeman EOLRails ignore
The EOLRails ignore and the accompanying migration-rule note were only
needed while the app ran Rails 7.2.3.1, whose support window closed
2026-08-09. Main has since moved to Rails 8.1.3, so the check no longer
warns and both changes are dead weight that only widen this PR's diff.
Keeps the PR focused on the transactions controller query optimization.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(merchants): preserve user-selected merchant colors on save (#2760)
Fixes#2760.
## Problem
FamilyMerchant#set_default_color callback ran on before_validation and unconditionally executed self.color = COLORS.sample.
As a result:
- Any user-selected color or color passed via API/import was discarded and replaced with a random palette color.
- Renaming or updating any merchant field re-assigned a new random color on every save.
## Fix
Guard set_default_color so it only assigns a random palette color when color is blank (self.color = COLORS.sample if color.blank?).
## Test
- Added test/models/family_merchant_test.rb testing color preservation on creation and update, as well as default color fallback when blank.
- Updated test/controllers/family_merchants_controller_test.rb to assert persisted color on create and update.
- Verified in dev container: 9 runs, 22 assertions, 0 failures, 0 errors.
- RuboCop: 0 offenses.
* fix(merchants): validate hex format and fallback to default color if invalid
Add hex format validation (/\A#[0-9A-Fa-f]{6}\z/) to FamilyMerchant#color (matching Category and Tag models) and ensure set_default_color replaces invalid hex values (e.g. from API/CSV imports) with a default palette color before saving.
Responds to Codex review feedback on PR #2802.
---------
Co-authored-by: erkdgn <erkdgn@users.noreply.github.com>
* feat(accounts): add Gains / ROI chart view for investment accounts
Adds a fourth chart view to the account details page showing the
historical unrealized gains series (market value - cost basis per
holding, summed daily with LOCF and FX conversion), following the
same ChartSeriesBuilder/Series pipeline as the existing views.
Holdings without a usable cost basis (nil, or unlocked zero from
providers) contribute zero gain, matching Holding#avg_cost semantics.
* fix(accounts): carry cost basis forward over gap-filled holdings in gains series
Gap-filled holding rows (weekends, price-history gaps) are persisted
without cost_basis even though the position and basis are unchanged,
which zeroed the gains series on those dates. Look up the basis from
the latest snapshot that has a usable one instead of reading it from
the current row, so already-persisted gap-filled rows are handled too.
* docs(accounts): add method docstrings for gains chart view code
Satisfies the pre-merge docstring coverage check on methods added or
touched by the Gains / ROI feature.
* fix(accounts): sign converted amount like main indicator in gains view
Extracts the gains sign-prefix logic into a shared signed_format helper
so the family-currency converted amount shown on foreign-currency
accounts matches the main indicator (+€79.53 / +$85.00), and adds
component tests covering the positive, negative, non-gains and
foreign-currency formatting paths.
* test(accounts): cover FX conversion path in gains series
The gains_series tests only used USD holdings against a USD target,
leaving the exchange_rates LATERAL join untested. Adds a case with EUR
holdings converted to USD, including LOCF rate carry-forward.
---------
Co-authored-by: Antoine GUYON <agy@ibanfirst.com>
* fix(akahu): pull full history on initial Akahu sync
Akahu-linked accounts only pulled ~90 days of history on their first
sync. Provider::Akahu#fetch_all already walks the full range via
Akahu's cursor pagination, so the paging logic was not the limit.
AkahuItem::Importer#determine_sync_start_date clamped the initial
window to 90.days.ago when an account had no stored transactions and
no configured sync_start_date, and Akahu's transactions endpoint only
returns data from the requested start onward, so that fallback capped
the first import at 90 days.
Request a 5.years lookback on the first sync instead. Incremental
syncs still continue from last_synced_at - 7.days, and an explicitly
configured sync_start_date still takes precedence.
Fixes#2609
* fix(akahu): omit start date on initial sync to pull full history
determine_sync_start_date still clamped the first import to INITIAL_SYNC_LOOKBACK.ago
(5 years), truncating Akahu apps/accounts that can access more history. Akahu's
account-transactions endpoint defaults to the entire accessible range when start/end
are omitted, so the no-config/no-stored-transactions case now returns nil (no start
date). Subsequent syncs still use the incremental last_synced_at - 7.days path.
Removes the now-unused INITIAL_SYNC_LOOKBACK constant and updates the test to assert
the initial sync omits the start date.
---------
Co-authored-by: agentloop <agentloop@localhost>
Co-authored-by: pro3958 <pro3958@users.noreply.github.com>
* fix(dashboard): add hover state to section header controls
The collapse chevron, widget-size settings, drag handle and cashflow
expand button signalled hover with a colour shift alone
(`text-secondary` → `text-primary`). On 14–16px glyphs that reads as
almost nothing, so there was no feedback for which of two adjacent
controls was about to be clicked.
Adopt the icon-control recipe already used for the top bar in
`layouts/application.html.erb`: add `hover:bg-container-inset-hover`
plus a radius alongside the existing colour shift. The token resolves to
gray-100 / gray-700, so light and dark are both covered and no design
system change is needed.
Replaces the ad-hoc `p-0.5` / `w-5 h-5` sizing with a uniform 24px
centred box, so the four controls present the same hover target and the
same tint area. 24px matches the header's `text-base` line-height, so
the row height is unchanged.
The drag handle moves from `hidden lg:block` to `hidden lg:flex` rather
than gaining a bare `flex`, which would have overridden `hidden` at the
mobile breakpoint depending on utility order.
* fix(dashboard): drop hover fill from the drag handle
The grip is a grab surface, not a button — clicking it does nothing. A
filled hover state reads as "clickable" and promises a click that never
lands, so it keeps cursor-grab plus the colour shift and skips the
background the genuinely clickable controls use.
Stays on the same 24px box so the icons remain aligned with the
widget-size control beside it.
SnapTrade delivers payroll-deducted 401k contributions as type "TRANSFER"
rather than "CONTRIBUTION", which normalize_cash_amount did not handle.
The value fell through to the pass-through branch and was stored positive,
violating the convention that inflows to an asset account are negative.
That stored sign drove both reported symptoms. Entry#classification reads a
positive amount as an expense, so the row rendered -$1,320.75 while the
brokerage showed +$1,320.75. Balance::ReverseCalculator reads it as a value
decrease, so walking backward from the provider-anchored current balance
produced a steadily declining history despite a correct present-day total.
TRANSFER_IN and TRANSFER_OUT encode direction in the type and can force the
sign with .abs. A bare TRANSFER does not, so the provider's sign is the only
directional signal available and is inverted into Sure's convention instead
of being passed through. Scoped to TRANSFER specifically: CASH_TYPES is
unreferenced, so every non-trade type reaches this method and broadening the
else branch would silently flip SPLIT, MERGER, JOURNAL and others.
Fixes#2756
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Add monthly net worth chart with group breakdown to Reports
Adds a net worth trend chart to the Reports > Net Worth section,
rendered in the same design scheme as the dashboard chart. The chart
shows one data point per month across the period selected at the top
of the Reports page, and its hover tooltip breaks the hovered month
down into per-account-group balances (Cash, Investments, Credit
Cards, Loans, etc.) under Assets and Liabilities headings with
section totals.
- BalanceSheet::NetWorthBreakdownSeriesBuilder builds the monthly
series by running Balance::ChartSeriesBuilder per account group
(grouped by accountable type), with liabilities reported as
positive magnitudes and all-zero groups omitted; cached with the
same invalidation pattern as the existing net worth series
- net_worth_chart Stimulus controller extends the existing
time_series_chart controller, overriding only data normalization
and the tooltip template
- Reports controller passes the series through the existing
net_worth_metrics hash; tooltip headings reuse existing locale keys
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address review: month-over-month tooltip deltas, Stimulus value labels
- Recompute each chart point's trend from the previous monthly point
instead of inheriting the raw series trend, which at a monthly
interval compared the underlying balance row's own start/end and so
reflected only the last balance update before the sample date
(chatgpt-codex-connector). The first point has no prior month and
renders the standard flat state.
- Pass tooltip section labels to the Stimulus controller as declared
values (data-*-value attributes) per coding guidelines (coderabbit)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(jobs): reap imports and exports stuck by lost background jobs
A hard-killed worker (OOM, SIGKILL during deploy) loses its in-flight
Sidekiq job permanently. Sync is the only model with a stale sweep;
everything else wedges in a non-terminal status forever:
- Import stuck "importing"/"reverting" (e.g. #2274)
- ImportSession stuck "importing" — unrecoverable, publish_later
refuses to re-publish while importing
- FamilyExport stuck "pending"/"processing" — exports index polls
every 3s indefinitely
- PdfImport's AI-processing claim held forever (ProcessPdfJob's
reclaim only runs if the job is redelivered)
- Provider activities_fetch_pending flags stranded when a
self-rescheduling fetch chain loses a link
SyncCleanerJob (existing hourly cron) now sweeps all of them, each
isolated so one failure doesn't block the rest, and records every
reaped record as a DebugLogEntry with the family attached.
Idempotency guards so a stray or redelivered job cannot corrupt a
record the reaper (or a user retry) has since moved on:
- Import#publish skips complete/reverting/revert_failed imports —
a redelivered ImportJob after completion double-applies data for
import types without row dedup. Failed imports deliberately stay
publishable: their transaction rolled back, so a re-run is a retry.
- FamilyDataExportJob refuses terminal exports but still allows
processing ones through, since graceful-shutdown redelivery is what
completes them.
Reaper thresholds (6h imports, 2h exports) key off updated_at and
dwarf legitimate runtimes, so live jobs are not swept in practice.
* fix(jobs): make the reapers race-safe and commit-aware
Review feedback on #2681 (jjmata, Codex, CodeRabbit):
- Every sweep now mutates under record.with_lock with a staleness
re-check, mirroring the guard Sync#perform gained in #2680 — a job
finishing between the sweep query and the write can no longer be
clobbered mid-flight
- Import.clean distinguishes which side of import!'s single transaction
the worker died on: rows attached means the data committed and only
the status write was lost, so the record is finalized as complete
(marking it failed invited a re-publish that double-imports types
without row dedup, e.g. TradeImport); no rows means a clean rollback
and the failed/try-again path stays
- PdfImport.clean applies the same split: no rows → the AI-extraction
claim died, reclaim to pending; rows → the publish died post-commit,
finalize complete instead of letting the same extracted rows be
published twice. Stuck PdfImport reverts (previously unswept by
either clean) now go to revert_failed like every other import
- ImportSession.clean reconciles chunks whose import! committed but
never got the complete-status write before failing the session, so
re-publish skips them instead of duplicating their rows via
SureImport's split path
- Import#publish redelivery skip is captured via DebugLogEntry instead
of Rails.logger so support can see it in /settings/debug
- The interrupted-error copy is i18n-backed (imports.errors.interrupted)
* fix(jobs): round-2 review feedback on the reapers
- Import.clean excludes session-owned chunks (import_session_id: nil) —
SyncCleanerJob runs it before ImportSession.clean, so it could
finalize or fail a SureImport chunk outside the session flow that
owns its lifecycle. Regression test added (CodeRabbit major)
- family.sync_later moved outside the row-lock transaction in both
reap paths — Rails doesn't defer enqueues to after-commit by default,
so Sidekiq could pick the job up before the status write was visible
- Per-record rescue in Import.clean / PdfImport.clean so one bad record
doesn't abort the rest of the hourly sweep
- ImportSession.clean uses the importing? enum predicate;
reconcile_committed_chunks! iterates with each (default-ordered
association made find_each warn, and chunk counts are tiny)
* fix(jobs): address round-3 reaper review (isolation + commit signal)
Per-record rescue isolation for the three sweeps that lacked it —
FamilyExport.clean, ImportSession.clean, and SyncCleanerJob's
activity-flag loop — mirroring Import.clean/PdfImport.clean. One bad
record (validation error, DB blip) no longer aborts the rest of that
sweep for the hour; the activity-flag guard is per-record so a failure
also stops skipping the models that follow.
data_committed? now covers Category/Rule/Merchant imports, whose
records hang off the family rather than the import (no entries or
accounts), via the new Import#committed_by_named_records? helper. A
committed one is reaped to complete instead of a retryable failed;
nameless RuleImport rows carry no stable key, so a nameless-only file
has no commit signal and stays retryable.
* fix(schema): drop duplicate enable_banking_accounts columns from merge
The merge of main into this branch re-appended product, credit_limit,
and identification_hashes after updated_at, so schema.rb declared each
twice and db:schema:load raised "you can't define an already defined
column 'product'". Removed the duplicate declarations (kept the new
treat_balance_as_available_credit column); the test database loads
again.
* Add "Money In / Out" dashboard widget
Adds a new dashboard section showing a monthly bar chart of cash
activity alongside a summary card (net balance, income, expenses),
with per-widget month navigation and account filtering.
- IncomeStatement#totals_for computes income/expense totals for an
arbitrary period, optionally scoped to a set of account ids
- New bar_chart_controller.js (D3) renders the monthly bars
- Income/expense rows link through to filtered transactions
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Fix tooltip/dropdown positioning and split money flow chart by income/expense
The widget's @container wrapper established a new containing block for
position:fixed descendants, so the month-picker and account-filter
menus (floating-ui, strategy: fixed) and the D3 tooltip (container-
relative coordinates) rendered away from their trigger/bar. Drop
@container in favor of regular viewport breakpoints for this
full-width widget, and position the tooltip with page-relative
coordinates like the other chart controllers.
Also replace the single combined bar per month with a grouped
expense/income pair (red/green, with a legend) so each month's
inflow and outflow are visible independently.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Cap and scroll the money flow widget's month picker
Add an optional max_height to DS::Menu (opt-in, backward compatible)
so a long item list scrolls inside a fixed-height panel instead of
overflowing the viewport. Use it for the money flow widget's 12-month
picker.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Use design system tokens for income/expense indicators in money flow widget
Swap the four bg-green-500/bg-red-500 dot indicators (legend + income/expense row links) in the money flow widget for bg-success/bg-destructive, matching the "functional tokens only" rule. The same partial already uses
text-success/text-destructive for the balance figure, and bg-success/ bg-destructive are already established elsewhere (DS::Alert, budget_categories/_budget_category.html.erb).
* Fix future-month 500 and pending-transaction link mismatch in money flow widget
Two CI review findings on the money flow widget:
- A future month passed via ?money_flow_month= (e.g. a bookmarked/hand-edited URL) made end_date earlier than month_start once capped at Date.current, which Period.custom rejects, causing a 500. money_flow_month_param now clamps future months to the current month, same as it already does for malformed input.
- The income/expense row links passed type/date/account filters but no status, so Transaction::Search included pending transactions even though the displayed totals (IncomeStatement#totals_for) exclude them via excluding_pending. Add status: ["confirmed"] to both links so the linked list matches the card total.
Also strengthens the widget's controller tests: asserts the highlighted bar's actual income/expense values instead of just its presence, and adds a regression test proving an account id outside the current user's accessible accounts is dropped (falls back to the unfiltered state) rather than leaking or erroring.
* Fix account filter eligibility and SVG dark mode fill in money flow widget
Two more CI review findings on the money flow widget:
The account filter iterated over all visible/accessible accounts, a broader set than IncomeStatement actually counts (accounts excluded from reports, tax-advantaged accounts like 401k/IRA, or shared accounts not included in the user's finances). Selecting one of these silently computed to zero while its drill-down link could still list its transactions. Add IncomeStatement#eligible_accounts, mirroring the same criteria already applied in the totals SQL, and use it for both the checkbox list and the account_ids intersection.
The D3 axis tick text elements used text-primary/text-secondary, which set CSS color, not SVG fill, so labels rendered with the default black fill and were unreadable in dark mode. Add fill-current, matching the pattern already used in sankey_chart_controller.js.
Adds controller/model tests for eligible_accounts (excluded from the account filter, ignored when passed as a filter id, excluded from totals) and verifies the dark-mode fill fix visually.
* Extract duplicated bar-chart JSON parsing into a test helper
The css_select("[data-controller='bar-chart']").first + JSON.parse(chart["data-bar-chart-data-value"]) pattern was repeated across four money flow widget tests in pages_controller_test.rb. Extract it into a private money_flow_bars helper and use it everywhere instead.
* Preserve eligible account scope in money flow drill-down links when unfiltered
The income/expense drill-down links used money_flow_data[:account_ids] directly, which is nil in the widget's default unfiltered state, so .compact dropped the account filter entirely from the link. TransactionsController treats an absent account_ids as all accessible accounts, a broader set than IncomeStatement#eligible_accounts (which excludes tax-advantaged, excluded-from-reports, and non-finance shared accounts). Users with any such account could click a displayed total and see transactions that were never counted in it.
Use selected_account_ids (already computed as money_flow_data[:account_ids] || accounts.map(&:id)) for both links instead, so they always pin to the same eligible accounts backing the total, filtered or not. Left the month-picker link on money_flow_data[:account_ids] so navigating months while unfiltered doesn't bloat the URL with every eligible account id.
Adds a regression test confirming the default (unfiltered) links include account_ids and exclude an ineligible account's id.
* Refine Money In/Out dashboard widget
- 6-month window (was 3), half-width default with responsive stack, income-first
bars, 2px floor + faded in-progress (partial) month
- Expense series/figures neutral (gray/text-primary) — app reserves red for
negative/overspend; neutral zero balance
- Account filter: outline DS::Button + list-filter icon trigger with in-panel
search (DS::SearchInput + list-filter), matching the app's filter convention
- i18n search_accounts (en + fr); bump money_flow bar-count test 3 -> 6
* Clarify Money In/Out scope: month label, 6-month caption, filter "All"
Addresses confusion between the widget's own month picker and the dashboard's
global period, and between the picked month and the 6-month chart span.
- Card now headed with the selected month (e.g. "July 2026") so it's clear the
totals below are that month's, driven by the picker
- Chart legend row captioned "Last N months" so the trailing window is explicit
- Info tooltip by the month picker: the widget scopes to the month you pick
here, independent of the dashboard's top-level period
- Account filter trigger reads "All accounts" when nothing is filtered out,
instead of "Filter accounts (N)"
- i18n (en + fr) for the new strings
* Match tooltip expense dot to the gray bar palette
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Guillem Arias <accounts@gariasf.com>
* Add support for trading212 integration in investments sync
* Respect Trading 212 history rate limits
* Replace hex colors with design tokens
* Changed withdrawal to withdraw
* Address PR review feedback for Trading 212 integration
- Replace all Rails.logger calls with DebugLogEntry.capture across
syncer, importer, processor, provider, controller, and unlinking
- Fix destroy action to surface unlink errors via DebugLogEntry
- Replace hardcoded 'GBP' currency fallback with Current.family.currency
- Replace hardcoded English status strings in Syncer with i18n
- Add config/initializers/trading212.rb with DEBUG_RAW ENV toggle
- Add comprehensive test coverage: item, account, data_helpers,
holdings_processor, activities_processor, importer, syncer,
and controller tests
- Rewrite syncer and controller tests to match actual interfaces
* Fix tests and bugs found during local test execution
- Fix standard_ticker: empty string caused nil.upcase via [].first
- Fix parse_date: DateTime < Date, so when Date caught DateTime first
- Fix ActivitiesProcessor/HoldingsProcessor: missing instruments_map
in DataHelpers caused NameError when processing dividends
- Fix test: security fixture ticker collision with test data
- Fix test: sync_status_summary i18n matching in assertions
- Fix test: trading212_provider ConfigurationError test path
- Fix test: controller invalid params needs Turbo-Frame header
- Fix test: syncer test uses stubs instead of strict expects
116 tests, 257 assertions, 0 failures, 0 errors
* Keep raw provider response bodies out of exception messages.
* Fix Secrets leak into page HTML, prevents the API key/secret from appearing in the HTML source while keeping the "leave blank to keep existing" UX.
* Address review findings: env gate, sync test, uniqueness test, destroy flow
- Gate TRADING212_DEBUG_RAW behind Rails.env.local? so staging/production
cannot accidentally enable raw payload logging
- Assert SyncJob enqueue in sync controller test, not just redirect
- Fix cross-item uniqueness test to actually use two different items
- Remove rescue in destroy so unlink failures stop the flow (matching
Brex/Akahu pattern) instead of proceeding to destroy_later silently
* Add Trading212 tables to db/schema.rb for CI test database
CI runs db:test:prepare which loads db/schema.rb, not migrations.
Without these table definitions, fixture loading fails with
PG::UndefinedTable: relation "trading212_accounts" does not exist.
* Removed lint complaint
* Register Trading212 in ProviderConnectionStatus::PROVIDERS
Fixes CI failure: test_provider_registry_covers_syncable_family_provider_item_associations
* Fixed bad merge
---------
Signed-off-by: jdcdp <47483528+jdcdp@users.noreply.github.com>
Co-authored-by: jdcdp <jdcdp@cdm4.net>
* Add RentCast and Realie AVM providers for property valuation
Adds Automated Valuation Model (AVM) provider support so self-hosted
users can create property accounts from a US address lookup instead of
entering details manually:
- Provider::Rentcast and Provider::Realie clients, each fetching the
property record (type, year built, square footage) and value estimate
in a single API request, registered under a new :property_valuations
registry concept
- API key fields (encrypted, ENV-overridable) with monthly usage display
in Settings > Self-Hosting under "Property Valuation Providers"
- New property flow: when a provider key is configured, the method
selector offers "Add via RentCast/Realie" alongside manual entry;
the lookup form reuses the manual flow's localized address fields
and creates the account active with fetched attributes, valuation as
balance, and the address saved
- SyncPropertyValuationsJob refreshes linked property valuations once a
day (never hourly) via config/schedule.yml; each provider enforces its
monthly request cap (RentCast 50, Realie 25) with a calendar-month
counter shared between creation lookups and refreshes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Allow ENV override of AVM provider monthly request caps
The RentCast (50/month) and Realie (25/month) budgets default to the
free tier limits but can now be raised for paid plans via
RENTCAST_MAX_REQUESTS_PER_MONTH / REALIE_MAX_REQUESTS_PER_MONTH,
mirroring the AlphaVantage and Tiingo request limit overrides. The
settings descriptions and usage display reflect the effective cap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address review feedback: durable request counters, safer refresh job
- Move monthly AVM request counters from Rails.cache to a new
provider_request_counts table with atomic upsert increments, so the
hard budget caps survive cache eviction, restarts, and Redis flushes
- SyncPropertyValuationsJob: only mark a property synced when the
balance update succeeds (wrapped in a transaction), process stalest
valuations first so a tight budget goes where it matters, and report
failures via DebugLogEntry.capture instead of Rails.logger
- Realie: reject lookups whose returned city/ZIP contradict the entered
address (street+state queries can match the wrong city); document that
numeric use codes are unpublished and leave the subtype unset
- Add Faraday open/request timeouts to both provider clients
- Localize all user-facing provider error messages
(config/locales/models/provider/en.yml)
- Add a check constraint restricting properties.avm_provider to known
providers
- Use the min-h-80 scale token instead of min-h-[320px]
- Tests: exact-argument expectations on the lookup stub, provider stubs
in the no-provider test, RentCast/Realie API key update tests,
ProviderRequestCount unit tests, failed-balance regression test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Stub AVM provider registry lookups in settings system test
Settings::HostingsController#show now resolves the RentCast and Realie
providers for usage display; the system test's partial get_provider
stubbing treats the new lookups as unexpected invocations without these.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address second-round review: candidate scan, provider reuse, shared row partial
- Realie: when the address lookup returns multiple candidates, pick the
first one consistent with the entered city/ZIP instead of judging only
the first array element; the location-mismatch error now only fires
when no candidate matches
- SyncPropertyValuationsJob: resolve one provider instance per key for
the whole run, so the per-instance request throttle actually spaces
requests across properties instead of resetting on each iteration
- Extract the AVM method selector row into a shared partial so the
manual and provider options render one shape
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add retry middleware and sync index from review feedback
- Both AVM clients now retry transient connection failures (same
Faraday retry config as Tiingo) so a network blip doesn't burn one of
the tight monthly budget's requests
- Partial index on properties(avm_provider, avm_last_synced_on) backing
the daily sync job's filter and stalest-first ordering
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Key the AVM sync index on avm_last_synced_on
The daily job orders by avm_last_synced_on ASC NULLS FIRST across all
AVM-linked properties, so leading the partial index with avm_provider
prevented it from serving the sort. Rekeyed on avm_last_synced_on with
matching null ordering; the partial predicate still covers the filter.
Amended in place since the migration is unmerged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Map Realie numeric use codes; skip incomplete addresses in sync job
- Realie documents its numeric use codes at
docs.realie.ai/api-reference/feature-key (earlier skip reason was
wrong — the table exists). Map the residential/land/agricultural codes
to Property subtypes, leaving codes without a subtype equivalent
(e.g. 1006 mobile/manufactured) unset like the RentCast mapping.
- The daily job now skips properties whose address is missing a street
or state before resolving a provider, logging a warn-level
DebugLogEntry, so a malformed address can't burn a monthly-budget
request every day.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Guard currency mismatches in refresh; reset AVM keys in test teardown
- The daily job now skips (with a warn-level DebugLogEntry) properties
whose account currency no longer matches the provider's valuation
currency, checked via the concept's valuation_currency before spending
a request — writing a USD valuation into a re-currencied account would
corrupt the balance
- Hostings controller test teardown now clears rentcast_api_key and
realie_api_key alongside the other cached global settings
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Validate AVM lookup inputs before spending a provider request
The form marks name and address fields required, but a forged or
JS-less submission bypasses that and would burn one of the tight
monthly-budget requests on a lookup that can't produce a property.
Property::AvmImport now validates name and the full address locally
(localized error) before calling the provider.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add preview-and-confirm step to the AVM property lookup
RentCast's AVM endpoint is location-based: a plausible but nonexistent
address still geocodes and returns an area-derived estimate with no
property record behind it. Instead of silently creating an account from
that, the lookup now shows what the provider returned — type, year
built, area (each "Unknown" when missing), and the estimated value —
with an explicit notice when no property record was found, and only the
user's confirmation creates the account.
The confirm step reuses the fetched data via the form (sanitized
server-side), so the flow still costs exactly one provider request.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Require a complete US address before daily valuation refresh
The job's guard only required street and state, but a later-blanked
city or ZIP silently disables the Realie wrong-city check (blank
entered fields count as "no mismatch"), so a refresh could accept
another property's valuation. The guard now mirrors the import-time
completeness check and also skips addresses edited to a non-US country.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Sign the AVM preview payload; blur valuation in privacy mode
- The confirm step now rebuilds everything from a signed, 1-hour
message-verifier token generated at lookup time, proving the lookup
(and its counted provider request) actually ran — a direct confirm
POST with fabricated data can no longer create provider-linked
properties that the daily refresh job would then spend quota on.
Forged/expired tokens re-render the lookup form with a clear error.
- The preview's market value now carries the privacy-sensitive class so
privacy mode blurs it like other account amounts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Treat zero Realie valuations as absent; bind preview token to family
Review follow-ups from PR #2727:
- Realie returns modelValue: 0 when it can't produce an AVM estimate;
zero now falls back to the assessed market value instead of syncing
a $0 balance, and an all-zero record errors out.
- Monetary values now parse via BigDecimal(value.to_s) rather than
Float#to_d, avoiding binary float rounding artifacts in balances.
- The signed AVM preview token's purpose is scoped to the family that
ran the lookup, so a leaked token can't be replayed cross-family.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Refresh AVM property valuations monthly instead of daily
Per @jjmata's review on #2727: valuations change slowly and providers
enforce tight monthly request caps, so a daily cron adds no value.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Api::V1::HoldingsController built both the index and show queries from
current_resource_owner.family.holdings, filtering only on account status.
A read-scoped API key could therefore read holdings (account id/name,
quantity, price, market value) for accounts owned by other family members
that were never shared with the key owner, and the account_id / account_ids
filters made targeted enumeration of those accounts trivial.
Scope the query through accounts.accessible_by(current_resource_owner) so
results are limited to accounts the token owner owns or has been granted
access to. Route both index and set_holding through the new
accessible_holdings scope. This matches Api::V1::BalancesController and
the web HoldingsController, which already scope the same way.
Fixes#2467
Co-authored-by: agentloop <agentloop@localhost>
* fix Prism issue - assigned but unused variable plaid_item
* measurement to improve accounts_controller__index performance based on skylight
* fix FEEDBACK - Stop eager-loading full sync histories on accounts index
* fix FEEDBACK - Remove sticky memoization from SimplefinItem#accounts
* fix lint error in PR review
* fix FEEDBACK - Avoid returning map misses as authoritative results
* fix test error
* fix FEEDBACK - Avoid redundant query by using already-loaded @manual_accounts
* Address maintainer review on accounts index sync preloading
Memoize SimplefinItem#accounts, align syncing? with Sync#visible?,
and add fallback regression tests.
* Add composite index for sync DISTINCT ON queries
Support latest_by_syncable ordering with syncable_type, syncable_id,
created_at DESC, and id DESC.
* fix error in dockerfile.preview file
* Suppress Pipelock false positives on CI database fixtures.
Pipelock 2.7.0 full-repo audit flags ephemeral postgres:// URLs in
workflow env blocks. Re-apply inline suppressions lost in the main merge.
* Address review: guard partial Current sync maps and drop unrelated diff
Add key? checks so partially populated Current sync maps fall back to DB
queries. Revert Dockerfile.preview and pipelock.yml changes unrelated to
the accounts index N+1 optimization.
* fix(sync): fully populate Current sync maps for all preloaded syncables
* fix(ci): skip scheduled preview cleanup on forks
Only run the hourly Cloudflare preview cleanup on we-promise/sure,
where the required secrets exist.
* Remove schema.rb Postgres-version churn, keep only new syncs index
Reset db/schema.rb to upstream/main and re-add only the
index_syncs_on_syncable_and_created_at_and_id index. The prior diff
included ~30 lines of noise (check-constraint reformatting, virtual
column reformat, and column reorderings) caused by dumping under a
different PostgreSQL version, which obscured the single intended
schema change and invited merge conflicts.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(snaptrade): replace device-flow OAuth with authorization-code + PKCE flow
Squashed from 16 commits on snaptrade-oauth-apps for a clean rebase onto
current upstream/main ahead of opening a PR.
* fix(snaptrade): address PR #2747 review feedback on OAuth PKCE flow
- Remove unreachable dead-code guard in import_latest_snaptrade_data
- Guard apply_oauth_tokens! against a malformed payload missing access_token
- Wrap token endpoint network errors in ApiError and retry like data calls
- Remove unused Provider::Snaptrade#revoke_token! instance method
- Preserve return_to/accountable_type through the SnapTrade portal callback
so the account-linking flow no longer drops users back to accounts_path
- Show the real absolute OAuth callback URL in self-hosted setup instructions
- Refresh brakeman.ignore fingerprint for the connect redirect after the
return_to/accountable_type params were added
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y8SCCmKX6RphB5E73WSUQQ
* fix(snaptrade): don't retry non-idempotent OAuth/API requests
CodeRabbit flagged that Provider::Snaptrade retried OAuth token
exchanges/refreshes and all API POST/DELETE calls (get_connection_url,
delete_connection) after timeouts/connection failures. If the response
is lost after SnapTrade already consumed a single-use auth code,
rotated the refresh token, or applied a POST/DELETE, replaying the
request either fails with invalid_grant on a token that actually
succeeded, or risks duplicate side effects. Retries are now limited to
GET requests; OAuth token requests and non-GET API calls translate a
network failure straight into an ApiError without replay.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NrrGkgSBEqhjjBmmH1fcXL
* fix(snaptrade): stop querying non-deterministically encrypted token via empty-string compare
CodeRabbit flagged that the syncable scope's where.not(oauth_access_token:
[nil, ""]) re-encrypts "" with a random IV on every query, so the ""
comparison can never match a stored ciphertext and is a silent no-op.
No code path ever persists oauth_access_token as "" (only nil or a real
token via apply_oauth_tokens!), so the exclusion is unnecessary --
narrowed the scope to a plain NULL check, which encryption handles
transparently since nil is never encrypted.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NrrGkgSBEqhjjBmmH1fcXL
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Prevent Yahoo crumb cache poisoning
* Make Yahoo Finance health status rate-limit aware
* Normalize Yahoo Colombian listings
Map Yahoo's BVC code to XBOG so search results derive Colombia from canonical exchange metadata and price requests use the .CL suffix with a COP fallback.
Spec: .scratch/normalize-yahoo-colombia-listings/spec.md
* Move Yahoo health checks to background
* fix: resolve insecure OAuth redirect URI registration
Reject unsafe redirect URIs during dynamic OAuth client registration so only HTTPS and loopback HTTP callbacks are accepted. This prevents token/code exfiltration through attacker-controlled redirect targets while preserving local native client flows.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: normalize IPv6 loopback hosts in OAuth redirect validation
Strip bracketed IPv6 hosts before comparing against LOOPBACK_HOSTS so
http://[::1] callbacks are accepted as intended.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix N+1 queries on categories index by batching lookups and removing partial fallbacks
* resolve Codex review suggestion about Guard subcategory against missing parents
* resolve coderabbitai review suggestion - Guard against empty categories when computing category_ids_with_transactions
* resolve coderabbitai review suggestion - Redundant pb-4 makes the conditional dead code
* resolve coderabbitai caution on PR review
* resolve sure-design review - DS Drift Patrol
* fix conflicting hidden/flex classes
* resolve jjmata review suggestion - schema.rb noise
* fix conflict and adjust related parts
* Ignore Brakeman EOLRails warning for Rails 7.2
Restore ignore entry lost during merge from main; documents upgrade
tracking and matches brakeman 7.1.2 in Gemfile.lock.
* db migration executed
* Remove deprecated focus-ring override from goals color picker.
The summary_class override was reintroduced during merge conflict
resolution and clobbered DS::Disclosure's canonical focus styling.
* fix merge conflict on disclosure.rb
* Restore parent-based semantics while keeping the performance win for root categories
* fix(ci): skip scheduled preview cleanup on forks
Only run the hourly Cloudflare preview cleanup on we-promise/sure,
where the required secrets exist.
* Revert schema.rb dump noise unrelated to categories N+1 fix
Restore db/schema.rb from main so PostgreSQL check-constraint and
column-order churn does not obscure the real PR changes.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Remove obsolete Rails 7.2 EOLRails Brakeman ignore
Main is already on Rails 8.1, so the 7.2.3.1 ignore entry is no longer needed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address review: bare disclosure docs, category picker aria label
* fix(test): wait for async exchange rate updates in system test
* fix(test): retry account edit submit around Turbo morph races
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: resolve ApiRateLimiter connecting to wrong Redis instance
Use REDIS_URL for API key rate limiting so hosted deployments share the
configured Redis backend instead of defaulting to localhost.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: align ApiRateLimiter specs with configured redis_url
Use ApiRateLimiter.redis_url for test cleanup and assert the public
redis_url reader instead of inspecting private Redis client internals.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(sync): allow EnableBanking credit card balances to be parsed as available credit
* feat(ui): expose Enable Banking balance interpretation toggle on credit card form
Adds a toggle to the credit card edit dialog, shown only when the account
is linked to an Enable Banking provider account. Toggling persists
treat_balance_as_available_credit and enqueues an item sync so the balance
is reinterpreted immediately.
* fix(sync): keep existing balance when credit limit is missing for available-credit cards
When treat_balance_as_available_credit is on and the API omits credit_limit,
the reported balance is known to be available credit, so recording it as
debt fabricates a liability. Skip the balance write in that case and only
update the available credit metadata.
Also extract the credit card branching into a helper and include
provider_key, account_provider and metadata in the debug log entries so
support can filter /settings/debug by the affected connection.
* refactor(ui): move provider lookup to Account and label the toggle for assistive tech
Adds Account#provider_account_for so the view no longer queries
account_providers directly, and wires aria-labelledby from the toggle to
its visible label.
* fix(ui): apply Enable Banking setting only after a successful account update
Runs the provider flag update after super and only on the redirect path,
so a failed account save no longer persists the flag or enqueues a sync.
Also permits the enable_banking params so malformed input is filtered
instead of raising.
* test(sync): extract relink helper in Enable Banking processor test
* feat(sync): fall back to manually set available credit as the credit limit
When the toggle is on, the accountable's available_credit field now holds
the credit limit (API-provided, or user-entered when the API omits it) and
is never overwritten with the reported balance. This lets users whose bank
reports available credit without a credit limit compute the outstanding
debt by entering their limit in the Available credit field, while keeping
the field stable across syncs so a stale reported balance can never be
mistaken for a limit.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
The "Advanced" settings section is gated to admins in the navigation, but
the controllers exposing family-wide data performed no server-side
authorization — any authenticated user could reach them by navigating
directly to the URL.
Add `before_action :require_admin!` to the controllers that expose
family-wide data:
- Settings::AiPromptsController (family AI assistant config)
- Settings::LlmUsagesController (family LLM usage/billing)
This mirrors the existing guards on Settings::ProvidersController and
Settings::HostingsController. Non-admins are redirected with the standard
"Only admins can perform this action" message (403 for turbo_stream/json).
API key and MCP token management are intentionally left open: both are
user-scoped self-management. Their actions are already scoped to
Current.user, and any signed-in user (not just admins) can create an API
key or authorize an MCP client, so gating them would leave members unable
to view or revoke their own credentials.
Adds controller tests covering admin-only access for the family-wide pages
(member and guest rejected) and member self-service access for the API key
and MCP pages.
`app/views/goals/new.html.erb` called `t("common.close")`, but that key is
only defined at the top level in `ru`, `tr` and `zh-CN`. The canonical key is
`defaults.common.close`, which is defined in `en` (and therefore resolves in
every locale via `config.i18n.fallbacks`) and is already used by
`settings/providers/_drawer_header.html.erb` and
`shared/notifications/_sync_toast.html.erb`.
Because `en` has no top-level `common.close`, the fallback had nothing to fall
back to, so the lookup failed everywhere except those three locales. The
failed lookup was passed straight into the button's `title` and `aria_label`
attributes, injecting literal markup into text meant for screen readers:
t("common.close") @ en
=> "<span class=\"translation_missing\" title=\"translation missing: en.common.close, locale: en\">Close</span>"
Point the view at `defaults.common.close` and drop the now-dead top-level
`common:` blocks from `ru`, `tr` and `zh-CN`. No `en.yml` changes needed.
Verified with `bin/rails runner`: `defaults.common.close` resolves in en, ca,
fr, it, ru, tr, zh-CN, and falls back cleanly to "Close" for de/es. No
`t("common.*")` call sites remain in `app/`.
Closes#2740
Co-authored-by: erkdgn <erkdgn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(settings): super-admin background jobs console
Neither managed nor self-hosted production deployments have any view
into background job state (/sidekiq is only mounted outside
production), so a stuck sync, import, or export is invisible until a
user complains — and even then there is no way to act on it.
Adds /settings/background_jobs, gated by the same super-admin
Admin::BaseController as /settings/debug:
- Worker status header: Sidekiq processes, busy count, queue depths
and latency, retry/scheduled/dead set sizes. Read through a
fail-closed PORO (BackgroundJobConsole) — when Redis is unreachable
the console says so and disables actions instead of pretending
health (deliberate contrast to BackgroundJobHealth's fail-open).
- In-flight operations across all families: incomplete Syncs, Imports
importing/reverting, ImportSessions importing, FamilyExports
pending/processing, each with a liveness verdict derived from
Sidekiq::Workers (job GlobalIDs in running payloads).
- One mutation: mark a presumed-lost operation as such (Sync → stale,
Import → failed/revert_failed, PdfImport → claim released back to
pending, ImportSession/FamilyExport → failed).
Guard rails on the mutation, server-side re-checked:
- refused while Redis state is unknown (fail closed)
- refused while the record's job is visibly executing
- refused until the record has been idle past 30 minutes, so a merely
queued job cannot be shot down and then still run
- refused for parent Syncs with children still in flight (a parent
legitimately has no live job of its own while children run)
- applied inside with_lock with a status re-check, so a job finishing
between render and click wins
- audited as a DebugLogEntry (actor, prior status, family)
The cancel endpoint gets its own Rack Attack throttle since the
console deliberately lives under /settings rather than the throttled
/admin prefix (its 10 req/min limit would fight the page's polling).
* fix(jobs-console): count waiting jobs as live and harden cancel paths
Review feedback on #2682 (Codex, CodeRabbit):
- Liveness now covers jobs sitting in queues, the retry set, and the
scheduled set, not just visibly-executing workers — a job waiting out
a backlog or retry backoff WILL run later, and most affected job
classes don't abort on a flipped status, so cancelling invited
duplicate work. The backlog scan is bounded (5k entries); a truncated
scan fails closed like redis_error?. The liveness column shows
"Queued" for these
- find_record! resolves STI subclass names (TransactionImport,
PdfImport, …) against a base-class whitelist via safe_constantize
instead of a fixed name map, so non-UI callers naming the subclass
don't 404
- A PdfImport stuck in reverting goes to revert_failed like every other
import instead of being released to pending — pending presented a
possibly half-reverted import as publishable again; only the AI
extraction claim (importing) is released to pending
- Redis-unreachable warning renders via DS::Alert; operation id cast
to_s before splitting; admin-cancel error copy moved to i18n
* fix(jobs-console): re-check the stuck window inside the cancel row lock
CodeRabbit round-2: cancellable? evaluates Sidekiq liveness outside the
with_lock transaction (re-running Redis calls under a row lock would be
worse), so a worker picking the job up between the liveness check and
the lock acquisition was invisible. Repeating the updated_at staleness
check inside the lock closes that window — a freshly-started job
touches the record, and the re-check refuses the cancel.
* fix(jobs-console): resolve record_type without reflection, i18n nits
Brakeman flagged safe_constantize on params[:record_type] as a
High-confidence UnsafeReflection (ci/scan_ruby). Replace the
constant lookup with a reverse lookup: find the id in each
cancellable base table and require the claimed type to match the
found record's class or its base class. STI subclass names still
resolve; unknown types still 404.
Also from DS Drift Patrol:
- "Sync · " label in _operation.html.erb now goes through
t(".sync_label", type:)
- drop the redundant default: on background_jobs_label now that
the key exists in the locale file
* fix(wise): redirect after token submission instead of rendering inline
The success path in create rendered select_profiles directly (200 OK),
which Turbo rejects for standard form submissions ('Form responses must
redirect to another location'). Now it redirects, carrying the encrypted
token through the session. Session-expired fallbacks now point at
settings_providers_path instead of new_wise_item_path, which has no view.
* test(wise): update controller specs for redirect-based create flow
* fix(wise): read pending token from session, not client params
link_profiles decrypted params[:encrypted_pending_token], even though create
already stores the encrypted token server-side in the session. The client
round-trip was unnecessary and untrusted; now link_profiles reads directly
from session[:wise_pending_encrypted_token].
* Fix statement links in account tab breaking out of Turbo frame, fixes#2614
* Fix statement navigation and split view/edit actions in account tab
- Change eye icon to open the actual file inline in a new tab
- Add pencil icon for navigating to the statement details/edit page
- Make filename click open the file directly (PDF inline, CSV/XLSX download)
* Refactor: extract file_attached local variable to reduce duplication
* added locale context for edit button for account statements
* added test for frame navigation for fix
* Assert turbo frame attribute on unlink form in test
* Fix turbo_frame attribute on unlink button_to form element
The transaction drawer already renders its category dropdown with the
badge variant (colour + icon + search); the transfer drawer and the
bulk-edit modal still show a bare alphabetical list. Bring those two
into line for a more consistent category-picking experience.
* Add send_email_notification rule action
Adds a new rule action that emails a digest of transactions matching a
rule. Re-syncs re-apply every active rule to all in-window matches, so a
notification_deliveries table (unique on rule_id + transaction_id) backs
deduplication and a per-rule watermark:
- Rule::ActionExecutor::SendEmailNotification plucks candidate ids, drops
ones already in notification_deliveries, records the remainder BEFORE
enqueuing (fail-safe: a crash suppresses rather than double-sends), and
returns the count of newly-notified transactions.
- RuleEmailNotificationJob loads the rule + transactions and delivers the
digest via RuleNotificationMailer.
- Creating the action pre-seeds all currently-matching transactions as
already-delivered (after_create_commit), so the rule only emails about
transactions appearing after the action exists.
Dedup keys on the DB row id, not provider identity, so a re-ingested
transaction (new id) may re-notify; accepted as benign.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add view transactions link to rule notification digest email
Include a "View transactions" CTA (APP_DOMAIN/transactions) in both the
HTML and text versions of the rule notification digest email.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address review feedback on rule email notifications
- Restrict digest delivery to family admins only (drop non-admin fallback)
- Make NotificationDelivery.record_for return only inserted ids and enqueue
off that result, preventing duplicate digests under concurrent rule runs
- Seed the notification baseline when an existing action is changed to
send_email_notification, so historical matches are not emailed
- Strengthen tests: assert the transactions CTA/link in the digest and the
enqueued job's transaction-id args
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Include super_admin owner as rule digest recipient
The admin-only recipient lookup used find_by(role: :admin), which excluded
super_admin owners. A self-hosted family is commonly a single super_admin, so
the digest was silently skipped (NullMail no-op) and no email was sent.
Match the recipient on %w[admin super_admin] (the same pattern used elsewhere,
and consistent with User#admin?), while still excluding regular members/guests.
Add tests for the super_admin recipient and the no-admin skip path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add mixed-role digest recipient test
Cover the case where a family has both an admin and a super_admin. The recipient
lookup uses find_by(role: %w[admin super_admin]) with no ORDER BY, so which one
is returned is non-deterministic; the contract is only that the recipient is an
admin-level user (never a member). Assert recipient.admin? rather than a brittle
precedence between the two roles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Deliver rule digest via deliver_later and order in DB
Use deliver_later so a slow/flaky SMTP connection doesn't tie up the
Sidekiq worker, and push the entry-date sort into the query instead of
materializing and sorting the result set in Ruby.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Replace raw hex colors with email-safe design tokens in digest template
The rule digest email hardcoded Tailwind slate-100/200 hex values for
table borders, which aren't part of this project's design system.
Resolve to the actual border-primary/border-secondary token values and
centralize them as a reusable .email-table class in the mailer layout.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(lunchflow): prune orphaned accounts deleted upstream (#1861)
Accounts deleted in Lunch Flow lingered in Sure as unlinked
LunchflowAccount records, permanently pinning the item to "Need setup".
The importer created/updated accounts returned by the API but never
removed records for accounts that disappeared upstream.
Add prune_orphaned_lunchflow_accounts, mirroring SimpleFin's
prune_orphaned_simplefin_accounts: after importing, delete LunchflowAccount
records whose account_id is no longer returned upstream and that are not
linked to an Account via AccountProvider. Linked accounts are kept so the
prune never cascade-destroys a user's Account. The prune is guarded to a
non-empty upstream list so a transient empty/failed response can't wipe out
all accounts.
Fixes#1861
* fix(lunchflow): prune NULL-id orphans + consistent import return shape (#1886)
* fix(lunchflow): make orphan prune resilient to destroy failures
Wrap the per-record destroy in begin/rescue so a single failed prune
doesn't abort the whole import. Pruning runs before transaction fetch,
so an unhandled error would have blocked transaction syncing entirely.
Matches the importer's existing continue-on-error convention.
* fix(lunchflow): only count pruned accounts when destroy succeeds
destroy returns false (without raising) when a before_destroy callback
halts deletion; the prior code incremented the pruned count regardless,
which could inflate it. Increment only on success and log when halted.
* fix(enable-banking): handle overdraft balance sync
* fix(enable-banking): skip nil provider balances
* test(enable-banking): cover nil provider balance
* fix(enable-banking): avoid stale and unsafe balance logs
* fix(enable-banking): guard unavailable balance writes
* test(enable-banking): stub unsaved account api id
* style(enable-banking): remove extra test blank line
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* fix(import): strip non-breaking-space thousands separators in sanitize_number
The French/Scandinavian number format ("1 234,56") only removed ASCII
whitespace via \s, which does not match a non-breaking space (U+00A0) or
a narrow no-break space (U+202F). Real-world European CSV exports use
those Unicode spaces as the thousands separator, so such amounts failed
the final numeric guard and were silently coerced to "", losing the
value on import. The branch also skipped the non-numeric junk stripping
that the other formats apply.
Strip everything that isn't a digit, the decimal separator, or a minus
sign in this branch, matching the else branch's behavior.
Fixes#2537
* fix(import): strip only whitespace for space-delimited number format
Addresses review feedback: the previous filter removed any non-numeric
character, so a misconfigured/mixed row using US separators ("1,234.56")
under the "1 234,56" format had its period dropped and its comma turned
into a decimal point, silently importing 1.23456 (off by ~1000x) instead
of being rejected.
Strip only whitespace via \p{Space}, which matches ASCII, non-breaking
(U+00A0), and narrow no-break (U+202F) spaces. Unexpected punctuation is
left in place so the existing numeric guard still rejects malformed
values. Add a regression test for the mixed-punctuation case and a
docstring for sanitize_number.
* fix(import): strip leading/trailing currency junk for space-delimited numbers
Follow-up to review on #2538: the whitespace-only strip did not cover
issue #2537's currency-suffix row ("1 234,56 kr") or a leading currency
symbol ("€1 234,56"), which still failed the numeric guard and imported
blank.
Strip non-numeric junk only at the leading/trailing edges (currency
symbols/codes), leaving interior characters untouched. This fully closes
the #2537 repro table while preserving the earlier fix for the mixed
US-format hazard: "1,234.56" keeps its interior period and is still
rejected rather than parsed as 1.23456. Digits, the decimal separator,
and a leading/trailing minus are preserved so signed values and the
guard still behave correctly.
Add regression tests for the currency prefix/suffix cases.
* test(import): add U+202F narrow no-break-space regression case
Complements the existing U+00A0 case so both Unicode thousands-separator
variants named in the fix are covered.
* Add Italian translations for various views and functionalities
- Introduced translations for splits, subscriptions, tag deletions, tags, trades, transactions, transfer matches, transfers, users, valuations, and vehicles.
- Enhanced user experience by providing localized content in Italian for better accessibility and understanding.
* Correzioni traduzioni italiano: API keys shared/index, super admin jobs, imports readback, snaptrade idiomatica, subscriptions progetto, valuations grammatica
* Aggiornamenti delle traduzioni italiane: miglioramenti e aggiunte per vari moduli, inclusi conti, obiettivi, trasferimenti e integrazione con SnapTrade.
* Add per-account toggle to disable automatic transaction categorization
Adds an `enable_category_matcher` boolean (default: true) to accounts so
users can opt out of Plaid's automatic category suggestions on a per-account
basis. When disabled, newly synced transactions arrive uncategorized so
rules or manual assignment take precedence.
- New migration adds `enable_category_matcher` column (default true, null: false)
- `PlaidEntry::Processor#matched_category` gates the CategoryMatcher call on the account flag
- Toggle rendered in the account edit modal for linked accounts (saves via main form submit)
- `AccountableResource#account_params` permits the new field
- `AccountsController#toggle_category_matcher` action added for potential API use
- i18n strings added for label and hint text
- Unit test covers the disabled-matcher path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Only show category matcher toggle for Plaid-linked accounts
Only PlaidEntry::Processor honors enable_category_matcher, but the toggle
rendered for every linked account, silently doing nothing for other
providers. Adds Account::Linkable#supports_category_matcher? (covering both
the legacy plaid_account_id link and AccountProvider rows) and gates the
form toggle on it. The SimpleFIN TODO now also points at this helper so the
toggle appears once SimpleFIN matching lands.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add controller tests for category matcher toggle persistence and rendering
Covers the two paths flagged in review as untested: the flag persisting
through the shared AccountableResource#update action via account_params
(both disable and re-enable), and the edit form rendering the toggle only
for accounts where supports_category_matcher? is true.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Signed-off-by: Mike Lloyd <49411532+mike-lloyd03@users.noreply.github.com>
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
The categorize view renders dates with :short (\"%b %d\"), which omits the
year — ambiguous when the uncategorized backlog spans more than one year.
Use format_date, which renders the family's Settings date format; every
selectable format includes the year.
/sidekiq was mounted `unless Rails.env.production?`, and the Docker
image bakes RAILS_ENV=production — so no self-hosted or managed
deployment has ever had queue tooling, while the production basic-auth
block (with default "sure"/"sure" credentials) was dead code. Worse,
any custom non-production env (e.g. staging) got the dashboard with no
authentication at all.
Now:
- Development keeps the open mount for convenience.
- Everywhere else the route only exists for a signed-in super admin,
via a routing constraint that resolves the signed session cookie the
same way Authentication#find_session_by_cookie does. The session's
user is always the true user (impersonation is resolved at the
Current level and impersonating a super admin is forbidden), and
sessions are only created after MFA verification, so neither can
bypass it. Fails closed — errors mean 404.
- The "sure"/"sure" default credentials are deleted. Basic auth now
activates only when BOTH SIDEKIQ_WEB_USERNAME and
SIDEKIQ_WEB_PASSWORD are explicitly set, as an optional second
layer on top of the constraint.
- Documented in .env.example and docs/hosting/docker.md, including
warnings that the dashboard is break-glass tooling: never manually
retry SimplefinConnectionUpdateJob (single-use token), and deleting
jobs does not update the corresponding Sure records.
The super-admin bar's Jobs link is feature-detected via
sidekiq_web_available? and lights up automatically now that the route
exists in production.
* feat(sync): family-facing cancellation for syncs, imports, and exports
Users had no way to stop or recover any background operation: a
mistaken "Sync all" runs to completion, and an import or export whose
job died (hard worker kills lose in-flight Sidekiq jobs) wedges with a
spinner forever.
Sync cancellation (cooperative — nothing is ever killed):
- New syncs.cancel_requested_at column. Only the cancelled sync
carries the flag: pending descendants are marked stale immediately
(their queued jobs no-op via the existing may_start? guard), while
descendants whose jobs are already executing finish their work
honestly.
- Family::Syncer stops fanning out child syncs once the flag is set
(fresh read per iteration — the flag comes from the web process).
- Finalization resolves a cancel-requested sync to stale instead of
completed, which also skips post-sync (transfer matching, rules,
broadcasts) via the existing stale gate.
- The `visible` scope excludes cancel-requested syncs, so spinners
clear immediately and — fixing a latent bug this feature would have
amplified — sync_later no longer piggybacks a new sync request onto
a dying sync it would silently swallow.
- Cancel button appears next to "Sync all" on the accounts page while
a family sync is visible. SyncsController#cancel scopes through
Sync.for_family with resource_owner, so cross-family ids 404 and
account-level syncs respect per-user account access.
Stuck import/export self-service:
- Import#force_fail! / FamilyExport#force_fail!: allowed only once the
record has been idle past PRESUMED_LOST_AFTER (1 hour — dwarfs any
legitimate run), and applied inside with_lock with a status
re-check, so a job finishing between page render and button click
wins. Imports fail into the existing retry path (reverting ->
revert_failed keeps the revert retryable); PdfImports release their
processing claim back to pending; exports fail so a new one can be
created.
- "Mark as failed" buttons appear on the imports/exports index rows
only when a record is presumed lost, behind the pages' existing
permission gates (statement-import permission for imports, admin
for exports).
* fix(sync-cancel): cascade pending cancels, guard late finalizers, scope provider syncs
Review feedback on #2685 (CodeRabbit, Codex):
- request_cancel! now cascades finalization for pending syncs too: a
pending child resolved to stale never runs its job, so nothing else
would ever call finalize_if_all_children_finalized — its waiting
parent hung in syncing until the 24h sweep (CodeRabbit critical)
- SimplefinItem::Syncer#mark_completed re-reads the sync under a row
lock and skips finalization once cancellation was requested or the
row went terminal — its in-memory copy predates the cancel, and the
unguarded complete! (plus the raw status fallback) resurrected a
cancelled sync and re-ran post-sync (Codex)
- Cancelling provider-item syncs now requires admin: for_family's
resource_owner only scopes the Account branch, so a restricted member
could cancel admin-managed provider syncs spanning accounts they
cannot see. Family- and account-level syncs stay member-cancellable,
matching the buttons the UI shows (Codex)
- Lost-import error copy moved behind i18n
(imports.errors.presumed_lost), resolved at call time (CodeRabbit)
- Tests: pending-child cancel finalizes the parent; late provider
complete! cannot resurrect a cancelled sync; provider-sync
cancellation is admin-only
* fix(sync-cancel): capture the skipped-finalization case via DebugLogEntry
CodeRabbit round-2: the mark_completed skip (cancelled/terminal sync)
is support-relevant — record it in the super-admin debug UI with the
family and provider attached instead of a raw Rails.logger line.
category: provider_sync, matching the other provider syncers.
* Fix keyboard navigation in DS::Select dropdowns.
Open the listbox on Tab focus and arrow/space/enter from the trigger so
account and category selectors in the transaction form are usable without
a mouse, and reset option tabindex when the menu closes.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Tab skipping DS::Select triggers in transaction form.
Keep focus on the trigger when Tab opens the menu, keep listbox options
at tabindex -1 with inert on the closed menu, and move focus into options
only via arrow keys so Tab no longer jumps to amount or date.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix search input losing focus during DS::Select filtering.
Only repoint focus in syncTabindex when a listbox option was focused and
became hidden; skip while the user is typing in the search field or when
clearSearch runs on menu open.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Escape closing the parent dialog from an open DS::Select.
Stop Escape propagation so only the listbox closes, and suppress
handleButtonFocus briefly after close+refocus so keyboard modality
does not reopen the menu via :focus-visible.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Tab leaving DS::Select after suppressReopenOnFocus change.
Focus the trigger before closing on Tab and advance to the next dialog
field manually so inert on the menu no longer drops focus to body.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(provider): resolve Tiingo security currency from countryCode
Tiingo's search API never returns the priceCurrency field the code was
reading, so currency detection silently failed. Currency is now derived
from countryCode via the countries gem's ISO 4217 data, with a
best-match tie-break for tickers that collide across countries so the
currency shown in search results always matches what fetch_security_prices
later returns.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(provider): guard tiingo currency cache against non-US downgrade
Addresses PR review feedback (jjmata): the per-ticker currency cache
written in search_securities was unconditionally overwritten on every
search, keyed only by ticker. A later search whose result set doesn't
happen to include a ticker's US cross-listing could silently downgrade
a previously-cached USD (the currency actually backing daily price
data) to a foreign currency. Only overwrite when the current search's
match is US, or nothing is cached yet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
