* fix(provider): resolve Tiingo security currency from countryCode
Tiingo's search API never returns the priceCurrency field the code was
reading, so currency detection silently failed. Currency is now derived
from countryCode via the countries gem's ISO 4217 data, with a
best-match tie-break for tickers that collide across countries so the
currency shown in search results always matches what fetch_security_prices
later returns.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(provider): guard tiingo currency cache against non-US downgrade
Addresses PR review feedback (jjmata): the per-ticker currency cache
written in search_securities was unconditionally overwritten on every
search, keyed only by ticker. A later search whose result set doesn't
happen to include a ticker's US cross-listing could silently downgrade
a previously-cached USD (the currency actually backing daily price
data) to a foreign currency. Only overwrite when the current search's
match is US, or nothing is cached yet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
CoinbaseAccount::Processor uses the compact class form, so its lexical
nesting is only [CoinbaseAccount::Processor]. The bare `HoldingsProcessor`
reference in process_holdings resolved against that nesting and raised
`uninitialized constant CoinbaseAccount::Processor::HoldingsProcessor`
instead of finding CoinbaseAccount::HoldingsProcessor.
The error was swallowed by the rescue around process_holdings, so every
Coinbase sync logged the failure and skipped the holdings step while the
balance path still completed. Holdings and value never refreshed.
Qualify the reference to CoinbaseAccount::HoldingsProcessor so it resolves
absolutely, matching every sibling processor (Binance, Kraken, Snaptrade,
Questrade, Indexa).
Fixes#2412
Co-authored-by: agentloop <agentloop@localhost>
A sync marked stale by SyncCleanerJob while its job is still running
hits two lost-update paths when that job finishes:
- success path: finalize skipped the status transition but still ran
perform_post_sync, re-applying transfer matching, rules, and
broadcasts for a sync the system had already written off
- failure path: the rescue's unguarded fail! silently overwrote the
terminal stale status with failed (the in-memory record still read
syncing, so the AASM guard never fired)
Fix: re-check state under a row lock (with_lock reloads) before
failing, and skip post-sync for stale syncs in finalize. Post-sync
still runs for failed syncs — that behavior is intentional and
covered by existing tests.
* feat(wise): add Wise integration with JAR savings account and activity support
- Add WiseItem/WiseAccount models with full sync pipeline (importer, syncer, processor)
- Detect income vs expense using targetAccount == recipientId from borderless accounts API
- Support JAR (SAVINGS) accounts with totalWorth balance and savings subtype
- Fetch JAR activity via profile activities API (INTERBALANCE, BALANCE_CASHBACK, BALANCE_ASSET_FEE)
- Route INTERBALANCE activities to both JAR and STANDARD accounts and link as Transfer records
- Add provider connection status registration, routes, views, and i18n
- Add migration for wise_items and wise_accounts tables
- Add tests for WiseAccount, WiseEntry::Processor, WiseActivity::Processor, WiseItem::Importer, and WiseItem#link_jar_transfers!
* chore(lint): add ignore to security scan (false positive)
* fix(wise): address PR review feedback on activity routing, HTML stripping, rate limiting, and scope extraction
- Replace title string matching in activity_for_account? with resource.id vs balance_id comparison to avoid breakage when users rename JAR accounts on Wise
- Replace gsub(/<[^>]+>/, "") with ActionController::Base.helpers.strip_tags to safely handle user-controlled HTML-like content
- Wrap paginated API calls in with_rate_limit_retry (up to 3 attempts, exponential backoff) to handle 429 responses during fetch_jar_activities and fetch_transfers
- Extract WiseAccount.unlinked scope and remove duplicated left_joins query from controller
* fix(wise): address PR #2433 review feedback
- Wire @wise_items into AccountsController#index so linked accounts appear on /accounts
- Fix find_wise_account_for_linking to preserve .active scope via .merge instead of .then
- Fix cross-currency incoming transfer amount to use targetValue instead of sourceValue
- Add missing select_profiles.session_expired locale key
- Add missing account_name interpolation to link_existing_account success notice
- Use i18n for profile type labels in profile_display_name
- Trigger wise_item.sync_later after account linking in all three linking actions
- Isolate fee transaction failure so it no longer aborts the main transfer import
- Use bare raise in WiseActivity/WiseEntry processors to preserve original backtrace
- Re-raise in WiseItem::Unlinking to prevent silently orphaned Holdings
- Fix avatar badge to use design system tokens (bg-container-inset, text-primary)
* fix(wise): fix INTERBALANCE routing and encrypt pending token in session
* fix(wise): replace hand-rolled buttons/links with DS::Button and DS::Link
Address repeated sure-design DS drift findings: migrate all manual
Tailwind button_to and link_to calls in Wise views to DS::Button
(outline/outline_destructive variants) and DS::Link (secondary variant).
Add missing provider_panel.disconnect locale key for the disconnect button text.
* fix(wise): use render_provider_panel_error in create instead of missing new template
* fix(wise): add missing comma in PROVIDERS array
CI failed with a SyntaxError because the questrade entry wasn't
comma-terminated before the wise entry.
* chore(wise): re-add pipelock:ignore for pending token param
Lost when the token source moved from session to an encrypted params
field in 0b5dc886, causing the CI secret scanner to flag a false positive.
* fix(test): widen random ticker suffix to avoid rare collision flake
hex(2) only yields 65536 possible tickers, so create_trade's 4 calls per
test run had a small but nonzero chance of colliding on the unique
ticker+exchange index. hex(8) makes collisions practically impossible.
* feat(exchange-rates): add Frankfurter as an exchange-rate provider
Frankfurter (frankfurter.dev) is a free, keyless FX rates API backed by
ECB daily reference rates, with no published rate limit and no auth
flow to maintain (unlike Yahoo Finance's reverse-engineered cookie/
crumb auth or TwelveData's fast-exhausting free tier).
Follows the Provider::MoexPublic template: Faraday client with retry
middleware, SslConfigurable for self-hosted CA support, a light
RateLimitable throttle, and a FRANKFURTER_URL env escape hatch for
self-hosters. Registered as exchange-rates-only (no security/stock
data) and added to the hosting settings dropdown.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* refactor(provider): switch Frankfurter to the v2 API
v1 is explicitly marked "frozen" on Frankfurter's own root endpoint; v2
is "current" and covers 201 currencies across 84 central banks vs v1's
~30 ECB-only. Confirmed via the v2 OpenAPI spec and live requests:
- Single-date lookups now use GET /rate/{base}/{quote}?date=..., which
carries weekends/holidays forward server-side (a Saturday returns a
real rate directly), so the provider no longer needs its own
lookback-window logic.
- Range lookups now use GET /rates?base=..."es=...&from=...&to=...,
a flat array of { date, base, quote, rate } records (v2's shape)
instead of v1's { "rates": { date => currencies } } hash.
- Every calendar day in a range is present (v2 gapfills itself), rather
than v1's omit-non-trading-days behavior.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(provider): sanitize currency codes before URL path interpolation
from/to were only upcased before being interpolated directly into the
URL path in fetch_exchange_rate (GET /rate/{from}/{to}). Low risk since
currency codes come from validated internal sources, but adds cheap
defense-in-depth: strip anything that isn't A-Z, matching the ISO 4217
format real currency codes always take.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* feat(insights): proactive financial intelligence feed
Adds a nightly job that analyzes each family's finances in pure Ruby and
surfaces typed, stateful insights on the dashboard and a new /insights page.
- Insight model (active/read/dismissed) with a per-family dedup_key unique
index so nightly re-runs refresh rows instead of duplicating them
- Seven generators (spending anomaly, cash-flow warning, net worth milestone,
subscription audit, savings rate change, idle cash, budget health) built on
IncomeStatement, BalanceSheet, RecurringTransaction, and BudgetCategory
- LLM used as a writer, not a reasoner: Insight::BodyWriter narrates
pre-computed facts via the configured provider, with an i18n template
fallback so self-hosted installs without API keys work identically; bodies
are only (re)written when an insight is new or its numbers changed
- GenerateInsightsJob: cron fan-out per family, per-family advisory lock,
metadata-diff upsert that preserves read/dismissed state for unchanged
signals and reactivates on material change
- Dashboard insights_feed section (top 3, collapsible/reorderable) and an
/insights feed page with turbo-stream dismissal; viewing the feed marks
insights read
- Tests for the model, job upsert semantics, and controller flows
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4
* fix(insights): address Codex review — expiry, budget metadata, LLM usage
- Expire visible insights whose condition cleared: generators declare the
insight types they produce, the registry reports which generators ran to
completion, and the job expires visible insights of those types whose
dedup_key was not regenerated. A crashing generator can't wipe out its
healthy insights, and an expired insight reactivates when its condition
returns — unlike a user-dismissed one, which stays dismissed.
- Include a bucketed budget-spent percent in budget_at_risk metadata so the
body refreshes when overall usage moves >=10 points even if the same
categories remain flagged.
- Pass family to chat_response so LLM narration is recorded in llm_usages.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4
* fix(insights): address CodeRabbit review
- Skip the mark-as-read write for Turbo hover-prefetch requests
(X-Sec-Purpose) so unread badges don't clear before a real visit
- Show the New pill on the dashboard feed (active = unread there; the
feed never marks insights read)
- Filter idle accounts in SQL instead of a per-account exists? loop
- Eager-load merchants in the subscription audit query
- Widen the advisory-lock key to the signed-bigint range and log when
acquisition fails so a skipped nightly run is observable
- Mirror the dedup_key unique index as a model validation
- Assert the refresh action enqueues for the signed-in family
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4
* fix(insights): clear stale lifecycle timestamps on reactivation
When an insight resurfaces, the row now leaves no contradictory state
behind: the material-change path clears both read_at and dismissed_at,
and the expired-recovery path clears read_at. Tests assert the contract,
and the dashboard feed test now also locks in the unread "New" badge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4
* fix(insights): drop redundant standalone family_id index
Every composite index on insights already leads with family_id, so the
auto-created single-column index was pure write overhead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4
* fix(insights): stop nightly drift from resurrecting dismissed insights
Three generators stored continuously drifting values (projected amounts,
starting balance, current net worth) in metadata, and any metadata diff
counts as a material change: the body is rewritten (an LLM call when a
provider is configured) and read/dismissed state is cleared. Dismissing a
cash-flow warning was undone at the next 6:00 run.
Metadata now carries only the signal identity plus a coarse bucket, the
same damping the budget generators already use; display values live in
facts. Also orders the idle-cash pick by balance so the selected accounts
don't flip between runs, and documents the dollar-scale threshold
assumption on the relevant constants.
* fix(ds): map DS::Button aria_label into the aria hash
A bare aria_label: option reaches the tag helpers as a literal aria_label
attribute (Rails only dasherizes the nested aria: hash), so the icon-only
fallback overrode it and screen readers announced the insight dismiss
button and the popover trigger as "X".
* fix(insights): gate LLM narration behind AI consent
The nightly job runs unprompted for every family, so narration now
requires someone in the family to have AI enabled: consent to share
financial facts with the provider, and a cost cap in managed mode. The
template fallback keeps behavior identical otherwise. Narration failures
are captured via DebugLogEntry so support can see them.
Adds BodyWriter coverage, including a template-interpolation test for
every generator template key.
* feat(insights): rework the dashboard feed
The feed rendered full insight cards inside the section shell — the only
widget nesting card-on-card — and appended below the fold for every
family with a saved section order. It now mirrors the outflows and
balance-sheet list idiom (inset well with an uppercase mini-header, white
row block, 28px sentiment-tinted icon circles via color-mix on the DS
CSS variables, right-aligned key figures) and leads the dashboard for
saved orders that predate it.
Icon color comes from sentiment, not priority — a savings-rate
improvement is high priority AND good news, and must not render red; red
is reserved for a projected-negative balance. Rows have no hover wash
(cursor plus a gentle icon scale, like the sibling widgets), links to
/insights disable Turbo prefetch so the mark-as-read actually fires for
mouse users, and the standard widget-size popover offers Half/Full. Full
stays the default: the feed is far shorter than any other single-width
widget, so a half default leaves a grid hole the masonry cannot backfill.
* feat(insights): actionable cards, dismiss undo, live refresh
Each row now persists its display facts (new jsonb column) alongside the
change-detection metadata. Facts refresh every run without touching the
body or user state, which is exactly why they are not part of the
material-change comparison.
The card gains what the stored data always supported: a type-and-period
meta line replacing "x minutes ago", a right-aligned key figure (green
only for good news), and a contextual link resolved from the subject ids
in metadata — category, account, recurring transaction, budget month —
omitted when the subject no longer exists.
Dismiss is forgiving: a toast in the notification tray offers undo, and
undismissing restores the row as read rather than re-badging it.
Milestone insights can never regenerate once dismissed, so this closes a
real loss path. Upsert failures are captured via DebugLogEntry.
Manual refresh gets feedback: the button swaps to a disabled checking
state, the page subscribes to a family-scoped stream, and the job
broadcasts the refreshed list and the idle button when it finishes (also
after a lock-skipped run, so the button cannot stay stuck).
Savings copy is sign-aware: a negative rate reads "you spent more than
you earned" with true minus signs. The empty state swaps Lucide sparkles
for the brand assistant glyph (DS::EmptyState learns icon_custom:).
* feat(insights): top-bar entry with unread count
The dashboard feed hides at zero insights and nothing else linked to
/insights, so the manual refresh (and the empty state) were unreachable
for exactly the families who need them: fresh setups before the first
6:00 run. The sticky top bar travels to every screen and its right
cluster had room, so insights get an icon entry there with a monochrome
unread badge. Prefetch is disabled on the link for the same
mark-as-read reason as the feed.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Guillem Arias <accounts@gariasf.com>
* fix: share family accounts with invited members on every sign-up path
A user who joined an existing family through an invitation was added to the
family but saw none of its accounts, even when the family's default sharing is
"share with all members". Only Invitation#accept_for created the AccountShare
records; the OIDC just-in-time sign-up, invite-token registration, and mobile
SSO onboarding paths all skipped it, so invitees landed in the family with an
empty account list. Signups routed into an invite-only default family had the
same gap.
Extract the sharing into Family#auto_share_existing_accounts_with, the single
entry point for "a member just joined, apply the family's sharing policy". It
honors default_account_sharing, only shares with a persisted member of the
family, grants read_only to guests and read_write to everyone else, excludes
accounts the user already owns, and is idempotent. accept_for and every
sign-up path call it, so no current or future join path can reintroduce the
empty-account bug or share accounts across families.
Also wrap the two SSO account-creation paths (OIDC JIT and mobile SSO) in a
transaction covering the user save, invitation acceptance, account sharing, and
identity creation, so a failure partway through can no longer leave a
half-onboarded user with no linked identity.
* address review: guest read_only symmetric in auto_share_with_family!, load-bearing guard comment, mobile SSO private-no-op test
Income trades created via Trade::CreateForm#create_income_trade set
price: 0 on the Trade record. This zero price was ingested by
PortfolioCache#load_prices as a valid price source, causing
ForwardCalculator#build_holdings to compute holding amount = qty * 0 = $0,
and overriding the security's market price on that date.
Additionally, Balance::BaseCalculator#flows_for_date misclassified income
trades (qty=0) as sells, producing spurious non_cash_outflows.
Fixes:
- PortfolioCache#load_prices: filter out zero-price trades from trade
price sources using .select { |t| t.entryable.price&.positive? }
- Balance::BaseCalculator#flows_for_date: separate income trades from
regular trades using their qty (income trades have qty=0), so they
contribute only to cash flows, not non-cash flows
- Balance::SyncCache#converted_entries: preserve the entryable association
target on duped entries so that e.entryable.qty access doesn't N+1
Fixes#2672
* Add native Questrade brokerage provider integration
Adds a per-family Questrade provider so users can sync their Questrade
investment accounts (TFSA, FHSA, RRSP, margin, etc.) directly via
Questrade's free personal API, with no paid aggregator.
- OAuth2 refresh-token flow with single-use token rotation, persisted
under a row lock. Tokens self-renew on each sync; the connected panel
lets users paste a fresh token if a connection goes stale (no need to
disconnect and re-link).
- Imports accounts, balances, positions and activities; multi-currency
holdings with per-currency cash holdings; Norbert's Gambit journals.
- New-account and link-existing-account flows, settings card with
desktop-only setup steps, and connect/update/disconnect.
- Restricted to Investment account types. Registered in the provider
connection-status registry with a syncable scope so it participates in
nightly family sync.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: linting error
* fix: refresh token encrypted
The OAuth token exchange passed the single-use refresh token as a GET
query parameter, so it could leak into URL-based logs (Sentry
breadcrumbs, APM spans, debug output). Switch to POST with a
form-encoded body (RFC 6749 3.2) so the credential stays out of URLs.
Verified Questrade's token endpoint accepts POST (returns 400 for a bad
token, not 405). Adds a test asserting the token travels in the body.
* Address PR review: authz, data integrity, retries, logging
Batch of fixes from the automated PR review:
- Require admin for all mutating/linking Questrade actions, and gate
existing-account linking through accessible_accounts + write permission
(was only Current.family scoped).
- Clear requires_update when a fresh token is accepted; use a real 302
redirect (not 422) on full-page failures.
- Require refresh_token on all saves (not just create) unless the item is
scheduled for deletion.
- Migrations target Rails 7.2; questrade_items state columns are NOT NULL.
- Background activity dedup keys on Questrade fields (matches the importer)
so multiple activities no longer collapse to one.
- Persist the normalized account payload; date-scope synthetic cash
holdings so daily history is not overwritten.
- Retry 429/5xx via a RetryableResponseError instead of hard-failing.
- Route provider error bodies to DebugLogEntry instead of Rails.logger /
exception messages, so payloads do not leak into application logs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address PR review: atomic linking, sync health, retry loop, USD cash
- Wrap account creation + provider linking (+ sync_start_date) in a
transaction in both link paths so a link failure rolls back the orphan
account.
- Surface per-account process/schedule failures in the item sync health
instead of always reporting healthy.
- Always stamp last_activities_sync once the background fetch completes,
so legitimately empty accounts stop being re-queued every sync.
- Treat only the account-currency (CAD) balance as primary cash; other
currencies (e.g. USD) now surface as separate cash holdings instead of
being hidden as primary.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address PR review: serialize token exchange, real processor tests
- Single-use token race: the SDK now wraps every token exchange (initial
and 401 re-auth) in a model-supplied lock that reloads and spends the
freshest persisted token (provided.rb#synchronize_exchange). Two
concurrent syncs/jobs can no longer double-spend the same refresh token.
Adds a test asserting the exchange runs inside the lock with the fresh
token.
- Replace the all-skipped QuestradeAccount processor test stubs with real
fixture-backed tests covering balance anchoring, holdings import, and
Buy-trade import (plus blank-symbol / blank-type guards).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix indentation of spliced Questrade schema blocks
The manually added questrade_accounts/questrade_items create_table blocks
sat at column 0 instead of the file 2-space indent, so rubocop flagged
them as inconsistent. Re-indent to match the rest of the schema.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Include currency and type in the Questrade activity merge key
Two activities that differ only by currency or type could collapse to a
single row in merge_activities. Add both fields to activity_key in the
importer and the background fetch job so multi-currency imports dedup
correctly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address review: infer account currency, Encryptable, safer flag clear
From @jjmata's review:
- Currency: QuestradeAccount#upsert_from_questrade! no longer hardcodes CAD
for every account. upsert_balances! now infers the home currency from the
per-currency balances (the currency holding the cash wins, ties broken by
total equity, default CAD) so USD-denominated accounts are labelled USD and
match the right combinedBalances anchor. Adds tests for USD and CAD cases.
- QuestradeItem now includes the shared Encryptable concern instead of
reimplementing encryption_ready? inline.
- QuestradeActivitiesFetchJob#clear_pending_flag is now best-effort so it can
never mask (and swallow) the original error in perform's rescue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix review issues: safe_return_to_path, DebugLogEntry, financial reset, turbo_prefetch, N+1 counts
- Add safe_return_to_path to QuestradeItemsController (blocks //evil.com
protocol-relative open redirect; same 3-check guard as PR #2591 Wise provider)
- Pass return_to through select_accounts and complete_account_setup so
users land back on the account they were linking from
- Replace Rails.logger.error/warn with DebugLogEntry.capture in controller
and unlinking concern (surface errors in the app debug log UI)
- Add questrade_items to Family::FinancialDataReset::PROVIDER_ITEM_ASSOCIATIONS
so Reset Financial Data actually removes Questrade data
- Add when "questrade" case to load_provider_items in providers_controller
so the settings panel lazy-load refresh works
- Fix turbo_prefetch: false on non-lunchflow provider links in
_method_selector.html.erb and select_provider.html.erb (prevents
prefetch-cache blank-modal bug for all generic sync providers)
- Preload questrade_accounts: :account_provider and build
@questrade_account_counts_map in AccountsController; read from map in
partial instead of calling .count on associations (eliminates N+1)
- Localize default connection name via I18n.t(questrade_items.default_name)
- Add default_name key to questrade_items locale
Patterns and bugs surfaced during review of PR #2591 (Wise provider).
* Cross-apply Wise learnings to Questrade provider
Encryption (matched convention from Wise/jjmata review):
- Add deterministic: true to QuestradeItem#refresh_token
- Add encrypts :raw_payload + :raw_institution_payload to QuestradeItem
- Add Encryptable + encrypts :raw_payload, :raw_holdings_payload,
:raw_activities_payload, :raw_balances_payload to QuestradeAccount
(brokerage-specific columns; matches MercuryAccount/UpAccount pattern)
Bug fix:
- Add missing RetryableResponseError class to Provider::Questrade
(used in with_retries rescue clause but never defined — would cause
NameError on any rate-limited or 5xx response)
Logging:
- Replace Rails.logger.error with DebugLogEntry.capture in
QuestradeItem#import_latest_questrade_data, #process_accounts,
and #schedule_account_syncs to surface errors in the support UI
Consistency:
- Extract update_sync_status(sync, key, **i18n_options) helper in
QuestradeItem::Syncer, replacing 5 inline sync.update! guard calls
- Use blank? instead of ||= for default name fallback in create action
Tests:
- Add QuestradeItemsControllerTest (18 tests: CRUD, sync, account
linking/setup flows, admin guard enforcement)
- Add questrade fixtures: questrade_items.yml, questrade_accounts.yml
- Add retry/backoff tests to Provider::QuestradeTest (network error,
429, 5xx — all verify MAX_RETRIES exhaustion raises Error)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Signed-off-by: Jestin Palamuttam <34907800+jestinjoshi@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(invitations): allow re-inviting after an unaccepted invite expires
An expired, never-accepted invitation still occupies the partial unique
index `index_invitations_on_email_and_family_id_pending` (predicate
`accepted_at IS NULL`), but the `pending` scope excludes it via the
`expires_at > now` clause. The duplicate-guard validation therefore
passes and the INSERT collides with the index, raising
ActiveRecord::RecordNotUnique — surfaced to the admin as a 500 with no
way to re-invite that address through the UI.
Remove the stale expired row inside the create transaction
(before_create) so the unique slot is freed and the re-invite succeeds.
It runs only after validations pass, so a still-pending (non-expired)
invitation can never be removed here. Add a controller-level rescue of
ActiveRecord::RecordNotUnique as a safety net against a concurrent
double-submit race.
Closes#2535
* refactor(invitations): scope RecordNotUnique rescue to the save
The method-level `rescue ActiveRecord::RecordNotUnique` wrapped the whole
create action, so any future write after @invitation.save (e.g.
accept_for) would silently be reported as a generic invite failure.
Extract save_invitation to scope the rescue to the save alone, and add a
regression test that drives the raced unique-index path directly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(sync): don't persist zero balances when a provider balance fetch fails
A nil current_balance means the sync's balance fetch did not succeed
(the snapshot upsert clears it; only a successful fetch repopulates it).
The Lunchflow and Enable Banking processors coerced nil to 0 and write
it (plus a currency fallback) onto the linked account, so any transient
provider failure persists wrong data with no user-visible signal.
Instead, treat nil as no-data: skip the account update. Also stop the
Lunchflow snapshot upsert resetting an established account's currency to
USD when the accounts endpoint omits currency.
* fix(lunchflow): normalize the preserved currency in snapshot upsert
Addresses the review comment on #2617: the preserved fallback reused the
record's raw in-memory currency, so a blank value would fail the presence
validation and break import, and an invalid code would persist instead of
falling back to USD. Run it through parse_currency like the payload value.
Regression test added.
* fix(lunchflow,eb): review round 2 — failure visibility and currency parity
Addresses the three review findings on #2617:
1. Capture the Lunch Flow balance-fetch failure via DebugLogEntry (the sync
otherwise reports success with no mention of the skip).
This is Lunch Flow only: Enable Banking's importer already surfaces the
failure through transactions_failed/@sync_error.
2. Apply the currency-preservation fix to Enable Banking's snapshot upsert
(same shape as the Lunch Flow fix: parity/safety). Regression test added.
3. Label the Enable Banking processor currency assertion as a parity check —
it also passes on main, since the reset defect was Lunch Flow-specific.
Trend#percent divided the delta by the signed previous value, so a negative
base inverted the sign of the result and contradicted #direction. This showed
an up arrow next to a negative percentage (and vice versa) for anything that
can go below zero — most visibly net worth in reports and balance sparklines.
Divide by the magnitude of the base instead, and carry the sign of current
when the base is zero so an all-negative move reports -Infinity rather than
+Infinity.
Fixes#2579
test "totals aggregate directly from trade entries" builds a month-to-date
period (beginning_of_month..Date.current) but places a trade at
start_date + 1.day. On the 1st of the month the period collapses to a single
day, so that trade falls outside the range and withdrawals aggregate to 0,
failing the assertion (expected 40, got 0). The test passes every other day.
Use a fixed multi-day period so the test is date-independent.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(up): flag internal transfers and round-ups as funds_movement
Up populates relationships.transferAccount on transactions that move money
between the user's own accounts (including round-ups swept into a Saver), but
flatten_transaction dropped it, so these imported as ordinary income/expense
and distorted budgets and cashflow.
- Provider::Up#flatten_transaction: lift transfer_account_id from
relationships.transferAccount.data.id.
- UpEntry::Processor: import transfers as funds_movement and persist
transfer_account_id in extra["up"].
- Account::ProviderImportAdapter#import_transaction: optional kind: param; an
explicit provider kind takes precedence over account-type auto-detection and
is applied after the sync-protection check, so user re-categorisations
survive re-sync.
Complementary to Family#auto_match_transfers!: two-sided transfers between
linked accounts are still paired into a Transfer (the matcher does not filter
on kind); one-sided movements and round-ups, which the matcher cannot pair,
are the cases this fixes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(up): account-type kind wins over provider transfer hint
Codex review caught that Up HOME_LOAN accounts map to a Loan account, so a
repayment carrying transferAccount would be reclassified from loan_payment to
funds_movement (budget-excluded). Make the provider kind: a fallback:
activity-label and account-type classification now take precedence, so
loan_payment and cc_payment survive. Adds a regression test (loan repayment
stays loan_payment), a depository-applies test, and a note that the up_test
stub ignores query: intentionally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Gavin Matthews <matthews.gav@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(mercury): pending transactions, kind/counterpartyId metadata, and test coverage
- Transaction::PENDING_PROVIDERS: add "mercury" so the existing pending
reconciliation pipeline (pending→posted amount matching in
ProviderImportAdapter) activates for Mercury entries
- MercuryEntry::Processor: pass extra: to import_transaction with
extra["mercury"]["pending"] = true/false (status == "pending")
extra["mercury"]["kind"] = ACH / Wire / Card / etc.
extra["mercury"]["counterparty_id"] = Mercury counterparty UUID
Pending transactions are now imported with the flag set rather than
being silently ignored; failed transactions continue to be skipped
- Tests (new files, 30 cases):
- test/models/mercury_entry/processor_test.rb — sign convention,
date fallback, name priority, notes concat, pending flag, kind,
counterpartyId, failed skip, idempotency, merchant creation,
no-linked-account guard
- test/models/mercury_item/importer_test.rb — account discovery,
no-duplicate unlinked records, balance update on linked accounts,
transaction dedup (append-only new ids), sync window (90-day first
sync, last_synced_at-7d subsequent), 401 marks requires_update
- test/models/mercury_account/processor_test.rb — balance update,
CreditCard sign negation, cash_balance parity, no-linked-account
no-op, transaction processing delegation
* fix(mercury): address review findings — pending SQL, dedup upsert, N+1, nil assertion
- provider_import_adapter: add mercury to all three find_pending_transaction*
SQL predicates so Mercury pending entries are found and claimed when the
posted version arrives (exact, fuzzy, and low-confidence paths)
- mercury_item/importer: replace append-only dedup with an upsert-by-id
that replaces the stored raw payload when status changes from pending to
non-pending; prevents pending flag from persisting indefinitely when Mercury
reuses the same transaction ID for the posted version
- kraken_account/ledger_processor: preload all existing kraken ledger
external_ids into a Set before the loop; replaces per-entry exists? query
(N+1) with an in-memory Set#include? lookup
- test/models/mercury_account/processor_test: capture return value from
process and add assert_nil to enforce the nil contract stated in the test name
* fix(mercury): route transaction-count diagnostics through DebugLogEntry
* feat(kraken): fetch and import Ledgers API for deposits, withdrawals, staking, fees
Closes#2450
Kraken TradesHistory only returns spot buy/sell trades. The Ledgers API
(/0/private/Ledgers) covers deposits, withdrawals, staking rewards, Earn
income, and standalone fees — everything that was missing from syncs.
Changes:
- Provider::Kraken#get_ledgers — new method forwarding start/type/offset params
- KrakenItem::Importer#fetch_ledgers — paginated fetch (up to 200 pages) with
graceful fallback if the API key lacks Query Ledger Entries permission
- Importer#upsert_kraken_account — stores "ledgers" alongside "trades" in
raw_transactions_payload
- KrakenAccount::LedgerProcessor — new class; maps each supported ledger type
(deposit, withdrawal, staking, earn, fee) to a Transaction entry with the
correct investment_activity_label, kind, and sign convention; skips trade/
transfer/margin types to avoid double-counting with TradesHistory
- KrakenAccount::Processor#process — calls LedgerProcessor after process_trades
- Multi-currency: fiat amounts converted via ExchangeRate (non-USD fiat bridged
through USD); crypto amounts use the spot price cached in raw_payload["assets"]
with a price_missing flag when no price is available
- Dedup guard: external_id "kraken_ledger_<id>" + source "kraken" prevents
re-importing on repeated syncs
* fix(kraken): correct sign convention, fee inclusion, and earn subtype filtering
- Sign convention: deposits/staking/earn → negative (inflow), withdrawals/fees
→ positive (outflow), matching Sure's global convention (inflow is negative)
- Fee inclusion: use (amount - fee).abs as abs_impact so withdrawal fees are
counted in the total outflow rather than discarded
- Earn subtypes: skip allocation/deallocation ledger entries (internal fund
movements); only import rewardallocation/bonusallocation as Interest income
- DebugLogEntry: replace Rails.logger.warn/error with DebugLogEntry.capture
throughout LedgerProcessor and the Ledgers permission fallback in Importer,
so support-relevant incidents surface in /settings/debug
- Importer test: stub get_ledgers in setup so existing tests do not error
on the new fetch_ledgers call
* fix(kraken): route duplicate ledger-id warning through DebugLogEntry
* perf(kraken): batch ledger idempotency check; strengthen tests
Address review feedback (jjmata):
- N+1: LedgerProcessor#process_ledger_entry ran `account.entries.exists?(...)`
per ledger entry (up to ~10k per sync). Load the existing Kraken external IDs
once into a Set and test membership in memory (newly created IDs are added so
the same run stays idempotent) — same pattern as #2452.
- Tests: the idempotency test now asserts the first pass actually creates the
entry (assert_difference) before asserting the second is a no-op; add a guard
asserting the second (all-skipped) pass issues a single bulk external_id pluck,
not one query per entry.
No behavior change to imported entries.
* perf(kraken): scope ledger idempotency pluck to kraken_ledger_ prefix
Only load existing ledger external IDs (not trade entries) into the idempotency
Set, matching the reviewed approach. No behavior change.
* feat(goals): earmark a portion of an account toward a goal
Goals currently count each linked account's whole balance, so an account
shared across goals double-counts and one account can't fund several goals
in distinct slices. Add a per-account earmark — the "GoalBacking" the v1
model already foreshadowed (goal.rb).
- goal_accounts.allocated_amount (nullable). NULL = "dedicate the whole
balance" (the v1 default: no backfill, existing goals unchanged); a set
amount reserves a fixed slice.
- Goal#current_balance is now the single chokepoint computing each account's
backing under a family-wide shared pool: fixed earmarks take their slice,
an unallocated link takes the remainder, and when fixed earmarks exceed the
balance every slice is scaled down pro-rata so the goals' shares can never
sum past the account balance (no double-counting).
- Account#free_to_earmark / #goal_earmarked_total (mirror Budget's
available_to_allocate) back a soft, non-blocking over-allocation hint.
- GoalsController threads a goal[allocations] hash through create/update.
Phase 1 of the goals earmarking work; investment-backed goals follow.
* feat(goals): earmark UI on the goal form + backing-aware funding breakdown
- Goal form: a per-account "earmark amount" input (blank = whole balance)
next to each funding-account checkbox, prefilled from the saved
allocation on edit.
- Goal#account_backing exposes a single linked account's share so the
funding-accounts breakdown shows each account's earmarked contribution
and percent instead of its whole balance — keeping the show page
consistent with the (now allocation-aware) progress ring.
- English strings for the earmark controls and the "earmarked of balance"
breakdown line.
* fix(goals): address review on the earmark shared-pool math
- Overdrawn (<= 0 balance) accounts now back nothing on both the fixed and
whole-balance paths. The fixed path previously produced negative backing and
let a goal claim money the account doesn't hold.
- An archived goal reads its OWN earmark from its own goal_accounts instead of
the shared pool (which excludes archived goals), so it no longer mis-reports
the whole account balance for itself.
- goals#index injects one family-wide earmark pool into every card
(Goal.pooled_allocations_for) instead of querying once per goal (N+1), and
preloads goal_accounts.
- The projection chart scales its whole-account historical series by the
backing ratio so the saved line meets current_balance at "today" rather than
dropping off a cliff for earmarked goals.
- Honest comments: free_to_earmark no longer claims a form warning that doesn't
exist yet; pace documents its deliberate whole-account basis.
* fix(goals): widen the earmark input so the 'Whole balance' placeholder isn't clipped
* fix(goals): address review on #2490
- autosave: true on goal_accounts so earmark edits to already-linked accounts
persist through goal.save! (Rails only auto-saves newly built children, so
changing/clearing an existing earmark was silently dropped). + test.
- Reset the balance/progress memos on AASM transitions, not just the status
memos, so a same-instance render after complete!/archive! isn't stale. + test.
- backing_ratio is 0 (not 1) when the linked-account total is non-positive, so
the projection saved series ends at 0 to match the forced-zero current_balance.
- Localize the funding-row subtype label via goals.form.subtypes.*.
- Add the earmark strings to zh-CN (the maintained second locale; goals has no
ca locale, so Catalan keeps falling back to en like the rest of goals).
* feat(goals): investment-backed goals (Phase 2)
Goals can now be funded by investment accounts, not just depository.
- Relax linked_accounts_must_be_depository -> _must_be_fundable
(depository || investment); the funding picker + counts include investment
accounts.
- Add goals.progress_basis ('balance' | 'contributions', default 'balance').
Investment-backed goals default to 'contributions' so a market swing doesn't
move the goal: current_balance = value - cumulative market gain
(Sum of balances.net_market_flows); depository accounts have zero
net_market_flows, so they're unchanged. Goal#market_value_money shows what
it's worth today next to the contributed figure on the show page.
- Pledge false-match guard: investment accounts never use manual_save /
valuation-delta matching (a market move isn't a deposit) - they resolve on
transfer (cash-inflow) entries only. Guarded in both
Account#default_pledge_kind and GoalPledge#matches?.
- Add a `reopen` AASM event (completed -> active) + route/action/menu item so a
manually-completed goal whose value later dips can be reopened.
Stacked on the earmarking branch (#2490). Full suite green; +6 goal tests.
* fix(goals): address Phase 2 review — allocation-aware contributions, N+1, basis-on-update
- Contributions basis now goes through the same earmark/shared-pool logic as
the balance basis: backing_balance_for -> backing_share_for(account, base),
where base is the live balance (balance basis) or net contributions
(contributions basis). Earmarks are respected and shared accounts no longer
double-count on contributions goals; market_value_money stays consistent.
- Batch the per-account net_market_flows sum (Goal.market_flows_for) and inject
it on index like pooled_allocations, killing the N+1 for contributions goals.
- Default the basis on update too (not just create), so adding an investment
account to an existing depository goal flips it to contributions instead of
silently tracking market value.
- Fix the stale reconciliation_manager comment (renamed validation) and the
orphaned zh-CN must_be_depository key.
* fix(goals): address review on #2491
- before_save (not before_validation) for the progress_basis default, so a goal
can be inspected via valid? without its basis flipping as a side effect (jjmata).
- Pledge copy keys off default_pledge_kind, not manual?, so a manual investment
account — which pledges via transfer — shows the transfer prompt instead of the
"update your manual balance" flow (codex). pledge_action_label_key and the
pledge modal's per-account helper flag both use it.
- Add the Phase 2 strings (reopen success/invalid_transition, show.reopen,
ring.market_value) to zh-CN.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* feat(goals): earmark a portion of an account toward a goal
Goals currently count each linked account's whole balance, so an account
shared across goals double-counts and one account can't fund several goals
in distinct slices. Add a per-account earmark — the "GoalBacking" the v1
model already foreshadowed (goal.rb).
- goal_accounts.allocated_amount (nullable). NULL = "dedicate the whole
balance" (the v1 default: no backfill, existing goals unchanged); a set
amount reserves a fixed slice.
- Goal#current_balance is now the single chokepoint computing each account's
backing under a family-wide shared pool: fixed earmarks take their slice,
an unallocated link takes the remainder, and when fixed earmarks exceed the
balance every slice is scaled down pro-rata so the goals' shares can never
sum past the account balance (no double-counting).
- Account#free_to_earmark / #goal_earmarked_total (mirror Budget's
available_to_allocate) back a soft, non-blocking over-allocation hint.
- GoalsController threads a goal[allocations] hash through create/update.
Phase 1 of the goals earmarking work; investment-backed goals follow.
* feat(goals): earmark UI on the goal form + backing-aware funding breakdown
- Goal form: a per-account "earmark amount" input (blank = whole balance)
next to each funding-account checkbox, prefilled from the saved
allocation on edit.
- Goal#account_backing exposes a single linked account's share so the
funding-accounts breakdown shows each account's earmarked contribution
and percent instead of its whole balance — keeping the show page
consistent with the (now allocation-aware) progress ring.
- English strings for the earmark controls and the "earmarked of balance"
breakdown line.
* fix(goals): address review on the earmark shared-pool math
- Overdrawn (<= 0 balance) accounts now back nothing on both the fixed and
whole-balance paths. The fixed path previously produced negative backing and
let a goal claim money the account doesn't hold.
- An archived goal reads its OWN earmark from its own goal_accounts instead of
the shared pool (which excludes archived goals), so it no longer mis-reports
the whole account balance for itself.
- goals#index injects one family-wide earmark pool into every card
(Goal.pooled_allocations_for) instead of querying once per goal (N+1), and
preloads goal_accounts.
- The projection chart scales its whole-account historical series by the
backing ratio so the saved line meets current_balance at "today" rather than
dropping off a cliff for earmarked goals.
- Honest comments: free_to_earmark no longer claims a form warning that doesn't
exist yet; pace documents its deliberate whole-account basis.
* fix(goals): widen the earmark input so the 'Whole balance' placeholder isn't clipped
* fix(goals): address review on #2490
- autosave: true on goal_accounts so earmark edits to already-linked accounts
persist through goal.save! (Rails only auto-saves newly built children, so
changing/clearing an existing earmark was silently dropped). + test.
- Reset the balance/progress memos on AASM transitions, not just the status
memos, so a same-instance render after complete!/archive! isn't stale. + test.
- backing_ratio is 0 (not 1) when the linked-account total is non-positive, so
the projection saved series ends at 0 to match the forced-zero current_balance.
- Localize the funding-row subtype label via goals.form.subtypes.*.
- Add the earmark strings to zh-CN (the maintained second locale; goals has no
ca locale, so Catalan keeps falling back to en like the rest of goals).
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
#2356 build the accountable inside Account#subtype= so a top-level subtype
survives create. That fix assumes accountable_type is already set when
subtype= runs, but the real controller path violates it: strong-params
permit preserves filter order, and account_params lists :subtype before
:accountable_type. So on create subtype= runs while accountable_type (and
accountable_class) is still blank, the build is skipped, and the chosen
subtype is silently dropped — the account renders with the type's fallback
label (e.g. a Depository shows 'Cash' instead of 'Savings').
The existing regression test only covered the accountable_type-first order,
so it never caught this.
Make the writer order-independent: when subtype arrives before the type,
stash it and apply it from an accountable_type= override once the type is
known. Add regression tests for the permit order and for create_and_sync.
- Show principal-only transfer amounts on both sides with separate fee and total lines (fixes inconsistent gross/net convention)
- Derive displayed fee amounts from fee_transactions entries (single source of truth) instead of stored columns
- Remove stored source_fee_amount/destination_fee_amount columns from transfers table
- Add foreign key for transactions.transfer_id -> transfers.id (replaces invalid CHECK subquery)
- Move destination fee line inside destination side div for consistent layout
- Remove orphaned view_fee_transaction locale keys from 7 locale files
- Rebuild schema.rb from origin/main to eliminate unrelated column reordering churn
Entries now hold principal only (no fee baked into amounts). Fee transactions created as standard kind with Fees category. Transfer#amount_abs returns principal from new amount column. Update handler recomputes entries and fee transactions on edit. Remove dead source_principal/destination_principal helpers. Schema regenerated cleanly with only transfer fee columns.
* Fix QIF import for dd mmm yyyy dates (#2498)
American Express QIF exports use dd mmm yyyy D-fields (e.g. D26 Jan 2026).
Import previously failed with "Unable to detect date format" for two reasons:
1. QifParser.normalize_qif_date stripped all internal whitespace, turning
26 Jan 2026 into the unparseable 26Jan2026. For month-name dates the
space IS the separator, so collapse multiples to a single space instead
of removing them. Numeric dates keep the existing strip-all behavior.
2. Family::DATE_FORMATS had no candidate mapping to dd mmm yyyy, so
detect_date_format could not match it. Add "%d %b %Y" (DD MMM YYYY).
Extend the 2-digit-year expansion separator class to include space so
month-name dates with 2-digit years also normalize (26 Jan 26 -> 26 Jan 2026).
Covered by normalize/parse/detect and Amex-style row-generation regression
tests in qif_import_test.rb.
* test(qif): cover month-name 2-digit year normalization & parse
Addresses CodeRabbit nitpick on #2500: the production change extends
the 2-digit-year expansion regex separator class to include a space
so month-name dates like "26 Jan 26" normalize to "26 Jan 2026".
Add normalize_qif_date and parse_qif_date regressions for that branch.
* Add SnapTrade OAuth device flow
* Add SnapTrade OAuth device flow
### Motivation
- Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint.
- Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling.
### Description
- Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers.
- Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly.
- Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity.
- Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses.
- Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI.
- Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence.
### Testing
- Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed.
- Ran `bin/rubocop` against modified files and no offenses were reported.
* Add SnapTrade OAuth device flow
### Motivation
- Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint.
- Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling.
### Description
- Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers.
- Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly.
- Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity.
- Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses.
- Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI.
- Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence.
### Testing
- Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed.
- Ran `bin/rubocop` against modified files and no offenses were reported.
* Add SnapTrade OAuth device flow
### Motivation
- Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint.
- Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling.
### Description
- Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers.
- Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly.
- Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity.
- Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses.
- Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI.
- Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence.
### Testing
- Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed.
- Ran `bin/rubocop` against modified files and no offenses were reported.
* Add SnapTrade OAuth device flow
### Motivation
- Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint.
- Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling.
### Description
- Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers.
- Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly.
- Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity.
- Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses.
- Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI.
- Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence.
### Testing
- Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed.
- Ran `bin/rubocop` against modified files and no offenses were reported.
* Add SnapTrade OAuth device flow (provider, model, controller, routes, migration, tests)
### Motivation
- Add support for SnapTrade OAuth 2.0 device authorization flow so users can authorize SnapTrade via device codes in addition to the existing portal flow.
- Persist OAuth token metadata on `SnaptradeItem` for subsequent polling and usage by the provider and UI.
### Description
- Implemented OAuth device flow in the provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token`, plus HTTP helper methods `oauth_connection`, `oauth_client_id`, and `parse_oauth_response` in `Provider::Snaptrade`.
- Added controller endpoints `start_oauth_device_flow` and `complete_oauth_device_flow` in `SnaptradeItemsController` with robust error handling and helpers `oauth_error_payload` and `parse_oauth_error_body` to surface OAuth error fields.
- Extended `SnaptradeItem` with encrypted columns for `oauth_access_token`, `oauth_refresh_token`, token metadata, `oauth_token_active?`, and model methods `start_oauth_device_flow` and `complete_oauth_device_flow!` to store token metadata.
- Added migration `AddOauthDeviceFlowToSnaptradeItems` and updated `db/schema.rb` to include the new columns, registered a new initializer `config/initializers/snaptrade.rb` to read `SNAPTRADE_OAUTH_CLIENT_ID`, and updated `.env*.example` files to document `SNAPTRADE_OAUTH_CLIENT_ID`.
- Exposed new routes `start_oauth_device_flow` and `complete_oauth_device_flow` for `snaptrade_items`.
### Testing
- Added unit tests for provider OAuth behavior in `test/models/provider/snaptrade_oauth_test.rb`, model token persistence in `test/models/snaptrade_item_oauth_test.rb`, and controller error propagation in `test/controllers/snaptrade_items_controller_test.rb`.
- Ran the test suite with `bin/rails test` and the full test run (including the new SnapTrade OAuth tests) passed.
* Add SnapTrade OAuth device flow (start/poll), store tokens, and tests
### Motivation
- Add support for SnapTrade OAuth Device Authorization flow so administrators can start device authorization and poll for tokens without exposing provider internals.
- Persist OAuth token metadata on `SnaptradeItem` so the app can reuse and surface token state for SnapTrade integrations.
- Improve error handling and sanitization for OAuth API errors returned by SnapTrade to avoid leaking upstream internals.
### Description
- Introduces new environment examples and initializer: adds `SNAPTRADE_OAUTH_CLIENT_ID` to `.env.local.example`/`.env.test.example` and configures `Rails.configuration.x.snaptrade.oauth_client_id` in `config/initializers/snaptrade.rb`.
- Extends `Provider::Snaptrade` with OAuth device flow support, adding discovery URL, device grant constant, Faraday `oauth_connection`, and methods `oauth_authorization_server_metadata`, `start_device_authorization`, `poll_device_token`, `oauth_client_id`, and `parse_oauth_response`, plus improved retry and API error wrapping.
- Adds DB migration and schema changes to store OAuth fields on `snaptrade_items` (`oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, `oauth_token_expires_at`) and marks those attributes encrypted when ActiveRecord encryption is enabled.
- Adds `SnaptradeItem` helpers `start_oauth_device_flow`, `complete_oauth_device_flow!`, and `oauth_token_active?` to start/poll and persist token metadata.
- Adds controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` with sanitized error responses and helper methods `oauth_error_payload` and `parse_oauth_error_body`, and wires new member routes in `config/routes.rb`.
- Updates `SnaptradeItemsController` before_action lists to include the new actions and adds user-facing error message helper `start_oauth_device_flow_error_message`.
### Testing
- Added unit tests `test/models/provider/snaptrade_oauth_test.rb` to validate discovery, device authorization request, token polling, missing client ID handling, and OAuth error propagation, and all assertions passed.
- Added `test/models/snaptrade_item_oauth_test.rb` to assert `complete_oauth_device_flow!` stores tokens and expiry and that `oauth_token_active?` reports correctly, and the test passed.
- Extended `test/controllers/snaptrade_items_controller_test.rb` with controller-level tests confirming sanitized OAuth error payloads and behavior for start/complete endpoints, and these controller tests passed.
* Add SnapTrade OAuth device flow support and token storage
### Motivation
- Add support for the OAuth 2.0 device authorization flow for SnapTrade so users can link brokerages via a device-code flow without traditional browser-based client redirects.
- Persist OAuth token metadata on the SnaptradeItem so tokens can be reused and expiration tracked.
- Surface and sanitize provider error payloads to callers while avoiding leakage of internal configuration details.
### Description
- Added new DB columns and a migration (`oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, `oauth_token_expires_at`) and updated `db/schema.rb` to reflect the change.
- Encrypted the new token fields on `SnaptradeItem` and added `oauth_token_active?`, `start_oauth_device_flow`, and `complete_oauth_device_flow!` helpers to the model.
- Implemented OAuth logic in `Provider::Snaptrade` including discovery (`OAUTH_DISCOVERY_URL`), `start_device_authorization`, `poll_device_token`, request parsing, retry handling, and a small Faraday connection wrapper; added configuration accessors for `oauth_client_id` via `config.x.snaptrade`.
- Added controller endpoints `start_oauth_device_flow` and `complete_oauth_device_flow` with safe error handling and helpers to format OAuth error payloads, and registered new member routes for `snaptrade_items`.
- Added an initializer to load `SNAPTRADE_OAUTH_CLIENT_ID` into `Rails.configuration.x.snaptrade`, and updated `.env.local.example` and `.env.test.example` to include the new env var.
- Added unit tests for provider OAuth behavior (`test/models/provider/snaptrade_oauth_test.rb`), SnaptradeItem token persistence (`test/models/snaptrade_item_oauth_test.rb`), and controller error handling (`test/controllers/snaptrade_items_controller_test.rb`), and updated controller tests accordingly.
### Testing
- Ran provider-level tests in `Provider::SnaptradeOauthTest` to validate discovery, device authorization, token polling, and error handling, which passed.
- Ran model tests in `SnaptradeItemOauthTest` to verify token metadata is stored and `oauth_token_active?` works, which passed.
- Ran controller tests in `SnaptradeItemsControllerTest` that exercise the start/complete endpoints and error sanitization, which passed.
* Add SnapTrade OAuth device-flow support and token storage
### Motivation
- Add support for SnapTrade OAuth device authorization so users can perform device-code based OAuth without embedding secrets in the browser.
- Persist OAuth token metadata on `SnaptradeItem` and expose programmatic start/complete endpoints while avoiding leaking provider internals on errors.
- Make OAuth client ID configurable via environment or credentials for Sure deployments.
### Description
- Introduce `SNAPTRADE_OAUTH_CLIENT_ID` to `.env` examples and add `config.x.snaptrade.oauth_client_id` initializer for configuration.
- Add migration `AddOauthDeviceFlowToSnaptradeItems` and update `schema.rb` to add `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at` to `snaptrade_items`.
- Persist and encrypt new token fields in `SnaptradeItem` and add helper methods `oauth_token_active?`, `start_oauth_device_flow`, and `complete_oauth_device_flow!`.
- Extend `Provider::Snaptrade` with OAuth discovery, device authorization (`start_device_authorization`), token polling (`poll_device_token`), Faraday-based `oauth_connection`, parsing and error-wrapping logic, and retry handling.
- Add controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` to `SnaptradeItemsController`, plus helper methods to format OAuth error payloads and user-facing error messages.
- Wire up new routes (`post :start_oauth_device_flow`, `post :complete_oauth_device_flow`) and add `config/initializers/snaptrade.rb`.
- Add comprehensive tests for the OAuth flow and controller error handling and update a system test stub to avoid provider leakage during UI tests.
### Testing
- Added `Provider::SnaptradeOauthTest` which stubs discovery, device authorization and token endpoints and asserts request payloads and responses; tests passed.
- Added `SnaptradeItemOauthTest` which verifies token metadata persistence and `oauth_token_active?`; test passed.
- Extended `SnaptradeItemsControllerTest` with scenarios for successful and failing device-flow completion and start flow error handling; tests passed.
- Ran the affected system test changes in `TradesTest` (provider stubbing and modal behavior); the updated tests passed.
* fix(sync): store EnableBanking credit card debt as balance instead of available credit
Previously, the EnableBanking processor forcibly overrode the primary account balance for credit cards to be the available credit (credit_limit - debt) rather than the actual outstanding debt. Since credit cards are modeled as Liability accounts, this caused the balance sheet (net worth) to treat available credit mathematically as a debt.
This PR aligns the EnableBanking processor with the Plaid and SimpleFIN processors by storing the absolute debt as the account balance, while tracking the available credit via accountable metadata.
Fixes#2458
* docs(sync): update EnableBanking credit card processor documentation
Updates the inline processor comments to reflect the new behavior introduced by the previous commit, clarifying that outstanding debt is stored sequentially as the primary balance rather than the UX available credit overriding it.
* refactor(sync): clarify balance and available credit calculations in EnableBanking processor
Refactors the debt polarity assignments to clarify why liability balances are strictly parsed as absolute positive numbers. Replaces the implicit ordering dependency between the '.abs' conversion and the 'available_credit' math with an explicit 'outstanding_debt' variable to prevent regression by future maintainers.
* style: remove trailing whitespace in EnableBanking processor
Adds a toggle to mark accounts as excluded from all financial reports
while keeping them active and visible individually.
- Migration: add exclude_from_reports boolean column to accounts
- Model: included_in_reports scope
- BalanceSheet: filter excluded from ClassificationGroup and AccountGroup totals,
HistoricalAccountScope, AccountRow flag
- IncomeStatement: exclude via SQL fragments in Totals, FamilyStats, CategoryStats
- InvestmentStatement/Budget: chain included_in_reports scope
- ReportsController: filter in breakdown view, export queries, trades
- AccountsController: toggle_exclude_from_reports action + route
- UI: DS::Toggle in account form, eye-off indicator in sidebar, menu toggle items
- Locale: all labels in en.yml
- Tests: model scope, controller toggle, income statement/balance sheet filtering
- 5070 tests pass (0 failures, 0 regressions)
* feat(api): allow multiple active API keys per user
Previously a user could hold only one active API key; creating a new one
silently revoked the existing key, breaking any app using it. Allow
multiple named active keys instead.
- Drop the one-active-key-per-source validation; add name uniqueness
among the user's active visible keys (revoked names are reusable,
same name allowed across users).
- Rewrite Settings::ApiKeysController as a RESTful collection
(index/show/new/create/destroy); create no longer revokes existing
keys, and key lookup is scoped to the current user's active keys.
- resource :api_key -> resources :api_keys.
- Add an API keys list view with per-key revoke and an empty state;
rewrite the show page for a single key.
- Update i18n (remove single-key copy, pluralise nav label) and tests.
* refactor(api): address review on multiple API keys
- Remove unreachable destroy branches (cannot_revoke is guarded by the
.visible 404; revoke! raises rather than returning false) and document
that .visible is the demo-key revocation guard.
- Delete the orphaned created.html.erb / created.turbo_stream.erb
templates (no action renders them) and their unused locale keys.
- Extract shared partials (_scope_badges, _status_indicator, _key_meta,
_key_reveal, _usage) to de-duplicate the index and show views; unify
the active-status indicator on the standard dot.
- Carry forward the @container / @lg:flex-row / min-w-0 responsive
fixes from #2079 into the shared key-reveal partial.
* test(api): cover newly-created API key confirmation render
* refactor(api): harden demo-key guard and address review nits
- Document the demo-key revocation guard on ApiKey's `visible` scope (the
authoritative spot) and add a model test locking the invariant that
`.visible` excludes the demo monitoring key.
- _scope_badges: use an i18n lookup with a humanize fallback instead of
bypassing translation for unknown scopes.
- _key_reveal: drop the hard-coded `id` from the shared partial; the
system test now locates the key via its data-clipboard-target.
* refactor(api-keys): migrate hand-rolled badges to DS::Pill
Replace raw span elements in scope badges and status indicator
with DS::Pill to align with the design system migration convention.
* fix(loans): opening anchor now uses current balance, not original principal
When creating a loan manually, the opening anchor valuation was being
set to `initial_balance` (the original loan principal) instead of
`account.balance` (the current outstanding balance). After the sync
job ran, `account.balance` was overwritten to match the anchor,
making every manually-created loan show its original principal as
the current balance.
Fix by always using `account.balance` for the opening anchor in
`create_and_sync`, and reading `Loan#original_balance` from the
`loans.initial_balance` column directly (with a fallback to
`first_valuation_amount` for provider-synced loans that may not
have the column populated).
* fix(api-keys): strip accidental loan changes; rescue revoke! failures
The fix(loans) commit was accidentally committed into this branch.
Remove the loan-related changes from account.rb, loan.rb, and both
test files, restoring them to their pre-loan-commit state.
Also fix the destroy action: revoke! uses update! internally which
raises ActiveRecord::RecordInvalid on failure rather than returning
false. Add rescue for RecordInvalid and RecordNotDestroyed so failures
produce a flash alert instead of a 500. Re-adds the revoke_failed
locale key that was dropped from settings.api_keys.destroy.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* fix: use ES256 instead of EdDSA for Coinbase CDP JWT signing
Coinbase Developer Platform issues EC P-256 keys, not Ed25519.
The previous EdDSA implementation would always return Unauthorized
for any key generated via portal.cdp.coinbase.com.
Switch to ES256 (ECDSA SHA-256) using OpenSSL, and convert the
DER-encoded signature to the raw r||s format required by the JWT spec.
API Secret field now accepts the full PEM private key directly
(including BEGIN/END headers) as provided by Coinbase.
* fix(coinbase): address PR review comments
- Normalize escaped \n in PEM key before parsing (fixes pasting directly
from the Coinbase CDP JSON download file where newlines are \n literals)
- Extract parse_ec_private_key helper with docstring explaining both
accepted PEM formats
- Fix double-space style nit on encoded_header assignment
- Remove ed25519 gem from Gemfile (no longer used after ES256 migration)
- Add Provider::CoinbaseTest covering: JWT 3-part structure, ES256 alg
header, kid claim, CDP payload claims, 64-byte raw r||s signature,
escaped-newline key acceptance, and cryptographic signature verification
* test(coinbase): fix base64url padding in JWT test assertions
* chore: update Gemfile.lock to remove ed25519 gem
* Sample credential in test
* Comments field, not real key use
* test(coinbase): generate EC key dynamically; drop scanner exclusions
Generate a fresh P-256 key per test run with OpenSSL::PKey::EC.generate
instead of hardcoding a PEM private key. This removes committed key
material and the need to exclude the provider and test files from the
pipelock secret scan, so those exclusions are removed too.
Addresses review findings on hardcoded test key + scan exclusions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chat): surface and recover from undelivered assistant responses
When the background worker that runs AssistantResponseJob is down or not
polling the high_priority queue, the eager "pending" assistant message had no
safeguard: chat hung on "Thinking…" forever with no error, timeout, or retry,
and the LLM provider was never even called.
Add three layers of resilience:
- Client watchdog (chat_controller.js): a pending "Thinking…" bubble that waits
past a threshold (default 90s) with no response asks the server to fail it.
Keyed on the pending marker — which disappears the instant a real response
streams — so a slow-but-working response is never falsely timed out.
- Server failure capture (Chat#handle_undelivered_response! +
MessagesController#report_timeout): clears the dead bubble, records a friendly
"the assistant didn't respond" error with Retry, and writes a DebugLogEntry so
support can see it in /settings/debug.
- Worker liveness signal (BackgroundJobHealth + warning banner): reads Sidekiq's
process/queue state directly from Redis in the web process — so a down worker
is detectable even though the worker is the thing that's broken — and warns
self-hosted users when no worker polls high_priority or it's badly backed up.
Fails open so a Sidekiq/Redis blip never blocks chat.
Tests: Chat model (3), MessagesController (2), BackgroundJobHealth (5).
* fix(chat): address review — server-side timeout guard + watchdog retry safety
- Chat#handle_undelivered_response!: gate the state change behind a row lock and
a server-side minimum age (UNDELIVERED_RESPONSE_TIMEOUT = 60s). The browser
watchdog is untrusted, so re-read the row under lock and only fail a bubble
that is still pending AND has genuinely waited past the timeout — never racing
a worker that is finishing a slow response. (Codex P2 + CodeRabbit)
- chat_controller.js: only mark a report URL as reported on response.ok (fetch
resolves on HTTP 4xx/5xx, rejecting only on network errors), and guard
concurrent duplicate POSTs with an in-flight set, so a failed report retries
instead of stranding the bubble. (CodeRabbit)
- _worker_health_warning: drop the unused `chat:` local + its render arg. (CodeRabbit)
* feat(mcp): add get/create/update tools for tags and categories
* test(mcp): add tests for tag and category assistant functions
* fix(mcp): harden category/tag assistant functions against bad tool input
* refactor(mcp): use UuidFormat.valid? instead of local UUID_PATTERN
* chore(mcp): remove account_filter component files that don't belong in this branch
