* feat(jobs): reap imports and exports stuck by lost background jobs
A hard-killed worker (OOM, SIGKILL during deploy) loses its in-flight
Sidekiq job permanently. Sync is the only model with a stale sweep;
everything else wedges in a non-terminal status forever:
- Import stuck "importing"/"reverting" (e.g. #2274)
- ImportSession stuck "importing" — unrecoverable, publish_later
refuses to re-publish while importing
- FamilyExport stuck "pending"/"processing" — exports index polls
every 3s indefinitely
- PdfImport's AI-processing claim held forever (ProcessPdfJob's
reclaim only runs if the job is redelivered)
- Provider activities_fetch_pending flags stranded when a
self-rescheduling fetch chain loses a link
SyncCleanerJob (existing hourly cron) now sweeps all of them, each
isolated so one failure doesn't block the rest, and records every
reaped record as a DebugLogEntry with the family attached.
Idempotency guards so a stray or redelivered job cannot corrupt a
record the reaper (or a user retry) has since moved on:
- Import#publish skips complete/reverting/revert_failed imports —
a redelivered ImportJob after completion double-applies data for
import types without row dedup. Failed imports deliberately stay
publishable: their transaction rolled back, so a re-run is a retry.
- FamilyDataExportJob refuses terminal exports but still allows
processing ones through, since graceful-shutdown redelivery is what
completes them.
Reaper thresholds (6h imports, 2h exports) key off updated_at and
dwarf legitimate runtimes, so live jobs are not swept in practice.
* fix(jobs): make the reapers race-safe and commit-aware
Review feedback on #2681 (jjmata, Codex, CodeRabbit):
- Every sweep now mutates under record.with_lock with a staleness
re-check, mirroring the guard Sync#perform gained in #2680 — a job
finishing between the sweep query and the write can no longer be
clobbered mid-flight
- Import.clean distinguishes which side of import!'s single transaction
the worker died on: rows attached means the data committed and only
the status write was lost, so the record is finalized as complete
(marking it failed invited a re-publish that double-imports types
without row dedup, e.g. TradeImport); no rows means a clean rollback
and the failed/try-again path stays
- PdfImport.clean applies the same split: no rows → the AI-extraction
claim died, reclaim to pending; rows → the publish died post-commit,
finalize complete instead of letting the same extracted rows be
published twice. Stuck PdfImport reverts (previously unswept by
either clean) now go to revert_failed like every other import
- ImportSession.clean reconciles chunks whose import! committed but
never got the complete-status write before failing the session, so
re-publish skips them instead of duplicating their rows via
SureImport's split path
- Import#publish redelivery skip is captured via DebugLogEntry instead
of Rails.logger so support can see it in /settings/debug
- The interrupted-error copy is i18n-backed (imports.errors.interrupted)
* fix(jobs): round-2 review feedback on the reapers
- Import.clean excludes session-owned chunks (import_session_id: nil) —
SyncCleanerJob runs it before ImportSession.clean, so it could
finalize or fail a SureImport chunk outside the session flow that
owns its lifecycle. Regression test added (CodeRabbit major)
- family.sync_later moved outside the row-lock transaction in both
reap paths — Rails doesn't defer enqueues to after-commit by default,
so Sidekiq could pick the job up before the status write was visible
- Per-record rescue in Import.clean / PdfImport.clean so one bad record
doesn't abort the rest of the hourly sweep
- ImportSession.clean uses the importing? enum predicate;
reconcile_committed_chunks! iterates with each (default-ordered
association made find_each warn, and chunk counts are tiny)
* fix(jobs): address round-3 reaper review (isolation + commit signal)
Per-record rescue isolation for the three sweeps that lacked it —
FamilyExport.clean, ImportSession.clean, and SyncCleanerJob's
activity-flag loop — mirroring Import.clean/PdfImport.clean. One bad
record (validation error, DB blip) no longer aborts the rest of that
sweep for the hour; the activity-flag guard is per-record so a failure
also stops skipping the models that follow.
data_committed? now covers Category/Rule/Merchant imports, whose
records hang off the family rather than the import (no entries or
accounts), via the new Import#committed_by_named_records? helper. A
committed one is reaped to complete instead of a retryable failed;
nameless RuleImport rows carry no stable key, so a nameless-only file
has no commit signal and stays retryable.
* fix(schema): drop duplicate enable_banking_accounts columns from merge
The merge of main into this branch re-appended product, credit_limit,
and identification_hashes after updated_at, so schema.rb declared each
twice and db:schema:load raised "you can't define an already defined
column 'product'". Removed the duplicate declarations (kept the new
treat_balance_as_available_credit column); the test database loads
again.
* Add "Money In / Out" dashboard widget
Adds a new dashboard section showing a monthly bar chart of cash
activity alongside a summary card (net balance, income, expenses),
with per-widget month navigation and account filtering.
- IncomeStatement#totals_for computes income/expense totals for an
arbitrary period, optionally scoped to a set of account ids
- New bar_chart_controller.js (D3) renders the monthly bars
- Income/expense rows link through to filtered transactions
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Fix tooltip/dropdown positioning and split money flow chart by income/expense
The widget's @container wrapper established a new containing block for
position:fixed descendants, so the month-picker and account-filter
menus (floating-ui, strategy: fixed) and the D3 tooltip (container-
relative coordinates) rendered away from their trigger/bar. Drop
@container in favor of regular viewport breakpoints for this
full-width widget, and position the tooltip with page-relative
coordinates like the other chart controllers.
Also replace the single combined bar per month with a grouped
expense/income pair (red/green, with a legend) so each month's
inflow and outflow are visible independently.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Cap and scroll the money flow widget's month picker
Add an optional max_height to DS::Menu (opt-in, backward compatible)
so a long item list scrolls inside a fixed-height panel instead of
overflowing the viewport. Use it for the money flow widget's 12-month
picker.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* Use design system tokens for income/expense indicators in money flow widget
Swap the four bg-green-500/bg-red-500 dot indicators (legend + income/expense row links) in the money flow widget for bg-success/bg-destructive, matching the "functional tokens only" rule. The same partial already uses
text-success/text-destructive for the balance figure, and bg-success/ bg-destructive are already established elsewhere (DS::Alert, budget_categories/_budget_category.html.erb).
* Fix future-month 500 and pending-transaction link mismatch in money flow widget
Two CI review findings on the money flow widget:
- A future month passed via ?money_flow_month= (e.g. a bookmarked/hand-edited URL) made end_date earlier than month_start once capped at Date.current, which Period.custom rejects, causing a 500. money_flow_month_param now clamps future months to the current month, same as it already does for malformed input.
- The income/expense row links passed type/date/account filters but no status, so Transaction::Search included pending transactions even though the displayed totals (IncomeStatement#totals_for) exclude them via excluding_pending. Add status: ["confirmed"] to both links so the linked list matches the card total.
Also strengthens the widget's controller tests: asserts the highlighted bar's actual income/expense values instead of just its presence, and adds a regression test proving an account id outside the current user's accessible accounts is dropped (falls back to the unfiltered state) rather than leaking or erroring.
* Fix account filter eligibility and SVG dark mode fill in money flow widget
Two more CI review findings on the money flow widget:
The account filter iterated over all visible/accessible accounts, a broader set than IncomeStatement actually counts (accounts excluded from reports, tax-advantaged accounts like 401k/IRA, or shared accounts not included in the user's finances). Selecting one of these silently computed to zero while its drill-down link could still list its transactions. Add IncomeStatement#eligible_accounts, mirroring the same criteria already applied in the totals SQL, and use it for both the checkbox list and the account_ids intersection.
The D3 axis tick text elements used text-primary/text-secondary, which set CSS color, not SVG fill, so labels rendered with the default black fill and were unreadable in dark mode. Add fill-current, matching the pattern already used in sankey_chart_controller.js.
Adds controller/model tests for eligible_accounts (excluded from the account filter, ignored when passed as a filter id, excluded from totals) and verifies the dark-mode fill fix visually.
* Extract duplicated bar-chart JSON parsing into a test helper
The css_select("[data-controller='bar-chart']").first + JSON.parse(chart["data-bar-chart-data-value"]) pattern was repeated across four money flow widget tests in pages_controller_test.rb. Extract it into a private money_flow_bars helper and use it everywhere instead.
* Preserve eligible account scope in money flow drill-down links when unfiltered
The income/expense drill-down links used money_flow_data[:account_ids] directly, which is nil in the widget's default unfiltered state, so .compact dropped the account filter entirely from the link. TransactionsController treats an absent account_ids as all accessible accounts, a broader set than IncomeStatement#eligible_accounts (which excludes tax-advantaged, excluded-from-reports, and non-finance shared accounts). Users with any such account could click a displayed total and see transactions that were never counted in it.
Use selected_account_ids (already computed as money_flow_data[:account_ids] || accounts.map(&:id)) for both links instead, so they always pin to the same eligible accounts backing the total, filtered or not. Left the month-picker link on money_flow_data[:account_ids] so navigating months while unfiltered doesn't bloat the URL with every eligible account id.
Adds a regression test confirming the default (unfiltered) links include account_ids and exclude an ineligible account's id.
* Refine Money In/Out dashboard widget
- 6-month window (was 3), half-width default with responsive stack, income-first
bars, 2px floor + faded in-progress (partial) month
- Expense series/figures neutral (gray/text-primary) — app reserves red for
negative/overspend; neutral zero balance
- Account filter: outline DS::Button + list-filter icon trigger with in-panel
search (DS::SearchInput + list-filter), matching the app's filter convention
- i18n search_accounts (en + fr); bump money_flow bar-count test 3 -> 6
* Clarify Money In/Out scope: month label, 6-month caption, filter "All"
Addresses confusion between the widget's own month picker and the dashboard's
global period, and between the picked month and the 6-month chart span.
- Card now headed with the selected month (e.g. "July 2026") so it's clear the
totals below are that month's, driven by the picker
- Chart legend row captioned "Last N months" so the trailing window is explicit
- Info tooltip by the month picker: the widget scopes to the month you pick
here, independent of the dashboard's top-level period
- Account filter trigger reads "All accounts" when nothing is filtered out,
instead of "Filter accounts (N)"
- i18n (en + fr) for the new strings
* Match tooltip expense dot to the gray bar palette
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Guillem Arias <accounts@gariasf.com>
* Add support for trading212 integration in investments sync
* Respect Trading 212 history rate limits
* Replace hex colors with design tokens
* Changed withdrawal to withdraw
* Address PR review feedback for Trading 212 integration
- Replace all Rails.logger calls with DebugLogEntry.capture across
syncer, importer, processor, provider, controller, and unlinking
- Fix destroy action to surface unlink errors via DebugLogEntry
- Replace hardcoded 'GBP' currency fallback with Current.family.currency
- Replace hardcoded English status strings in Syncer with i18n
- Add config/initializers/trading212.rb with DEBUG_RAW ENV toggle
- Add comprehensive test coverage: item, account, data_helpers,
holdings_processor, activities_processor, importer, syncer,
and controller tests
- Rewrite syncer and controller tests to match actual interfaces
* Fix tests and bugs found during local test execution
- Fix standard_ticker: empty string caused nil.upcase via [].first
- Fix parse_date: DateTime < Date, so when Date caught DateTime first
- Fix ActivitiesProcessor/HoldingsProcessor: missing instruments_map
in DataHelpers caused NameError when processing dividends
- Fix test: security fixture ticker collision with test data
- Fix test: sync_status_summary i18n matching in assertions
- Fix test: trading212_provider ConfigurationError test path
- Fix test: controller invalid params needs Turbo-Frame header
- Fix test: syncer test uses stubs instead of strict expects
116 tests, 257 assertions, 0 failures, 0 errors
* Keep raw provider response bodies out of exception messages.
* Fix Secrets leak into page HTML, prevents the API key/secret from appearing in the HTML source while keeping the "leave blank to keep existing" UX.
* Address review findings: env gate, sync test, uniqueness test, destroy flow
- Gate TRADING212_DEBUG_RAW behind Rails.env.local? so staging/production
cannot accidentally enable raw payload logging
- Assert SyncJob enqueue in sync controller test, not just redirect
- Fix cross-item uniqueness test to actually use two different items
- Remove rescue in destroy so unlink failures stop the flow (matching
Brex/Akahu pattern) instead of proceeding to destroy_later silently
* Add Trading212 tables to db/schema.rb for CI test database
CI runs db:test:prepare which loads db/schema.rb, not migrations.
Without these table definitions, fixture loading fails with
PG::UndefinedTable: relation "trading212_accounts" does not exist.
* Removed lint complaint
* Register Trading212 in ProviderConnectionStatus::PROVIDERS
Fixes CI failure: test_provider_registry_covers_syncable_family_provider_item_associations
* Fixed bad merge
---------
Signed-off-by: jdcdp <47483528+jdcdp@users.noreply.github.com>
Co-authored-by: jdcdp <jdcdp@cdm4.net>
* Add RentCast and Realie AVM providers for property valuation
Adds Automated Valuation Model (AVM) provider support so self-hosted
users can create property accounts from a US address lookup instead of
entering details manually:
- Provider::Rentcast and Provider::Realie clients, each fetching the
property record (type, year built, square footage) and value estimate
in a single API request, registered under a new :property_valuations
registry concept
- API key fields (encrypted, ENV-overridable) with monthly usage display
in Settings > Self-Hosting under "Property Valuation Providers"
- New property flow: when a provider key is configured, the method
selector offers "Add via RentCast/Realie" alongside manual entry;
the lookup form reuses the manual flow's localized address fields
and creates the account active with fetched attributes, valuation as
balance, and the address saved
- SyncPropertyValuationsJob refreshes linked property valuations once a
day (never hourly) via config/schedule.yml; each provider enforces its
monthly request cap (RentCast 50, Realie 25) with a calendar-month
counter shared between creation lookups and refreshes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Allow ENV override of AVM provider monthly request caps
The RentCast (50/month) and Realie (25/month) budgets default to the
free tier limits but can now be raised for paid plans via
RENTCAST_MAX_REQUESTS_PER_MONTH / REALIE_MAX_REQUESTS_PER_MONTH,
mirroring the AlphaVantage and Tiingo request limit overrides. The
settings descriptions and usage display reflect the effective cap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address review feedback: durable request counters, safer refresh job
- Move monthly AVM request counters from Rails.cache to a new
provider_request_counts table with atomic upsert increments, so the
hard budget caps survive cache eviction, restarts, and Redis flushes
- SyncPropertyValuationsJob: only mark a property synced when the
balance update succeeds (wrapped in a transaction), process stalest
valuations first so a tight budget goes where it matters, and report
failures via DebugLogEntry.capture instead of Rails.logger
- Realie: reject lookups whose returned city/ZIP contradict the entered
address (street+state queries can match the wrong city); document that
numeric use codes are unpublished and leave the subtype unset
- Add Faraday open/request timeouts to both provider clients
- Localize all user-facing provider error messages
(config/locales/models/provider/en.yml)
- Add a check constraint restricting properties.avm_provider to known
providers
- Use the min-h-80 scale token instead of min-h-[320px]
- Tests: exact-argument expectations on the lookup stub, provider stubs
in the no-provider test, RentCast/Realie API key update tests,
ProviderRequestCount unit tests, failed-balance regression test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Stub AVM provider registry lookups in settings system test
Settings::HostingsController#show now resolves the RentCast and Realie
providers for usage display; the system test's partial get_provider
stubbing treats the new lookups as unexpected invocations without these.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Address second-round review: candidate scan, provider reuse, shared row partial
- Realie: when the address lookup returns multiple candidates, pick the
first one consistent with the entered city/ZIP instead of judging only
the first array element; the location-mismatch error now only fires
when no candidate matches
- SyncPropertyValuationsJob: resolve one provider instance per key for
the whole run, so the per-instance request throttle actually spaces
requests across properties instead of resetting on each iteration
- Extract the AVM method selector row into a shared partial so the
manual and provider options render one shape
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add retry middleware and sync index from review feedback
- Both AVM clients now retry transient connection failures (same
Faraday retry config as Tiingo) so a network blip doesn't burn one of
the tight monthly budget's requests
- Partial index on properties(avm_provider, avm_last_synced_on) backing
the daily sync job's filter and stalest-first ordering
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Key the AVM sync index on avm_last_synced_on
The daily job orders by avm_last_synced_on ASC NULLS FIRST across all
AVM-linked properties, so leading the partial index with avm_provider
prevented it from serving the sort. Rekeyed on avm_last_synced_on with
matching null ordering; the partial predicate still covers the filter.
Amended in place since the migration is unmerged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Map Realie numeric use codes; skip incomplete addresses in sync job
- Realie documents its numeric use codes at
docs.realie.ai/api-reference/feature-key (earlier skip reason was
wrong — the table exists). Map the residential/land/agricultural codes
to Property subtypes, leaving codes without a subtype equivalent
(e.g. 1006 mobile/manufactured) unset like the RentCast mapping.
- The daily job now skips properties whose address is missing a street
or state before resolving a provider, logging a warn-level
DebugLogEntry, so a malformed address can't burn a monthly-budget
request every day.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Guard currency mismatches in refresh; reset AVM keys in test teardown
- The daily job now skips (with a warn-level DebugLogEntry) properties
whose account currency no longer matches the provider's valuation
currency, checked via the concept's valuation_currency before spending
a request — writing a USD valuation into a re-currencied account would
corrupt the balance
- Hostings controller test teardown now clears rentcast_api_key and
realie_api_key alongside the other cached global settings
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Validate AVM lookup inputs before spending a provider request
The form marks name and address fields required, but a forged or
JS-less submission bypasses that and would burn one of the tight
monthly-budget requests on a lookup that can't produce a property.
Property::AvmImport now validates name and the full address locally
(localized error) before calling the provider.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add preview-and-confirm step to the AVM property lookup
RentCast's AVM endpoint is location-based: a plausible but nonexistent
address still geocodes and returns an area-derived estimate with no
property record behind it. Instead of silently creating an account from
that, the lookup now shows what the provider returned — type, year
built, area (each "Unknown" when missing), and the estimated value —
with an explicit notice when no property record was found, and only the
user's confirmation creates the account.
The confirm step reuses the fetched data via the form (sanitized
server-side), so the flow still costs exactly one provider request.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Require a complete US address before daily valuation refresh
The job's guard only required street and state, but a later-blanked
city or ZIP silently disables the Realie wrong-city check (blank
entered fields count as "no mismatch"), so a refresh could accept
another property's valuation. The guard now mirrors the import-time
completeness check and also skips addresses edited to a non-US country.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Sign the AVM preview payload; blur valuation in privacy mode
- The confirm step now rebuilds everything from a signed, 1-hour
message-verifier token generated at lookup time, proving the lookup
(and its counted provider request) actually ran — a direct confirm
POST with fabricated data can no longer create provider-linked
properties that the daily refresh job would then spend quota on.
Forged/expired tokens re-render the lookup form with a clear error.
- The preview's market value now carries the privacy-sensitive class so
privacy mode blurs it like other account amounts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Treat zero Realie valuations as absent; bind preview token to family
Review follow-ups from PR #2727:
- Realie returns modelValue: 0 when it can't produce an AVM estimate;
zero now falls back to the assessed market value instead of syncing
a $0 balance, and an all-zero record errors out.
- Monetary values now parse via BigDecimal(value.to_s) rather than
Float#to_d, avoiding binary float rounding artifacts in balances.
- The signed AVM preview token's purpose is scoped to the family that
ran the lookup, so a leaked token can't be replayed cross-family.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Refresh AVM property valuations monthly instead of daily
Per @jjmata's review on #2727: valuations change slowly and providers
enforce tight monthly request caps, so a daily cron adds no value.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Api::V1::HoldingsController built both the index and show queries from
current_resource_owner.family.holdings, filtering only on account status.
A read-scoped API key could therefore read holdings (account id/name,
quantity, price, market value) for accounts owned by other family members
that were never shared with the key owner, and the account_id / account_ids
filters made targeted enumeration of those accounts trivial.
Scope the query through accounts.accessible_by(current_resource_owner) so
results are limited to accounts the token owner owns or has been granted
access to. Route both index and set_holding through the new
accessible_holdings scope. This matches Api::V1::BalancesController and
the web HoldingsController, which already scope the same way.
Fixes#2467
Co-authored-by: agentloop <agentloop@localhost>
* fix Prism issue - assigned but unused variable plaid_item
* measurement to improve accounts_controller__index performance based on skylight
* fix FEEDBACK - Stop eager-loading full sync histories on accounts index
* fix FEEDBACK - Remove sticky memoization from SimplefinItem#accounts
* fix lint error in PR review
* fix FEEDBACK - Avoid returning map misses as authoritative results
* fix test error
* fix FEEDBACK - Avoid redundant query by using already-loaded @manual_accounts
* Address maintainer review on accounts index sync preloading
Memoize SimplefinItem#accounts, align syncing? with Sync#visible?,
and add fallback regression tests.
* Add composite index for sync DISTINCT ON queries
Support latest_by_syncable ordering with syncable_type, syncable_id,
created_at DESC, and id DESC.
* fix error in dockerfile.preview file
* Suppress Pipelock false positives on CI database fixtures.
Pipelock 2.7.0 full-repo audit flags ephemeral postgres:// URLs in
workflow env blocks. Re-apply inline suppressions lost in the main merge.
* Address review: guard partial Current sync maps and drop unrelated diff
Add key? checks so partially populated Current sync maps fall back to DB
queries. Revert Dockerfile.preview and pipelock.yml changes unrelated to
the accounts index N+1 optimization.
* fix(sync): fully populate Current sync maps for all preloaded syncables
* fix(ci): skip scheduled preview cleanup on forks
Only run the hourly Cloudflare preview cleanup on we-promise/sure,
where the required secrets exist.
* Remove schema.rb Postgres-version churn, keep only new syncs index
Reset db/schema.rb to upstream/main and re-add only the
index_syncs_on_syncable_and_created_at_and_id index. The prior diff
included ~30 lines of noise (check-constraint reformatting, virtual
column reformat, and column reorderings) caused by dumping under a
different PostgreSQL version, which obscured the single intended
schema change and invited merge conflicts.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(snaptrade): replace device-flow OAuth with authorization-code + PKCE flow
Squashed from 16 commits on snaptrade-oauth-apps for a clean rebase onto
current upstream/main ahead of opening a PR.
* fix(snaptrade): address PR #2747 review feedback on OAuth PKCE flow
- Remove unreachable dead-code guard in import_latest_snaptrade_data
- Guard apply_oauth_tokens! against a malformed payload missing access_token
- Wrap token endpoint network errors in ApiError and retry like data calls
- Remove unused Provider::Snaptrade#revoke_token! instance method
- Preserve return_to/accountable_type through the SnapTrade portal callback
so the account-linking flow no longer drops users back to accounts_path
- Show the real absolute OAuth callback URL in self-hosted setup instructions
- Refresh brakeman.ignore fingerprint for the connect redirect after the
return_to/accountable_type params were added
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y8SCCmKX6RphB5E73WSUQQ
* fix(snaptrade): don't retry non-idempotent OAuth/API requests
CodeRabbit flagged that Provider::Snaptrade retried OAuth token
exchanges/refreshes and all API POST/DELETE calls (get_connection_url,
delete_connection) after timeouts/connection failures. If the response
is lost after SnapTrade already consumed a single-use auth code,
rotated the refresh token, or applied a POST/DELETE, replaying the
request either fails with invalid_grant on a token that actually
succeeded, or risks duplicate side effects. Retries are now limited to
GET requests; OAuth token requests and non-GET API calls translate a
network failure straight into an ApiError without replay.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NrrGkgSBEqhjjBmmH1fcXL
* fix(snaptrade): stop querying non-deterministically encrypted token via empty-string compare
CodeRabbit flagged that the syncable scope's where.not(oauth_access_token:
[nil, ""]) re-encrypts "" with a random IV on every query, so the ""
comparison can never match a stored ciphertext and is a silent no-op.
No code path ever persists oauth_access_token as "" (only nil or a real
token via apply_oauth_tokens!), so the exclusion is unnecessary --
narrowed the scope to a plain NULL check, which encryption handles
transparently since nil is never encrypted.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NrrGkgSBEqhjjBmmH1fcXL
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* Prevent Yahoo crumb cache poisoning
* Make Yahoo Finance health status rate-limit aware
* Normalize Yahoo Colombian listings
Map Yahoo's BVC code to XBOG so search results derive Colombia from canonical exchange metadata and price requests use the .CL suffix with a COP fallback.
Spec: .scratch/normalize-yahoo-colombia-listings/spec.md
* Move Yahoo health checks to background
* Bump version to next iteration after v0.7.3-alpha.4 release
* Use GitHub token for automated version bump PRs
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix: resolve insecure OAuth redirect URI registration
Reject unsafe redirect URIs during dynamic OAuth client registration so only HTTPS and loopback HTTP callbacks are accepted. This prevents token/code exfiltration through attacker-controlled redirect targets while preserving local native client flows.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: normalize IPv6 loopback hosts in OAuth redirect validation
Strip bracketed IPv6 hosts before comparing against LOOPBACK_HOSTS so
http://[::1] callbacks are accepted as intended.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix N+1 queries on categories index by batching lookups and removing partial fallbacks
* resolve Codex review suggestion about Guard subcategory against missing parents
* resolve coderabbitai review suggestion - Guard against empty categories when computing category_ids_with_transactions
* resolve coderabbitai review suggestion - Redundant pb-4 makes the conditional dead code
* resolve coderabbitai caution on PR review
* resolve sure-design review - DS Drift Patrol
* fix conflicting hidden/flex classes
* resolve jjmata review suggestion - schema.rb noise
* fix conflict and adjust related parts
* Ignore Brakeman EOLRails warning for Rails 7.2
Restore ignore entry lost during merge from main; documents upgrade
tracking and matches brakeman 7.1.2 in Gemfile.lock.
* db migration executed
* Remove deprecated focus-ring override from goals color picker.
The summary_class override was reintroduced during merge conflict
resolution and clobbered DS::Disclosure's canonical focus styling.
* fix merge conflict on disclosure.rb
* Restore parent-based semantics while keeping the performance win for root categories
* fix(ci): skip scheduled preview cleanup on forks
Only run the hourly Cloudflare preview cleanup on we-promise/sure,
where the required secrets exist.
* Revert schema.rb dump noise unrelated to categories N+1 fix
Restore db/schema.rb from main so PostgreSQL check-constraint and
column-order churn does not obscure the real PR changes.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Remove obsolete Rails 7.2 EOLRails Brakeman ignore
Main is already on Rails 8.1, so the 7.2.3.1 ignore entry is no longer needed.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Address review: bare disclosure docs, category picker aria label
* fix(test): wait for async exchange rate updates in system test
* fix(test): retry account edit submit around Turbo morph races
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: resolve ApiRateLimiter connecting to wrong Redis instance
Use REDIS_URL for API key rate limiting so hosted deployments share the
configured Redis backend instead of defaulting to localhost.
Co-authored-by: Cursor <cursoragent@cursor.com>
* test: align ApiRateLimiter specs with configured redis_url
Use ApiRateLimiter.redis_url for test cleanup and assert the public
redis_url reader instead of inspecting private Redis client internals.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(sync): allow EnableBanking credit card balances to be parsed as available credit
* feat(ui): expose Enable Banking balance interpretation toggle on credit card form
Adds a toggle to the credit card edit dialog, shown only when the account
is linked to an Enable Banking provider account. Toggling persists
treat_balance_as_available_credit and enqueues an item sync so the balance
is reinterpreted immediately.
* fix(sync): keep existing balance when credit limit is missing for available-credit cards
When treat_balance_as_available_credit is on and the API omits credit_limit,
the reported balance is known to be available credit, so recording it as
debt fabricates a liability. Skip the balance write in that case and only
update the available credit metadata.
Also extract the credit card branching into a helper and include
provider_key, account_provider and metadata in the debug log entries so
support can filter /settings/debug by the affected connection.
* refactor(ui): move provider lookup to Account and label the toggle for assistive tech
Adds Account#provider_account_for so the view no longer queries
account_providers directly, and wires aria-labelledby from the toggle to
its visible label.
* fix(ui): apply Enable Banking setting only after a successful account update
Runs the provider flag update after super and only on the redirect path,
so a failed account save no longer persists the flag or enqueues a sync.
Also permits the enable_banking params so malformed input is filtered
instead of raising.
* test(sync): extract relink helper in Enable Banking processor test
* feat(sync): fall back to manually set available credit as the credit limit
When the toggle is on, the accountable's available_credit field now holds
the credit limit (API-provided, or user-entered when the API omits it) and
is never overwritten with the reported balance. This lets users whose bank
reports available credit without a credit limit compute the outstanding
debt by entering their limit in the Available credit field, while keeping
the field stable across syncs so a stale reported balance can never be
mistaken for a limit.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
The "Advanced" settings section is gated to admins in the navigation, but
the controllers exposing family-wide data performed no server-side
authorization — any authenticated user could reach them by navigating
directly to the URL.
Add `before_action :require_admin!` to the controllers that expose
family-wide data:
- Settings::AiPromptsController (family AI assistant config)
- Settings::LlmUsagesController (family LLM usage/billing)
This mirrors the existing guards on Settings::ProvidersController and
Settings::HostingsController. Non-admins are redirected with the standard
"Only admins can perform this action" message (403 for turbo_stream/json).
API key and MCP token management are intentionally left open: both are
user-scoped self-management. Their actions are already scoped to
Current.user, and any signed-in user (not just admins) can create an API
key or authorize an MCP client, so gating them would leave members unable
to view or revoke their own credentials.
Adds controller tests covering admin-only access for the family-wide pages
(member and guest rejected) and member self-service access for the API key
and MCP pages.
`app/views/goals/new.html.erb` called `t("common.close")`, but that key is
only defined at the top level in `ru`, `tr` and `zh-CN`. The canonical key is
`defaults.common.close`, which is defined in `en` (and therefore resolves in
every locale via `config.i18n.fallbacks`) and is already used by
`settings/providers/_drawer_header.html.erb` and
`shared/notifications/_sync_toast.html.erb`.
Because `en` has no top-level `common.close`, the fallback had nothing to fall
back to, so the lookup failed everywhere except those three locales. The
failed lookup was passed straight into the button's `title` and `aria_label`
attributes, injecting literal markup into text meant for screen readers:
t("common.close") @ en
=> "<span class=\"translation_missing\" title=\"translation missing: en.common.close, locale: en\">Close</span>"
Point the view at `defaults.common.close` and drop the now-dead top-level
`common:` blocks from `ru`, `tr` and `zh-CN`. No `en.yml` changes needed.
Verified with `bin/rails runner`: `defaults.common.close` resolves in en, ca,
fr, it, ru, tr, zh-CN, and falls back cleanly to "Close" for de/es. No
`t("common.*")` call sites remain in `app/`.
Closes#2740
Co-authored-by: erkdgn <erkdgn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(settings): super-admin background jobs console
Neither managed nor self-hosted production deployments have any view
into background job state (/sidekiq is only mounted outside
production), so a stuck sync, import, or export is invisible until a
user complains — and even then there is no way to act on it.
Adds /settings/background_jobs, gated by the same super-admin
Admin::BaseController as /settings/debug:
- Worker status header: Sidekiq processes, busy count, queue depths
and latency, retry/scheduled/dead set sizes. Read through a
fail-closed PORO (BackgroundJobConsole) — when Redis is unreachable
the console says so and disables actions instead of pretending
health (deliberate contrast to BackgroundJobHealth's fail-open).
- In-flight operations across all families: incomplete Syncs, Imports
importing/reverting, ImportSessions importing, FamilyExports
pending/processing, each with a liveness verdict derived from
Sidekiq::Workers (job GlobalIDs in running payloads).
- One mutation: mark a presumed-lost operation as such (Sync → stale,
Import → failed/revert_failed, PdfImport → claim released back to
pending, ImportSession/FamilyExport → failed).
Guard rails on the mutation, server-side re-checked:
- refused while Redis state is unknown (fail closed)
- refused while the record's job is visibly executing
- refused until the record has been idle past 30 minutes, so a merely
queued job cannot be shot down and then still run
- refused for parent Syncs with children still in flight (a parent
legitimately has no live job of its own while children run)
- applied inside with_lock with a status re-check, so a job finishing
between render and click wins
- audited as a DebugLogEntry (actor, prior status, family)
The cancel endpoint gets its own Rack Attack throttle since the
console deliberately lives under /settings rather than the throttled
/admin prefix (its 10 req/min limit would fight the page's polling).
* fix(jobs-console): count waiting jobs as live and harden cancel paths
Review feedback on #2682 (Codex, CodeRabbit):
- Liveness now covers jobs sitting in queues, the retry set, and the
scheduled set, not just visibly-executing workers — a job waiting out
a backlog or retry backoff WILL run later, and most affected job
classes don't abort on a flipped status, so cancelling invited
duplicate work. The backlog scan is bounded (5k entries); a truncated
scan fails closed like redis_error?. The liveness column shows
"Queued" for these
- find_record! resolves STI subclass names (TransactionImport,
PdfImport, …) against a base-class whitelist via safe_constantize
instead of a fixed name map, so non-UI callers naming the subclass
don't 404
- A PdfImport stuck in reverting goes to revert_failed like every other
import instead of being released to pending — pending presented a
possibly half-reverted import as publishable again; only the AI
extraction claim (importing) is released to pending
- Redis-unreachable warning renders via DS::Alert; operation id cast
to_s before splitting; admin-cancel error copy moved to i18n
* fix(jobs-console): re-check the stuck window inside the cancel row lock
CodeRabbit round-2: cancellable? evaluates Sidekiq liveness outside the
with_lock transaction (re-running Redis calls under a row lock would be
worse), so a worker picking the job up between the liveness check and
the lock acquisition was invisible. Repeating the updated_at staleness
check inside the lock closes that window — a freshly-started job
touches the record, and the re-check refuses the cancel.
* fix(jobs-console): resolve record_type without reflection, i18n nits
Brakeman flagged safe_constantize on params[:record_type] as a
High-confidence UnsafeReflection (ci/scan_ruby). Replace the
constant lookup with a reverse lookup: find the id in each
cancellable base table and require the claimed type to match the
found record's class or its base class. STI subclass names still
resolve; unknown types still 404.
Also from DS Drift Patrol:
- "Sync · " label in _operation.html.erb now goes through
t(".sync_label", type:)
- drop the redundant default: on background_jobs_label now that
the key exists in the locale file
* feat(i18n): complete Turkish (tr) locale coverage and Türkiye formatting
Brings the Turkish (tr) locale to full key parity with the base English
locale and finalizes Türkiye-appropriate number/currency/date formatting.
Locale coverage:
- Add 64 missing tr.yml files and complete 61 partial ones, covering all
views, models, mailers, breadcrumbs, doorkeeper, and defaults namespaces
(125 tr.yml files total, matching the en.yml set 1:1 by key).
- Translate all user-facing values to natural Turkish (formal "siz"
register), consistent with existing tr terminology (Hesap, İşlem, Bütçe,
Kategori, Transfer, Satıcı, Bakiye, Para Birimi, ...). Brand names
(Sure, Plaid, SimpleFIN, Brex, Questrade, etc.) and format strings are
intentionally left in English.
Türkiye formatting:
- defaults/tr.yml: number.human decimal units translated
(Bin/Milyon/Milyar/Trilyon/Katrilyon; Byte -> Bayt).
- views/application/tr.yml already provides TRY currency (₺, separator
",", delimiter "."); DD.MM.YYYY date format is selectable via
Family::DATE_FORMATS.
Fixes:
- Correct broken interpolation in 5 tr keys where "%%{percent}" rendered
as the literal string instead of the value (goals.show.ring.aria_label,
reports.print.summary.of_income/vs_prior,
reports.trends.insight_higher_weekday/weekend). Aligned to en's working
"%{percent}%" pattern so the percent value now interpolates correctly
(verified at runtime via I18n.t).
Verification:
- i18n-tasks: tr has full key parity with en. The 33 keys reported as
"missing" for tr are pre-existing upstream app-level gaps (referenced
in views/controllers but defined in no locale, including en), not a
tr deficiency.
- check-consistent-interpolations: tr placeholder sets match en for all
shared keys.
- Runtime render checks via `rails runner` confirm interpolations produce
expected output (e.g. "Hedef 50% tamamlandı", "gelirin 30%'i").
Builds on prior Turkish PRs #4 and #31 (musabustun, 2025-07/08) by
backfilling all keys added to en since then and fixing interpolation
regressions.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(i18n): address tr locale review findings on PR #2737
Restore Turkish number.currency.format keys (format/unit/delimiter/separator/
precision) that were dropped — number_to_currency without explicit unit: now
renders "1.234,56 ₺" again. Plus translation fixes from CodeRabbit/Codex review:
- doorkeeper.tr.yml: translate go_back/authorization_code_label/copy_instructions;
use %d.%m.%Y date_format for authorized_applications.created_at
- brex_item: official_hosts_only now says 'cannot be blank' (was contradictory)
- category/deletions: informal -> formal Turkish (silin/atayın/atayabilirsiniz)
- goal_pledges/reports: percent possessive uses 'yüzde %{percent}'i' form
- ibkr_items: Interactive Brokers'tan (hard-consonant ablative)
- indexa_capital_items: savings -> 'Tasarruf Hesabı' (was 'Vadeli Hesap')
- kraken_items: 'spot işlem gerçekleşmeleri' (was 'dolguları')
- properties: 'Metrekare' (was 'Metre Kare')
- registrations: 'vb.' (was 'etc')
- settings/api_keys: drop stray 'Maybe' brand word
- settings: 'Go to ...' -> '<a>...</a> sayfasına gidin'
- simplefin_items + simplefin_account: 'SimpleFin' -> 'SimpleFIN' brand casing
- transactions: 'Kopya?'/'İncele' (was truncated 'Kop?'/'İnc')
- invitation_mailer/invitations: reword to avoid hard-coded case suffixes on
interpolations (Turkish vowel harmony)
Verified: i18n-tasks health clean for tr, all YAML parses, runtime render checks
pass (currency, percent, invitation, doorkeeper keys).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(i18n): restore YAML arrays stringified in tr locale
Two keys were serialized as strings containing JSON-array literals instead
of YAML lists, causing NoMethodError (undefined method 'each' for String):
- settings.securities.show.encryption_warning.keys
-> crashed Settings::Securities#show (t(...).each)
- recurring_transactions.info.triggers
Both now proper YAML arrays matching en. Verified via runtime I18n.t
(Array#each works) and an en-vs-tr type-mismatch scan (no remaining
array/string regressions; the only structural diffs are the pre-existing
plain-string million/trillion units in origin/main).
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(i18n): restore missing 2FA keys, off status key, and moniker placeholders in tr locale
- Add 5 missing settings.securities.show 2FA keys in config/locales/views/settings/securities/tr.yml
- Fix settings.providers.status.'false' key back to off in config/locales/views/settings/tr.yml
- Restore %{moniker} placeholders in account, registrations, settings, invitations, and merchants tr files
- Remove double percent sign in goals velocity_delta_down and velocity_delta_up
* fix(i18n): update settings security page_title from 'Menkul Kıymet' to 'Güvenlik'
---------
Co-authored-by: erkdgn <erkdgn@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
* fix(wise): redirect after token submission instead of rendering inline
The success path in create rendered select_profiles directly (200 OK),
which Turbo rejects for standard form submissions ('Form responses must
redirect to another location'). Now it redirects, carrying the encrypted
token through the session. Session-expired fallbacks now point at
settings_providers_path instead of new_wise_item_path, which has no view.
* test(wise): update controller specs for redirect-based create flow
* fix(wise): read pending token from session, not client params
link_profiles decrypted params[:encrypted_pending_token], even though create
already stores the encrypted token server-side in the session. The client
round-trip was unnecessary and untrusted; now link_profiles reads directly
from session[:wise_pending_encrypted_token].
* Fix statement links in account tab breaking out of Turbo frame, fixes#2614
* Fix statement navigation and split view/edit actions in account tab
- Change eye icon to open the actual file inline in a new tab
- Add pencil icon for navigating to the statement details/edit page
- Make filename click open the file directly (PDF inline, CSV/XLSX download)
* Refactor: extract file_attached local variable to reduce duplication
* added locale context for edit button for account statements
* added test for frame navigation for fix
* Assert turbo frame attribute on unlink form in test
* Fix turbo_frame attribute on unlink button_to form element
The transaction drawer already renders its category dropdown with the
badge variant (colour + icon + search); the transfer drawer and the
bulk-edit modal still show a bare alphabetical list. Bring those two
into line for a more consistent category-picking experience.
* Add send_email_notification rule action
Adds a new rule action that emails a digest of transactions matching a
rule. Re-syncs re-apply every active rule to all in-window matches, so a
notification_deliveries table (unique on rule_id + transaction_id) backs
deduplication and a per-rule watermark:
- Rule::ActionExecutor::SendEmailNotification plucks candidate ids, drops
ones already in notification_deliveries, records the remainder BEFORE
enqueuing (fail-safe: a crash suppresses rather than double-sends), and
returns the count of newly-notified transactions.
- RuleEmailNotificationJob loads the rule + transactions and delivers the
digest via RuleNotificationMailer.
- Creating the action pre-seeds all currently-matching transactions as
already-delivered (after_create_commit), so the rule only emails about
transactions appearing after the action exists.
Dedup keys on the DB row id, not provider identity, so a re-ingested
transaction (new id) may re-notify; accepted as benign.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add view transactions link to rule notification digest email
Include a "View transactions" CTA (APP_DOMAIN/transactions) in both the
HTML and text versions of the rule notification digest email.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address review feedback on rule email notifications
- Restrict digest delivery to family admins only (drop non-admin fallback)
- Make NotificationDelivery.record_for return only inserted ids and enqueue
off that result, preventing duplicate digests under concurrent rule runs
- Seed the notification baseline when an existing action is changed to
send_email_notification, so historical matches are not emailed
- Strengthen tests: assert the transactions CTA/link in the digest and the
enqueued job's transaction-id args
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Include super_admin owner as rule digest recipient
The admin-only recipient lookup used find_by(role: :admin), which excluded
super_admin owners. A self-hosted family is commonly a single super_admin, so
the digest was silently skipped (NullMail no-op) and no email was sent.
Match the recipient on %w[admin super_admin] (the same pattern used elsewhere,
and consistent with User#admin?), while still excluding regular members/guests.
Add tests for the super_admin recipient and the no-admin skip path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add mixed-role digest recipient test
Cover the case where a family has both an admin and a super_admin. The recipient
lookup uses find_by(role: %w[admin super_admin]) with no ORDER BY, so which one
is returned is non-deterministic; the contract is only that the recipient is an
admin-level user (never a member). Assert recipient.admin? rather than a brittle
precedence between the two roles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Deliver rule digest via deliver_later and order in DB
Use deliver_later so a slow/flaky SMTP connection doesn't tie up the
Sidekiq worker, and push the entry-date sort into the query instead of
materializing and sorting the result set in Ruby.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Replace raw hex colors with email-safe design tokens in digest template
The rule digest email hardcoded Tailwind slate-100/200 hex values for
table borders, which aren't part of this project's design system.
Resolve to the actual border-primary/border-secondary token values and
centralize them as a reusable .email-table class in the mailer layout.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(lunchflow): prune orphaned accounts deleted upstream (#1861)
Accounts deleted in Lunch Flow lingered in Sure as unlinked
LunchflowAccount records, permanently pinning the item to "Need setup".
The importer created/updated accounts returned by the API but never
removed records for accounts that disappeared upstream.
Add prune_orphaned_lunchflow_accounts, mirroring SimpleFin's
prune_orphaned_simplefin_accounts: after importing, delete LunchflowAccount
records whose account_id is no longer returned upstream and that are not
linked to an Account via AccountProvider. Linked accounts are kept so the
prune never cascade-destroys a user's Account. The prune is guarded to a
non-empty upstream list so a transient empty/failed response can't wipe out
all accounts.
Fixes#1861
* fix(lunchflow): prune NULL-id orphans + consistent import return shape (#1886)
* fix(lunchflow): make orphan prune resilient to destroy failures
Wrap the per-record destroy in begin/rescue so a single failed prune
doesn't abort the whole import. Pruning runs before transaction fetch,
so an unhandled error would have blocked transaction syncing entirely.
Matches the importer's existing continue-on-error convention.
* fix(lunchflow): only count pruned accounts when destroy succeeds
destroy returns false (without raising) when a before_destroy callback
halts deletion; the prior code incremented the pruned count regardless,
which could inflate it. Increment only on success and log when halted.
* fix(enable-banking): handle overdraft balance sync
* fix(enable-banking): skip nil provider balances
* test(enable-banking): cover nil provider balance
* fix(enable-banking): avoid stale and unsafe balance logs
* fix(enable-banking): guard unavailable balance writes
* test(enable-banking): stub unsaved account api id
* style(enable-banking): remove extra test blank line
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Bump version to next iteration after v0.7.3-alpha.3 release
* Fix automated version bump pull requests
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* fix(import): strip non-breaking-space thousands separators in sanitize_number
The French/Scandinavian number format ("1 234,56") only removed ASCII
whitespace via \s, which does not match a non-breaking space (U+00A0) or
a narrow no-break space (U+202F). Real-world European CSV exports use
those Unicode spaces as the thousands separator, so such amounts failed
the final numeric guard and were silently coerced to "", losing the
value on import. The branch also skipped the non-numeric junk stripping
that the other formats apply.
Strip everything that isn't a digit, the decimal separator, or a minus
sign in this branch, matching the else branch's behavior.
Fixes#2537
* fix(import): strip only whitespace for space-delimited number format
Addresses review feedback: the previous filter removed any non-numeric
character, so a misconfigured/mixed row using US separators ("1,234.56")
under the "1 234,56" format had its period dropped and its comma turned
into a decimal point, silently importing 1.23456 (off by ~1000x) instead
of being rejected.
Strip only whitespace via \p{Space}, which matches ASCII, non-breaking
(U+00A0), and narrow no-break (U+202F) spaces. Unexpected punctuation is
left in place so the existing numeric guard still rejects malformed
values. Add a regression test for the mixed-punctuation case and a
docstring for sanitize_number.
* fix(import): strip leading/trailing currency junk for space-delimited numbers
Follow-up to review on #2538: the whitespace-only strip did not cover
issue #2537's currency-suffix row ("1 234,56 kr") or a leading currency
symbol ("€1 234,56"), which still failed the numeric guard and imported
blank.
Strip non-numeric junk only at the leading/trailing edges (currency
symbols/codes), leaving interior characters untouched. This fully closes
the #2537 repro table while preserving the earlier fix for the mixed
US-format hazard: "1,234.56" keeps its interior period and is still
rejected rather than parsed as 1.23456. Digits, the decimal separator,
and a leading/trailing minus are preserved so signed values and the
guard still behave correctly.
Add regression tests for the currency prefix/suffix cases.
* test(import): add U+202F narrow no-break-space regression case
Complements the existing U+00A0 case so both Unicode thousands-separator
variants named in the fix are covered.
* Add Italian translations for various views and functionalities
- Introduced translations for splits, subscriptions, tag deletions, tags, trades, transactions, transfer matches, transfers, users, valuations, and vehicles.
- Enhanced user experience by providing localized content in Italian for better accessibility and understanding.
* Correzioni traduzioni italiano: API keys shared/index, super admin jobs, imports readback, snaptrade idiomatica, subscriptions progetto, valuations grammatica
* Aggiornamenti delle traduzioni italiane: miglioramenti e aggiunte per vari moduli, inclusi conti, obiettivi, trasferimenti e integrazione con SnapTrade.
* Add per-account toggle to disable automatic transaction categorization
Adds an `enable_category_matcher` boolean (default: true) to accounts so
users can opt out of Plaid's automatic category suggestions on a per-account
basis. When disabled, newly synced transactions arrive uncategorized so
rules or manual assignment take precedence.
- New migration adds `enable_category_matcher` column (default true, null: false)
- `PlaidEntry::Processor#matched_category` gates the CategoryMatcher call on the account flag
- Toggle rendered in the account edit modal for linked accounts (saves via main form submit)
- `AccountableResource#account_params` permits the new field
- `AccountsController#toggle_category_matcher` action added for potential API use
- i18n strings added for label and hint text
- Unit test covers the disabled-matcher path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Only show category matcher toggle for Plaid-linked accounts
Only PlaidEntry::Processor honors enable_category_matcher, but the toggle
rendered for every linked account, silently doing nothing for other
providers. Adds Account::Linkable#supports_category_matcher? (covering both
the legacy plaid_account_id link and AccountProvider rows) and gates the
form toggle on it. The SimpleFIN TODO now also points at this helper so the
toggle appears once SimpleFIN matching lands.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add controller tests for category matcher toggle persistence and rendering
Covers the two paths flagged in review as untested: the flag persisting
through the shared AccountableResource#update action via account_params
(both disable and re-enable), and the edit form rendering the toggle only
for accounts where supports_category_matcher? is true.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Signed-off-by: Mike Lloyd <49411532+mike-lloyd03@users.noreply.github.com>
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
The categorize view renders dates with :short (\"%b %d\"), which omits the
year — ambiguous when the uncategorized backlog spans more than one year.
Use format_date, which renders the family's Settings date format; every
selectable format includes the year.
* docs(ai): document local LLM context window tuning
* Update compose.example.yml
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
---------
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
/sidekiq was mounted `unless Rails.env.production?`, and the Docker
image bakes RAILS_ENV=production — so no self-hosted or managed
deployment has ever had queue tooling, while the production basic-auth
block (with default "sure"/"sure" credentials) was dead code. Worse,
any custom non-production env (e.g. staging) got the dashboard with no
authentication at all.
Now:
- Development keeps the open mount for convenience.
- Everywhere else the route only exists for a signed-in super admin,
via a routing constraint that resolves the signed session cookie the
same way Authentication#find_session_by_cookie does. The session's
user is always the true user (impersonation is resolved at the
Current level and impersonating a super admin is forbidden), and
sessions are only created after MFA verification, so neither can
bypass it. Fails closed — errors mean 404.
- The "sure"/"sure" default credentials are deleted. Basic auth now
activates only when BOTH SIDEKIQ_WEB_USERNAME and
SIDEKIQ_WEB_PASSWORD are explicitly set, as an optional second
layer on top of the constraint.
- Documented in .env.example and docs/hosting/docker.md, including
warnings that the dashboard is break-glass tooling: never manually
retry SimplefinConnectionUpdateJob (single-use token), and deleting
jobs does not update the corresponding Sure records.
The super-admin bar's Jobs link is feature-detected via
sidekiq_web_available? and lights up automatically now that the route
exists in production.
* feat(sync): family-facing cancellation for syncs, imports, and exports
Users had no way to stop or recover any background operation: a
mistaken "Sync all" runs to completion, and an import or export whose
job died (hard worker kills lose in-flight Sidekiq jobs) wedges with a
spinner forever.
Sync cancellation (cooperative — nothing is ever killed):
- New syncs.cancel_requested_at column. Only the cancelled sync
carries the flag: pending descendants are marked stale immediately
(their queued jobs no-op via the existing may_start? guard), while
descendants whose jobs are already executing finish their work
honestly.
- Family::Syncer stops fanning out child syncs once the flag is set
(fresh read per iteration — the flag comes from the web process).
- Finalization resolves a cancel-requested sync to stale instead of
completed, which also skips post-sync (transfer matching, rules,
broadcasts) via the existing stale gate.
- The `visible` scope excludes cancel-requested syncs, so spinners
clear immediately and — fixing a latent bug this feature would have
amplified — sync_later no longer piggybacks a new sync request onto
a dying sync it would silently swallow.
- Cancel button appears next to "Sync all" on the accounts page while
a family sync is visible. SyncsController#cancel scopes through
Sync.for_family with resource_owner, so cross-family ids 404 and
account-level syncs respect per-user account access.
Stuck import/export self-service:
- Import#force_fail! / FamilyExport#force_fail!: allowed only once the
record has been idle past PRESUMED_LOST_AFTER (1 hour — dwarfs any
legitimate run), and applied inside with_lock with a status
re-check, so a job finishing between page render and button click
wins. Imports fail into the existing retry path (reverting ->
revert_failed keeps the revert retryable); PdfImports release their
processing claim back to pending; exports fail so a new one can be
created.
- "Mark as failed" buttons appear on the imports/exports index rows
only when a record is presumed lost, behind the pages' existing
permission gates (statement-import permission for imports, admin
for exports).
* fix(sync-cancel): cascade pending cancels, guard late finalizers, scope provider syncs
Review feedback on #2685 (CodeRabbit, Codex):
- request_cancel! now cascades finalization for pending syncs too: a
pending child resolved to stale never runs its job, so nothing else
would ever call finalize_if_all_children_finalized — its waiting
parent hung in syncing until the 24h sweep (CodeRabbit critical)
- SimplefinItem::Syncer#mark_completed re-reads the sync under a row
lock and skips finalization once cancellation was requested or the
row went terminal — its in-memory copy predates the cancel, and the
unguarded complete! (plus the raw status fallback) resurrected a
cancelled sync and re-ran post-sync (Codex)
- Cancelling provider-item syncs now requires admin: for_family's
resource_owner only scopes the Account branch, so a restricted member
could cancel admin-managed provider syncs spanning accounts they
cannot see. Family- and account-level syncs stay member-cancellable,
matching the buttons the UI shows (Codex)
- Lost-import error copy moved behind i18n
(imports.errors.presumed_lost), resolved at call time (CodeRabbit)
- Tests: pending-child cancel finalizes the parent; late provider
complete! cannot resurrect a cancelled sync; provider-sync
cancellation is admin-only
* fix(sync-cancel): capture the skipped-finalization case via DebugLogEntry
CodeRabbit round-2: the mark_completed skip (cancelled/terminal sync)
is support-relevant — record it in the super-admin debug UI with the
family and provider attached instead of a raw Rails.logger line.
category: provider_sync, matching the other provider syncers.
* Add missing French translations for Insights, Questrade, Wise, SimpleFIN update
Fills in every French locale key that was missing across the app (0 remaining per a full en/fr key comparison over config/locales/**).
- Add insights/fr.yml, questrade_items/fr.yml, wise_items/fr.yml, and simplefin_items/update.fr.yml — previously untranslated features (insights and questrade_items had been left as raw copies of en.yml).
- Add the 5 remaining stray keys in layout/fr.yml, pages/fr.yml,
settings/fr.yml, and settings/hostings/fr.yml.
- Fix a pre-existing duplicate `transactions_title` key in settings/fr.yml.
Terminology kept consistent with existing translations (e.g. "Patrimoine net" for Net Worth, "Trésorerie" for Cash flow, matching reports/fr.yml and pages/fr.yml; Questrade's provider strings mirrored from indexa_capital_items/fr.yml, which shares the same structure).
* Translate 'Insights' as 'Analyses' in French locale
Replace the untranslated anglicism "Insight(s)" with "Analyse(s)" across the French locale files touched by the recent translation pass (views/insights/fr.yml, views/pages/fr.yml, views/layout/fr.yml).
Includes gender agreement fixes that follow from the new feminine noun (e.g. "Analyse ignorée" instead of "Insight ignoré", "Elles se rafraîchissent" instead of "Ils se rafraîchissent").
* Fix leftover English strings and anglicize chart period labels in French locale
Corrects the remaining untranslated fragments found while auditing the French locale (config/locales/**):
- akahu_items/fr.yml, up_items/fr.yml: token field labels were left in English ("App Token", "User Token", "Personal Access Token") while the matching placeholders right below them were already translated.
- settings/fr.yml: drop the now-redundant English parentheticals in the Akahu setup instructions.
- loans/fr.yml: "N/A" → "N/D", to match the same field's translation a few lines down (edit.overview.not_applicable).
- transactions/fr.yml: "A/M" → "C/A" so the abbreviation matches its own French expansion ("Correspondance automatique") instead of the English one ("Auto-Matched").
- models/period/fr.yml: replace the English finance abbreviations (1D/7D/30D/90D/365D/5Y/10Y, WTD/MTD/YTD) used for chart period chips with French-derived ones (1J/7J/30J/90J/365J/5A/10A, SC/MC/AC).
Left untouched: third-party dashboard field names quoted verbatim (SnapTrade/Plaid "Client ID", "Consumer Key"; IBKR Flex Query section names) - these intentionally mirror the exact English labels users see on those providers' own sites, US retirement-account terms (401(k), IRA) with no French equivalent, and brand names.
* Update French translations for clarity and grammar
Improve French localization across multiple provider connection and insight features:
- Fix grammar: change "Derniers" to "Dernières" (feminine agreement)
- Add proper pluralization for Questrade account creation messages
- Enhance SimpleFIN success and error messages for clarity
- Reword Wise account linking success message for better phrasing
* Fix keyboard navigation in DS::Select dropdowns.
Open the listbox on Tab focus and arrow/space/enter from the trigger so
account and category selectors in the transaction form are usable without
a mouse, and reset option tabindex when the menu closes.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Tab skipping DS::Select triggers in transaction form.
Keep focus on the trigger when Tab opens the menu, keep listbox options
at tabindex -1 with inert on the closed menu, and move focus into options
only via arrow keys so Tab no longer jumps to amount or date.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix search input losing focus during DS::Select filtering.
Only repoint focus in syncTabindex when a listbox option was focused and
became hidden; skip while the user is typing in the search field or when
clearSearch runs on menu open.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Escape closing the parent dialog from an open DS::Select.
Stop Escape propagation so only the listbox closes, and suppress
handleButtonFocus briefly after close+refocus so keyboard modality
does not reopen the menu via :focus-visible.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix Tab leaving DS::Select after suppressReopenOnFocus change.
Focus the trigger before closing on Tab and advance to the next dialog
field manually so inert on the menu no longer drops focus to body.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
