* Performance improvements in balance sync cache
Balance::SyncCache#converted_holdings called account.holdings.map { |h| h.dup }
which duplicated every holding record into a new ActiveRecord object, converted
its currency, and stored the full object in a holdings_by_date array hash.
For an investment account with years of history this allocates 100,000+
AR objects on every sync - one per holding row - creating proportional GC
pressure that scaled with account age.
The only consumer of get_holdings(date) was BaseCalculator#holdings_value_for_date,
which immediately discarded the objects after calling .sum(&:amount). The
individual holding objects were never accessed for any other attribute.
Replace the dup-and-group approach with a single aggregation pass that stores
only the per-date sum:
holdings_value_by_date: account.holdings.each_with_object(Hash.new(0)) do |h, totals|
converted = Money.new(h.amount, h.currency).exchange_to(account.currency, date: h.date).amount
totals[h.date] += converted
end
Interface change: get_holdings(date) -> get_holdings_value(date) returns a
Numeric directly rather than an Array. BaseCalculator#holdings_value_for_date
is updated accordingly, and its own per-date memoization layer is removed
since holdings_value_by_date is already fully memoized at the SyncCache level.
* fall back to 1:1 rate in SyncCache when holding exchange rate is missing; update tests to use investment class
* Improve chat LLM error messages
* Fix chat visibility regression in tests
* Harden chat error handling for review feedback
* Fix rubocop private method indentation
* Fix nil presentable_error_message, i18n strings, bare rescue
- Guard `presentable_error_message` with `return nil if error.blank?` so
chats with no error return nil instead of the fallback string; this
prevents the API serialisers from emitting a spurious error message and
stops the mobile polling guard from firing on every successful chat
- Move all hardcoded user-facing error strings into
config/locales/models/chat/en.yml and reference them via I18n.t()
- Replace bare `rescue` in `error_message_for` with `rescue StandardError`
to avoid swallowing system-level exceptions
- Update tests to reference I18n keys instead of raw strings, and add
tests for the nil-error case and the unrecognized-error fallback
https://claude.ai/code/session_01YFMjEds5WVyKPL42xBqMCX
---------
Co-authored-by: SureBot <sure-bot@we-promise.com>
Co-authored-by: Claude <noreply@anthropic.com>
* fix: change postal_code column from integer to string
Allows non-numeric postal codes such as UK format (e.g. SW1A 2AA).
The integer column was silently dropping any alphanumeric input.
The migration is marked irreversible — once alphanumeric postal codes
exist, they cannot be safely cast back to integer.
* fix: update schema.rb, quote fixture postal_code, and add alphanumeric test
* fix: use conventional migration timestamp
* Fix SimpleFIN inverting Loan account balances
SimplefinAccount::Processor#process_account! routes every liability
through OverpaymentAnalyzer + normalize_liability_balance. That path
is built around credit-like liabilities, where transaction history
distinguishes debt vs. credit. For a Loan account with only the
opening anchor (no payment history), the analyzer returns :unknown
and the fallback negates the observed value:
def normalize_liability_balance(observed, bal, avail)
...
-observed
end
That's wrong for loans: the bank reports the principal outstanding
as a positive number from its own books. Negating it stores the loan
balance as negative, so BalanceSheet#net_worth = assets - liabilities
ends up _adding_ the loan instead of subtracting it (off by 2× the
loan amount). Example with a hypothetical mortgage:
raw_balance = 100000.00 (positive — bank's own report)
Sure stored = -100000.00 (negated by the fallback)
Net worth shown = inflated by 2 × 100000
Short-circuit Loan accountables straight to observed.abs and skip the
analyzer/fallback entirely. Loans don't have credit-vs-debt
ambiguity — if the loan is paid off the balance is 0, not negative.
Credit cards still go through the existing heuristic.
* Add observability for the SimpleFIN loan sign branch
Mirrors the logging + Sentry breadcrumb the credit-card branches emit
when the OverpaymentAnalyzer classifies as :credit / :debt, so the
loan short-circuit shows up in production traces too. Per CodeRabbit
review on #1574.
* Test that positive bank-reported loan balances are preserved
The existing "inverts negative balance for loan liabilities" test only
covers a bank that reports the loan as negative — both the old (buggy)
fallback and the new short-circuit produce the same +50000 there, which
is why the inversion bug went undetected. Add a sibling test where the
bank reports +50000 (the common mortgage convention); under the old
code that became -50000 and inflated net worth.
* Redact monetary amounts from SimpleFIN liability info logs
Move raw observed/stored amounts and metric totals from `Rails.logger.info`
and `Sentry.add_breadcrumb` payloads to a `Rails.logger.debug` line.
The info-level message and breadcrumb data now carry only identifiers
(`sfa_id`) plus the classification (`loan` / `credit` / `debt` /
`unknown`) and `tx_count`, so log aggregators and Sentry no longer
receive raw monetary values for any of the four liability branches.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add missing IndexaCapitalItem::SyncCompleteEvent
Syncable#sync_broadcaster instantiates self.class::SyncCompleteEvent,
which is implemented for every other provider (Plaid, Lunchflow,
Mercury, etc.) but was missing for IndexaCapitalItem. The error was
swallowed by Sync#perform_post_sync's rescue, so syncs appeared to
succeed but post-sync UI broadcasts never fired:
Error performing post-sync for IndexaCapitalItem (...):
uninitialized constant IndexaCapitalItem::SyncCompleteEvent
This adds the class, modeled on LunchflowItem::SyncCompleteEvent,
restoring per-account and per-item Turbo broadcasts after Indexa
Capital syncs.
* Fix IndexaCapital account setup never creating accounts
complete_account_setup read params[:accounts], but the form in
setup_accounts.html.erb submits account_ids[] (array) and
sync_start_dates[<id>] (hash). The hash was always empty, so every
submit hit the empty-config branch and bounced back with
"No accounts to set up." — accounts were never created.
The controller also branched on config[:account_type] / config[:subtype]
even though the form has no account-type picker (Indexa Capital is an
investment-only broker). Rewrote complete_account_setup to consume the
form's actual params and infer the accountable type as Investment from
indexa_capital_account.account_type.
* Fix IndexaCapital balance double-count and account type
Two more issues in the IndexaCapital flow that surfaced once accounts
could actually be created (see prior commit):
1. Accountable type was inferred from indexa_capital_account.account_type
("mutual" / "pension"), but infer_accountable_type doesn't recognize
those values and falls through to "Depository". The result: every
imported Indexa account showed up as a Cash depository account
instead of an Investment account, hiding holdings/trades surfaces.
Indexa Capital is investment-only, so hard-code the accountable
type to Investment.
2. Account::Processor#calculate_total_balance summed every row in
raw_holdings_payload. Indexa returns a time series — one row per
security per date — so the naive sum double-counts (observed:
reported €91,633 became stored balance €180,039). Trust the API's
current_balance when present, and if we have to fall back to a
computed total, dedupe by instrument and take the latest-dated
amount per security.
* Fix IndexaCapital holdings reflecting oldest snapshot per security
HoldingsProcessor#process iterated every row in raw_holdings_payload.
Indexa returns a time series (many rows per security across dates),
and each iteration upserts the same (account, security, today) holding
row, so the LAST row processed wins. The payload is ordered with
newer dates first, so the last row processed is the OLDEST snapshot —
the holdings shown in the UI reflected tiny early positions instead
of the current ones (e.g. 3.8 shares of US 500 stored vs 62.34 actual).
Reduce the payload to one row per security (latest date) before
processing. The cost-basis update is now also driven by the latest
snapshot for the same reason.
* Fix IndexaCapital holdings using per-lot detail instead of totals
Importer#normalize_holdings_response read data[:fiscal_results], which
the Indexa API returns as per-tax-lot detail — many rows per security
covering each subscription_date, plus virtual sell/buy rows generated
by rebalances. Iterating it produced wildly wrong stored holdings:
e.g. 9.61 shares stored for Vanguard US 500 vs 62.34 actual; total
weights summed to ~10% instead of 100%.
The same response also includes data[:total_fiscal_results] — one
aggregated row per security with current titles/amount/cost matching
the Indexa UI and the user-downloadable positions CSV. Prefer it,
falling back to the per-lot field only when the totals are absent.
* Address CodeRabbit review on IndexaCapital fixes
Four review items, all fixed:
* Share instrument-key extraction
HoldingsProcessor#extract_ticker and Processor#calculate_holdings_value
used different fallback orders (one looked at :isin, the other at
:isin_code), so they could disagree on which rows referred to the same
security. Moved a single extract_instrument_key helper into
IndexaCapitalAccount::DataHelpers and routed both callers through it.
* Simplify Processor#calculate_holdings_value
The date-based dedupe was a workaround for the bug already fixed in
the importer (which now stores total_fiscal_results — one row per
security). Replaced the date comparison with a per-security map
populated via the shared key extractor. Same end result, fewer
moving parts, no fragile string-date comparison.
* Drop dead config key passed to create_account_from_indexa_capital
create_account_from_indexa_capital only reads :subtype and :balance
from its config arg. Passing :sync_start_date there was inert.
* Don't mark created accounts as skipped on post-create errors
In complete_account_setup, ensure_account_provider! and
update!(sync_start_date:) ran inside the same begin/rescue as the
Account.create!. If either raised after the Account row was already
persisted, control jumped to the rescue with created_count not yet
incremented and the account was wrongly counted as skipped. Now:
parse the form-supplied sync_start_date up front (a malformed value
is silently dropped instead of bubbling out of the loop), bump
created_count immediately after persisted?, and isolate the post-
create steps in their own rescue so failures there are logged but
don't desync the success counter.
* Fall back to /portfolio so pension plans get holdings imported
Indexa's /accounts/{id}/fiscal-results endpoint returns
{fiscal_results: [], total_fiscal_results: []} for pension plan
accounts (e.g. type "pension"). The same positions are exposed via
/accounts/{id}/portfolio in instrument_accounts[].positions[] for
both mutual funds and pensions, so use it as a fallback when
fiscal-results is empty.
The portfolio response uses the same field names HoldingsProcessor
already understands (instrument, titles, price, amount, cost_amount)
plus a derived cost_price (cost_amount / titles) added during
adaptation. No HoldingsProcessor changes needed.
Verified against the user-downloadable "Posiciones" CSV for an
SH71ZPMY pension account: two positions (N5138 Acciones, N5137
Bonos) and balance €8,273.56 match exactly.
* Fix CI: update tests for new IndexaCapital flow + rubocop blank line
* Lint: drop trailing blank line before `end` in
IndexaCapitalAccount::Processor (Layout/EmptyLinesAroundClassBody).
* Controller test: complete_account_setup#creates was posting
params: { accounts: { id => { account_type:, subtype: } } } against
the old controller schema. The new endpoint reads
params[:account_ids] and infers Investment for Indexa Capital, so
switch the test to that shape (and update the matching skip-already-
linked / no-selected-accounts cases).
* Processor test: "updates account balance from holdings value" set
current_balance: 38905.21 alongside holdings summing to 27093.01
and asserted the latter wins. After the fix
(calculate_total_balance prefers the API-reported current_balance
when present), the API value is the right answer. Renamed to
"trusts API current_balance over holdings sum when present" and
added a sibling test that nils current_balance to exercise the
holdings-sum fallback path explicitly (still asserts 27093.01).
* Wrap account creation+linking in a transaction to avoid orphans
complete_account_setup created the Account row first, incremented
created_count, and only then called ensure_account_provider! / the
sync_start_date update inside an inner rescue. If the link or the
sync_start_date update raised after the Account was already persisted,
control fell into the inner rescue: the orphaned Account row stayed
in the database, the failure was silently logged, and the success
counter was inflated.
Wrap creation, ensure_account_provider!, and the optional
sync_start_date update in a single ActiveRecord::Base.transaction.
Increment created_count only after the transaction commits; on any
exception the outer rescue rolls the whole step into skipped_count
with a clear log line tagged with the indexa_capital_account id.
* Add Sophtron Provider
* fix syncer test issue
* fix schema wrong merge
* sync #588
* sync code for #588
* fixed a view issue
* modified by comment
* modified
* modifed
* modified
* modified
* fixed a schema issue
* use global subtypes
* add some locales
* fix a safe_return_to_path
* fix exposing raw exception messages issue
* fix a merged issue
* update schema.rb
* fix a schema issue
* fix some issue
* Update bank sync controller to reflect beta status
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
* Rename settings section title to 'Sophtron (alpha)'
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
* Consistency in alpha/beta for Sophtron
* Good PR suggestions from CodeRabbit
---------
Signed-off-by: soky srm <sokysrm@gmail.com>
Signed-off-by: Sophtron Rocky <rocky@sophtron.com>
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* SimpleFIN: setup UX + same-provider relink + card-replacement detection
Fixes three bugs and adds auto-detection for credit-card fraud replacement.
Bugs:
- Importer: per-institution auth errors no longer flip the whole item to
requires_update. Partial errors stay on sync_stats so other institutions
keep syncing.
- Setup page: new activity badges (recent / dormant / empty / likely-closed)
via SimplefinAccount::ActivitySummary. Likely-closed (dormant + near-zero
balance + prior history) defaults to "skip" in the type picker.
- Relink: link_existing_account allows SimpleFIN to SimpleFIN swaps by
atomically detaching the old AccountProvider inside a transaction. Adds
"Change SimpleFIN account" menu item on linked-account dropdowns.
Feature (credit-card scope only):
- SimplefinItem::ReplacementDetector runs post-sync. Pairs a linked dormant
zero-balance sfa with an unlinked active sfa at the same institution and
account type. Persists suggestions on Sync#sync_stats.
- Inline banner on the SimpleFIN item card prompts relink via CustomConfirm.
Per-pair dismiss button scoped to the current sync (resurfaces on next
sync if still applicable). Auto-suppresses once the relink has landed.
Dev tooling:
- bin/rails simplefin:seed_fraud_scenario[email] creates a realistic broken
pair for manual QA; cleanup_fraud_scenario reverses it.
* Address review feedback on #1493
- ReplacementDetector: symmetric one-to-one matching. Two dormant cards
pointing at the same active card are now both skipped — previously the
detector could emit two suggestions that would clobber each other if
the user accepted both.
- ReplacementDetector: require non-blank institution names on both sides
before matching. Blank-vs-blank was accidentally treated as equal,
risking cross-provider false matches when SimpleFIN omitted org_data.
- ActivitySummary: fall back to "posted" when "transacted_at" is 0
(SimpleFIN's "unknown" sentinel). Integer 0 is truthy in Ruby, so the
previous `|| fallback` short-circuited and ignored posted.
- Controller: dismiss key is now the (dormant, active) pair so dismissing
one candidate for a dormant card doesn't suppress others.
- Helper test: freeze time around "6.hours.ago" and "5.days.ago"
assertions so they don't flake when the suite runs before 06:00.
* Address second review pass on #1493
- ReplacementDetector: canonicalize account_type in one place so filtering
(supported_type?) and matching (type_matches?) agree on "credit card"
vs "credit_card" variants.
- ReplacementDetector: skip candidates with nil current_balance. nil is
"unknown," not "zero" — previously fell back to 0 and passed the near-
zero gate, allowing suggestions without balance evidence.
* EnableBanking: skip CARD-* counterparty in name
# Conflicts:
# test/models/enable_banking_entry/processor_test.rb
# Conflicts:
# test/models/enable_banking_entry/processor_test.rb
* Fix whitespace in remittance_information array
Whitespace added before 'ACME SHOP' in remittance_information.
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
* Fix merchant creation for Wise and prefer remittance for Entry name if counterparty is CARD-XXX
* Fix review
* Handle scalars
* Handle empty strings
* Fix review
* Make truncate not use ellipsis at the end
---------
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: quentinreytinas <quentin@reytinas.fr>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* fix: Restore legacy fallback for credit card balance calculation in Enable Banking
* test: update test following new behavior
* test: keep old test
* fix: use absolute value for balance computation
Ensure accessible_account_ids filtering is applied whenever account scope is provided, including empty arrays, so users with no shared accounts cannot see family-wide transactions.
Also make totals robust when scoped queries return no rows and add regression tests for both visibility and totals behavior with empty accessible account lists.
* Optimize UI in budget
* update locales
* Optimize UI
* optimize suggested_daily_spending
* try over_budget and on_track
* update locale
* optimize
* add budgets_helper.rb
* fix
* hide no buget and no expense sub-catogory
* Optimize
* Optimize button on phone
* Fix Pipelock CI noise
* using section to render both overbudget and onTrack
* hide last ruler
* fix
* update test
---------
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* feat: add currency management for families with enabled currencies
* feat: update currency selection logic and improve accessibility
* feat: update currency preferences to use group moniker in titles
---------
Signed-off-by: Ang Wei Feng (Ted) <hello@tedawf.com>
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* fix(enable-banking): refactor error handling and add missing GIN index
* fix(enable-banking): handle wrapped network errors and fix concurrent index migration
* fix(enable-banking): extract network errors to frozen constant
* fix(enable-banking): consolidate error handling and enforce strict localization
* fix(enable-banking): improve sync error handling and fix invalid test status
* test(enable_banking): use OpenStruct instead of mock for provider
* Binance as securities provider
* Disable twelve data crypto results
* Add logo support and new currency pairs
* FIX importer fallback
* Add price clamping and optiimize retrieval
* Review
* Update adding-a-securities-provider.md
* day gap miss fix
* New fixes
* Brandfetch doesn't support crypto. add new CDN
* Update _investment_performance.html.erb
* feat: add Binance support (Items, Accounts, Importers, Processor, and Sync)
* refactor: deduplicate 'stablecoins' constant and push stale_rate filter to SQL
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Add Quick Categorize Wizard (iteration 1)
Adds a step-by-step wizard for bulk-categorizing uncategorized transactions
and optionally creating auto-categorization rules, reducing friction after
connecting a new bank account.
New files:
- Transaction::Grouper abstraction + ByMerchantOrName strategy (groups by
merchant name when present, falls back to entry name; sorted by count desc)
- Transactions::CategorizesController (GET show / POST create)
- Wizard view at app/views/transactions/categorizes/show.html.erb
- Stimulus categorize_controller.js (Enter-key-to-select-first)
- Tests for grouper and controller
Modified files:
- routes.rb: resource :categorize inside namespace :transactions
- transactions_controller.rb: expose @uncategorized_count to index
- transactions/index.html.erb: Categorize (N) button in header
- family.rb: uncategorized_transaction_count query
- rules_controller.rb: return_to param support for wizard → rule editor flow
- rules/_form.html.erb, rules/new.html.erb: pass return_to through form
- i18n: categorizes show/create keys + rules.create.success
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Quick Categorize Wizard — iteration 2 polish
Six improvements from live testing:
- Breadcrumb: Home > Transactions > Categorize
- Layout: category picker + confirmation dialog above transaction list
- Inline confirmation dialog: clicking a category pill shows a <dialog>
summarising what will happen (N transactions → category, rule if checked)
with Confirm and Cancel buttons — no redirect to rule editor
- Direct rule creation: rule created with active: true in the controller
instead of redirecting to the rule editor; revert return_to plumbing from
RulesController, rules/_form, rules/new, rules/en.yml
- Individual row assignment: per-row category <select> submits via
PATCH /transactions/categorize/assign_entry and removes the row via
Turbo Stream (assign_entry action + route)
- Enter key guard: selectFirst only fires when exactly 1 pill is visible
after filtering
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Quick Categorize Wizard — iteration 3 reliability fixes and UX polish
- Fix Stimulus controller not loading: remove invalid `@hotwired/turbo` named
import (not in importmap); use global `Turbo.renderStreamMessage` instead
- Fix Enter key submitting form with wrong category when search field is
unfocused: move keydown listener to document so it fires regardless of focus
- Prevent Enter from submitting when multiple categories are visible
- Clear search filter after bulk category assignment (pill click or Enter),
but not after individual row dropdown assignment
- Update group transaction count and total amount live as entries are assigned
via row dropdown or partial bulk assignment
- Add turbo frames for remaining count and group summary so they update
without a full page reload
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Quick categorization polish
* refactoring
* Remove unused GROUPS_PER_BATCH constant, fix ERB self-closing tags
Wizard only ever uses one group at a time so limit: 1 is correct and
more honest than fetching 20 and discarding 19. ERB linter fixes are
whitespace/void-element corrections with no functional change.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Move Categorize button into ... menu on transactions index
Reduces header clutter by putting it in the overflow menu at the bottom,
where it only appears when there are uncategorized transactions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Scope categorize wizard to accessible entries only
Fixes a security issue where users with restricted account access via
account sharing could view and categorize transactions from accounts
they cannot access through normal transaction flows.
- Pass Current.accessible_entries to Transaction::Grouper so the wizard
only displays groups from accounts the user can see
- Use Current.accessible_entries on all write paths in create and
assign_entry, matching the pattern in TransactionCategoriesController
- Refactor Grouper to accept an entries scope instead of a family object,
keeping authorization concerns in the controller
- Add tests verifying inaccessible entries are hidden from the wizard
and cannot be categorized via forged POST/PATCH params
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Clamp position param to >= 0 to guard against negative offset
Prevents ArgumentError from Array#drop when a negative position is
passed via a tampered query string or form value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Surface rule creation failure and add accessible names to entry row
- Capture Rule.create_from_grouping! return value; set flash[:alert] when
nil so users who checked "Create Rule" know it wasn't created (e.g. a
duplicate already exists); stream the notification for partial updates
- Add aria-label to the per-row checkbox and category select in
_entry_row so screen readers can identify which transaction each
control belongs to
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Localize breadcrumb labels in categorizes controller
Follows the pattern used by FamilyExportsController and ImportsController.
Adds 'transactions' and 'categorize' keys to the breadcrumbs locale file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add error handling to categorize controller fetch calls
Check response.ok before parsing the body and add .catch handlers
so network failures and non-2xx responses are logged rather than
silently swallowed. On assignment failure the per-row select is
reset to empty so the user can retry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Scope preview_rule to accessible entries only
Entry.uncategorized_matching now accepts an entries scope instead of a
family object, matching the same pattern used for Transaction::Grouper.
The preview_rule action passes Current.accessible_entries so rule
previews respect account sharing permissions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Scope remaining count to accessible entries
Adds Entry.uncategorized_count(entries) following the same pattern as
uncategorized_matching. Replaces all three uses of
Current.family.uncategorized_transaction_count in the categorize
controller so the remaining-count badge reflects only the transactions
the current user can actually access and categorize.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Comments got separated from their function
* Remove quick-categorize-wizard dev notes
This was a planning document used during development, not intended
for the final branch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Recompute remaining entries from server state after writes
Adds uncategorized_entries_for helper that reloads remaining entries
from the DB with a category_id IS NULL filter after each write, so
the partial-update Turbo Stream reflects server-side state rather than
trusting the client-provided remaining_ids. This handles the case where
a concurrent request has categorized one of the remaining entries
between page render and form submit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Rename create_from_grouping! to create_from_grouping
The method rescues RecordInvalid and returns nil, which contradicts
the bang convention. Dropping the ! correctly signals that callers
should check the return value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Clamp offset in grouper to guard against negative values
The controller already clamps position before passing it as offset,
but clamping in the grouper itself prevents ArgumentError from
Array#drop if the grouper is ever called directly with a negative offset.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* feat: improve QIF import date format selection
- Added a reusable date format auto-detection method.
- Show a live preview of the first parsed date that updates client-side
as the user changes the dropdown selection, via a new
qif-date-format Stimulus controller.
- Show an error alert and disable the submit button when no supported
date format can parse the file's dates.
* A few polishing fixes:
- Missing return on redirects
Stale REASONABLE_DATE_RANGE constant.
- Replaced the frozen constant with a class method
Bare inline rescue — Replaced Date.strptime(s, fmt) rescue nil with an explicit begin/rescue catching.
- save!(validate: false) in controller — Changed to update_column(:column_mappings, ...) in qif_category_selections_controller.rb:22, matching the pattern used in detect_and_set_qif_date_format!.
- Unescaped JSON in HTML attribute — Replaced the raw <div> with tag.div ... do block in show.html.erb:16, letting Rails properly escape the data attribute value.
* fix: address review feedback for QIF date format feature
- Add missing `return` after redirect for non-QIF imports
- Pass date_format to parse_opening_balance in will_adjust_opening_anchor?
- Return empty array when no usable date sample exists for format preview
- Add sr-only label to date format select for accessibility
- Consolidate duplicate try_parse_date/parse_qif_date into single method
- Remove misleading ambiguity scoring comment from detect_date_format
- Skip redundant sync_mappings when date format already triggered a sync
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Use %{product_name} interpolation in locale strings
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Added server-side validation to Category model to enforce 6-digit hex format for colors.
- Added HTML pattern attribute to category form for client-side validation.
- Updated tests to cover validation and fixed existing tests using shorthand hex colors.
* Record dividends and interest as Trades in investment accounts
All investment income (dividends and interest) is now modeled as a
Trade with qty: 0 and price: 0, keeping security_id NOT NULL on trades
intact. Dividends require a security; interest falls back to a
per-account synthetic cash security (kind: "cash", offline: true) when
none is selected, matching how brokerages handle uninvested cash
internally.
- Add `kind` column to securities ("standard" | "cash") with DB check
constraint; `Security.cash_for(account)` lazily finds or creates the
synthetic cash security; `scope :standard` excludes synthetic
securities from user-facing pickers
- Trade::CreateForm: new `dividend` type (security required); `interest`
now creates a Trade instead of a Transaction
- Trade form: Dividend and Interest in the type dropdown with a security
combobox (required for dividend, optional for interest)
- transactions table: untouched
* UI fixes
* HealthChecker — both scopes now chain .standard to exclude cash securities from provider health checks.
DB query moved to model — Account#traded_standard_securities in app/models/account.rb, view uses account.traded_standard_securities.
DRY income creation — create_income_trade(sec:, label:, name:) extracted as shared private method; create_dividend_income and create_interest_income delegate to it.
show.html.erb blocks merged — single unless trade.qty.zero? block covers qty/price/fee fields.
Test extended — assert_response :unprocessable_entity added after the assert_no_difference block.
* Hide cash account ticker from no-security trade detail
* Fix CodeRabbit review issues from PR #1311
- Remove duplicate YAML keys in translation files (de, es, fr)
- Add error handling for security resolution in create_dividend_income
- Extract income trade check to reduce duplication in header template
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* Include holdings in dividend/interest security picker
The security picker for dividend/interest trades should include all securities
in holdings, not just those with trade history. This fixes the issue where
accounts with imported holdings (e.g., SimpleFIN) but no trades would have an
empty picker and be unable to record dividends.
Uses UNION to combine securities from both trades and holdings.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* scope picker to holdings only (a trade creates a holding anyway)
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Add an optional fee field (decimal, precision: 19, scale: 4) to trades.
Fee is included in the total amount calculation (qty * price + fee) for
both create and update flows. The fee field appears on both the create
and edit forms, defaults to 0, and auto-submits like other trade fields.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: deduplicate Enable Banking API transactions with different entry_reference IDs (#954)
Enable Banking API sometimes returns the same logical transaction multiple
times with different entry_reference values in a single response. This causes
duplicate entries because the existing ID-based deduplication treats them as
distinct transactions.
Add content-based deduplication that compares (date, amount, currency,
creditor, debtor, remittance_information, status) to detect and remove these
API-level duplicates before storing them. The first occurrence is kept.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add Enable Banking processor and importer deduplication tests (#954)
Add tests for:
- EnableBankingEntry::Processor: verifies entry_reference fallback for
external_id, idempotent re-processing, string key handling
- EnableBankingItem::Importer: verifies content-based deduplication removes
API-level duplicates while preserving legitimate distinct transactions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: handle nil values in remittance_information array for dedup key (#954)
Call compact and map(&:to_s) before sort.join when remittance_information
is an array, preventing ArgumentError when it contains nil elements.
Also document the known limitation of content-based deduplication
collapsing genuinely distinct identical transactions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add coverage for nil values in remittance_information array (#954)
Verify that deduplication handles remittance_information arrays containing
nil elements without raising ArgumentError, and correctly treats arrays
with different nil positions but same non-nil content as duplicates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prefer transaction_id over content-based dedup to preserve legit duplicates (#954)
When transaction_id is present, use it as the dedup key instead of falling
back to content-based deduplication. This preserves legitimately distinct
transactions with identical content (e.g. two laundromat payments of the
same amount on the same day) while still deduplicating the Enable Banking
duplicate entry_reference issue when transaction_id is nil.
Addresses review feedback from @jjmata about legitimate duplicate
transactions being incorrectly collapsed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use composite key for dedup instead of transaction_id alone (#954)
Per the Enable Banking API docs, transaction_id is not guaranteed to be
unique. Include it as one component of the composite content key rather
than using it as the sole dedup criterion. This preserves transactions
with non-unique transaction_ids but different content, while still
deduplicating true API-level duplicates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add value_date fallback coverage for dedup key (#954)
build_transaction_content_key falls back to value_date when booking_date
is absent. This test exercises that path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: document known limitation of content-based dedup (#954)
When transaction_id is nil for both transactions, pure content comparison
applies, which could theoretically collapse two genuinely distinct
transactions with identical fields. Document this trade-off inline for
future maintainers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add credit_debit_indicator to dedup composite key (#954)
transaction_amount.amount is always positive in the Enable Banking API,
with direction encoded separately in credit_debit_indicator (CRDT/DBIT).
Without it in the composite key, a payment and a same-day refund of the
same amount to the same merchant would produce identical keys, silently
dropping one transaction.
- Add credit_debit_indicator to build_transaction_content_key
- Add test for payment + same-day refund scenario
- Update docstring to document the rationale
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* third party provider scoping
* Simplify logic and allow only admins to mange providers
* Broadcast fixes
* FIX tests and build
* Fixes
* Reviews
* Scope merchants
* DRY fixes
* Reorganize import UI with Financial Tools / Raw Data tabs
Split the flat list of import sources into two tabbed sections using
DS::Tabs: "Financial Tools" (Mint, Quicken/QIF, YNAB coming soon) and
"Raw Data" (transactions, investments, accounts, categories, rules,
documents). This prepares for adding more tool-specific importers
without cluttering the list.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* Fix import controller test to account for YNAB coming soon entry
The new YNAB "coming soon" disabled entry adds a 5th aria-disabled
element to the import dialog.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* Fix system tests to click Raw Data tab before selecting import type
Transaction, trade, and account imports are now under the Raw Data tab
and need an explicit tab click before the buttons are visible.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* feat: Add bulk import for NDJSON export files
Implements an import flow that accepts the full all.ndjson file from data exports,
allowing users to restore their complete data including:
- Accounts with accountable types
- Categories with parent relationships
- Tags and merchants
- Transactions with category, merchant, and tag references
- Trades with securities
- Valuations
- Budgets and budget categories
- Rules with conditions and actions (including compound conditions)
Key changes:
- Add BulkImport model extending Import base class
- Add Family::DataImporter to handle NDJSON parsing and import logic
- Update imports controller and views to support NDJSON workflow
- Skip configuration/mapping steps for structured NDJSON imports
- Add i18n translations for bulk import UI
- Add tests for BulkImport and DataImporter
* fix: Fix category import and test query issues
- Add default lucide_icon ("shapes") for categories when not provided
- Fix valuation test to use proper ActiveRecord joins syntax
* Linter errors
* fix: Add default color for tags when not provided in import
* fix: Add default kind for transactions when not provided in import
* Fix test
* Fix tests
* Fix remaining merge conflicts from PR 766 cherry-pick
Resolve conflict markers in test fixtures and clean up BulkImport
entry in new.html.erb to use the _import_option partial consistently.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* Import Sure `.ndjson`
* Remove `.ndjson` import from raw data
* Fix support for Sure "bulk" import from old branch
* Linter
* Fix CI test
* Fix more CI tests
* Fix tests
* Fix tests / move PDF import to first tab
* Remove redundant title
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add AI merchant enhancement and dedup
* Enhancements
Add error if job is already running
add note that we also merge merchants
* Allow updating provider website
* Review fixes
* Update provider_merchant.rb
* Linter and fixes
* FIX transaction quick menu modal
* Initial split transaction support
* Add support to unsplit and edit split
* Update show.html.erb
* FIX address reviews
* Improve UX
* Update show.html.erb
* Reviews
* Update edit.html.erb
* Add parent category to dialog
* Update en.yml
* Add UI indication to totals
* FIX ui update
* Add category select like rest of app
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Add conditional migration for vector_store_chunks table
Creates the pgvector-backed chunks table when VECTOR_STORE_PROVIDER=pgvector.
Enables the vector extension, adds store_id/file_id indexes, and uses
vector(1024) column type for embeddings.
* Add VectorStore::Embeddable concern for text extraction and embedding
Shared concern providing extract_text (PDF via pdf-reader, plain-text as-is),
paragraph-boundary chunking (~2000 chars, ~200 overlap), and embed/embed_batch
via OpenAI-compatible /v1/embeddings endpoint using Faraday. Configurable via
EMBEDDING_MODEL, EMBEDDING_URI_BASE, with fallback to OPENAI_* env vars.
* Implement VectorStore::Pgvector adapter with raw SQL
Replaces the stub with a full implementation using
ActiveRecord::Base.connection with parameterized binds. Supports
create_store, delete_store, upload_file (extract+chunk+embed+insert),
remove_file, and cosine-similarity search via the <=> operator.
* Add registry test for pgvector adapter selection
* Configure pgvector in compose.example.ai.yml
Switch db image to pgvector/pgvector:pg16, add VECTOR_STORE_PROVIDER,
EMBEDDING_MODEL, and EMBEDDING_DIMENSIONS env vars, and include
nomic-embed-text in Ollama's pre-loaded models.
* Update pgvector docs from scaffolded to ready
Document env vars, embedding model setup, pgvector Docker image
requirement, and Ollama pull instructions.
* Address PR review feedback
- Migration: remove env guard, use pgvector_available? check so it runs
on plain Postgres (CI) but creates the table on pgvector-capable servers.
Add NOT NULL constraints on content/embedding/metadata, unique index on
(store_id, file_id, chunk_index).
- Pgvector adapter: wrap chunk inserts in a DB transaction to prevent
partial file writes. Override supported_extensions to match formats
that extract_text can actually parse.
- Embeddable: add hard_split fallback for paragraphs exceeding CHUNK_SIZE
to avoid overflowing embedding model token limits.
* Bump schema version to include vector_store_chunks migration
CI uses db:schema:load which checks the version — without this bump,
the migration is detected as pending and tests fail to start.
* Update 20260316120000_create_vector_store_chunks.rb
---------
Co-authored-by: sokiee <sokysrm@gmail.com>
DeveloperMessage was a debug-only STI subclass of Message that was never
created by any production code. Remove the model, view partial, test,
and fixtures, and simplify Chat#conversation_messages accordingly.
https://claude.ai/code/session_012pm5HKGKFs1tpAsvXMr4Tp
Co-authored-by: Claude <noreply@anthropic.com>
