* fix(enable-banking): refactor error handling and add missing GIN index
* fix(enable-banking): handle wrapped network errors and fix concurrent index migration
* fix(enable-banking): extract network errors to frozen constant
* fix(enable-banking): consolidate error handling and enforce strict localization
* fix(enable-banking): improve sync error handling and fix invalid test status
* test(enable_banking): use OpenStruct instead of mock for provider
* Binance as securities provider
* Disable twelve data crypto results
* Add logo support and new currency pairs
* FIX importer fallback
* Add price clamping and optiimize retrieval
* Review
* Update adding-a-securities-provider.md
* day gap miss fix
* New fixes
* Brandfetch doesn't support crypto. add new CDN
* Update _investment_performance.html.erb
* feat: add Binance support (Items, Accounts, Importers, Processor, and Sync)
* refactor: deduplicate 'stablecoins' constant and push stale_rate filter to SQL
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Extract Entry.uncategorized_transactions scope, remove Family#uncategorized_transaction_count
Adds a single Entry.uncategorized_transactions scope containing the
shared conditions (transactions join, active accounts, category nil,
not transfer kinds, not excluded). All callers now use this scope:
- Entry.uncategorized_matching builds on it
- Transaction::Grouper::ByMerchantOrName#uncategorized_entries uses it
- categorizes_controller#uncategorized_entries_for uses it (also fixes
missing status/excluded filters that were silently absent before)
- Both controllers replace Current.family.uncategorized_transaction_count
with Current.accessible_entries.uncategorized_transactions.count so
the button count and wizard count both respect account sharing
Family#uncategorized_transaction_count removed as it is now unused and
was family-scoped rather than user-scoped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Scope assign_entry write to Current.accessible_entries
Replaces unscoped Entry.where(id:) with Current.accessible_entries.where(id:)
so the write path is consistent with the find above it. Not exploitable
given the find would 404 first, but removes the pattern inconsistency.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add privacy-sensitive class to amounts in categorize wizard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Extract uncategorized_count helper in CategorizesController
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix comment on uncategorized_transactions scope to mention draft accounts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Use uncategorized_count helper in assign_entry action
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fixed crypto and loan add menus
* Changed unspecified crypto account type to none for consistency
* Added default add message for loan subtype
* Made the short form of non-mortgage loans in loans.rb match the long form
* Edited the crypto tooltip to be country generic
* Update config/locales/views/loans/en.yml
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Geoffrey <79559478+CYBRXT@users.noreply.github.com>
* Update app/views/loans/_form.html.erb
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Geoffrey <79559478+CYBRXT@users.noreply.github.com>
* Following Dosu's comment on my issue for consistency
---------
Signed-off-by: Geoffrey <79559478+CYBRXT@users.noreply.github.com>
Co-authored-by: Geoffrey <geoffrey@github.worker>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Add Quick Categorize Wizard (iteration 1)
Adds a step-by-step wizard for bulk-categorizing uncategorized transactions
and optionally creating auto-categorization rules, reducing friction after
connecting a new bank account.
New files:
- Transaction::Grouper abstraction + ByMerchantOrName strategy (groups by
merchant name when present, falls back to entry name; sorted by count desc)
- Transactions::CategorizesController (GET show / POST create)
- Wizard view at app/views/transactions/categorizes/show.html.erb
- Stimulus categorize_controller.js (Enter-key-to-select-first)
- Tests for grouper and controller
Modified files:
- routes.rb: resource :categorize inside namespace :transactions
- transactions_controller.rb: expose @uncategorized_count to index
- transactions/index.html.erb: Categorize (N) button in header
- family.rb: uncategorized_transaction_count query
- rules_controller.rb: return_to param support for wizard → rule editor flow
- rules/_form.html.erb, rules/new.html.erb: pass return_to through form
- i18n: categorizes show/create keys + rules.create.success
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Quick Categorize Wizard — iteration 2 polish
Six improvements from live testing:
- Breadcrumb: Home > Transactions > Categorize
- Layout: category picker + confirmation dialog above transaction list
- Inline confirmation dialog: clicking a category pill shows a <dialog>
summarising what will happen (N transactions → category, rule if checked)
with Confirm and Cancel buttons — no redirect to rule editor
- Direct rule creation: rule created with active: true in the controller
instead of redirecting to the rule editor; revert return_to plumbing from
RulesController, rules/_form, rules/new, rules/en.yml
- Individual row assignment: per-row category <select> submits via
PATCH /transactions/categorize/assign_entry and removes the row via
Turbo Stream (assign_entry action + route)
- Enter key guard: selectFirst only fires when exactly 1 pill is visible
after filtering
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Quick Categorize Wizard — iteration 3 reliability fixes and UX polish
- Fix Stimulus controller not loading: remove invalid `@hotwired/turbo` named
import (not in importmap); use global `Turbo.renderStreamMessage` instead
- Fix Enter key submitting form with wrong category when search field is
unfocused: move keydown listener to document so it fires regardless of focus
- Prevent Enter from submitting when multiple categories are visible
- Clear search filter after bulk category assignment (pill click or Enter),
but not after individual row dropdown assignment
- Update group transaction count and total amount live as entries are assigned
via row dropdown or partial bulk assignment
- Add turbo frames for remaining count and group summary so they update
without a full page reload
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Quick categorization polish
* refactoring
* Remove unused GROUPS_PER_BATCH constant, fix ERB self-closing tags
Wizard only ever uses one group at a time so limit: 1 is correct and
more honest than fetching 20 and discarding 19. ERB linter fixes are
whitespace/void-element corrections with no functional change.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Move Categorize button into ... menu on transactions index
Reduces header clutter by putting it in the overflow menu at the bottom,
where it only appears when there are uncategorized transactions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Scope categorize wizard to accessible entries only
Fixes a security issue where users with restricted account access via
account sharing could view and categorize transactions from accounts
they cannot access through normal transaction flows.
- Pass Current.accessible_entries to Transaction::Grouper so the wizard
only displays groups from accounts the user can see
- Use Current.accessible_entries on all write paths in create and
assign_entry, matching the pattern in TransactionCategoriesController
- Refactor Grouper to accept an entries scope instead of a family object,
keeping authorization concerns in the controller
- Add tests verifying inaccessible entries are hidden from the wizard
and cannot be categorized via forged POST/PATCH params
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Clamp position param to >= 0 to guard against negative offset
Prevents ArgumentError from Array#drop when a negative position is
passed via a tampered query string or form value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Surface rule creation failure and add accessible names to entry row
- Capture Rule.create_from_grouping! return value; set flash[:alert] when
nil so users who checked "Create Rule" know it wasn't created (e.g. a
duplicate already exists); stream the notification for partial updates
- Add aria-label to the per-row checkbox and category select in
_entry_row so screen readers can identify which transaction each
control belongs to
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Localize breadcrumb labels in categorizes controller
Follows the pattern used by FamilyExportsController and ImportsController.
Adds 'transactions' and 'categorize' keys to the breadcrumbs locale file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add error handling to categorize controller fetch calls
Check response.ok before parsing the body and add .catch handlers
so network failures and non-2xx responses are logged rather than
silently swallowed. On assignment failure the per-row select is
reset to empty so the user can retry.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Scope preview_rule to accessible entries only
Entry.uncategorized_matching now accepts an entries scope instead of a
family object, matching the same pattern used for Transaction::Grouper.
The preview_rule action passes Current.accessible_entries so rule
previews respect account sharing permissions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Scope remaining count to accessible entries
Adds Entry.uncategorized_count(entries) following the same pattern as
uncategorized_matching. Replaces all three uses of
Current.family.uncategorized_transaction_count in the categorize
controller so the remaining-count badge reflects only the transactions
the current user can actually access and categorize.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Comments got separated from their function
* Remove quick-categorize-wizard dev notes
This was a planning document used during development, not intended
for the final branch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Recompute remaining entries from server state after writes
Adds uncategorized_entries_for helper that reloads remaining entries
from the DB with a category_id IS NULL filter after each write, so
the partial-update Turbo Stream reflects server-side state rather than
trusting the client-provided remaining_ids. This handles the case where
a concurrent request has categorized one of the remaining entries
between page render and form submit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Rename create_from_grouping! to create_from_grouping
The method rescues RecordInvalid and returns nil, which contradicts
the bang convention. Dropping the ! correctly signals that callers
should check the return value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Clamp offset in grouper to guard against negative values
The controller already clamps position before passing it as offset,
but clamping in the grouper itself prevents ArgumentError from
Array#drop if the grouper is ever called directly with a negative offset.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* collect transaction stats after sync in EnableBanking provider
* update logic to follow the pattern used by other bank providers
* fix n+1 issue
* fix N+1 issues
* feat: improve QIF import date format selection
- Added a reusable date format auto-detection method.
- Show a live preview of the first parsed date that updates client-side
as the user changes the dropdown selection, via a new
qif-date-format Stimulus controller.
- Show an error alert and disable the submit button when no supported
date format can parse the file's dates.
* A few polishing fixes:
- Missing return on redirects
Stale REASONABLE_DATE_RANGE constant.
- Replaced the frozen constant with a class method
Bare inline rescue — Replaced Date.strptime(s, fmt) rescue nil with an explicit begin/rescue catching.
- save!(validate: false) in controller — Changed to update_column(:column_mappings, ...) in qif_category_selections_controller.rb:22, matching the pattern used in detect_and_set_qif_date_format!.
- Unescaped JSON in HTML attribute — Replaced the raw <div> with tag.div ... do block in show.html.erb:16, letting Rails properly escape the data attribute value.
* fix: address review feedback for QIF date format feature
- Add missing `return` after redirect for non-QIF imports
- Pass date_format to parse_opening_balance in will_adjust_opening_anchor?
- Return empty array when no usable date sample exists for format preview
- Add sr-only label to date format select for accessibility
- Consolidate duplicate try_parse_date/parse_qif_date into single method
- Remove misleading ambiguity scoring comment from detect_date_format
- Skip redundant sync_mappings when date format already triggered a sync
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Use %{product_name} interpolation in locale strings
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Added server-side validation to Category model to enforce 6-digit hex format for colors.
- Added HTML pattern attribute to category form for client-side validation.
- Updated tests to cover validation and fixed existing tests using shorthand hex colors.
* Fix missing translation fallback for account subtypes in LunchFlow and Mercury
The translate_subtypes lambda called t() without a default: fallback, causing
"Translation missing" for the ~39 investment subtypes not in the locale files.
Now falls back to the :long label from the model's SUBTYPES hash (e.g. "457(b)",
"SEP IRA", "UGMA Custodial Account"), matching the pattern used by Accountable#subtype_label_for.
https://claude.ai/code/session_01HBJzNkvpky8mKrLkzv83Mm
* Default to single-column dashboard
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Fix NoMethodError when entry has nil entryable
Guard against orphaned entries where the entryable record has been
deleted but the entry still exists, preventing a crash on the
account show page.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add dependent: :destroy to Entryable has_one :entry
The polymorphic has_one :entry association lacked a dependent option,
meaning if a Transaction/Trade/Valuation was ever deleted directly
(bypassing the Entry), the Entry would be left orphaned with a nil
entryable — causing NoMethodError in the activity feed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add nil entryable guard to _split_group.html.erb
Same defensive check as _entry.html.erb for consistency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Record dividends and interest as Trades in investment accounts
All investment income (dividends and interest) is now modeled as a
Trade with qty: 0 and price: 0, keeping security_id NOT NULL on trades
intact. Dividends require a security; interest falls back to a
per-account synthetic cash security (kind: "cash", offline: true) when
none is selected, matching how brokerages handle uninvested cash
internally.
- Add `kind` column to securities ("standard" | "cash") with DB check
constraint; `Security.cash_for(account)` lazily finds or creates the
synthetic cash security; `scope :standard` excludes synthetic
securities from user-facing pickers
- Trade::CreateForm: new `dividend` type (security required); `interest`
now creates a Trade instead of a Transaction
- Trade form: Dividend and Interest in the type dropdown with a security
combobox (required for dividend, optional for interest)
- transactions table: untouched
* UI fixes
* HealthChecker — both scopes now chain .standard to exclude cash securities from provider health checks.
DB query moved to model — Account#traded_standard_securities in app/models/account.rb, view uses account.traded_standard_securities.
DRY income creation — create_income_trade(sec:, label:, name:) extracted as shared private method; create_dividend_income and create_interest_income delegate to it.
show.html.erb blocks merged — single unless trade.qty.zero? block covers qty/price/fee fields.
Test extended — assert_response :unprocessable_entity added after the assert_no_difference block.
* Hide cash account ticker from no-security trade detail
* Fix CodeRabbit review issues from PR #1311
- Remove duplicate YAML keys in translation files (de, es, fr)
- Add error handling for security resolution in create_dividend_income
- Extract income trade check to reduce duplication in header template
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* Include holdings in dividend/interest security picker
The security picker for dividend/interest trades should include all securities
in holdings, not just those with trade history. This fixes the issue where
accounts with imported holdings (e.g., SimpleFIN) but no trades would have an
empty picker and be unable to record dividends.
Uses UNION to combine securities from both trades and holdings.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
* scope picker to holdings only (a trade creates a holding anyway)
---------
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
Add an optional fee field (decimal, precision: 19, scale: 4) to trades.
Fee is included in the total amount calculation (qty * price + fee) for
both create and update flows. The fee field appears on both the create
and edit forms, defaults to 0, and auto-submits like other trade fields.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: deduplicate Enable Banking API transactions with different entry_reference IDs (#954)
Enable Banking API sometimes returns the same logical transaction multiple
times with different entry_reference values in a single response. This causes
duplicate entries because the existing ID-based deduplication treats them as
distinct transactions.
Add content-based deduplication that compares (date, amount, currency,
creditor, debtor, remittance_information, status) to detect and remove these
API-level duplicates before storing them. The first occurrence is kept.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add Enable Banking processor and importer deduplication tests (#954)
Add tests for:
- EnableBankingEntry::Processor: verifies entry_reference fallback for
external_id, idempotent re-processing, string key handling
- EnableBankingItem::Importer: verifies content-based deduplication removes
API-level duplicates while preserving legitimate distinct transactions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: handle nil values in remittance_information array for dedup key (#954)
Call compact and map(&:to_s) before sort.join when remittance_information
is an array, preventing ArgumentError when it contains nil elements.
Also document the known limitation of content-based deduplication
collapsing genuinely distinct identical transactions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add coverage for nil values in remittance_information array (#954)
Verify that deduplication handles remittance_information arrays containing
nil elements without raising ArgumentError, and correctly treats arrays
with different nil positions but same non-nil content as duplicates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prefer transaction_id over content-based dedup to preserve legit duplicates (#954)
When transaction_id is present, use it as the dedup key instead of falling
back to content-based deduplication. This preserves legitimately distinct
transactions with identical content (e.g. two laundromat payments of the
same amount on the same day) while still deduplicating the Enable Banking
duplicate entry_reference issue when transaction_id is nil.
Addresses review feedback from @jjmata about legitimate duplicate
transactions being incorrectly collapsed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use composite key for dedup instead of transaction_id alone (#954)
Per the Enable Banking API docs, transaction_id is not guaranteed to be
unique. Include it as one component of the composite content key rather
than using it as the sole dedup criterion. This preserves transactions
with non-unique transaction_ids but different content, while still
deduplicating true API-level duplicates.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add value_date fallback coverage for dedup key (#954)
build_transaction_content_key falls back to value_date when booking_date
is absent. This test exercises that path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: document known limitation of content-based dedup (#954)
When transaction_id is nil for both transactions, pure content comparison
applies, which could theoretically collapse two genuinely distinct
transactions with identical fields. Document this trade-off inline for
future maintainers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add credit_debit_indicator to dedup composite key (#954)
transaction_amount.amount is always positive in the Enable Banking API,
with direction encoded separately in credit_debit_indicator (CRDT/DBIT).
Without it in the composite key, a payment and a same-day refund of the
same amount to the same merchant would produce identical keys, silently
dropping one transaction.
- Add credit_debit_indicator to build_transaction_content_key
- Add test for payment + same-day refund scenario
- Update docstring to document the rationale
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Show inflow/outflow totals when filtering by transfers
When filtering transactions by "Transfer" type, the summary bar previously
showed $0 for both Income and Expenses because transfers were excluded from
those sums. Now computes transfer inflow/outflow in the same SQL pass and
switches labels to "Inflow"/"Outflow" when transfer amounts are non-zero.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add mixed filter comment and transfer-only test coverage
Document the intentional mixed filter behavior where transfer amounts
are excluded from the summary bar when non-transfer types are present.
Add test exercising Inflow/Outflow label switching for transfer-only results.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* third party provider scoping
* Simplify logic and allow only admins to mange providers
* Broadcast fixes
* FIX tests and build
* Fixes
* Reviews
* Scope merchants
* DRY fixes
* Perf: Index Balance::SyncCache lookups by date to eliminate O(N×D) scans
Each call to get_holdings(date) and get_entries(date) previously did a
linear scan over the full converted_holdings / converted_entries arrays.
The balance calculators call these once per day across the full account
history, making the overall complexity O(N×D) where N is the total number
of holding/entry rows and D is the number of days in the account history.
For a typical investment account (20 securities, 2 years of history):
- Holdings: 20 × 730 = 14,600 rows
- Balance loop: 730 date iterations
- Comparisons: 14,600 × 730 ≈ 10.7 million per materialise run
This change builds a hash index (grouped by date) once on first access and
reuses it for all subsequent lookups, reducing per-call complexity to O(1).
Total complexity becomes O(N) — load once, look up cheaply.
Observed wall-clock improvement on a real account: ~36 s → ~5 s for a full
Balance::Materializer run. The nightly sync benefits equally.
No behavioural change: get_holdings, get_entries, and get_valuation return
identical data — they are now just fetched via a hash key rather than a
repeated array scan.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix: Return defensive copy from get_holdings to prevent cache mutation
get_holdings was returning a direct reference to the internal cached
array from holdings_by_date. A caller appending to the result (e.g.
via <<) would silently corrupt the cache for all subsequent date
lookups in the same materialise run.
Use &.dup to return a shallow copy of the group array. Callers only
read from the result (sum, map, etc.) so this has no behavioural
impact and negligible performance cost.
get_entries is already safe — Array#select always returns a new array.
get_valuation returns a single object, not an array, so no issue there.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Remove unnecessary dup in get_holdings for consistency
No caller mutates the returned array (only .sum is called), so the
defensive copy is unnecessary overhead. This aligns get_holdings with
get_entries and get_valuation which also return cached references directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove "value" from the market_value fallback chain in the SimpleFIN
HoldingsProcessor and add it to the cost_basis fallback chain instead.
Some brokerages (Vanguard, Fidelity) use "value" to represent cost basis,
causing the system to display average cost per share as the current price
and show massive phantom losses.
https://claude.ai/code/session_01V2gC6BPT3sF7Hu4XQgUQT4
Co-authored-by: Claude <noreply@anthropic.com>
* Reorganize import UI with Financial Tools / Raw Data tabs
Split the flat list of import sources into two tabbed sections using
DS::Tabs: "Financial Tools" (Mint, Quicken/QIF, YNAB coming soon) and
"Raw Data" (transactions, investments, accounts, categories, rules,
documents). This prepares for adding more tool-specific importers
without cluttering the list.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* Fix import controller test to account for YNAB coming soon entry
The new YNAB "coming soon" disabled entry adds a 5th aria-disabled
element to the import dialog.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* Fix system tests to click Raw Data tab before selecting import type
Transaction, trade, and account imports are now under the Raw Data tab
and need an explicit tab click before the buttons are visible.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* feat: Add bulk import for NDJSON export files
Implements an import flow that accepts the full all.ndjson file from data exports,
allowing users to restore their complete data including:
- Accounts with accountable types
- Categories with parent relationships
- Tags and merchants
- Transactions with category, merchant, and tag references
- Trades with securities
- Valuations
- Budgets and budget categories
- Rules with conditions and actions (including compound conditions)
Key changes:
- Add BulkImport model extending Import base class
- Add Family::DataImporter to handle NDJSON parsing and import logic
- Update imports controller and views to support NDJSON workflow
- Skip configuration/mapping steps for structured NDJSON imports
- Add i18n translations for bulk import UI
- Add tests for BulkImport and DataImporter
* fix: Fix category import and test query issues
- Add default lucide_icon ("shapes") for categories when not provided
- Fix valuation test to use proper ActiveRecord joins syntax
* Linter errors
* fix: Add default color for tags when not provided in import
* fix: Add default kind for transactions when not provided in import
* Fix test
* Fix tests
* Fix remaining merge conflicts from PR 766 cherry-pick
Resolve conflict markers in test fixtures and clean up BulkImport
entry in new.html.erb to use the _import_option partial consistently.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* Import Sure `.ndjson`
* Remove `.ndjson` import from raw data
* Fix support for Sure "bulk" import from old branch
* Linter
* Fix CI test
* Fix more CI tests
* Fix tests
* Fix tests / move PDF import to first tab
* Remove redundant title
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add AI merchant enhancement and dedup
* Enhancements
Add error if job is already running
add note that we also merge merchants
* Allow updating provider website
* Review fixes
* Update provider_merchant.rb
* Linter and fixes
* FIX transaction quick menu modal
* Initial split transaction support
* Add support to unsplit and edit split
* Update show.html.erb
* FIX address reviews
* Improve UX
* Update show.html.erb
* Reviews
* Update edit.html.erb
* Add parent category to dialog
* Update en.yml
* Add UI indication to totals
* FIX ui update
* Add category select like rest of app
* Add split ui
* Add settings configuration for split transactions
- Adds a new settings section for appearance changes
- Also adds extra checks for delete and API calls
- Also adds checks for parent/child changes
* fixes
- split transactions dark mode fix
- add split transactions to context menu
* Update entry.rb
1. New validation split_child_date_matches_parent — prevents saving a split child with a date different from its parent. This is the root-cause fix that
protects all flows at once.
2. Bulk update guard — bulk_update! now strips :date from attributes when processing split children, preventing the validation from raising and silently
skipping the date change instead.
* N+1 fix for split_parent?
* Update entry.rb
Problem: In bulk_update!, when a split child has :date removed from attrs (line 432) and the remaining attrs is empty (e.g., the bulk update only
changed the date), entry.update! {} still ran as a no-op. But lock_saved_attributes! and mark_user_modified! at lines 443-444 executed unconditionally,
incorrectly marking untouched split children as user-modified and opting them out of future syncs.
Fix:
1. Added a changed flag to track whether any actual modification happened
2. Wrapped entry.update! in an if attrs.present? check so no-op updates are skipped
3. Gated lock_saved_attributes! and mark_user_modified! behind if changed, so they only run when the entry was actually modified (either via attribute
update or tag update)
* fixes
1. Indentation in show.html.erb Settings section — The split button block and delete block had extra indentation making them appear nested inside guard
blocks they weren't part of. Fixed to match actual nesting.
2. Skip @split_parents query when grouping is off — The controller now only loads split parent entries when show_split_grouped? is true, saving a query
with joins when the feature is disabled.
The "transaction merchant" condition filter used `family.assigned_merchants`
which only returned merchants already assigned to a transaction. This meant
newly created merchants wouldn't appear in the rule condition dropdown until
manually assigned to a transaction.
Changed to `family.available_merchants` which includes all family merchants
(even unassigned ones), provider merchants on transactions, and recently
unlinked merchants — consistent with the transaction form merchant selector.
Fixes#1197
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Initial split transaction support
* Add support to unsplit and edit split
* Update show.html.erb
* FIX address reviews
* Improve UX
* Update show.html.erb
* Reviews
* Update edit.html.erb
* Add parent category to dialog
* Update en.yml
* Add UI indication to totals
* FIX ui update
* Add category select like rest of app
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Add conditional migration for vector_store_chunks table
Creates the pgvector-backed chunks table when VECTOR_STORE_PROVIDER=pgvector.
Enables the vector extension, adds store_id/file_id indexes, and uses
vector(1024) column type for embeddings.
* Add VectorStore::Embeddable concern for text extraction and embedding
Shared concern providing extract_text (PDF via pdf-reader, plain-text as-is),
paragraph-boundary chunking (~2000 chars, ~200 overlap), and embed/embed_batch
via OpenAI-compatible /v1/embeddings endpoint using Faraday. Configurable via
EMBEDDING_MODEL, EMBEDDING_URI_BASE, with fallback to OPENAI_* env vars.
* Implement VectorStore::Pgvector adapter with raw SQL
Replaces the stub with a full implementation using
ActiveRecord::Base.connection with parameterized binds. Supports
create_store, delete_store, upload_file (extract+chunk+embed+insert),
remove_file, and cosine-similarity search via the <=> operator.
* Add registry test for pgvector adapter selection
* Configure pgvector in compose.example.ai.yml
Switch db image to pgvector/pgvector:pg16, add VECTOR_STORE_PROVIDER,
EMBEDDING_MODEL, and EMBEDDING_DIMENSIONS env vars, and include
nomic-embed-text in Ollama's pre-loaded models.
* Update pgvector docs from scaffolded to ready
Document env vars, embedding model setup, pgvector Docker image
requirement, and Ollama pull instructions.
* Address PR review feedback
- Migration: remove env guard, use pgvector_available? check so it runs
on plain Postgres (CI) but creates the table on pgvector-capable servers.
Add NOT NULL constraints on content/embedding/metadata, unique index on
(store_id, file_id, chunk_index).
- Pgvector adapter: wrap chunk inserts in a DB transaction to prevent
partial file writes. Override supported_extensions to match formats
that extract_text can actually parse.
- Embeddable: add hard_split fallback for paragraphs exceeding CHUNK_SIZE
to avoid overflowing embedding model token limits.
* Bump schema version to include vector_store_chunks migration
CI uses db:schema:load which checks the version — without this bump,
the migration is detected as pending and tests fail to start.
* Update 20260316120000_create_vector_store_chunks.rb
---------
Co-authored-by: sokiee <sokysrm@gmail.com>
