Commit Graph
19 Commits
Author SHA1 Message Date
GFRandGerald 07ce130405 fix: respect SURE_IMPORT_MAX_NDJSON_SIZE_MB in Sure import GUI upload (#3111)
The GUI upload paths (imports_controller#create_sure_import and
Import::UploadsController#update_sure_import_upload) checked file
size against the hardcoded SureImport::MAX_NDJSON_SIZE constant
instead of SureImport.max_ndjson_size, so self-hosted admins raising
SURE_IMPORT_MAX_NDJSON_SIZE_MB had no effect on the GUI — only the
API upload paths honored it. Removes the now-unused constant.

Fixes #3010.

Co-authored-by: Gerald <248542187+gfr-free@users.noreply.github.com>
2026-08-22 04:33:54 +02:00
Juan José MataandClaude Opus 5 8ca65ffd27 Reconcile PDF statement imports against transactions that already exist (#3105)
* Reset to main, keeping only the account matcher improvements

Backs out the LLM-driven reconciliation work (PR #1382's approach and the two
commits hardening it). That approach compares whole-statement aggregates, which
is all-or-nothing: when 18 of 20 transactions have already synced the totals
disagree, the import proceeds, and 18 duplicates are created. Reconciliation is
a row-level problem and belongs in the import path, where TransactionImport
already solves it via Account::ProviderImportAdapter.

Kept from that work, because it stands on its own:

- AccountMatcher gains a hint-based class-level entry point so callers without
  an AccountStatement row can score against the same rules. The instance path
  used by AccountStatement#assign_account_match is unchanged.
- It also refuses to guess between equally-confident candidates rather than
  letting max_by take whichever the scan reached first. Account names are not
  unique within a family, so that tie was reachable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

* Reconcile PDF statement imports against transactions that already exist

Reconciliation is a row-level problem. Comparing whole-statement totals is
all-or-nothing: when 18 of 20 transactions have already synced the totals
disagree, the import proceeds, and 18 duplicates are created. This matches each
extracted transaction against what the account already holds, so only genuinely
new transactions are ever offered for import and the rest are marked reconciled.

TransactionImport#import! already does this for CSV via
Account::ProviderImportAdapter. PdfImport#import! called it zero times and built
one Transaction per row unconditionally -- the only import path in the repo with
no duplicate protection.

Reconciliation state follows Quicken's uncleared / cleared / reconciled, but only
the last state is stored:

- "Cleared" means the institution acknowledged the transaction, which is exactly
  what entries.source and entries.external_id already record. It stays accurate
  on its own, because the adapter stamps both onto a manual entry when a provider
  transaction claims it, so a hand-entered transaction that later appears in a
  download becomes cleared with no extra bookkeeping. Deriving it also keeps it
  non-editable, which is right: it is a fact about provenance, not an opinion.
- "Reconciled" means a statement was matched against the transaction. Nothing can
  derive that, so entries gains reconciled_at and reconciled_by_statement_id. It
  is a judgement, so it can be set and unset, and it survives the statement being
  deleted (the FK nullifies, the timestamp stays).

Matching:

- find_duplicate_transaction grows include_provider_entries, which is what makes
  this work for Provider-backed accounts -- the existing where(external_id: nil)
  filter hid synced transactions from every import path, so this gap affected CSV
  imports equally. Default stays false: provider sync must not claim another
  provider's entry.
- It also grows date_window, because a statement's posting date routinely differs
  by a day or two from the date a provider recorded. Nearest date wins.
- Name is deliberately not matched on: statement descriptions and provider names
  for the same transaction rarely agree. The adapter makes the same choice for
  sync.
- Candidates are built as real Import::Row objects so matching uses the same
  signed_amount and date_iso the import itself would write, rather than a second
  interpretation of signage that could drift.
- Matching is per-account, so with no account assigned every row is offered and
  re-judged on assignment; reassigning also releases the previous account's
  reconciliations.
- A row whose date or amount will not parse is offered for import rather than
  dropped, so nothing disappears silently.
- import! re-checks at publish, since a sync can land between review and publish,
  and new transactions are born reconciled: the statement is their evidence.

Provider-backed accounts are now offered in the import target picker. The
manual-only restriction existed because importing into a synced account would
duplicate what sync brought in, which is precisely what this removes.

Also fixes a bug this uncovered on main: extract_transactions stored the
extractor's symbol-keyed hash, while every reader digs with strings. jsonb keeps
the hash as assigned until reload and ProcessPdfJob never reloads, so
has_extracted_transactions? was false and PDF imports generated zero rows. The
existing tests missed it -- one uses a YAML fixture, the other stubs the
extractor with string keys.

Supersedes #1382. Refs #1379.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

* Fix three review findings in statement reconciliation

All three confirmed against the code before fixing.

Publish-time recheck consumed the same entry twice. import! started its
exclusion list empty, so a statement carrying two same-amount transactions
against an account holding only one would re-match the surviving row against the
entry row generation had already consumed -- silently dropping a genuinely new
transaction instead of creating it. Seed the exclusions with the entries this
statement already reconciled; newly synced entries are still caught, since only
already-reconciled ones are excluded.

A regeneration that emptied the row set left the import stranded. In the normal
upload flow the account is assigned after extraction, so assign_account!
regenerates -- and if everything then reconciled, rows_count went to zero while
status stayed pending. _pdf_import.html.erb renders pending-with-no-rows as the
processing screen, and process_with_ai_later cannot restart because
ai_processed? is already true, so the import was stuck with no way forward.
Status now follows the same rule ProcessPdfJob applies after initial processing,
in both directions: no rows completes it, rows returning sends it back to
pending. Guarded by data_committed? so a published import is never reopened.

Unevaluatable rows went only to the Rails log. AGENTS.md asks for
DebugLogEntry.capture on recoverable import failures so they surface in
/settings/debug with structured context. Capture family, account, import,
statement, row number and the raw date/amount that would not parse.

Adds regression coverage for each.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

* Fix row regeneration collision and over-broad reconciliation release

test_unit caught one error in 6462 tests, and it was real.

Row regeneration collided on the second call. insert_all! bypasses
ActiveRecord, so the rows association is never populated with what it wrote.
Calling generate_rows_from_extracted_data twice on the same in-memory record --
which assign_account! now does after ProcessPdfJob has already generated once --
made rows.destroy_all clear a stale empty collection, delete nothing, and then
collide on (import_id, source_row_number). Reload before destroying, and reset
the association after inserting so sync_mappings and the view read what was
actually written.

Releasing reconciliations was scoped to the statement, not the account. A
statement is evidence for exactly one account at a time but can back more than
one import, so reassigning an account cleared reconciliations another account
still relied on. Scoped to the account being moved away from; a blank scope
releases nothing, which is correct because nothing is reconciled while no
account is assigned.

The second finding was raised by CodeRabbit. Its other flagged risk -- entries
being marked reconciled before the import is published -- is deliberate and
stays: the statement is the evidence, reconciling is reversible via
unmark_reconciled!, and deferring it to publish would leave a fully reconciled
statement with nothing to publish and therefore nothing ever marked.

Adds regression coverage for both.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

* Address review nitpicks: model validation, scope placement, mock style, lock-safe DDL

All four checked against the repo's own conventions before applying.

Mirror the check constraint as a model validation. Assigning
reconciled_by_statement without reconciled_at raised StatementInvalid rather
than a validation error. CLAUDE.md Convention 5 asks for exactly this pairing --
the constraint in the database, an ActiveRecord validation for form-friendly
errors.

Stop hijacking the pending-scope comment. The reconciliation scopes were
inserted directly under "Pending transaction scopes", so that header read as
documentation for them and the provider note below read as a continuation of
reconciled_by. Given the reconciliation scopes their own header.

Use OpenStruct for the provider response double, per "Always prefer OpenStruct
when creating mock instances". Verified OpenStruct.new(success?: true) responds
to success?, and ostruct is already a dependency used elsewhere in test/.

Make the migration lock-safe. entries is the largest table in the app: both
indexes now build concurrently, and the check constraint is added unvalidated
then validated separately so VALIDATE takes only SHARE UPDATE EXCLUSIVE instead
of holding ACCESS EXCLUSIVE for a full scan. This follows existing practice --
13 migrations already use disable_ddl_transaction! and 11 use algorithm:
:concurrently, with add_offline_reason_to_securities combining add_column and a
concurrent index in one migration exactly like this. The suggested follow-up
migration for validation was not needed: validating in the same non-
transactional migration gets the same lock behavior without a second file, and
the repo has no validate: false precedent in 400 migrations.

schema.rb is unchanged: a validated constraint and a concurrently-built index
dump identically.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

* Add the two review-requested regression tests

Covers the invalid-amount half of "malformed rows are offered, not dropped" --
the existing test only exercised an unparseable date. Asserts the raw value is
stored verbatim rather than coerced to 0, so the review step shows the user what
the statement actually said.

Also covers the Entry validation added in f5ba646: assigning
reconciled_by_statement without reconciled_at must fail model validation rather
than reaching chk_entries_reconciled_at_present_when_statement_set and raising
StatementInvalid.

Both requested by CodeRabbit on f5ba646.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

* Assert both reconciliation fields when an account is reassigned

reconciled? only reads reconciled_at, so the test proved the state was cleared
but not that the statement evidence went with it, nor that the sibling account
kept its own. Entry#unmark_reconciled! clears the pair, so assert the pair.

Raised by CodeRabbit on eac82cf.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

* Release reconciliation on revert and guard account reassignment

Two correctness findings from review.

Import#revert destroyed the import's own entries but nothing else, so a
statement import left stale evidence behind: entries it had only *matched*
kept reconciled_at and reconciled_by_statement_id pointing at a statement
that no longer claimed them. Worse, a statement that reconciled every line
carries zero rows, so revert returned it to pending with rows_count 0 --
the exact combination the pdf import view renders as the processing screen,
with no way to regenerate rows or re-trigger extraction.

Import#revert now calls two hooks inside its transaction: revert_derived_state!
for state a subclass keeps outside its own rows and entries, and
status_after_revert for where the record lands. The base behavior is unchanged.
PdfImport releases its reconciliations, re-judges every statement line against
what the account actually holds now, and finishes as complete when there is
nothing left to offer.

PdfImport#assign_account! had no guard against an already-published import.
A back-button or replayed PATCH ran release_reconciliations! and
generate_rows_from_extracted_data unconditionally, releasing evidence and
destroying the rows that documented what was published while the created
entries stayed put -- only refresh_status_after_regeneration! checked
data_committed?. It now takes the row lock, refuses when the import has
committed data or a job owns the record, and returns false so the controller
can explain rather than report a save that did not happen. An import that
reconciled every line is still re-targetable: it is complete, but committed
nothing of its own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

* Show what became of each statement transaction

Row-level matching made the import's outcome invisible. A statement whose
lines were all already on file finishes with rows_count 0 and renders the
generic "Document analyzed" screen, which cannot be told apart from a
statement nothing was extracted from. The user is told the import is done
and nothing else.

Two counts also became wrong rather than merely absent. The ready-for-review
screen labels rows_count as "Transactions Extracted", but after matching that
is the *unmatched* count: a 20-line statement against an account holding 18 of
them read "2 transactions extracted". ready_for_review_description made the
same claim in prose.

Adds a summary dialog at GET /imports/:id/summary, linked from both the
ready-for-review and complete screens, breaking the statement down into what
was found, what was already recorded, what was imported, and what is still
waiting. The counts are derived from the entries rather than stored, so they
stay true if a later sync or edit changes the picture.

Two of them need care. Entries this import creates are born reconciled, so
already_recorded_count has to exclude them or it double-counts what the
account genuinely already had. And publishing does not destroy rows -- they
remain as the record of what was written -- so awaiting_review_count reports
zero once the data is committed rather than repeating rows_count.

The complete screen now explains a fully reconciled import instead of
claiming to have found something, and the extracted count reports the
statement's real size with the matched count beside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

* Memoize the import outcome counts

Rendering the summary dialog issued roughly fifteen queries for four
numbers. already_recorded_count, imported_count and awaiting_review_count
are each read two or three times per template, every read is its own
COUNT, reconciled_anything? adds another by calling already_recorded_count
internally, and awaiting_review_count consults data_committed? -- two more
EXISTS queries -- on every invocation.

Memoizing on the model rather than assigning locals in the template fixes
the review screen too, which reads the same counts, and keeps the
arithmetic out of the view.

These report a finished outcome for display. Anything that re-judges the
import recomputes from the entries directly, so a value cached for the life
of the request is what the callers want.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KvWSJotDiqP6qSR7qEiC8Q

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-22 03:14:03 +02:00
Guillem Arias Fauste 5105752bbd feat(sync): family-facing cancellation for syncs, imports, and exports (#2685)
* feat(sync): family-facing cancellation for syncs, imports, and exports

Users had no way to stop or recover any background operation: a
mistaken "Sync all" runs to completion, and an import or export whose
job died (hard worker kills lose in-flight Sidekiq jobs) wedges with a
spinner forever.

Sync cancellation (cooperative — nothing is ever killed):

- New syncs.cancel_requested_at column. Only the cancelled sync
  carries the flag: pending descendants are marked stale immediately
  (their queued jobs no-op via the existing may_start? guard), while
  descendants whose jobs are already executing finish their work
  honestly.
- Family::Syncer stops fanning out child syncs once the flag is set
  (fresh read per iteration — the flag comes from the web process).
- Finalization resolves a cancel-requested sync to stale instead of
  completed, which also skips post-sync (transfer matching, rules,
  broadcasts) via the existing stale gate.
- The `visible` scope excludes cancel-requested syncs, so spinners
  clear immediately and — fixing a latent bug this feature would have
  amplified — sync_later no longer piggybacks a new sync request onto
  a dying sync it would silently swallow.
- Cancel button appears next to "Sync all" on the accounts page while
  a family sync is visible. SyncsController#cancel scopes through
  Sync.for_family with resource_owner, so cross-family ids 404 and
  account-level syncs respect per-user account access.

Stuck import/export self-service:

- Import#force_fail! / FamilyExport#force_fail!: allowed only once the
  record has been idle past PRESUMED_LOST_AFTER (1 hour — dwarfs any
  legitimate run), and applied inside with_lock with a status
  re-check, so a job finishing between page render and button click
  wins. Imports fail into the existing retry path (reverting ->
  revert_failed keeps the revert retryable); PdfImports release their
  processing claim back to pending; exports fail so a new one can be
  created.
- "Mark as failed" buttons appear on the imports/exports index rows
  only when a record is presumed lost, behind the pages' existing
  permission gates (statement-import permission for imports, admin
  for exports).

* fix(sync-cancel): cascade pending cancels, guard late finalizers, scope provider syncs

Review feedback on #2685 (CodeRabbit, Codex):

- request_cancel! now cascades finalization for pending syncs too: a
  pending child resolved to stale never runs its job, so nothing else
  would ever call finalize_if_all_children_finalized — its waiting
  parent hung in syncing until the 24h sweep (CodeRabbit critical)
- SimplefinItem::Syncer#mark_completed re-reads the sync under a row
  lock and skips finalization once cancellation was requested or the
  row went terminal — its in-memory copy predates the cancel, and the
  unguarded complete! (plus the raw status fallback) resurrected a
  cancelled sync and re-ran post-sync (Codex)
- Cancelling provider-item syncs now requires admin: for_family's
  resource_owner only scopes the Account branch, so a restricted member
  could cancel admin-managed provider syncs spanning accounts they
  cannot see. Family- and account-level syncs stay member-cancellable,
  matching the buttons the UI shows (Codex)
- Lost-import error copy moved behind i18n
  (imports.errors.presumed_lost), resolved at call time (CodeRabbit)
- Tests: pending-child cancel finalizes the parent; late provider
  complete! cannot resurrect a cancelled sync; provider-sync
  cancellation is admin-only

* fix(sync-cancel): capture the skipped-finalization case via DebugLogEntry

CodeRabbit round-2: the mark_completed skip (cancelled/terminal sync)
is support-relevant — record it in the super-admin debug UI with the
family and provider attached instead of a raw Rails.logger line.
category: provider_sync, matching the other provider syncers.
2026-07-17 06:57:54 +02:00
ghost 6ea931f3fe feat(imports): add YNAB CSV import (#2361)
Adds YnabImport (mirroring ActualImport) for YNAB "Export budget" register CSVs:

- Amount — combines the split Outflow/Inflow columns into a single signed amount
  (inflow - |outflow|), stripping currency symbols and thousands separators. A
  single signed Amount column takes precedence when present.
- Category — resolves across export shapes: the combined "Category Group/Category"
  column, the split "Category Group" + "Category", or legacy YNAB 4
  "Master Category" + "Sub Category".
- Names — falls back from a blank Payee to the Memo, then the default row name.
- Validation — requires at least one amount source (Outflow/Inflow or Amount); a
  file exposing none leaves rows un-clean instead of importing zero-dollar entries.

Enables the previously-disabled YNAB option on the imports screen (using the YNAB
logo, like Mint) with its configuration partial, and removes the now-dead
imports.new.coming_soon locale key. Documents the type in the API import-type
enums (rswag request spec + swagger_helper + generated openapi.yaml).

Closes #1255.
2026-06-16 08:32:17 +02:00
dripsmvcp 1157ea8f20 fix(sharing): scope import account selects to accessible_by (#1803) (#2194)
* fix(sharing): scope import account selects to accessible_by (#1803)

Three CSV / QIF account selects in import/uploads/show.html.erb and one
PDF-import account select in imports/_pdf_import.html.erb pulled their
options from `@import.family.accounts`. That listed every account in
the family — including the family admin's unshared personal accounts —
in the dropdown shown to any member running an import. Swap each call
site to `Current.user.accessible_accounts` (owned + explicitly shared
accounts only), matching the existing scoping used by the dashboard
sidebar, transactions controller, transfers controller, etc.

Adds a regression test that signs in as family_member and asserts the
unshared-account names from the dylan_family fixtures never appear in
the rendered upload page.

* test(import): scope leak assertions to account select (#2194 CodeRabbit)

CodeRabbit nitpick: assert_match on response.body could pass/fail on
text outside the account dropdown (sidebar, breadcrumb, error message,
etc.) and gave false confidence in the refute_match exclusions. Switch
to assert_select 'select[name="import[account_id]"] option', text: …
so the assertions only see the option nodes the leak test actually
cares about.

* test(import): cover PDF account-select scoping; pluck PDF partial (#2194 review)

jjmata: the _pdf_import.html.erb scoping change was not covered by the
existing test (which only hit /import/uploads). Add a regression test
hitting GET /imports/:id with a PdfImport fixture, asserting the
account dropdown options match accessible accounts only.

Also swap the PDF partial's accounts.map { |a| [a.name, a.id] } for
.pluck(:name, :id) to match the .pluck pattern the other three CSV/QIF
selects already use.

* test(import): stub pdf_uploaded? on PDF leak test (#2194 ci)

The new regression test hit ImportsController#show which redirects to
the upload page when @import.pdf_uploaded? is false. The pdf_with_rows
fixture has neither a pdf_file attached nor a statement, so the
redirect fired before the partial under test ever rendered, failing
with 302 in CI. Stub PdfImport#pdf_uploaded? to true so the test
exercises the account-select scoping path it was written to cover.

* fix(import): scope PDF form to :import so field names match (#2194 ci)

The PDF-import account-select form was `form_with model: import` with
no explicit scope. Because the model is a PdfImport, Rails derived the
param namespace from the class name, so the rendered field was named
`pdf_import[account_id]` — not `import[account_id]`. The
ImportsController#update action accepts either via
`params.dig(:pdf_import, :account_id) || params.dig(:import,
:account_id)` so live submissions still worked, but the regression
test added in 0685bbdf asserted on `select[name="import[account_id]"]`
and matched zero options.

Add `scope: :import` to align the rendered name with both the test
selector and the convention used by the CSV/QIF forms on the upload
page (which all use `scope: :import`).
2026-06-11 16:56:35 +02:00
ghost 3bcc86f4a8 refactor(imports): back PDF imports with statements (#1786) 2026-05-30 00:22:25 +02:00
Sure Admin (bot) 4fd460d551 Add Actual Budget CSV import flow (#1830)
* Add Actual Budget CSV import flow

* Address Actual import review feedback
2026-05-18 18:38:53 +02:00
Juan José MataandClaude 2595885eb7 Full .ndjson import / reorganize UI with Financial Tools / Raw Data tabs (#1208)
* Reorganize import UI with Financial Tools / Raw Data tabs

Split the flat list of import sources into two tabbed sections using
DS::Tabs: "Financial Tools" (Mint, Quicken/QIF, YNAB coming soon) and
"Raw Data" (transactions, investments, accounts, categories, rules,
documents). This prepares for adding more tool-specific importers
without cluttering the list.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* Fix import controller test to account for YNAB coming soon entry

The new YNAB "coming soon" disabled entry adds a 5th aria-disabled
element to the import dialog.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* Fix system tests to click Raw Data tab before selecting import type

Transaction, trade, and account imports are now under the Raw Data tab
and need an explicit tab click before the buttons are visible.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* feat: Add bulk import for NDJSON export files

Implements an import flow that accepts the full all.ndjson file from data exports,
allowing users to restore their complete data including:
- Accounts with accountable types
- Categories with parent relationships
- Tags and merchants
- Transactions with category, merchant, and tag references
- Trades with securities
- Valuations
- Budgets and budget categories
- Rules with conditions and actions (including compound conditions)

Key changes:
- Add BulkImport model extending Import base class
- Add Family::DataImporter to handle NDJSON parsing and import logic
- Update imports controller and views to support NDJSON workflow
- Skip configuration/mapping steps for structured NDJSON imports
- Add i18n translations for bulk import UI
- Add tests for BulkImport and DataImporter

* fix: Fix category import and test query issues

- Add default lucide_icon ("shapes") for categories when not provided
- Fix valuation test to use proper ActiveRecord joins syntax

* Linter errors

* fix: Add default color for tags when not provided in import

* fix: Add default kind for transactions when not provided in import

* Fix test

* Fix tests

* Fix remaining merge conflicts from PR 766 cherry-pick

Resolve conflict markers in test fixtures and clean up BulkImport
entry in new.html.erb to use the _import_option partial consistently.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* Import Sure `.ndjson`

* Remove `.ndjson` import from raw data

* Fix support for Sure "bulk" import from old branch

* Linter

* Fix CI test

* Fix more CI tests

* Fix tests

* Fix tests / move PDF import to first tab

* Remove redundant title

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-23 14:27:41 +01:00
Serge LandClaude Sonnet 4.6 57199d6eb9 Feat: Add QIF (Quicken Interchange Format) import functionality (#1074)
* Feat: Add QIF (Quicken Interchange Format) import functionality
- Add the ability to import QIF files for users coming from Quicken
- Includes categories and tags
- Comprehensive tests for QifImport, including parsing, row generation, and import functionality.
- Ensure handling of hierarchical categories (ex "Home:Home Improvement" is imported as Parent:Child)

* Fix QIF import issues raised in code review

- Fix two-digit year windowing in QIF date parser (e.g. '99 → 1999, not 2099)
- Fix ArgumentError from invalid `undef: :raise` encoding option
- Nil-safe `leaf_category_name` with blank guard and `.to_s` coercion
- Memoize `qif_account_type` to avoid re-parsing the full QIF file
- Add strong parameters (`selection_params`) to QifCategorySelectionsController
- Wrap all mutations in DB transactions in uploads and category-selections controllers
- Skip unchanged tag rows (only write rows where tags actually differ)
- Replace hardcoded strings with i18n keys across QIF views and nav
- Fix potentially colliding checkbox/label IDs in category selection view
- Improve keyboard accessibility: use semantic `<label>` for file picker area

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix QIF import test count and Brakeman mass assignment warning

- Update ImportsControllerTest to expect 4 disabled import options (was 3),
  accounting for the new QIF import type added in this branch
- Remove :account_id from upload_params permit list; it was never accessed
  through strong params (always via params.dig with Current.family scope),
  so this resolves the Brakeman high-confidence mass assignment warning

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Fix: QIF import security, safety, and i18n issues raised in code review
- Added french, spanish and german translations for newly added i18n keys
- Replace params.dig(:import, :account_id) with a proper strong-params
  accessor (import_account_id) in UploadsController to satisfy Rails
  parameter filtering requirements
- Guard ImportsController#show against QIF imports reaching the publish
  screen before a file has been uploaded, preventing an unrescued error
  on publish
- Gate the QIF "Clean" nav step link on import.uploaded? to prevent
  routing to CleansController with an unconfigured import (which would
  raise "Unknown import type: QifImport" via ImportsHelper)
- Replace hard-coded "txn" pluralize calls in the category/tag selection
  view with t(".txn_count") and add pluralization keys to the locale file
- Localize all hard-coded strings in the QIF upload section of
  uploads/show.html.erb and add corresponding en.yml keys
- Convert the CSV upload drop zone from a clickable <div> (JS-only) to
  a semantic <label> element, making it keyboard-accessible without
  JavaScript

* Fix: missing translations keys

* Add icon mapping and random color assignment to new categories

* fix a lint issue

* Add a warning about splits and some plumbing for future support.
Updated locales.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 20:22:39 +01:00
Juan José MataandClaude 8264943f23 Show disabled import options when no accounts exist (#977)
* Show disabled import options before accounts exist

Keep account-dependent import choices visible on /imports/new and render them as disabled with guidance when no accounts are available.

* Refactor disabled import options: extract partial, fix accessibility (#986)

- Extract _import_option partial to eliminate duplicated enabled/disabled
  markup across TransactionImport, TradeImport, and MintImport (also
  used by AccountImport, CategoryImport, RuleImport for consistency)
- Replace misleading chevron-right with lock icon in disabled state
- Add aria-disabled="true" for screen reader accessibility
- Remove redundant default: parameter from t() call
- Fix locale key ordering (requires_account after import_* keys)
- Fix extra blank line in test file
- Add assertion for aria-disabled attribute in test

https://claude.ai/code/session_016j9tDYEBfWX9Dzd99rAYjX

Co-authored-by: Claude <noreply@anthropic.com>

* Tailwind fixes

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-14 01:10:25 +01:00
Juan José Mata d3c4986bfd Fix flaky expectations in import-related tests (#963) 2026-02-11 19:42:26 +01:00
Juan José Mata 0eedd533bb Generalize from PDF import to just files 2026-02-11 17:56:49 +00:00
Zach Gollwitzer 398b246965 CSV Imports Overhaul (Transactions, Trades, Accounts, and Mint import support) (#1209)
* Remove stale 1.0 import logic and model

* Fresh start

* Checkpoint before removing nav

* First working prototype

* Add trade, account, and mint import flows

* Basic working version with tests

* System tests for each import type

* Clean up mappings flow

* Clean up PR, refactor stale code, tests

* Add back row validations

* Row validations

* Fix import job test

* Fix import navigation

* Fix mint import configuration form

* Currency preset for new accounts
2024-10-01 10:47:59 -04:00
Pedro Carmona 0c1ff00c1e Refactor: Allow other import files (#1099)
* Rename stimulus controller

* feature: rename raw_csv_str to raw_file_str
2024-08-19 09:25:07 -04:00
Alexander SchrotandZach Gollwitzer 4527482aa2 Add support for different column separator in csv import logic (#1096)
* add col_sep to import model

* add validation for col_sep column

* add col_sep option to csv import model

* make use of col_sep option in import model

* add column separator field to new/edit action of an import

* add col_sep parameter to create/update action

* fix spacing between fields

Co-authored-by: Zach Gollwitzer <zach.gollwitzer@gmail.com>
Signed-off-by: Alexander Schrot <alexander@axs-labs.com>

---------

Signed-off-by: Alexander Schrot <alexander@axs-labs.com>
Co-authored-by: Zach Gollwitzer <zach.gollwitzer@gmail.com>
2024-08-16 14:00:16 -04:00
Zach Gollwitzer fa08f027c7 Sync notifications and troubleshooting guides (#998)
* Add help articles

* Broadcast sync messages as notifications

* Lint fixes

* more lint fixes

* Remove redundant code
2024-07-18 14:39:38 -04:00
Tony Vincent cdbca5aff3 Allow CSV file upload in import flow (#986)
* Add .tool-versions to gitignore

* Add dropzone js for drag and drop file uploads

* UI for csv file uploads for import

* dropzone controller and use lucide_icon instead of svg

* Preview for file chosen

* File upload

* Remove dropzone

* Normalize I18n keys and fix lint issues

* Add system tests

* Cleanup

* Remove unwanted
2024-07-16 09:23:45 -04:00
Zach Gollwitzer 457247da8e Create tagging system (#792)
* Repro

* Fix

* Update signage

* Create tagging system

* Add tags to transaction imports

* Build tagging UI

* Cleanup

* More cleanup
2024-05-23 08:09:33 -04:00
Zach Gollwitzer 45ae4a9737 CSV Transaction Imports (#708)
Introduces a basic CSV import module for bulk-importing account transactions.

Changes include:

- User can load a CSV
- User can configure the column mappings for a CSV
- Imported CSV shows invalid cells
- User can clean up their data directly in the UI
- User can see a preview of the import rows and confirm import
- Layout refactor + Import nav stepper
- System test stability improvements
2024-05-17 09:09:32 -04:00