Commit Graph

6 Commits

Author SHA1 Message Date
ghost
6e04c6927d feat(imports): add SureImport session batches (#1785)
* feat(imports): add SureImport session batches

Add first-class SureImport sessions for ordered multi-file NDJSON imports.

Persist source mappings across chunks, make session/chunk processing idempotent, expose progress readback, and keep existing single-file import behavior compatible.

Includes the devcontainer libvips runtime dependency needed by ActiveStorage variant tests.

Addresses #1610.

Related to #1458.

* fix(imports): avoid scanner-like API key test data

* test(imports): assert skipped balances are not persisted

* fix(imports): harden session publish retries

Validate expected import chunk sequences exactly before publish, and restore session state with error details when enqueueing the publish job fails.

* fix(imports): close session retry edge cases

Backfill expected chunk counts after client-session insert races and enqueue import-session jobs after the status transition commits. Persist a safe enqueue failure body so API readback does not expose raw queue errors.

* fix(imports): address session publish review gaps

Remove dead transaction external-id assignment, harden session publish retry/sync behavior, align session chunk status docs, and add regression coverage for partial retries and safe enqueue error readback.

* fix(imports): include sessions in family reset

Clear import sessions through the family reset job so chunk imports and source mappings do not survive a reset.

Expose import session and source mapping counts in the reset status response and regenerated OpenAPI schema so polling reflects the full reset surface.

* test(imports): cover split import mapping invariants

* test(imports): cover session verification invariants

* fix(imports): scope SureImport session reimports

* Tighten SureImport session batching

* fix(imports): export rule source ids for sessions

* test(imports): stabilize rule id export assertion

* test(imports): restore reset status session fixture
2026-06-04 11:48:44 +02:00
ghost
cb660ebf65 refactor(imports): focus Sure preflight scope (#1833) 2026-06-03 00:37:21 +02:00
ghost
655895341d feat(imports): verify Sure NDJSON import readback (#1869)
* feat(imports): verify Sure NDJSON readback

* fix(imports): tighten Sure readback verification

* fix(imports): polish Sure verification review nits
2026-05-20 21:35:22 +02:00
ghost
1fedc43f68 feat(api): add import preflight validation (#1755)
* feat(api): add import preflight validation

* fix(api): harden import preflight validation
2026-05-12 00:00:49 +02:00
ghost
c4414c4fbb feat(api): expose import status details (#1599)
* feat(api): expose import status details

* fix(api): reuse import status validation counts

* fix(api): cache Sure import status reads

* fix(imports): invalidate cached Sure import blobs

* docs(api): split import status schemas

* fix(api): refine import status detail contract
2026-05-01 22:59:32 +02:00
Juan José Mata
2595885eb7 Full .ndjson import / reorganize UI with Financial Tools / Raw Data tabs (#1208)
* Reorganize import UI with Financial Tools / Raw Data tabs

Split the flat list of import sources into two tabbed sections using
DS::Tabs: "Financial Tools" (Mint, Quicken/QIF, YNAB coming soon) and
"Raw Data" (transactions, investments, accounts, categories, rules,
documents). This prepares for adding more tool-specific importers
without cluttering the list.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* Fix import controller test to account for YNAB coming soon entry

The new YNAB "coming soon" disabled entry adds a 5th aria-disabled
element to the import dialog.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* Fix system tests to click Raw Data tab before selecting import type

Transaction, trade, and account imports are now under the Raw Data tab
and need an explicit tab click before the buttons are visible.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* feat: Add bulk import for NDJSON export files

Implements an import flow that accepts the full all.ndjson file from data exports,
allowing users to restore their complete data including:
- Accounts with accountable types
- Categories with parent relationships
- Tags and merchants
- Transactions with category, merchant, and tag references
- Trades with securities
- Valuations
- Budgets and budget categories
- Rules with conditions and actions (including compound conditions)

Key changes:
- Add BulkImport model extending Import base class
- Add Family::DataImporter to handle NDJSON parsing and import logic
- Update imports controller and views to support NDJSON workflow
- Skip configuration/mapping steps for structured NDJSON imports
- Add i18n translations for bulk import UI
- Add tests for BulkImport and DataImporter

* fix: Fix category import and test query issues

- Add default lucide_icon ("shapes") for categories when not provided
- Fix valuation test to use proper ActiveRecord joins syntax

* Linter errors

* fix: Add default color for tags when not provided in import

* fix: Add default kind for transactions when not provided in import

* Fix test

* Fix tests

* Fix remaining merge conflicts from PR 766 cherry-pick

Resolve conflict markers in test fixtures and clean up BulkImport
entry in new.html.erb to use the _import_option partial consistently.

https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3

* Import Sure `.ndjson`

* Remove `.ndjson` import from raw data

* Fix support for Sure "bulk" import from old branch

* Linter

* Fix CI test

* Fix more CI tests

* Fix tests

* Fix tests / move PDF import to first tab

* Remove redundant title

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-23 14:27:41 +01:00