* feat(sync): family-facing cancellation for syncs, imports, and exports
Users had no way to stop or recover any background operation: a
mistaken "Sync all" runs to completion, and an import or export whose
job died (hard worker kills lose in-flight Sidekiq jobs) wedges with a
spinner forever.
Sync cancellation (cooperative — nothing is ever killed):
- New syncs.cancel_requested_at column. Only the cancelled sync
carries the flag: pending descendants are marked stale immediately
(their queued jobs no-op via the existing may_start? guard), while
descendants whose jobs are already executing finish their work
honestly.
- Family::Syncer stops fanning out child syncs once the flag is set
(fresh read per iteration — the flag comes from the web process).
- Finalization resolves a cancel-requested sync to stale instead of
completed, which also skips post-sync (transfer matching, rules,
broadcasts) via the existing stale gate.
- The `visible` scope excludes cancel-requested syncs, so spinners
clear immediately and — fixing a latent bug this feature would have
amplified — sync_later no longer piggybacks a new sync request onto
a dying sync it would silently swallow.
- Cancel button appears next to "Sync all" on the accounts page while
a family sync is visible. SyncsController#cancel scopes through
Sync.for_family with resource_owner, so cross-family ids 404 and
account-level syncs respect per-user account access.
Stuck import/export self-service:
- Import#force_fail! / FamilyExport#force_fail!: allowed only once the
record has been idle past PRESUMED_LOST_AFTER (1 hour — dwarfs any
legitimate run), and applied inside with_lock with a status
re-check, so a job finishing between page render and button click
wins. Imports fail into the existing retry path (reverting ->
revert_failed keeps the revert retryable); PdfImports release their
processing claim back to pending; exports fail so a new one can be
created.
- "Mark as failed" buttons appear on the imports/exports index rows
only when a record is presumed lost, behind the pages' existing
permission gates (statement-import permission for imports, admin
for exports).
* fix(sync-cancel): cascade pending cancels, guard late finalizers, scope provider syncs
Review feedback on #2685 (CodeRabbit, Codex):
- request_cancel! now cascades finalization for pending syncs too: a
pending child resolved to stale never runs its job, so nothing else
would ever call finalize_if_all_children_finalized — its waiting
parent hung in syncing until the 24h sweep (CodeRabbit critical)
- SimplefinItem::Syncer#mark_completed re-reads the sync under a row
lock and skips finalization once cancellation was requested or the
row went terminal — its in-memory copy predates the cancel, and the
unguarded complete! (plus the raw status fallback) resurrected a
cancelled sync and re-ran post-sync (Codex)
- Cancelling provider-item syncs now requires admin: for_family's
resource_owner only scopes the Account branch, so a restricted member
could cancel admin-managed provider syncs spanning accounts they
cannot see. Family- and account-level syncs stay member-cancellable,
matching the buttons the UI shows (Codex)
- Lost-import error copy moved behind i18n
(imports.errors.presumed_lost), resolved at call time (CodeRabbit)
- Tests: pending-child cancel finalizes the parent; late provider
complete! cannot resurrect a cancelled sync; provider-sync
cancellation is admin-only
* fix(sync-cancel): capture the skipped-finalization case via DebugLogEntry
CodeRabbit round-2: the mark_completed skip (cancelled/terminal sync)
is support-relevant — record it in the super-admin debug UI with the
family and provider attached instead of a raw Rails.logger line.
category: provider_sync, matching the other provider syncers.
A sync marked stale by SyncCleanerJob while its job is still running
hits two lost-update paths when that job finishes:
- success path: finalize skipped the status transition but still ran
perform_post_sync, re-applying transfer matching, rules, and
broadcasts for a sync the system had already written off
- failure path: the rescue's unguarded fail! silently overwrote the
terminal stale status with failed (the in-memory record still read
syncing, so the AASM guard never fired)
Fix: re-check state under a row lock (with_lock reloads) before
failing, and skip post-sync for stale syncs in finalize. Post-sync
still runs for failed syncs — that behavior is intentional and
covered by existing tests.
* fix(jobs): delegate recurring-transaction sync gate to Sync.for_family
`IdentifyRecurringTransactionsJob#family_has_incomplete_syncs?` hand-rolled
the list of provider `*_items` associations it polled — plaid, simplefin,
lunchflow, enable_banking, sophtron — missing nine other `Syncable`
provider concerns on `Family`: coinbase, binance, kraken, coinstats,
snaptrade, mercury, brex, indexa_capital, ibkr. When a sync on any of those
nine was in flight, the debounce gate fell through and
`RecurringTransaction::Identifier` ran against a partial dataset; the
follow-up re-enqueue then hit the `find_or_initialize_by` upsert path and
inherited the stale `occurrence_count`. Same drift pattern that bolted
sophtron on as the 5th entry (#591) was already an iteration of.
The maintainers' own `Sync.for_family` (sync.rb:61) already enumerates every
`*_items` association via `Family.reflect_on_all_associations(:has_many)`
filtered by inclusion of `Syncable` — exactly the helper the gate should
delegate to so the list cannot drift again.
- Add `Sync.any_incomplete_for?(family)` class method that wraps
`for_family(family).incomplete.exists?`.
- Rewrite `family_has_incomplete_syncs?` to delegate. 14 lines → 1.
- New test file `test/jobs/identify_recurring_transactions_job_test.rb`
covers in-flight Coinbase + Mercury (gate fires), idle (identifier runs),
missing family, and superseded-by-newer-schedule.
- `test/models/sync_test.rb` gets 2 new tests pinning
`any_incomplete_for?` against a provider `_items` sync and a
family-itself sync.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(jobs): stub Rails.cache.read for supersession test (NullStore in test env)
`Rails.cache` is `ActiveSupport::Cache::NullStore` in the Rails test env, so
the previous test's `Rails.cache.write(cache_key, @scheduled_at + 10, ...)`
was a no-op and `Rails.cache.read(cache_key)` returned `nil`. The
supersession short-circuit `return if latest_scheduled && latest_scheduled
> scheduled_at` then fell through, the job proceeded to invoke
`RecurringTransaction::Identifier`, and the Mocha
`.expects(:identify_recurring_patterns).never` failed in CI.
Switch to `Rails.cache.stubs(:read).with(cache_key).returns(...)` — the
same idiom `test/models/provider/twelve_data_test.rb:186-197` already uses
for the cache layer. Add an `assert_nil` on the bare `perform` return so
Minitest's assertion counter sees an explicit assertion (silences the
"missing assertions" warning).
No production-code change. Behavior under test is unchanged; only the test
mechanism for simulating "newer scheduled run already in cache" is fixed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Basic plaid data model and linking
* Remove institutions, add plaid items
* Improve schema and Plaid provider
* Add webhook verification sketch
* Webhook verification
* Item accounts and balances sync setup
* Provide test encryption keys
* Fix test
* Only provide encryption keys in prod
* Try defining keys in test env
* Consolidate account sync logic
* Add back plaid account initialization
* Plaid transaction sync
* Sync UI overhaul for Plaid
* Add liability and investment syncing
* Handle investment webhooks and process current day holdings
* Remove logs
* Remove "all" period select for performance
* fix amount calc
* Remove todo comment
* Coming soon for investment historical data
* Document Plaid configuration
* Listen for holding updates
