Files
sure/app/models/pdf_import.rb
Guillem Arias Fauste 5105752bbd feat(sync): family-facing cancellation for syncs, imports, and exports (#2685)
* feat(sync): family-facing cancellation for syncs, imports, and exports

Users had no way to stop or recover any background operation: a
mistaken "Sync all" runs to completion, and an import or export whose
job died (hard worker kills lose in-flight Sidekiq jobs) wedges with a
spinner forever.

Sync cancellation (cooperative — nothing is ever killed):

- New syncs.cancel_requested_at column. Only the cancelled sync
  carries the flag: pending descendants are marked stale immediately
  (their queued jobs no-op via the existing may_start? guard), while
  descendants whose jobs are already executing finish their work
  honestly.
- Family::Syncer stops fanning out child syncs once the flag is set
  (fresh read per iteration — the flag comes from the web process).
- Finalization resolves a cancel-requested sync to stale instead of
  completed, which also skips post-sync (transfer matching, rules,
  broadcasts) via the existing stale gate.
- The `visible` scope excludes cancel-requested syncs, so spinners
  clear immediately and — fixing a latent bug this feature would have
  amplified — sync_later no longer piggybacks a new sync request onto
  a dying sync it would silently swallow.
- Cancel button appears next to "Sync all" on the accounts page while
  a family sync is visible. SyncsController#cancel scopes through
  Sync.for_family with resource_owner, so cross-family ids 404 and
  account-level syncs respect per-user account access.

Stuck import/export self-service:

- Import#force_fail! / FamilyExport#force_fail!: allowed only once the
  record has been idle past PRESUMED_LOST_AFTER (1 hour — dwarfs any
  legitimate run), and applied inside with_lock with a status
  re-check, so a job finishing between page render and button click
  wins. Imports fail into the existing retry path (reverting ->
  revert_failed keeps the revert retryable); PdfImports release their
  processing claim back to pending; exports fail so a new one can be
  created.
- "Mark as failed" buttons appear on the imports/exports index rows
  only when a record is presumed lost, behind the pages' existing
  permission gates (statement-import permission for imports, admin
  for exports).

* fix(sync-cancel): cascade pending cancels, guard late finalizers, scope provider syncs

Review feedback on #2685 (CodeRabbit, Codex):

- request_cancel! now cascades finalization for pending syncs too: a
  pending child resolved to stale never runs its job, so nothing else
  would ever call finalize_if_all_children_finalized — its waiting
  parent hung in syncing until the 24h sweep (CodeRabbit critical)
- SimplefinItem::Syncer#mark_completed re-reads the sync under a row
  lock and skips finalization once cancellation was requested or the
  row went terminal — its in-memory copy predates the cancel, and the
  unguarded complete! (plus the raw status fallback) resurrected a
  cancelled sync and re-ran post-sync (Codex)
- Cancelling provider-item syncs now requires admin: for_family's
  resource_owner only scopes the Account branch, so a restricted member
  could cancel admin-managed provider syncs spanning accounts they
  cannot see. Family- and account-level syncs stay member-cancellable,
  matching the buttons the UI shows (Codex)
- Lost-import error copy moved behind i18n
  (imports.errors.presumed_lost), resolved at call time (CodeRabbit)
- Tests: pending-child cancel finalizes the parent; late provider
  complete! cannot resurrect a cancelled sync; provider-sync
  cancellation is admin-only

* fix(sync-cancel): capture the skipped-finalization case via DebugLogEntry

CodeRabbit round-2: the mark_completed skip (cancelled/terminal sync)
is support-relevant — record it in the super-admin debug UI with the
family and provider attached instead of a raw Rails.logger line.
category: provider_sync, matching the other provider syncers.
2026-07-17 06:57:54 +02:00

285 lines
7.6 KiB
Ruby

class PdfImport < Import
has_one_attached :pdf_file, dependent: :purge_later
validates :document_type, inclusion: { in: DOCUMENT_TYPES }, allow_nil: true
validate :account_statement_matches_import
class << self
def create_from_upload!(family:, file:, user:)
statement = AccountStatement.create_from_prepared_upload!(
family: family,
account: nil,
prepared_upload: AccountStatement.prepare_upload!(file)
)
create_from_statement!(statement: statement)
rescue AccountStatement::DuplicateUploadError => e
raise unless e.statement.manageable_by?(user)
create_from_statement!(statement: e.statement)
end
def create_from_statement!(statement:)
reusable_import = statement.latest_reusable_pdf_import
return reusable_import if reusable_import &&
reusable_import.account_id == statement.account_id &&
reusable_import.date_format == statement.family.date_format
create!(family: statement.family, account: statement.account, account_statement: statement, date_format: statement.family.date_format, status: :pending)
end
end
# A PdfImport's importing status is a processing claim (AI extraction or
# publish). Release a lost claim back to pending so the user can re-trigger
# processing, mirroring ProcessPdfJob's own reclaim; lost reverts keep the
# base revert_failed behavior.
def force_fail!(error_message = Import.lost_error_message)
return super if reverting?
with_lock do
return false unless presumed_lost?
update!(status: :pending)
end
true
end
def import!
raise "Account required for PDF import" unless account.present?
transaction do
mappings.each(&:create_mappable!)
new_transactions = rows.map do |row|
category = mappings.categories.mappable_for(row.category)
Transaction.new(
category: category,
entry: Entry.new(
account: account,
date: row.date_iso,
amount: row.signed_amount,
name: row.name,
currency: row.currency,
notes: row.notes,
import: self,
import_locked: true
)
)
end
Transaction.import!(new_transactions, recursive: true) if new_transactions.any?
end
end
def assign_account!(account)
transaction do
update!(account: account)
if (statement = account_statement)
statement.lock!
statement.link_to_account!(account) if statement.account_id != account.id
end
end
end
def pdf_uploaded?
statement_backed? || pdf_file.attached?
end
def ai_processed?
ai_summary.present?
end
def process_with_ai_later
return false unless with_lock { pending? && !ai_processed? && rows_count.zero? && pdf_uploaded? && update!(status: :importing) }
begin
ProcessPdfJob.perform_later(self)
true
rescue StandardError => e
Rails.logger.error("Failed to enqueue PDF processing for import #{id}: #{e.class.name} - #{e.message}")
reload.with_lock { update!(status: :pending) }
false
end
end
def process_with_ai
# Honors Setting.llm_provider (issue #2113) — Provider::Anthropic implements
# process_pdf (PR #1985).
provider = Provider::Registry.preferred_llm_provider
raise "AI provider not configured" unless provider
raise "AI provider does not support PDF processing" unless provider.supports_pdf_processing?
response = provider.process_pdf(
pdf_content: pdf_file_content,
family: family
)
unless response.success?
error_message = response.error&.message || "Unknown PDF processing error"
raise error_message
end
result = response.data
update!(
ai_summary: result.summary,
document_type: result.document_type
)
result
end
def extract_transactions
return unless statement_with_transactions?
# Honors Setting.llm_provider (issue #2113) — Provider::Anthropic implements
# extract_bank_statement (PR #1985).
provider = Provider::Registry.preferred_llm_provider
raise "AI provider not configured" unless provider
response = provider.extract_bank_statement(
pdf_content: pdf_file_content,
family: family
)
unless response.success?
error_message = response.error&.message || "Unknown extraction error"
raise error_message
end
update!(extracted_data: response.data)
response.data
end
def bank_statement?
document_type == "bank_statement"
end
def statement_with_transactions?
document_type.in?(%w[bank_statement credit_card_statement])
end
def has_extracted_transactions?
extracted_data.present? && extracted_data["transactions"].present?
end
def extracted_transactions
extracted_data&.dig("transactions") || []
end
def generate_rows_from_extracted_data
transaction do
rows.destroy_all
unless has_extracted_transactions?
update_column(:rows_count, 0)
return
end
currency = account&.currency || family.currency
mapped_rows = extracted_transactions.map.with_index(1) do |txn, index|
{
import_id: id,
source_row_number: index,
date: format_date_for_import(txn["date"]),
amount: txn["amount"].to_s,
name: txn["name"].to_s,
category: txn["category"].to_s,
notes: txn["notes"].to_s,
currency: currency
}
end
Import::Row.insert_all!(mapped_rows) if mapped_rows.any?
update_column(:rows_count, mapped_rows.size)
end
end
def send_next_steps_email(user)
PdfImportMailer.with(
user: user,
pdf_import: self
).next_steps.deliver_later
end
def uploaded?
pdf_uploaded?
end
def configured?
ai_processed? && rows_count > 0
end
def cleaned?
configured? && rows.all?(&:valid?)
end
def publishable?
account.present? && statement_with_transactions? && cleaned? && mappings.all?(&:valid?)
end
def cleaned_from_validation_stats?(invalid_rows_count:)
account.present? && statement_with_transactions? && super
end
def publishable_from_validation_stats?(invalid_rows_count:)
account.present? && statement_with_transactions? && super
end
def column_keys
%i[date amount name category notes]
end
def requires_csv_workflow?
false
end
def pdf_file_content
return @pdf_file_content if defined?(@pdf_file_content)
return @pdf_file_content = account_statement.original_file.download if statement_backed?
@pdf_file_content = pdf_file.download if pdf_file.attached?
end
def pdf_filename
return account_statement.filename if statement_backed?
pdf_file.filename.to_s if pdf_file.attached?
end
def statement_backed?
account_statement&.original_file&.attached?
end
def required_column_keys
%i[date amount]
end
def mapping_steps
base = []
# Only include CategoryMapping if rows have non-empty categories
base << Import::CategoryMapping if rows.where.not(category: [ nil, "" ]).exists?
# Note: PDF imports use direct account selection in the UI, not AccountMapping
# AccountMapping is designed for CSV imports where rows have different account values
base
end
private
def format_date_for_import(date_str)
return "" if date_str.blank?
Date.parse(date_str).strftime(date_format)
rescue ArgumentError
date_str.to_s
end
def account_statement_matches_import
return if account_statement.blank? || (account_statement.family_id == family_id && account_statement.pdf?)
errors.add(:account_statement, :invalid)
end
end