Files
sure/app/models/import_source_mapping.rb
ghost 6e04c6927d feat(imports): add SureImport session batches (#1785)
* feat(imports): add SureImport session batches

Add first-class SureImport sessions for ordered multi-file NDJSON imports.

Persist source mappings across chunks, make session/chunk processing idempotent, expose progress readback, and keep existing single-file import behavior compatible.

Includes the devcontainer libvips runtime dependency needed by ActiveStorage variant tests.

Addresses #1610.

Related to #1458.

* fix(imports): avoid scanner-like API key test data

* test(imports): assert skipped balances are not persisted

* fix(imports): harden session publish retries

Validate expected import chunk sequences exactly before publish, and restore session state with error details when enqueueing the publish job fails.

* fix(imports): close session retry edge cases

Backfill expected chunk counts after client-session insert races and enqueue import-session jobs after the status transition commits. Persist a safe enqueue failure body so API readback does not expose raw queue errors.

* fix(imports): address session publish review gaps

Remove dead transaction external-id assignment, harden session publish retry/sync behavior, align session chunk status docs, and add regression coverage for partial retries and safe enqueue error readback.

* fix(imports): include sessions in family reset

Clear import sessions through the family reset job so chunk imports and source mappings do not survive a reset.

Expose import session and source mapping counts in the reset status response and regenerated OpenAPI schema so polling reflects the full reset surface.

* test(imports): cover split import mapping invariants

* test(imports): cover session verification invariants

* fix(imports): scope SureImport session reimports

* Tighten SureImport session batching

* fix(imports): export rule source ids for sessions

* test(imports): stabilize rule id export assertion

* test(imports): restore reset status session fixture
2026-06-04 11:48:44 +02:00

42 lines
1.5 KiB
Ruby

class ImportSourceMapping < ApplicationRecord
SOURCE_TYPES = %w[Account Category Tag Merchant RecurringTransaction Transaction Budget Security Rule].freeze
belongs_to :family
belongs_to :import_session
belongs_to :target, polymorphic: true, optional: true
validates :source_type, :source_id, :target_type, :target_id, presence: true
validates :source_type, inclusion: { in: SOURCE_TYPES }
validates :target_type, inclusion: { in: SOURCE_TYPES }, allow_blank: true
validates :source_type, length: { maximum: 64 }
validates :source_id, length: { maximum: 255 }
validates :source_id, uniqueness: { scope: [ :import_session_id, :source_type ] }
normalizes :source_type, :source_id, with: ->(value) { value.strip.presence }
validate :family_matches_import_session
validate :target_exists
validate :target_matches_family
private
def family_matches_import_session
return if import_session.blank? || family_id == import_session.family_id
errors.add(:family, "must match import session")
end
def target_exists
return if target_type.blank? || target_id.blank? || !SOURCE_TYPES.include?(target_type)
return if target.present?
errors.add(:target, "must exist")
end
def target_matches_family
return if target_type.blank? || !SOURCE_TYPES.include?(target_type)
return if target.blank?
return unless target.respond_to?(:family_id)
return if target.family_id == family_id
errors.add(:target, "must belong to your family")
end
end