Files
sure/app/models/transaction.rb
LPW 3658e812a8 Add pending transaction handling and duplicate reconciliation logic (#602)
* Add pending transaction handling and duplicate reconciliation logic

- Implemented logic to exclude pending transactions from budgets and analytics calculations.
- Introduced mechanisms for reconciling pending transactions with posted versions.
- Added duplicate detection with support for merging or dismissing matches.
- Updated transaction search filters to include a `status_filter` for pending/confirmed transactions.
- Introduced UI elements for reviewing and resolving duplicates.
- Enhanced `ProviderSyncSummary` with stats for reconciled and stale pending transactions.

* Refactor translation handling and enhance transaction and sync logic

- Moved hardcoded strings to locale files for improved translation support.
- Refined styling for duplicate transaction indicators and sync summaries.
- Improved logic for excluding stale pending transactions and updating timestamps on batch exclusion.
- Added unique IDs to status filters for better element targeting in UI.
- Optimized database queries to avoid N+1 issues in stale pending calculations.

* Add sync settings and enhance pending transaction handling

- Introduced a new "Sync Settings" section in hosting settings with UI to toggle inclusion of pending transactions.
- Updated handling of pending transactions with improved inference logic for `posted=0` and `transacted_at` in processors.
- Added priority order for pending transaction inclusion: explicit argument > environment variable > runtime configurable setting.
- Refactored settings and controllers to store updated sync preferences.

* Refactor sync settings and pending transaction reconciliation

- Extracted logic for pending transaction reconciliation, stale exclusion, and unmatched tracking into dedicated methods for better maintainability.
- Updated sync settings to infer defaults from multiple provider environment variables (`SIMPLEFIN_INCLUDE_PENDING`, `PLAID_INCLUDE_PENDING`).
- Refined UI and messaging to handle multi-provider configurations in sync settings.

# Conflicts:
#	app/models/simplefin_item/importer.rb

* Debounce transaction reconciliation during imports

- Added per-run reconciliation debouncing to prevent repeated scans for the same account during chunked history imports.
- Trimmed size of reconciliation stats to retain recent details only.
- Introduced error tracking for reconciliation steps to improve UI visibility of issues.

* Apply ABS() in pending transaction queries and improve error handling

- Updated pending transaction logic to use ABS() for consistent handling of negative amounts.
- Adjusted amount bounds calculations to ensure accuracy for both positive and negative values.
- Refined exception handling in `merge_duplicate` to log failures and update user alert.
- Replaced `Date.today` with `Date.current` in tests to ensure timezone consistency.
- Minor optimization to avoid COUNT queries by loading limited records directly.

* Improve error handling in duplicate suggestion and dismissal logic

- Added exception handling for `store_duplicate_suggestion` to log failures and prevent crashes during fuzzy/low-confidence matches.
- Enhanced `dismiss_duplicate` action to handle `ActiveRecord::RecordInvalid` and display appropriate user alerts.

---------

Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
2026-01-10 20:11:00 +01:00

149 lines
4.7 KiB
Ruby

class Transaction < ApplicationRecord
include Entryable, Transferable, Ruleable
belongs_to :category, optional: true
belongs_to :merchant, optional: true
has_many :taggings, as: :taggable, dependent: :destroy
has_many :tags, through: :taggings
accepts_nested_attributes_for :taggings, allow_destroy: true
after_save :clear_merchant_unlinked_association, if: :merchant_id_previously_changed?
enum :kind, {
standard: "standard", # A regular transaction, included in budget analytics
funds_movement: "funds_movement", # Movement of funds between accounts, excluded from budget analytics
cc_payment: "cc_payment", # A CC payment, excluded from budget analytics (CC payments offset the sum of expense transactions)
loan_payment: "loan_payment", # A payment to a Loan account, treated as an expense in budgets
one_time: "one_time" # A one-time expense/income, excluded from budget analytics
}
# Pending transaction scopes - filter based on provider pending flags in extra JSONB
# Works with any provider that stores pending status in extra["provider_name"]["pending"]
scope :pending, -> {
where(<<~SQL.squish)
(transactions.extra -> 'simplefin' ->> 'pending')::boolean = true
OR (transactions.extra -> 'plaid' ->> 'pending')::boolean = true
SQL
}
scope :excluding_pending, -> {
where(<<~SQL.squish)
(transactions.extra -> 'simplefin' ->> 'pending')::boolean IS DISTINCT FROM true
AND (transactions.extra -> 'plaid' ->> 'pending')::boolean IS DISTINCT FROM true
SQL
}
# Overarching grouping method for all transfer-type transactions
def transfer?
funds_movement? || cc_payment? || loan_payment?
end
def set_category!(category)
if category.is_a?(String)
category = entry.account.family.categories.find_or_create_by!(
name: category
)
end
update!(category: category)
end
def pending?
extra_data = extra.is_a?(Hash) ? extra : {}
ActiveModel::Type::Boolean.new.cast(extra_data.dig("simplefin", "pending")) ||
ActiveModel::Type::Boolean.new.cast(extra_data.dig("plaid", "pending"))
rescue
false
end
# Potential duplicate matching methods
# These help users review and resolve fuzzy-matched pending/posted pairs
def has_potential_duplicate?
potential_posted_match_data.present? && !potential_duplicate_dismissed?
end
def potential_duplicate_entry
return nil unless has_potential_duplicate?
Entry.find_by(id: potential_posted_match_data["entry_id"])
end
def potential_duplicate_reason
potential_posted_match_data&.dig("reason")
end
def potential_duplicate_confidence
potential_posted_match_data&.dig("confidence") || "medium"
end
def low_confidence_duplicate?
potential_duplicate_confidence == "low"
end
def potential_duplicate_posted_amount
potential_posted_match_data&.dig("posted_amount")&.to_d
end
def potential_duplicate_dismissed?
potential_posted_match_data&.dig("dismissed") == true
end
# Merge this pending transaction with its suggested posted match
# This DELETES the pending entry since the posted version is canonical
def merge_with_duplicate!
return false unless has_potential_duplicate?
posted_entry = potential_duplicate_entry
return false unless posted_entry
pending_entry_id = entry.id
pending_entry_name = entry.name
# Delete this pending entry completely (no need to keep it around)
entry.destroy!
Rails.logger.info("User merged pending entry #{pending_entry_id} (#{pending_entry_name}) with posted entry #{posted_entry.id}")
true
end
# Dismiss the duplicate suggestion - user says these are NOT the same transaction
def dismiss_duplicate_suggestion!
return false unless potential_posted_match_data.present?
updated_extra = (extra || {}).deep_dup
updated_extra["potential_posted_match"]["dismissed"] = true
update!(extra: updated_extra)
Rails.logger.info("User dismissed duplicate suggestion for entry #{entry.id}")
true
end
# Clear the duplicate suggestion entirely
def clear_duplicate_suggestion!
return false unless potential_posted_match_data.present?
updated_extra = (extra || {}).deep_dup
updated_extra.delete("potential_posted_match")
update!(extra: updated_extra)
true
end
private
def potential_posted_match_data
return nil unless extra.is_a?(Hash)
extra["potential_posted_match"]
end
def clear_merchant_unlinked_association
return unless merchant_id.present? && merchant.is_a?(ProviderMerchant)
family = entry&.account&.family
return unless family
FamilyMerchantAssociation.where(family: family, merchant: merchant).delete_all
end
end