Files
sure/app/models/entry_search.rb
LPW 3658e812a8 Add pending transaction handling and duplicate reconciliation logic (#602)
* Add pending transaction handling and duplicate reconciliation logic

- Implemented logic to exclude pending transactions from budgets and analytics calculations.
- Introduced mechanisms for reconciling pending transactions with posted versions.
- Added duplicate detection with support for merging or dismissing matches.
- Updated transaction search filters to include a `status_filter` for pending/confirmed transactions.
- Introduced UI elements for reviewing and resolving duplicates.
- Enhanced `ProviderSyncSummary` with stats for reconciled and stale pending transactions.

* Refactor translation handling and enhance transaction and sync logic

- Moved hardcoded strings to locale files for improved translation support.
- Refined styling for duplicate transaction indicators and sync summaries.
- Improved logic for excluding stale pending transactions and updating timestamps on batch exclusion.
- Added unique IDs to status filters for better element targeting in UI.
- Optimized database queries to avoid N+1 issues in stale pending calculations.

* Add sync settings and enhance pending transaction handling

- Introduced a new "Sync Settings" section in hosting settings with UI to toggle inclusion of pending transactions.
- Updated handling of pending transactions with improved inference logic for `posted=0` and `transacted_at` in processors.
- Added priority order for pending transaction inclusion: explicit argument > environment variable > runtime configurable setting.
- Refactored settings and controllers to store updated sync preferences.

* Refactor sync settings and pending transaction reconciliation

- Extracted logic for pending transaction reconciliation, stale exclusion, and unmatched tracking into dedicated methods for better maintainability.
- Updated sync settings to infer defaults from multiple provider environment variables (`SIMPLEFIN_INCLUDE_PENDING`, `PLAID_INCLUDE_PENDING`).
- Refined UI and messaging to handle multi-provider configurations in sync settings.

# Conflicts:
#	app/models/simplefin_item/importer.rb

* Debounce transaction reconciliation during imports

- Added per-run reconciliation debouncing to prevent repeated scans for the same account during chunked history imports.
- Trimmed size of reconciliation stats to retain recent details only.
- Introduced error tracking for reconciliation steps to improve UI visibility of issues.

* Apply ABS() in pending transaction queries and improve error handling

- Updated pending transaction logic to use ABS() for consistent handling of negative amounts.
- Adjusted amount bounds calculations to ensure accuracy for both positive and negative values.
- Refined exception handling in `merge_duplicate` to log failures and update user alert.
- Replaced `Date.today` with `Date.current` in tests to ensure timezone consistency.
- Minor optimization to avoid COUNT queries by loading limited records directly.

* Improve error handling in duplicate suggestion and dismissal logic

- Added exception handling for `store_duplicate_suggestion` to log failures and prevent crashes during fuzzy/low-confidence matches.
- Enhanced `dismiss_duplicate` action to handle `ActiveRecord::RecordInvalid` and display appropriate user alerts.

---------

Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
2026-01-10 20:11:00 +01:00

110 lines
3.3 KiB
Ruby

class EntrySearch
include ActiveModel::Model
include ActiveModel::Attributes
attribute :search, :string
attribute :amount, :string
attribute :amount_operator, :string
attribute :types, :string
attribute :status, array: true
attribute :accounts, array: true
attribute :account_ids, array: true
attribute :start_date, :string
attribute :end_date, :string
class << self
def apply_search_filter(scope, search)
return scope if search.blank?
query = scope
query = query.where("entries.name ILIKE :search",
search: "%#{ActiveRecord::Base.sanitize_sql_like(search)}%"
)
query
end
def apply_date_filters(scope, start_date, end_date)
return scope if start_date.blank? && end_date.blank?
query = scope
query = query.where("entries.date >= ?", start_date) if start_date.present?
query = query.where("entries.date <= ?", end_date) if end_date.present?
query
end
def apply_amount_filter(scope, amount, amount_operator)
return scope if amount.blank? || amount_operator.blank?
query = scope
case amount_operator
when "equal"
query = query.where("ABS(ABS(entries.amount) - ?) <= 0.01", amount.to_f.abs)
when "less"
query = query.where("ABS(entries.amount) < ?", amount.to_f.abs)
when "greater"
query = query.where("ABS(entries.amount) > ?", amount.to_f.abs)
end
query
end
def apply_accounts_filter(scope, accounts, account_ids)
return scope if accounts.blank? && account_ids.blank?
query = scope
query = query.where(accounts: { name: accounts }) if accounts.present?
query = query.where(accounts: { id: account_ids }) if account_ids.present?
query
end
def apply_status_filter(scope, statuses)
return scope unless statuses.present?
return scope if statuses.uniq.sort == %w[confirmed pending] # Both selected = no filter
pending_condition = <<~SQL.squish
entries.entryable_type = 'Transaction'
AND EXISTS (
SELECT 1 FROM transactions t
WHERE t.id = entries.entryable_id
AND (
(t.extra -> 'simplefin' ->> 'pending')::boolean = true
OR (t.extra -> 'plaid' ->> 'pending')::boolean = true
)
)
SQL
confirmed_condition = <<~SQL.squish
entries.entryable_type != 'Transaction'
OR NOT EXISTS (
SELECT 1 FROM transactions t
WHERE t.id = entries.entryable_id
AND (
(t.extra -> 'simplefin' ->> 'pending')::boolean = true
OR (t.extra -> 'plaid' ->> 'pending')::boolean = true
)
)
SQL
case statuses.sort
when [ "pending" ]
scope.where(pending_condition)
when [ "confirmed" ]
scope.where(confirmed_condition)
else
scope
end
end
end
def build_query(scope)
query = scope.joins(:account)
query = self.class.apply_search_filter(query, search)
query = self.class.apply_date_filters(query, start_date, end_date)
query = self.class.apply_amount_filter(query, amount, amount_operator)
query = self.class.apply_accounts_filter(query, accounts, account_ids)
query = self.class.apply_status_filter(query, status)
query
end
end