mirror of
https://github.com/we-promise/sure.git
synced 2026-04-08 14:54:49 +00:00
* Add pending transaction handling and duplicate reconciliation logic - Implemented logic to exclude pending transactions from budgets and analytics calculations. - Introduced mechanisms for reconciling pending transactions with posted versions. - Added duplicate detection with support for merging or dismissing matches. - Updated transaction search filters to include a `status_filter` for pending/confirmed transactions. - Introduced UI elements for reviewing and resolving duplicates. - Enhanced `ProviderSyncSummary` with stats for reconciled and stale pending transactions. * Refactor translation handling and enhance transaction and sync logic - Moved hardcoded strings to locale files for improved translation support. - Refined styling for duplicate transaction indicators and sync summaries. - Improved logic for excluding stale pending transactions and updating timestamps on batch exclusion. - Added unique IDs to status filters for better element targeting in UI. - Optimized database queries to avoid N+1 issues in stale pending calculations. * Add sync settings and enhance pending transaction handling - Introduced a new "Sync Settings" section in hosting settings with UI to toggle inclusion of pending transactions. - Updated handling of pending transactions with improved inference logic for `posted=0` and `transacted_at` in processors. - Added priority order for pending transaction inclusion: explicit argument > environment variable > runtime configurable setting. - Refactored settings and controllers to store updated sync preferences. * Refactor sync settings and pending transaction reconciliation - Extracted logic for pending transaction reconciliation, stale exclusion, and unmatched tracking into dedicated methods for better maintainability. - Updated sync settings to infer defaults from multiple provider environment variables (`SIMPLEFIN_INCLUDE_PENDING`, `PLAID_INCLUDE_PENDING`). - Refined UI and messaging to handle multi-provider configurations in sync settings. # Conflicts: # app/models/simplefin_item/importer.rb * Debounce transaction reconciliation during imports - Added per-run reconciliation debouncing to prevent repeated scans for the same account during chunked history imports. - Trimmed size of reconciliation stats to retain recent details only. - Introduced error tracking for reconciliation steps to improve UI visibility of issues. * Apply ABS() in pending transaction queries and improve error handling - Updated pending transaction logic to use ABS() for consistent handling of negative amounts. - Adjusted amount bounds calculations to ensure accuracy for both positive and negative values. - Refined exception handling in `merge_duplicate` to log failures and update user alert. - Replaced `Date.today` with `Date.current` in tests to ensure timezone consistency. - Minor optimization to avoid COUNT queries by loading limited records directly. * Improve error handling in duplicate suggestion and dismissal logic - Added exception handling for `store_duplicate_suggestion` to log failures and prevent crashes during fuzzy/low-confidence matches. - Enhanced `dismiss_duplicate` action to handle `ActiveRecord::RecordInvalid` and display appropriate user alerts. --------- Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
183 lines
6.3 KiB
Ruby
183 lines
6.3 KiB
Ruby
class Transaction::Search
|
|
include ActiveModel::Model
|
|
include ActiveModel::Attributes
|
|
|
|
attribute :search, :string
|
|
attribute :amount, :string
|
|
attribute :amount_operator, :string
|
|
attribute :types, array: true
|
|
attribute :status, array: true
|
|
attribute :accounts, array: true
|
|
attribute :account_ids, array: true
|
|
attribute :start_date, :string
|
|
attribute :end_date, :string
|
|
attribute :categories, array: true
|
|
attribute :merchants, array: true
|
|
attribute :tags, array: true
|
|
attribute :active_accounts_only, :boolean, default: true
|
|
|
|
attr_reader :family
|
|
|
|
def initialize(family, filters: {})
|
|
@family = family
|
|
super(filters)
|
|
end
|
|
|
|
def transactions_scope
|
|
@transactions_scope ||= begin
|
|
# This already joins entries + accounts. To avoid expensive double-joins, don't join them again (causes full table scan)
|
|
query = family.transactions
|
|
|
|
query = apply_active_accounts_filter(query, active_accounts_only)
|
|
query = apply_category_filter(query, categories)
|
|
query = apply_type_filter(query, types)
|
|
query = apply_status_filter(query, status)
|
|
query = apply_merchant_filter(query, merchants)
|
|
query = apply_tag_filter(query, tags)
|
|
query = EntrySearch.apply_search_filter(query, search)
|
|
query = EntrySearch.apply_date_filters(query, start_date, end_date)
|
|
query = EntrySearch.apply_amount_filter(query, amount, amount_operator)
|
|
query = EntrySearch.apply_accounts_filter(query, accounts, account_ids)
|
|
|
|
query
|
|
end
|
|
end
|
|
|
|
# Computes totals for the specific search
|
|
def totals
|
|
@totals ||= begin
|
|
Rails.cache.fetch("transaction_search_totals/#{cache_key_base}") do
|
|
result = transactions_scope
|
|
.select(
|
|
"COALESCE(SUM(CASE WHEN entries.amount >= 0 AND transactions.kind NOT IN ('funds_movement', 'cc_payment') THEN ABS(entries.amount * COALESCE(er.rate, 1)) ELSE 0 END), 0) as expense_total",
|
|
"COALESCE(SUM(CASE WHEN entries.amount < 0 AND transactions.kind NOT IN ('funds_movement', 'cc_payment') THEN ABS(entries.amount * COALESCE(er.rate, 1)) ELSE 0 END), 0) as income_total",
|
|
"COUNT(entries.id) as transactions_count"
|
|
)
|
|
.joins(
|
|
ActiveRecord::Base.sanitize_sql_array([
|
|
"LEFT JOIN exchange_rates er ON (er.date = entries.date AND er.from_currency = entries.currency AND er.to_currency = ?)",
|
|
family.currency
|
|
])
|
|
)
|
|
.take
|
|
|
|
Totals.new(
|
|
count: result.transactions_count.to_i,
|
|
income_money: Money.new(result.income_total, family.currency),
|
|
expense_money: Money.new(result.expense_total, family.currency)
|
|
)
|
|
end
|
|
end
|
|
end
|
|
|
|
def cache_key_base
|
|
[
|
|
family.id,
|
|
Digest::SHA256.hexdigest(attributes.sort.to_h.to_json), # cached by filters
|
|
family.entries_cache_version
|
|
].join("/")
|
|
end
|
|
|
|
private
|
|
Totals = Data.define(:count, :income_money, :expense_money)
|
|
|
|
def apply_active_accounts_filter(query, active_accounts_only_filter)
|
|
if active_accounts_only_filter
|
|
query.where(accounts: { status: [ "draft", "active" ] })
|
|
else
|
|
query
|
|
end
|
|
end
|
|
|
|
|
|
def apply_category_filter(query, categories)
|
|
return query unless categories.present?
|
|
|
|
# Get parent category IDs for the given category names
|
|
parent_category_ids = family.categories.where(name: categories).pluck(:id)
|
|
|
|
# Build condition based on whether parent_category_ids is empty
|
|
if parent_category_ids.empty?
|
|
query = query.left_joins(:category).where(
|
|
"categories.name IN (?) OR (
|
|
categories.id IS NULL AND (transactions.kind NOT IN ('funds_movement', 'cc_payment'))
|
|
)",
|
|
categories
|
|
)
|
|
else
|
|
query = query.left_joins(:category).where(
|
|
"categories.name IN (?) OR categories.parent_id IN (?) OR (
|
|
categories.id IS NULL AND (transactions.kind NOT IN ('funds_movement', 'cc_payment'))
|
|
)",
|
|
categories, parent_category_ids
|
|
)
|
|
end
|
|
|
|
if categories.exclude?("Uncategorized")
|
|
query = query.where.not(category_id: nil)
|
|
end
|
|
|
|
query
|
|
end
|
|
|
|
def apply_type_filter(query, types)
|
|
return query unless types.present?
|
|
return query if types.sort == [ "expense", "income", "transfer" ]
|
|
|
|
transfer_condition = "transactions.kind IN ('funds_movement', 'cc_payment', 'loan_payment')"
|
|
expense_condition = "entries.amount >= 0"
|
|
income_condition = "entries.amount <= 0"
|
|
|
|
condition = case types.sort
|
|
when [ "transfer" ]
|
|
transfer_condition
|
|
when [ "expense" ]
|
|
Arel.sql("#{expense_condition} AND NOT (#{transfer_condition})")
|
|
when [ "income" ]
|
|
Arel.sql("#{income_condition} AND NOT (#{transfer_condition})")
|
|
when [ "expense", "transfer" ]
|
|
Arel.sql("#{expense_condition} OR #{transfer_condition}")
|
|
when [ "income", "transfer" ]
|
|
Arel.sql("#{income_condition} OR #{transfer_condition}")
|
|
when [ "expense", "income" ]
|
|
Arel.sql("NOT (#{transfer_condition})")
|
|
end
|
|
|
|
query.where(condition)
|
|
end
|
|
|
|
def apply_merchant_filter(query, merchants)
|
|
return query unless merchants.present?
|
|
query.joins(:merchant).where(merchants: { name: merchants })
|
|
end
|
|
|
|
def apply_tag_filter(query, tags)
|
|
return query unless tags.present?
|
|
query.joins(:tags).where(tags: { name: tags })
|
|
end
|
|
|
|
def apply_status_filter(query, statuses)
|
|
return query unless statuses.present?
|
|
return query if statuses.uniq.sort == [ "confirmed", "pending" ] # Both selected = no filter
|
|
|
|
pending_condition = <<~SQL.squish
|
|
(transactions.extra -> 'simplefin' ->> 'pending')::boolean = true
|
|
OR (transactions.extra -> 'plaid' ->> 'pending')::boolean = true
|
|
SQL
|
|
|
|
confirmed_condition = <<~SQL.squish
|
|
(transactions.extra -> 'simplefin' ->> 'pending')::boolean IS DISTINCT FROM true
|
|
AND (transactions.extra -> 'plaid' ->> 'pending')::boolean IS DISTINCT FROM true
|
|
SQL
|
|
|
|
case statuses.sort
|
|
when [ "pending" ]
|
|
query.where(pending_condition)
|
|
when [ "confirmed" ]
|
|
query.where(confirmed_condition)
|
|
else
|
|
query
|
|
end
|
|
end
|
|
end
|