Files
sure/app/models/income_statement.rb
LPW 3658e812a8 Add pending transaction handling and duplicate reconciliation logic (#602)
* Add pending transaction handling and duplicate reconciliation logic

- Implemented logic to exclude pending transactions from budgets and analytics calculations.
- Introduced mechanisms for reconciling pending transactions with posted versions.
- Added duplicate detection with support for merging or dismissing matches.
- Updated transaction search filters to include a `status_filter` for pending/confirmed transactions.
- Introduced UI elements for reviewing and resolving duplicates.
- Enhanced `ProviderSyncSummary` with stats for reconciled and stale pending transactions.

* Refactor translation handling and enhance transaction and sync logic

- Moved hardcoded strings to locale files for improved translation support.
- Refined styling for duplicate transaction indicators and sync summaries.
- Improved logic for excluding stale pending transactions and updating timestamps on batch exclusion.
- Added unique IDs to status filters for better element targeting in UI.
- Optimized database queries to avoid N+1 issues in stale pending calculations.

* Add sync settings and enhance pending transaction handling

- Introduced a new "Sync Settings" section in hosting settings with UI to toggle inclusion of pending transactions.
- Updated handling of pending transactions with improved inference logic for `posted=0` and `transacted_at` in processors.
- Added priority order for pending transaction inclusion: explicit argument > environment variable > runtime configurable setting.
- Refactored settings and controllers to store updated sync preferences.

* Refactor sync settings and pending transaction reconciliation

- Extracted logic for pending transaction reconciliation, stale exclusion, and unmatched tracking into dedicated methods for better maintainability.
- Updated sync settings to infer defaults from multiple provider environment variables (`SIMPLEFIN_INCLUDE_PENDING`, `PLAID_INCLUDE_PENDING`).
- Refined UI and messaging to handle multi-provider configurations in sync settings.

# Conflicts:
#	app/models/simplefin_item/importer.rb

* Debounce transaction reconciliation during imports

- Added per-run reconciliation debouncing to prevent repeated scans for the same account during chunked history imports.
- Trimmed size of reconciliation stats to retain recent details only.
- Introduced error tracking for reconciliation steps to improve UI visibility of issues.

* Apply ABS() in pending transaction queries and improve error handling

- Updated pending transaction logic to use ABS() for consistent handling of negative amounts.
- Adjusted amount bounds calculations to ensure accuracy for both positive and negative values.
- Refined exception handling in `merge_duplicate` to log failures and update user alert.
- Replaced `Date.today` with `Date.current` in tests to ensure timezone consistency.
- Minor optimization to avoid COUNT queries by loading limited records directly.

* Improve error handling in duplicate suggestion and dismissal logic

- Added exception handling for `store_duplicate_suggestion` to log failures and prevent crashes during fuzzy/low-confidence matches.
- Enhanced `dismiss_duplicate` action to handle `ActiveRecord::RecordInvalid` and display appropriate user alerts.

---------

Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
2026-01-10 20:11:00 +01:00

141 lines
5.2 KiB
Ruby

require "digest/md5"
class IncomeStatement
include Monetizable
monetize :median_expense, :median_income
attr_reader :family
def initialize(family)
@family = family
end
def totals(transactions_scope: nil, date_range:)
# Default to excluding pending transactions from budget/analytics calculations
# Pending transactions shouldn't affect budget totals until they post
transactions_scope ||= family.transactions.visible.excluding_pending
result = totals_query(transactions_scope: transactions_scope, date_range: date_range)
total_income = result.select { |t| t.classification == "income" }.sum(&:total)
total_expense = result.select { |t| t.classification == "expense" }.sum(&:total)
ScopeTotals.new(
transactions_count: result.sum(&:transactions_count),
income_money: Money.new(total_income, family.currency),
expense_money: Money.new(total_expense, family.currency)
)
end
def expense_totals(period: Period.current_month)
build_period_total(classification: "expense", period: period)
end
def income_totals(period: Period.current_month)
build_period_total(classification: "income", period: period)
end
def median_expense(interval: "month", category: nil)
if category.present?
category_stats(interval: interval).find { |stat| stat.classification == "expense" && stat.category_id == category.id }&.median || 0
else
family_stats(interval: interval).find { |stat| stat.classification == "expense" }&.median || 0
end
end
def avg_expense(interval: "month", category: nil)
if category.present?
category_stats(interval: interval).find { |stat| stat.classification == "expense" && stat.category_id == category.id }&.avg || 0
else
family_stats(interval: interval).find { |stat| stat.classification == "expense" }&.avg || 0
end
end
def median_income(interval: "month")
family_stats(interval: interval).find { |stat| stat.classification == "income" }&.median || 0
end
private
ScopeTotals = Data.define(:transactions_count, :income_money, :expense_money)
PeriodTotal = Data.define(:classification, :total, :currency, :category_totals)
CategoryTotal = Data.define(:category, :total, :currency, :weight)
def categories
@categories ||= family.categories.all.to_a
end
def build_period_total(classification:, period:)
# Exclude pending transactions from budget calculations
totals = totals_query(transactions_scope: family.transactions.visible.excluding_pending.in_period(period), date_range: period.date_range).select { |t| t.classification == classification }
classification_total = totals.sum(&:total)
uncategorized_category = family.categories.uncategorized
other_investments_category = family.categories.other_investments
category_totals = [ *categories, uncategorized_category, other_investments_category ].map do |category|
subcategory = categories.find { |c| c.id == category.parent_id }
parent_category_total = if category.uncategorized?
# Regular uncategorized: NULL category_id and NOT uncategorized investment
totals.select { |t| t.category_id.nil? && !t.is_uncategorized_investment }&.sum(&:total) || 0
elsif category.other_investments?
# Other investments: NULL category_id AND is_uncategorized_investment
totals.select { |t| t.category_id.nil? && t.is_uncategorized_investment }&.sum(&:total) || 0
else
totals.select { |t| t.category_id == category.id }&.sum(&:total) || 0
end
children_totals = if category.synthetic?
0
else
totals.select { |t| t.parent_category_id == category.id }&.sum(&:total) || 0
end
category_total = parent_category_total + children_totals
weight = (category_total.zero? ? 0 : category_total.to_f / classification_total) * 100
CategoryTotal.new(
category: category,
total: category_total,
currency: family.currency,
weight: weight,
)
end
PeriodTotal.new(
classification: classification,
total: category_totals.reject { |ct| ct.category.subcategory? }.sum(&:total),
currency: family.currency,
category_totals: category_totals
)
end
def family_stats(interval: "month")
@family_stats ||= {}
@family_stats[interval] ||= Rails.cache.fetch([
"income_statement", "family_stats", family.id, interval, family.entries_cache_version
]) { FamilyStats.new(family, interval:).call }
end
def category_stats(interval: "month")
@category_stats ||= {}
@category_stats[interval] ||= Rails.cache.fetch([
"income_statement", "category_stats", family.id, interval, family.entries_cache_version
]) { CategoryStats.new(family, interval:).call }
end
def totals_query(transactions_scope:, date_range:)
sql_hash = Digest::MD5.hexdigest(transactions_scope.to_sql)
Rails.cache.fetch([
"income_statement", "totals_query", family.id, sql_hash, family.entries_cache_version
]) { Totals.new(family, transactions_scope: transactions_scope, date_range: date_range).call }
end
def monetizable_currency
family.currency
end
end