Files
sure/app/views/simplefin_items/_simplefin_item.html.erb
LPW 3658e812a8 Add pending transaction handling and duplicate reconciliation logic (#602)
* Add pending transaction handling and duplicate reconciliation logic

- Implemented logic to exclude pending transactions from budgets and analytics calculations.
- Introduced mechanisms for reconciling pending transactions with posted versions.
- Added duplicate detection with support for merging or dismissing matches.
- Updated transaction search filters to include a `status_filter` for pending/confirmed transactions.
- Introduced UI elements for reviewing and resolving duplicates.
- Enhanced `ProviderSyncSummary` with stats for reconciled and stale pending transactions.

* Refactor translation handling and enhance transaction and sync logic

- Moved hardcoded strings to locale files for improved translation support.
- Refined styling for duplicate transaction indicators and sync summaries.
- Improved logic for excluding stale pending transactions and updating timestamps on batch exclusion.
- Added unique IDs to status filters for better element targeting in UI.
- Optimized database queries to avoid N+1 issues in stale pending calculations.

* Add sync settings and enhance pending transaction handling

- Introduced a new "Sync Settings" section in hosting settings with UI to toggle inclusion of pending transactions.
- Updated handling of pending transactions with improved inference logic for `posted=0` and `transacted_at` in processors.
- Added priority order for pending transaction inclusion: explicit argument > environment variable > runtime configurable setting.
- Refactored settings and controllers to store updated sync preferences.

* Refactor sync settings and pending transaction reconciliation

- Extracted logic for pending transaction reconciliation, stale exclusion, and unmatched tracking into dedicated methods for better maintainability.
- Updated sync settings to infer defaults from multiple provider environment variables (`SIMPLEFIN_INCLUDE_PENDING`, `PLAID_INCLUDE_PENDING`).
- Refined UI and messaging to handle multi-provider configurations in sync settings.

# Conflicts:
#	app/models/simplefin_item/importer.rb

* Debounce transaction reconciliation during imports

- Added per-run reconciliation debouncing to prevent repeated scans for the same account during chunked history imports.
- Trimmed size of reconciliation stats to retain recent details only.
- Introduced error tracking for reconciliation steps to improve UI visibility of issues.

* Apply ABS() in pending transaction queries and improve error handling

- Updated pending transaction logic to use ABS() for consistent handling of negative amounts.
- Adjusted amount bounds calculations to ensure accuracy for both positive and negative values.
- Refined exception handling in `merge_duplicate` to log failures and update user alert.
- Replaced `Date.today` with `Date.current` in tests to ensure timezone consistency.
- Minor optimization to avoid COUNT queries by loading limited records directly.

* Improve error handling in duplicate suggestion and dismissal logic

- Added exception handling for `store_duplicate_suggestion` to log failures and prevent crashes during fuzzy/low-confidence matches.
- Enhanced `dismiss_duplicate` action to handle `ActiveRecord::RecordInvalid` and display appropriate user alerts.

---------

Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
2026-01-10 20:11:00 +01:00

246 lines
12 KiB
Plaintext

<%# locals: (simplefin_item:) %>
<%= tag.div id: dom_id(simplefin_item) do %>
<details open class="group bg-container p-4 shadow-border-xs rounded-xl">
<summary class="flex items-center justify-between gap-2">
<div class="flex items-center gap-2">
<%= icon "chevron-right", class: "group-open:transform group-open:rotate-90" %>
<div class="flex items-center justify-center h-8 w-8 bg-success/10 rounded-full">
<% if simplefin_item.logo.attached? %>
<%= image_tag simplefin_item.logo, class: "rounded-full h-full w-full", loading: "lazy" %>
<% else %>
<div class="flex items-center justify-center">
<%= tag.p simplefin_item.name.first.upcase, class: "text-success text-xs font-medium" %>
</div>
<% end %>
</div>
<%# Compute unlinked count early for badge display %>
<% header_unlinked_count = if defined?(@simplefin_unlinked_count_map) && @simplefin_unlinked_count_map
@simplefin_unlinked_count_map[simplefin_item.id] || 0
else
begin
simplefin_item.simplefin_accounts
.left_joins(:account, :account_provider)
.where(accounts: { id: nil }, account_providers: { id: nil })
.count
rescue => e
0
end
end %>
<div class="pl-1 text-sm">
<div class="flex items-center gap-2">
<%= tag.p simplefin_item.name, class: "font-medium text-primary" %>
<% if header_unlinked_count.to_i > 0 %>
<%= link_to setup_accounts_simplefin_item_path(simplefin_item),
data: { turbo_frame: :modal },
class: "inline-flex items-center gap-1 px-2 py-0.5 rounded-full text-xs font-medium bg-warning/10 text-warning hover:bg-warning/20 transition-colors" do %>
<%= icon "alert-circle", size: "xs" %>
<span><%= header_unlinked_count %> <%= header_unlinked_count == 1 ? "account" : "accounts" %> need setup</span>
<% end %>
<% end %>
<% if simplefin_item.scheduled_for_deletion? %>
<p class="text-destructive text-sm animate-pulse"><%= t(".deletion_in_progress") %></p>
<% end %>
</div>
<% if simplefin_item.accounts.any? %>
<p class="text-xs text-secondary">
<%= simplefin_item.institution_summary %>
</p>
<%# Extra inline badges from latest sync stats %>
<% stats = (@simplefin_sync_stats_map || {})[simplefin_item.id] || {} %>
<% if stats.present? %>
<div class="flex items-center gap-2 mt-1">
<% if stats["unlinked_accounts"].to_i > 0 %>
<%= render DS::Tooltip.new(text: "Accounts need setup", icon: "link-2", size: "sm") %>
<span class="text-xs text-secondary">Unlinked: <%= stats["unlinked_accounts"].to_i %></span>
<% end %>
<% if stats["accounts_skipped"].to_i > 0 %>
<%= render DS::Tooltip.new(text: "Some accounts were skipped due to errors during sync", icon: "alert-triangle", size: "sm", color: "warning") %>
<span class="text-xs text-warning">Skipped: <%= stats["accounts_skipped"].to_i %></span>
<% end %>
<% if stats["rate_limited"].present? || stats["rate_limited_at"].present? %>
<% ts = stats["rate_limited_at"] %>
<% ago = (ts.present? ? (begin; time_ago_in_words(Time.parse(ts)); rescue StandardError; nil; end) : nil) %>
<%= render DS::Tooltip.new(
text: (ago ? "Rate limited (" + ago + " ago)" : "Rate limited recently"),
icon: "clock",
size: "sm",
color: "warning"
) %>
<% end %>
<% if stats["total_errors"].to_i > 0 || (stats["errors"].is_a?(Array) && stats["errors"].any?) %>
<% tooltip_text = simplefin_error_tooltip(stats) %>
<% if tooltip_text.present? %>
<%= render DS::Tooltip.new(text: tooltip_text, icon: "alert-octagon", size: "sm", color: "warning") %>
<% end %>
<% end %>
<% if stats["total_accounts"].to_i > 0 %>
<span class="text-xs text-secondary">Total: <%= stats["total_accounts"].to_i %></span>
<% end %>
</div>
<% end %>
<% end %>
<%# Determine if all reported errors are benign duplicate-skips (suppress scary banner). Computed in controller for testability. %>
<% duplicate_only_errors = (@simplefin_duplicate_only_map || {})[simplefin_item.id] || false %>
<% if simplefin_item.syncing? %>
<div class="text-secondary flex items-center gap-1">
<%= icon "loader", size: "sm", class: "animate-spin" %>
<%= tag.span t(".syncing") %>
</div>
<% elsif simplefin_item.requires_update? %>
<div class="text-warning flex items-center gap-1">
<%= icon "alert-triangle", size: "sm", color: "warning" %>
<%= tag.span t(".requires_update") %>
</div>
<% elsif (stale_status = simplefin_item.stale_sync_status)[:stale] %>
<div class="text-warning flex items-center gap-1">
<%= icon "alert-circle", size: "sm", color: "warning" %>
<%= tag.span stale_status[:message], class: "text-sm" %>
</div>
<% elsif (pending_status = simplefin_item.stale_pending_status)[:count] > 0 %>
<div class="text-secondary">
<div class="flex items-center gap-1">
<%= icon "clock", size: "sm", color: "secondary" %>
<%= tag.span pending_status[:message], class: "text-sm" %>
<span class="text-xs text-tertiary"><%= t(".stale_pending_note") %></span>
</div>
<% if pending_status[:accounts]&.any? %>
<div class="text-xs text-tertiary ml-5">
<%= t(".stale_pending_accounts", accounts: pending_status[:accounts].join(", ")) %>
</div>
<% end %>
</div>
<% elsif (reconciled_status = simplefin_item.last_sync_reconciled_status)[:count] > 0 %>
<div class="text-success flex items-center gap-1">
<%= icon "check-circle", size: "sm", color: "success" %>
<%= tag.span reconciled_status[:message], class: "text-sm" %>
<span class="text-xs text-tertiary"><%= t(".reconciled_details_note") %></span>
</div>
<% elsif simplefin_item.rate_limited_message.present? %>
<div class="text-warning flex items-center gap-1">
<%= icon "clock", size: "sm", color: "warning" %>
<%= tag.span simplefin_item.rate_limited_message %>
</div>
<% elsif simplefin_item.sync_error.present? && !duplicate_only_errors %>
<div class="text-secondary flex items-center gap-1">
<%= render DS::Tooltip.new(text: simplefin_item.sync_error, icon: "alert-circle", size: "sm", color: "destructive") %>
<%= tag.span t(".error"), class: "text-destructive" %>
</div>
<% elsif duplicate_only_errors %>
<div class="text-secondary flex items-center gap-1">
<%= icon "info", size: "sm" %>
<%= tag.span t(".duplicate_accounts_skipped"), class: "text-secondary" %>
</div>
<% else %>
<p class="text-secondary">
<% if simplefin_item.last_synced_at %>
<% if simplefin_item.sync_status_summary %>
<%= t(".status_with_summary", timestamp: time_ago_in_words(simplefin_item.last_synced_at), summary: simplefin_item.sync_status_summary) %>
<% else %>
<%= t(".status", timestamp: time_ago_in_words(simplefin_item.last_synced_at)) %>
<% end %>
<% else %>
<%= t(".status_never") %>
<% end %>
</p>
<% end %>
</div>
</div>
<div class="flex items-center gap-2">
<% if simplefin_item.requires_update? %>
<%= render DS::Link.new(
text: t(".update"),
icon: "refresh-cw",
variant: "secondary",
href: edit_simplefin_item_path(simplefin_item),
frame: "modal"
) %>
<% elsif Rails.env.development? %>
<%= icon(
"refresh-cw",
as_button: true,
href: sync_simplefin_item_path(simplefin_item)
) %>
<% end %>
<%= render DS::Menu.new do |menu| %>
<% menu.with_item(
variant: "button",
text: t(".delete"),
icon: "trash-2",
href: simplefin_item_path(simplefin_item),
method: :delete,
confirm: CustomConfirm.for_resource_deletion(simplefin_item.name, high_severity: true)
) %>
<% end %>
</div>
</summary>
<% unless simplefin_item.scheduled_for_deletion? %>
<div class="space-y-4 mt-4">
<% if simplefin_item.accounts.any? %>
<%= render "accounts/index/account_groups", accounts: simplefin_item.accounts %>
<% end %>
<%# Sync summary (collapsible) - using shared ProviderSyncSummary component
Prefer controller-provided map; fallback to latest sync stats so Turbo broadcasts
can render the summary without requiring a full page refresh. %>
<% stats = if defined?(@simplefin_sync_stats_map) && @simplefin_sync_stats_map
@simplefin_sync_stats_map[simplefin_item.id] || {}
else
# `latest_sync` is private on Syncable; access via association for broadcast renders.
simplefin_item.syncs.ordered.first&.sync_stats || {}
end %>
<%= render ProviderSyncSummary.new(
stats: stats,
provider_item: simplefin_item,
institutions_count: simplefin_item.connected_institutions.size
) %>
<%# Compute unlinked SimpleFin accounts (no legacy account and no AccountProvider link)
# Prefer controller-provided map; fallback to a local query so the card stays accurate after Turbo broadcasts %>
<% unlinked_count = if defined?(@simplefin_unlinked_count_map) && @simplefin_unlinked_count_map
@simplefin_unlinked_count_map[simplefin_item.id] || 0
else
begin
simplefin_item.simplefin_accounts
.left_joins(:account, :account_provider)
.where(accounts: { id: nil }, account_providers: { id: nil })
.count
rescue => e
Rails.logger.warn("SimpleFin card: unlinked_count fallback failed: #{e.class} - #{e.message}")
0
end
end %>
<% if unlinked_count.to_i > 0 %>
<div class="p-4 flex flex-col gap-3 items-center justify-center">
<p class="text-primary font-medium text-sm"><%= t(".setup_needed") %></p>
<p class="text-secondary text-sm"><%= t(".setup_description") %></p>
<%= render DS::Link.new(
text: t(".setup_action"),
icon: "settings",
variant: "primary",
href: setup_accounts_simplefin_item_path(simplefin_item),
frame: :modal
) %>
</div>
<% elsif simplefin_item.accounts.empty? %>
<div class="p-4 flex flex-col gap-3 items-center justify-center">
<p class="text-primary font-medium text-sm"><%= t(".no_accounts_title") %></p>
<p class="text-secondary text-sm"><%= t(".no_accounts_description") %></p>
</div>
<% end %>
</div>
<% end %>
</details>
<% end %>