mirror of
https://github.com/we-promise/sure.git
synced 2026-04-19 12:04:08 +00:00
Harden SimpleFin sync: retries, safer imports, manual relinking, and data-quality reconciliation (#544)
* Add tests and enhance logic for SimpleFin account synchronization and reconciliation - Added retry logic with exponential backoff for network errors in `Provider::Simplefin`. - Introduced tests to verify retry functionality and error handling for rate-limit, server errors, and stale data. - Updated `SimplefinItem` to detect stale sync status and reconciliation issues. - Enhanced UI to display stale sync warnings and data integrity notices. - Improved SimpleFin account matching during updates with multi-tier strategy (ID, fingerprint, fuzzy match). - Added transaction reconciliation logic to detect data gaps, transaction count drops, and duplicate transaction IDs. * Introduce `SimplefinConnectionUpdateJob` for asynchronous SimpleFin connection updates - Moved SimpleFin connection update logic to `SimplefinConnectionUpdateJob` to improve response times by offloading network retries, data fetching, and reconciliation tasks. - Enhanced SimpleFin account matching with a multi-tier strategy (ID, fingerprint, fuzzy name match). - Added retry logic and bounded latency for token claim requests in `Provider::Simplefin`. - Updated tests to cover the new job flow and ensure correct account reconciliation during updates. * Remove unused SimpleFin account matching logic and improve error handling in `SimplefinConnectionUpdateJob` - Deleted the multi-tier account matching logic from `SimplefinItemsController` as it is no longer used. - Enhanced error handling in `SimplefinConnectionUpdateJob` to gracefully handle import failures, ensuring orphaned items can be manually resolved. - Updated job flow to conditionally set item status based on the success of import operations. * Fix SimpleFin sync: check both legacy FK and AccountProvider for linked accounts * Add crypto, checking, savings, and cash account detection; refine subtype selection and linking - Enhanced `Simplefin::AccountTypeMapper` to include detection for crypto, checking, savings, and standalone cash accounts. - Improved subtype selection UI with validation and warning indicators for missing selections. - Updated SimpleFin account linking to handle both legacy FK and `AccountProvider` associations consistently. - Refined job flow and importer logic for better handling of linked accounts and subtype inference. * Improve `SimplefinConnectionUpdateJob` and holdings processing logic - Fixed race condition in `SimplefinConnectionUpdateJob` by moving `destroy_later` calls outside of transactions. - Updated fuzzy name match logic to use Levenshtein distance for better accuracy. - Enhanced synthetic ticker generation in holdings processor with hash suffix for uniqueness. * Refine SimpleFin entry processing logic and ensure `extra` data persistence - Simplified pending flag determination to rely solely on provider-supplied values. - Fixed potential stale values in `extra` by ensuring deep merge overwrite with `entry.transaction.save!`. * Replace hardcoded fallback transaction description with localized string * Refine pending flag logic in SimpleFin processor tests - Adjust test to prevent falsely inferring pending status from missing posted dates. - Ensure provider explicitly sets pending flag for transactions. * Add `has_many :holdings` association to `AccountProvider` with `dependent: :nullify` --------- Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
This commit is contained in:
@@ -39,27 +39,31 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Sync Date Range Selection -->
|
||||
<!-- Sync Data Information -->
|
||||
<div class="bg-surface border border-primary p-4 rounded-lg">
|
||||
<div class="flex items-start gap-3">
|
||||
<%= icon "calendar", size: "sm", class: "text-primary mt-0.5 flex-shrink-0" %>
|
||||
<%= icon "clock", size: "sm", class: "text-primary mt-0.5 flex-shrink-0" %>
|
||||
<div class="flex-1">
|
||||
<p class="text-sm text-primary mb-3">
|
||||
<strong>Historical Data Range:</strong>
|
||||
<p class="text-sm text-primary mb-2">
|
||||
<strong>Transaction History:</strong>
|
||||
</p>
|
||||
<p class="text-xs text-secondary">
|
||||
SimpleFin typically provides <strong>60-90 days</strong> of transaction history, depending on your bank.
|
||||
After initial setup, new transactions will sync automatically going forward.
|
||||
Historical data availability varies by institution and account type.
|
||||
</p>
|
||||
<%= form.date_field :sync_start_date,
|
||||
label: "Start syncing transactions from:",
|
||||
value: @simplefin_item.sync_start_date || 1.year.ago.to_date,
|
||||
min: 3.years.ago.to_date,
|
||||
max: Date.current,
|
||||
class: "w-full max-w-xs",
|
||||
help_text: "Select how far back you want to sync transaction history. Maximum 3 years of history available." %>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<% @simplefin_accounts.each do |simplefin_account| %>
|
||||
<div class="border border-primary rounded-lg p-4">
|
||||
<% inferred = @inferred_map[simplefin_account.id] || {} %>
|
||||
<% selected_type = inferred[:confidence] == :high ? inferred[:type] : "skip" %>
|
||||
<%# Check if this account needs user attention (type selected but subtype missing) %>
|
||||
<% types_with_subtypes = %w[Depository Investment Loan] %>
|
||||
<% needs_subtype_attention = selected_type != "skip" && types_with_subtypes.include?(selected_type) && inferred[:subtype].blank? %>
|
||||
|
||||
<div class="rounded-lg p-4 <%= needs_subtype_attention ? "border-2 border-warning bg-warning/5" : "border border-primary" %>">
|
||||
<div class="flex items-center justify-between mb-3">
|
||||
<div>
|
||||
<h3 class="font-medium text-primary">
|
||||
@@ -78,8 +82,6 @@
|
||||
<div>
|
||||
<%= label_tag "account_types[#{simplefin_account.id}]", "Account Type:",
|
||||
class: "block text-sm font-medium text-primary mb-2" %>
|
||||
<% inferred = @inferred_map[simplefin_account.id] || {} %>
|
||||
<% selected_type = inferred[:confidence] == :high ? inferred[:type] : "skip" %>
|
||||
<%= select_tag "account_types[#{simplefin_account.id}]",
|
||||
options_for_select(@account_type_options, selected_type),
|
||||
{ class: "appearance-none bg-container border border-primary rounded-md px-3 py-2 text-sm leading-6 text-primary focus:border-primary focus:ring-1 focus:ring-primary focus:outline-none w-full",
|
||||
|
||||
Reference in New Issue
Block a user