mirror of
https://github.com/we-promise/sure.git
synced 2026-04-20 12:34:12 +00:00
Harden SimpleFin sync: retries, safer imports, manual relinking, and data-quality reconciliation (#544)
* Add tests and enhance logic for SimpleFin account synchronization and reconciliation - Added retry logic with exponential backoff for network errors in `Provider::Simplefin`. - Introduced tests to verify retry functionality and error handling for rate-limit, server errors, and stale data. - Updated `SimplefinItem` to detect stale sync status and reconciliation issues. - Enhanced UI to display stale sync warnings and data integrity notices. - Improved SimpleFin account matching during updates with multi-tier strategy (ID, fingerprint, fuzzy match). - Added transaction reconciliation logic to detect data gaps, transaction count drops, and duplicate transaction IDs. * Introduce `SimplefinConnectionUpdateJob` for asynchronous SimpleFin connection updates - Moved SimpleFin connection update logic to `SimplefinConnectionUpdateJob` to improve response times by offloading network retries, data fetching, and reconciliation tasks. - Enhanced SimpleFin account matching with a multi-tier strategy (ID, fingerprint, fuzzy name match). - Added retry logic and bounded latency for token claim requests in `Provider::Simplefin`. - Updated tests to cover the new job flow and ensure correct account reconciliation during updates. * Remove unused SimpleFin account matching logic and improve error handling in `SimplefinConnectionUpdateJob` - Deleted the multi-tier account matching logic from `SimplefinItemsController` as it is no longer used. - Enhanced error handling in `SimplefinConnectionUpdateJob` to gracefully handle import failures, ensuring orphaned items can be manually resolved. - Updated job flow to conditionally set item status based on the success of import operations. * Fix SimpleFin sync: check both legacy FK and AccountProvider for linked accounts * Add crypto, checking, savings, and cash account detection; refine subtype selection and linking - Enhanced `Simplefin::AccountTypeMapper` to include detection for crypto, checking, savings, and standalone cash accounts. - Improved subtype selection UI with validation and warning indicators for missing selections. - Updated SimpleFin account linking to handle both legacy FK and `AccountProvider` associations consistently. - Refined job flow and importer logic for better handling of linked accounts and subtype inference. * Improve `SimplefinConnectionUpdateJob` and holdings processing logic - Fixed race condition in `SimplefinConnectionUpdateJob` by moving `destroy_later` calls outside of transactions. - Updated fuzzy name match logic to use Levenshtein distance for better accuracy. - Enhanced synthetic ticker generation in holdings processor with hash suffix for uniqueness. * Refine SimpleFin entry processing logic and ensure `extra` data persistence - Simplified pending flag determination to rely solely on provider-supplied values. - Fixed potential stale values in `extra` by ensuring deep merge overwrite with `entry.transaction.save!`. * Replace hardcoded fallback transaction description with localized string * Refine pending flag logic in SimpleFin processor tests - Adjust test to prevent falsely inferring pending status from missing posted dates. - Ensure provider explicitly sets pending flag for transactions. * Add `has_many :holdings` association to `AccountProvider` with `dependent: :nullify` --------- Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
This commit is contained in:
@@ -56,7 +56,10 @@ class SimplefinItem < ApplicationRecord
|
||||
end
|
||||
|
||||
def process_accounts
|
||||
simplefin_accounts.joins(:account).each do |simplefin_account|
|
||||
# Process accounts linked via BOTH legacy FK and AccountProvider
|
||||
simplefin_accounts.includes(:account, account_provider: :account).each do |simplefin_account|
|
||||
# Only process if there's a linked account (via either system)
|
||||
next unless simplefin_account.current_account.present?
|
||||
SimplefinAccount::Processor.new(simplefin_account).process
|
||||
end
|
||||
end
|
||||
@@ -192,6 +195,58 @@ class SimplefinItem < ApplicationRecord
|
||||
end
|
||||
end
|
||||
|
||||
# Detect if sync data appears stale (no new transactions for extended period)
|
||||
# Returns a hash with :stale (boolean) and :message (string) if stale
|
||||
def stale_sync_status
|
||||
return { stale: false } unless last_synced_at.present?
|
||||
|
||||
# Check if last sync was more than 3 days ago
|
||||
days_since_sync = (Date.current - last_synced_at.to_date).to_i
|
||||
if days_since_sync > 3
|
||||
return {
|
||||
stale: true,
|
||||
days_since_sync: days_since_sync,
|
||||
message: "Last successful sync was #{days_since_sync} days ago. Your SimpleFin connection may need attention."
|
||||
}
|
||||
end
|
||||
|
||||
# Check if linked accounts have recent transactions
|
||||
linked_accounts = accounts
|
||||
return { stale: false } if linked_accounts.empty?
|
||||
|
||||
# Find the most recent transaction date across all linked accounts
|
||||
latest_transaction_date = Entry.where(account_id: linked_accounts.map(&:id))
|
||||
.where(entryable_type: "Transaction")
|
||||
.maximum(:date)
|
||||
|
||||
if latest_transaction_date.present?
|
||||
days_since_transaction = (Date.current - latest_transaction_date).to_i
|
||||
if days_since_transaction > 14
|
||||
return {
|
||||
stale: true,
|
||||
days_since_transaction: days_since_transaction,
|
||||
message: "No new transactions in #{days_since_transaction} days. Check your SimpleFin dashboard to ensure your bank connections are active."
|
||||
}
|
||||
end
|
||||
end
|
||||
|
||||
{ stale: false }
|
||||
end
|
||||
|
||||
# Check if the SimpleFin connection needs user attention
|
||||
def needs_attention?
|
||||
requires_update? || stale_sync_status[:stale] || pending_account_setup?
|
||||
end
|
||||
|
||||
# Get a summary of issues requiring attention
|
||||
def attention_summary
|
||||
issues = []
|
||||
issues << "Connection needs update" if requires_update?
|
||||
issues << stale_sync_status[:message] if stale_sync_status[:stale]
|
||||
issues << "Accounts need setup" if pending_account_setup?
|
||||
issues
|
||||
end
|
||||
|
||||
private
|
||||
def remove_simplefin_item
|
||||
# SimpleFin doesn't require server-side cleanup like Plaid
|
||||
|
||||
Reference in New Issue
Block a user