* Introduce SnapTrade integration with models, migrations, views, and activity processing logic.
* Refactor SnapTrade activities processing: improve activity fetching flow, handle pending states, and update UI elements for enhanced user feedback.
* Update Brakeman ignore file to include intentional redirect for SnapTrade OAuth portal.
* Refactor SnapTrade models, views, and processing logic: add currency extraction helper, improve pending state handling, optimize migration checks, and enhance user feedback in UI.
* Remove encryption for SnapTrade `snaptrade_user_id`, as it is an identifier, not a secret.
* Introduce `SnaptradeConnectionCleanupJob` to asynchronously handle SnapTrade connection cleanup and improve i18n for SnapTrade item status messages.
* Update SnapTrade encryption: make `snaptrade_user_secret` non-deterministic to enhance security.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Implement entry protection flags for sync overwrites
- Added `user_modified` and `import_locked` flags to `entries` table to prevent provider sync from overwriting user-edited and imported data.
- Introduced backfill migration to mark existing entries based on conditions.
- Enhanced sync and processing logic to respect protection flags, track skipped entries, and log detailed stats.
- Updated UI to display skipped/protected entries and reasons in sync summaries.
* Localize error details summary text and adjust `sync_account_later` method placement
* Restored schema.rb
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* Add pending transaction handling and duplicate reconciliation logic
- Implemented logic to exclude pending transactions from budgets and analytics calculations.
- Introduced mechanisms for reconciling pending transactions with posted versions.
- Added duplicate detection with support for merging or dismissing matches.
- Updated transaction search filters to include a `status_filter` for pending/confirmed transactions.
- Introduced UI elements for reviewing and resolving duplicates.
- Enhanced `ProviderSyncSummary` with stats for reconciled and stale pending transactions.
* Refactor translation handling and enhance transaction and sync logic
- Moved hardcoded strings to locale files for improved translation support.
- Refined styling for duplicate transaction indicators and sync summaries.
- Improved logic for excluding stale pending transactions and updating timestamps on batch exclusion.
- Added unique IDs to status filters for better element targeting in UI.
- Optimized database queries to avoid N+1 issues in stale pending calculations.
* Add sync settings and enhance pending transaction handling
- Introduced a new "Sync Settings" section in hosting settings with UI to toggle inclusion of pending transactions.
- Updated handling of pending transactions with improved inference logic for `posted=0` and `transacted_at` in processors.
- Added priority order for pending transaction inclusion: explicit argument > environment variable > runtime configurable setting.
- Refactored settings and controllers to store updated sync preferences.
* Refactor sync settings and pending transaction reconciliation
- Extracted logic for pending transaction reconciliation, stale exclusion, and unmatched tracking into dedicated methods for better maintainability.
- Updated sync settings to infer defaults from multiple provider environment variables (`SIMPLEFIN_INCLUDE_PENDING`, `PLAID_INCLUDE_PENDING`).
- Refined UI and messaging to handle multi-provider configurations in sync settings.
# Conflicts:
# app/models/simplefin_item/importer.rb
* Debounce transaction reconciliation during imports
- Added per-run reconciliation debouncing to prevent repeated scans for the same account during chunked history imports.
- Trimmed size of reconciliation stats to retain recent details only.
- Introduced error tracking for reconciliation steps to improve UI visibility of issues.
* Apply ABS() in pending transaction queries and improve error handling
- Updated pending transaction logic to use ABS() for consistent handling of negative amounts.
- Adjusted amount bounds calculations to ensure accuracy for both positive and negative values.
- Refined exception handling in `merge_duplicate` to log failures and update user alert.
- Replaced `Date.today` with `Date.current` in tests to ensure timezone consistency.
- Minor optimization to avoid COUNT queries by loading limited records directly.
* Improve error handling in duplicate suggestion and dismissal logic
- Added exception handling for `store_duplicate_suggestion` to log failures and prevent crashes during fuzzy/low-confidence matches.
- Enhanced `dismiss_duplicate` action to handle `ActiveRecord::RecordInvalid` and display appropriate user alerts.
---------
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
* Add shared sync statistics collection and provider sync summary UI
- Introduced `SyncStats::Collector` concern to centralize sync statistics logic, including account, transaction, holdings, and health stats collection.
- Added collapsible `ProviderSyncSummary` component for displaying sync summaries across providers.
- Updated syncers (e.g., `LunchflowItem::Syncer`) to use the shared collector methods for consistent stats calculation.
- Added rake tasks under `dev:sync_stats` for testing and development purposes, including fake stats generation with optional issues.
- Enhanced provider-specific views to include sync summaries using the new shared component.
* Refactor `ProviderSyncSummary` to improve maintainability
- Extracted `severity_color_class` to simplify severity-to-CSS mapping.
- Replaced `holdings_label` with `holdings_label_key` for streamlined localization.
- Updated locale file to separate `found` and `processed` translations for clarity.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
