Commit Graph

14 Commits

Author SHA1 Message Date
David Gil
ba442d5f26 Implement Indexa Capital provider with real API integration (#933)
* Add Indexa Capital provider scaffold

Generate Indexa Capital provider scaffolding and align credential fields with the API authentication requirements.

* Fix PR 926 lint and schema CI failures

* Implement Indexa Capital provider with real API integration

- Rewrite all broken view templates (were meta-ERB from code generator)
- Create missing select_accounts.html.erb template
- Implement real API calls: list_accounts via /users/me, get_holdings
  via /accounts/{number}/fiscal-results, get_account_balance via
  /accounts/{number}/performance
- Add API token auth support (stored token > env token > credentials)
- Add api_token column with encryption support
- Redesign settings panel: API token prominent, credentials collapsible
- Fix account balances display using performance endpoint portfolios
- Fix accounts index empty-state guard missing indexa_capital_items
- Simplify activities fetch job (no activities API endpoint exists)
- Fix i18n interpolation (%%{ -> %{) throughout locale file

* Add tests for Indexa Capital provider integration

- IndexaCapitalItemTest: validations, credentials, scopes, sync status
- IndexaCapitalAccountTest: upsert, holdings, account provider linking
- Provider::IndexaCapitalTest: auth modes, API stubs, error handling
- IndexaCapitalItemsControllerTest: CRUD, setup, linking, authorization
- Fixtures for items (token + credentials) and accounts (mutual + pension)

52 tests, 98 assertions, 0 failures

* Address code review feedback from PR #933

- Fix zero balance bug: use `nil?` instead of `present?` so 0 is stored
- Fix has_indexa_capital_credentials? to check api_token (was ignored)
- Fix build_provider to delegate to Provided concern (was ignoring token)
- Fix IndexaCapital section outside encryption_error guard in settings
- Add account_number sanitization to prevent path traversal in API URLs
- Replace all skipped processor tests with real working tests
- Add zero-balance and path-traversal test coverage

61 tests, 107 assertions, 0 failures

* Address code review round 2: credentials validation, RuboCop, test quality

- Fix RuboCop SpaceInsideArrayLiteralBrackets in credentials check
- Chain where.not calls so all three username/document/password must be present
- Require all three credentials (||) instead of any one (&&) in validate_configuration!
- Move attr_reader to private to avoid exposing credentials publicly
- Parse dates with Date.parse in extract_balance for robustness
- Remove stale TODO and Crypto from supported_account_types
- Order build_provider query deterministically by created_at
- Replace no-op holdings assertion with meaningful assert_difference

* Address code review round 3: JSON parse safety and test precision

- Rescue JSON::ParserError on 2xx responses for clearer error messages
- Fix weak balance assertion: set balance to 0 before processing, assert
  expected value (27093.01 = sum of holdings amounts)

* Include Indexa Capital in automatic family sync

Add indexa_capital_items to Family::Syncer#child_syncables so balances
and holdings refresh on daily auto-sync and login sync, not only on
manual sync button clicks.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-02-08 18:19:37 +01:00
Juan José Mata
71f10c5e4a fix: Handle missing encryption keys gracefully on Poviders config page (#753)
* fix: Handle missing encryption keys gracefully on providers page

When Active Record encryption keys are not configured, the settings/providers
page would crash with an unhandled exception. This change catches the
ActiveRecord::Encryption::Errors::Configuration error and displays a
friendly error message instead, explaining that encryption credentials
need to be configured before using sync providers.

https://claude.ai/code/session_015nPsLWkr12i5ok5bwLtA7p

* Simplify rescue block

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-23 19:15:04 +01:00
Juan José Mata
bf76d6b88d Let /settings/providers work in demo site 2026-01-22 20:01:18 +00:00
LPW
a83f70425f Add SnapTrade brokerage integration with full trade history support (#737)
* Introduce SnapTrade integration with models, migrations, views, and activity processing logic.

* Refactor SnapTrade activities processing: improve activity fetching flow, handle pending states, and update UI elements for enhanced user feedback.

* Update Brakeman ignore file to include intentional redirect for SnapTrade OAuth portal.

* Refactor SnapTrade models, views, and processing logic: add currency extraction helper, improve pending state handling, optimize migration checks, and enhance user feedback in UI.

* Remove encryption for SnapTrade `snaptrade_user_id`, as it is an identifier, not a secret.

* Introduce `SnaptradeConnectionCleanupJob` to asynchronously handle SnapTrade connection cleanup and improve i18n for SnapTrade item status messages.

* Update SnapTrade encryption: make `snaptrade_user_secret` non-deterministic to enhance security.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-22 20:52:49 +01:00
soky srm
179552657c Mercury integration (#723)
* Initial mercury impl

* FIX both mercury and generator class

* Finish mercury integration and provider generator

* Fix schema

* Fix linter and tags

* Update routes.rb

* Avoid schema drift

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-22 20:37:07 +01:00
LPW
dd991fa339 Add Coinbase exchange integration with CDP API support (#704)
* **Add Coinbase integration with item and account management**
- Creates migrations for `coinbase_items` and `coinbase_accounts`.
- Adds models, controllers, views, and background tasks to support account linking, syncing, and transaction handling.
- Implements Coinbase API client and adapter for seamless integration.
- Supports ActiveRecord encryption for secure credential storage.
- Adds UI components for provider setup, account management, and synchronization.

* Localize Coinbase-related UI strings, refine account linking for security, and add timeouts to Coinbase API requests.

* Localize Coinbase account handling to support native currencies (USD, EUR, GBP, etc.) across balances, trades, holdings, and transactions.

* Improve Coinbase processing with timezone-safe parsing, native currency support, and immediate holdings updates.

* Improve trend percentage formatting and enhance race condition handling for Coinbase account linking.

* Fix log message wording for orphan cleanup

* Ensure `selected_accounts` parameter is sanitized by rejecting blank entries.

* Add tests for Coinbase integration: account, item, and controller coverage

- Adds unit tests for `CoinbaseAccount` and `CoinbaseItem` models.
- Adds integration tests for `CoinbaseItemsController`.
- Introduces Stimulus `select-all` controller for UI checkbox handling.
- Localizes UI strings and logging for Coinbase integration.

* Update test fixtures to use consistent placeholder API keys and secrets

* Refine `coinbase_item` tests to ensure deterministic ordering and improve scope assertions.

* Integrate `SyncStats::Collector` into Coinbase syncer to streamline statistics collection and enhance consistency.

* Localize Coinbase sync status messages and improve sync summary test coverage.

* Update `CoinbaseItem` encryption: use deterministic encryption for `api_key` and standard for `api_secret`.

* fix schema drift

* Beta labels to lower expectations

---------

Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-21 22:56:39 +01:00
Ethan
3b4ab735b0 Add (beta) CoinStats Crypto Wallet Integration with Balance and Transaction Syncing (#512)
* Feat(CoinStats): Scaffold implementation, not yet functional

* Feat(CoinStats): Implement crypto wallet balance and transactions

* Feat(CoinStats): Add tests, Minor improvements

* Feat(CoinStats): Utilize bulk fetch API endpoints

* Feat(CoinStats): Migrate strings to i8n

* Feat(CoinStats): Fix error handling in wallet link modal

* Feat(CoinStats): Implement hourly provider sync job

* Feat(CoinStats): Generate docstrings

* Fix(CoinStats): Validate API Key on provider update

* Fix(Providers): Safely handle race condition in merchance creation

* Fix(CoinStats): Don't catch system signals in account processor

* Fix(CoinStats): Preload before iterating accounts

* Fix(CoinStats): Add no opener / referrer to API dashboard link

* Fix(CoinStats): Use strict matching for symbols

* Fix(CoinStats): Remove dead code in transactions importer

* Fix(CoinStats): Avoid transaction fallback ID collisions

* Fix(CoinStats): Improve Blockchains fetch error handling

* Fix(CoinStats): Enforce NOT NULL constraint for API Key schema

* Fix(CoinStats): Migrate sync status strings to i8n

* Fix(CoinStats): Use class name rather than hardcoded string

* Fix(CoinStats): Use account currency rather than hardcoded USD

* Fix(CoinStats): Migrate from standalone to Provider class

* Fix(CoinStats): Fix test failures due to string changes
2026-01-07 15:59:04 +01:00
soky srm
4a29d030af Initial enable banking implementation (#382)
* Initial enable banking implementation

* Handle multiple connections

* Amount fixes

* Account type mapping

* Add option to skip accounts

* Update schema.rb

* Transaction fixes

* Provider fixes

* FIX account identifier

* FIX support unlinking

* UI style fixes

* FIX safe redirect and brakeman issue

* FIX

- pagination max fix
- wrap crud in transaction logic

* FIX api uid access

- The Enable Banking API expects the UUID (uid from the API response) to fetch balances/transactions, not the identification_hash

* FIX add new connection

* FIX erb code

* Alert/notice box overflow protection

* Give alert/notification boxes room to grow (3 lines max)

* Add "Enable Banking (beta)" to `/settings/bank_sync`

* Make Enable Banking section collapsible like all others

* Add callback hint to error message

---------

Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-11-29 13:31:08 +01:00
soky srm
91a91c3834 Improvements (#379)
* Improvements

- Fix button visibility in reports on light theme
- Unify logic for provider syncs
- Add default option is to skip accounts linking ( no op default )

* Stability fixes and UX improvements

* FIX add unlinking when deleting lunch flow connection as well

* Wrap updates in transaction

* Some more improvements

* FIX proper provider setup check

* Make provider section collapsible

* Fix balance calculation

* Restore focus ring

* Use browser default focus

* Fix lunch flow balance for credit cards
2025-11-25 20:21:29 +01:00
soky srm
be0b20dfd9 Lunchflow settings family (#363)
* Move provider config to family

* remove global settings

* Remove turbo auto  submit

* Fix flash location

* Fix mssing syncer for lunchflow

* Update schema.rb

* FIX tests and encryption config

* FIX make rabbit happy

* FIX run migration in SQL

* FIX turbo frame modal

* Branding fixes

* FIX rabbit

* OCD with product names

* More OCD

* No other console.log|warn in codebase

---------

Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-11-22 02:14:29 +01:00
LPW
61eb611529 Simplefin enhancements v2 (#267)
* SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint

# Conflicts:
#	db/schema.rb

# Conflicts:
#	app/controllers/simplefin_items_controller.rb

* fix testing

* fix linting

* xfix linting x2

* Review PR #267 on we-promise/sure (SimpleFin enhancements v2). Address all 15 actionable CodeRabbit comments: Add UUID validations in rakes (e.g., simplefin_unlink), swap Ruby pattern matching/loops for efficient DB queries (e.g., where LOWER(name) LIKE ?), generate docstrings for low-coverage areas (31%), consolidate routes for simplefin_items, move view logic to helpers (e.g., format_transaction_extra), strengthen tests with exact assertions/fixtures for dedup/relink failures. Also, check for overlaps with merged #262 (merchants fix): Ensure merchant creation in simplefin_entry/processor.rb aligns with new payee-based flow and MD5 IDs; add tests for edge cases like empty payees or over-merging pendings. Prioritize security (PII redaction in logs, no hardcoded secrets).

* SimpleFin: address CodeRabbit comments (batch 1)

- Consolidate simplefin_items routes under a single resources block; keep URLs stable
- Replace inline JS with Stimulus auto-relink controller; auto-load relink modal via global modal frame
- Improve a11y in relink modal by wrapping rows in labels
- Harden unlink rake: default dry_run=true, UUID validation, redact PII in outputs, clearer errors
- Backfill rake: default dry_run=true, UUID validation; groundwork for per-SFA counters
- Fix-was-merged rake: default dry_run=true, UUID validation; clearer outputs
- Idempotent transfer auto-match (find_or_create_by! + RecordNotUnique rescue)
- Extract SimpleFin error tooltip assembly into helper and use it in view

RuboCop: maintain 2-space indentation, spaces inside array brackets, spaces after commas, and no redundant returns

* Linter noise

* removed filed commited by mistake.

* manual relink flow and tighten composite matching

* enforce manual relink UI; fix adapter keywords; guarantee extra.simplefin hash

* refactor(simplefin): extract relink service; enforce manual relink UI; tighten composite match; migration 7.2

* add provider date parser; refactor rake; move view queries; partial resilience

* run balances-only import in background job. make update flow enqueue balances-only job

* persists across all update redirects and initialize
used_manual_ids to prevent NameError in relink candidate computation.

* SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint

* Fixed failed test after rebase.

* scan_ruby fix

* Calming the rabbit:
Fix AccountProvider linking when accounts change
Drop the legacy unique index instead of duplicating it
Fix dynamic constant assignment
Use fixtures consistently; avoid rescue for control flow.
Replace bare rescue with explicit exception class.
Move business logic out of the view.
Critical: Transaction boundary excludes recompute phase, risking data loss.
Inconsistency between documentation and implementation for zero-error case.
Refactor to use the compute_unlinked_count helper for consistency.
Fix cleanup task default: it deletes by default.
Move sync stats computation to controller to avoid N+1 queries.
Consolidate duplicate sync query.
Clarify the intent of setting flash notice on the error path.
Fix Date/Time comparison in should_be_inactive?.
Move stats retrieval logic to controller.
Remove duplicate Sync summary section.
Remove the unnecessary sleep statement; use Capybara's built-in waiting.
Add label wrappers for accessibility and consistency.

* FIX SimpleFIN new account modal

Now new account properly loads as a Modal, instead of new page.
Fixes also form showing dashboard instead of settings page.

* Remove SimpleFin legacy UI components, migrate schema, and refine linking behavior.

# Conflicts:
#	app/helpers/settings_helper.rb

* Extract SimpleFin-related logic to `prepare_show_context` helper and refactor for consistency. Adjust conditional checks and ensure controller variables are properly initialized.

* Remove unused SimpleFin maps from prepare_show_context; select IDs to avoid N+1
Replace Tailwind bg-green-500 with semantic bg-success in _simplefin_panel/_provider_form
Add f.label :setup_token in simplefin_items/new for a11y
Remove duplicate require in AccountsControllerSimplefinCtaTest

* Remove unnecessary blank lines

* Reduce unnecessary changes

This reduces the diff against main

* Simplefin Account Setup: Display in modal

This fixes an issue with the `X` dismiss button in the top right corner

* Removed unnecessary comment.

* removed unnecessary function.

* fixed broken links

* Removed unnecessary file

* changed to database query

* set to use UTC and gaurd against null

* set dry_run=true

* Fixed comment

* Changed to use a database-level query

* matched test name to test behavior.

* Eliminate code duplication and Time.zone dependency

* make final summary surface failures

* lint fix

* Revised timezone comment. better handle missing selectors.

* sanitized LIKE wildcards

* Fixed SimpleFin import to avoid “Currency can’t be blank” validation failures when providers return an empty currency string.

* Added helper methods for admin and self-hosted checks

* Specify exception types in rescue clauses.

* Refined logic to determine transaction dates for credit accounts.

* Refined stats calculation for `total_accounts` to track the maximum unique accounts per run instead of accumulating totals.

* Moved `unlink_all!` logic to `SimplefinItem::Unlinking` concern and deprecated `SimplefinItem::Unlinker`. Updated related references.

* Refined legacy unlinking logic, improved `current_holdings` formatting, and added ENV-based overrides for self-hosted checks.

* Enhanced `unlink_all!` with explicit error handling, improved transaction safety, and refined ENV-based self-hosted checks. Adjusted exception types and cleaned up private method handling.

* Improved currency assignment logic by adding fallback to `current_account` and `family` currencies.

* Enhanced error tracking during SimpleFin account imports by adding categorized error buckets, limiting stored errors to the last 5, and improving `stats` calculations.

* typo fix

* Didn't realize rabbit was still mad...
Refactored SimpleFin error handling and CTA logic: centralized duplicate detection and relink visibility into controller, improved task counters, adjusted redirect notices, and fixed form indexing.

* Dang rabbit never stops... Centralized SimpleFin maps logic into `MapsHelper` concern and integrated it into relevant controllers and rake tasks. Optimized queries, reduced redundancy, and improved unlinked counts and manual account checks with batch processing. Adjusted task arguments for clarity.

* Persistent rabbit. Optimized SimpleFin maps logic by implementing batch queries for manual account and unlinked count checks, reducing N+1 issues. Improved clarity of rake task argument descriptions and error messages for better usability.

* Lost a commit somehow, resolved here. Refactored transaction extra details logic by introducing `build_transaction_extra_details` helper to improve clarity, reusability, and reduce view complexity. Enhanced rake tasks with strict dry-run validation and better error handling. Updated schema to allow nullable `merchant_id` and added conditional unique indexes for recurring transactions.

* Refactored sensitive data redaction in `simplefin_unlink` task for recursive handling, optimized SQL sanitization in `simplefin_holdings_backfill`, improved error handling in `transactions_helper`, and streamlined day change calculation logic in `Holding` model.

* Lint fix

* Removed per PR comments.

* Also removing per PR comment.

* git commit -m "SimpleFIN polish: preserve #manual-accounts wrapper, unify \"manual\" scope, and correct unlinked counts
- Preserve #manual-accounts wrapper: switch non-empty updates to turbo_stream.update and background broadcast_update_to; keep empty-path replace to render <div id=\"manual-accounts\"></div>
- Unify definition of manual accounts via Account.visible_manual (visible + legacy-nil + no AccountProvider); reuse in controllers, jobs, and helper
- Correct setup/unlinked counts: SimplefinItem::Syncer#finalize_setup_counts and maps now consider AccountProvider links (legacy account AND provider must be absent)
Deleted:
- app/models/simplefin_item/relink_service.rb
- app/controllers/concerns/simplefin_items/relink_helpers.rb
- app/javascript/controllers/auto_relink_controller.js
- app/views/simplefin_items/_relink_modal.html.erb
- app/views/simplefin_items/manual_relink.html.erb
- app/views/simplefin_items/relink.html.erb
- test/services/simplefin_item/relink_service_test.rb
Refs: PR #318 unified link/unlink; PR #267 SimpleFIN; follow-up to fix wrapper ID loss and counting drift."

* Extend unlinked account check to include "Investment" type

* set SimpleFIN item for `balances`, remove redundant unpacking, and improve holdings task error

* SimpleFIN: add `errors` action + modal; do not reintroduce legacy relink actions; removed dead helper

* FIX simpleFIN linking

* Add delay back, tests benefit from it

* Put cache back in

* Remove empty `rake` task

* Small spelling fixes.

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: sokie <sokysrm@gmail.com>
Co-authored-by: Dylan Corrales <deathcamel58@gmail.com>
2025-11-17 21:51:37 +01:00
soky srm
fad241c416 Fixes & Improvements (#316)
* Some improvements

- Fix issue with lunch flow accounts that were imported
- Remove the period comparison section from reports

* Add cleanup migration

* FIX for dynamic config

* Fix linter

* FIX settings setter

Reuse the base class’ atomic setter to leverage its locking and cache invalidation.

* Make upsert atomic

* Remove migration file

Signed-off-by: soky srm <sokysrm@gmail.com>

* Delete db/migrate/20251111094448_migrate_dynamic_fields_to_individual_entries.rb

Signed-off-by: soky srm <sokysrm@gmail.com>

* Fix cache reset

* Revert "Remove migration file"

This reverts commit 1f2a21ef58.

* Revert "Delete db/migrate/20251111094448_migrate_dynamic_fields_to_individual_entries.rb"

This reverts commit 29dcaaafb2.

* Fix Plaid initialiser

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
2025-11-11 19:51:07 +01:00
soky srm
164fe5375d FIX Read-Modify-Write issue with dynamic fields (#290)
* FIX Read-Modify-Write issue with dynamic fields

Ruby caching + queueing updates might cause some dynamic fields to not be updated.

* Small fix for true dynamic fields

* Add suite of tests for new settings page

* Treat nil values as deletions to keep the hash clean

* Test fix
2025-11-05 12:19:33 +01:00
soky srm
96713ee8b4 Add support for dynamic config UI (#256)
* Add support for dynamic config UI

* Add support for section description

* Better dynamic class settings

Added dynamic_fields hash field - Stores all undeclared settings
[] method - Checks declared fields first, then falls back to dynamic hash
[]= method - Updates declared fields normally, stores others in hash
No runtime field declaration - Fields are never dynamically created on the class

* FIX proper lookup for provider keys

- Also validate configurable values properly.
- Change Provider factory to use Rails autoloading (Zeitwerk)

* Fix factory

The derive_adapter_name method relies on string manipulation ("PlaidAccount".sub(/Account$/, "") + "Adapter" → "PlaidAdapter"), but we already have explicit registration in place.

* Make updates atomic, field-aware, and handle blanks explicitly

* Small UX detail

* Add support for PlaidEU in UI also

- This looks like partial support atm
2025-10-29 13:11:04 +01:00