Providers factory (#250)

* Implement providers factory

* Multiple providers sync support

- Proper Multi-Provider Syncing: When you click sync on an account with multiple providers (e.g., both Plaid and SimpleFin), all provider items are synced
- Better API: The existing account.providers method already returns all providers, and account.provider returns the first one for backward compatibility
- Correct Holdings Deletion Logic: Holdings can only be deleted if ALL providers allow it, preventing accidental deletions that would be recreated on next sync
TODO: validate this is the way we want to go? We would need to check holdings belong to which account, and then check provider allows deletion. More complex
- Database Constraints: The existing validations ensure an account can have at most one provider of each type (one PlaidAccount, one SimplefinAccount, etc.)

* Add generic provider_import_adapter

* Finish unified import strategy

* Update app/models/plaid_account.rb

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: soky srm <sokysrm@gmail.com>

* Update app/models/provider/factory.rb

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: soky srm <sokysrm@gmail.com>

* Fix account linked by plaid_id instead of external_id

* Parse numerics to BigDecimal

Parse numerics to BigDecimal before computing amount; guard nils.

Avoid String * String and float drift; also normalize date.

* Fix incorrect usage of assert_raises.

* Fix linter

* Fix processor test.

* Update current_balance_manager.rb

* Test fixes

* Fix plaid linked account test

* Add support for holding per account_provider

* Fix proper account access

Also fix account deletion for simpefin too

* FIX match tests for consistency

* Some more factory updates

* Fix account schema for multipe providers

  Can do:
  - Account #1 → PlaidAccount + SimplefinAccount (multiple different providers)
  - Account #2 → PlaidAccount only
  - Account #3 → SimplefinAccount only

  Cannot do:
  - Account #1 → PlaidAccount + PlaidAccount (duplicate provider type)
  - PlaidAccount #123 → Account #1 + Account #2 (provider linked to multiple accounts)

* Fix account setup

- An account CAN have multiple providers (the schema shows account_providers with unique index on [account_id, provider_type])
  - Each provider should maintain its own separate entries
  - We should NOT update one provider's entry when another provider syncs

* Fix linter and guard migration

* FIX linter issues.

* Fixes

- Remove duplicated index
- Pass account_provider_id
- Guard holdings call to avoid NoMethodError

* Update schema and provider import fix

* Plaid doesn't allow holdings deletion

* Use ClimateControl for proper env setup

* No need for this in .git

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
This commit is contained in:
soky srm
2025-10-28 19:32:27 +01:00
committed by GitHub
parent 72e7d7736b
commit 4fb0a3856e
67 changed files with 2338 additions and 315 deletions

View File

@@ -0,0 +1,15 @@
class CreateAccountProviders < ActiveRecord::Migration[7.2]
def change
create_table :account_providers, id: :uuid do |t|
t.references :account, type: :uuid, null: false, foreign_key: true, index: true
t.references :provider, type: :uuid, null: false, polymorphic: true, index: true
t.timestamps
end
# Ensure an account can only have one provider of each type
add_index :account_providers, [ :account_id, :provider_type ], unique: true
# Ensure a provider can only be linked to one account
add_index :account_providers, [ :provider_type, :provider_id ], unique: true
end
end

View File

@@ -0,0 +1,36 @@
class MigrateAccountProvidersData < ActiveRecord::Migration[7.2]
def up
# Migrate Plaid accounts
execute <<-SQL
INSERT INTO account_providers (id, account_id, provider_type, provider_id, created_at, updated_at)
SELECT
gen_random_uuid(),
accounts.id,
'PlaidAccount',
accounts.plaid_account_id,
NOW(),
NOW()
FROM accounts
WHERE accounts.plaid_account_id IS NOT NULL
SQL
# Migrate SimpleFin accounts
execute <<-SQL
INSERT INTO account_providers (id, account_id, provider_type, provider_id, created_at, updated_at)
SELECT
gen_random_uuid(),
accounts.id,
'SimplefinAccount',
accounts.simplefin_account_id,
NOW(),
NOW()
FROM accounts
WHERE accounts.simplefin_account_id IS NOT NULL
SQL
end
def down
# Delete all account provider records
execute "DELETE FROM account_providers"
end
end

View File

@@ -0,0 +1,9 @@
class AddExternalIdAndSourceToEntries < ActiveRecord::Migration[7.2]
def change
add_column :entries, :external_id, :string
add_column :entries, :source, :string
# Add unique index on external_id + source combination to prevent duplicates
add_index :entries, [ :external_id, :source ], unique: true, where: "external_id IS NOT NULL AND source IS NOT NULL"
end
end

View File

@@ -0,0 +1,9 @@
class AddProviderMerchantIdIndexToMerchants < ActiveRecord::Migration[7.2]
def change
# Add unique index on provider_merchant_id + source for ProviderMerchant
add_index :merchants, [ :provider_merchant_id, :source ],
unique: true,
where: "provider_merchant_id IS NOT NULL AND type = 'ProviderMerchant'",
name: "index_merchants_on_provider_merchant_id_and_source"
end
end

View File

@@ -0,0 +1,5 @@
class AddAccountProviderToHoldings < ActiveRecord::Migration[7.2]
def change
add_reference :holdings, :account_provider, null: true, foreign_key: true, type: :uuid, index: true
end
end

View File

@@ -0,0 +1,18 @@
class FixAccountProvidersIndexes < ActiveRecord::Migration[7.2]
def change
# Remove the overly restrictive unique index on account_id alone
# This was preventing an account from having multiple providers
remove_index :account_providers, name: "index_account_providers_on_account_id"
# Add proper composite unique index to match model validation
# This allows an account to have multiple providers, but only one of each type
# e.g., Account can have PlaidAccount + SimplefinAccount, but not two PlaidAccounts
add_index :account_providers, [ :account_id, :provider_type ],
unique: true,
name: "index_account_providers_on_account_and_provider_type"
# Remove redundant non-unique index (cleanup)
# Line 30 already has a unique index on the same columns
remove_index :account_providers, name: "index_account_providers_on_provider"
end
end

View File

@@ -0,0 +1,26 @@
class BackfillEntriesExternalIdFromPlaidId < ActiveRecord::Migration[7.2]
def up
# Backfill external_id from plaid_id for entries that have plaid_id but no external_id
# Set source to 'plaid' for these entries as well
execute <<-SQL
UPDATE entries e
SET external_id = e.plaid_id,
source = COALESCE(e.source, 'plaid')
WHERE e.plaid_id IS NOT NULL
AND e.external_id IS NULL
AND (e.source IS NULL OR e.source = 'plaid')
SQL
end
def down
# Reverse the migration by clearing external_id and source for entries where they match plaid_id
execute <<-SQL
UPDATE entries
SET external_id = NULL,
source = NULL
WHERE plaid_id IS NOT NULL
AND external_id = plaid_id
AND source = 'plaid'
SQL
end
end

View File

@@ -0,0 +1,25 @@
class UpdateEntriesExternalIdIndex < ActiveRecord::Migration[7.2]
def up
# Remove the old index on [external_id, source]
remove_index :entries, name: "index_entries_on_external_id_and_source", if_exists: true
# Add new index on [account_id, source, external_id] with WHERE clause
# This ensures external_id is unique per (account, source) combination
# Allows same account to have multiple providers with separate entries
add_index :entries, [ :account_id, :source, :external_id ],
unique: true,
where: "(external_id IS NOT NULL) AND (source IS NOT NULL)",
name: "index_entries_on_account_source_and_external_id"
end
def down
# Remove the new index
remove_index :entries, name: "index_entries_on_account_source_and_external_id", if_exists: true
# Restore the old index
add_index :entries, [ :external_id, :source ],
unique: true,
where: "(external_id IS NOT NULL) AND (source IS NOT NULL)",
name: "index_entries_on_external_id_and_source"
end
end

View File

@@ -0,0 +1,5 @@
class RemoveDuplicateHoldingsIndex < ActiveRecord::Migration[7.2]
def change
remove_index :holdings, name: "index_holdings_on_account_and_external_id"
end
end