Files
sure/db/migrate/20260725000000_create_redbark_items_and_accounts.rb
Oscar c6a240a183 feat(redbark): add australian bank sync (redbark) (#2794)
* add redbark provider integration

- per family api key provider, built like the lunchflow integration
- syncs accounts, balances and transactions from api.redbark.com
- account setup flow, settings panel, locales and routes
- tests and fixtures

* harden redbark integration based on prior provider pr feedback

- use DebugLogEntry.capture for sync/import/unlink failures
- retry 429s and 5xxs with backoff, raise on page cap instead of truncating
- keep raw response bodies out of logs and errors
- not null constraints on account columns, migration base 7.2
- persist ignored flag for skipped accounts so they stop nagging setup
- validate api key on every save, re-arm status on key rotation
- destroy aborts if unlink fails, atomic account create and link
- require_admin on mutating actions, see_other on error redirects
- single grouped query for item account counts
- i18n default connection name, blank password field value
- controller and provider tests

* fix issues found in second review sweep

- add missing syncable scope, without it every family sync raises
- kick off a sync on connection create and on key rotation
- setup dialog fetches accounts inline for fresh connections and shows api errors
- skip balance write when no balance has been fetched yet, never anchor a false zero
- exclude stale and non banking accounts from the batched balances call, per account fallback if the batch is rejected
- detect the server row ceiling and empty pages instead of silently truncating history
- user sync start date only governs the initial backfill, incremental after that
- fetch connections before the per account loop so auth errors propagate once
- drop untemplated index/show/new/edit routes and dead preload/link_accounts actions
- stable dom id on the settings panel so repeat turbo replaces keep working

* skip brokerage connections, found in live testing

- the transactions endpoint 400s for brokerage connections, they belong to /v1/trades
- only import accounts from banking and documents connections
- guard transaction fetches for any legacy linked non banking account

* address review feedback

- treat the truncation header as a pagination signal: split the date window and refetch instead of failing the account
- prune stale pending rows from the snapshot so settled pendings cant come back as duplicates
- block linking a sure account that already has another provider feed
- count setup failures separately from skips and surface an error instead of "all skipped"
- add not nulls on redbark_items name and api key
- enqueue the destroy job after the flag commits, not inside the transaction
- swap bg-gray-400 for bg-surface-inset, drop amounts from info logs, remove i18n default fallbacks
- tests for window splitting, pending pruning and encrypted payload round trip

* fix issues from convention review

- benign skips (unlinked account, blank id, unparseable rows) no longer count as failures, tracked separately so a clean batch reports success
- currency parsing goes through extract_currency so hash shaped payloads resolve instead of falling to the default
- merchant ids use truncated sha256 instead of md5
- debug log entries for import failures and account sync scheduling failures

* bound the raw transactions snapshot to the fetch window

- trim raw_transactions_payload to the current fetch window on merge, same as brex
- keep rows without a parseable date, drop settled pendings as before
- surface skipped rows in the aggregate debug log entry with imported/skipped counts
2026-07-26 07:40:25 +02:00

71 lines
2.1 KiB
Ruby

# frozen_string_literal: true
class CreateRedbarkItemsAndAccounts < ActiveRecord::Migration[7.2]
def change
# Create provider items table (stores per-family connection credentials)
create_table :redbark_items, id: :uuid do |t|
t.references :family, null: false, foreign_key: true, type: :uuid
t.string :name, null: false
# Institution metadata
t.string :institution_id
t.string :institution_name
t.string :institution_domain
t.string :institution_url
t.string :institution_color
# Status and lifecycle
t.string :status, default: "good", null: false
t.boolean :scheduled_for_deletion, default: false, null: false
t.boolean :pending_account_setup, default: false, null: false
# Sync settings
t.datetime :sync_start_date
# Raw data storage
t.jsonb :raw_payload
t.jsonb :raw_institution_payload
# Provider-specific credential fields
t.text :api_key, null: false
t.timestamps
end
add_index :redbark_items, :status
# Create provider accounts table (stores individual account data from provider)
create_table :redbark_accounts, id: :uuid do |t|
t.references :redbark_item, null: false, foreign_key: true, type: :uuid
# Account identification
t.string :name, null: false
t.string :redbark_account_id, null: false
t.string :connection_id
t.string :account_number
# Account details
t.string :currency, null: false
t.decimal :current_balance, precision: 19, scale: 4
t.string :account_status
t.string :account_type
t.string :provider
# Setup state - accounts the user chose to skip stay hidden from setup prompts
t.boolean :ignored, default: false, null: false
# Metadata and raw data
t.jsonb :institution_metadata
t.jsonb :raw_payload
t.jsonb :raw_transactions_payload
# Sync settings
t.date :sync_start_date
t.timestamps
end
add_index :redbark_accounts, [ :redbark_item_id, :redbark_account_id ], unique: true, name: "index_redbark_accounts_on_item_and_account_id"
end
end