Files
sure/app/models/provider_connection_status.rb
Jake dc2a565b6a feat(up): add Up Bank (AU) provider integration (#2391)
* feat(up): add Up Bank (AU) provider integration

Adds Up Bank as a per-family, token-based bank sync provider, modelled on
the existing Akahu integration. Up uses a JSON:API REST API with a personal
access token (Bearer), cursor pagination via links.next, and returns both
HELD (pending) and SETTLED transactions from one endpoint.

New:
- Provider::Up client (JSON:API unwrap, links.next pagination, retries,
  typed errors, /util/ping) + Provider::UpAdapter (Factory-registered,
  Depository + Loan).
- UpItem / UpAccount models with Provided, Unlinking, Syncer,
  SyncCompleteEvent, Importer, Processor, Transactions::Processor, and
  UpEntry::Processor (amount sign flip, HELD->pending, foreignAmount FX,
  merchant from description, stale-pending pruning).
- Family::UpConnectable, UpItemsController, routes, settings panel + connect
  flow views, accounts index wiring, initializer, en locale, and model tests.

Core wiring:
- "up" added to Transaction::PENDING_PROVIDERS, the three pending-match SQL
  blocks in Account::ProviderImportAdapter, Provider::Metadata::REGISTRY,
  ProviderMerchant/DataEnrichment source enums, ProviderConnectionStatus,
  settings provider panels, and financial data reset.

Migration create_up_items_and_accounts must be run before use. No external
API endpoints added (no OpenAPI changes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(up): dump up tables to schema and make since filter TZ-safe

The feature commit added the up_items/up_accounts migration but never
re-dumped db/schema.rb, leaving the schema version and tables stale.
Add the two table definitions and foreign keys and bump the schema
version so a fresh DB load matches the migration.

Also format a bare Date `since` as UTC midnight instead of the server's
local zone, so `filter[since]` is deterministic regardless of where the
app runs (previously shifted by the local UTC offset).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(up): address code review feedback

Behavior/correctness:
- Persist skipped accounts via a new up_accounts.ignored flag and a
  needs_setup scope, so skipped accounts stop resurfacing as "needs
  setup" on every sync. Linking clears the flag.
- destroy now checks unlink_all! per-account results and aborts deletion
  (alert) if any unlink failed, instead of swallowing failures.
- render_provider_panel_error redirect uses :see_other (was an invalid
  4xx redirect status).
- Up provider adapter falls back to item institution name/url when
  institution_metadata is absent (early return previously blocked it).

Resilience/security:
- fetch_all_resources guards against an API repeating the same
  links.next cursor (Set#add?), preventing infinite pagination.
- HTTP client validates absolute URLs (from links.next) against Up's
  HTTPS host before sending the bearer token, preventing credential
  leakage to untrusted hosts.

Diagnostics:
- Route provider sync/import failures through DebugLogEntry.capture
  (controller, UpItem, syncer, unlinking) with family/account context.
  Low-level HTTP client and currency-normalization warnings keep
  Rails.logger to match existing provider conventions.

Data integrity:
- up_accounts.name and currency are NOT NULL (align with model presence
  validations); account_id stays nullable (allow_nil uniqueness).

Forms:
- select_existing_account radio is required; controller guards a blank/
  unknown up_account_id with a friendly alert instead of RecordNotFound.

Tests:
- Add UpAccount needs_setup scope test, pagination loop guard test,
  untrusted-host rejection test; tighten filter[since] assertion to the
  exact UTC timestamp.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(up): address second-round review feedback

- Capture sync/import failures via DebugLogEntry so swallowed errors in
  account/transaction fetching and transaction processing surface in
  /settings/debug instead of only Rails.logger.
- Gate UP_DEBUG_RAW raw payload dump to local envs to avoid leaking PII
  (merchant names, amounts, account IDs) in managed/production logs.
- Collapse linked/unlinked/total account counts into one memoized query
  instead of 3 separate COUNTs per rendered item.
- Rename "Set Up Up Accounts" locale title to "Link Up Accounts".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(up): add method docstrings and align failed_result keys

Add docstrings to all Up provider source files (controller, models,
providers, concerns) to satisfy the 80% docstring coverage threshold.

Third-round review: failed_result now mirrors import's result shape
(accounts_updated/created/failed, transactions_imported/failed) instead
of the stale accounts_imported key, so failure results stay consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 20:33:09 +02:00

254 lines
8.6 KiB
Ruby

# frozen_string_literal: true
class ProviderConnectionStatus
PROVIDERS = [
{ key: "akahu", type: "AkahuItem", association: :akahu_items, accounts: :akahu_accounts },
{ key: "up", type: "UpItem", association: :up_items, accounts: :up_accounts },
{ key: "plaid", type: "PlaidItem", association: :plaid_items, accounts: :plaid_accounts },
{ key: "simplefin", type: "SimplefinItem", association: :simplefin_items, accounts: :simplefin_accounts },
{ key: "lunchflow", type: "LunchflowItem", association: :lunchflow_items, accounts: :lunchflow_accounts },
{ key: "enable_banking", type: "EnableBankingItem", association: :enable_banking_items, accounts: :enable_banking_accounts },
{ key: "coinbase", type: "CoinbaseItem", association: :coinbase_items, accounts: :coinbase_accounts },
{ key: "binance", type: "BinanceItem", association: :binance_items, accounts: :binance_accounts },
{ key: "kraken", type: "KrakenItem", association: :kraken_items, accounts: :kraken_accounts },
{ key: "coinstats", type: "CoinstatsItem", association: :coinstats_items, accounts: :coinstats_accounts },
{ key: "snaptrade", type: "SnaptradeItem", association: :snaptrade_items, accounts: :snaptrade_accounts, linked_accounts: :linked_accounts },
{ key: "ibkr", type: "IbkrItem", association: :ibkr_items, accounts: :ibkr_accounts },
{ key: "mercury", type: "MercuryItem", association: :mercury_items, accounts: :mercury_accounts },
{ key: "brex", type: "BrexItem", association: :brex_items, accounts: :brex_accounts },
{ key: "sophtron", type: "SophtronItem", association: :sophtron_items, accounts: :sophtron_accounts },
{ key: "indexa_capital", type: "IndexaCapitalItem", association: :indexa_capital_items, accounts: :indexa_capital_accounts }
].freeze
class << self
def for_family(family)
PROVIDERS.flat_map do |provider|
relation = family.public_send(provider[:association])
items = relation.includes(association_includes_for(relation, provider)).ordered.to_a
sync_contexts = sync_contexts_for(provider[:type], items)
items.map do |item|
new(provider, item, sync_contexts.fetch(item.id, {})).to_h
end
end
end
private
def association_includes_for(relation, provider)
includes = [ { provider[:accounts] => :account_provider } ]
includes << provider[:linked_accounts] if provider[:linked_accounts]
includes << :accounts if relation.klass.reflect_on_association(:accounts)
includes
end
def sync_contexts_for(provider_type, items)
item_ids = items.map(&:id)
return {} if item_ids.empty?
latest_syncs = latest_syncs_for(provider_type, item_ids)
latest_completed_syncs = latest_syncs_for(provider_type, item_ids, scope: Sync.completed)
syncing_item_ids = Sync.visible
.where(syncable_type: provider_type, syncable_id: item_ids)
.distinct
.pluck(:syncable_id)
item_ids.index_with do |item_id|
{
latest_sync: latest_syncs[item_id],
latest_completed_sync: latest_completed_syncs[item_id],
syncing: syncing_item_ids.include?(item_id)
}
end
end
def latest_syncs_for(provider_type, item_ids, scope: Sync.all)
ranked_syncs = scope.where(syncable_type: provider_type, syncable_id: item_ids)
.select(
"syncs.*, " \
"ROW_NUMBER() OVER (PARTITION BY syncable_id ORDER BY created_at DESC, id DESC) AS sync_rank"
)
Sync.from(ranked_syncs, :syncs).where("sync_rank = 1").index_by(&:syncable_id)
end
end
def initialize(provider, item, sync_context = {})
@provider = provider
@item = item
@sync_context = sync_context
end
def to_h
{
id: item.id,
provider: provider[:key],
provider_type: provider[:type],
name: item_value(:name, provider[:key].humanize),
status: item_value(:status),
requires_update: item_boolean(:requires_update?),
credentials_configured: credentials_configured?,
scheduled_for_deletion: item_boolean(:scheduled_for_deletion?),
pending_account_setup: pending_account_setup?,
institution: institution_payload,
accounts: accounts_payload,
sync: sync_payload,
created_at: item.created_at,
updated_at: item.updated_at
}
end
private
attr_reader :provider, :item, :sync_context
def credentials_configured?
item_boolean(:credentials_configured?)
end
def pending_account_setup?
item_boolean(:pending_account_setup?)
end
def institution_payload
{
name: item_value(:institution_display_name, item_value(:name, provider[:key].humanize)),
domain: item_value(:institution_domain),
url: item_value(:institution_url)
}
end
def accounts_payload
@accounts_payload ||= begin
total = provider_account_count
linked = linked_account_count
{
total_count: total,
linked_count: linked,
unlinked_count: [ total - linked, 0 ].max
}
end
end
def provider_account_count
records = provider_account_records
return records.size if records
return item.total_accounts_count if item.respond_to?(:total_accounts_count)
0
end
def linked_account_count
records = provider_account_records
return records.count { |provider_account| linked_provider_account?(provider_account) } if records
return item.linked_accounts_count if item.respond_to?(:linked_accounts_count)
if provider[:linked_accounts] && item.respond_to?(provider[:linked_accounts])
return item.public_send(provider[:linked_accounts]).size
end
return item.accounts.size if item.respond_to?(:accounts)
0
end
def provider_account_records
return unless item.respond_to?(provider[:accounts])
@provider_account_records ||= item.public_send(provider[:accounts]).to_a
end
def linked_provider_account?(provider_account)
return false unless provider_account.respond_to?(:account_provider)
association = provider_account.association(:account_provider)
association.loaded? ? association.target.present? : provider_account.account_provider.present?
end
def sync_payload
{
syncing: syncing?,
status_summary: sync_status_summary,
last_synced_at: latest_completed_sync&.completed_at,
latest: latest_sync_payload(latest_sync)
}
end
def sync_status_summary
stats = latest_completed_sync_stats
counts = accounts_payload
total = stats.fetch("total_accounts", counts[:total_count]).to_i
linked = stats.fetch("linked_accounts", counts[:linked_count]).to_i
unlinked = stats.fetch("unlinked_accounts", [ total - linked, 0 ].max).to_i
if total.zero?
"No accounts found"
elsif unlinked.zero?
"#{linked} #{'account'.pluralize(linked)} synced"
else
"#{linked} synced, #{unlinked} need setup"
end
end
def syncing?
return sync_context[:syncing] if sync_context.key?(:syncing)
item_boolean(:syncing?)
end
def latest_sync
sync_context[:latest_sync]
end
def latest_completed_sync
sync_context[:latest_completed_sync]
end
def latest_completed_sync_stats
stats = latest_completed_sync&.sync_stats
return stats.stringify_keys if stats.is_a?(Hash)
return {} unless stats.is_a?(String)
parsed = JSON.parse(stats)
parsed.is_a?(Hash) ? parsed.stringify_keys : {}
rescue JSON::ParserError
{}
end
def latest_sync_payload(sync)
return unless sync
{
id: sync.id,
status: sync.status,
created_at: sync.created_at,
syncing_at: sync.syncing_at,
completed_at: sync.completed_at,
failed_at: sync.failed_at,
error: sync_error_payload(sync)
}
end
def sync_error_payload(sync)
return unless sync.failed? || sync.stale?
# Provider health treats stale connections as actionable even when the
# generic sync API suppresses stale-without-error payloads.
{
present: true,
message: sync.stale? ? "Sync became stale before completion" : "Sync failed"
}
end
def item_boolean(method_name)
item_value(method_name, false) == true
end
def item_value(method_name, default = nil)
return default unless item.respond_to?(method_name)
item.public_send(method_name)
end
end