Files
sure/app/models/provider/up.rb
Jake dc2a565b6a feat(up): add Up Bank (AU) provider integration (#2391)
* feat(up): add Up Bank (AU) provider integration

Adds Up Bank as a per-family, token-based bank sync provider, modelled on
the existing Akahu integration. Up uses a JSON:API REST API with a personal
access token (Bearer), cursor pagination via links.next, and returns both
HELD (pending) and SETTLED transactions from one endpoint.

New:
- Provider::Up client (JSON:API unwrap, links.next pagination, retries,
  typed errors, /util/ping) + Provider::UpAdapter (Factory-registered,
  Depository + Loan).
- UpItem / UpAccount models with Provided, Unlinking, Syncer,
  SyncCompleteEvent, Importer, Processor, Transactions::Processor, and
  UpEntry::Processor (amount sign flip, HELD->pending, foreignAmount FX,
  merchant from description, stale-pending pruning).
- Family::UpConnectable, UpItemsController, routes, settings panel + connect
  flow views, accounts index wiring, initializer, en locale, and model tests.

Core wiring:
- "up" added to Transaction::PENDING_PROVIDERS, the three pending-match SQL
  blocks in Account::ProviderImportAdapter, Provider::Metadata::REGISTRY,
  ProviderMerchant/DataEnrichment source enums, ProviderConnectionStatus,
  settings provider panels, and financial data reset.

Migration create_up_items_and_accounts must be run before use. No external
API endpoints added (no OpenAPI changes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(up): dump up tables to schema and make since filter TZ-safe

The feature commit added the up_items/up_accounts migration but never
re-dumped db/schema.rb, leaving the schema version and tables stale.
Add the two table definitions and foreign keys and bump the schema
version so a fresh DB load matches the migration.

Also format a bare Date `since` as UTC midnight instead of the server's
local zone, so `filter[since]` is deterministic regardless of where the
app runs (previously shifted by the local UTC offset).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(up): address code review feedback

Behavior/correctness:
- Persist skipped accounts via a new up_accounts.ignored flag and a
  needs_setup scope, so skipped accounts stop resurfacing as "needs
  setup" on every sync. Linking clears the flag.
- destroy now checks unlink_all! per-account results and aborts deletion
  (alert) if any unlink failed, instead of swallowing failures.
- render_provider_panel_error redirect uses :see_other (was an invalid
  4xx redirect status).
- Up provider adapter falls back to item institution name/url when
  institution_metadata is absent (early return previously blocked it).

Resilience/security:
- fetch_all_resources guards against an API repeating the same
  links.next cursor (Set#add?), preventing infinite pagination.
- HTTP client validates absolute URLs (from links.next) against Up's
  HTTPS host before sending the bearer token, preventing credential
  leakage to untrusted hosts.

Diagnostics:
- Route provider sync/import failures through DebugLogEntry.capture
  (controller, UpItem, syncer, unlinking) with family/account context.
  Low-level HTTP client and currency-normalization warnings keep
  Rails.logger to match existing provider conventions.

Data integrity:
- up_accounts.name and currency are NOT NULL (align with model presence
  validations); account_id stays nullable (allow_nil uniqueness).

Forms:
- select_existing_account radio is required; controller guards a blank/
  unknown up_account_id with a friendly alert instead of RecordNotFound.

Tests:
- Add UpAccount needs_setup scope test, pagination loop guard test,
  untrusted-host rejection test; tighten filter[since] assertion to the
  exact UTC timestamp.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(up): address second-round review feedback

- Capture sync/import failures via DebugLogEntry so swallowed errors in
  account/transaction fetching and transaction processing surface in
  /settings/debug instead of only Rails.logger.
- Gate UP_DEBUG_RAW raw payload dump to local envs to avoid leaking PII
  (merchant names, amounts, account IDs) in managed/production logs.
- Collapse linked/unlinked/total account counts into one memoized query
  instead of 3 separate COUNTs per rendered item.
- Rename "Set Up Up Accounts" locale title to "Link Up Accounts".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(up): add method docstrings and align failed_result keys

Add docstrings to all Up provider source files (controller, models,
providers, concerns) to satisfy the 80% docstring coverage threshold.

Third-round review: failed_result now mirrors import's result shape
(accounts_updated/created/failed, transactions_imported/failed) instead
of the stale accounts_imported key, so failure results stay consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 20:33:09 +02:00

225 lines
7.9 KiB
Ruby

class Provider::Up
include HTTParty
extend SslConfigurable
DEFAULT_BASE_URL = "https://api.up.com.au/api/v1".freeze
DEFAULT_PAGE_SIZE = 100
# Host that authenticated requests (bearer token) may be sent to. Absolute URLs
# taken from API responses (links.next) are validated against this.
ALLOWED_HOST = URI.parse(DEFAULT_BASE_URL).host.freeze
headers "User-Agent" => "Sure Finance Up Client"
default_options.merge!({ timeout: 120 }.merge(httparty_ssl_options))
attr_reader :access_token
# Build a client with the family's Up personal access token. Raises if blank.
def initialize(access_token)
@access_token = access_token.to_s.strip
raise UpError.new("Up access token is required", :configuration_error) if @access_token.blank?
end
# GET /util/ping - validates the personal access token.
# Returns the parsed payload (contains meta.id / meta.statusEmoji) or raises UpError.
def ping
get("util/ping")
end
# GET /accounts - returns an array of flattened account hashes.
# Each hash: { id:, displayName:, accountType:, ownershipType:, balance: {...}, createdAt: }
def get_accounts
fetch_all_resources("accounts").map { |resource| flatten_account(resource) }
end
# GET /accounts/{id}/transactions - returns an array of flattened transaction hashes.
# Both HELD (pending) and SETTLED (posted) transactions are returned; callers derive
# pending status from the :status field.
def get_account_transactions(account_id:, since: nil, until_date: nil, page_size: DEFAULT_PAGE_SIZE)
query = { "page[size]" => page_size }
query["filter[since]"] = format_api_time(since) if since.present?
query["filter[until]"] = format_api_time(until_date) if until_date.present?
path = "accounts/#{ERB::Util.url_encode(account_id.to_s)}/transactions"
fetch_all_resources(path, query: query).map { |resource| flatten_transaction(resource) }
end
private
RETRYABLE_ERRORS = [
SocketError,
Net::OpenTimeout,
Net::ReadTimeout,
Errno::ECONNRESET,
Errno::ECONNREFUSED,
Errno::ETIMEDOUT,
EOFError
].freeze
MAX_RETRIES = 3
INITIAL_RETRY_DELAY = 2
# Follows JSON:API cursor pagination via links.next (absolute URLs) until exhausted,
# concatenating each page's `data` array.
def fetch_all_resources(path, query: {})
results = []
payload = get(path, query: query.presence)
seen_urls = Set.new
loop do
results.concat(Array(payload[:data]))
next_url = payload.dig(:links, :next)
break if next_url.blank?
# Guard against an API that returns the same cursor repeatedly.
break unless seen_urls.add?(next_url)
payload = get(next_url)
end
results
end
# Flattens a JSON:API account resource into a single hash with id + attributes.
def flatten_account(resource)
data = resource.with_indifferent_access
attributes = data[:attributes].is_a?(Hash) ? data[:attributes] : {}
attributes.merge(
id: data[:id],
type: data[:type]
).with_indifferent_access
end
# Flattens a JSON:API transaction resource, lifting attributes to the top level and
# extracting the related account/category ids from relationships.
def flatten_transaction(resource)
data = resource.with_indifferent_access
attributes = data[:attributes].is_a?(Hash) ? data[:attributes] : {}
attributes.merge(
id: data[:id],
account_id: data.dig(:relationships, :account, :data, :id),
category_id: data.dig(:relationships, :category, :data, :id)
).with_indifferent_access
end
def format_api_time(value)
return value if value.is_a?(String)
# A bare Date has no time/zone, so interpret it as UTC midnight rather than
# the server's local zone (which would shift `filter[since]` by the offset).
return value.to_time(:utc).iso8601 if value.instance_of?(Date)
value.to_time.utc.iso8601
end
# Issues a GET request. `path_or_url` may be a relative path (prefixed with the base URL)
# or an absolute URL (used when following pagination links).
def get(path_or_url, query: nil)
with_retries("GET #{path_or_url}") do
url = resolve_url(path_or_url)
response = self.class.get(url, headers: auth_headers, query: query)
handle_response(response)
end
end
# Resolves a relative path against the base URL, or validates an absolute URL
# so the bearer token is only ever sent to Up's HTTPS host.
def resolve_url(path_or_url)
value = path_or_url.to_s
return "#{DEFAULT_BASE_URL}/#{value}" unless value.start_with?("http")
uri = URI.parse(value)
unless uri.scheme == "https" && uri.host == ALLOWED_HOST
raise UpError.new("Refusing to send credentials to untrusted host: #{uri.host.inspect}", :invalid_url)
end
value
rescue URI::InvalidURIError
raise UpError.new("Invalid Up API URL", :invalid_url)
end
# Bearer-auth headers sent with every request.
def auth_headers
{
"Authorization" => "Bearer #{access_token}",
"Accept" => "application/json"
}
end
# Run the block, retrying transient network errors with exponential backoff.
def with_retries(operation_name, max_retries: MAX_RETRIES)
retries = 0
begin
yield
rescue *RETRYABLE_ERRORS => e
retries += 1
if retries <= max_retries
delay = calculate_retry_delay(retries)
Rails.logger.warn(
"Up API: #{operation_name} failed (attempt #{retries}/#{max_retries}): " \
"#{e.class}: #{e.message}. Retrying in #{delay}s..."
)
Kernel.sleep(delay)
retry
end
Rails.logger.error("Up API: #{operation_name} failed after #{max_retries} retries: #{e.class}: #{e.message}")
raise UpError.new("Network error after #{max_retries} retries: #{e.message}", :network_error)
end
end
# Exponential backoff delay (with jitter), capped at 30 seconds.
def calculate_retry_delay(retry_count)
base_delay = INITIAL_RETRY_DELAY * (2 ** (retry_count - 1))
jitter = base_delay * rand * 0.25
[ base_delay + jitter, 30 ].min
end
# Map an HTTP response to parsed data or a typed UpError by status code.
def handle_response(response)
case response.code
when 200, 201
parse_response_body(response)
when 204
{}
when 400
raise UpError.new("Bad request to Up API (status=#{response.code})", :bad_request)
when 401
raise UpError.new("Invalid Up access token", :unauthorized)
when 403
raise UpError.new("Up access forbidden - check token permissions", :access_forbidden)
when 404
raise UpError.new("Up resource not found", :not_found)
when 429
raise UpError.new("Up rate limit exceeded. Please try again later.", :rate_limited)
when 500..599
raise UpError.new("Up server error (#{response.code}). Please try again later.", :server_error)
else
Rails.logger.error "Up API: Unexpected response status=#{response.code}"
raise UpError.new("Failed to fetch Up data", :fetch_failed)
end
end
# Parse a JSON response body into a symbol-keyed hash, raising on bad JSON.
def parse_response_body(response)
return {} if response.body.blank?
JSON.parse(response.body, symbolize_names: true)
rescue JSON::ParserError => e
Rails.logger.error "Up API: Failed to parse response: #{e.class}"
raise UpError.new("Failed to parse Up API response", :parse_error)
end
# Error raised for Up API failures, tagged with a symbolic +error_type+.
class UpError < StandardError
attr_reader :error_type
# Build the error with a +message+ and a categorizing +error_type+.
def initialize(message, error_type = :unknown)
super(message)
@error_type = error_type
end
end
end