mirror of
https://github.com/we-promise/sure.git
synced 2026-07-18 15:55:22 +00:00
* feat(up): add Up Bank (AU) provider integration Adds Up Bank as a per-family, token-based bank sync provider, modelled on the existing Akahu integration. Up uses a JSON:API REST API with a personal access token (Bearer), cursor pagination via links.next, and returns both HELD (pending) and SETTLED transactions from one endpoint. New: - Provider::Up client (JSON:API unwrap, links.next pagination, retries, typed errors, /util/ping) + Provider::UpAdapter (Factory-registered, Depository + Loan). - UpItem / UpAccount models with Provided, Unlinking, Syncer, SyncCompleteEvent, Importer, Processor, Transactions::Processor, and UpEntry::Processor (amount sign flip, HELD->pending, foreignAmount FX, merchant from description, stale-pending pruning). - Family::UpConnectable, UpItemsController, routes, settings panel + connect flow views, accounts index wiring, initializer, en locale, and model tests. Core wiring: - "up" added to Transaction::PENDING_PROVIDERS, the three pending-match SQL blocks in Account::ProviderImportAdapter, Provider::Metadata::REGISTRY, ProviderMerchant/DataEnrichment source enums, ProviderConnectionStatus, settings provider panels, and financial data reset. Migration create_up_items_and_accounts must be run before use. No external API endpoints added (no OpenAPI changes). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(up): dump up tables to schema and make since filter TZ-safe The feature commit added the up_items/up_accounts migration but never re-dumped db/schema.rb, leaving the schema version and tables stale. Add the two table definitions and foreign keys and bump the schema version so a fresh DB load matches the migration. Also format a bare Date `since` as UTC midnight instead of the server's local zone, so `filter[since]` is deterministic regardless of where the app runs (previously shifted by the local UTC offset). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(up): address code review feedback Behavior/correctness: - Persist skipped accounts via a new up_accounts.ignored flag and a needs_setup scope, so skipped accounts stop resurfacing as "needs setup" on every sync. Linking clears the flag. - destroy now checks unlink_all! per-account results and aborts deletion (alert) if any unlink failed, instead of swallowing failures. - render_provider_panel_error redirect uses :see_other (was an invalid 4xx redirect status). - Up provider adapter falls back to item institution name/url when institution_metadata is absent (early return previously blocked it). Resilience/security: - fetch_all_resources guards against an API repeating the same links.next cursor (Set#add?), preventing infinite pagination. - HTTP client validates absolute URLs (from links.next) against Up's HTTPS host before sending the bearer token, preventing credential leakage to untrusted hosts. Diagnostics: - Route provider sync/import failures through DebugLogEntry.capture (controller, UpItem, syncer, unlinking) with family/account context. Low-level HTTP client and currency-normalization warnings keep Rails.logger to match existing provider conventions. Data integrity: - up_accounts.name and currency are NOT NULL (align with model presence validations); account_id stays nullable (allow_nil uniqueness). Forms: - select_existing_account radio is required; controller guards a blank/ unknown up_account_id with a friendly alert instead of RecordNotFound. Tests: - Add UpAccount needs_setup scope test, pagination loop guard test, untrusted-host rejection test; tighten filter[since] assertion to the exact UTC timestamp. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(up): address second-round review feedback - Capture sync/import failures via DebugLogEntry so swallowed errors in account/transaction fetching and transaction processing surface in /settings/debug instead of only Rails.logger. - Gate UP_DEBUG_RAW raw payload dump to local envs to avoid leaking PII (merchant names, amounts, account IDs) in managed/production logs. - Collapse linked/unlinked/total account counts into one memoized query instead of 3 separate COUNTs per rendered item. - Rename "Set Up Up Accounts" locale title to "Link Up Accounts". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(up): add method docstrings and align failed_result keys Add docstrings to all Up provider source files (controller, models, providers, concerns) to satisfy the 80% docstring coverage threshold. Third-round review: failed_result now mirrors import's result shape (accounts_updated/created/failed, transactions_imported/failed) instead of the stale accounts_imported key, so failure results stay consistent. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
225 lines
7.9 KiB
Ruby
225 lines
7.9 KiB
Ruby
class Provider::Up
|
|
include HTTParty
|
|
extend SslConfigurable
|
|
|
|
DEFAULT_BASE_URL = "https://api.up.com.au/api/v1".freeze
|
|
DEFAULT_PAGE_SIZE = 100
|
|
# Host that authenticated requests (bearer token) may be sent to. Absolute URLs
|
|
# taken from API responses (links.next) are validated against this.
|
|
ALLOWED_HOST = URI.parse(DEFAULT_BASE_URL).host.freeze
|
|
|
|
headers "User-Agent" => "Sure Finance Up Client"
|
|
default_options.merge!({ timeout: 120 }.merge(httparty_ssl_options))
|
|
|
|
attr_reader :access_token
|
|
|
|
# Build a client with the family's Up personal access token. Raises if blank.
|
|
def initialize(access_token)
|
|
@access_token = access_token.to_s.strip
|
|
|
|
raise UpError.new("Up access token is required", :configuration_error) if @access_token.blank?
|
|
end
|
|
|
|
# GET /util/ping - validates the personal access token.
|
|
# Returns the parsed payload (contains meta.id / meta.statusEmoji) or raises UpError.
|
|
def ping
|
|
get("util/ping")
|
|
end
|
|
|
|
# GET /accounts - returns an array of flattened account hashes.
|
|
# Each hash: { id:, displayName:, accountType:, ownershipType:, balance: {...}, createdAt: }
|
|
def get_accounts
|
|
fetch_all_resources("accounts").map { |resource| flatten_account(resource) }
|
|
end
|
|
|
|
# GET /accounts/{id}/transactions - returns an array of flattened transaction hashes.
|
|
# Both HELD (pending) and SETTLED (posted) transactions are returned; callers derive
|
|
# pending status from the :status field.
|
|
def get_account_transactions(account_id:, since: nil, until_date: nil, page_size: DEFAULT_PAGE_SIZE)
|
|
query = { "page[size]" => page_size }
|
|
query["filter[since]"] = format_api_time(since) if since.present?
|
|
query["filter[until]"] = format_api_time(until_date) if until_date.present?
|
|
|
|
path = "accounts/#{ERB::Util.url_encode(account_id.to_s)}/transactions"
|
|
fetch_all_resources(path, query: query).map { |resource| flatten_transaction(resource) }
|
|
end
|
|
|
|
private
|
|
|
|
RETRYABLE_ERRORS = [
|
|
SocketError,
|
|
Net::OpenTimeout,
|
|
Net::ReadTimeout,
|
|
Errno::ECONNRESET,
|
|
Errno::ECONNREFUSED,
|
|
Errno::ETIMEDOUT,
|
|
EOFError
|
|
].freeze
|
|
|
|
MAX_RETRIES = 3
|
|
INITIAL_RETRY_DELAY = 2
|
|
|
|
# Follows JSON:API cursor pagination via links.next (absolute URLs) until exhausted,
|
|
# concatenating each page's `data` array.
|
|
def fetch_all_resources(path, query: {})
|
|
results = []
|
|
payload = get(path, query: query.presence)
|
|
seen_urls = Set.new
|
|
|
|
loop do
|
|
results.concat(Array(payload[:data]))
|
|
next_url = payload.dig(:links, :next)
|
|
break if next_url.blank?
|
|
# Guard against an API that returns the same cursor repeatedly.
|
|
break unless seen_urls.add?(next_url)
|
|
|
|
payload = get(next_url)
|
|
end
|
|
|
|
results
|
|
end
|
|
|
|
# Flattens a JSON:API account resource into a single hash with id + attributes.
|
|
def flatten_account(resource)
|
|
data = resource.with_indifferent_access
|
|
attributes = data[:attributes].is_a?(Hash) ? data[:attributes] : {}
|
|
|
|
attributes.merge(
|
|
id: data[:id],
|
|
type: data[:type]
|
|
).with_indifferent_access
|
|
end
|
|
|
|
# Flattens a JSON:API transaction resource, lifting attributes to the top level and
|
|
# extracting the related account/category ids from relationships.
|
|
def flatten_transaction(resource)
|
|
data = resource.with_indifferent_access
|
|
attributes = data[:attributes].is_a?(Hash) ? data[:attributes] : {}
|
|
|
|
attributes.merge(
|
|
id: data[:id],
|
|
account_id: data.dig(:relationships, :account, :data, :id),
|
|
category_id: data.dig(:relationships, :category, :data, :id)
|
|
).with_indifferent_access
|
|
end
|
|
|
|
def format_api_time(value)
|
|
return value if value.is_a?(String)
|
|
# A bare Date has no time/zone, so interpret it as UTC midnight rather than
|
|
# the server's local zone (which would shift `filter[since]` by the offset).
|
|
return value.to_time(:utc).iso8601 if value.instance_of?(Date)
|
|
|
|
value.to_time.utc.iso8601
|
|
end
|
|
|
|
# Issues a GET request. `path_or_url` may be a relative path (prefixed with the base URL)
|
|
# or an absolute URL (used when following pagination links).
|
|
def get(path_or_url, query: nil)
|
|
with_retries("GET #{path_or_url}") do
|
|
url = resolve_url(path_or_url)
|
|
response = self.class.get(url, headers: auth_headers, query: query)
|
|
handle_response(response)
|
|
end
|
|
end
|
|
|
|
# Resolves a relative path against the base URL, or validates an absolute URL
|
|
# so the bearer token is only ever sent to Up's HTTPS host.
|
|
def resolve_url(path_or_url)
|
|
value = path_or_url.to_s
|
|
return "#{DEFAULT_BASE_URL}/#{value}" unless value.start_with?("http")
|
|
|
|
uri = URI.parse(value)
|
|
unless uri.scheme == "https" && uri.host == ALLOWED_HOST
|
|
raise UpError.new("Refusing to send credentials to untrusted host: #{uri.host.inspect}", :invalid_url)
|
|
end
|
|
|
|
value
|
|
rescue URI::InvalidURIError
|
|
raise UpError.new("Invalid Up API URL", :invalid_url)
|
|
end
|
|
|
|
# Bearer-auth headers sent with every request.
|
|
def auth_headers
|
|
{
|
|
"Authorization" => "Bearer #{access_token}",
|
|
"Accept" => "application/json"
|
|
}
|
|
end
|
|
|
|
# Run the block, retrying transient network errors with exponential backoff.
|
|
def with_retries(operation_name, max_retries: MAX_RETRIES)
|
|
retries = 0
|
|
|
|
begin
|
|
yield
|
|
rescue *RETRYABLE_ERRORS => e
|
|
retries += 1
|
|
if retries <= max_retries
|
|
delay = calculate_retry_delay(retries)
|
|
Rails.logger.warn(
|
|
"Up API: #{operation_name} failed (attempt #{retries}/#{max_retries}): " \
|
|
"#{e.class}: #{e.message}. Retrying in #{delay}s..."
|
|
)
|
|
Kernel.sleep(delay)
|
|
retry
|
|
end
|
|
|
|
Rails.logger.error("Up API: #{operation_name} failed after #{max_retries} retries: #{e.class}: #{e.message}")
|
|
raise UpError.new("Network error after #{max_retries} retries: #{e.message}", :network_error)
|
|
end
|
|
end
|
|
|
|
# Exponential backoff delay (with jitter), capped at 30 seconds.
|
|
def calculate_retry_delay(retry_count)
|
|
base_delay = INITIAL_RETRY_DELAY * (2 ** (retry_count - 1))
|
|
jitter = base_delay * rand * 0.25
|
|
[ base_delay + jitter, 30 ].min
|
|
end
|
|
|
|
# Map an HTTP response to parsed data or a typed UpError by status code.
|
|
def handle_response(response)
|
|
case response.code
|
|
when 200, 201
|
|
parse_response_body(response)
|
|
when 204
|
|
{}
|
|
when 400
|
|
raise UpError.new("Bad request to Up API (status=#{response.code})", :bad_request)
|
|
when 401
|
|
raise UpError.new("Invalid Up access token", :unauthorized)
|
|
when 403
|
|
raise UpError.new("Up access forbidden - check token permissions", :access_forbidden)
|
|
when 404
|
|
raise UpError.new("Up resource not found", :not_found)
|
|
when 429
|
|
raise UpError.new("Up rate limit exceeded. Please try again later.", :rate_limited)
|
|
when 500..599
|
|
raise UpError.new("Up server error (#{response.code}). Please try again later.", :server_error)
|
|
else
|
|
Rails.logger.error "Up API: Unexpected response status=#{response.code}"
|
|
raise UpError.new("Failed to fetch Up data", :fetch_failed)
|
|
end
|
|
end
|
|
|
|
# Parse a JSON response body into a symbol-keyed hash, raising on bad JSON.
|
|
def parse_response_body(response)
|
|
return {} if response.body.blank?
|
|
|
|
JSON.parse(response.body, symbolize_names: true)
|
|
rescue JSON::ParserError => e
|
|
Rails.logger.error "Up API: Failed to parse response: #{e.class}"
|
|
raise UpError.new("Failed to parse Up API response", :parse_error)
|
|
end
|
|
|
|
# Error raised for Up API failures, tagged with a symbolic +error_type+.
|
|
class UpError < StandardError
|
|
attr_reader :error_type
|
|
|
|
# Build the error with a +message+ and a categorizing +error_type+.
|
|
def initialize(message, error_type = :unknown)
|
|
super(message)
|
|
@error_type = error_type
|
|
end
|
|
end
|
|
end
|