Files
sure/app/models/kraken_account/processor.rb
ghost d329a4f69d feat(kraken): import deposits, withdrawals, staking & fees via Ledgers API (#2451)
* feat(kraken): fetch and import Ledgers API for deposits, withdrawals, staking, fees

Closes #2450

Kraken TradesHistory only returns spot buy/sell trades. The Ledgers API
(/0/private/Ledgers) covers deposits, withdrawals, staking rewards, Earn
income, and standalone fees — everything that was missing from syncs.

Changes:
- Provider::Kraken#get_ledgers — new method forwarding start/type/offset params
- KrakenItem::Importer#fetch_ledgers — paginated fetch (up to 200 pages) with
  graceful fallback if the API key lacks Query Ledger Entries permission
- Importer#upsert_kraken_account — stores "ledgers" alongside "trades" in
  raw_transactions_payload
- KrakenAccount::LedgerProcessor — new class; maps each supported ledger type
  (deposit, withdrawal, staking, earn, fee) to a Transaction entry with the
  correct investment_activity_label, kind, and sign convention; skips trade/
  transfer/margin types to avoid double-counting with TradesHistory
- KrakenAccount::Processor#process — calls LedgerProcessor after process_trades
- Multi-currency: fiat amounts converted via ExchangeRate (non-USD fiat bridged
  through USD); crypto amounts use the spot price cached in raw_payload["assets"]
  with a price_missing flag when no price is available
- Dedup guard: external_id "kraken_ledger_<id>" + source "kraken" prevents
  re-importing on repeated syncs

* fix(kraken): correct sign convention, fee inclusion, and earn subtype filtering

- Sign convention: deposits/staking/earn → negative (inflow), withdrawals/fees
  → positive (outflow), matching Sure's global convention (inflow is negative)
- Fee inclusion: use (amount - fee).abs as abs_impact so withdrawal fees are
  counted in the total outflow rather than discarded
- Earn subtypes: skip allocation/deallocation ledger entries (internal fund
  movements); only import rewardallocation/bonusallocation as Interest income
- DebugLogEntry: replace Rails.logger.warn/error with DebugLogEntry.capture
  throughout LedgerProcessor and the Ledgers permission fallback in Importer,
  so support-relevant incidents surface in /settings/debug
- Importer test: stub get_ledgers in setup so existing tests do not error
  on the new fetch_ledgers call

* fix(kraken): route duplicate ledger-id warning through DebugLogEntry

* perf(kraken): batch ledger idempotency check; strengthen tests

Address review feedback (jjmata):

- N+1: LedgerProcessor#process_ledger_entry ran `account.entries.exists?(...)`
  per ledger entry (up to ~10k per sync). Load the existing Kraken external IDs
  once into a Set and test membership in memory (newly created IDs are added so
  the same run stays idempotent) — same pattern as #2452.
- Tests: the idempotency test now asserts the first pass actually creates the
  entry (assert_difference) before asserting the second is a no-op; add a guard
  asserting the second (all-skipped) pass issues a single bulk external_id pluck,
  not one query per entry.

No behavior change to imported entries.

* perf(kraken): scope ledger idempotency pluck to kraken_ledger_ prefix

Only load existing ledger external IDs (not trade entries) into the idempotency
Set, matching the reviewed approach. No behavior change.
2026-06-30 07:35:14 +02:00

124 lines
3.7 KiB
Ruby

# frozen_string_literal: true
class KrakenAccount::Processor
include KrakenAccount::UsdConverter
attr_reader :kraken_account
def initialize(kraken_account)
@kraken_account = kraken_account
end
def process
return unless kraken_account.current_account.present?
KrakenAccount::HoldingsProcessor.new(kraken_account).process
process_account!
process_trades
KrakenAccount::LedgerProcessor.new(kraken_account).process
end
private
def target_currency
kraken_account.kraken_item&.family&.currency
end
def process_account!
account = kraken_account.current_account
amount, stale, rate_date = convert_from_usd((kraken_account.current_balance || 0).to_d, date: Date.current)
account.update!(
balance: amount,
cash_balance: 0,
currency: target_currency
)
kraken_account.update!(extra: kraken_account.extra.to_h.deep_merge(build_stale_extra(stale, rate_date, Date.current)))
end
def process_trades
raw_trades.each do |txid, trade|
process_trade(txid, trade)
end
rescue StandardError => e
Rails.logger.error "KrakenAccount::Processor - trade processing failed: #{e.message}"
end
def raw_trades
kraken_account.raw_transactions_payload&.dig("trades") || {}
end
def process_trade(txid, trade)
account = kraken_account.current_account
return unless account
external_id = "kraken_trade_#{txid}"
return if account.entries.exists?(external_id: external_id, source: "kraken")
type = trade["type"].to_s.downcase
return unless %w[buy sell].include?(type)
pair = trade["pair"].to_s
base_symbol, quote_symbol = infer_pair_symbols(pair, trade)
return if base_symbol.blank?
qty = trade["vol"].to_d
return if qty.zero?
price = trade["price"].to_d
cost = trade["cost"].presence&.to_d
cost ||= (qty * price).round(8)
fee = trade["fee"].presence&.to_d || 0
currency = quote_symbol.presence || "USD"
date = Time.zone.at(trade["time"].to_d).to_date
security = KrakenAccount::SecurityResolver.resolve("CRYPTO:#{base_symbol}", base_symbol)
return unless security
entry_amount = type == "buy" ? -cost : cost
trade_qty = type == "buy" ? qty : -qty
label = type == "buy" ? "Buy" : "Sell"
account.entries.create!(
date: date,
name: "#{label} #{qty.round(8)} #{base_symbol}",
amount: entry_amount,
currency: currency,
external_id: external_id,
source: "kraken",
notes: trade["ordertxid"].presence,
entryable: Trade.new(
security: security,
qty: trade_qty,
price: price,
currency: currency,
fee: fee,
investment_activity_label: label
)
)
rescue StandardError => e
Rails.logger.error "KrakenAccount::Processor - failed to process trade #{txid}: #{e.message}"
end
def infer_pair_symbols(pair, trade)
pair_metadata = kraken_account.raw_payload&.dig("pair_metadata") || {}
metadata = pair_metadata[pair] || pair_metadata.values.find { |candidate| candidate["altname"].to_s == pair }
normalizer = KrakenAccount::AssetNormalizer.new(kraken_account.raw_payload&.dig("asset_metadata") || {})
if metadata
base = normalizer.normalize(metadata["base"])[:symbol]
quote = normalizer.normalize(metadata["quote"])[:symbol]
return [ base, quote ]
end
altname = trade["pair"].to_s
%w[USDT USDC USD EUR GBP BTC ETH].each do |quote|
next unless altname.end_with?(quote)
return [ normalizer.normalize(altname.delete_suffix(quote))[:symbol], quote ]
end
[ altname, "USD" ]
end
end