Files
sure/app/models/indexa_capital_account/data_helpers.rb
GermanDZ c5503320af Fix IndexaCapital sync, account setup, and balance/type bugs (#1562)
* Add missing IndexaCapitalItem::SyncCompleteEvent

Syncable#sync_broadcaster instantiates self.class::SyncCompleteEvent,
which is implemented for every other provider (Plaid, Lunchflow,
Mercury, etc.) but was missing for IndexaCapitalItem. The error was
swallowed by Sync#perform_post_sync's rescue, so syncs appeared to
succeed but post-sync UI broadcasts never fired:

  Error performing post-sync for IndexaCapitalItem (...):
  uninitialized constant IndexaCapitalItem::SyncCompleteEvent

This adds the class, modeled on LunchflowItem::SyncCompleteEvent,
restoring per-account and per-item Turbo broadcasts after Indexa
Capital syncs.

* Fix IndexaCapital account setup never creating accounts

complete_account_setup read params[:accounts], but the form in
setup_accounts.html.erb submits account_ids[] (array) and
sync_start_dates[<id>] (hash). The hash was always empty, so every
submit hit the empty-config branch and bounced back with
"No accounts to set up." — accounts were never created.

The controller also branched on config[:account_type] / config[:subtype]
even though the form has no account-type picker (Indexa Capital is an
investment-only broker). Rewrote complete_account_setup to consume the
form's actual params and infer the accountable type as Investment from
indexa_capital_account.account_type.

* Fix IndexaCapital balance double-count and account type

Two more issues in the IndexaCapital flow that surfaced once accounts
could actually be created (see prior commit):

1. Accountable type was inferred from indexa_capital_account.account_type
   ("mutual" / "pension"), but infer_accountable_type doesn't recognize
   those values and falls through to "Depository". The result: every
   imported Indexa account showed up as a Cash depository account
   instead of an Investment account, hiding holdings/trades surfaces.
   Indexa Capital is investment-only, so hard-code the accountable
   type to Investment.

2. Account::Processor#calculate_total_balance summed every row in
   raw_holdings_payload. Indexa returns a time series — one row per
   security per date — so the naive sum double-counts (observed:
   reported €91,633 became stored balance €180,039). Trust the API's
   current_balance when present, and if we have to fall back to a
   computed total, dedupe by instrument and take the latest-dated
   amount per security.

* Fix IndexaCapital holdings reflecting oldest snapshot per security

HoldingsProcessor#process iterated every row in raw_holdings_payload.
Indexa returns a time series (many rows per security across dates),
and each iteration upserts the same (account, security, today) holding
row, so the LAST row processed wins. The payload is ordered with
newer dates first, so the last row processed is the OLDEST snapshot —
the holdings shown in the UI reflected tiny early positions instead
of the current ones (e.g. 3.8 shares of US 500 stored vs 62.34 actual).

Reduce the payload to one row per security (latest date) before
processing. The cost-basis update is now also driven by the latest
snapshot for the same reason.

* Fix IndexaCapital holdings using per-lot detail instead of totals

Importer#normalize_holdings_response read data[:fiscal_results], which
the Indexa API returns as per-tax-lot detail — many rows per security
covering each subscription_date, plus virtual sell/buy rows generated
by rebalances. Iterating it produced wildly wrong stored holdings:
e.g. 9.61 shares stored for Vanguard US 500 vs 62.34 actual; total
weights summed to ~10% instead of 100%.

The same response also includes data[:total_fiscal_results] — one
aggregated row per security with current titles/amount/cost matching
the Indexa UI and the user-downloadable positions CSV. Prefer it,
falling back to the per-lot field only when the totals are absent.

* Address CodeRabbit review on IndexaCapital fixes

Four review items, all fixed:

* Share instrument-key extraction
  HoldingsProcessor#extract_ticker and Processor#calculate_holdings_value
  used different fallback orders (one looked at :isin, the other at
  :isin_code), so they could disagree on which rows referred to the same
  security. Moved a single extract_instrument_key helper into
  IndexaCapitalAccount::DataHelpers and routed both callers through it.

* Simplify Processor#calculate_holdings_value
  The date-based dedupe was a workaround for the bug already fixed in
  the importer (which now stores total_fiscal_results — one row per
  security). Replaced the date comparison with a per-security map
  populated via the shared key extractor. Same end result, fewer
  moving parts, no fragile string-date comparison.

* Drop dead config key passed to create_account_from_indexa_capital
  create_account_from_indexa_capital only reads :subtype and :balance
  from its config arg. Passing :sync_start_date there was inert.

* Don't mark created accounts as skipped on post-create errors
  In complete_account_setup, ensure_account_provider! and
  update!(sync_start_date:) ran inside the same begin/rescue as the
  Account.create!. If either raised after the Account row was already
  persisted, control jumped to the rescue with created_count not yet
  incremented and the account was wrongly counted as skipped. Now:
  parse the form-supplied sync_start_date up front (a malformed value
  is silently dropped instead of bubbling out of the loop), bump
  created_count immediately after persisted?, and isolate the post-
  create steps in their own rescue so failures there are logged but
  don't desync the success counter.

* Fall back to /portfolio so pension plans get holdings imported

Indexa's /accounts/{id}/fiscal-results endpoint returns
{fiscal_results: [], total_fiscal_results: []} for pension plan
accounts (e.g. type "pension"). The same positions are exposed via
/accounts/{id}/portfolio in instrument_accounts[].positions[] for
both mutual funds and pensions, so use it as a fallback when
fiscal-results is empty.

The portfolio response uses the same field names HoldingsProcessor
already understands (instrument, titles, price, amount, cost_amount)
plus a derived cost_price (cost_amount / titles) added during
adaptation. No HoldingsProcessor changes needed.

Verified against the user-downloadable "Posiciones" CSV for an
SH71ZPMY pension account: two positions (N5138 Acciones, N5137
Bonos) and balance €8,273.56 match exactly.

* Fix CI: update tests for new IndexaCapital flow + rubocop blank line

* Lint: drop trailing blank line before `end` in
  IndexaCapitalAccount::Processor (Layout/EmptyLinesAroundClassBody).

* Controller test: complete_account_setup#creates was posting
  params: { accounts: { id => { account_type:, subtype: } } } against
  the old controller schema. The new endpoint reads
  params[:account_ids] and infers Investment for Indexa Capital, so
  switch the test to that shape (and update the matching skip-already-
  linked / no-selected-accounts cases).

* Processor test: "updates account balance from holdings value" set
  current_balance: 38905.21 alongside holdings summing to 27093.01
  and asserted the latter wins. After the fix
  (calculate_total_balance prefers the API-reported current_balance
  when present), the API value is the right answer. Renamed to
  "trusts API current_balance over holdings sum when present" and
  added a sibling test that nils current_balance to exercise the
  holdings-sum fallback path explicitly (still asserts 27093.01).

* Wrap account creation+linking in a transaction to avoid orphans

complete_account_setup created the Account row first, incremented
created_count, and only then called ensure_account_provider! / the
sync_start_date update inside an inner rescue. If the link or the
sync_start_date update raised after the Account was already persisted,
control fell into the inner rescue: the orphaned Account row stayed
in the database, the failure was silently logged, and the success
counter was inflated.

Wrap creation, ensure_account_provider!, and the optional
sync_start_date update in a single ActiveRecord::Base.transaction.
Increment created_count only after the transaction commits; on any
exception the outer rescue rolls the whole step into skipped_count
with a clear log line tagged with the indexa_capital_account id.
2026-04-27 18:33:22 +02:00

176 lines
5.6 KiB
Ruby

# frozen_string_literal: true
module IndexaCapitalAccount::DataHelpers
extend ActiveSupport::Concern
private
# Convert SDK objects to hashes via JSON round-trip
# Many SDKs return objects that don't have proper #to_h methods
def sdk_object_to_hash(obj)
return obj if obj.is_a?(Hash)
if obj.respond_to?(:to_json)
JSON.parse(obj.to_json)
elsif obj.respond_to?(:to_h)
obj.to_h
else
obj
end
rescue JSON::ParserError, TypeError
obj.respond_to?(:to_h) ? obj.to_h : {}
end
def parse_decimal(value)
return nil if value.nil?
case value
when BigDecimal
value
when String
BigDecimal(value)
when Numeric
BigDecimal(value.to_s)
else
nil
end
rescue ArgumentError => e
Rails.logger.error("IndexaCapitalAccount::DataHelpers - Failed to parse decimal value: #{value.inspect} - #{e.message}")
nil
end
# Extract the canonical security key from an Indexa fiscal-results row.
# Indexa's response shape varies between endpoints (and across time), so
# try the nested instrument hash first, then a few flat fallbacks. Both
# HoldingsProcessor and Processor#calculate_holdings_value go through
# this helper so they can't disagree on which rows refer to the same
# security.
def extract_instrument_key(data)
return nil unless data.respond_to?(:[])
hash = data.respond_to?(:with_indifferent_access) ? data.with_indifferent_access : data
instrument = hash[:instrument]
if instrument.is_a?(Hash)
nested = instrument.with_indifferent_access
return nested[:identifier] || nested[:isin_code] || nested[:isin]
end
hash[:identifier] || hash[:isin_code] || hash[:isin] || hash[:symbol] || hash[:ticker]
end
def parse_date(date_value)
return nil if date_value.nil?
case date_value
when Date
date_value
when String
# Use Time.zone.parse for external timestamps (Rails timezone guidelines)
Time.zone.parse(date_value)&.to_date
when Time, DateTime, ActiveSupport::TimeWithZone
date_value.to_date
else
nil
end
rescue ArgumentError, TypeError => e
Rails.logger.error("IndexaCapitalAccount::DataHelpers - Failed to parse date: #{date_value.inspect} - #{e.message}")
nil
end
# Find or create security with race condition handling
def resolve_security(symbol, symbol_data = {})
ticker = symbol.to_s.upcase.strip
return nil if ticker.blank?
security = Security.find_by(ticker: ticker)
# If security exists but has a bad name (looks like a hash), update it
if security && security.name&.start_with?("{")
new_name = extract_security_name(symbol_data, ticker)
Rails.logger.info "IndexaCapitalAccount::DataHelpers - Fixing security name: #{security.name.first(50)}... -> #{new_name}"
security.update!(name: new_name)
end
return security if security
# Create new security
security_name = extract_security_name(symbol_data, ticker)
Rails.logger.info "IndexaCapitalAccount::DataHelpers - Creating security: ticker=#{ticker}, name=#{security_name}"
Security.create!(
ticker: ticker,
name: security_name,
exchange_mic: extract_exchange(symbol_data),
country_code: extract_country_code(symbol_data)
)
rescue ActiveRecord::RecordInvalid, ActiveRecord::RecordNotUnique => e
# Handle race condition - another process may have created it
Rails.logger.error "IndexaCapitalAccount::DataHelpers - Failed to create security #{ticker}: #{e.message}"
Security.find_by(ticker: ticker)
end
def extract_security_name(symbol_data, fallback_ticker)
symbol_data = symbol_data.with_indifferent_access if symbol_data.respond_to?(:with_indifferent_access)
# Try various paths where the name might be
name = symbol_data[:name] || symbol_data[:description]
# If description is missing or looks like a type description, use ticker
if name.blank? || name.is_a?(Hash) || name =~ /^(COMMON STOCK|CRYPTOCURRENCY|ETF|MUTUAL FUND)$/i
name = fallback_ticker
end
# Titleize for readability if it's all caps
name = name.titleize if name == name.upcase && name.length > 4
name
end
def extract_exchange(symbol_data)
symbol_data = symbol_data.with_indifferent_access if symbol_data.respond_to?(:with_indifferent_access)
exchange = symbol_data[:exchange]
return nil unless exchange.is_a?(Hash)
exchange.with_indifferent_access[:mic_code] || exchange.with_indifferent_access[:id]
end
def extract_country_code(symbol_data)
symbol_data = symbol_data.with_indifferent_access if symbol_data.respond_to?(:with_indifferent_access)
# Try to extract country from currency or exchange
currency = symbol_data[:currency]
currency = currency.dig(:code) if currency.is_a?(Hash)
case currency
when "USD"
"US"
when "CAD"
"CA"
when "GBP", "GBX"
"GB"
when "EUR"
nil # Could be many countries
else
nil
end
end
# Handle currency as string or object (API inconsistency)
def extract_currency(data, fallback: nil)
data = data.with_indifferent_access if data.respond_to?(:with_indifferent_access)
currency_data = data[:currency]
return fallback if currency_data.blank?
if currency_data.is_a?(Hash)
currency_data.with_indifferent_access[:code] || fallback
elsif currency_data.is_a?(String)
currency_data.upcase
else
fallback
end
end
end