Files
sure/app/controllers/accounts_controller.rb
GermanDZ 7e1de420ca perf(accounts): kill sidebar/sparkline N+1s and cache the sidebar (#1683)
* perf(accounts): kill sidebar/sparkline N+1s and cache the sidebar

The dashboard was issuing hundreds of per-account `SELECT 1` and
polymorphic `accountable` lookups on every page load. Sidebar render
alone hit the DB ~50–100× and ran twice per request (mobile + desktop).

Changes:

- AccountableSparklinesController: short-circuit
  `requires_normalized_aggregation?` to Investment/Crypto only and
  collapse the per-account `linked?` loop into a single `EXISTS`. Kills
  the N+1 `AccountProvider Exists?` queries on every sparkline endpoint.

- BalanceSheet::AccountTotals#visible_accounts: preload `:accountable`,
  `:plaid_account`, `:simplefin_account`, and
  `account_providers: :provider` so the sidebar's
  `account.subtype` / `account.linked?` / `account.provider` calls don't
  trigger per-row polymorphic loads.

- AccountsController#index: same preloads on `@manual_accounts`.

- accounts/index/_account_groups.erb: extend the existing `Preloader`
  call to batch-load accountable + provider associations so the
  per-provider-item partials (Plaid, SimpleFIN, Coinbase, etc.) stop
  re-issuing N+1s when rendering account rows on /accounts.

- accounts/_account_sidebar_tabs.html.erb: wrap the partial in a
  `cache` block keyed on the family's data-version, the current user,
  shares fingerprint, locale, mobile flag, active tab, and a
  path-derived "current account" component (`sidebar_active_account_id`
  helper). The sidebar is rendered on every page in the layout
  (twice — mobile + desktop drawers), so most navigations now serve
  the cached fragment instead of re-walking accounts/balances.

Local impact (DZG family, 23 accounts, 6.1k transactions):
- Dashboard `/`: ~6.5s → ~1.95s
- /accounts: ~2.7s → ~0.85s on warm cache
- /accountable_sparklines/*: per-request N+1s eliminated; remaining
  cost is request boilerplate which can be addressed by bumping
  `RAILS_MAX_THREADS` (the dashboard fans out 5 sparkline turbo frames
  in parallel and Puma's default 3 threads serialize them).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(perf): address PR review on sidebar/sparkline perf changes

- AccountableSparklinesController#requires_normalized_aggregation?
  also matches legacy plaid_account_id / simplefin_account_id links,
  not just new-style account_providers, so investment/crypto accounts
  in the legacy linking state still get LinkedInvestmentSeriesNormalizer
  applied (Codex P1 / CodeRabbit major).

- Sidebar share fingerprint includes both `count` and `max(updated_at)`
  so deleting a non-most-recent AccountShare invalidates the cached
  fragment for users who lost access (Codex P1).

- Move the sidebar cache-key construction (incl. the AccountShare
  query) from the ERB into a new `account_sidebar_tabs_cache_key`
  helper, per the project's "no heavy logic in ERB" rule (CodeRabbit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(perf): address human review on perf PR

- Account.linked: new SQL-level scope mirroring `Account#linked?` so
  the controller and per-instance method share one definition. Removes
  the duplicated raw SQL string in
  `AccountableSparklinesController#requires_normalized_aggregation?`,
  which now reads `accounts.linked.exists?` (jjmata, sure-design).

- AccountsHelper: move `sidebar_active_account_id` and
  `account_sidebar_tabs_cache_key` out of `ApplicationHelper`. The
  cache-key helper also collapses the AccountShare `count` + `max(updated_at)`
  fingerprint into a single `pick` query so we don't pay two round-trips
  on every render (jjmata, sure-design).

- test/models/account/linkable_test.rb: pin the `Account.linked` scope
  against all three link types (account_providers, legacy plaid_account,
  legacy simplefin_account) so any future schema change that diverges
  the SQL definition from `linked?` breaks a test instead of silently
  serving wrong sparkline aggregations (sure-design).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(perf): correct shares cache fingerprint on raw-SQL pick

`pick(Arel.sql("count(*), max(updated_at)"))` passes a single comma-
separated fragment, which Rails returns as a String (per the documented
behavior of `pluck` with SQL fragments). The previous `max_at&.to_i`
silently truncated `"2025-05-06 12:34:56.789 UTC"` to `2025`, so the
sidebar cache key would not change for share `updated_at` movements
within the same calendar year — including share deletions — leaving
revoked users with a stale sidebar until the 12h expiry.

Pass the aggregates as two separate `Arel.sql` args and just concatenate
the raw String values into the cache key. The values only need to be
stable for a given DB state, not numerically meaningful.

Caught by CodeRabbit on PR #1683.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 17:31:16 +02:00

341 lines
13 KiB
Ruby

class AccountsController < ApplicationController
include StreamExtensions
before_action :set_account, only: %i[show sparkline sync set_default remove_default]
before_action :set_manageable_account, only: %i[toggle_active destroy unlink confirm_unlink select_provider]
include Periodable
def index
@accessible_account_ids = Current.user.accessible_accounts.pluck(:id)
@manual_accounts = family.accounts
.listable_manual
.where(id: @accessible_account_ids)
.includes(:accountable, :account_providers, :plaid_account, :simplefin_account)
.order(:name)
@plaid_items = visible_provider_items(family.plaid_items.ordered.includes(:syncs, :plaid_accounts))
@simplefin_items = visible_provider_items(family.simplefin_items.ordered.includes(:syncs))
@lunchflow_items = visible_provider_items(family.lunchflow_items.ordered.includes(:syncs, :lunchflow_accounts))
@enable_banking_items = visible_provider_items(family.enable_banking_items.ordered.includes(:syncs))
@coinstats_items = visible_provider_items(family.coinstats_items.ordered.includes(:coinstats_accounts, :accounts, :syncs))
@mercury_items = visible_provider_items(family.mercury_items.ordered.includes(:syncs, :mercury_accounts))
@coinbase_items = visible_provider_items(family.coinbase_items.ordered.includes(:coinbase_accounts, :accounts, :syncs))
@snaptrade_items = visible_provider_items(family.snaptrade_items.ordered.includes(:syncs, :snaptrade_accounts))
@indexa_capital_items = visible_provider_items(family.indexa_capital_items.ordered.includes(:syncs, :indexa_capital_accounts))
@sophtron_items = visible_provider_items(family.sophtron_items.ordered.includes(:syncs, :sophtron_accounts))
# Build sync stats maps for all providers
build_sync_stats_maps
# Prevent Turbo Drive from caching this page to ensure fresh account lists
expires_now
render layout: "settings"
end
def new
# Get all registered providers with any credentials configured
@provider_configs = Provider::Factory.registered_adapters.flat_map do |adapter_class|
adapter_class.connection_configs(family: family)
end
end
def sync_all
family.sync_later
redirect_to accounts_path, notice: t("accounts.sync_all.syncing")
end
def show
@chart_view = params[:chart_view] || "balance"
@tab = params[:tab]
@q = params.fetch(:q, {}).permit(:search, status: [])
entries = @account.entries.where(excluded: false).search(@q).reverse_chronological
@pagy, @entries = pagy(
entries,
limit: safe_per_page,
params: request.query_parameters.except("tab").merge("tab" => "activity")
)
@activity_feed_data = Account::ActivityFeedData.new(@account, @entries)
end
def sync
unless @account.syncing?
if @account.linked?
# Sync all provider items for this account
# Each provider item will trigger an account sync when complete
@account.account_providers.each do |account_provider|
item = account_provider.adapter&.item
item&.sync_later if item && !item.syncing?
end
else
# Manual accounts just need balance materialization
@account.sync_later
end
end
redirect_to account_path(@account)
end
def sparkline
etag_key = @account.family.build_cache_key("#{@account.id}_sparkline_#{Account::Chartable::SPARKLINE_CACHE_VERSION}", invalidate_on_data_updates: true)
# Short-circuit with 304 Not Modified when the client already has the latest version.
# We defer the expensive series computation until we know the content is stale.
if stale?(etag: etag_key, last_modified: @account.family.latest_sync_completed_at)
@sparkline_series = @account.sparkline_series
render layout: false
end
end
def toggle_active
if @account.active?
@account.disable!
elsif @account.disabled?
@account.enable!
end
redirect_to accounts_path
end
def set_default
unless @account.eligible_for_transaction_default?
redirect_to accounts_path, alert: t("accounts.set_default.depository_only")
return
end
Current.user.update!(default_account: @account)
redirect_to accounts_path
end
def remove_default
Current.user.update!(default_account: nil)
redirect_to accounts_path
end
def destroy
if @account.linked?
redirect_to account_path(@account), alert: t("accounts.destroy.cannot_delete_linked")
else
begin
@account.destroy_later
redirect_to accounts_path, notice: t("accounts.destroy.success", type: @account.accountable_type)
rescue => e
Rails.logger.error "Failed to schedule account #{@account.id} for deletion: #{e.message}"
redirect_to accounts_path, alert: t("accounts.destroy.failed")
end
end
end
def confirm_unlink
unless @account.linked?
redirect_to account_path(@account), alert: t("accounts.unlink.not_linked")
end
end
def unlink
unless @account.linked?
redirect_to account_path(@account), alert: t("accounts.unlink.not_linked")
return
end
begin
Account.transaction do
# Detach holdings from provider links before destroying them
provider_link_ids = @account.account_providers.pluck(:id)
if provider_link_ids.any?
Holding.where(account_provider_id: provider_link_ids).update_all(account_provider_id: nil)
end
# Capture provider accounts before clearing links (so we can destroy them)
simplefin_account_to_destroy = @account.simplefin_account
# Remove new system links (account_providers join table)
# SnaptradeAccount records are preserved (not destroyed) so users can relink later.
# This follows the Plaid pattern where the provider account survives as "unlinked".
# SnapTrade has limited connection slots (5 free), so preserving the record avoids
# wasting a slot on reconnect.
@account.account_providers.destroy_all
# Remove legacy system links (foreign keys)
@account.update!(plaid_account_id: nil, simplefin_account_id: nil)
# Destroy the SimplefinAccount record so it doesn't cause stale account issues
# This is safe because:
# - Account data (transactions, holdings, balances) lives on the Account, not SimplefinAccount
# - SimplefinAccount only caches API data which is regenerated on reconnect
# - If user reconnects SimpleFin later, a new SimplefinAccount will be created
simplefin_account_to_destroy&.destroy!
end
redirect_to accounts_path, notice: t("accounts.unlink.success")
rescue ActiveRecord::RecordInvalid => e
redirect_to account_path(@account), alert: t("accounts.unlink.error", error: e.message)
rescue StandardError => e
Rails.logger.error "Failed to unlink account #{@account.id}: #{e.message}"
redirect_to account_path(@account), alert: t("accounts.unlink.error", error: t("accounts.unlink.generic_error"))
end
end
def select_provider
if @account.linked?
redirect_to account_path(@account), alert: t("accounts.select_provider.already_linked")
return
end
account_type_name = @account.accountable_type
# Get all available provider configs dynamically for this account type
provider_configs = Provider::Factory.connection_configs_for_account_type(
account_type: account_type_name,
family: family
)
# Build available providers list with paths resolved for this specific account
# Filter out providers that don't support linking to existing accounts
@available_providers = provider_configs.filter_map do |config|
next unless config[:existing_account_path].present?
{
name: config[:name],
key: config[:key],
description: config[:description],
path: config[:existing_account_path].call(@account.id)
}
end
if @available_providers.empty?
redirect_to account_path(@account), alert: t("accounts.select_provider.no_providers")
end
end
private
def family
Current.family
end
def set_account
@account = Current.user.accessible_accounts.find(params[:id])
end
def set_manageable_account
@account = Current.user.accessible_accounts.find(params[:id])
permission = @account.permission_for(Current.user)
unless permission.in?([ :owner, :full_control ])
respond_to do |format|
format.html { redirect_to account_path(@account), alert: t("accounts.not_authorized") }
format.turbo_stream { stream_redirect_to(account_path(@account), alert: t("accounts.not_authorized")) }
end
nil
end
end
def visible_provider_items(items)
items.select do |item|
Current.user.admin? ||
(item.respond_to?(:accounts) && (item.accounts.map(&:id) & @accessible_account_ids).any?)
end
end
# Builds sync stats maps for all provider types to avoid N+1 queries in views
def build_sync_stats_maps
# SimpleFIN sync stats
@simplefin_sync_stats_map = {}
@simplefin_has_unlinked_map = {}
@simplefin_unlinked_count_map = {}
@simplefin_show_relink_map = {}
@simplefin_duplicate_only_map = {}
@simplefin_items.each do |item|
latest_sync = item.syncs.ordered.first
stats = latest_sync&.sync_stats || {}
@simplefin_sync_stats_map[item.id] = stats
@simplefin_has_unlinked_map[item.id] = item.family.accounts.listable_manual.exists?
# Count unlinked accounts
count = item.simplefin_accounts
.left_joins(:account, :account_provider)
.where(accounts: { id: nil }, account_providers: { id: nil })
.count
@simplefin_unlinked_count_map[item.id] = count
# CTA visibility
manuals_exist = @simplefin_has_unlinked_map[item.id]
sfa_any = item.simplefin_accounts.loaded? ? item.simplefin_accounts.any? : item.simplefin_accounts.exists?
@simplefin_show_relink_map[item.id] = (count.to_i == 0 && manuals_exist && sfa_any)
# Check if all errors are duplicate-skips
errors = Array(stats["errors"]).map { |e| e.is_a?(Hash) ? e["message"] || e[:message] : e.to_s }
@simplefin_duplicate_only_map[item.id] = errors.present? && errors.all? { |m| m.to_s.downcase.include?("duplicate upstream account detected") }
rescue => e
Rails.logger.warn("SimpleFin stats map build failed for item #{item.id}: #{e.class} - #{e.message}")
@simplefin_sync_stats_map[item.id] = {}
@simplefin_show_relink_map[item.id] = false
@simplefin_duplicate_only_map[item.id] = false
end
# Plaid sync stats
@plaid_sync_stats_map = {}
@plaid_items.each do |item|
latest_sync = item.syncs.ordered.first
@plaid_sync_stats_map[item.id] = latest_sync&.sync_stats || {}
end
# Lunchflow sync stats
@lunchflow_sync_stats_map = {}
@lunchflow_items.each do |item|
latest_sync = item.syncs.ordered.first
@lunchflow_sync_stats_map[item.id] = latest_sync&.sync_stats || {}
end
# Enable Banking sync stats
@enable_banking_sync_stats_map = {}
@enable_banking_latest_sync_error_map = {}
@enable_banking_items.each do |item|
latest_sync = item.syncs.ordered.first
@enable_banking_sync_stats_map[item.id] = latest_sync&.sync_stats || {}
@enable_banking_latest_sync_error_map[item.id] = latest_sync&.error
end
# CoinStats sync stats
@coinstats_sync_stats_map = {}
@coinstats_items.each do |item|
latest_sync = item.syncs.ordered.first
@coinstats_sync_stats_map[item.id] = latest_sync&.sync_stats || {}
end
# Sophtron sync stats
@sophtron_sync_stats_map = {}
@sophtron_items.each do |item|
latest_sync = item.syncs.ordered.first
@sophtron_sync_stats_map[item.id] = latest_sync&.sync_stats || {}
end
# Mercury sync stats
@mercury_sync_stats_map = {}
@mercury_items.each do |item|
latest_sync = item.syncs.ordered.first
@mercury_sync_stats_map[item.id] = latest_sync&.sync_stats || {}
end
# Coinbase sync stats
@coinbase_sync_stats_map = {}
@coinbase_unlinked_count_map = {}
@coinbase_items.each do |item|
latest_sync = item.syncs.ordered.first
@coinbase_sync_stats_map[item.id] = latest_sync&.sync_stats || {}
# Count unlinked accounts
count = item.coinbase_accounts
.left_joins(:account_provider)
.where(account_providers: { id: nil })
.count
@coinbase_unlinked_count_map[item.id] = count
end
# IndexaCapital sync stats
@indexa_capital_sync_stats_map = {}
@indexa_capital_items.each do |item|
latest_sync = item.syncs.ordered.first
@indexa_capital_sync_stats_map[item.id] = latest_sync&.sync_stats || {}
end
end
end