Files
sure/app/models/insight/generators/spending_anomaly_generator.rb
Juan José Mata 66cf9e7f0b feat(insights): proactive financial intelligence feed (#2550)
* feat(insights): proactive financial intelligence feed

Adds a nightly job that analyzes each family's finances in pure Ruby and
surfaces typed, stateful insights on the dashboard and a new /insights page.

- Insight model (active/read/dismissed) with a per-family dedup_key unique
  index so nightly re-runs refresh rows instead of duplicating them
- Seven generators (spending anomaly, cash-flow warning, net worth milestone,
  subscription audit, savings rate change, idle cash, budget health) built on
  IncomeStatement, BalanceSheet, RecurringTransaction, and BudgetCategory
- LLM used as a writer, not a reasoner: Insight::BodyWriter narrates
  pre-computed facts via the configured provider, with an i18n template
  fallback so self-hosted installs without API keys work identically; bodies
  are only (re)written when an insight is new or its numbers changed
- GenerateInsightsJob: cron fan-out per family, per-family advisory lock,
  metadata-diff upsert that preserves read/dismissed state for unchanged
  signals and reactivates on material change
- Dashboard insights_feed section (top 3, collapsible/reorderable) and an
  /insights feed page with turbo-stream dismissal; viewing the feed marks
  insights read
- Tests for the model, job upsert semantics, and controller flows

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): address Codex review — expiry, budget metadata, LLM usage

- Expire visible insights whose condition cleared: generators declare the
  insight types they produce, the registry reports which generators ran to
  completion, and the job expires visible insights of those types whose
  dedup_key was not regenerated. A crashing generator can't wipe out its
  healthy insights, and an expired insight reactivates when its condition
  returns — unlike a user-dismissed one, which stays dismissed.
- Include a bucketed budget-spent percent in budget_at_risk metadata so the
  body refreshes when overall usage moves >=10 points even if the same
  categories remain flagged.
- Pass family to chat_response so LLM narration is recorded in llm_usages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): address CodeRabbit review

- Skip the mark-as-read write for Turbo hover-prefetch requests
  (X-Sec-Purpose) so unread badges don't clear before a real visit
- Show the New pill on the dashboard feed (active = unread there; the
  feed never marks insights read)
- Filter idle accounts in SQL instead of a per-account exists? loop
- Eager-load merchants in the subscription audit query
- Widen the advisory-lock key to the signed-bigint range and log when
  acquisition fails so a skipped nightly run is observable
- Mirror the dedup_key unique index as a model validation
- Assert the refresh action enqueues for the signed-in family

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): clear stale lifecycle timestamps on reactivation

When an insight resurfaces, the row now leaves no contradictory state
behind: the material-change path clears both read_at and dismissed_at,
and the expired-recovery path clears read_at. Tests assert the contract,
and the dashboard feed test now also locks in the unread "New" badge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): drop redundant standalone family_id index

Every composite index on insights already leads with family_id, so the
auto-created single-column index was pure write overhead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): stop nightly drift from resurrecting dismissed insights

Three generators stored continuously drifting values (projected amounts,
starting balance, current net worth) in metadata, and any metadata diff
counts as a material change: the body is rewritten (an LLM call when a
provider is configured) and read/dismissed state is cleared. Dismissing a
cash-flow warning was undone at the next 6:00 run.

Metadata now carries only the signal identity plus a coarse bucket, the
same damping the budget generators already use; display values live in
facts. Also orders the idle-cash pick by balance so the selected accounts
don't flip between runs, and documents the dollar-scale threshold
assumption on the relevant constants.

* fix(ds): map DS::Button aria_label into the aria hash

A bare aria_label: option reaches the tag helpers as a literal aria_label
attribute (Rails only dasherizes the nested aria: hash), so the icon-only
fallback overrode it and screen readers announced the insight dismiss
button and the popover trigger as "X".

* fix(insights): gate LLM narration behind AI consent

The nightly job runs unprompted for every family, so narration now
requires someone in the family to have AI enabled: consent to share
financial facts with the provider, and a cost cap in managed mode. The
template fallback keeps behavior identical otherwise. Narration failures
are captured via DebugLogEntry so support can see them.

Adds BodyWriter coverage, including a template-interpolation test for
every generator template key.

* feat(insights): rework the dashboard feed

The feed rendered full insight cards inside the section shell — the only
widget nesting card-on-card — and appended below the fold for every
family with a saved section order. It now mirrors the outflows and
balance-sheet list idiom (inset well with an uppercase mini-header, white
row block, 28px sentiment-tinted icon circles via color-mix on the DS
CSS variables, right-aligned key figures) and leads the dashboard for
saved orders that predate it.

Icon color comes from sentiment, not priority — a savings-rate
improvement is high priority AND good news, and must not render red; red
is reserved for a projected-negative balance. Rows have no hover wash
(cursor plus a gentle icon scale, like the sibling widgets), links to
/insights disable Turbo prefetch so the mark-as-read actually fires for
mouse users, and the standard widget-size popover offers Half/Full. Full
stays the default: the feed is far shorter than any other single-width
widget, so a half default leaves a grid hole the masonry cannot backfill.

* feat(insights): actionable cards, dismiss undo, live refresh

Each row now persists its display facts (new jsonb column) alongside the
change-detection metadata. Facts refresh every run without touching the
body or user state, which is exactly why they are not part of the
material-change comparison.

The card gains what the stored data always supported: a type-and-period
meta line replacing "x minutes ago", a right-aligned key figure (green
only for good news), and a contextual link resolved from the subject ids
in metadata — category, account, recurring transaction, budget month —
omitted when the subject no longer exists.

Dismiss is forgiving: a toast in the notification tray offers undo, and
undismissing restores the row as read rather than re-badging it.
Milestone insights can never regenerate once dismissed, so this closes a
real loss path. Upsert failures are captured via DebugLogEntry.

Manual refresh gets feedback: the button swaps to a disabled checking
state, the page subscribes to a family-scoped stream, and the job
broadcasts the refreshed list and the idle button when it finishes (also
after a lock-skipped run, so the button cannot stay stuck).

Savings copy is sign-aware: a negative rate reads "you spent more than
you earned" with true minus signs. The empty state swaps Lucide sparkles
for the brand assistant glyph (DS::EmptyState learns icon_custom:).

* feat(insights): top-bar entry with unread count

The dashboard feed hides at zero insights and nothing else linked to
/insights, so the manual refresh (and the empty state) were unreachable
for exactly the families who need them: fresh setups before the first
6:00 run. The sticky top bar travels to every screen and its right
cluster had room, so insights get an icon entry there with a monochrome
unread badge. Prefetch is disabled on the link for the same
mark-as-read reason as the feed.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Guillem Arias <accounts@gariasf.com>
2026-07-14 02:19:39 +02:00

98 lines
3.8 KiB
Ruby

# Flags parent categories whose current-month spending pace deviates from
# their average over the previous three full months.
class Insight::Generators::SpendingAnomalyGenerator < Insight::Generator
produces "spending_anomaly"
MIN_BASELINE = 50 # ignore categories with a negligible baseline
MIN_ELAPSED_DAYS = 7 # too early in the month = too noisy to project
DEVIATION_THRESHOLD_PCT = 25
HIGH_PRIORITY_PCT = 50
BASELINE_MONTHS = 3
MAX_INSIGHTS = 3
def generate
period = Period.current_month_for(family)
elapsed_days = (Date.current - period.start_date).to_i + 1
return [] if elapsed_days < MIN_ELAPSED_DAYS
current = category_spend(period)
return [] if current.empty?
baseline = baseline_spend(period)
pace_factor = period.days.to_f / elapsed_days
anomalies = current.filter_map do |category_id, data|
baseline_amount = baseline[category_id]
next unless baseline_amount && baseline_amount >= MIN_BASELINE
projected = data[:total] * pace_factor
deviation_pct = (projected - baseline_amount) / baseline_amount * 100
next if deviation_pct.abs < DEVIATION_THRESHOLD_PCT
{ category_id: category_id, name: data[:name], projected: projected,
baseline: baseline_amount, deviation_pct: deviation_pct }
end
anomalies
.sort_by { |a| -a[:deviation_pct].abs }
.first(MAX_INSIGHTS)
.map { |anomaly| anomaly_insight(anomaly, period) }
end
private
def anomaly_insight(anomaly, period)
direction = anomaly[:deviation_pct].positive? ? "above" : "below"
build_insight(
insight_type: "spending_anomaly",
priority: anomaly[:deviation_pct].abs >= HIGH_PRIORITY_PCT ? "high" : "medium",
title: I18n.t("insights.titles.spending_anomaly.#{direction}", category: anomaly[:name]),
template_key: "spending_anomaly.#{direction}",
facts: {
category: anomaly[:name],
deviation_pct: round(anomaly[:deviation_pct].abs, 0).to_i,
projected_spend: format_money(anomaly[:projected]),
baseline_spend: format_money(anomaly[:baseline])
},
# The projection moves every night by construction (spend accrues and
# the pace factor shrinks as the month elapses), so exact amounts here
# would rewrite the body and resurrect dismissals nightly. Bucket the
# deviation instead; the display numbers live in `facts` only.
metadata: {
category_id: anomaly[:category_id],
direction: direction,
deviation_bucket: (round(anomaly[:deviation_pct].abs, 0).to_i / 25) * 25
},
period: period,
dedup_key: "spending_anomaly:#{anomaly[:category_id]}:#{month_token(period.start_date)}"
)
end
# { category_id => { name:, total: } } for persisted parent categories only.
# Subcategory spend is already rolled into its parent's total, and synthetic
# categories (uncategorized / other investments) are too noisy to flag.
def category_spend(period)
income_statement.expense_totals(period: period).category_totals.each_with_object({}) do |ct, totals|
next if ct.category.synthetic? || ct.category.parent_id.present?
next unless ct.total.positive?
totals[ct.category.id] = { name: ct.category.name, total: ct.total.to_d }
end
end
def baseline_spend(current_period)
sums = Hash.new { |h, k| h[k] = 0.to_d }
BASELINE_MONTHS.times do |i|
start_date = current_period.start_date - (i + 1).months
month = Period.custom(start_date: start_date, end_date: start_date + 1.month - 1.day)
category_spend(month).each do |category_id, data|
sums[category_id] += data[:total]
end
end
sums.transform_values { |total| total / BASELINE_MONTHS }
end
end