Files
sure/app/controllers/pages_controller.rb
Juan José Mata 66cf9e7f0b feat(insights): proactive financial intelligence feed (#2550)
* feat(insights): proactive financial intelligence feed

Adds a nightly job that analyzes each family's finances in pure Ruby and
surfaces typed, stateful insights on the dashboard and a new /insights page.

- Insight model (active/read/dismissed) with a per-family dedup_key unique
  index so nightly re-runs refresh rows instead of duplicating them
- Seven generators (spending anomaly, cash-flow warning, net worth milestone,
  subscription audit, savings rate change, idle cash, budget health) built on
  IncomeStatement, BalanceSheet, RecurringTransaction, and BudgetCategory
- LLM used as a writer, not a reasoner: Insight::BodyWriter narrates
  pre-computed facts via the configured provider, with an i18n template
  fallback so self-hosted installs without API keys work identically; bodies
  are only (re)written when an insight is new or its numbers changed
- GenerateInsightsJob: cron fan-out per family, per-family advisory lock,
  metadata-diff upsert that preserves read/dismissed state for unchanged
  signals and reactivates on material change
- Dashboard insights_feed section (top 3, collapsible/reorderable) and an
  /insights feed page with turbo-stream dismissal; viewing the feed marks
  insights read
- Tests for the model, job upsert semantics, and controller flows

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): address Codex review — expiry, budget metadata, LLM usage

- Expire visible insights whose condition cleared: generators declare the
  insight types they produce, the registry reports which generators ran to
  completion, and the job expires visible insights of those types whose
  dedup_key was not regenerated. A crashing generator can't wipe out its
  healthy insights, and an expired insight reactivates when its condition
  returns — unlike a user-dismissed one, which stays dismissed.
- Include a bucketed budget-spent percent in budget_at_risk metadata so the
  body refreshes when overall usage moves >=10 points even if the same
  categories remain flagged.
- Pass family to chat_response so LLM narration is recorded in llm_usages.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): address CodeRabbit review

- Skip the mark-as-read write for Turbo hover-prefetch requests
  (X-Sec-Purpose) so unread badges don't clear before a real visit
- Show the New pill on the dashboard feed (active = unread there; the
  feed never marks insights read)
- Filter idle accounts in SQL instead of a per-account exists? loop
- Eager-load merchants in the subscription audit query
- Widen the advisory-lock key to the signed-bigint range and log when
  acquisition fails so a skipped nightly run is observable
- Mirror the dedup_key unique index as a model validation
- Assert the refresh action enqueues for the signed-in family

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): clear stale lifecycle timestamps on reactivation

When an insight resurfaces, the row now leaves no contradictory state
behind: the material-change path clears both read_at and dismissed_at,
and the expired-recovery path clears read_at. Tests assert the contract,
and the dashboard feed test now also locks in the unread "New" badge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): drop redundant standalone family_id index

Every composite index on insights already leads with family_id, so the
auto-created single-column index was pure write overhead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* fix(insights): stop nightly drift from resurrecting dismissed insights

Three generators stored continuously drifting values (projected amounts,
starting balance, current net worth) in metadata, and any metadata diff
counts as a material change: the body is rewritten (an LLM call when a
provider is configured) and read/dismissed state is cleared. Dismissing a
cash-flow warning was undone at the next 6:00 run.

Metadata now carries only the signal identity plus a coarse bucket, the
same damping the budget generators already use; display values live in
facts. Also orders the idle-cash pick by balance so the selected accounts
don't flip between runs, and documents the dollar-scale threshold
assumption on the relevant constants.

* fix(ds): map DS::Button aria_label into the aria hash

A bare aria_label: option reaches the tag helpers as a literal aria_label
attribute (Rails only dasherizes the nested aria: hash), so the icon-only
fallback overrode it and screen readers announced the insight dismiss
button and the popover trigger as "X".

* fix(insights): gate LLM narration behind AI consent

The nightly job runs unprompted for every family, so narration now
requires someone in the family to have AI enabled: consent to share
financial facts with the provider, and a cost cap in managed mode. The
template fallback keeps behavior identical otherwise. Narration failures
are captured via DebugLogEntry so support can see them.

Adds BodyWriter coverage, including a template-interpolation test for
every generator template key.

* feat(insights): rework the dashboard feed

The feed rendered full insight cards inside the section shell — the only
widget nesting card-on-card — and appended below the fold for every
family with a saved section order. It now mirrors the outflows and
balance-sheet list idiom (inset well with an uppercase mini-header, white
row block, 28px sentiment-tinted icon circles via color-mix on the DS
CSS variables, right-aligned key figures) and leads the dashboard for
saved orders that predate it.

Icon color comes from sentiment, not priority — a savings-rate
improvement is high priority AND good news, and must not render red; red
is reserved for a projected-negative balance. Rows have no hover wash
(cursor plus a gentle icon scale, like the sibling widgets), links to
/insights disable Turbo prefetch so the mark-as-read actually fires for
mouse users, and the standard widget-size popover offers Half/Full. Full
stays the default: the feed is far shorter than any other single-width
widget, so a half default leaves a grid hole the masonry cannot backfill.

* feat(insights): actionable cards, dismiss undo, live refresh

Each row now persists its display facts (new jsonb column) alongside the
change-detection metadata. Facts refresh every run without touching the
body or user state, which is exactly why they are not part of the
material-change comparison.

The card gains what the stored data always supported: a type-and-period
meta line replacing "x minutes ago", a right-aligned key figure (green
only for good news), and a contextual link resolved from the subject ids
in metadata — category, account, recurring transaction, budget month —
omitted when the subject no longer exists.

Dismiss is forgiving: a toast in the notification tray offers undo, and
undismissing restores the row as read rather than re-badging it.
Milestone insights can never regenerate once dismissed, so this closes a
real loss path. Upsert failures are captured via DebugLogEntry.

Manual refresh gets feedback: the button swaps to a disabled checking
state, the page subscribes to a family-scoped stream, and the job
broadcasts the refreshed list and the idle button when it finishes (also
after a lock-skipped run, so the button cannot stay stuck).

Savings copy is sign-aware: a negative rate reads "you spent more than
you earned" with true minus signs. The empty state swaps Lucide sparkles
for the brand assistant glyph (DS::EmptyState learns icon_custom:).

* feat(insights): top-bar entry with unread count

The dashboard feed hides at zero insights and nothing else linked to
/insights, so the manual refresh (and the empty state) were unreachable
for exactly the families who need them: fresh setups before the first
6:00 run. The sticky top bar travels to every screen and its right
cluster had room, so insights get an icon entry there with a monochrome
unread badge. Prefetch is disabled on the link for the same
mark-as-read reason as the feed.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Guillem Arias <accounts@gariasf.com>
2026-07-14 02:19:39 +02:00

409 lines
16 KiB
Ruby

class PagesController < ApplicationController
include Periodable
# Per-widget dashboard layout guardrails. Deterministic defaults the masonry
# packer reads; users may override a grow widget's height via presets.
# col_span: "single" | "full" (full spans both columns in 2-col mode)
# grow: true for charts that should fill an allotted height,
# false for content-sized widgets (tables, stat grids)
# min_height: floor in px
DASHBOARD_SECTION_LAYOUTS = {
# Width-toggleable but full by default: the feed is much shorter than any
# other single-width widget, so defaulting to half leaves a grid hole the
# masonry can't backfill (dense placement needs a later card short enough
# to fit beside it, and none is). Users who pair it manually can go half.
"insights_feed" => { col_span: "full", grow: false, min_height: 0, width_toggle: true },
"cashflow_sankey" => { col_span: "full", grow: false, min_height: 384, width_toggle: true },
"outflows_donut" => { col_span: "single", grow: false, min_height: 0 },
"investment_summary" => { col_span: "single", grow: false, min_height: 0, width_toggle: true },
"net_worth_chart" => { col_span: "single", grow: true, min_height: 208, width_toggle: true },
"balance_sheet" => { col_span: "single", grow: false, min_height: 0, width_toggle: true }
}.freeze
# Selectable height presets (px) for grow widgets.
DASHBOARD_HEIGHT_PRESETS = { "compact" => 208, "auto" => 288, "tall" => 416 }.freeze
DEFAULT_HEIGHT_PRESET = "auto"
skip_authentication only: %i[redis_configuration_error privacy terms]
before_action :ensure_intro_guest!, only: :intro
def dashboard
if Current.user&.ui_layout_intro?
redirect_to chats_path and return
end
@balance_sheet = Current.family.balance_sheet
@investment_statement = Current.family.investment_statement
@accounts = Current.user.accessible_accounts.visible.with_attached_logo
family_currency = Current.family.currency
# Use IncomeStatement for all cashflow data (now includes categorized trades)
income_statement = Current.family.income_statement
income_totals = income_statement.income_totals(period: @period)
expense_totals = income_statement.expense_totals(period: @period)
net_totals = income_statement.net_category_totals(period: @period)
@cashflow_sankey_data = build_cashflow_sankey_data(net_totals, income_totals, expense_totals, family_currency)
@outflows_data = build_outflows_donut_data(net_totals)
@feed_insights = Current.family.insights.visible.ordered.limit(3)
@dashboard_sections = build_dashboard_sections
@breadcrumbs = [ [ t("breadcrumbs.home"), root_path ], [ t("breadcrumbs.dashboard"), nil ] ]
end
def intro
@breadcrumbs = [ [ t("breadcrumbs.home"), chats_path ], [ t("breadcrumbs.intro"), nil ] ]
end
def update_preferences
if Current.user.update_dashboard_preferences(preferences_params)
head :ok
else
head :unprocessable_entity
end
end
def changelog
@release_notes = github_provider.fetch_latest_release_notes
# Fallback if no release notes are available
if @release_notes.nil?
@release_notes = {
avatar: "https://github.com/we-promise.png",
username: "we-promise",
name: t("pages.release_notes_unavailable.name"),
published_at: Date.current,
body: t("pages.release_notes_unavailable.body_html")
}
end
render layout: "settings"
end
def feedback
render layout: "settings"
end
def redis_configuration_error
render layout: "blank"
end
def privacy
render layout: "blank"
end
def terms
render layout: "blank"
end
private
def preferences_params
prefs = params.require(:preferences)
{}.tap do |permitted|
permitted["collapsed_sections"] = prefs[:collapsed_sections].to_unsafe_h if prefs[:collapsed_sections].respond_to?(:to_unsafe_h)
permitted["section_order"] = prefs[:section_order] if prefs[:section_order].is_a?(Array)
permitted["dashboard_section_layout"] = prefs[:dashboard_section_layout].to_unsafe_h if prefs[:dashboard_section_layout].respond_to?(:to_unsafe_h)
end
end
def build_dashboard_sections
all_sections = [
{
key: "insights_feed",
title: "pages.dashboard.insights_feed.title",
partial: "pages/dashboard/insights_feed",
layout: section_layout("insights_feed"),
locals: { insights: @feed_insights },
visible: @feed_insights.any?,
collapsible: true
},
{
key: "cashflow_sankey",
title: "pages.dashboard.cashflow_sankey.title",
partial: "pages/dashboard/cashflow_sankey",
layout: section_layout("cashflow_sankey"),
locals: { sankey_data: @cashflow_sankey_data, period: @period },
visible: @accounts.any?,
collapsible: true
},
{
key: "outflows_donut",
title: "pages.dashboard.outflows_donut.title",
partial: "pages/dashboard/outflows_donut",
layout: section_layout("outflows_donut"),
locals: { outflows_data: @outflows_data, period: @period },
visible: @accounts.any? && @outflows_data[:categories].present?,
collapsible: true
},
{
key: "investment_summary",
title: "pages.dashboard.investment_summary.title",
partial: "pages/dashboard/investment_summary",
layout: section_layout("investment_summary"),
locals: { investment_statement: @investment_statement, period: @period },
visible: @accounts.any? && @investment_statement.investment_accounts.any?,
collapsible: true
},
{
key: "net_worth_chart",
title: "pages.dashboard.net_worth_chart.title",
partial: "pages/dashboard/net_worth_chart",
layout: section_layout("net_worth_chart"),
locals: { balance_sheet: @balance_sheet, period: @period },
visible: @accounts.any?,
collapsible: true
},
{
key: "balance_sheet",
title: "pages.dashboard.balance_sheet.title",
partial: "pages/dashboard/balance_sheet",
layout: section_layout("balance_sheet"),
locals: { balance_sheet: @balance_sheet },
visible: @accounts.any?,
collapsible: true
}
]
# Order sections according to user preference
section_order = Current.user.dashboard_section_order
ordered_sections = section_order.map do |key|
all_sections.find { |s| s[:key] == key }
end.compact
# Add any new sections that aren't in the saved order (future-proofing).
# The insights feed leads instead of appending: it's a proactive surface,
# and appending would bury it below the fold for every family with a
# saved order. Users can still drag it back down — that choice persists.
all_sections.each do |section|
next if ordered_sections.include?(section)
if section[:key] == "insights_feed"
ordered_sections.unshift(section)
else
ordered_sections << section
end
end
ordered_sections
end
# Resolves a section's layout guardrails, applying the user's height preset
# override (falling back to the deterministic default) for grow widgets.
def section_layout(key)
base = DASHBOARD_SECTION_LAYOUTS.fetch(key, { col_span: "single", grow: false, min_height: 0, width_toggle: false })
preset = Current.user.dashboard_section_height(key)
preset = DEFAULT_HEIGHT_PRESET unless DASHBOARD_HEIGHT_PRESETS.key?(preset)
col_span = base[:col_span]
if base[:width_toggle]
user_span = Current.user.dashboard_section_width(key)
col_span = user_span if %w[single full].include?(user_span)
end
base.merge(col_span: col_span, height_preset: preset, height_px: DASHBOARD_HEIGHT_PRESETS.fetch(preset))
end
def github_provider
Provider::Registry.get_provider(:github)
end
def build_cashflow_sankey_data(net_totals, income_totals, expense_totals, currency)
nodes = []
links = []
node_indices = {}
add_node = ->(unique_key, display_name, value, percentage, color) {
node_indices[unique_key] ||= begin
nodes << { id: unique_key, name: display_name, value: value.to_f.round(2), percentage: percentage.to_f.round(1), color: color }
nodes.size - 1
end
}
total_income = net_totals.total_net_income.to_f.round(2)
total_expense = net_totals.total_net_expense.to_f.round(2)
# Central Cash Flow node
cash_flow_idx = add_node.call("cash_flow_node", "Cash Flow", total_income, 100.0, "var(--color-success)")
# Build netted subcategory data from raw totals
net_subcategories_by_parent = build_net_subcategories(expense_totals, income_totals)
# Process net income categories (flow: subcategory -> parent -> cash_flow)
process_net_category_nodes(
categories: net_totals.net_income_categories,
total: total_income,
prefix: "income",
net_subcategories_by_parent: net_subcategories_by_parent,
add_node: add_node,
links: links,
cash_flow_idx: cash_flow_idx,
flow_direction: :inbound
)
# Process net expense categories (flow: cash_flow -> parent -> subcategory)
process_net_category_nodes(
categories: net_totals.net_expense_categories,
total: total_expense,
prefix: "expense",
net_subcategories_by_parent: net_subcategories_by_parent,
add_node: add_node,
links: links,
cash_flow_idx: cash_flow_idx,
flow_direction: :outbound
)
# Surplus/Deficit
net = (total_income - total_expense).round(2)
if net.positive?
percentage = total_income.zero? ? 0 : (net / total_income * 100).round(1)
idx = add_node.call("surplus_node", "Surplus", net, percentage, "var(--color-success)")
links << { source: cash_flow_idx, target: idx, value: net, color: "var(--color-success)", percentage: percentage }
end
{ nodes: nodes, links: links, currency_symbol: Money::Currency.new(currency).symbol }
end
# Nets subcategory expense and income totals, grouped by parent_id.
# Returns { parent_id => [ { category:, total: net_amount }, ... ] }
# Only includes subcategories with positive net (same direction as parent).
def build_net_subcategories(expense_totals, income_totals)
expense_subs = expense_totals.category_totals
.select { |ct| ct.category.parent_id.present? }
.index_by { |ct| ct.category.id }
income_subs = income_totals.category_totals
.select { |ct| ct.category.parent_id.present? }
.index_by { |ct| ct.category.id }
all_sub_ids = (expense_subs.keys + income_subs.keys).uniq
result = {}
all_sub_ids.each do |sub_id|
exp_ct = expense_subs[sub_id]
inc_ct = income_subs[sub_id]
exp_total = exp_ct&.total || 0
inc_total = inc_ct&.total || 0
net = exp_total - inc_total
category = exp_ct&.category || inc_ct&.category
next if net.zero?
parent_id = category.parent_id
result[parent_id] ||= []
result[parent_id] << { category: category, total: net.abs, net_direction: net > 0 ? :expense : :income }
end
result
end
# Builds sankey nodes/links for net categories with subcategory hierarchy.
# Subcategories matching the parent's flow direction are shown as children.
# Subcategories with opposite net direction appear on the OTHER side of the
# sankey (handled when the other side calls this method).
#
# flow_direction: :inbound (subcategory -> parent -> cash_flow) for income
# :outbound (cash_flow -> parent -> subcategory) for expenses
def process_net_category_nodes(categories:, total:, prefix:, net_subcategories_by_parent:, add_node:, links:, cash_flow_idx:, flow_direction:)
matching_direction = flow_direction == :inbound ? :income : :expense
categories.each do |ct|
val = ct.total.to_f.round(2)
next if val.zero?
percentage = total.zero? ? 0 : (val / total * 100).round(1)
color = ct.category.color.presence || Category::UNCATEGORIZED_COLOR
node_key = "#{prefix}_#{ct.category.id || ct.category.name}"
all_subs = ct.category.id ? (net_subcategories_by_parent[ct.category.id] || []) : []
same_side_subs = all_subs.select { |s| s[:net_direction] == matching_direction }
# Also check if any subcategory has opposite direction — those will be
# rendered by the OTHER side's call to this method, linked to cash_flow
# directly (they appear as independent nodes on the opposite side).
opposite_subs = all_subs.select { |s| s[:net_direction] != matching_direction }
if same_side_subs.any?
parent_idx = add_node.call(node_key, ct.category.name, val, percentage, color)
if flow_direction == :inbound
links << { source: parent_idx, target: cash_flow_idx, value: val, color: color, percentage: percentage }
else
links << { source: cash_flow_idx, target: parent_idx, value: val, color: color, percentage: percentage }
end
same_side_subs.each do |sub|
sub_val = sub[:total].to_f.round(2)
sub_pct = val.zero? ? 0 : (sub_val / val * 100).round(1)
sub_color = sub[:category].color.presence || color
sub_key = "#{prefix}_sub_#{sub[:category].id}"
sub_idx = add_node.call(sub_key, sub[:category].name, sub_val, sub_pct, sub_color)
if flow_direction == :inbound
links << { source: sub_idx, target: parent_idx, value: sub_val, color: sub_color, percentage: sub_pct }
else
links << { source: parent_idx, target: sub_idx, value: sub_val, color: sub_color, percentage: sub_pct }
end
end
else
idx = add_node.call(node_key, ct.category.name, val, percentage, color)
if flow_direction == :inbound
links << { source: idx, target: cash_flow_idx, value: val, color: color, percentage: percentage }
else
links << { source: cash_flow_idx, target: idx, value: val, color: color, percentage: percentage }
end
end
# Render opposite-direction subcategories as standalone nodes on this side,
# linked directly to cash_flow. They represent subcategory surplus/deficit
# that goes against the parent's overall direction.
opposite_prefix = flow_direction == :inbound ? "expense" : "income"
opposite_subs.each do |sub|
sub_val = sub[:total].to_f.round(2)
sub_pct = total.zero? ? 0 : (sub_val / total * 100).round(1)
sub_color = sub[:category].color.presence || color
sub_key = "#{opposite_prefix}_sub_#{sub[:category].id}"
sub_idx = add_node.call(sub_key, sub[:category].name, sub_val, sub_pct, sub_color)
# Opposite direction: if parent is outbound (expense), this sub is inbound (income)
if flow_direction == :inbound
links << { source: cash_flow_idx, target: sub_idx, value: sub_val, color: sub_color, percentage: sub_pct }
else
links << { source: sub_idx, target: cash_flow_idx, value: sub_val, color: sub_color, percentage: sub_pct }
end
end
end
end
def build_outflows_donut_data(net_totals)
currency_symbol = Money::Currency.new(net_totals.currency).symbol
total = net_totals.total_net_expense
categories = net_totals.net_expense_categories
.reject { |ct| ct.total.zero? }
.sort_by { |ct| -ct.total }
.map do |ct|
{
id: ct.category.id,
name: ct.category.name,
amount: ct.total.to_f.round(2),
currency: ct.currency,
percentage: ct.weight.round(1),
color: ct.category.color.presence || Category::UNCATEGORIZED_COLOR,
icon: ct.category.lucide_icon,
clickable: !ct.category.other_investments?
}
end
{ categories: categories, total: total.to_f.round(2), currency: net_totals.currency, currency_symbol: currency_symbol }
end
def ensure_intro_guest!
return if Current.user&.guest?
redirect_to root_path, alert: t("pages.intro.not_authorized", default: "Intro is only available to guest users.")
end
end