feat(insights): gate the insights feed behind preview features (#2788)

* feat(insights): gate the insights feed behind preview features

Insights shipped to everyone in #2550. Make it opt-in via Settings →
Preferences until it's proven, so users who haven't enabled preview
features see nothing and cost nothing.

Entry points gated:
- InsightsController — require_preview_features! covers all four actions,
  including the refresh action that enqueues the job
- Dashboard — the insights_feed section is omitted from the section list
  rather than left in it hidden, so the saved-order lookup and the
  insights_feed unshift special-case never fire; the feed query is skipped
- Top bar — the lightbulb entry and its unread COUNT, which previously ran
  on every page render

The job is gated too, departing from the guide's default that background
jobs keep running. That default fits a job like SweepExpiredGoalPledgesJob,
which only walks records opted-in users created and is naturally inert.
GenerateInsightsJob instead manufactures data for every family nightly —
seven generators over the income statement and balance sheet, plus paid LLM
narration — so it would have kept spending on families who can't see the
result. The fan-out filters with Family.with_preview_features (one indexed
jsonb containment query, not load-and-iterate), and generate_for_family
re-checks above the advisory lock so a gated family skips the broadcast too.

Adds Family#preview_features_enabled? and the matching scopes, keeping the
predicate name identical on User and Family so the guide's GA-removal grep
finds every call site. Verified the SQL scope and the Ruby predicate agree
for true / false / "yes" / nil. Documents the job-gating pattern in
docs/llm-guides/gating-a-preview-feature.md, which previously said the gate
does nothing for jobs.

Existing insight rows are left alone: invisible without the flag, and the
next nightly run refreshes facts and expires anything stale if a family
opts in.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* perf(insights): use EXISTS for the family preview rollup

Family#preview_features_enabled? is asked once per family by the nightly
job; the block form loaded and instantiated every member to answer a
boolean. Delegate to the scope instead.

The predicate now shares an implementation with the scope, so the
truthy-non-boolean test asserts against User#preview_features_enabled? —
the predicate the UI actually gates on — to keep the cross-check
meaningful rather than tautological.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

* docs(insights): fix guide/code drift and stale cron description

Review follow-ups from @gariasf:

- The guide's family-rollup snippet still showed the block form after the
  EXISTS commit changed it. It mattered more than normal doc drift: the
  paragraph below calls GenerateInsightsJob "the reference implementation",
  so the next person writing a gated job would have copied the form
  family.rb's comment explicitly rejects.
- schedule.yml still described the job as running for "all families" — the
  string someone reads while debugging why a family got no insights.
- Document that the shared predicate name is per-user on User but
  "anyone in the household" on Family, and prohibit gating UI on the family
  form: Current.family.preview_features_enabled? reads naturally and would
  show the feature to a user who explicitly opted out. Noted in both the
  model and the guide.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y6ZRA6cgCRM4UdFKct3wm4

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Guillem Arias Fauste <accounts@gariasf.com>

app/controllers/insights_controller.rb

@@ -1,4 +1,5 @@
 class InsightsController < ApplicationController
+  before_action :require_preview_features!
   before_action :set_insight, only: %i[dismiss undismiss]
 
   def index

app/controllers/pages_controller.rb

@@ -51,7 +51,9 @@ class PagesController < ApplicationController
 
     @cashflow_sankey_data = build_cashflow_sankey_data(net_totals, income_totals, expense_totals, family_currency)
     @outflows_data = build_outflows_donut_data(net_totals)
-    @feed_insights = Current.family.insights.visible.ordered.limit(3)
+    # Preview-gated: skip the query outright rather than loading rows the
+    # section won't be built from.
+    @feed_insights = preview_features_enabled? ? Current.family.insights.visible.ordered.limit(3) : Insight.none
 
     @money_flow_accounts = income_statement.eligible_accounts
     @money_flow_month = money_flow_month_param
@@ -118,17 +120,27 @@ class PagesController < ApplicationController
       end
     end
 
+    # Preview-gated, and omitted from the section list entirely rather than
+    # left in it with `visible: false`. Dropping it here means the two
+    # downstream behaviors fall out for free: the saved-order lookup finds
+    # nothing to map, and the insights_feed unshift special-case never fires.
+    def insights_feed_section
+      return nil unless preview_features_enabled?
+
+      {
+        key: "insights_feed",
+        title: "pages.dashboard.insights_feed.title",
+        partial: "pages/dashboard/insights_feed",
+        layout: section_layout("insights_feed"),
+        locals: { insights: @feed_insights },
+        visible: @feed_insights.any?,
+        collapsible: true
+      }
+    end
+
     def build_dashboard_sections
       all_sections = [
-        {
-          key: "insights_feed",
-          title: "pages.dashboard.insights_feed.title",
-          partial: "pages/dashboard/insights_feed",
-          layout: section_layout("insights_feed"),
-          locals: { insights: @feed_insights },
-          visible: @feed_insights.any?,
-          collapsible: true
-        },
+        insights_feed_section,
         {
           key: "cashflow_sankey",
           title: "pages.dashboard.cashflow_sankey.title",
@@ -183,7 +195,7 @@ class PagesController < ApplicationController
           visible: @accounts.any?,
           collapsible: true
         }
-      ]
+      ].compact
 
       # Order sections according to user preference
       section_order = Current.user.dashboard_section_order

app/jobs/generate_insights_job.rb

@@ -13,8 +13,14 @@ class GenerateInsightsJob < ApplicationJob
   end
 
   private
+    # Insights are a preview feature, so the nightly sweep only visits families
+    # with an opted-in member. Unlike a job that moves existing data around,
+    # this one manufactures data per family — seven generators over the income
+    # statement and balance sheet, plus paid LLM narration — so running it for
+    # families who can't see the result is pure waste. Scoped in SQL rather
+    # than checked per family to keep the fan-out a single indexed query.
     def fan_out
-      Family.find_each do |family|
+      Family.with_preview_features.find_each do |family|
         GenerateInsightsJob.perform_later(family_id: family.id)
       rescue => e
         Rails.logger.error("Failed to enqueue insight generation for family #{family.id}: #{e.message}")
@@ -25,6 +31,11 @@ class GenerateInsightsJob < ApplicationJob
       family = Family.find_by(id: family_id)
       return unless family
       return if family.accounts.none?
+      # Also checked here, not just at the fan-out: this path is reachable
+      # directly via perform_later(family_id:) from the refresh action and the
+      # console. Returning above the lock means a gated family skips the
+      # broadcast below too, not just the generation.
+      return unless family.preview_features_enabled?
 
       with_advisory_lock(family_id) do
         I18n.with_locale(family.locale) do

app/models/family.rb

@@ -116,6 +116,27 @@ class Family < ApplicationRecord
   has_many :recurring_transactions, dependent: :destroy
   has_many :insights, dependent: :destroy
 
+  # Families with at least one opted-in member. Lets a job filter in one
+  # indexed query rather than loading every family and asking each in Ruby.
+  scope :with_preview_features, -> { where(id: User.with_preview_features.select(:family_id)) }
+
+  # Family-level rollup of the per-user preview flag, for callers that run
+  # without a Current.user (the nightly insights job). Preview access is a
+  # personal preference but the data it produces is family-scoped, so one
+  # opted-in member is enough to generate for the family.
+  #
+  # EXISTS rather than `users.any?(&:preview_features_enabled?)`: the job asks
+  # this once per family, and the block form would load and instantiate every
+  # member just to answer a boolean.
+  #
+  # Never gate UI on this — visibility is per-user, and this answers "somebody
+  # in the household opted in", so a view using it would show the feature to a
+  # user who explicitly opted out. Use the PreviewGateable helper (Current.user)
+  # for anything a person sees.
+  def preview_features_enabled?
+    users.with_preview_features.exists?
+  end
+
   validates :locale, inclusion: { in: I18n.available_locales.map(&:to_s) }
   validates :date_format, inclusion: { in: DATE_FORMATS.map(&:last) }
   validates :month_start_day, inclusion: { in: 1..28 }

app/models/user.rb

@@ -56,6 +56,16 @@ class User < ApplicationRecord
   normalizes :first_name, :last_name, with: ->(value) { value.strip.presence }
 
   enum :role, { guest: "guest", member: "member", admin: "admin", super_admin: "super_admin" }, validate: true
+
+  # SQL counterpart to #preview_features_enabled?, for callers that filter
+  # users (or their families) in one query instead of loading and iterating.
+  # The `@>` containment operator uses index_users_on_preferences (GIN) and
+  # matches only a JSON boolean true, so it agrees with that predicate's
+  # strict `== true` — a stray "yes" enables neither.
+  scope :with_preview_features, -> {
+    where("preferences @> ?", { preview_features_enabled: true }.to_json)
+  }
+
   attribute :ui_layout, :string
   enum :ui_layout, { dashboard: "dashboard", intro: "intro" }, validate: true, prefix: true
 

app/views/insights/index.html.erb

@@ -3,7 +3,10 @@
 <div class="space-y-6 pb-6">
   <header class="flex items-start justify-between gap-4">
     <div>
-      <h1 class="text-2xl font-semibold text-primary"><%= t(".title") %></h1>
+      <div class="flex items-center gap-2">
+        <h1 class="text-2xl font-semibold text-primary"><%= t(".title") %></h1>
+        <%= render DS::Pill.new(label: t("shared.preview"), size: :md) %>
+      </div>
       <p class="text-sm text-secondary mt-1"><%= t(".subtitle") %></p>
     </div>
 

app/views/layouts/application.html.erb

@@ -159,7 +159,11 @@ end %>
           </div>
 
           <div class="flex items-center gap-2">
-            <% if Current.family %>
+            <%# Preview-gated. Gating the whole block (rather than just the
+                link) also keeps the unread COUNT off every page render for
+                users who haven't opted in. No preview dot on the icon: the
+                unread badge already occupies that corner. %>
+            <% if Current.family && preview_features_enabled? %>
               <% unread_insights_count = Current.family.insights.active.count %>
               <%# Prefetch disabled: /insights marks insights read on real visits
                   and deliberately skips prefetch requests, so a prefetched

app/views/pages/dashboard/_insights_feed.html.erb

@@ -13,11 +13,14 @@
   <div class="bg-container-inset rounded-xl p-1 space-y-1">
     <div class="px-4 py-2 flex items-center justify-between uppercase text-xs font-medium text-secondary">
       <% new_count = insights.count(&:active?) %>
-      <% if new_count.positive? %>
-        <p><%= t("insights.feed.header_new") %><span class="text-subdued mx-1">&middot;</span><%= new_count %></p>
-      <% else %>
-        <p><%= t("insights.feed.header") %></p>
-      <% end %>
+      <div class="flex items-center gap-2">
+        <% if new_count.positive? %>
+          <p><%= t("insights.feed.header_new") %><span class="text-subdued mx-1">&middot;</span><%= new_count %></p>
+        <% else %>
+          <p><%= t("insights.feed.header") %></p>
+        <% end %>
+        <%= render DS::Pill.new(label: t("shared.preview")) %>
+      </div>
 
       <%= render DS::Link.new(
         text: t("insights.feed.view_all"),

config/schedule.yml

@@ -59,7 +59,7 @@ generate_insights:
   cron: "0 6 * * *" # daily at 6:00 AM UTC, after the nightly cleanup jobs
   class: "GenerateInsightsJob"
   queue: "scheduled"
-  description: "Generates proactive financial insights for all families"
+  description: "Generates proactive financial insights for families with preview features enabled"
 
 sweep_expired_goal_pledges:
   cron: "*/15 * * * *" # every 15 minutes

docs/llm-guides/gating-a-preview-feature.md

@@ -140,6 +140,25 @@ Grep for `require_preview_features!` and `preview_features_enabled?` near your f
 
 The flag is per-user, not per-family. Two users in the same family can see different versions of the product if one opts in and the other doesn't. That's intentional. Data is family-scoped, but visibility is a personal preference. If you write a feature that creates family-shared data (goals, budgets, etc.), the data persists when a user toggles preview off. The UI just disappears from their view while still showing up for opted-in family members.
 
-The gate does nothing for background jobs. If your feature has a Sidekiq cron job, it runs regardless of who has preview enabled. That's usually correct (data should keep flowing), but if the job sends notifications or emails, gate those at the send site too.
+The gate does nothing for background jobs on its own — `PreviewGateable` reads `Current.user`, which a Sidekiq worker doesn't have. A cron job runs regardless of who has preview enabled. That's usually correct: data should keep flowing, so it's there when someone opts in. `SweepExpiredGoalPledgesJob` is the typical shape — it only walks pledges that opted-in users created, so it's naturally inert for everyone else and needs no gate.
+
+Gate the job when it does work that isn't free. Two cases: it sends something outward (notifications, emails — gate at the send site), or it *manufactures* data for every family rather than moving existing data around, burning compute or paid API calls per family. `GenerateInsightsJob` is the second case: seven generators over the income statement and balance sheet, plus optional LLM narration, for every family nightly.
+
+For a family-scoped job, roll the per-user flag up to the family:
+
+```ruby
+# app/models/family.rb
+scope :with_preview_features, -> { where(id: User.with_preview_features.select(:family_id)) }
+
+def preview_features_enabled?
+  users.with_preview_features.exists?
+end
+```
+
+Filter the fan-out with the scope (one indexed query — `User.with_preview_features` is a jsonb containment match against the GIN index on `users.preferences`, not a load-and-iterate), and re-check with the predicate inside the per-family path, which is reachable directly from controllers and the console. Keep the same predicate name on both models so the GA-removal grep finds every call site. `GenerateInsightsJob` is the reference implementation.
+
+One opted-in member is enough to generate for the whole family — consistent with the per-user-visibility / family-scoped-data split described above.
+
+Note what the shared name costs: `preview_features_enabled?` means "I opted in" on `User` but "somebody in my household opted in" on `Family`. Only ever gate background work on the family form. Gating UI on it (`Current.family.preview_features_enabled?` reads naturally, which is the trap) would show the feature to someone who explicitly opted out because a family member opted in. For anything a person sees, use the `PreviewGateable` helper, which reads `Current.user`.
 
 The redirect target is `/`. If you want gated controllers to land somewhere else (a docs page, an opt-in nudge), override `require_preview_features!` in the controller, or write a thin custom `before_action` that calls `preview_features_enabled?` directly.

test/controllers/insights_controller_test.rb

@@ -3,6 +3,7 @@ require "test_helper"
 class InsightsControllerTest < ActionDispatch::IntegrationTest
   setup do
     sign_in @user = users(:family_admin)
+    enable_preview_features
     @insight = insights(:spending_anomaly_dining)
     ensure_tailwind_build
   end
@@ -97,4 +98,55 @@ class InsightsControllerTest < ActionDispatch::IntegrationTest
 
     assert_redirected_to insights_path
   end
+
+  # Preview gate. Insights is opt-in via Settings → Preferences, so a user
+  # without the flag reaches none of it — not the page, not the dashboard
+  # section, not the top-bar entry, and not the job the refresh action would
+  # otherwise enqueue.
+  test "redirects users without preview access" do
+    disable_preview_features
+
+    get insights_url
+
+    assert_redirected_to root_path
+    assert_match(/preview/i, flash[:alert])
+  end
+
+  test "refresh does not enqueue generation for users without preview access" do
+    disable_preview_features
+
+    assert_no_enqueued_jobs only: GenerateInsightsJob do
+      post refresh_insights_url
+    end
+
+    assert_redirected_to root_path
+  end
+
+  test "dismiss is blocked for users without preview access" do
+    disable_preview_features
+
+    patch dismiss_insight_url(@insight), as: :turbo_stream
+
+    assert_redirected_to root_path
+    assert @insight.reload.active?
+  end
+
+  test "dashboard omits the insights feed and top-bar entry without preview access" do
+    disable_preview_features
+
+    get root_url
+
+    assert_response :success
+    assert_select "#insights-feed", count: 0
+    assert_select "a[href=?]", insights_path, count: 0
+  end
+
+  private
+    def enable_preview_features
+      @user.update!(preferences: (@user.preferences || {}).merge("preview_features_enabled" => true))
+    end
+
+    def disable_preview_features
+      @user.update!(preferences: (@user.preferences || {}).merge("preview_features_enabled" => false))
+    end
 end

test/jobs/generate_insights_job_test.rb

@@ -3,12 +3,56 @@ require "test_helper"
 class GenerateInsightsJobTest < ActiveJob::TestCase
   setup do
     @family = families(:dylan_family)
+    enable_preview_features(@family)
   end
 
-  test "without args enqueues one job per family" do
-    assert_enqueued_jobs Family.count, only: GenerateInsightsJob do
+  test "without args enqueues one job per preview-enabled family" do
+    assert_operator Family.count, :>, Family.with_preview_features.count,
+      "fixture setup should leave some families without preview access"
+
+    assert_enqueued_jobs Family.with_preview_features.count, only: GenerateInsightsJob do
       GenerateInsightsJob.perform_now
     end
+
+    assert_enqueued_with(job: GenerateInsightsJob, args: [ { family_id: @family.id } ])
+  end
+
+  # Insights is a preview feature and the job manufactures data (and can spend
+  # LLM budget) per family, so families with nobody opted in are skipped
+  # entirely rather than generated for and hidden.
+  test "without args enqueues nothing when no family has preview access" do
+    disable_preview_features(@family)
+
+    assert_no_enqueued_jobs only: GenerateInsightsJob do
+      GenerateInsightsJob.perform_now
+    end
+  end
+
+  test "does nothing for a family without preview access" do
+    disable_preview_features(@family)
+
+    assert_no_difference "Insight.count" do
+      GenerateInsightsJob.perform_now(family_id: @family.id)
+    end
+  end
+
+  test "does not broadcast for a family without preview access" do
+    disable_preview_features(@family)
+
+    Turbo::StreamsChannel.expects(:broadcast_replace_to).never
+
+    GenerateInsightsJob.perform_now(family_id: @family.id)
+  end
+
+  test "generates for a family where only one member opted in" do
+    @family.users.each { |user| set_preview_features(user, false) }
+    set_preview_features(@family.users.first, true)
+
+    stub_generated([ generated_insight ])
+
+    assert_difference "@family.insights.count", 1 do
+      GenerateInsightsJob.perform_now(family_id: @family.id)
+    end
   end
 
   test "does nothing for an unknown family" do
@@ -157,6 +201,18 @@ class GenerateInsightsJobTest < ActiveJob::TestCase
   end
 
   private
+    def enable_preview_features(family)
+      family.users.each { |user| set_preview_features(user, true) }
+    end
+
+    def disable_preview_features(family)
+      family.users.each { |user| set_preview_features(user, false) }
+    end
+
+    def set_preview_features(user, enabled)
+      user.update!(preferences: (user.preferences || {}).merge("preview_features_enabled" => enabled))
+    end
+
     def stub_generated(generated_insights, succeeded_types: nil)
       result = Insight::GeneratorRegistry::Result.new(
         insights: generated_insights,

test/models/family_test.rb

@@ -312,4 +312,48 @@ class FamilyTest < ActiveSupport::TestCase
     assert_equal count_after_first, AccountShare.where(user: newcomer).count,
       "re-running must not create duplicate shares"
   end
+
+  # Preview access is per-user, but jobs that act on family-scoped data have no
+  # Current.user. One opted-in member enables the family.
+  test "preview_features_enabled? is true when any member has opted in" do
+    family = families(:dylan_family)
+    family.users.each { |user| set_preview_features(user, false) }
+
+    assert_not family.reload.preview_features_enabled?
+
+    set_preview_features(family.users.first, true)
+
+    assert family.reload.preview_features_enabled?
+  end
+
+  test "with_preview_features scope agrees with the predicate" do
+    family = families(:dylan_family)
+    family.users.each { |user| set_preview_features(user, false) }
+
+    assert_not_includes Family.with_preview_features, family.reload
+
+    set_preview_features(family.users.first, true)
+
+    assert_includes Family.with_preview_features, family.reload
+  end
+
+  # The family rollup is a jsonb containment match; the UI gates on
+  # User#preview_features_enabled?'s strict `== true`. If containment were the
+  # looser of the two, the nightly job would generate for families whose UI
+  # still hides the feature — so assert the user-level predicate agrees.
+  test "with_preview_features ignores truthy non-boolean values" do
+    family = families(:dylan_family)
+    family.users.each { |user| set_preview_features(user, false) }
+    set_preview_features(family.users.first, "yes")
+
+    assert_not family.users.first.reload.preview_features_enabled?,
+      "the per-user predicate the UI reads must reject a non-boolean"
+    assert_not family.reload.preview_features_enabled?
+    assert_not_includes Family.with_preview_features, family
+  end
+
+  private
+    def set_preview_features(user, enabled)
+      user.update!(preferences: (user.preferences || {}).merge("preview_features_enabled" => enabled))
+    end
 end