QT/sure - sure

QT/sure

Fork 0

mirror of https://github.com/we-promise/sure.git synced 2026-09-05 23:01:26 +00:00

Commit Graph

Author	SHA1	Message	Date
Jonathan KaiserandClaude Sonnet 5	b9b5a7c58b	feat: verify AI configuration and provider liveness from worker processes Closes #3169. The AI status page (#3145, PR #3155) proves only that the `web` process resolved a valid-looking configuration and can reach the configured provider from its own network context. Most AI workloads -- assistant responses, PDF processing, embeddings, auto-categorization, and merchant detection -- actually run in Sidekiq `worker` processes, which can differ from `web` in environment, DNS, proxy rules, network policy, or even loaded credentials (workload-specific overrides, an updated Secret without a pod restart, `web` recreated without `worker`). A passing web check says nothing about whether a worker can do the same. ## What this adds `WorkerAiHealthCheckJob`, queued on demand from a new "Verify worker configuration" button on System health -> AI status. It runs the same bounded, non-destructive probes `AiHealth` already runs, but from inside whichever Sidekiq worker process dequeues it, and records the result via `WorkerAiHealth`: process identity (hostname:pid), checked-at time, a non-secret configuration fingerprint (effective provider, model, redacted endpoint, vector-store adapter/embedding config), and probe outcomes. The AI status tab lists every recorded result -- most recent first, kept for `WorkerAiHealth::RETENTION` (15 minutes) -- each labeled with a status pill (`Passing` / `Failing` / `Stale`, the last once older than `STALE_AFTER`) and a configuration pill comparing it against the web snapshot (`Matches web` / `Differs from web`), with a failure-reason list reusing the existing failure-code translations when a probe failed. ## Implementation constraints from the issue, addressed directly - Cannot reuse a web-cached probe result, or vice versa. `AiHealth.new` gained an injectable `probe_cache:` (default `Rails.cache`, matching today's behavior). The worker job passes a fresh `ActiveSupport::Cache::NullStore` instead, so every worker check is a live call that neither reads a web-cached entry nor leaves one behind. - A single job only verifies one worker. Documented on the button (`coverage_notice`) and in the docs: with multiple replicas, a passing result names one process, not the fleet. Queuing again samples another. - Never persists or displays a raw credential. `WorkerAiHealth::Snapshot` only carries redacted endpoints (AiHealth already redacts these before they reach the job) and provider/model/status fields -- there is no field for a token to occupy. A structural test asserts this stays true. - Failures land in both places an operator already checks. Same destinations as `AiHealth::Probe`'s own failures: `Rails.logger` and `DebugLogEntry` (new `ai_health_worker` category), tagged with the process identity. - Results carry a clear status, including the `pending` case implicitly (no result yet renders an explanatory empty state) and `stale` for a result whose process may no longer reflect current state. - DB-backed vs ENV-backed settings are labeled. A new info block next to the worker results explains which UI settings propagate automatically (rails-settings-cached invalidates the shared cache on write) versus which require restarting/recreating both `web` and `worker`. ## What this deliberately doesn't do Full-fleet coverage (every process publishing a periodic fingerprint) -- the issue lists this under "Other options to consider," not the acceptance criteria, and Sidekiq's normal dispatch doesn't target every process without an explicit per-process coordination mechanism. This PR implements the on-demand, single-check design the acceptance criteria actually describes ("An administrator can request an asynchronous worker-side check", "does not imply full-fleet coverage"); periodic fleet-wide publishing is a natural follow-up if operators need it. ## Testing - `WorkerAiHealth`: recording/reading, same-process replacement vs. cross-process coexistence, MAX_RESULTS bounding, staleness, status derivation (failure codes / component statuses / function-calling refusal), `matches_web?` comparison, and the credential-field structural guard. - `WorkerAiHealthCheckJob`: records a passing/failing snapshot naming this process, writes failures to Rails.logger + DebugLogEntry (and only on failure), never leaks the access token, and -- the defining property -- is proven to construct `AiHealth.new` with an isolated `NullStore` rather than the shared web-facing cache. - `Admin::SystemHealthController`: empty state, a rendered result with matching/mismatched configuration, a failing result's failure reason, a stale result, the `verify_worker_ai` action enqueuing the job and redirecting with a flash notice, and that non-super-admins and unauthenticated requests cannot trigger it. I could not run the Rails test suite in this environment (no working Ruby/Bundler toolchain available locally -- Ruby 2.6 system Ruby vs. the project's required 3.4.9, no way to install without sudo/Docker access). Every file was checked with `ruby -c`, YAML files with `YAML.load_file`, and the ERB view with `ERB.new(...).src`, plus careful manual tracing of each test against the production code paths it exercises, but CI should be treated as the first real run of this suite per the repository's own guidance for exactly this situation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A9494Pxh4LZKnNLTGfFXPw	2026-09-02 07:15:54 +02:00
Juan José Mata	fd6f4ff078	Add live AI checks to system health (#3155 ) * Add live AI checks to system health Give super admins a dedicated AI status view with bounded liveness probes for LLMs, vector stores, pgvector, and embedding endpoints. Record sanitized failures in both the system debug log and Rails logger, and document the recommended local configuration.\n\nCloses #3145 * Fix AI health CI checks * Address AI health review feedback * Correct Ollama model preload guidance * Distinguish OpenAI-compatible providers * Make Ollama startup readiness explicit * Recognize Cloudflare AI endpoints	2026-08-24 22:41:08 +02:00
Khaostica John Baillie Khaostica	6439a731ab	feat(self-hosting): surface Sidekiq-unhealthy nudge + admin system health page (#1906 ) * feat(self-hosting): surface Sidekiq-unhealthy nudge + admin system health page (#1481) When the Sidekiq worker container isn't running — the most common Docker Compose misconfiguration in self-hosted setups — every background job silently never executes. Balance calculations, net-worth updates, and account syncs stall. The UI shows zeros and "No balance data available for this date" without explaining why (#1481, #1047). Per jjmata's resolution on the issue, this PR ships both halves of the fix in one pass: 1. A user-facing nudge banner that appears on every authenticated page when Sidekiq isn't processing jobs. Tells the user their data may be stale; doesn't pretend zeros are real. 2. An admin-only deep link from that banner into a new `/settings/admin/system_health` page (super-admin gated, matching the existing admin namespace contract) showing live Sidekiq state: process count, last heartbeat, max queue latency, job counters, and per-queue depth. ## What changed - New `SidekiqHealth` PORO (`app/models/sidekiq_health.rb`) eagerly loads ProcessSet + Queue + Stats in one pass and exposes `healthy?` plus a stable `reason` symbol (`:redis_unreachable`, `:no_worker_processes`, `:stale_heartbeat`, `:queue_backed_up`). Any Redis/Sidekiq failure during the eager load is caught and surfaced as `:redis_unreachable` so a degraded broker never crashes the layout. - `ApplicationController#current_sidekiq_health` memoizes a single instance per request via `helper_method` so the layout, banner partial, and any controller checks share one Redis round-trip. - New `app/views/shared/_sidekiq_health_banner.html.erb` rendered from `_htmldoc.html.erb` when `Current.user` is present and the health check is failing. Banner shows the user-facing message to everyone; the "View system health" CTA + reason detail are gated on `Current.user&.super_admin?`. - New `Admin::SystemHealthController#show` (inherits the existing `Admin::BaseController`, so super-admin gating is enforced for free) + view rendering status, counters, and per-queue breakdown. - Routes: `resource :system_health, only: :show` inside the existing `namespace :admin`. - Settings nav: new "System health" entry under the Advanced section, gated on `super_admin?` to match `sso_providers_label` and `users_label`. - i18n: new `shared.sidekiq_health_banner.` keys (title, body, CTA, per-reason explanations) and a full `admin.system_health.show.` namespace for the new admin page. English-only, matching how `ds.pill.` and other DS keys are scoped. ## Why - jjmata: "Let's take both approaches ... a nudge about 'data unavailable' which hyperlinks to the admin UI if you are an admin only (not for other types of users) sounds like the best path forward. Any takers for the PR?" (#1481) - smurfpandey: "We can add a section in Settings for superadmins to see 'health' of the application/host." - The detection signal is conservative on purpose: - `PROCESS_HEARTBEAT_TIMEOUT = 2.minutes` tolerates deploy restarts and brief Redis blips without flapping. - `LATENCY_THRESHOLD = 5.minutes` is well above the sync-job tail under default `config/sidekiq.yml` concurrency. ## Validation This worktree runs on Windows without a local Ruby toolchain, so I could not run `bin/rubocop`, `bundle exec erb_lint`, `bin/brakeman`, or `bin/rails test` locally. CI will run the full matrix on the PR: - `lint` — `bin/rubocop -f github` - `lint_js` — `npm run lint` (no JS touched, should be green) - `scan_ruby` — `bin/brakeman --no-pager` - `scan_js` — `bin/importmap audit` - `test_unit` — `bin/rails test` (includes 7 new tests under `test/models/sidekiq_health_test.rb` and 4 new under `test/controllers/admin/system_health_controller_test.rb`) - `test_system` — `DISABLE_PARALLELIZATION=true bin/rails test:system` - `pipelock` — secret + agent-security diff scan Manual checks done in this worktree: - Re-read `CONTRIBUTING.md` and `.cursor/rules/project-conventions.mdc`. PORO under `app/models/` per Convention 2. No new gem dependency per Convention 1. Banner uses semantic tokens (`bg-warning/10`, `text-warning`) per the design-system rules. No `lucide_icon` direct call — uses the `icon` helper per CLAUDE.md. - Confirmed `Sidekiq::ProcessSet` / `Sidekiq::Queue` / `Sidekiq::Stats` are the same APIs Sidekiq 7+ exposes (we're on Sidekiq 8.x per the `Gemfile.lock` comment in `config/initializers/sidekiq.rb`). - Tests stub `Sidekiq::ProcessSet.new` / `Sidekiq::Queue.all` / `Sidekiq::Stats.new` so the suite doesn't need Redis populated. - The admin route lives inside the existing `namespace :admin` so `Admin::BaseController#require_super_admin!` enforces auth — no new authorization surface added. ## Notes - No public API endpoints, no rswag specs, no OpenAPI changes. - No migrations, no model changes outside the new PORO. - No background jobs touched. - English-only locale entry, mirroring the `ds.` / `admin.invitations.` precedent in this repo. Other locales fall back to English. - Detection thresholds are constants on `SidekiqHealth` so they're easy to tune from a follow-up PR if the defaults turn out to flap on any real-world deployment. - The banner positions itself at `top-20` (below the impersonation / super-admin bars) and uses `z-40` (below the `z-50` notification tray). Single-screen overlap with mobile flash toasts is acceptable for V1. Refs: #1481, #1047 fix(self-hosting): address review on Sidekiq health PR (#1481) - `Admin::SystemHealthController#show` now reads from the request-memoized `current_sidekiq_health` instead of building a fresh `SidekiqHealth.new`, so the controller and the layout banner share one Redis round-trip. - `SidekiqHealth#reason` now treats `last_heartbeat_at.nil?` the same as a stale beat: a registered process that hasn't published a heartbeat is not "healthy". Previously the check short-circuited on the nil guard and silently fell through to the queue-latency branch. Added a unit test covering the `ProcessSet` entry with `"beat" => nil` case. - Settings nav: switched the "System health" entry's icon from `activity` to `heart-pulse` so it no longer duplicates the LLM Usage icon. - Routes: dropped the redundant `controller: "system_health"` option from the `resource :system_health` declaration — Rails infers `Admin::SystemHealthController` from the namespace, matching the style of the sibling `:sso_providers`, `:users`, `:invitations`, and `:families` admin resources. * fix(self-hosting): scope + cache Sidekiq health, admin-only banner (#1481) Addresses the second round of maintainer review on the Sidekiq health PR. - Skip the check entirely in managed mode. `current_sidekiq_health` returns `nil` unless `Rails.application.config.app_mode.self_hosted?`, so authenticated requests in managed deployments add zero Redis round-trips for this feature. - Cache the snapshot across requests via `SidekiqHealth.current` (Rails.cache, TTL `CACHE_TTL` = 60s default, env-overridable). The per-request memoization on `ApplicationController` is preserved on top, so even back-to-back self-hosted pages share one fetch. - Make thresholds operator-tunable. `PROCESS_HEARTBEAT_TIMEOUT`, `LATENCY_THRESHOLD`, and the new `CACHE_TTL` read from `SIDEKIQ_HEALTH_HEARTBEAT_TIMEOUT`, `SIDEKIQ_HEALTH_LATENCY_THRESHOLD`, and `SIDEKIQ_HEALTH_CACHE_TTL` env vars (seconds), with the previous values as defaults. Comments now explain the tuning rationale. - Gate the banner on `Current.user&.super_admin?` at the layout level rather than rendering a vague warning to family members who can't act on it. The partial no longer carries an internal admin check since the call site does it; non-admins see nothing. - Replace the hard-coded `top-20` offset with a computed offset based on which impersonation bars are visible (`top-4` / `top-20` / `top-36`) so the banner doesn't collide with the super-admin or approval bars when both are stacked above it. - `Admin::SystemHealthController#show` now bypasses the cache (`SidekiqHealth.expire_cache!` + `SidekiqHealth.new`) so an operator who just restarted the worker sees fresh state instead of a stale 60-second snapshot. Also lets the page render in managed mode where `current_sidekiq_health` is nil. - Tests: add coverage for `.current` cache reuse and `.expire_cache!` forcing a re-query, swapping `Rails.cache` to a MemoryStore since the test env defaults to `:null_store`. * fix(self-hosting): route singular resource + drop assert_same on cached snapshot (#1481) Two CI failures surfaced once the full pipeline ran on this branch for the first time (it was gated on contributor approval until d04b78e): - Admin system-health controller tests returned 404. Singular `resource :system_health` in `config/routes.rb` makes Rails infer `Admin::SystemHealthsController` (it pluralizes the controller name even for singular resources), but the controller file is named `system_health_controller.rb` / `Admin::SystemHealthController`. Restore the explicit `controller: "system_health"` override that the previous "address review" commit dropped on the (mistaken) premise that Rails would infer it from the namespace — the sibling admin routes all use plural `resources` so they round-trip cleanly, this one doesn't. Comment now spells the gotcha out so the next reviewer doesn't try to "simplify" it again. - `SidekiqHealthTest#test_current_memoizes_across_calls_inside_the_cache_TTL` used `assert_same` on the two returns from `SidekiqHealth.current`. `ActiveSupport::Cache::MemoryStore` defaults to `dup_values: true` and Marshals on read, so a cache hit returns an `==`-equal but `equal?`-different instance. Replace the identity check with the behavioral assertion we actually care about: re-stub `ProcessSet` to raise on the second call, then assert the second `current` return is still healthy (proving Redis was not re-queried). * fix(i18n): drop redundant inline default on system_health nav label (#1481) `system_health_label` is already defined in config/locales/views/settings/en.yml, so the inline `default: "System health"` was a hard-coded English string in the template (DS Drift Patrol Rule 5). Use the bare locale lookup like the sibling nav entries. --------- Co-authored-by: John Baillie <johnbaillie2007@gmail.com> Co-authored-by: Khaostica <256858950+Khaostica@users.noreply.github.com>	2026-08-22 07:17:29 +02:00

Author

SHA1

Message

Date

Jonathan KaiserandClaude Sonnet 5

b9b5a7c58b

feat: verify AI configuration and provider liveness from worker processes

Closes #3169.

The AI status page (#3145, PR #3155) proves only that the `web` process
resolved a valid-looking configuration and can reach the configured
provider from its own network context. Most AI workloads -- assistant
responses, PDF processing, embeddings, auto-categorization, and merchant
detection -- actually run in Sidekiq `worker` processes, which can differ
from `web` in environment, DNS, proxy rules, network policy, or even
loaded credentials (workload-specific overrides, an updated Secret without
a pod restart, `web` recreated without `worker`). A passing web check says
nothing about whether a worker can do the same.

## What this adds

`WorkerAiHealthCheckJob`, queued on demand from a new "Verify worker
configuration" button on System health -> AI status. It runs the same
bounded, non-destructive probes `AiHealth` already runs, but from inside
whichever Sidekiq worker process dequeues it, and records the result via
`WorkerAiHealth`: process identity (hostname:pid), checked-at time, a
non-secret configuration fingerprint (effective provider, model, redacted
endpoint, vector-store adapter/embedding config), and probe outcomes.

The AI status tab lists every recorded result -- most recent first, kept
for `WorkerAiHealth::RETENTION` (15 minutes) -- each labeled with a status
pill (`Passing` / `Failing` / `Stale`, the last once older than
`STALE_AFTER`) and a configuration pill comparing it against the web
snapshot (`Matches web` / `Differs from web`), with a failure-reason list
reusing the existing failure-code translations when a probe failed.

## Implementation constraints from the issue, addressed directly

- **Cannot reuse a web-cached probe result, or vice versa.** `AiHealth.new`
  gained an injectable `probe_cache:` (default `Rails.cache`, matching
  today's behavior). The worker job passes a fresh
  `ActiveSupport::Cache::NullStore` instead, so every worker check is a
  live call that neither reads a web-cached entry nor leaves one behind.
- **A single job only verifies one worker.** Documented on the button
  (`coverage_notice`) and in the docs: with multiple replicas, a passing
  result names one process, not the fleet. Queuing again samples another.
- **Never persists or displays a raw credential.** `WorkerAiHealth::Snapshot`
  only carries redacted endpoints (AiHealth already redacts these before
  they reach the job) and provider/model/status fields -- there is no field
  for a token to occupy. A structural test asserts this stays true.
- **Failures land in both places an operator already checks.** Same
  destinations as `AiHealth::Probe`'s own failures: `Rails.logger` and
  `DebugLogEntry` (new `ai_health_worker` category), tagged with the
  process identity.
- **Results carry a clear status**, including the `pending` case implicitly
  (no result yet renders an explanatory empty state) and `stale` for a
  result whose process may no longer reflect current state.
- **DB-backed vs ENV-backed settings are labeled.** A new info block next
  to the worker results explains which UI settings propagate automatically
  (rails-settings-cached invalidates the shared cache on write) versus
  which require restarting/recreating both `web` and `worker`.

## What this deliberately doesn't do

Full-fleet coverage (every process publishing a periodic fingerprint) --
the issue lists this under "Other options to consider," not the acceptance
criteria, and Sidekiq's normal dispatch doesn't target every process
without an explicit per-process coordination mechanism. This PR implements
the on-demand, single-check design the acceptance criteria actually
describes ("An administrator can request an asynchronous worker-side
check", "does not imply full-fleet coverage"); periodic fleet-wide
publishing is a natural follow-up if operators need it.

## Testing

- `WorkerAiHealth`: recording/reading, same-process replacement vs.
  cross-process coexistence, MAX_RESULTS bounding, staleness, status
  derivation (failure codes / component statuses / function-calling
  refusal), `matches_web?` comparison, and the credential-field structural
  guard.
- `WorkerAiHealthCheckJob`: records a passing/failing snapshot naming this
  process, writes failures to Rails.logger + DebugLogEntry (and only on
  failure), never leaks the access token, and -- the defining property --
  is proven to construct `AiHealth.new` with an isolated `NullStore`
  rather than the shared web-facing cache.
- `Admin::SystemHealthController`: empty state, a rendered result with
  matching/mismatched configuration, a failing result's failure reason, a
  stale result, the `verify_worker_ai` action enqueuing the job and
  redirecting with a flash notice, and that non-super-admins and
  unauthenticated requests cannot trigger it.

I could not run the Rails test suite in this environment (no working
Ruby/Bundler toolchain available locally -- Ruby 2.6 system Ruby vs. the
project's required 3.4.9, no way to install without sudo/Docker access).
Every file was checked with `ruby -c`, YAML files with `YAML.load_file`,
and the ERB view with `ERB.new(...).src`, plus careful manual tracing of
each test against the production code paths it exercises, but CI should
be treated as the first real run of this suite per the repository's own
guidance for exactly this situation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A9494Pxh4LZKnNLTGfFXPw

2026-09-02 07:15:54 +02:00

Juan José Mata

fd6f4ff078

Add live AI checks to system health (#3155 )

* Add live AI checks to system health

Give super admins a dedicated AI status view with bounded liveness probes for LLMs, vector stores, pgvector, and embedding endpoints. Record sanitized failures in both the system debug log and Rails logger, and document the recommended local configuration.\n\nCloses #3145

* Fix AI health CI checks

* Address AI health review feedback

* Correct Ollama model preload guidance

* Distinguish OpenAI-compatible providers

* Make Ollama startup readiness explicit

* Recognize Cloudflare AI endpoints

2026-08-24 22:41:08 +02:00

6439a731ab

feat(self-hosting): surface Sidekiq-unhealthy nudge + admin system health page (#1906 )

* feat(self-hosting): surface Sidekiq-unhealthy nudge + admin system health page (#1481)

When the Sidekiq worker container isn't running — the most common Docker
Compose misconfiguration in self-hosted setups — every background job
silently never executes. Balance calculations, net-worth updates, and
account syncs stall. The UI shows zeros and "No balance data available
for this date" without explaining why (#1481, #1047).

Per jjmata's resolution on the issue, this PR ships both halves of the
fix in one pass:

1. A user-facing nudge banner that appears on every authenticated page
when Sidekiq isn't processing jobs. Tells the user their data may be
stale; doesn't pretend zeros are real.

2. An admin-only deep link from that banner into a new
`/settings/admin/system_health` page (super-admin gated, matching the
existing admin namespace contract) showing live Sidekiq state:
process count, last heartbeat, max queue latency, job counters, and
per-queue depth.

## What changed

- New `SidekiqHealth` PORO (`app/models/sidekiq_health.rb`) eagerly
loads ProcessSet + Queue + Stats in one pass and exposes `healthy?`
plus a stable `reason` symbol (`:redis_unreachable`,
`:no_worker_processes`, `:stale_heartbeat`, `:queue_backed_up`).
Any Redis/Sidekiq failure during the eager load is caught and
surfaced as `:redis_unreachable` so a degraded broker never crashes
the layout.

- `ApplicationController#current_sidekiq_health` memoizes a single
instance per request via `helper_method` so the layout, banner
partial, and any controller checks share one Redis round-trip.

- New `app/views/shared/_sidekiq_health_banner.html.erb` rendered from
`_htmldoc.html.erb` when `Current.user` is present and the health
check is failing. Banner shows the user-facing message to everyone;
the "View system health" CTA + reason detail are gated on
`Current.user&.super_admin?`.

- New `Admin::SystemHealthController#show` (inherits the existing
`Admin::BaseController`, so super-admin gating is enforced for free)
+ view rendering status, counters, and per-queue breakdown.

- Routes: `resource :system_health, only: :show` inside the existing
`namespace :admin`.

- Settings nav: new "System health" entry under the Advanced section,
gated on `super_admin?` to match `sso_providers_label` and
`users_label`.

- i18n: new `shared.sidekiq_health_banner.*` keys (title, body, CTA,
per-reason explanations) and a full `admin.system_health.show.*`
namespace for the new admin page. English-only, matching how
`ds.pill.*` and other DS keys are scoped.

## Why

- jjmata: "Let's take both approaches ... a nudge about 'data
unavailable' which hyperlinks to the admin UI if you are an admin
only (not for other types of users) sounds like the best path forward.
**Any takers for the PR?**" (#1481)
- smurfpandey: "We can add a section in Settings for superadmins to see
'health' of the application/host."
- The detection signal is conservative on purpose:
- `PROCESS_HEARTBEAT_TIMEOUT = 2.minutes` tolerates deploy restarts
and brief Redis blips without flapping.
- `LATENCY_THRESHOLD = 5.minutes` is well above the sync-job tail
under default `config/sidekiq.yml` concurrency.

## Validation

This worktree runs on Windows without a local Ruby toolchain, so I
could not run `bin/rubocop`, `bundle exec erb_lint`, `bin/brakeman`, or
`bin/rails test` locally. CI will run the full matrix on the PR:

- `lint` — `bin/rubocop -f github`
- `lint_js` — `npm run lint` (no JS touched, should be green)
- `scan_ruby` — `bin/brakeman --no-pager`
- `scan_js` — `bin/importmap audit`
- `test_unit` — `bin/rails test` (includes 7 new tests under
`test/models/sidekiq_health_test.rb` and 4 new under
`test/controllers/admin/system_health_controller_test.rb`)
- `test_system` — `DISABLE_PARALLELIZATION=true bin/rails test:system`
- `pipelock` — secret + agent-security diff scan

Manual checks done in this worktree:

- Re-read `CONTRIBUTING.md` and `.cursor/rules/project-conventions.mdc`.
PORO under `app/models/` per Convention 2. No new gem dependency per
Convention 1. Banner uses semantic tokens (`bg-warning/10`,
`text-warning`) per the design-system rules. No `lucide_icon` direct
call — uses the `icon` helper per CLAUDE.md.
- Confirmed `Sidekiq::ProcessSet` / `Sidekiq::Queue` / `Sidekiq::Stats`
are the same APIs Sidekiq 7+ exposes (we're on Sidekiq 8.x per the
`Gemfile.lock` comment in `config/initializers/sidekiq.rb`).
- Tests stub `Sidekiq::ProcessSet.new` / `Sidekiq::Queue.all` /
`Sidekiq::Stats.new` so the suite doesn't need Redis populated.
- The admin route lives inside the existing `namespace :admin` so
`Admin::BaseController#require_super_admin!` enforces auth — no new
authorization surface added.

## Notes

- No public API endpoints, no rswag specs, no OpenAPI changes.
- No migrations, no model changes outside the new PORO.
- No background jobs touched.
- English-only locale entry, mirroring the `ds.*` / `admin.invitations.*`
precedent in this repo. Other locales fall back to English.
- Detection thresholds are constants on `SidekiqHealth` so they're easy
to tune from a follow-up PR if the defaults turn out to flap on any
real-world deployment.
- The banner positions itself at `top-20` (below the impersonation /
super-admin bars) and uses `z-40` (below the `z-50` notification
tray). Single-screen overlap with mobile flash toasts is acceptable
for V1.

Refs: #1481, #1047

* fix(self-hosting): address review on Sidekiq health PR (#1481)

- `Admin::SystemHealthController#show` now reads from the request-memoized
`current_sidekiq_health` instead of building a fresh `SidekiqHealth.new`,
so the controller and the layout banner share one Redis round-trip.
- `SidekiqHealth#reason` now treats `last_heartbeat_at.nil?` the same as a
stale beat: a registered process that hasn't published a heartbeat is
not "healthy". Previously the check short-circuited on the nil guard
and silently fell through to the queue-latency branch. Added a unit
test covering the `ProcessSet` entry with `"beat" => nil` case.
- Settings nav: switched the "System health" entry's icon from `activity`
to `heart-pulse` so it no longer duplicates the LLM Usage icon.
- Routes: dropped the redundant `controller: "system_health"` option from
the `resource :system_health` declaration — Rails infers
`Admin::SystemHealthController` from the namespace, matching the style
of the sibling `:sso_providers`, `:users`, `:invitations`, and
`:families` admin resources.

* fix(self-hosting): scope + cache Sidekiq health, admin-only banner (#1481)

Addresses the second round of maintainer review on the Sidekiq health PR.

- Skip the check entirely in managed mode. `current_sidekiq_health`
returns `nil` unless `Rails.application.config.app_mode.self_hosted?`,
so authenticated requests in managed deployments add zero Redis
round-trips for this feature.
- Cache the snapshot across requests via `SidekiqHealth.current`
(Rails.cache, TTL `CACHE_TTL` = 60s default, env-overridable). The
per-request memoization on `ApplicationController` is preserved on
top, so even back-to-back self-hosted pages share one fetch.
- Make thresholds operator-tunable. `PROCESS_HEARTBEAT_TIMEOUT`,
`LATENCY_THRESHOLD`, and the new `CACHE_TTL` read from
`SIDEKIQ_HEALTH_HEARTBEAT_TIMEOUT`, `SIDEKIQ_HEALTH_LATENCY_THRESHOLD`,
and `SIDEKIQ_HEALTH_CACHE_TTL` env vars (seconds), with the previous
values as defaults. Comments now explain the tuning rationale.
- Gate the banner on `Current.user&.super_admin?` at the layout level
rather than rendering a vague warning to family members who can't
act on it. The partial no longer carries an internal admin check
since the call site does it; non-admins see nothing.
- Replace the hard-coded `top-20` offset with a computed offset based
on which impersonation bars are visible (`top-4` / `top-20` / `top-36`)
so the banner doesn't collide with the super-admin or approval bars
when both are stacked above it.
- `Admin::SystemHealthController#show` now bypasses the cache
(`SidekiqHealth.expire_cache!` + `SidekiqHealth.new`) so an operator
who just restarted the worker sees fresh state instead of a stale
60-second snapshot. Also lets the page render in managed mode where
`current_sidekiq_health` is nil.
- Tests: add coverage for `.current` cache reuse and `.expire_cache!`
forcing a re-query, swapping `Rails.cache` to a MemoryStore since the
test env defaults to `:null_store`.

* fix(self-hosting): route singular resource + drop assert_same on cached snapshot (#1481)

Two CI failures surfaced once the full pipeline ran on this branch for
the first time (it was gated on contributor approval until d04b78e):

- Admin system-health controller tests returned 404. Singular
`resource :system_health` in `config/routes.rb` makes Rails infer
`Admin::SystemHealthsController` (it pluralizes the controller name
even for singular resources), but the controller file is named
`system_health_controller.rb` / `Admin::SystemHealthController`.
Restore the explicit `controller: "system_health"` override that
the previous "address review" commit dropped on the (mistaken)
premise that Rails would infer it from the namespace — the sibling
admin routes all use plural `resources` so they round-trip cleanly,
this one doesn't. Comment now spells the gotcha out so the next
reviewer doesn't try to "simplify" it again.
- `SidekiqHealthTest#test_current_memoizes_across_calls_inside_the_cache_TTL`
used `assert_same` on the two returns from `SidekiqHealth.current`.
`ActiveSupport::Cache::MemoryStore` defaults to `dup_values: true`
and Marshals on read, so a cache hit returns an `==`-equal but
`equal?`-different instance. Replace the identity check with the
behavioral assertion we actually care about: re-stub `ProcessSet`
to raise on the second call, then assert the second `current`
return is still healthy (proving Redis was not re-queried).

* fix(i18n): drop redundant inline default on system_health nav label (#1481)

`system_health_label` is already defined in
config/locales/views/settings/en.yml, so the inline
`default: "System health"` was a hard-coded English string in the
template (DS Drift Patrol Rule 5). Use the bare locale lookup like the
sibling nav entries.

---------

Co-authored-by: John Baillie <johnbaillie2007@gmail.com>
Co-authored-by: Khaostica <256858950+Khaostica@users.noreply.github.com>

2026-08-22 07:17:29 +02:00