Commit Graph

85 Commits

Author SHA1 Message Date
Juan José Mata
f6f9feba8a Bank Sync cleanup (#1710)
* feat(settings/providers): surface connection status in section headers

Lifts the per-panel status indicator up to each collapsed accordion
header so admins can see at a glance which providers are connected
without expanding every section. Connected providers sort first.

- Add optional status: and meta: locals to settings/_section partial;
  pill hides via group-open:hidden when the section is expanded
- New settings/providers/_status_pill partial (ok/warn/err/off states)
- Add SettingsHelper#provider_summary to centralise the connected-vs-not
  logic already scattered across panel partials
- Refactor show.html.erb to pass status to every section and sort
  family_panels by connection state
- Add settings.providers.status.* i18n keys
- Add system tests asserting pill renders and sort order

https://claude.ai/code/session_01KW2HCN9rP1fiyQuw7Cju9D

* feat(settings/providers): group providers into Connected and Available

Partition the provider list in the controller into @connected_providers
and @available_providers based on provider_summary status, and render
each group under its own heading with a count. Auto-open the section
when only one provider is connected. Adds an empty-state line when
nothing is connected yet.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(settings/providers): health strip, action-needed group, and sync error surfacing

- Extend provider_summary to return :err/:warn with meta text by checking
  latest sync per item (window function, same pattern as ProviderConnectionStatus)
  and Enable Banking session expiry within 7 days
- Partition provider entries into three groups: Connected (:ok), Action needed
  (:warn/:err, auto-opened), Available (:off)
- Add Settings::HealthSummary ViewComponent — four-tile grid showing Connected,
  Action needed, Errors, and Accounts synced counts
- Render health strip directly under page description; omit Action needed heading
  when group is empty
- Add i18n keys for tile labels, group heading, and all meta strings

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(settings/providers): card grid for available providers with connect drawer

- Add Provider::Metadata registry with static display data (region, kind,
  tier, maturity, logo) for all 11 providers
- Add Settings::ProviderCard ViewComponent rendering logo square, name,
  Beta/Alpha pill, meta line (region · type · tier), tagline, and Connect link
- Add connect_form action + route (GET /settings/providers/:key/connect_form)
  that opens the existing panel partial or config form in a DS::Dialog drawer
- Replace the Available accordion loop with a 2-column responsive card grid;
  empty state when all providers are connected
- Fix layout override: use turbo_rails/frame layout for frame requests so the
  drawer response is not wrapped in the full settings layout (was causing
  Turbo to pick the empty outer drawer frame instead of the filled one)
- Add SyncAllProvidersJob and last_sync_all_attempted_at migration (sync-all
  throttle support)
- Unify Connected + Action needed into a single "Your connections" section;
  items with warn/err status auto-open
- Fix Enable Banking grouping: items with expired sessions were returning
  :off (Available) instead of :warn (Your connections); gate now checks
  any? instead of any?(&:session_valid?)
- Add reconsent_required locale key for fully-expired EB sessions
- Surface Beta/Alpha maturity pills on connected provider accordion rows
  via new badge: param on settings_section helper
- Add i18n taglines for all 11 providers; add connect and empty_available keys

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(settings): retire /settings/bank_sync; merge into providers page

- Delete Settings::BankSyncController and its views (the providers page is
  now a strict superset of what bank_sync offered)
- Add permanent 301 redirect: GET /settings/bank_sync → /settings/providers
- Collapse nav to a single "Bank Sync" entry pointing at /settings/providers;
  remove the duplicate admin-only "Providers" entry from the Advanced section
- Remove "Providers" from SETTINGS_ORDER; point "Bank Sync" at
  settings_providers_path for next/prev navigation
- Rename page title to "Bank Sync"; replace admin-credential lede with
  user-facing copy ("Connect external accounts…")
- Update breadcrumb: Home → Bank sync
- Add controller test asserting 301 status and Location header

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Migrations are 7.2 here

* Minimize schema noise

* Schema duplication

* Small copy edits

* Fix tests

* Address provider settings review feedback

* refactor(settings/providers): finish design-review cleanup pass

Picks up the remaining items from Claude Design's review of #1710
that the previous review-feedback commit didn't cover.

DS / casing
- Sentence-case the page title ("Bank Sync" -> "Bank sync") and
  align the nav label.
- Drop the card hover-lift (shadow-border-sm) in favour of
  bg-container-hover; per the DS, card hover is colour-only.
- Whole-tile click target on each provider card — the inner
  "Connect ->" link was a hit-target inversion.
- Set Sync all to whitespace-nowrap so the label stops wrapping at
  narrow viewport widths.

UX simplifications
- Drop the four health-summary tiles (per-row warn/err pills already
  surface the signal at the scale this app sees). Removes
  Settings::HealthSummary, the @health_counts controller block, and
  the now-unused health.* locale keys.
- Hide "Your connections" heading + empty-state line when no
  providers are connected — the lede already invites a connect.
- Drop the redundant "Free" tier from per-card meta lines (printed
  10x for one fact); "Paid" still surfaces on Plaid.

Tests updated to drop the obsolete tiles assertion and switch the
provider-card click selector to look up the (now whole-card) anchor
by provider name.

* feat(settings/providers): replace Add another provider CTA with a search + kind filter

Per the design review, the "Add another provider · Browse providers"
card was a redirect to content one scroll-tick away. A search input
plus kind chips lets users self-segment the catalog and is the right
tool once it grows beyond the four to twelve providers we ship today.

- New providers_filter Stimulus controller — case-insensitive free
  text search across name/region/kind, plus a chip group with
  All / Banks / Crypto / Investment that toggle visibility via
  Tailwind's `hidden` class.
- _search_filters partial: search box (count-pluralized placeholder)
  + chip group, ARIA-labelled and aria-pressed for the chips.
- ProviderCard exposes filter_data (target + name/region/kind data
  attrs) so the controller can match without re-rendering.
- Lunchflow's `kind` was "Lunch" — switched to "Bank" so it falls
  under the Banks chip alongside its actual offering (it aggregates
  banks).
- Drops the add_provider_cta partial and its locale entries; adds
  search_filters.* and an empty_filter message.

* Private method fix

* refactor(settings/providers): drawer cleanup, header lock-up, trust statement

Per the design review's §07.

- Drop the trailing "Configured / Not configured" footer status from
  every provider panel (binance, coinbase, coinstats, indexa_capital,
  lunchflow, mercury, simplefin, snaptrade, sophtron, provider_form).
  The parent details section's status pill already carries that
  signal; the footer was redundant — and the copy/styling was
  inconsistent across panels (free-text vs. dot pill, "configured"
  vs. "not connected").
- Connect drawer gets a header lock-up: small logo chip + provider
  name + maturity badge, mirroring the available-card layout.
  Implemented as _drawer_header partial; connect_form passes
  custom_header: true to DS::Dialog so we own the row.
- Drawer footer trust statement: "Read-only — Sure can never move
  money. Stored encrypted." A single-line reassurance covering all
  panels.
- Sentence-case the hardcoded primary buttons that were Title Case:
  "Save Configuration" -> "Save and connect"
  "Update Configuration" -> "Update connection"
  "Connect Bank" -> "Connect bank"
  Affects simplefin, lunchflow, enable_banking, provider_form. The
  i18n'd panels (binance, coinbase, coinstats, indexa_capital,
  mercury, snaptrade, sophtron) keep their existing keys.

* chore(locales): drop unused provider-panel status strings

Footer "Configured / Not configured" status was removed from each
provider panel partial in the prior drawer-cleanup pass; the matching
i18n keys are no longer referenced. Removing them across every
locale to keep the catalogue clean.

Dropped (15 keys × varying locale coverage, 36 line removals across
24 files):

- coinstats_items.new.{status_configured_html, status_not_configured}
- indexa_capital_items.panel.{status_configured_html, status_not_configured}
- mercury_items.provider_panel.{configured_html, not_configured, accounts_link}
- sophtron_items.sophtron_panel.status.{configured_html, not_configured}
  (parent `status:` removed where it became empty)
- providers.snaptrade.{status_needs_registration, status_not_configured}
  (status_connected stays — still used by the lazy-load summary)
- settings.providers.{binance_panel, coinbase_panel}.{status_connected, status_not_connected}

* feat(settings/providers): connected-state polish per design §05 + Linked institutions rename

Building the next phase of the design review. Pulls forward the
slim health strip, denser connection rows, and "Linked institutions"
heading rename — the small Phase A lift the designer flagged in
§08 of the doc.

- New _health_strip partial: single-line at-a-glance pulse —
  connected count + needs-attention count + accounts syncing +
  last-synced timestamp. Renders only when at least one provider
  is linked or needs action.
- New _connection_row partial replaces the generic settings_section
  call for providers. Tighter rows: text-sm title (was text-lg),
  px-4 py-3.5 padding, single-line summary (chevron + name +
  maturity badge + meta + status pill + sync action). Warn/error
  rows get a coloured outline (border-warning/25 or
  border-destructive/25) so the at-risk row stands out without
  shouting.
- "Sync all" button restyled to match the design's secondary
  button: text-primary, alpha-black-100 border, rounded-[10px],
  padding 7px 12px (was the broader px-3 py-1.5 ghost).
- "Your connections" → "Linked institutions" heading, lifted from
  the designer's Phase-C reconciliation note. Primes users for the
  Option-C institution-search wizard six months early; existing
  i18n key stays as `groups.your_connections` for now to keep the
  rename to a single value flip.
- Controller computes the new @health hash (connected,
  needs_attention, accounts_syncing, last_synced_at) feeding the
  strip; brings back the single accounts query that was removed
  with the four-tile component.

System test updated for the new heading copy.

* fix(settings/providers): align connected state with the final design mock

Tightening the §05 polish to match the user-confirmed final design.

- Revert "Linked institutions" → "Your connections". The §08
  designer note about the Phase-A heading rename didn't carry
  forward to the final mock; keep the original wording.
- Drop the warn/err auto-open on connection rows. The design shows
  Enable Banking collapsed with a warn-outline and a status pill —
  no auto-expanded form. Single-connection auto-open kept (handy
  when the page is otherwise empty).
- Hide the "accounts syncing" segment in the health strip when the
  count is 0 — the design mock assumes a populated number; an
  always-visible "0 accounts syncing" reads as a placeholder.
- Strip the leading "about " from `time_ago_in_words` everywhere
  the result is shown to the user (health strip "Last synced %{time}
  ago" plus per-row "Synced %{time} ago" meta). Matches the design's
  shorter copy.

* refactor(settings/providers): tighten paddings, dedupe maturity badge, semantic + a11y fixes

Pixel-level alignment to the design's §05 mock + cleanup from a DS
audit pass.

Paddings, margins, font sizes
- Health strip: my-4 → mt-4 mb-5 to match the design's 16px / 20px
  vertical breathing room.
- Search filters bar: gap-2 → gap-2.5; mt-2 → mt-5 mb-3 (was missing
  the 12px bottom margin entirely).
- Search box: rounded-lg → rounded-[10px]; px-3 py-2 → px-[14px]
  py-[9px]. Search icon downsized w-4 → w-3.5 to match.
- Chip group: p-1 → p-[3px]; rounded-lg → rounded-[10px].
- Chip: py-1 → py-[5px]; rounded-md → rounded-lg.
- Group heading: mt-2 → mt-[18px]; mb-1 → mb-1.5.
- Status pill: text-xs → text-[11px].
- Provider card: gap-3 → gap-2.5 (outer + top); name gets explicit
  text-sm; tagline + foot 14px → 13px; arrow icon w-4 → w-3.5.
- Sync icon button: p-1 → fixed w-7 h-7 (28×28) so the row hit
  target matches the design's column width.
- Connect drawer header logo glyph: text-[10px] → text-xs (matches
  the available card's logo-glyph treatment).

Component / partial cleanup (DS audit follow-ups)
- New _maturity_badge partial replaces the inline span that was
  duplicated in 3 places (_connection_row, _drawer_header,
  provider_card.html.erb).
- Settings::ProviderCard.maturity_label class method centralizes the
  MATURITY_LABELS lookup; callers no longer reach into the constant.
- _connection_row title: <h2> → <h3> (the row sits inside the
  "Your connections" h2 group heading; nested h2s flattened the
  outline).
- show.html.erb encryption error: <h3> → <h2> for the same reason.

Locale
- Drop orphaned keys: settings.providers.groups.connected and
  groups.needs_attention (no view code uses them) plus the leftover
  show.coinbase_title block.
- Health strip "needs reconsent" → "needs attention" so the strip
  copy lines up with the per-row status pill ("Action needed") and
  the original group heading wording.

A11y
- focus-visible:ring-2 on chip buttons, provider-card link, and
  focus-within:ring-2 on the search input wrapper. Keyboard users
  now get a visible focus state.
- Search input: explicit autocomplete="off" (erb_lint hint).

* fix(settings/providers): icons + search input height

- Icons were rendering at 20px because the application_helper's `icon`
  default size (`md` = w-5 h-5) was beating the inline class override
  in compiled CSS source order. Pass `size: "sm"` and use the project's
  `!w-3.5 !h-3.5` important-prefix pattern (precedent: dashboard.html.erb)
  so chevron, refresh-cw, search, check, circle-alert, and arrow-right
  all render at the design's 14px.
- Search input was 54px tall because @tailwindcss/forms applies
  `padding: 8px 12px` to bare `<input type="search">`. Override with
  `!p-0 focus:ring-0 focus:shadow-none` so the wrapping div's padding
  alone defines the box (38px total — matches the design).

* refactor(settings/providers): align Sync all + search input with DS, address review feedback

- Sync all: replace the hand-rolled `button_to` with `DS::Link.new(variant: "outline", method: :post)` — same component as the
  "Identify Patterns" button on the recurring-transactions page.
- Search input: switch to the icon-overlay pattern used by the
  Manage-currencies and transaction filter rows
  (relative wrapper + absolutely positioned search icon +
  bordered input with `focus:ring-gray-500`). Brings the keyboard
  focus state in line with the rest of the app's filterable lists.
- SnapTrade panel: restore the "needs registration" status row that
  the drawer-cleanup pass dropped along with the redundant
  Configured/Not configured footer. The unregistered case is
  meaningful state, not redundant chrome.
- Move the slim health-strip computation out of the controller and
  into `SettingsHelper#provider_health_strip` (Convention 2: skinny
  controllers).
- Extract `concise_time_ago` helper so the "drop leading 'about '"
  trick stops being duplicated 3x.
- `Settings::ProviderCard#maturity_label` (instance) now delegates
  to `.maturity_label` (class) instead of duplicating the lookup.
- Drop unused `warn_or_err` local in `_connection_row`.
- Replace the `data-controller` string-injection + html_safe in
  `_connection_row` with `tag.details(data: ...)`; safer and more
  idiomatic.
- Add a system test for the empty-filter message wiring.

* fix(settings/providers): drawer trust statement uses border-tertiary

`border-secondary/10` was reaching for the text-foreground token at
10% opacity for a divider. The project ships a dedicated divider
token (`border-tertiary`, ~8% black) used by DS::Menu, the holdings
page, and admin/sso forms. Switching to it makes the trust-statement
HR match every other thin divider in Sure and stops misusing the
text token as a border.

* refactor(settings/providers): swap arbitrary Tailwind values for scale tokens

Per the user's directive — DS-compliance over pixel-perfect alignment
with the design mock. Walked the design audit and applied every swap
that lands within ±2px of the original.

Swaps:
- _health_strip: gap-[18px] → gap-5 (+2), px-[14px] → px-3.5 (=),
  text-[13px] → text-sm (+1).
- _search_filters: chip group p-[3px] → p-1, rounded-[10px] →
  rounded-xl (concentric with rounded-lg inner pills), chip py-[5px]
  → py-1.
- _status_pill: text-[11px] → text-xs.
- _group_heading: mt-[18px] → mt-5.
- _maturity_badge: text-[10px] → text-xs.
- provider_card: tagline + foot text-[13px] → text-sm.

Kept arbitrary: `min-w-[200px]` in _search_filters — nearest scale
tokens are min-w-48 (192px) and min-w-52 (208px); both are noticeable
layout shifts for a one-off responsive guard. Worth keeping the
arbitrary here.

Net: 9 of 10 arbitrary values gone. Visual delta: max +2px on a
single value. Design mock and DS scale now agree.

* revert(settings/providers): drop the slim health strip

Per-row status pills already carry the at-a-glance signal (connected
/ action needed) at the scale this app sees (1–4 connections per
family). The strip was redundant chrome for almost every user; only
worth bringing back if the catalog grows to a point where the row
list itself stops fitting on a single screen.

- Delete _health_strip.html.erb partial.
- Drop @health controller assignment + provider_health_strip helper.
- Drop unused settings.providers.health_strip.* locale keys.
- concise_time_ago helper stays — still used by per-row meta text.

* refactor(settings/providers): align with DS conventions

Two consistency wins from the screenshot/DS audit pass.

Sync icon button now renders DS::Button (variant: icon, size: sm)
instead of a hand-rolled `button_to`. Same component used by other
icon-only actions across the app (settings/profiles, layouts/imports).
Visual delta: 28×28 → 32×32 (DS sm size). Accept the +4px for
consistency. `event.stopPropagation()` still wired via the form opt
so the row's <details> doesn't toggle when the user clicks the
button.

Group heading now follows the established Sure section-label style
(`text-xs font-medium text-secondary uppercase`) used by
`_settings_nav` and the imports/categories surfaces. The previous
sentence-case `text-sm text-primary` was a one-off that didn't
match the rest of the app. Locale strings stay sentence-case;
uppercase comes from CSS `text-transform`. Tests updated to
case-insensitively match the rendered heading text.

* fix(provider/metadata): add plaid_eu entry

`plaid_eu` is registered as a separate Provider::ConfigurationRegistry
entry but had no Provider::Metadata row, so its card in the
Available grid fell through to the gray-500 default and rendered
empty (no region, kind, tier, or tagline). The title also came out
as "Plaid Eu" because `titleize` doesn't know "EU" is an initialism.

- Add a `plaid_eu` row to Provider::Metadata::REGISTRY with the same
  shape as `plaid` (US → EU, otherwise identical).
- Introduce an optional `name:` field in metadata; controller falls
  back to it before titleizing the provider key. Lets `plaid_eu`
  render as "Plaid EU".
- Add the missing `settings.providers.taglines.plaid_eu` translation.

* fix(settings/providers): center-align Sync all next to the lede

`items-start` made the button hug the first line when the lede wrapped;
on a single line the button sat at the top of the text bounding box
which read slightly off. Center matches the dominant convention
across the rest of settings (api_keys, securities, hostings, _section,
_settings_nav_link_large).

* fix(settings/providers): drop colour palette + filter polish + drawer warnings

Round of design-feedback fixes.

Provider chips
- Drop the per-provider raw Tailwind palette (bg-blue-600 etc.) from
  Provider::Metadata. All cards + drawer logo lock-up now use
  bg-surface-inset + text-primary, matching the design's §04 "drop
  colour entirely" recommendation. Solves the long-standing §01
  BLOCKER without externalising brand assets. Re-introducing logos
  later just means an optional logo_svg: field on metadata.
- ProviderCard component drops the `logo_bg:` parameter; the chip
  is now styled in the template.

Filter / search
- "Available · N" count and the empty-filter state now update
  client-side as the chip filter and free-text search narrow the
  grid (new `count` Stimulus target + dedicated update path).
- Empty-filter state now offers a Clear filters button that resets
  both the search input and the active chip in one click.
- Search placeholder drops the drifting "Search 9 providers" count
  for plain "Search providers" — the section heading carries the
  number.
- Chip labels normalised to plural where natural: "Banks · Crypto ·
  Investments" (Crypto stays as the mass noun).

Drawer copy / treatment
- "IP Whitelisting Required" → "IP whitelisting required" (DS
  sentence-case).
- Binance "do NOT enable withdrawal permissions" lifted out of
  inline red-text into a proper bg-warning-50 border-warning-200
  alert block with an alert-triangle icon. Matches the api_keys /
  hosting alert pattern.
- SnapTrade free-tier inline alert-triangle now uses `size: "sm"`
  so the icon stops rendering at 20px next to 14px body text.

Spacing
- Group-heading margin top bumped 5 → 6 (20→24px) so the eyebrow
  has more breathing room above the search bar.

* refactor(settings/providers): drawer alerts use DS::Alert; drop card-in-card

Two consistency fixes from a design-review pass.

DS::Alert adoption
- Replaces 9 hand-rolled error blocks across the provider panels
  (`bg-destructive/10 text-destructive ... line-clamp-3`) with
  `DS::Alert(variant: :error)` — the project's existing primitive.
- Replaces the just-shipped Binance no-withdraw warning block with
  `DS::Alert(variant: :warning)` instead of a hand-rolled
  `bg-warning-50 border-warning-200` card.
- Replaces the SnapTrade free-tier inline icon-prefixed warning
  paragraph with `DS::Alert(variant: :warning)` — proper alert
  treatment for an actual warning, not body copy.
- Replaces the Enable Banking "Configuration locked" inline
  `bg-warning/10` two-paragraph block with `DS::Alert(variant: :warning)`
  using `safe_join` for the title + body.
- Replaces the encryption-error block at the top of show.html.erb
  with `DS::Alert(variant: :error)`, again via `safe_join`.

Mercury card-within-card
- The "Add another Mercury connection" form was wrapped in a
  `<details>` `bg-container shadow-border-xs rounded-xl` card. In
  the Connect drawer (always 0 existing connections), that wrapping
  card-inside-the-drawer-card has no value — the form is the only
  thing on the surface. Drop the wrapper when no connections exist;
  keep the heading + form inline. When 1+ connections exist (the
  section page) the heading hints "+ Add another connection"
  without the disclosure indirection.

Trade-off: the error-alert blocks lose their `line-clamp-3` /
`title=` truncation. Acceptable for now — DS::Alert can grow a
truncate option as a follow-up if needed.

Open follow-up: DS::Alert itself uses raw Tailwind palette
(`bg-yellow-50` etc.) instead of semantic tokens, and only accepts
a single string `message:`. A separate issue tracks this.

* fix(settings/providers): hoist warning alerts to top of drawer

DS::Alert convention across the rest of the app: alerts sit at the
top of the form / page / section, not floating between content
blocks. The Binance no-withdraw warning and SnapTrade free-tier
warning were rendering between the setup-instructions list and the
form fields — visually wonky.

Move both to the top of their respective panels so the warning is
the first thing the user sees when the connect drawer opens.

Existing precedents this aligns with:
- accounts/_form.html.erb (error alert above form)
- valuations/new.html.erb (error alert above form)
- other_assets/new.html.erb (info alert above form)
- holdings/show.html.erb (warn alerts above content)

* fix(DS::Alert): align icon to cap-height of first text line

`items-start` on the container made the icon's top edge flush with
the text's top edge, leaving the icon's optical center sitting below
the text's first-line center. The hand-rolled alerts elsewhere in
the codebase (api_keys/new, hostings/_sync_settings, holdings/show)
all add `mt-0.5` to the icon for the same reason — fold that into
the primitive so every caller gets the cap-height alignment.

* copy(settings/providers): tighten alert messaging per voice review

Copy expert pass on the new provider drawer alerts. House style:
sentence case for titles, lead with the action, drop "Warning:" /
"Please" filler (the alert variant icon already signals tone),
prefer one short sentence + optional title-paragraph for emphasis.

- Binance no-withdraw warning: was a single line "Warning: do NOT
  enable withdrawal permissions" — alarmist without context. Now
  splits into "Read-only key only" (title) + "Don't enable
  withdrawal permissions when creating your Binance API key — Sure
  only needs read access." (body).
- SnapTrade free-tier note: "Free tier includes 5 brokerage
  connections. Additional connections require a paid SnapTrade
  plan." → "SnapTrade's free tier covers 5 brokerage connections.
  Upgrade on SnapTrade for more."
- SnapTrade connection-limit-info inside the brokerage list: cut
  entirely. The drawer already shows the cap; restating it in the
  list was noise.
- SnapTrade needs-registration: "Credentials saved — finish
  registration to connect a brokerage." → "Credentials saved.
  Finish setup to connect a brokerage." ("registration" was
  ambiguous — register where, with whom?)
- Enable Banking "Configuration locked" body: "Credentials cannot
  be changed while you have active bank connections. Remove all
  connections first to update credentials." → "Disconnect all
  linked banks before changing these credentials." Same meaning,
  half the words.
- Encryption-error block: title-cased "Encryption Configuration
  Required" → "Encryption keys missing"; body strips "Please
  ensure" filler and the parenthetical credential dump, leaving
  the three credential names inline as a clean list. Self-hosters
  still get exactly the names they need to set.

* feat(settings/providers): SetupSteps partial for connect-drawer instructions

Per the design's drawer-cleanup follow-up. Replaces the per-panel
"Setup instructions:" + ordered list + "Field descriptions:" block
with a shared boxed-step component.

The new partial — `_setup_steps.html.erb` — takes a `steps:` array
of strings (or html_safe strings for inline links / code) plus an
optional `help:` hash for a docs link below the steps. The eyebrow
label is "Setup" (uppercase, tracking-wider) matching Sure's other
section labels.

Applied across all eleven provider panels:
- _provider_form (Plaid + Plaid EU): field descriptions move to
  per-field helper text below the input.
- _binance, _coinbase, _coinstats, _indexa_capital,
  _lunchflow, _mercury, _simplefin, _snaptrade, _sophtron,
  _enable_banking: ordered list + duplicate "Field descriptions"
  block both replaced by the partial.
- Some panels' inline copy tightened in the same pass (Lunch Flow,
  SimpleFIN, Enable Banking) — the design copy is shorter than the
  current legacy strings; a copy-pass through every panel can
  follow as a separate cleanup.

Token notes: uses scale tokens (`rounded-xl`, `text-xs`/`text-sm`,
`tracking-wider`) instead of the design mock's exact arbitrary
values, per the consistency-over-design-specs directive on this
branch.

* fix(settings/providers): tighten panel spacing + relocate per-panel notes

Read-flow audit on each connect drawer. The uniform `space-y-4`
treated every block (alert, steps, info card, fields, button) the
same — visually they were five sibling boxes with no grouping. The
fix is per panel; some notes belong as helper text on a specific
field, others as a tightly-grouped pre-fill primer.

Per panel:

- Binance: IP-whitelisting card now matches the setup_steps box
  (`bg-surface-inset rounded-xl`) and is wrapped with setup_steps
  in an inner `space-y-2` so they read as a single pre-fill primer
  cluster. Same eyebrow treatment ("IP whitelisting required") so
  the two boxes look like sister panels, not unrelated chrome.

- SnapTrade: drop the description paragraph above setup_steps. The
  available-providers card grid already markets SnapTrade
  ("Connect brokerage accounts via the SnapTrade aggregation
  network."); repeating in the drawer was duplication.

- Mercury: move the sandbox-API note out of its standalone <p>
  below setup_steps and into per-field helper text under the
  base_url field — the user only cares about the sandbox URL when
  they're filling that field. Applied to both the per-item edit
  form and the add-new form.

- _setup_steps partial: drop the now-pointless `mb-2` (outer
  `space-y-4` already controls the gap; bottom-margin was dead
  CSS thanks to margin-collapse rules with the next sibling's
  margin-top).

* fix(settings/providers): plaid + indexa drawers join the SetupSteps look

Two unifying fixes after the panel-by-panel screenshots showed
mixed treatments.

Plaid + Plaid EU
- The registry-driven panel (_provider_form) was still rendering
  each adapter's markdown `description` block as plain prose
  ("Setup instructions: 1. Visit the Plaid Dashboard ..."). Other
  panels switched to the SetupSteps box; Plaid was the odd one out.
- Drop the markdown `description` block from both plaid_adapter
  and plaid_eu_adapter. Render setup_steps in _provider_form for
  these two provider keys via inline ERB (link helper handles the
  Plaid Dashboard link cleanly; the regional differences fold to
  the same dashboard URL with a different account scope).
- Other registry-based providers fall through to the previous
  markdown description path — no behavior change for them.

Indexa Capital
- The API token field was wrapped in a `bg-surface border` "card"
  that duplicated the field label inside as a heading and put the
  description above the input. Same pattern the user flagged as
  the "card within input" anti-shape.
- Drop the wrapper. The styled-form input renders its own label;
  description moves to per-field helper text below the input,
  matching the pattern used by Plaid (provider_form) and Mercury.

* fix(settings/providers): surface configured plaid_eu + dedup show context

provider_summary had no plaid_eu branch — configured plaid_eu was
falling through to status :off and rendering in Available even with
credentials set. Collapse plaid + plaid_eu into a single registry
check.

Drawer title for non-panel configurations was provider_key.titleize,
which produced "Plaid Eu" while the available card grid used
metadata[:name] = "Plaid EU". Read from metadata first.

While here:
- compute_provider_sync_health no longer relies on
  instance_variable_get; pass family_panel_items explicitly so the
  hash-key/ivar-name coupling is gone.
- drop unused .includes(:syncs, :mercury_accounts) and
  .includes(:snaptrade_accounts) from prepare_show_context. The show
  view only consults summary[:status]; the eager-loads were carried
  over from connect_form (which has its own load_provider_items).

* i18n(settings/providers): localize plaid setup steps + drop dead defaults

The plaid + plaid_eu setup steps in _provider_form.html.erb were
hardcoded English strings. Move them to settings.providers.plaid_panel
(shared) + plaid_eu_panel (EU-specific step 1) so they can be
translated like every other panel.

_setup_steps.html.erb was passing default: "Setup" / "Need help?" to
t(), masking missing translations in non-EN locales. Both keys exist
in en.yml — drop the defaults so missing translations actually
surface.

* test(settings/providers): cover plaid_eu, clear filters, warn outline

Three system test additions:

- Configured plaid_eu surfaces in Your connections (regression guard
  for the helper fix; previously fell through to Available).
- Clear filters button resets input + chip state and brings cards
  back into view.
- :warn-state connection row carries the border-warning/25 outline
  that distinguishes it from an :ok row.

* copy(settings/providers): drop em dashes, naturalize phrasing

Sweep through every string this branch added and replace em-dash
splices with full sentences or simple connectives.

en.yml:
- drawer_trust_statement now reads "Read-only access. Sure can never
  move money, and your credentials are stored encrypted." instead
  of em-dash splicing.
- sync_all_recently / recently_synced split into two sentences.
- binance_panel.no_withdraw_body, plaid_panel.step_1_html / step_2,
  plaid_eu_panel.step_1_html same treatment.

Hardcoded panel steps (enable_banking, lunchflow, simplefin) become
"Go to <link> and …" or "Go to <link> for …" instead of the
"<link> — get …" splice. Same setup_steps comment cleaned up.

* fix(settings/providers): address CodeRabbit pass on PR #1717

Fixed:
- Localize the setup steps in _enable_banking_panel,
  _lunchflow_panel, and _simplefin_panel. The em-dash sweep had
  rewritten these into hardcoded English; they now route through
  settings.providers.{enable_banking,lunchflow,simplefin}_panel
  step_1_html / step_2 / step_3 keys, mirroring the plaid_panel
  treatment.
- connect_form: silent redirect when provider_key is unknown now
  carries an alert (settings.providers.not_found) so misrouted
  links don't drop users on the page with no feedback.
- sync action: redirect notice now reflects whether anything was
  actually scheduled — adds settings.providers.sync_provider_no_items
  for the "all items already syncing or none exist" path.
- Family::Syncer test: count plaid_items via the .syncable scope to
  match what Family::Syncer actually schedules (already done for
  binance_items in the same test).

Skipped, with reasons:
- focus:ring-gray-500/-gray-900 in coinstats / coinbase / simplefin /
  search_filters: tracked under issue #1715 as part of the raw-palette
  → DS-token sweep across the whole codebase.
- Coinbase #0052FF brand-color wrapper: tracked under PR #1710's
  follow-up tracking comment as the deferred Provider::Metadata
  colour-palette decision (designer §01).
- Sophtron submit-button extraction into DS::Button: same
  deferred sweep — every panel hand-rolls this class string;
  one-off extraction would just churn.
- Redundant .html_safe on _html keys in coinstats: tracked in #1715.
- _provider_form.html.erb env hint, "Optional" placeholder, "Save and
  connect" submit: pre-existing strings not added on this branch.
- Renaming sync_health_for's :stale to :data_stale: pre-existing
  shape, refactor scope.
- Plaid_eu using plaid_panel.step_2/step_3 keys: deliberate. Same
  English copy across both providers; duplicating keys would just
  give translators twice the work for identical strings.
- _enable_banking_panel / _lunchflow_panel / _simplefin_panel
  alert + submit + button labels: pre-existing hardcoded strings
  from before this branch. Setup steps were the strings actually
  touched in the em-dash sweep, so those got localized; the rest
  belong in a broader panel-i18n pass.

Verified:
- bundle exec erb_lint on the three panels: clean.
- bin/rubocop on controller + test: clean.
- bin/rails test test/models/family/syncer_test.rb
  test/controllers/settings/providers_controller_test.rb:
  23 runs, 85 assertions, 0 failures.
- DISABLE_PARALLELIZATION=true bin/rails test
  test/system/settings/providers_test.rb:
  15 runs, 38 assertions, 0 failures.

* fix(db): rename migration to clear collision with main's 20260508120000

Main's PR #1705 (Sophtron manual sync) shipped a migration with
the same 20260508120000 timestamp as our
add_last_sync_all_attempted_at_to_families migration. The merge
that brought main into this branch left both files at the same
prefix, which trips Rails' "Duplicate migration" guard at
db:schema:load time and broke CI.

Renaming our migration to 20260510120000 keeps the column it adds
intact (already in db/schema.rb) and bumps the schema version to
match. No DB-level change.

* fix(settings/providers): card + strip a11y polish

- Bring back the slim health strip; gate behind 10+ accounts
  (HEALTH_STRIP_MIN_ACCOUNTS) so it stays out of the way for
  small libraries where per-row pills already carry the signal.
- Status pill: drop the bg-{c}/10 text-{c} pattern (failed AA
  on warn / err); switch to bg-surface-inset text-primary with
  the dot still carrying semantic colour. Passes AA in both
  themes; the dot is the only colourful affordance.
- Maturity badge: bg-alpha-black-50 was invisible against the
  hovered card bg in light mode and against bg-container in
  dark mode. Move to bg-surface-inset + border-tertiary so it
  stays delineated through hover and dark theme.
- Provider card: keep the bg shift on hover (now bg-surface-inset
  for a perceptible delta), focus ring promoted alpha-black-100
  -> alpha-black-300 (visible to keyboard users), meta line
  text-subdued -> text-secondary (text-subdued failed AA at
  2.86:1 against bg-container).
- Restore the per-provider logo palette dropped in 6abceb07.
  Yellow-on-white was the BLOCKER then; bumped Binance to
  yellow-600 and CoinStats to pink-600 (distinct from Binance
  and AA-safe with white text).
- Health strip dividers: bg-alpha-black-100 was invisible in
  dark mode. Switch to border-l border-secondary so the DS
  variant flips correctly.

* fix(settings/providers): keep row height on open

The right-side meta + status pill + sync button group is hidden
via group-open:hidden, but the sync button (DS::Button size sm,
h-8) is what dictated the row's natural height. With it gone,
the row collapsed from 60px to 48px and the title appeared to
jump upward.

Pin a min-h-15 on the <summary> so the height stays constant
through open/close.

* Let's not regress IPv6

* Keep the only real change in schema.rb

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Signed-off-by: Guillem Arias Fauste <accounts@gariasf.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Guillem Arias <accounts@gariasf.com>
Co-authored-by: Guillem Arias Fauste <gariasf@proton.me>
2026-05-10 22:13:57 +02:00
ghost
911aa34ba9 feat(auth): add WebAuthn MFA credentials (#1628)
* feat(auth): add WebAuthn MFA credentials

* fix(auth): harden WebAuthn MFA review paths

* fix(auth): polish WebAuthn error handling

* fix(auth): handle duplicate WebAuthn credential races

* fix(auth): permit WebAuthn credential params

* fix(auth): trim WebAuthn registration controller cleanup

* fix(auth): tighten WebAuthn MFA handling

* fix(auth): pin WebAuthn relying party config
2026-05-03 22:13:28 +02:00
ghost
6c84fc760e fix(mercury): support named multiple API connections (#1627)
* fix(mercury): support named multiple connections

* fix(mercury): address multi-connection review feedback

* fix(mercury): localize connection labels

* fix(mercury): strip API tokens before provider calls

* test(mercury): localize provider config assertions

* fix(mercury): address multi-connection review

* refactor(mercury): simplify connection selection failure
2026-05-03 10:56:31 +02:00
David Gil
7f17fbf6da security: sanitize exception messages in v1 API responses (FIX-11) (#1521)
* fix(security): sanitize exception messages in API responses (FIX-11)

Replace raw e.message/error.message interpolations in response bodies
with generic error strings, and log class+message server-side. Prevents
leaking internal exception details (stack traces, SQL fragments, record
data) to API clients.

Covers:
- API v1 accounts, categories (index/show), holdings, sync, trades,
  transactions (index/show/create/update/destroy), valuations
  (show/create/update): replace "Error: #{e.message}" with
  "An unexpected error occurred".
- API v1 auth: device-registration rescue paths now log
  "[Auth] Device registration failed: ..." and respond with
  "Failed to register device".
- WebhooksController#plaid and #plaid_eu: log full error and respond
  with "Invalid webhook".
- Settings::ProvidersController: generic user-facing flash alert,
  detailed log line with error class + message.

Updates providers_controller_test assertion to match sanitized flash.

* fix(security): address CodeRabbit review

Major — partial-commit on device registration failure:
- Strengthened valid_device_info? to also run MobileDevice's model
  validations up-front (device_type inclusion, attribute presence), not
  just a flat "are the keys present?" check. A client that sends a bad
  device_type ("windows", etc.) is now rejected at the API boundary
  BEFORE signup commits any user/family/invite state.
- Wrapped the signup path (user.save + InviteCode.claim + MobileDevice
  upsert + token issuance) in ActiveRecord::Base.transaction. A
  post-save RecordInvalid from device registration (e.g., racing
  uniqueness on device_id) now rolls back the user/invite/family so
  clients don't see a partial-account state.
- Rescue branch logs the exception class + message ("#{e.class} - #{e.message}")
  for better postmortem debugging, matching the providers controller
  pattern.

Nit:
- Tightened providers_controller_test log expectation regex to assert on
  both the exception class name AND the message ("StandardError - Database
  error"), so a regression that drops either still fails the test.

Tests:
- New: "should reject signup with invalid device_type before committing
  any state" — POST /api/v1/auth/signup with device_type="windows"
  returns 400 AND asserts no User, MobileDevice, or Doorkeeper::AccessToken
  row was created.

Note on SSO path (sso_exchange → issue_mobile_tokens, lines 173/225): the
device_info in those flows comes from Rails.cache (populated by an earlier
request that already passed valid_device_info?), so the pre-validation
covers it indirectly. Wrapping the full SSO account creation (user +
invitation + OidcIdentity + issue_mobile_tokens) in one transaction would
be a meaningful architectural cleanup but is out of scope for this
error-hygiene PR — filed it as a mental note for a follow-up.
2026-04-19 18:38:23 +02:00
Sophtron Rocky
b32e9dbc45 Add Sophtron Provider (#596)
* Add Sophtron Provider

* fix syncer test issue

* fix schema  wrong merge

* sync #588

* sync code for #588

* fixed a view issue

* modified by comment

* modified

* modifed

* modified

* modified

* fixed a schema issue

* use global subtypes

* add some locales

* fix a safe_return_to_path

* fix exposing raw exception messages issue

* fix a merged issue

* update schema.rb

* fix a schema issue

* fix some issue

* Update bank sync controller to reflect beta status

Signed-off-by: Juan José Mata <jjmata@jjmata.com>

* Rename settings section title to 'Sophtron (alpha)'

Signed-off-by: Juan José Mata <jjmata@jjmata.com>

* Consistency in alpha/beta for Sophtron

* Good PR suggestions from CodeRabbit

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Signed-off-by: Sophtron Rocky <rocky@sophtron.com>
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-04-19 11:16:04 +02:00
Juan José Mata
7b2b1dd367 Rebase PR #784 and fix OpenAI model/chat regressions (#1384)
* Wire conversation history through OpenAI responses API

* Fix RuboCop hash brace spacing in assistant tests

* Pipelock ignores

* Batch fixes

---------

Co-authored-by: sokiee <sokysrm@gmail.com>
2026-04-15 18:45:24 +02:00
soky srm
7908f7d8a4 Expand financial providers (#1407)
* Initial implementation

* Tiingo fixes

* Adds 2 providers, remove 2

* Add  extra checks

* FIX a big hotwire race condition

// Fix hotwire_combobox race condition: when typing quickly, a slow response for
// an early query (e.g. "A") can overwrite the correct results for the final query
// (e.g. "AAPL"). We abort the previous in-flight request whenever a new one fires,
// so stale Turbo Stream responses never reach the DOM.

* pipelock

* Update price_test.rb

* Reviews

* i8n

* fixes

* fixes

* Update tiingo.rb

* fixes

* Improvements

* Big revamp

* optimisations

* Update 20260408151837_add_offline_reason_to_securities.rb

* Add missing tests, fixes

* small rank tests

* FIX tests

* Update show.html.erb

* Update resolver.rb

* Update usd_converter.rb

* Update holdings_controller.rb

* Update holdings_controller.rb

* Update holdings_controller.rb

* Update holdings_controller.rb

* Update holdings_controller.rb

* Update _yahoo_finance_settings.html.erb
2026-04-09 18:33:59 +02:00
Juan José Mata
8970211476 LunchFlow referral codes 2026-03-31 15:27:38 +02:00
soky srm
65f439e4da Add appearance setting for 2 column layout (#1285) 2026-03-24 19:19:05 +01:00
soky srm
0cda69ebb0 Split UI (#1245)
* Initial split transaction support

* Add support to unsplit and edit split

* Update show.html.erb

* FIX address reviews

* Improve UX

* Update show.html.erb

* Reviews

* Update edit.html.erb

* Add parent category to dialog

* Update en.yml

* Add UI indication to totals

* FIX ui update

* Add category select like rest of app

* Add split ui

* Add settings configuration for split transactions

- Adds a new settings section for appearance changes
- Also adds extra checks for delete and API calls
- Also adds checks for parent/child changes

* fixes

- split transactions dark mode fix
- add split transactions to context menu

* Update entry.rb

1. New validation split_child_date_matches_parent — prevents saving a split child with a date different from its parent. This is the root-cause fix that
   protects all flows at once.
  2. Bulk update guard — bulk_update! now strips :date from attributes when processing split children, preventing the validation from raising and silently
   skipping the date change instead.

* N+1 fix for split_parent?

* Update entry.rb

  Problem: In bulk_update!, when a split child has :date removed from attrs (line 432) and the remaining attrs is empty (e.g., the bulk update only
  changed the date), entry.update! {} still ran as a no-op. But lock_saved_attributes! and mark_user_modified! at lines 443-444 executed unconditionally,
  incorrectly marking untouched split children as user-modified and opting them out of future syncs.

  Fix:
  1. Added a changed flag to track whether any actual modification happened
  2. Wrapped entry.update! in an if attrs.present? check so no-op updates are skipped
  3. Gated lock_saved_attributes! and mark_user_modified! behind if changed, so they only run when the entry was actually modified (either via attribute
  update or tag update)

* fixes

1. Indentation in show.html.erb Settings section — The split button block and delete block had extra indentation making them appear nested inside guard
  blocks they weren't part of. Fixed to match actual nesting.
  2. Skip @split_parents query when grouping is off — The controller now only loads split parent entries when show_split_grouped? is true, saving a query
  with joins when the feature is disabled.
2026-03-22 12:02:58 +01:00
Juan José Mata
7ae9077935 Add default family selection for invite-only onboarding mode (#1174)
* Add default family selection for invite-only onboarding mode

When onboarding is set to invite-only, admins can now choose a default
family that new users without an invitation are automatically placed into
as members, instead of creating a new family for each signup.

https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx

* Restrict invite codes and onboarding settings to super_admin only

The Invite Codes section on /settings/hosting was visible to any
authenticated user via the show action, leaking all family names/IDs
through the default-family dropdown. This tightens access:

- Hide the entire Invite Codes section in the view behind super_admin?
- Add before_action :ensure_super_admin to InviteCodesController for
  all actions (index, create, destroy), replacing the inline admin? check
- Add ensure_super_admin_for_onboarding filter on hostings#update that
  blocks non-super_admin users from changing onboarding_state or
  invite_only_default_family_id

https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx

* Fix tests for super_admin-only invite codes and onboarding settings

- Hostings controller test: sign in as sure_support_staff (super_admin)
  for the onboarding_state update test, since ensure_super_admin_for_onboarding
  now requires super_admin role
- Invite codes tests: use super_admin fixture for the success case and
  verify that a regular admin gets redirected instead of raising StandardError

https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx

* Fix system test to use super_admin for self-hosting settings

The invite codes section is now only visible to super_admin users,
so the system test needs to sign in as sure_support_staff to find
the onboarding_state select element.

https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx

* Skip invite code requirement when a default family is configured

When onboarding is invite-only but a default family is set, the
claim_invite_code before_action was blocking registration before
the create action could assign the user to the default family.
Now invite_code_required? returns false when
invite_only_default_family_id is present, allowing codeless
signups to land in the configured default family.

https://claude.ai/code/session_01U9KgikKjV6xbyBZ5wMYsYx

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-10 18:12:53 +01:00
LPW
84bfe5b7ab Add external AI assistant with Pipelock security proxy (#1069)
* feat(helm): add Pipelock ConfigMap, scanning config, and consolidate compose

- Add ConfigMap template rendering DLP, response scanning, MCP input/tool
  scanning, and forward proxy settings from values
- Mount ConfigMap as /etc/pipelock/pipelock.yaml volume in deployment
- Add checksum/config annotation for automatic pod restart on config change
- Gate HTTPS_PROXY/HTTP_PROXY env injection on forwardProxy.enabled (skip
  in MCP-only mode)
- Use hasKey for all boolean values to prevent Helm default swallowing false
- Single source of truth for ports (forwardProxy.port/mcpProxy.port)
- Pipelock-specific imagePullSecrets with fallback to app secrets
- Merge standalone compose.example.pipelock.yml into compose.example.ai.yml
- Add pipelock.example.yaml for Docker Compose users
- Add exclude-paths to CI workflow for locale file false positives

* Add external assistant support (OpenAI-compatible SSE proxy)

Allow self-hosted instances to delegate chat to an external AI agent
via an OpenAI-compatible streaming endpoint. Configurable per-family
through Settings UI or ASSISTANT_TYPE env override.

- Assistant::External::Client: SSE streaming HTTP client (no new gems)
- Settings UI with type selector, env lock indicator, config status
- Helm chart and Docker Compose env var support
- 45 tests covering client, config, routing, controller, integration

* Add session key routing, email allowlist, and config plumbing

Route to the actual OpenClaw session via x-openclaw-session-key header
instead of creating isolated sessions. Gate external assistant access
behind an email allowlist (EXTERNAL_ASSISTANT_ALLOWED_EMAILS env var).
Plumb session_key and allowedEmails through Helm chart, compose, and
env template.

* Add HTTPS_PROXY support to External::Client for Pipelock integration

Net::HTTP does not auto-read HTTPS_PROXY/HTTP_PROXY env vars (unlike
Faraday). Explicitly resolve proxy from environment in build_http so
outbound traffic to the external assistant routes through Pipelock's
forward proxy when enabled. Respects NO_PROXY for internal hosts.

* Add UI fields for external assistant config (Setting-backed with env fallback)

Follow the same pattern as OpenAI settings: database-backed Setting
fields with env var defaults. Self-hosters can now configure the
external assistant URL, token, and agent ID from the browser
(Settings > Self-Hosting > AI Assistant) instead of requiring env vars.
Fields disable when the corresponding env var is set.

* Improve external assistant UI labels and add help text

Change placeholder to generic OpenAI-compatible URL pattern. Add help
text under each field explaining where the values come from: URL from
agent provider, token for authentication, agent ID for multi-agent
routing.

* Add external assistant docs and fix URL help text

Add External AI Assistant section to docs/hosting/ai.md covering setup
(UI and env vars), how it works, Pipelock security scanning, access
control, and Docker Compose example. Drop "chat completions" jargon
from URL help text.

* Harden external assistant: retry logic, disconnect UI, error handling, and test coverage

- Add retry with backoff for transient network errors (no retry after streaming starts)
- Add disconnect button with confirmation modal in self-hosting settings
- Narrow rescue scope with fallback logging for unexpected errors
- Safe cleanup of partial responses on stream interruption
- Gate ai_available? on family assistant_type instead of OR-ing all providers
- Truncate conversation history to last 20 messages
- Proxy-aware HTTP client with NO_PROXY support
- Sanitize protocol to use generic headers (X-Agent-Id, X-Session-Key)
- Full test coverage for streaming, retries, proxy routing, config, and disconnect

* Exclude external assistant client from Pipelock scan-diff

False positive: `@token` instance variable flagged as "Credential in URL".
Temporary workaround until Pipelock supports inline suppression.

* Address review feedback: NO_PROXY boundary fix, SSE done flag, design tokens

- Fix NO_PROXY matching to require domain boundary (exact match or .suffix),
  case-insensitive. Prevents badexample.com matching example.com.
- Add done flag to SSE streaming so read_body stops after [DONE]
- Move MAX_CONVERSATION_MESSAGES to class level
- Use bg-success/bg-destructive design tokens for status indicators
- Add rationale comment for pipelock scan exclusion
- Update docs last-updated date

* Address second round of review feedback

- Allowlist email comparison is now case-insensitive and nil-safe
- Cap SSE buffer at 1 MB to prevent memory blowup from malformed streams
- Don't expose upstream HTTP response body in user-facing errors (log it instead)
- Fix frozen string warning on buffer initialization
- Fix "builtin" typo in docs (should be "built-in")

* Protect completed responses from cleanup, sanitize error messages

- Don't destroy a fully streamed assistant message if post-stream
  metadata update fails (only cleanup partial responses)
- Log raw connection/HTTP errors internally, show generic messages
  to users to avoid leaking network/proxy details
- Update test assertions for new error message wording

* Fix SSE content guard and NO_PROXY test correctness

Use nil check instead of present? for SSE delta content to preserve
whitespace-only chunks (newlines, spaces) that can occur in code output.

Fix NO_PROXY test to use HTTP_PROXY matching the http:// client URL so
the proxy resolution and NO_PROXY bypass logic are actually exercised.

* Forward proxy credentials to Net::HTTP

Pass proxy_uri.user and proxy_uri.password to Net::HTTP.new so
authenticated proxies (http://user:pass@host:port) work correctly.
Without this, credentials parsed from the proxy URL were silently
dropped. Nil values are safe as positional args when no creds exist.

* Update pipelock integration to v0.3.1 with full scanning config

Bump Helm image tag from 0.2.7 to 0.3.1. Add missing security
sections to both the Helm ConfigMap and compose example config:
mcp_tool_policy, mcp_session_binding, and tool_chain_detection.
These protect the /mcp endpoint against tool injection, session
hijacking, and multi-step exfiltration chains.

Add version and mode fields to config files. Enable include_defaults
for DLP and response scanning to merge user patterns with the 35
built-in patterns. Remove redundant --mode CLI flag from the Helm
deployment template since mode is now in the config file.
2026-03-03 15:47:51 +01:00
Juan José Mata
705b5a8b26 First cut of a simplified "intro" UI layout (#265)
* First cut of a simplified "intro" UI layout

* Linter

* Add guest role and intro-only access

* Fix guest role UI defaults (#940)

Use enum predicate to avoid missing role helper.

* Remove legacy user role mapping (#941)

Drop the unused user role references in role normalization
and SSO role mapping forms to avoid implying a role that
never existed.

Refs: #0

* Remove role normalization (#942)

Remove role normalization

Roles are now stored directly without legacy mappings.

* Revert role mapping logic

* Remove `normalize_role_settings`

* Remove unnecessary migration

* Make `member` the default

* Broken `.erb`

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
2026-02-09 11:09:25 +01:00
David Gil
ba442d5f26 Implement Indexa Capital provider with real API integration (#933)
* Add Indexa Capital provider scaffold

Generate Indexa Capital provider scaffolding and align credential fields with the API authentication requirements.

* Fix PR 926 lint and schema CI failures

* Implement Indexa Capital provider with real API integration

- Rewrite all broken view templates (were meta-ERB from code generator)
- Create missing select_accounts.html.erb template
- Implement real API calls: list_accounts via /users/me, get_holdings
  via /accounts/{number}/fiscal-results, get_account_balance via
  /accounts/{number}/performance
- Add API token auth support (stored token > env token > credentials)
- Add api_token column with encryption support
- Redesign settings panel: API token prominent, credentials collapsible
- Fix account balances display using performance endpoint portfolios
- Fix accounts index empty-state guard missing indexa_capital_items
- Simplify activities fetch job (no activities API endpoint exists)
- Fix i18n interpolation (%%{ -> %{) throughout locale file

* Add tests for Indexa Capital provider integration

- IndexaCapitalItemTest: validations, credentials, scopes, sync status
- IndexaCapitalAccountTest: upsert, holdings, account provider linking
- Provider::IndexaCapitalTest: auth modes, API stubs, error handling
- IndexaCapitalItemsControllerTest: CRUD, setup, linking, authorization
- Fixtures for items (token + credentials) and accounts (mutual + pension)

52 tests, 98 assertions, 0 failures

* Address code review feedback from PR #933

- Fix zero balance bug: use `nil?` instead of `present?` so 0 is stored
- Fix has_indexa_capital_credentials? to check api_token (was ignored)
- Fix build_provider to delegate to Provided concern (was ignoring token)
- Fix IndexaCapital section outside encryption_error guard in settings
- Add account_number sanitization to prevent path traversal in API URLs
- Replace all skipped processor tests with real working tests
- Add zero-balance and path-traversal test coverage

61 tests, 107 assertions, 0 failures

* Address code review round 2: credentials validation, RuboCop, test quality

- Fix RuboCop SpaceInsideArrayLiteralBrackets in credentials check
- Chain where.not calls so all three username/document/password must be present
- Require all three credentials (||) instead of any one (&&) in validate_configuration!
- Move attr_reader to private to avoid exposing credentials publicly
- Parse dates with Date.parse in extract_balance for robustness
- Remove stale TODO and Crypto from supported_account_types
- Order build_provider query deterministically by created_at
- Replace no-op holdings assertion with meaningful assert_difference

* Address code review round 3: JSON parse safety and test precision

- Rescue JSON::ParserError on 2xx responses for clearer error messages
- Fix weak balance assertion: set balance to 0 before processing, assert
  expected value (27093.01 = sum of holdings amounts)

* Include Indexa Capital in automatic family sync

Add indexa_capital_items to Family::Syncer#child_syncables so balances
and holdings refresh on daily auto-sync and login sync, not only on
manual sync button clicks.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-02-08 18:19:37 +01:00
Juan José Mata
61ce5c8514 Protect demo API key from deletion (#919)
* feat: Protect demo monitoring API key from deletion

- Add DEMO_MONITORING_KEY constant to ApiKey model
- Add `demo_monitoring_key?` method to identify the monitoring key
- Add `visible` scope to exclude monitoring key from UI queries
- Update controller to use `visible` scope, hiding the monitoring key
- Prevent revocation of the monitoring key with explicit error handling
- Update Demo::Generator to use the shared constant

Users on the demo instance can still create their own API keys,
but cannot see or delete the monitoring key used for uptime checks.

https://claude.ai/code/session_01RQFsw39K7PB5kztboVdBdB

* Linter

* Protect demo monitoring API key from deletion

* Use monitoring source for demo API key

* Add test for demo monitoring revoke guard

* Disable Rack::Attack in test and development

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-06 21:25:52 +01:00
MkDev11
eeff4edbea Add warning for TwelveData plan-restricted tickers (#803)
* Add warning for TwelveData plan-restricted tickers

Fixes #800

- Add Security::PlanRestrictionTracker concern using Rails cache
- Detect plan upgrade errors during Security::Price::Importer sync
- Display amber warning on /settings/hosting with affected tickers
- Include unit tests for the new functionality

* Scope plan restriction cache by provider

Addresses review feedback:
- Cache key now includes provider name to support multiple data providers
- Methods now require provider parameter for proper scoping
- Added tests for provider-scoped restrictions
- Added documentation explaining instance-level API key architecture

* Fix RuboCop array bracket spacing

* Fix empty array bracket spacing

* Move plan upgrade detection to Provider::TwelveData

* Fix provider scoping tests to use direct cache writes

---------

Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
2026-01-27 15:45:50 +01:00
Juan José Mata
71f10c5e4a fix: Handle missing encryption keys gracefully on Poviders config page (#753)
* fix: Handle missing encryption keys gracefully on providers page

When Active Record encryption keys are not configured, the settings/providers
page would crash with an unhandled exception. This change catches the
ActiveRecord::Encryption::Errors::Configuration error and displays a
friendly error message instead, explaining that encryption credentials
need to be configured before using sync providers.

https://claude.ai/code/session_015nPsLWkr12i5ok5bwLtA7p

* Simplify rescue block

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-23 19:15:04 +01:00
Juan José Mata
e0fb585bda Hide contribution payments from demo user(s) (#738)
* Hide payment contribution options from demo and manually created users

Demo data users and manually created users don't have stripe_customer_id
set on their family, so they should not see payment/contribution options.

Changes:
- Add can_manage_subscription? method to Family::Subscribeable that checks
  for presence of stripe_customer_id
- Guard Settings::PaymentsController to return 403 for users without
  stripe_customer_id
- Guard SubscriptionsController#show action (Stripe portal redirect) for
  users without stripe_customer_id
- Update settings navigation to hide the payment link when
  stripe_customer_id is not present
- Add tests for the new behavior

* Fix broken test

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-23 12:35:49 +01:00
Juan José Mata
bf76d6b88d Let /settings/providers work in demo site 2026-01-22 20:01:18 +00:00
LPW
a83f70425f Add SnapTrade brokerage integration with full trade history support (#737)
* Introduce SnapTrade integration with models, migrations, views, and activity processing logic.

* Refactor SnapTrade activities processing: improve activity fetching flow, handle pending states, and update UI elements for enhanced user feedback.

* Update Brakeman ignore file to include intentional redirect for SnapTrade OAuth portal.

* Refactor SnapTrade models, views, and processing logic: add currency extraction helper, improve pending state handling, optimize migration checks, and enhance user feedback in UI.

* Remove encryption for SnapTrade `snaptrade_user_id`, as it is an identifier, not a secret.

* Introduce `SnaptradeConnectionCleanupJob` to asynchronously handle SnapTrade connection cleanup and improve i18n for SnapTrade item status messages.

* Update SnapTrade encryption: make `snaptrade_user_secret` non-deterministic to enhance security.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-22 20:52:49 +01:00
soky srm
179552657c Mercury integration (#723)
* Initial mercury impl

* FIX both mercury and generator class

* Finish mercury integration and provider generator

* Fix schema

* Fix linter and tags

* Update routes.rb

* Avoid schema drift

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-22 20:37:07 +01:00
LPW
dd991fa339 Add Coinbase exchange integration with CDP API support (#704)
* **Add Coinbase integration with item and account management**
- Creates migrations for `coinbase_items` and `coinbase_accounts`.
- Adds models, controllers, views, and background tasks to support account linking, syncing, and transaction handling.
- Implements Coinbase API client and adapter for seamless integration.
- Supports ActiveRecord encryption for secure credential storage.
- Adds UI components for provider setup, account management, and synchronization.

* Localize Coinbase-related UI strings, refine account linking for security, and add timeouts to Coinbase API requests.

* Localize Coinbase account handling to support native currencies (USD, EUR, GBP, etc.) across balances, trades, holdings, and transactions.

* Improve Coinbase processing with timezone-safe parsing, native currency support, and immediate holdings updates.

* Improve trend percentage formatting and enhance race condition handling for Coinbase account linking.

* Fix log message wording for orphan cleanup

* Ensure `selected_accounts` parameter is sanitized by rejecting blank entries.

* Add tests for Coinbase integration: account, item, and controller coverage

- Adds unit tests for `CoinbaseAccount` and `CoinbaseItem` models.
- Adds integration tests for `CoinbaseItemsController`.
- Introduces Stimulus `select-all` controller for UI checkbox handling.
- Localizes UI strings and logging for Coinbase integration.

* Update test fixtures to use consistent placeholder API keys and secrets

* Refine `coinbase_item` tests to ensure deterministic ordering and improve scope assertions.

* Integrate `SyncStats::Collector` into Coinbase syncer to streamline statistics collection and enhance consistency.

* Localize Coinbase sync status messages and improve sync summary test coverage.

* Update `CoinbaseItem` encryption: use deterministic encryption for `api_key` and standard for `api_secret`.

* fix schema drift

* Beta labels to lower expectations

---------

Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-21 22:56:39 +01:00
Juan José Mata
8e36c8e736 Rename billing to payment throughout the codebase (#726)
* Rename billing to payment throughout the codebase

This change updates terminology from "billing" to "payment" to better
reflect that these are contributions/payments rather than bills.

Changes include:
- Rename BillingsController to PaymentsController
- Rename billing_email to payment_email
- Rename next_billing_date to next_payment_date
- Rename create_billing_portal_session_url to create_payment_portal_session_url
- Update routes from billing to payment
- Update all 12 locale files with new terminology
- Update views, helpers, and tests

* Update app/views/subscriptions/upgrade.html.erb

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>

---------

Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-21 19:06:00 +01:00
soky srm
abab66675c Implement a setting to retrieve high res logos (#725)
* Implement a setting to retrieve high res logos

* Update _brand_fetch_settings.html.erb

* Add fallback for stock tickers also to use Brandfetch

* Update security.rb

* Update toggle logic for high-res logos setting

Signed-off-by: Juan José Mata <jjmata@jjmata.com>

* Update security.rb

* Update security.rb

---------

Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-01-21 17:16:51 +01:00
soky srm
064833621e Merge pull request #538 from luckyPipewrench/sso-upgrades
Multi-provider SSO with admin UI and SAML support
2026-01-12 15:38:59 +01:00
sokie
30923b7296 FIX address comments 2026-01-12 15:05:28 +01:00
Alessio Cappa
5b74e8d90b feat: Add parameters in setting to enable/disable auto-sync and define timing 2026-01-11 19:42:35 +01:00
LPW
3658e812a8 Add pending transaction handling and duplicate reconciliation logic (#602)
* Add pending transaction handling and duplicate reconciliation logic

- Implemented logic to exclude pending transactions from budgets and analytics calculations.
- Introduced mechanisms for reconciling pending transactions with posted versions.
- Added duplicate detection with support for merging or dismissing matches.
- Updated transaction search filters to include a `status_filter` for pending/confirmed transactions.
- Introduced UI elements for reviewing and resolving duplicates.
- Enhanced `ProviderSyncSummary` with stats for reconciled and stale pending transactions.

* Refactor translation handling and enhance transaction and sync logic

- Moved hardcoded strings to locale files for improved translation support.
- Refined styling for duplicate transaction indicators and sync summaries.
- Improved logic for excluding stale pending transactions and updating timestamps on batch exclusion.
- Added unique IDs to status filters for better element targeting in UI.
- Optimized database queries to avoid N+1 issues in stale pending calculations.

* Add sync settings and enhance pending transaction handling

- Introduced a new "Sync Settings" section in hosting settings with UI to toggle inclusion of pending transactions.
- Updated handling of pending transactions with improved inference logic for `posted=0` and `transacted_at` in processors.
- Added priority order for pending transaction inclusion: explicit argument > environment variable > runtime configurable setting.
- Refactored settings and controllers to store updated sync preferences.

* Refactor sync settings and pending transaction reconciliation

- Extracted logic for pending transaction reconciliation, stale exclusion, and unmatched tracking into dedicated methods for better maintainability.
- Updated sync settings to infer defaults from multiple provider environment variables (`SIMPLEFIN_INCLUDE_PENDING`, `PLAID_INCLUDE_PENDING`).
- Refined UI and messaging to handle multi-provider configurations in sync settings.

# Conflicts:
#	app/models/simplefin_item/importer.rb

* Debounce transaction reconciliation during imports

- Added per-run reconciliation debouncing to prevent repeated scans for the same account during chunked history imports.
- Trimmed size of reconciliation stats to retain recent details only.
- Introduced error tracking for reconciliation steps to improve UI visibility of issues.

* Apply ABS() in pending transaction queries and improve error handling

- Updated pending transaction logic to use ABS() for consistent handling of negative amounts.
- Adjusted amount bounds calculations to ensure accuracy for both positive and negative values.
- Refined exception handling in `merge_duplicate` to log failures and update user alert.
- Replaced `Date.today` with `Date.current` in tests to ensure timezone consistency.
- Minor optimization to avoid COUNT queries by loading limited records directly.

* Improve error handling in duplicate suggestion and dismissal logic

- Added exception handling for `store_duplicate_suggestion` to log failures and prevent crashes during fuzzy/low-confidence matches.
- Enhanced `dismiss_duplicate` action to handle `ActiveRecord::RecordInvalid` and display appropriate user alerts.

---------

Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
2026-01-10 20:11:00 +01:00
Josh Waldrep
238fa8e0ca Merge remote-tracking branch 'upstream/main' into sso-upgrades
# Conflicts:
#	app/views/simplefin_items/_simplefin_item.html.erb
#	db/schema.rb
2026-01-10 11:57:23 -05:00
Ethan
3b4ab735b0 Add (beta) CoinStats Crypto Wallet Integration with Balance and Transaction Syncing (#512)
* Feat(CoinStats): Scaffold implementation, not yet functional

* Feat(CoinStats): Implement crypto wallet balance and transactions

* Feat(CoinStats): Add tests, Minor improvements

* Feat(CoinStats): Utilize bulk fetch API endpoints

* Feat(CoinStats): Migrate strings to i8n

* Feat(CoinStats): Fix error handling in wallet link modal

* Feat(CoinStats): Implement hourly provider sync job

* Feat(CoinStats): Generate docstrings

* Fix(CoinStats): Validate API Key on provider update

* Fix(Providers): Safely handle race condition in merchance creation

* Fix(CoinStats): Don't catch system signals in account processor

* Fix(CoinStats): Preload before iterating accounts

* Fix(CoinStats): Add no opener / referrer to API dashboard link

* Fix(CoinStats): Use strict matching for symbols

* Fix(CoinStats): Remove dead code in transactions importer

* Fix(CoinStats): Avoid transaction fallback ID collisions

* Fix(CoinStats): Improve Blockchains fetch error handling

* Fix(CoinStats): Enforce NOT NULL constraint for API Key schema

* Fix(CoinStats): Migrate sync status strings to i8n

* Fix(CoinStats): Use class name rather than hardcoded string

* Fix(CoinStats): Use account currency rather than hardcoded USD

* Fix(CoinStats): Migrate from standalone to Provider class

* Fix(CoinStats): Fix test failures due to string changes
2026-01-07 15:59:04 +01:00
Josh Waldrep
d3055b2e0b refactor: remove SSO settings page; consolidate SSO identity management under Security settings
- Removed the `Settings::SsoIdentitiesController` and views for a simplified user experience.
- Moved SSO identity management to the Security settings page (`Settings::SecuritiesController`).
- Updated locale keys and layout for the new structure.
- Fixed unlink protection warnings and adjusted redirection path.
- Cleaned up routes, helper methods, and redundant code.
2026-01-03 20:49:31 -05:00
Josh Waldrep
14993d871c feat: comprehensive SSO/OIDC upgrade with enterprise features
Multi-provider SSO support:
   - Database-backed SSO provider management with admin UI
   - Support for OpenID Connect, Google OAuth2, GitHub, and SAML 2.0
   - Flipper feature flag (db_sso_providers) for dynamic provider loading
   - ProviderLoader service for YAML or database configuration

   Admin functionality:
   - Admin::SsoProvidersController for CRUD operations
   - Admin::UsersController for super_admin role management
   - Pundit policies for authorization
   - Test connection endpoint for validating provider config

   User provisioning improvements:
   - JIT (just-in-time) account creation with configurable default role
   - Changed default JIT role from admin to member (security)
   - User attribute sync on each SSO login
   - Group/role mapping from IdP claims

   SSO identity management:
   - Settings::SsoIdentitiesController for users to manage connected accounts
   - Issuer validation for OIDC identities
   - Unlink protection when no password set

   Audit logging:
   - SsoAuditLog model tracking login, logout, link, unlink, JIT creation
   - Captures IP address, user agent, and metadata

   Advanced OIDC features:
   - Custom scopes per provider
   - Configurable prompt parameter (login, consent, select_account, none)
   - RP-initiated logout (federated logout to IdP)
   - id_token storage for logout

   SAML 2.0 support:
   - omniauth-saml gem integration
   - IdP metadata URL or manual configuration
   - Certificate and fingerprint validation
   - NameID format configuration
2026-01-03 17:56:42 -05:00
soky srm
88952e4714 Small llms improvements (#400)
* Initial implementation

* FIX keys

* Add langfuse evals support

* FIX trace upload

* Delete .claude/settings.local.json

Signed-off-by: soky srm <sokysrm@gmail.com>

* Update client.rb

* Small LLMs improvements

* Keep batch size normal

* Update categorizer

* FIX json mode

* Add reasonable alternative to matching

* FIX thinking blocks for llms

* Implement json mode support with AUTO mode

* Make auto default for everyone

* FIX linter

* Address review

* Allow export manual categories

* FIX user export

* FIX oneshot example pollution

* Update categorization_golden_v1.yml

* Update categorization_golden_v1.yml

* Trim to 100 items

* Update auto_categorizer.rb

* FIX for auto retry in auto mode

* Separate the Eval Logic from the Auto-Categorizer

The expected_null_count parameter conflates eval-specific logic with production categorization logic.

* Force json mode on evals

* Introduce a more mixed dataset

150 items, performance from a local model:

By Difficulty:
  easy: 93.22% accuracy (55/59)
  medium: 93.33% accuracy (42/45)
  hard: 92.86% accuracy (26/28)
  edge_case: 100.0% accuracy (18/18)

* Improve datasets

Remove Data leakage from prompts

* Create eval runs as "pending"

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-12-07 18:11:34 +01:00
soky srm
4a29d030af Initial enable banking implementation (#382)
* Initial enable banking implementation

* Handle multiple connections

* Amount fixes

* Account type mapping

* Add option to skip accounts

* Update schema.rb

* Transaction fixes

* Provider fixes

* FIX account identifier

* FIX support unlinking

* UI style fixes

* FIX safe redirect and brakeman issue

* FIX

- pagination max fix
- wrap crud in transaction logic

* FIX api uid access

- The Enable Banking API expects the UUID (uid from the API response) to fetch balances/transactions, not the identification_hash

* FIX add new connection

* FIX erb code

* Alert/notice box overflow protection

* Give alert/notification boxes room to grow (3 lines max)

* Add "Enable Banking (beta)" to `/settings/bank_sync`

* Make Enable Banking section collapsible like all others

* Add callback hint to error message

---------

Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-11-29 13:31:08 +01:00
soky srm
91a91c3834 Improvements (#379)
* Improvements

- Fix button visibility in reports on light theme
- Unify logic for provider syncs
- Add default option is to skip accounts linking ( no op default )

* Stability fixes and UX improvements

* FIX add unlinking when deleting lunch flow connection as well

* Wrap updates in transaction

* Some more improvements

* FIX proper provider setup check

* Make provider section collapsible

* Fix balance calculation

* Restore focus ring

* Use browser default focus

* Fix lunch flow balance for credit cards
2025-11-25 20:21:29 +01:00
soky srm
be0b20dfd9 Lunchflow settings family (#363)
* Move provider config to family

* remove global settings

* Remove turbo auto  submit

* Fix flash location

* Fix mssing syncer for lunchflow

* Update schema.rb

* FIX tests and encryption config

* FIX make rabbit happy

* FIX run migration in SQL

* FIX turbo frame modal

* Branding fixes

* FIX rabbit

* OCD with product names

* More OCD

* No other console.log|warn in codebase

---------

Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-11-22 02:14:29 +01:00
LPW
61eb611529 Simplefin enhancements v2 (#267)
* SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint

# Conflicts:
#	db/schema.rb

# Conflicts:
#	app/controllers/simplefin_items_controller.rb

* fix testing

* fix linting

* xfix linting x2

* Review PR #267 on we-promise/sure (SimpleFin enhancements v2). Address all 15 actionable CodeRabbit comments: Add UUID validations in rakes (e.g., simplefin_unlink), swap Ruby pattern matching/loops for efficient DB queries (e.g., where LOWER(name) LIKE ?), generate docstrings for low-coverage areas (31%), consolidate routes for simplefin_items, move view logic to helpers (e.g., format_transaction_extra), strengthen tests with exact assertions/fixtures for dedup/relink failures. Also, check for overlaps with merged #262 (merchants fix): Ensure merchant creation in simplefin_entry/processor.rb aligns with new payee-based flow and MD5 IDs; add tests for edge cases like empty payees or over-merging pendings. Prioritize security (PII redaction in logs, no hardcoded secrets).

* SimpleFin: address CodeRabbit comments (batch 1)

- Consolidate simplefin_items routes under a single resources block; keep URLs stable
- Replace inline JS with Stimulus auto-relink controller; auto-load relink modal via global modal frame
- Improve a11y in relink modal by wrapping rows in labels
- Harden unlink rake: default dry_run=true, UUID validation, redact PII in outputs, clearer errors
- Backfill rake: default dry_run=true, UUID validation; groundwork for per-SFA counters
- Fix-was-merged rake: default dry_run=true, UUID validation; clearer outputs
- Idempotent transfer auto-match (find_or_create_by! + RecordNotUnique rescue)
- Extract SimpleFin error tooltip assembly into helper and use it in view

RuboCop: maintain 2-space indentation, spaces inside array brackets, spaces after commas, and no redundant returns

* Linter noise

* removed filed commited by mistake.

* manual relink flow and tighten composite matching

* enforce manual relink UI; fix adapter keywords; guarantee extra.simplefin hash

* refactor(simplefin): extract relink service; enforce manual relink UI; tighten composite match; migration 7.2

* add provider date parser; refactor rake; move view queries; partial resilience

* run balances-only import in background job. make update flow enqueue balances-only job

* persists across all update redirects and initialize
used_manual_ids to prevent NameError in relink candidate computation.

* SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint

* Fixed failed test after rebase.

* scan_ruby fix

* Calming the rabbit:
Fix AccountProvider linking when accounts change
Drop the legacy unique index instead of duplicating it
Fix dynamic constant assignment
Use fixtures consistently; avoid rescue for control flow.
Replace bare rescue with explicit exception class.
Move business logic out of the view.
Critical: Transaction boundary excludes recompute phase, risking data loss.
Inconsistency between documentation and implementation for zero-error case.
Refactor to use the compute_unlinked_count helper for consistency.
Fix cleanup task default: it deletes by default.
Move sync stats computation to controller to avoid N+1 queries.
Consolidate duplicate sync query.
Clarify the intent of setting flash notice on the error path.
Fix Date/Time comparison in should_be_inactive?.
Move stats retrieval logic to controller.
Remove duplicate Sync summary section.
Remove the unnecessary sleep statement; use Capybara's built-in waiting.
Add label wrappers for accessibility and consistency.

* FIX SimpleFIN new account modal

Now new account properly loads as a Modal, instead of new page.
Fixes also form showing dashboard instead of settings page.

* Remove SimpleFin legacy UI components, migrate schema, and refine linking behavior.

# Conflicts:
#	app/helpers/settings_helper.rb

* Extract SimpleFin-related logic to `prepare_show_context` helper and refactor for consistency. Adjust conditional checks and ensure controller variables are properly initialized.

* Remove unused SimpleFin maps from prepare_show_context; select IDs to avoid N+1
Replace Tailwind bg-green-500 with semantic bg-success in _simplefin_panel/_provider_form
Add f.label :setup_token in simplefin_items/new for a11y
Remove duplicate require in AccountsControllerSimplefinCtaTest

* Remove unnecessary blank lines

* Reduce unnecessary changes

This reduces the diff against main

* Simplefin Account Setup: Display in modal

This fixes an issue with the `X` dismiss button in the top right corner

* Removed unnecessary comment.

* removed unnecessary function.

* fixed broken links

* Removed unnecessary file

* changed to database query

* set to use UTC and gaurd against null

* set dry_run=true

* Fixed comment

* Changed to use a database-level query

* matched test name to test behavior.

* Eliminate code duplication and Time.zone dependency

* make final summary surface failures

* lint fix

* Revised timezone comment. better handle missing selectors.

* sanitized LIKE wildcards

* Fixed SimpleFin import to avoid “Currency can’t be blank” validation failures when providers return an empty currency string.

* Added helper methods for admin and self-hosted checks

* Specify exception types in rescue clauses.

* Refined logic to determine transaction dates for credit accounts.

* Refined stats calculation for `total_accounts` to track the maximum unique accounts per run instead of accumulating totals.

* Moved `unlink_all!` logic to `SimplefinItem::Unlinking` concern and deprecated `SimplefinItem::Unlinker`. Updated related references.

* Refined legacy unlinking logic, improved `current_holdings` formatting, and added ENV-based overrides for self-hosted checks.

* Enhanced `unlink_all!` with explicit error handling, improved transaction safety, and refined ENV-based self-hosted checks. Adjusted exception types and cleaned up private method handling.

* Improved currency assignment logic by adding fallback to `current_account` and `family` currencies.

* Enhanced error tracking during SimpleFin account imports by adding categorized error buckets, limiting stored errors to the last 5, and improving `stats` calculations.

* typo fix

* Didn't realize rabbit was still mad...
Refactored SimpleFin error handling and CTA logic: centralized duplicate detection and relink visibility into controller, improved task counters, adjusted redirect notices, and fixed form indexing.

* Dang rabbit never stops... Centralized SimpleFin maps logic into `MapsHelper` concern and integrated it into relevant controllers and rake tasks. Optimized queries, reduced redundancy, and improved unlinked counts and manual account checks with batch processing. Adjusted task arguments for clarity.

* Persistent rabbit. Optimized SimpleFin maps logic by implementing batch queries for manual account and unlinked count checks, reducing N+1 issues. Improved clarity of rake task argument descriptions and error messages for better usability.

* Lost a commit somehow, resolved here. Refactored transaction extra details logic by introducing `build_transaction_extra_details` helper to improve clarity, reusability, and reduce view complexity. Enhanced rake tasks with strict dry-run validation and better error handling. Updated schema to allow nullable `merchant_id` and added conditional unique indexes for recurring transactions.

* Refactored sensitive data redaction in `simplefin_unlink` task for recursive handling, optimized SQL sanitization in `simplefin_holdings_backfill`, improved error handling in `transactions_helper`, and streamlined day change calculation logic in `Holding` model.

* Lint fix

* Removed per PR comments.

* Also removing per PR comment.

* git commit -m "SimpleFIN polish: preserve #manual-accounts wrapper, unify \"manual\" scope, and correct unlinked counts
- Preserve #manual-accounts wrapper: switch non-empty updates to turbo_stream.update and background broadcast_update_to; keep empty-path replace to render <div id=\"manual-accounts\"></div>
- Unify definition of manual accounts via Account.visible_manual (visible + legacy-nil + no AccountProvider); reuse in controllers, jobs, and helper
- Correct setup/unlinked counts: SimplefinItem::Syncer#finalize_setup_counts and maps now consider AccountProvider links (legacy account AND provider must be absent)
Deleted:
- app/models/simplefin_item/relink_service.rb
- app/controllers/concerns/simplefin_items/relink_helpers.rb
- app/javascript/controllers/auto_relink_controller.js
- app/views/simplefin_items/_relink_modal.html.erb
- app/views/simplefin_items/manual_relink.html.erb
- app/views/simplefin_items/relink.html.erb
- test/services/simplefin_item/relink_service_test.rb
Refs: PR #318 unified link/unlink; PR #267 SimpleFIN; follow-up to fix wrapper ID loss and counting drift."

* Extend unlinked account check to include "Investment" type

* set SimpleFIN item for `balances`, remove redundant unpacking, and improve holdings task error

* SimpleFIN: add `errors` action + modal; do not reintroduce legacy relink actions; removed dead helper

* FIX simpleFIN linking

* Add delay back, tests benefit from it

* Put cache back in

* Remove empty `rake` task

* Small spelling fixes.

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: sokie <sokysrm@gmail.com>
Co-authored-by: Dylan Corrales <deathcamel58@gmail.com>
2025-11-17 21:51:37 +01:00
soky srm
0d798c522a Add support for provider selection (#346) 2025-11-17 18:13:20 +01:00
soky srm
fad241c416 Fixes & Improvements (#316)
* Some improvements

- Fix issue with lunch flow accounts that were imported
- Remove the period comparison section from reports

* Add cleanup migration

* FIX for dynamic config

* Fix linter

* FIX settings setter

Reuse the base class’ atomic setter to leverage its locking and cache invalidation.

* Make upsert atomic

* Remove migration file

Signed-off-by: soky srm <sokysrm@gmail.com>

* Delete db/migrate/20251111094448_migrate_dynamic_fields_to_individual_entries.rb

Signed-off-by: soky srm <sokysrm@gmail.com>

* Fix cache reset

* Revert "Remove migration file"

This reverts commit 1f2a21ef58.

* Revert "Delete db/migrate/20251111094448_migrate_dynamic_fields_to_individual_entries.rb"

This reverts commit 29dcaaafb2.

* Fix Plaid initialiser

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
2025-11-11 19:51:07 +01:00
soky srm
164fe5375d FIX Read-Modify-Write issue with dynamic fields (#290)
* FIX Read-Modify-Write issue with dynamic fields

Ruby caching + queueing updates might cause some dynamic fields to not be updated.

* Small fix for true dynamic fields

* Add suite of tests for new settings page

* Treat nil values as deletions to keep the hash clean

* Test fix
2025-11-05 12:19:33 +01:00
soky srm
96713ee8b4 Add support for dynamic config UI (#256)
* Add support for dynamic config UI

* Add support for section description

* Better dynamic class settings

Added dynamic_fields hash field - Stores all undeclared settings
[] method - Checks declared fields first, then falls back to dynamic hash
[]= method - Updates declared fields normally, stores others in hash
No runtime field declaration - Fields are never dynamically created on the class

* FIX proper lookup for provider keys

- Also validate configurable values properly.
- Change Provider factory to use Rails autoloading (Zeitwerk)

* Fix factory

The derive_adapter_name method relies on string manipulation ("PlaidAccount".sub(/Account$/, "") + "Adapter" → "PlaidAdapter"), but we already have explicit registration in place.

* Make updates atomic, field-aware, and handle blanks explicitly

* Small UX detail

* Add support for PlaidEU in UI also

- This looks like partial support atm
2025-10-29 13:11:04 +01:00
João Felipe
9fefe57de5 Feature/yahoo finance (#123)
* Implement Yahoo Finance

* Added tests

* Updated hosting controller to check for managed app_mode instead of env_override

* Suggestions from CodeRabbit and Fixes on tests

* Remove Css changes

* Fix yahoo finance impl and i18n

* Updated view to use healthy method

* remove usage

* Updated env example

* keep usage on class just to keep same format

* Ci test

* Remove some useless validations

* Remove logs

* Linter fixes

* Broke this in my conflict merge

* Wrong indentation level

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-10-28 23:15:14 +01:00
Juan José Mata
72e7d7736b Add onboarding state selector for self-hosted signup (#251)
* Add onboarding modes to self-hosted signup

* Style form consistently

* Configure ONBOARDING_STATE via ENV
2025-10-27 21:52:37 +01:00
Juan José Mata
768e85ce08 Add OpenID Connect login support (#77)
* Add OpenID Connect login support
* Add docs for OIDC config with Google Auth
* Use Google styles for log in
- Add support for linking existing account
- Force users to sign-in with passoword first, when linking existing accounts
- Add support to create new user when using OIDC
- Add identities to user to prevent account take-ver
- Make tests mocking instead of being integration tests
- Manage session handling correctly
- use OmniAuth.config.mock_auth instead of passing auth data via request env
* Conditionally render Oauth button

- Set a config item `configuration.x.auth.oidc_enabled`
- Hide button if disabled

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: sokie <sokysrm@gmail.com>
2025-10-24 16:07:45 +02:00
soky srm
bb364fab38 LLM cost estimation (#223)
* Password reset back button also after confirmation

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>

* Implement a filter for category (#215)

- Also implement an is empty/is null condition.

* Implement an LLM cost estimation page

Track costs across all the cost categories: auto categorization, auto merchant detection and chat.
Show warning with estimated cost when running a rule that contains AI.

* Update pricing

* Add google pricing

and fix inferred model everywhere.

* Update app/models/llm_usage.rb

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: soky srm <sokysrm@gmail.com>

* FIX address review

* Linter

* Address review

- Lowered log level
- extracted the duplicated record_usage method into a shared concern

* Update app/controllers/settings/llm_usages_controller.rb

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: soky srm <sokysrm@gmail.com>

* Moved attr_reader out of private

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-10-24 00:08:59 +02:00
soky srm
8cd109a5b2 Implement support for generic OpenAI api (#213)
* Implement support for generic OpenAI api

- Implements support to route requests to any openAI capable provider ( Deepsek, Qwen, VLLM, LM Studio, Ollama ).
- Keeps support for pure OpenAI and uses the new better responses api
- Uses the /chat/completions api for the generic providers
- If uri_base is not set, uses default implementation.

* Fix json handling and indentation

* Fix linter error indent

* Fix tests to set env vars

* Fix updating settings

* Change to prefix checking for OAI models

* FIX check model if custom uri is set

* Change chat to sync calls

Some local models don't support streaming. Revert to sync calls for generic OAI api

* Fix tests

* Fix tests

* Fix for gpt5 message extraction

- Finds the message output by filtering for "type" == "message" instead of assuming it's at index 0
- Safely extracts the text using safe navigation operators (&.)
- Raises a clear error if no message content is found
- Parses the JSON as before

* Add more langfuse logging

- Add Langfuse to auto categorizer and merchant detector
- Fix monitoring on streaming chat responses
- Add Langfuse traces also for model errors now

* Update app/models/provider/openai.rb

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: soky srm <sokysrm@gmail.com>

* handle nil function results explicitly

* Exposing some config vars.

* Linter and nitpick comments

* Drop back to `gpt-4.1` as default for now

* Linter

* Fix for strict tool schema in Gemini

- This fixes tool calling in Gemini OpenAI api
- Fix for getTransactions function, page size is not used.

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-10-22 16:02:50 +02:00
Juan José Mata
5d6915a994 Add OpenAI token configuration to self-hosting settings (#122)
* feat: add OpenAI settings partial
2025-08-23 08:04:59 +02:00
Juan José Mata
d054cd0bb2 Reorganize Settings sections + add LLM model/prompt configs (#116)
* Reshuffle/organize settings UI
* Settings: AI prompt display/minor touch-ups
* API key settings tests
* Moved import/export together
* Collapsible LLM prompt DIVs
* Add export tests
2025-08-22 20:43:24 +02:00
Amr Awad
26c18427c7 Add a 'Bank Sync' page in Settings (#111)
* add initial pages for Bank Sync and Lunch Flow

* update breadcrumbs on Lunch Flow page

* update content for Lunch Flow page

* add norefeerrer to Github link

* update lunch flow url

* nest lunch_flow resource under bank_sync

* add a provider link partial

* remove trailing whitespaces

* update providers style to match merchants page

* remove separate lunch flow page

* fix hover on dark mode

* point lunch flow to custom sure landing page

* [i18n] Bank Sync label

* [i18n] API Keys

* [i18n] Self-Hosting consistency

* Security breadcrum, not "securities" default

---------

Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-08-15 03:07:15 +02:00
Juan José Mata
dd0cb60b56 feat: Add Brand Fetch logo link for logos (see #43) (#99)
* feat: Add Brand Fetch logo link for logos

* docs: brand fetch integration docs

* Document CLIENT_ID location?

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>

---------

Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Vincent Teo <vinteo@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-08-11 22:34:54 +02:00