mirror of
https://github.com/we-promise/sure.git
synced 2026-05-29 15:34:58 +00:00
fdffcd0dfdba7f8e53b0bc44064951fa3d267c25
77 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Juan José Mata
|
f6f9feba8a |
Bank Sync cleanup (#1710)
* feat(settings/providers): surface connection status in section headers
Lifts the per-panel status indicator up to each collapsed accordion
header so admins can see at a glance which providers are connected
without expanding every section. Connected providers sort first.
- Add optional status: and meta: locals to settings/_section partial;
pill hides via group-open:hidden when the section is expanded
- New settings/providers/_status_pill partial (ok/warn/err/off states)
- Add SettingsHelper#provider_summary to centralise the connected-vs-not
logic already scattered across panel partials
- Refactor show.html.erb to pass status to every section and sort
family_panels by connection state
- Add settings.providers.status.* i18n keys
- Add system tests asserting pill renders and sort order
https://claude.ai/code/session_01KW2HCN9rP1fiyQuw7Cju9D
* feat(settings/providers): group providers into Connected and Available
Partition the provider list in the controller into @connected_providers
and @available_providers based on provider_summary status, and render
each group under its own heading with a count. Auto-open the section
when only one provider is connected. Adds an empty-state line when
nothing is connected yet.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(settings/providers): health strip, action-needed group, and sync error surfacing
- Extend provider_summary to return :err/:warn with meta text by checking
latest sync per item (window function, same pattern as ProviderConnectionStatus)
and Enable Banking session expiry within 7 days
- Partition provider entries into three groups: Connected (:ok), Action needed
(:warn/:err, auto-opened), Available (:off)
- Add Settings::HealthSummary ViewComponent — four-tile grid showing Connected,
Action needed, Errors, and Accounts synced counts
- Render health strip directly under page description; omit Action needed heading
when group is empty
- Add i18n keys for tile labels, group heading, and all meta strings
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(settings/providers): card grid for available providers with connect drawer
- Add Provider::Metadata registry with static display data (region, kind,
tier, maturity, logo) for all 11 providers
- Add Settings::ProviderCard ViewComponent rendering logo square, name,
Beta/Alpha pill, meta line (region · type · tier), tagline, and Connect link
- Add connect_form action + route (GET /settings/providers/:key/connect_form)
that opens the existing panel partial or config form in a DS::Dialog drawer
- Replace the Available accordion loop with a 2-column responsive card grid;
empty state when all providers are connected
- Fix layout override: use turbo_rails/frame layout for frame requests so the
drawer response is not wrapped in the full settings layout (was causing
Turbo to pick the empty outer drawer frame instead of the filled one)
- Add SyncAllProvidersJob and last_sync_all_attempted_at migration (sync-all
throttle support)
- Unify Connected + Action needed into a single "Your connections" section;
items with warn/err status auto-open
- Fix Enable Banking grouping: items with expired sessions were returning
:off (Available) instead of :warn (Your connections); gate now checks
any? instead of any?(&:session_valid?)
- Add reconsent_required locale key for fully-expired EB sessions
- Surface Beta/Alpha maturity pills on connected provider accordion rows
via new badge: param on settings_section helper
- Add i18n taglines for all 11 providers; add connect and empty_available keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(settings): retire /settings/bank_sync; merge into providers page
- Delete Settings::BankSyncController and its views (the providers page is
now a strict superset of what bank_sync offered)
- Add permanent 301 redirect: GET /settings/bank_sync → /settings/providers
- Collapse nav to a single "Bank Sync" entry pointing at /settings/providers;
remove the duplicate admin-only "Providers" entry from the Advanced section
- Remove "Providers" from SETTINGS_ORDER; point "Bank Sync" at
settings_providers_path for next/prev navigation
- Rename page title to "Bank Sync"; replace admin-credential lede with
user-facing copy ("Connect external accounts…")
- Update breadcrumb: Home → Bank sync
- Add controller test asserting 301 status and Location header
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Migrations are 7.2 here
* Minimize schema noise
* Schema duplication
* Small copy edits
* Fix tests
* Address provider settings review feedback
* refactor(settings/providers): finish design-review cleanup pass
Picks up the remaining items from Claude Design's review of #1710
that the previous review-feedback commit didn't cover.
DS / casing
- Sentence-case the page title ("Bank Sync" -> "Bank sync") and
align the nav label.
- Drop the card hover-lift (shadow-border-sm) in favour of
bg-container-hover; per the DS, card hover is colour-only.
- Whole-tile click target on each provider card — the inner
"Connect ->" link was a hit-target inversion.
- Set Sync all to whitespace-nowrap so the label stops wrapping at
narrow viewport widths.
UX simplifications
- Drop the four health-summary tiles (per-row warn/err pills already
surface the signal at the scale this app sees). Removes
Settings::HealthSummary, the @health_counts controller block, and
the now-unused health.* locale keys.
- Hide "Your connections" heading + empty-state line when no
providers are connected — the lede already invites a connect.
- Drop the redundant "Free" tier from per-card meta lines (printed
10x for one fact); "Paid" still surfaces on Plaid.
Tests updated to drop the obsolete tiles assertion and switch the
provider-card click selector to look up the (now whole-card) anchor
by provider name.
* feat(settings/providers): replace Add another provider CTA with a search + kind filter
Per the design review, the "Add another provider · Browse providers"
card was a redirect to content one scroll-tick away. A search input
plus kind chips lets users self-segment the catalog and is the right
tool once it grows beyond the four to twelve providers we ship today.
- New providers_filter Stimulus controller — case-insensitive free
text search across name/region/kind, plus a chip group with
All / Banks / Crypto / Investment that toggle visibility via
Tailwind's `hidden` class.
- _search_filters partial: search box (count-pluralized placeholder)
+ chip group, ARIA-labelled and aria-pressed for the chips.
- ProviderCard exposes filter_data (target + name/region/kind data
attrs) so the controller can match without re-rendering.
- Lunchflow's `kind` was "Lunch" — switched to "Bank" so it falls
under the Banks chip alongside its actual offering (it aggregates
banks).
- Drops the add_provider_cta partial and its locale entries; adds
search_filters.* and an empty_filter message.
* Private method fix
* refactor(settings/providers): drawer cleanup, header lock-up, trust statement
Per the design review's §07.
- Drop the trailing "Configured / Not configured" footer status from
every provider panel (binance, coinbase, coinstats, indexa_capital,
lunchflow, mercury, simplefin, snaptrade, sophtron, provider_form).
The parent details section's status pill already carries that
signal; the footer was redundant — and the copy/styling was
inconsistent across panels (free-text vs. dot pill, "configured"
vs. "not connected").
- Connect drawer gets a header lock-up: small logo chip + provider
name + maturity badge, mirroring the available-card layout.
Implemented as _drawer_header partial; connect_form passes
custom_header: true to DS::Dialog so we own the row.
- Drawer footer trust statement: "Read-only — Sure can never move
money. Stored encrypted." A single-line reassurance covering all
panels.
- Sentence-case the hardcoded primary buttons that were Title Case:
"Save Configuration" -> "Save and connect"
"Update Configuration" -> "Update connection"
"Connect Bank" -> "Connect bank"
Affects simplefin, lunchflow, enable_banking, provider_form. The
i18n'd panels (binance, coinbase, coinstats, indexa_capital,
mercury, snaptrade, sophtron) keep their existing keys.
* chore(locales): drop unused provider-panel status strings
Footer "Configured / Not configured" status was removed from each
provider panel partial in the prior drawer-cleanup pass; the matching
i18n keys are no longer referenced. Removing them across every
locale to keep the catalogue clean.
Dropped (15 keys × varying locale coverage, 36 line removals across
24 files):
- coinstats_items.new.{status_configured_html, status_not_configured}
- indexa_capital_items.panel.{status_configured_html, status_not_configured}
- mercury_items.provider_panel.{configured_html, not_configured, accounts_link}
- sophtron_items.sophtron_panel.status.{configured_html, not_configured}
(parent `status:` removed where it became empty)
- providers.snaptrade.{status_needs_registration, status_not_configured}
(status_connected stays — still used by the lazy-load summary)
- settings.providers.{binance_panel, coinbase_panel}.{status_connected, status_not_connected}
* feat(settings/providers): connected-state polish per design §05 + Linked institutions rename
Building the next phase of the design review. Pulls forward the
slim health strip, denser connection rows, and "Linked institutions"
heading rename — the small Phase A lift the designer flagged in
§08 of the doc.
- New _health_strip partial: single-line at-a-glance pulse —
connected count + needs-attention count + accounts syncing +
last-synced timestamp. Renders only when at least one provider
is linked or needs action.
- New _connection_row partial replaces the generic settings_section
call for providers. Tighter rows: text-sm title (was text-lg),
px-4 py-3.5 padding, single-line summary (chevron + name +
maturity badge + meta + status pill + sync action). Warn/error
rows get a coloured outline (border-warning/25 or
border-destructive/25) so the at-risk row stands out without
shouting.
- "Sync all" button restyled to match the design's secondary
button: text-primary, alpha-black-100 border, rounded-[10px],
padding 7px 12px (was the broader px-3 py-1.5 ghost).
- "Your connections" → "Linked institutions" heading, lifted from
the designer's Phase-C reconciliation note. Primes users for the
Option-C institution-search wizard six months early; existing
i18n key stays as `groups.your_connections` for now to keep the
rename to a single value flip.
- Controller computes the new @health hash (connected,
needs_attention, accounts_syncing, last_synced_at) feeding the
strip; brings back the single accounts query that was removed
with the four-tile component.
System test updated for the new heading copy.
* fix(settings/providers): align connected state with the final design mock
Tightening the §05 polish to match the user-confirmed final design.
- Revert "Linked institutions" → "Your connections". The §08
designer note about the Phase-A heading rename didn't carry
forward to the final mock; keep the original wording.
- Drop the warn/err auto-open on connection rows. The design shows
Enable Banking collapsed with a warn-outline and a status pill —
no auto-expanded form. Single-connection auto-open kept (handy
when the page is otherwise empty).
- Hide the "accounts syncing" segment in the health strip when the
count is 0 — the design mock assumes a populated number; an
always-visible "0 accounts syncing" reads as a placeholder.
- Strip the leading "about " from `time_ago_in_words` everywhere
the result is shown to the user (health strip "Last synced %{time}
ago" plus per-row "Synced %{time} ago" meta). Matches the design's
shorter copy.
* refactor(settings/providers): tighten paddings, dedupe maturity badge, semantic + a11y fixes
Pixel-level alignment to the design's §05 mock + cleanup from a DS
audit pass.
Paddings, margins, font sizes
- Health strip: my-4 → mt-4 mb-5 to match the design's 16px / 20px
vertical breathing room.
- Search filters bar: gap-2 → gap-2.5; mt-2 → mt-5 mb-3 (was missing
the 12px bottom margin entirely).
- Search box: rounded-lg → rounded-[10px]; px-3 py-2 → px-[14px]
py-[9px]. Search icon downsized w-4 → w-3.5 to match.
- Chip group: p-1 → p-[3px]; rounded-lg → rounded-[10px].
- Chip: py-1 → py-[5px]; rounded-md → rounded-lg.
- Group heading: mt-2 → mt-[18px]; mb-1 → mb-1.5.
- Status pill: text-xs → text-[11px].
- Provider card: gap-3 → gap-2.5 (outer + top); name gets explicit
text-sm; tagline + foot 14px → 13px; arrow icon w-4 → w-3.5.
- Sync icon button: p-1 → fixed w-7 h-7 (28×28) so the row hit
target matches the design's column width.
- Connect drawer header logo glyph: text-[10px] → text-xs (matches
the available card's logo-glyph treatment).
Component / partial cleanup (DS audit follow-ups)
- New _maturity_badge partial replaces the inline span that was
duplicated in 3 places (_connection_row, _drawer_header,
provider_card.html.erb).
- Settings::ProviderCard.maturity_label class method centralizes the
MATURITY_LABELS lookup; callers no longer reach into the constant.
- _connection_row title: <h2> → <h3> (the row sits inside the
"Your connections" h2 group heading; nested h2s flattened the
outline).
- show.html.erb encryption error: <h3> → <h2> for the same reason.
Locale
- Drop orphaned keys: settings.providers.groups.connected and
groups.needs_attention (no view code uses them) plus the leftover
show.coinbase_title block.
- Health strip "needs reconsent" → "needs attention" so the strip
copy lines up with the per-row status pill ("Action needed") and
the original group heading wording.
A11y
- focus-visible:ring-2 on chip buttons, provider-card link, and
focus-within:ring-2 on the search input wrapper. Keyboard users
now get a visible focus state.
- Search input: explicit autocomplete="off" (erb_lint hint).
* fix(settings/providers): icons + search input height
- Icons were rendering at 20px because the application_helper's `icon`
default size (`md` = w-5 h-5) was beating the inline class override
in compiled CSS source order. Pass `size: "sm"` and use the project's
`!w-3.5 !h-3.5` important-prefix pattern (precedent: dashboard.html.erb)
so chevron, refresh-cw, search, check, circle-alert, and arrow-right
all render at the design's 14px.
- Search input was 54px tall because @tailwindcss/forms applies
`padding: 8px 12px` to bare `<input type="search">`. Override with
`!p-0 focus:ring-0 focus:shadow-none` so the wrapping div's padding
alone defines the box (38px total — matches the design).
* refactor(settings/providers): align Sync all + search input with DS, address review feedback
- Sync all: replace the hand-rolled `button_to` with `DS::Link.new(variant: "outline", method: :post)` — same component as the
"Identify Patterns" button on the recurring-transactions page.
- Search input: switch to the icon-overlay pattern used by the
Manage-currencies and transaction filter rows
(relative wrapper + absolutely positioned search icon +
bordered input with `focus:ring-gray-500`). Brings the keyboard
focus state in line with the rest of the app's filterable lists.
- SnapTrade panel: restore the "needs registration" status row that
the drawer-cleanup pass dropped along with the redundant
Configured/Not configured footer. The unregistered case is
meaningful state, not redundant chrome.
- Move the slim health-strip computation out of the controller and
into `SettingsHelper#provider_health_strip` (Convention 2: skinny
controllers).
- Extract `concise_time_ago` helper so the "drop leading 'about '"
trick stops being duplicated 3x.
- `Settings::ProviderCard#maturity_label` (instance) now delegates
to `.maturity_label` (class) instead of duplicating the lookup.
- Drop unused `warn_or_err` local in `_connection_row`.
- Replace the `data-controller` string-injection + html_safe in
`_connection_row` with `tag.details(data: ...)`; safer and more
idiomatic.
- Add a system test for the empty-filter message wiring.
* fix(settings/providers): drawer trust statement uses border-tertiary
`border-secondary/10` was reaching for the text-foreground token at
10% opacity for a divider. The project ships a dedicated divider
token (`border-tertiary`, ~8% black) used by DS::Menu, the holdings
page, and admin/sso forms. Switching to it makes the trust-statement
HR match every other thin divider in Sure and stops misusing the
text token as a border.
* refactor(settings/providers): swap arbitrary Tailwind values for scale tokens
Per the user's directive — DS-compliance over pixel-perfect alignment
with the design mock. Walked the design audit and applied every swap
that lands within ±2px of the original.
Swaps:
- _health_strip: gap-[18px] → gap-5 (+2), px-[14px] → px-3.5 (=),
text-[13px] → text-sm (+1).
- _search_filters: chip group p-[3px] → p-1, rounded-[10px] →
rounded-xl (concentric with rounded-lg inner pills), chip py-[5px]
→ py-1.
- _status_pill: text-[11px] → text-xs.
- _group_heading: mt-[18px] → mt-5.
- _maturity_badge: text-[10px] → text-xs.
- provider_card: tagline + foot text-[13px] → text-sm.
Kept arbitrary: `min-w-[200px]` in _search_filters — nearest scale
tokens are min-w-48 (192px) and min-w-52 (208px); both are noticeable
layout shifts for a one-off responsive guard. Worth keeping the
arbitrary here.
Net: 9 of 10 arbitrary values gone. Visual delta: max +2px on a
single value. Design mock and DS scale now agree.
* revert(settings/providers): drop the slim health strip
Per-row status pills already carry the at-a-glance signal (connected
/ action needed) at the scale this app sees (1–4 connections per
family). The strip was redundant chrome for almost every user; only
worth bringing back if the catalog grows to a point where the row
list itself stops fitting on a single screen.
- Delete _health_strip.html.erb partial.
- Drop @health controller assignment + provider_health_strip helper.
- Drop unused settings.providers.health_strip.* locale keys.
- concise_time_ago helper stays — still used by per-row meta text.
* refactor(settings/providers): align with DS conventions
Two consistency wins from the screenshot/DS audit pass.
Sync icon button now renders DS::Button (variant: icon, size: sm)
instead of a hand-rolled `button_to`. Same component used by other
icon-only actions across the app (settings/profiles, layouts/imports).
Visual delta: 28×28 → 32×32 (DS sm size). Accept the +4px for
consistency. `event.stopPropagation()` still wired via the form opt
so the row's <details> doesn't toggle when the user clicks the
button.
Group heading now follows the established Sure section-label style
(`text-xs font-medium text-secondary uppercase`) used by
`_settings_nav` and the imports/categories surfaces. The previous
sentence-case `text-sm text-primary` was a one-off that didn't
match the rest of the app. Locale strings stay sentence-case;
uppercase comes from CSS `text-transform`. Tests updated to
case-insensitively match the rendered heading text.
* fix(provider/metadata): add plaid_eu entry
`plaid_eu` is registered as a separate Provider::ConfigurationRegistry
entry but had no Provider::Metadata row, so its card in the
Available grid fell through to the gray-500 default and rendered
empty (no region, kind, tier, or tagline). The title also came out
as "Plaid Eu" because `titleize` doesn't know "EU" is an initialism.
- Add a `plaid_eu` row to Provider::Metadata::REGISTRY with the same
shape as `plaid` (US → EU, otherwise identical).
- Introduce an optional `name:` field in metadata; controller falls
back to it before titleizing the provider key. Lets `plaid_eu`
render as "Plaid EU".
- Add the missing `settings.providers.taglines.plaid_eu` translation.
* fix(settings/providers): center-align Sync all next to the lede
`items-start` made the button hug the first line when the lede wrapped;
on a single line the button sat at the top of the text bounding box
which read slightly off. Center matches the dominant convention
across the rest of settings (api_keys, securities, hostings, _section,
_settings_nav_link_large).
* fix(settings/providers): drop colour palette + filter polish + drawer warnings
Round of design-feedback fixes.
Provider chips
- Drop the per-provider raw Tailwind palette (bg-blue-600 etc.) from
Provider::Metadata. All cards + drawer logo lock-up now use
bg-surface-inset + text-primary, matching the design's §04 "drop
colour entirely" recommendation. Solves the long-standing §01
BLOCKER without externalising brand assets. Re-introducing logos
later just means an optional logo_svg: field on metadata.
- ProviderCard component drops the `logo_bg:` parameter; the chip
is now styled in the template.
Filter / search
- "Available · N" count and the empty-filter state now update
client-side as the chip filter and free-text search narrow the
grid (new `count` Stimulus target + dedicated update path).
- Empty-filter state now offers a Clear filters button that resets
both the search input and the active chip in one click.
- Search placeholder drops the drifting "Search 9 providers" count
for plain "Search providers" — the section heading carries the
number.
- Chip labels normalised to plural where natural: "Banks · Crypto ·
Investments" (Crypto stays as the mass noun).
Drawer copy / treatment
- "IP Whitelisting Required" → "IP whitelisting required" (DS
sentence-case).
- Binance "do NOT enable withdrawal permissions" lifted out of
inline red-text into a proper bg-warning-50 border-warning-200
alert block with an alert-triangle icon. Matches the api_keys /
hosting alert pattern.
- SnapTrade free-tier inline alert-triangle now uses `size: "sm"`
so the icon stops rendering at 20px next to 14px body text.
Spacing
- Group-heading margin top bumped 5 → 6 (20→24px) so the eyebrow
has more breathing room above the search bar.
* refactor(settings/providers): drawer alerts use DS::Alert; drop card-in-card
Two consistency fixes from a design-review pass.
DS::Alert adoption
- Replaces 9 hand-rolled error blocks across the provider panels
(`bg-destructive/10 text-destructive ... line-clamp-3`) with
`DS::Alert(variant: :error)` — the project's existing primitive.
- Replaces the just-shipped Binance no-withdraw warning block with
`DS::Alert(variant: :warning)` instead of a hand-rolled
`bg-warning-50 border-warning-200` card.
- Replaces the SnapTrade free-tier inline icon-prefixed warning
paragraph with `DS::Alert(variant: :warning)` — proper alert
treatment for an actual warning, not body copy.
- Replaces the Enable Banking "Configuration locked" inline
`bg-warning/10` two-paragraph block with `DS::Alert(variant: :warning)`
using `safe_join` for the title + body.
- Replaces the encryption-error block at the top of show.html.erb
with `DS::Alert(variant: :error)`, again via `safe_join`.
Mercury card-within-card
- The "Add another Mercury connection" form was wrapped in a
`<details>` `bg-container shadow-border-xs rounded-xl` card. In
the Connect drawer (always 0 existing connections), that wrapping
card-inside-the-drawer-card has no value — the form is the only
thing on the surface. Drop the wrapper when no connections exist;
keep the heading + form inline. When 1+ connections exist (the
section page) the heading hints "+ Add another connection"
without the disclosure indirection.
Trade-off: the error-alert blocks lose their `line-clamp-3` /
`title=` truncation. Acceptable for now — DS::Alert can grow a
truncate option as a follow-up if needed.
Open follow-up: DS::Alert itself uses raw Tailwind palette
(`bg-yellow-50` etc.) instead of semantic tokens, and only accepts
a single string `message:`. A separate issue tracks this.
* fix(settings/providers): hoist warning alerts to top of drawer
DS::Alert convention across the rest of the app: alerts sit at the
top of the form / page / section, not floating between content
blocks. The Binance no-withdraw warning and SnapTrade free-tier
warning were rendering between the setup-instructions list and the
form fields — visually wonky.
Move both to the top of their respective panels so the warning is
the first thing the user sees when the connect drawer opens.
Existing precedents this aligns with:
- accounts/_form.html.erb (error alert above form)
- valuations/new.html.erb (error alert above form)
- other_assets/new.html.erb (info alert above form)
- holdings/show.html.erb (warn alerts above content)
* fix(DS::Alert): align icon to cap-height of first text line
`items-start` on the container made the icon's top edge flush with
the text's top edge, leaving the icon's optical center sitting below
the text's first-line center. The hand-rolled alerts elsewhere in
the codebase (api_keys/new, hostings/_sync_settings, holdings/show)
all add `mt-0.5` to the icon for the same reason — fold that into
the primitive so every caller gets the cap-height alignment.
* copy(settings/providers): tighten alert messaging per voice review
Copy expert pass on the new provider drawer alerts. House style:
sentence case for titles, lead with the action, drop "Warning:" /
"Please" filler (the alert variant icon already signals tone),
prefer one short sentence + optional title-paragraph for emphasis.
- Binance no-withdraw warning: was a single line "Warning: do NOT
enable withdrawal permissions" — alarmist without context. Now
splits into "Read-only key only" (title) + "Don't enable
withdrawal permissions when creating your Binance API key — Sure
only needs read access." (body).
- SnapTrade free-tier note: "Free tier includes 5 brokerage
connections. Additional connections require a paid SnapTrade
plan." → "SnapTrade's free tier covers 5 brokerage connections.
Upgrade on SnapTrade for more."
- SnapTrade connection-limit-info inside the brokerage list: cut
entirely. The drawer already shows the cap; restating it in the
list was noise.
- SnapTrade needs-registration: "Credentials saved — finish
registration to connect a brokerage." → "Credentials saved.
Finish setup to connect a brokerage." ("registration" was
ambiguous — register where, with whom?)
- Enable Banking "Configuration locked" body: "Credentials cannot
be changed while you have active bank connections. Remove all
connections first to update credentials." → "Disconnect all
linked banks before changing these credentials." Same meaning,
half the words.
- Encryption-error block: title-cased "Encryption Configuration
Required" → "Encryption keys missing"; body strips "Please
ensure" filler and the parenthetical credential dump, leaving
the three credential names inline as a clean list. Self-hosters
still get exactly the names they need to set.
* feat(settings/providers): SetupSteps partial for connect-drawer instructions
Per the design's drawer-cleanup follow-up. Replaces the per-panel
"Setup instructions:" + ordered list + "Field descriptions:" block
with a shared boxed-step component.
The new partial — `_setup_steps.html.erb` — takes a `steps:` array
of strings (or html_safe strings for inline links / code) plus an
optional `help:` hash for a docs link below the steps. The eyebrow
label is "Setup" (uppercase, tracking-wider) matching Sure's other
section labels.
Applied across all eleven provider panels:
- _provider_form (Plaid + Plaid EU): field descriptions move to
per-field helper text below the input.
- _binance, _coinbase, _coinstats, _indexa_capital,
_lunchflow, _mercury, _simplefin, _snaptrade, _sophtron,
_enable_banking: ordered list + duplicate "Field descriptions"
block both replaced by the partial.
- Some panels' inline copy tightened in the same pass (Lunch Flow,
SimpleFIN, Enable Banking) — the design copy is shorter than the
current legacy strings; a copy-pass through every panel can
follow as a separate cleanup.
Token notes: uses scale tokens (`rounded-xl`, `text-xs`/`text-sm`,
`tracking-wider`) instead of the design mock's exact arbitrary
values, per the consistency-over-design-specs directive on this
branch.
* fix(settings/providers): tighten panel spacing + relocate per-panel notes
Read-flow audit on each connect drawer. The uniform `space-y-4`
treated every block (alert, steps, info card, fields, button) the
same — visually they were five sibling boxes with no grouping. The
fix is per panel; some notes belong as helper text on a specific
field, others as a tightly-grouped pre-fill primer.
Per panel:
- Binance: IP-whitelisting card now matches the setup_steps box
(`bg-surface-inset rounded-xl`) and is wrapped with setup_steps
in an inner `space-y-2` so they read as a single pre-fill primer
cluster. Same eyebrow treatment ("IP whitelisting required") so
the two boxes look like sister panels, not unrelated chrome.
- SnapTrade: drop the description paragraph above setup_steps. The
available-providers card grid already markets SnapTrade
("Connect brokerage accounts via the SnapTrade aggregation
network."); repeating in the drawer was duplication.
- Mercury: move the sandbox-API note out of its standalone <p>
below setup_steps and into per-field helper text under the
base_url field — the user only cares about the sandbox URL when
they're filling that field. Applied to both the per-item edit
form and the add-new form.
- _setup_steps partial: drop the now-pointless `mb-2` (outer
`space-y-4` already controls the gap; bottom-margin was dead
CSS thanks to margin-collapse rules with the next sibling's
margin-top).
* fix(settings/providers): plaid + indexa drawers join the SetupSteps look
Two unifying fixes after the panel-by-panel screenshots showed
mixed treatments.
Plaid + Plaid EU
- The registry-driven panel (_provider_form) was still rendering
each adapter's markdown `description` block as plain prose
("Setup instructions: 1. Visit the Plaid Dashboard ..."). Other
panels switched to the SetupSteps box; Plaid was the odd one out.
- Drop the markdown `description` block from both plaid_adapter
and plaid_eu_adapter. Render setup_steps in _provider_form for
these two provider keys via inline ERB (link helper handles the
Plaid Dashboard link cleanly; the regional differences fold to
the same dashboard URL with a different account scope).
- Other registry-based providers fall through to the previous
markdown description path — no behavior change for them.
Indexa Capital
- The API token field was wrapped in a `bg-surface border` "card"
that duplicated the field label inside as a heading and put the
description above the input. Same pattern the user flagged as
the "card within input" anti-shape.
- Drop the wrapper. The styled-form input renders its own label;
description moves to per-field helper text below the input,
matching the pattern used by Plaid (provider_form) and Mercury.
* fix(settings/providers): surface configured plaid_eu + dedup show context
provider_summary had no plaid_eu branch — configured plaid_eu was
falling through to status :off and rendering in Available even with
credentials set. Collapse plaid + plaid_eu into a single registry
check.
Drawer title for non-panel configurations was provider_key.titleize,
which produced "Plaid Eu" while the available card grid used
metadata[:name] = "Plaid EU". Read from metadata first.
While here:
- compute_provider_sync_health no longer relies on
instance_variable_get; pass family_panel_items explicitly so the
hash-key/ivar-name coupling is gone.
- drop unused .includes(:syncs, :mercury_accounts) and
.includes(:snaptrade_accounts) from prepare_show_context. The show
view only consults summary[:status]; the eager-loads were carried
over from connect_form (which has its own load_provider_items).
* i18n(settings/providers): localize plaid setup steps + drop dead defaults
The plaid + plaid_eu setup steps in _provider_form.html.erb were
hardcoded English strings. Move them to settings.providers.plaid_panel
(shared) + plaid_eu_panel (EU-specific step 1) so they can be
translated like every other panel.
_setup_steps.html.erb was passing default: "Setup" / "Need help?" to
t(), masking missing translations in non-EN locales. Both keys exist
in en.yml — drop the defaults so missing translations actually
surface.
* test(settings/providers): cover plaid_eu, clear filters, warn outline
Three system test additions:
- Configured plaid_eu surfaces in Your connections (regression guard
for the helper fix; previously fell through to Available).
- Clear filters button resets input + chip state and brings cards
back into view.
- :warn-state connection row carries the border-warning/25 outline
that distinguishes it from an :ok row.
* copy(settings/providers): drop em dashes, naturalize phrasing
Sweep through every string this branch added and replace em-dash
splices with full sentences or simple connectives.
en.yml:
- drawer_trust_statement now reads "Read-only access. Sure can never
move money, and your credentials are stored encrypted." instead
of em-dash splicing.
- sync_all_recently / recently_synced split into two sentences.
- binance_panel.no_withdraw_body, plaid_panel.step_1_html / step_2,
plaid_eu_panel.step_1_html same treatment.
Hardcoded panel steps (enable_banking, lunchflow, simplefin) become
"Go to <link> and …" or "Go to <link> for …" instead of the
"<link> — get …" splice. Same setup_steps comment cleaned up.
* fix(settings/providers): address CodeRabbit pass on PR #1717
Fixed:
- Localize the setup steps in _enable_banking_panel,
_lunchflow_panel, and _simplefin_panel. The em-dash sweep had
rewritten these into hardcoded English; they now route through
settings.providers.{enable_banking,lunchflow,simplefin}_panel
step_1_html / step_2 / step_3 keys, mirroring the plaid_panel
treatment.
- connect_form: silent redirect when provider_key is unknown now
carries an alert (settings.providers.not_found) so misrouted
links don't drop users on the page with no feedback.
- sync action: redirect notice now reflects whether anything was
actually scheduled — adds settings.providers.sync_provider_no_items
for the "all items already syncing or none exist" path.
- Family::Syncer test: count plaid_items via the .syncable scope to
match what Family::Syncer actually schedules (already done for
binance_items in the same test).
Skipped, with reasons:
- focus:ring-gray-500/-gray-900 in coinstats / coinbase / simplefin /
search_filters: tracked under issue #1715 as part of the raw-palette
→ DS-token sweep across the whole codebase.
- Coinbase #0052FF brand-color wrapper: tracked under PR #1710's
follow-up tracking comment as the deferred Provider::Metadata
colour-palette decision (designer §01).
- Sophtron submit-button extraction into DS::Button: same
deferred sweep — every panel hand-rolls this class string;
one-off extraction would just churn.
- Redundant .html_safe on _html keys in coinstats: tracked in #1715.
- _provider_form.html.erb env hint, "Optional" placeholder, "Save and
connect" submit: pre-existing strings not added on this branch.
- Renaming sync_health_for's :stale to :data_stale: pre-existing
shape, refactor scope.
- Plaid_eu using plaid_panel.step_2/step_3 keys: deliberate. Same
English copy across both providers; duplicating keys would just
give translators twice the work for identical strings.
- _enable_banking_panel / _lunchflow_panel / _simplefin_panel
alert + submit + button labels: pre-existing hardcoded strings
from before this branch. Setup steps were the strings actually
touched in the em-dash sweep, so those got localized; the rest
belong in a broader panel-i18n pass.
Verified:
- bundle exec erb_lint on the three panels: clean.
- bin/rubocop on controller + test: clean.
- bin/rails test test/models/family/syncer_test.rb
test/controllers/settings/providers_controller_test.rb:
23 runs, 85 assertions, 0 failures.
- DISABLE_PARALLELIZATION=true bin/rails test
test/system/settings/providers_test.rb:
15 runs, 38 assertions, 0 failures.
* fix(db): rename migration to clear collision with main's 20260508120000
Main's PR #1705 (Sophtron manual sync) shipped a migration with
the same 20260508120000 timestamp as our
add_last_sync_all_attempted_at_to_families migration. The merge
that brought main into this branch left both files at the same
prefix, which trips Rails' "Duplicate migration" guard at
db:schema:load time and broke CI.
Renaming our migration to 20260510120000 keeps the column it adds
intact (already in db/schema.rb) and bumps the schema version to
match. No DB-level change.
* fix(settings/providers): card + strip a11y polish
- Bring back the slim health strip; gate behind 10+ accounts
(HEALTH_STRIP_MIN_ACCOUNTS) so it stays out of the way for
small libraries where per-row pills already carry the signal.
- Status pill: drop the bg-{c}/10 text-{c} pattern (failed AA
on warn / err); switch to bg-surface-inset text-primary with
the dot still carrying semantic colour. Passes AA in both
themes; the dot is the only colourful affordance.
- Maturity badge: bg-alpha-black-50 was invisible against the
hovered card bg in light mode and against bg-container in
dark mode. Move to bg-surface-inset + border-tertiary so it
stays delineated through hover and dark theme.
- Provider card: keep the bg shift on hover (now bg-surface-inset
for a perceptible delta), focus ring promoted alpha-black-100
-> alpha-black-300 (visible to keyboard users), meta line
text-subdued -> text-secondary (text-subdued failed AA at
2.86:1 against bg-container).
- Restore the per-provider logo palette dropped in
|
||
|
Juan José Mata
|
81cdccb768 |
[codex] Complete Sophtron account mapping (#1698)
* Complete Sophtron account mapping * Clarify Sophtron login challenge flow * Add Sophtron connection UI timeout * Treat Sophtron timeout jobs as failed * Reset failed Sophtron connection state * Handle stale Sophtron connection jobs * Advance Sophtron polling timeout * Shorten Sophtron connection timeout * Fix Sophtron modal polling updates * Stabilize Sophtron MFA polling * Give Sophtron OTP challenges more time * Clarify Sophtron institution login failures * Extend Sophtron polling during login progress * Probe Sophtron accounts after completed MFA step * Align Sophtron dialogs with design system * Start Sophtron initial load after linking accounts * Fix Sophtron initial transaction load * Fail Sophtron sync without institution connection * Fix tests * Wrap Sophtron account linking in transaction * Wrap Sophtron provider responses * Fix Sophtron MFA security tests * Guard Sophtron MFA challenge arrays * Respect Sophtron initial load window * Use unique Sophtron MFA answer field ids * Address Sophtron review follow-ups * Fix Sophtron transaction sync refresh * Avoid blocking Sophtron refresh polling * Move Sophtron account helpers to model * Keep Sophtron grouping provider-level * Start new Sophtron institution links * Isolate Sophtron institution connections --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
Michal Tajchert
|
ccd6a53071 |
fix(chat): eager pending AssistantMessage to fix Turbo subscribe race (#1657) (#1658)
* fix(chat): persist eager pending assistant message to fix subscribe race When the LLM replies in ~1-2s the assistant message broadcast could fire before the client's Turbo stream subscription was established, leaving the UI stuck on the thinking indicator while the response was already persisted. Create the AssistantMessage as `pending` synchronously in `Chat#ask_assistant_later`, so it is rendered server-side on the chat show page with a "Thinking ..." inline placeholder. The worker then finds and updates the existing row via `append_text!`, which flips the status to `complete` and broadcasts updates against a DOM id that is already in the page — no race possible. On error, the placeholder is destroyed if no content streamed, otherwise demoted to `failed`. Replaces the standalone thinking indicator partial and the `Assistant::Broadcastable` thinking helpers, both now redundant. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(chat): bind each assistant job to its specific pending placeholder Addressing review feedback on #1658: 1. The pending placeholder lookup based on `last pending` was racy — back-to-back user messages would let one job fill another job's placeholder. Pass the placeholder through the job arguments (`AssistantResponseJob.perform_later(user_message, pending)`) so each turn is bound to its own row. 2. In `Assistant::External#respond_to`, the configured/authorized guards raise before the local was bound, leaving rescue cleanup with `nil` and the placeholder visible forever. Bind the parameter first so cleanup can destroy it on the misconfigured path. The kwarg defaults to nil so the API#retry path (`AssistantResponseJob.perform_later(new_message)`) and the model-level test calls continue to work — they fall back to an in-memory new message, restoring the original test count assertions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(chat): i18n the pending assistant placeholder string Move the hardcoded "Thinking ..." indicator into the locale file per CLAUDE.md i18n guidelines. With i18n.fallbacks enabled, non-en locales fall back to English until translated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Add thinking label translations * Fix chat pending assistant expectations * Fix external assistant pending test lookup * Scope chat stream targets per chat * Update message broadcast target tests --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
wps260
|
33bb1f524d |
fix: skip DemoFamilyRefreshJob in self-hosted mode (#1578)
DemoFamilyRefreshJob runs daily via cron and recreates a demo family from scratch. This is a managed-mode-only feature (the public hosted instance), but the job had no app_mode guard, so self-hosted instances were also creating and refreshing a demo family every day. This results in every self-host instance of sure getting the demo family with the well-known credentials created. It may be worth considering a separate one-time fix that deactivates the demo family users, and/or entries in the release notes or a separate security advisories alerting users that they need to deactivate the demo users. |
||
|
Sophtron Rocky
|
b32e9dbc45 |
Add Sophtron Provider (#596)
* Add Sophtron Provider * fix syncer test issue * fix schema wrong merge * sync #588 * sync code for #588 * fixed a view issue * modified by comment * modified * modifed * modified * modified * fixed a schema issue * use global subtypes * add some locales * fix a safe_return_to_path * fix exposing raw exception messages issue * fix a merged issue * update schema.rb * fix a schema issue * fix some issue * Update bank sync controller to reflect beta status Signed-off-by: Juan José Mata <jjmata@jjmata.com> * Rename settings section title to 'Sophtron (alpha)' Signed-off-by: Juan José Mata <jjmata@jjmata.com> * Consistency in alpha/beta for Sophtron * Good PR suggestions from CodeRabbit --------- Signed-off-by: soky srm <sokysrm@gmail.com> Signed-off-by: Sophtron Rocky <rocky@sophtron.com> Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Signed-off-by: Juan José Mata <jjmata@jjmata.com> Co-authored-by: soky srm <sokysrm@gmail.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Juan José Mata <jjmata@jjmata.com> |
||
|
Juan José Mata
|
65c5f8eb07 |
Fix DemoFamilyRefreshJob demo email config lookup (#1426)
* Fix demo refresh email config lookup Use indifferent access when reading demo config so the refresh job works whether config keys are strings or symbols. Add a regression test covering symbol-keyed config_for output. * Remove unnecessary blank line in test file Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
sentry[bot]
|
117ab2b45f |
refactor(accounts): Improve destroy_later atomicity and add controller error handling (#1395)
* refactor(accounts): Improve destroy_later atomicity and add controller error handling * Address PR comment --------- Co-authored-by: sentry[bot] <39604003+sentry[bot]@users.noreply.github.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
soky srm
|
9410e5b38d |
Providers sharing (#1273)
* third party provider scoping * Simplify logic and allow only admins to mange providers * Broadcast fixes * FIX tests and build * Fixes * Reviews * Scope merchants * DRY fixes |
||
|
soky srm
|
12d2f4e36d |
Provider merchants enhancement (#1254)
* Add AI merchant enhancement and dedup * Enhancements Add error if job is already running add note that we also merge merchants * Allow updating provider website * Review fixes * Update provider_merchant.rb * Linter and fixes * FIX transaction quick menu modal |
||
|
Juan José Mata
|
53f478a77b |
Add scheduled DemoFamilyRefreshJob to rebuild demo data daily (#1217)
* Add scheduled demo family refresh job Rebuild demo data daily at 5am UTC by anonymizing and enqueueing deletion of the existing demo family while immediately generating new sample data. Add super-admin email notifications with 24-hour session and signup metrics, plus tests for the new job and mailer. * Delete demo monitoring key before family refresh Ensure DemoFamilyRefreshJob removes ApiKey::DEMO_MONITORING_KEY from the old demo family before enqueueing async family destruction and generating replacement sample data. Adds a regression assertion that the key is gone before generator execution. |
||
|
Juan José Mata
|
5b0ddd06a4 |
Add post-trial inactive Family cleanup with data archival (#1199)
* Add post-trial inactive family cleanup with data archival Families that expire their trial without subscribing now get cleaned up daily. Empty families (no accounts) are destroyed immediately after a 14-day grace period. Families with meaningful data (12+ transactions, some recent) get their data exported as NDJSON/ZIP to an ArchivedExport record before deletion, downloadable via a token-based URL for 90 days. - Add InactiveFamilyCleanerJob (scheduled daily at 4 AM, managed mode only) - Add ArchivedExport model with token-based downloads - Add inactive_trial_for_cleanup scope and requires_data_archive? to Family - Extend DataCleanerJob to purge expired archived exports - Add ArchivedExportsController for unauthenticated token downloads https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ * Fix Brakeman redirect warning in ArchivedExportsController Use rails_blob_path instead of redirecting directly to the ActiveStorage attachment, which avoids the allow_other_host: true open redirect. https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ * Update schema.rb with archived_exports table Add the archived_exports table definition to schema.rb to match the pending migration, unblocking CI tests. https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ * Fix broken CI tests for ArchivedExports and InactiveFamilyCleaner - ArchivedExportsController 404 test: use assert_response :not_found instead of assert_raises since Rails rescues RecordNotFound in integration tests and returns a 404 response. - InactiveFamilyCleanerJob test: remove assert_no_difference on Family.count since the inactive_trial fixture gets cleaned up by the job. The test intent is to verify the active family survives, which is checked by assert Family.exists?. https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ * Wrap ArchivedExport creation in a transaction Ensure the ArchivedExport record and its file attachment succeed atomically. If the attach fails, the transaction rolls back so no orphaned record is left without an export file. https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ * Store only a digest of the download token for ArchivedExport Replace plaintext download_token column with download_token_digest (SHA-256 hex). The raw token is generated via SecureRandom on create, exposed transiently via attr_reader for use in emails/logs, and only its digest is persisted. Lookup uses find_by_download_token! which digests the incoming token before querying. https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ * Remove raw download token from cleanup job logs Log a truncated digest prefix instead of the raw token, which is the sole credential for the unauthenticated download endpoint. https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ * Fix empty assert_no_difference block in cleaner job test Wrap the perform_now call with both assertions so the ArchivedExport.count check actually exercises the job. https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
soky srm
|
d79d86d848 |
PDF ai import (#1006)
Add support to review transactions for AI pdf import |
||
|
Juan José Mata
|
8276800859 | Fix tests | ||
|
Juan José Mata
|
e7c609dc91 | Index all PDF imports into vector store with type metadata | ||
|
Juan José Mata
|
17df5179ae |
Remove reference to nonexistent Sidekiq::Throttled gem (#950)
The sidekiq-throttled gem is not in the Gemfile, so including Sidekiq::Throttled::Job causes an uninitialized constant NameError at boot time, breaking production builds. https://claude.ai/code/session_01Bj7xgndJt28BcUHW1v1M9S Co-authored-by: Claude <noreply@anthropic.com> |
||
|
David Gil
|
ba442d5f26 |
Implement Indexa Capital provider with real API integration (#933)
* Add Indexa Capital provider scaffold
Generate Indexa Capital provider scaffolding and align credential fields with the API authentication requirements.
* Fix PR 926 lint and schema CI failures
* Implement Indexa Capital provider with real API integration
- Rewrite all broken view templates (were meta-ERB from code generator)
- Create missing select_accounts.html.erb template
- Implement real API calls: list_accounts via /users/me, get_holdings
via /accounts/{number}/fiscal-results, get_account_balance via
/accounts/{number}/performance
- Add API token auth support (stored token > env token > credentials)
- Add api_token column with encryption support
- Redesign settings panel: API token prominent, credentials collapsible
- Fix account balances display using performance endpoint portfolios
- Fix accounts index empty-state guard missing indexa_capital_items
- Simplify activities fetch job (no activities API endpoint exists)
- Fix i18n interpolation (%%{ -> %{) throughout locale file
* Add tests for Indexa Capital provider integration
- IndexaCapitalItemTest: validations, credentials, scopes, sync status
- IndexaCapitalAccountTest: upsert, holdings, account provider linking
- Provider::IndexaCapitalTest: auth modes, API stubs, error handling
- IndexaCapitalItemsControllerTest: CRUD, setup, linking, authorization
- Fixtures for items (token + credentials) and accounts (mutual + pension)
52 tests, 98 assertions, 0 failures
* Address code review feedback from PR #933
- Fix zero balance bug: use `nil?` instead of `present?` so 0 is stored
- Fix has_indexa_capital_credentials? to check api_token (was ignored)
- Fix build_provider to delegate to Provided concern (was ignoring token)
- Fix IndexaCapital section outside encryption_error guard in settings
- Add account_number sanitization to prevent path traversal in API URLs
- Replace all skipped processor tests with real working tests
- Add zero-balance and path-traversal test coverage
61 tests, 107 assertions, 0 failures
* Address code review round 2: credentials validation, RuboCop, test quality
- Fix RuboCop SpaceInsideArrayLiteralBrackets in credentials check
- Chain where.not calls so all three username/document/password must be present
- Require all three credentials (||) instead of any one (&&) in validate_configuration!
- Move attr_reader to private to avoid exposing credentials publicly
- Parse dates with Date.parse in extract_balance for robustness
- Remove stale TODO and Crypto from supported_account_types
- Order build_provider query deterministically by created_at
- Replace no-op holdings assertion with meaningful assert_difference
* Address code review round 3: JSON parse safety and test precision
- Rescue JSON::ParserError on 2xx responses for clearer error messages
- Fix weak balance assertion: set balance to 0 before processing, assert
expected value (27093.01 = sum of holdings amounts)
* Include Indexa Capital in automatic family sync
Add indexa_capital_items to Family::Syncer#child_syncables so balances
and holdings refresh on daily auto-sync and login sync, not only on
manual sync button clicks.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
|
||
|
MkDev11
|
0afdb1d0fd |
Feature/pdf import transaction rows (#846)
* Add import row generation from PDF extracted data - Add generate_rows_from_extracted_data method to PdfImport - Add import! method to create transactions from PDF rows - Update ProcessPdfJob to generate rows after extraction - Update configured?, cleaned?, publishable? for PDF workflow - Add column_keys, required_column_keys, mapping_steps - Set bank statements to pending status for user review - Add tests for new functionality Closes #844 * Add tests for BankStatementExtractor - Test transaction extraction from PDF content - Test deduplication across chunk boundaries - Test amount normalization for various formats - Test graceful handling of malformed JSON responses - Test error handling for empty/nil PDF content * Fix supports_pdf_processing? to validate effective model The validation was always checking @default_model, but process_pdf allows overriding the model via parameter. This could cause a vision-capable override model to be rejected, or a non-vision-capable override to pass validation only to fail during processing. Changes: - supports_pdf_processing? now accepts optional model parameter - process_pdf passes effective model to validation - Raise Provider::Openai::Error inside with_provider_response for consistent error handling Addresses review feedback from PR#808 * Fix insert_all! bug: explicitly set import_id Rails insert_all! on associations does NOT auto-set the foreign key. Added import_id explicitly and use Import::Row.insert_all! directly. Also reload rows before counting to ensure accurate count. * Fix pending status showing as processing for bank statements with rows When bank statement PDF imports have extracted rows, show a 'Ready for Review' screen with a link to the confirm path instead of the 'Processing' spinner. This addresses the PR feedback that users couldn't reach the review flow even though rows were created. * Gate publishable? on account.present? to prevent import failure PDF imports are created without an account, and import! raises if account is missing. This prevents users from hitting publish and having the job fail. * Wrap generate_rows_from_extracted_data in transaction for atomicity - Clear rows and reset count even when no transactions extracted - Use transaction block to prevent partial updates on failure - Use mapped_rows.size instead of reload for count * Localize transactions count string with i18n helper * Add AccountMapping step for PDF imports when account is nil PDF imports need account selection before publishing. This adds Import::AccountMapping to mapping_steps when account is nil, matching the behavior of TransactionImport and TradeImport. Addresses PR#846 feedback about account selection for PDF imports. * Only include CategoryMapping when rows have non-empty categories PDF extraction doesn't extract categories from bank statements, so the CategoryMapping step would show empty. Now we only include CategoryMapping if rows actually have non-empty category values. This prevents showing an empty mapping step for PDF imports. * Fix PDF import UI flow and account selection - Add direct account selection in PDF import UI instead of AccountMapping - AccountMapping designed for CSV imports with multiple account values - PDF imports need single account for all transactions - Add update action and route for imports controller - Fix controller to handle pdf_import param format from form_with - Show Publish button when import is publishable (account set) - Fix stepper nav: Upload/Configure/Clean non-clickable for PDF imports - Redirect PDF imports from configuration step (auto-configured) - Improve AI prompt to recognize M-PESA/mobile money as bank statements - Fix migration ordering for import_rows table columns * Add guard for invalid account_id in imports#update Prevents silently clearing account when invalid ID is passed. Returns error message instead of confusing 'Account saved' notice. * Localize step names in import nav and add account guard - Use t() helper for all step names (Upload, Configure, Clean, Map, Confirm) - Add guard for invalid account_id in imports#update - Prevents silently clearing account when invalid ID is passed * Make category column migrations idempotent Check if columns exist before adding to prevent duplicate column errors when migrations are re-run with new timestamps. * Add match_path for PDF import step highlighting Fixes step detection when path is nil by using separate match_path for current step highlighting while keeping links disabled. * Rename category migrations and update to Rails 7.2 - Rename class to EnsureCategoryFieldsOnImportRows to avoid conflicts - Rename class to EnsureCategoryIconOnImportRows - Update migration version from 7.1 to 7.2 per guidelines - Rename files to match class names - Add match_path for PDF import step highlighting * Use primary (black) style for Create Account and Save buttons * Remove match_path from auto-completed PDF steps Only step 4 (Confirm) needs match_path for active-step detection. Steps 1-3 are purely informational and always complete. * Add fallback for document type translation Handles nil or unexpected document_type values gracefully. Also removes match_path from auto-completed PDF steps. * Use index-based step number for mobile indicator Fixes 'Step 5 of 4' issue when Map step is dynamically removed. * Fix hostings_controller_test: use blank? instead of nil Setting returns empty string not nil for unset values. * Localize step progress label and use design token * Fix button styling: use design system Tailwind classes btn--primary and btn--secondary CSS classes don't exist. Use actual design system classes from DS::Buttonish. * Fix CRLF line endings in tags_controller_test.rb --------- Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com> |
||
|
MkDev11
|
6f8858b1a6 |
feat/Add AI-Powered Bank Statement Import (step 1, PDF import & analysis) (#808)
* feat: Add PDF import with AI-powered document analysis This enhances the import functionality to support PDF files with AI-powered document analysis. When a PDF is uploaded, it is processed by AI to: - Identify the document type (bank statement, credit card statement, etc.) - Generate a summary of the document contents - Extract key metadata (institution, dates, balances, transaction count) After processing, an email is sent to the user asking for next steps. Key changes: - Add PdfImport model for handling PDF document imports - Add Provider::Openai::PdfProcessor for AI document analysis - Add ProcessPdfJob for async PDF processing - Add PdfImportMailer for user notification emails - Update imports controller to detect and handle PDF uploads - Add PDF import option to the new import page - Add i18n translations for all new strings - Add comprehensive tests for the new functionality * Add bank statement import with AI extraction - Create ImportBankStatement assistant function for MCP - Add BankStatementExtractor with chunked processing for small context windows - Register function in assistant configurable - Make PdfImport#pdf_file_content public for extractor access - Increase OpenAI request timeout to 600s for slow local models - Increase DB connection pool to 20 for concurrent operations Tested with M-Pesa bank statement via remote Ollama (qwen3:8b): - Successfully extracted 18 transactions - Generated CSV and created TransactionImport - Works with 3000 char chunks for small context windows * Add pdf-reader gem dependency The BankStatementExtractor uses PDF::Reader to parse bank statement PDFs, but the gem was not properly declared in the Gemfile. This would cause NameError in production when processing bank statements. Added pdf-reader ~> 2.12 to Gemfile dependencies. * Fix transaction deduplication to preserve legitimate duplicates The previous deduplication logic removed ALL duplicate transactions based on [date, amount, name], which would drop legitimate same-day duplicates like multiple ATM withdrawals or card authorizations. Changed to only deduplicate transactions that appear in consecutive chunks (chunking artifacts) while preserving all legitimate duplicates within the same chunk or non-adjacent chunks. * Refactor bank statement extraction to use public provider method Address code review feedback: - Add public extract_bank_statement method to Provider::Openai - Remove direct access to private client via send(:client) - Update ImportBankStatement to use new public method - Add require 'set' to BankStatementExtractor - Remove PII-sensitive content from error logs - Add defensive check for nil response.error - Handle oversized PDF pages in chunking logic - Remove unused process_native and process_generic methods - Update email copy to reflect feature availability - Add guard for nil document_type in email template - Document pdf-reader gem rationale in Gemfile Tested with both OpenAI (gpt-4o) and Ollama (qwen3:8b): - OpenAI: 49 transactions extracted in 30s - Ollama: 40 transactions extracted in 368s - All encapsulation and error handling working correctly * Update schema.rb with ai_summary and document_type columns * Address PR #808 review comments - Rename :csv_file to :import_file across controllers/views/tests - Add PDF test fixture (sample_bank_statement.pdf) - Add supports_pdf_processing? method for graceful degradation - Revert unrelated database.yml pool change (600->3) - Remove month_start_day schema bleed from other PR - Fix PdfProcessor: use .strip instead of .strip_heredoc - Add server-side PDF magic byte validation - Conditionally show PDF import option when AI provider available - Fix ProcessPdfJob: sanitize errors, handle update failure - Move pdf_file attachment from Import to PdfImport - Document deduplication logic limitations - Fix ImportBankStatement: catch specific exceptions only - Remove unnecessary require 'set' - Remove dead json_schema method from PdfProcessor - Reduce default OpenAI timeout from 600s to 60s - Fix nil guard in text mailer template - Add require 'csv' to ImportBankStatement - Remove Gemfile pdf-reader comment * Fix RuboCop indentation in ProcessPdfJob * Refactor PDF import check to use model predicate method Replace is_a?(PdfImport) type check with requires_csv_workflow? predicate that leverages STI inheritance for cleaner controller logic. * Fix missing 'unknown' locale key and schema version mismatch - Add 'unknown: Unknown Document' to document_types locale - Fix schema version to match latest migration (2026_01_24_180211) * Document OPENAI_REQUEST_TIMEOUT env variable Added to .env.local.example and docs/hosting/ai.md * Rename ALLOWED_MIME_TYPES to ALLOWED_CSV_MIME_TYPES for clarity * Add comment explaining requires_csv_workflow? predicate * Remove redundant required_column_keys from PdfImport Base class already returns [] by default * Add ENV toggle to disable PDF processing for non-vision endpoints OPENAI_SUPPORTS_PDF_PROCESSING=false can be used for OpenAI-compatible endpoints (e.g., Ollama) that don't support vision/PDF processing. * Wire up transaction extraction for PDF bank statements - Add extracted_data JSONB column to imports - Add extract_transactions method to PdfImport - Call extraction in ProcessPdfJob for bank statements - Store transactions in extracted_data for later review * Fix ProcessPdfJob retry logic, sanitize and localize errors - Allow retries after partial success (classification ok, extraction failed) - Log sanitized error message instead of raw message to avoid data leakage - Use i18n for user-facing error messages * Add vision-capable model validation for PDF processing * Fix drag-and-drop test to use correct field name csv_file * Schema bleedover from another branch * Fix drag-drop import form field name to match controller * Add vision capability guard to process_pdf method --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com> Co-authored-by: Juan José Mata <jjmata@jjmata.com> |
||
|
eureka928
|
23e9749ed1 |
Refactor clear_ai_cache to use family_scope pattern
- Move family-scoped queries to models via family_scope class method - Remove hardcoded model names from Enrichable concern - Replace inline rescue with proper respond_to? check - Add count tracking for better logging |
||
|
eureka928
|
029d09685e |
Address PR review feedback
- Preserve user locks: Only unlock attributes where current value still matches what AI set. If user changed the value, they took ownership. - Add nil guard clause for family parameter in ClearAiCacheJob - Add partial failure handling so one model's failure doesn't block the other |
||
|
eureka928
|
46ab1d8373 |
Add AI cache clearing infrastructure
Add ClearAiCacheJob for async cache clearing with low priority. Extend Enrichable concern with clear_ai_cache methods to unlock AI-enriched attributes and delete AI enrichment records. Trigger automatic cache clearing when OpenAI model setting changes. |
||
|
LPW
|
9858b36dc7 |
Centralize sdk_object_to_hash logic in DataHelpers module and update all references for improved reusability and consistency. Add error handling for partial and failed SnapTrade account linking. (#741)
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me> |
||
|
LPW
|
a83f70425f |
Add SnapTrade brokerage integration with full trade history support (#737)
* Introduce SnapTrade integration with models, migrations, views, and activity processing logic. * Refactor SnapTrade activities processing: improve activity fetching flow, handle pending states, and update UI elements for enhanced user feedback. * Update Brakeman ignore file to include intentional redirect for SnapTrade OAuth portal. * Refactor SnapTrade models, views, and processing logic: add currency extraction helper, improve pending state handling, optimize migration checks, and enhance user feedback in UI. * Remove encryption for SnapTrade `snaptrade_user_id`, as it is an identifier, not a secret. * Introduce `SnaptradeConnectionCleanupJob` to asynchronously handle SnapTrade connection cleanup and improve i18n for SnapTrade item status messages. * Update SnapTrade encryption: make `snaptrade_user_secret` non-deterministic to enhance security. --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: luckyPipewrench <luckypipewrench@proton.me> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
LPW
|
7c3af7d85e |
refactor: streamline SimpleFIN connection updates for improved efficiency (#631)
- Introduced `update_access_url!` method to reuse existing SimpleFIN items during reconnections, preserving account linkages. - Refactored `SimplefinConnectionUpdateJob` to update access URLs in place without creating new items or transferring accounts. - Adjusted sync logic to leverage `repair_stale_linkages` for seamless reconnections. - Enhanced `SimplefinItem::Importer` to auto-recover the `good` status if no auth errors are found during sync. - Updated tests to validate in-place updates and preserved account relationships. Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com> |
||
|
soky srm
|
76dc91377c |
Merchants improvements (#594)
* FIX logos * Implement merchant mods * FIX confirm issue * FIX linter * Add recently seen merchants to re-add if needed * Update merge.html.erb * FIX do security check * Add error handling for update failures. |
||
|
soky srm
|
e37c03d1d4 | Implement Run all rules (#582) | ||
|
Ethan
|
3b4ab735b0 |
Add (beta) CoinStats Crypto Wallet Integration with Balance and Transaction Syncing (#512)
* Feat(CoinStats): Scaffold implementation, not yet functional * Feat(CoinStats): Implement crypto wallet balance and transactions * Feat(CoinStats): Add tests, Minor improvements * Feat(CoinStats): Utilize bulk fetch API endpoints * Feat(CoinStats): Migrate strings to i8n * Feat(CoinStats): Fix error handling in wallet link modal * Feat(CoinStats): Implement hourly provider sync job * Feat(CoinStats): Generate docstrings * Fix(CoinStats): Validate API Key on provider update * Fix(Providers): Safely handle race condition in merchance creation * Fix(CoinStats): Don't catch system signals in account processor * Fix(CoinStats): Preload before iterating accounts * Fix(CoinStats): Add no opener / referrer to API dashboard link * Fix(CoinStats): Use strict matching for symbols * Fix(CoinStats): Remove dead code in transactions importer * Fix(CoinStats): Avoid transaction fallback ID collisions * Fix(CoinStats): Improve Blockchains fetch error handling * Fix(CoinStats): Enforce NOT NULL constraint for API Key schema * Fix(CoinStats): Migrate sync status strings to i8n * Fix(CoinStats): Use class name rather than hardcoded string * Fix(CoinStats): Use account currency rather than hardcoded USD * Fix(CoinStats): Migrate from standalone to Provider class * Fix(CoinStats): Fix test failures due to string changes |
||
|
LPW
|
c12c585a0e |
Harden SimpleFin sync: retries, safer imports, manual relinking, and data-quality reconciliation (#544)
* Add tests and enhance logic for SimpleFin account synchronization and reconciliation - Added retry logic with exponential backoff for network errors in `Provider::Simplefin`. - Introduced tests to verify retry functionality and error handling for rate-limit, server errors, and stale data. - Updated `SimplefinItem` to detect stale sync status and reconciliation issues. - Enhanced UI to display stale sync warnings and data integrity notices. - Improved SimpleFin account matching during updates with multi-tier strategy (ID, fingerprint, fuzzy match). - Added transaction reconciliation logic to detect data gaps, transaction count drops, and duplicate transaction IDs. * Introduce `SimplefinConnectionUpdateJob` for asynchronous SimpleFin connection updates - Moved SimpleFin connection update logic to `SimplefinConnectionUpdateJob` to improve response times by offloading network retries, data fetching, and reconciliation tasks. - Enhanced SimpleFin account matching with a multi-tier strategy (ID, fingerprint, fuzzy name match). - Added retry logic and bounded latency for token claim requests in `Provider::Simplefin`. - Updated tests to cover the new job flow and ensure correct account reconciliation during updates. * Remove unused SimpleFin account matching logic and improve error handling in `SimplefinConnectionUpdateJob` - Deleted the multi-tier account matching logic from `SimplefinItemsController` as it is no longer used. - Enhanced error handling in `SimplefinConnectionUpdateJob` to gracefully handle import failures, ensuring orphaned items can be manually resolved. - Updated job flow to conditionally set item status based on the success of import operations. * Fix SimpleFin sync: check both legacy FK and AccountProvider for linked accounts * Add crypto, checking, savings, and cash account detection; refine subtype selection and linking - Enhanced `Simplefin::AccountTypeMapper` to include detection for crypto, checking, savings, and standalone cash accounts. - Improved subtype selection UI with validation and warning indicators for missing selections. - Updated SimpleFin account linking to handle both legacy FK and `AccountProvider` associations consistently. - Refined job flow and importer logic for better handling of linked accounts and subtype inference. * Improve `SimplefinConnectionUpdateJob` and holdings processing logic - Fixed race condition in `SimplefinConnectionUpdateJob` by moving `destroy_later` calls outside of transactions. - Updated fuzzy name match logic to use Levenshtein distance for better accuracy. - Enhanced synthetic ticker generation in holdings processor with hash suffix for uniqueness. * Refine SimpleFin entry processing logic and ensure `extra` data persistence - Simplified pending flag determination to rely solely on provider-supplied values. - Fixed potential stale values in `extra` by ensuring deep merge overwrite with `entry.transaction.save!`. * Replace hardcoded fallback transaction description with localized string * Refine pending flag logic in SimpleFin processor tests - Adjust test to prevent falsely inferring pending status from missing posted dates. - Ensure provider explicitly sets pending flag for transactions. * Add `has_many :holdings` association to `AccountProvider` with `dependent: :nullify` --------- Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com> |
||
|
Juan José Mata
|
1af0f91779 |
Rescue me from errors
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
soky srm
|
0300bf9c24 |
Recurring fixes (#454)
* Fix record violation and add toggle for recurring feature * Run only once per sync cycle ( 30 sec ) * FIX params passing * Add collapsible to recurring section * FIX preferences error catch |
||
|
LPW
|
4d3d9d10df |
Address remaining CodeRabbit comments from PR #267 #351 (#451)
* Address remaining CodeRabbit comments from PR #267 This commit addresses the remaining unresolved code review comments: 1. Fix down migration in drop_was_merged_from_transactions.rb - Add null: false, default: false constraints to match original column - Ensures proper rollback compatibility 2. Fix bare rescue in maps_helper.rb compute_duplicate_only_flag - Replace bare rescue with rescue StandardError => e - Add proper logging for debugging - Follows Ruby best practices by being explicit about exception handling These changes improve code quality and follow Rails/Ruby best practices. * Refactor `SimplefinItemsController` and add tests for balances sync and account relinking behavior - Replaced direct sync execution with `SyncJob` for asynchronous handling of balances sync. - Updated account relinking logic to prevent disabling accounts with other active provider links. - Removed unused `compute_relink_candidates` method. - Added tests to verify `balances` action enqueues `SyncJob` and relinking respects account-provider relationships. * Refactor balances sync to use runtime-only `balances_only` flag - Replaced persistent `sync_stats` usage with runtime `balances_only?` predicate via `define_singleton_method`. - Updated `SimplefinItemsController` `balances` action to pass `balances_only` flag to `SyncJob`. - Enhanced `SyncJob` to attach transient `balances_only?` flag for execution. - Adjusted `SimplefinItem::Syncer` logic to rely on the runtime `balances_only?` method. - Updated controller tests to validate runtime flag usage in `SyncJob`. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com> |
||
|
Juan José Mata
|
bf90cad9a0 |
Add Recent Runs visibility for rule executions (#376)
* Add Recent Runs visibility for rule executions Adds a comprehensive tracking system for rule execution history with the following features: - Creates RuleRun model to track execution metadata: * Date/time of execution * Execution type (manual/scheduled) * Success/failure status * Rule reference * Transaction counts (processed and modified) * Error messages for failed runs - Updates RuleJob to automatically record execution results: * Captures transaction processing statistics * Handles success/failure states * Stores error details for debugging - Adds "Recent Runs" section to rules index page: * Paginated display (20 runs per page) * Columnar layout similar to LLM usage page * Visual status indicators (success/failed badges) * Error tooltips for failed runs * Responsive design with design system tokens - Includes i18n translations for all user-facing strings This provides users with visibility into rule execution history, making it easier to debug issues and monitor rule performance. * Update schema.rb with rule_runs table definition * Linter noise * Separate transaction counts into Queued, Processed, and Modified Previously, the code eagerly reported transactions as "processed" when they were only queued for processing. This commit separates the counts into three distinct metrics: - Transactions Queued: Count of transactions matching the rule's filter conditions before any processing begins - Transactions Processed: Count of transactions that were actually processed and modified by the rule actions - Transactions Modified: Count of transactions that had their values changed (currently same as Processed, but allows for future differentiation) Changes: - Add transactions_queued column to rule_runs table - Update RuleJob to track all three counts separately - Update action executors to return count of modified transactions - Update Rule#apply to aggregate modification counts from actions - Add transactions_queued label to locales - Update Recent Runs view to display new column - Add validation for transactions_queued in RuleRun model The tracking now correctly reports: 1. How many transactions matched the filter (queued) 2. How many were actually modified (processed/modified) 3. Distinguishes between matching and modifying transactions * Add Pending status to track async rule execution progress Introduced a new "pending" status for rule runs to properly track async AI operations. The system now: - Tracks pending async jobs with a counter that decrements as jobs complete - Updates transactions_modified incrementally as each job finishes - Only counts transactions that were actually modified (not just queued) - Displays pending status with yellow badge in the UI - Automatically transitions from pending to success when all jobs complete This provides better visibility into long-running AI categorization and merchant detection operations, showing real-time progress as Sidekiq processes the batches. * Fix migration version to 7.2 as per project standards * Consolidate rule_runs migrations into single migration file Merged three separate migrations (create, add_transactions_queued, add_pending_jobs_count) into a single CreateRuleRuns migration. This provides better clarity and maintains a clean migration history. Changes: - Updated CreateRuleRuns migration to include all columns upfront - Removed redundant add_column migrations - Updated schema version to 2025_11_24_000000 * Linter and test fixes * Space optimization * LLM l10n is better than no l10n * Fix implementation for tags/AI rules * Fix tests * Use batch_size * Consider jobs "unknown" status sometimes * Rabbit suggestion * Rescue block for RuleRun.create! --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
LPW
|
a91a4397e9 |
Simplefin liabilities recording fix (#410)
* Add tests and logic for Simplefin account balance normalization - Introduced `SimplefinAccountProcessorTest` to verify balance normalization logic. - Updated `SimplefinAccount::Processor` to invert negative balances for liability accounts (credit cards and loans) while keeping asset balances unchanged. - Added comments to clarify balance conventions and sign normalization rules. * Refactor balances-only sync logic and improve tests for edge cases - Updated `SimplefinItem::Importer` and `SimplefinItem::Syncer` to ensure `last_synced_at` remains nil during balances-only runs, preserving chunked-history behavior for full syncs. - Introduced additional comments to clarify balances-only implications and syncing logic. - Added test case in `SimplefinAccountProcessorTest` to verify correct handling of overpayment for credit card liabilities. - Refined balance normalization in `SimplefinAccount::Processor` to always invert liability balances for consistency. --------- Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com> |
||
|
Josh Waldrep
|
195dd18a52 |
Add tests and refactor for Simplefin holdings import and processing
- Introduced tests for importer post-import logic and `SimplefinHoldingsApplyJob`. - Refactored `ProviderImportAdapter` to improve holding resolution strategy. - Added handling of investment and crypto holdings in importer with debounce logic for job enqueuing. - Updated rake task to use `SimplefinHoldingsApplyJob` for holding materialization. |
||
|
Juan José Mata
|
d6cbf300c7 |
Add scheduled job to sync all accounts every 24 hours (#330)
* Add scheduled job to sync all accounts every 30 minutes Signed-off-by: Nikhil Badyal <nikhill773384@gmail.com> * Change job queue from default to scheduled Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> * Flatten job into single directory * Every 30 minutes is a bit much and will trigger Sentry warnings * Locking and logging improvements * Add support for extra Sidekiq goodies --------- Signed-off-by: Nikhil Badyal <nikhill773384@gmail.com> Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Nikhil Badyal <nikhill773384@gmail.com> |
||
|
LPW
|
61eb611529 |
Simplefin enhancements v2 (#267)
* SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint # Conflicts: # db/schema.rb # Conflicts: # app/controllers/simplefin_items_controller.rb * fix testing * fix linting * xfix linting x2 * Review PR #267 on we-promise/sure (SimpleFin enhancements v2). Address all 15 actionable CodeRabbit comments: Add UUID validations in rakes (e.g., simplefin_unlink), swap Ruby pattern matching/loops for efficient DB queries (e.g., where LOWER(name) LIKE ?), generate docstrings for low-coverage areas (31%), consolidate routes for simplefin_items, move view logic to helpers (e.g., format_transaction_extra), strengthen tests with exact assertions/fixtures for dedup/relink failures. Also, check for overlaps with merged #262 (merchants fix): Ensure merchant creation in simplefin_entry/processor.rb aligns with new payee-based flow and MD5 IDs; add tests for edge cases like empty payees or over-merging pendings. Prioritize security (PII redaction in logs, no hardcoded secrets). * SimpleFin: address CodeRabbit comments (batch 1) - Consolidate simplefin_items routes under a single resources block; keep URLs stable - Replace inline JS with Stimulus auto-relink controller; auto-load relink modal via global modal frame - Improve a11y in relink modal by wrapping rows in labels - Harden unlink rake: default dry_run=true, UUID validation, redact PII in outputs, clearer errors - Backfill rake: default dry_run=true, UUID validation; groundwork for per-SFA counters - Fix-was-merged rake: default dry_run=true, UUID validation; clearer outputs - Idempotent transfer auto-match (find_or_create_by! + RecordNotUnique rescue) - Extract SimpleFin error tooltip assembly into helper and use it in view RuboCop: maintain 2-space indentation, spaces inside array brackets, spaces after commas, and no redundant returns * Linter noise * removed filed commited by mistake. * manual relink flow and tighten composite matching * enforce manual relink UI; fix adapter keywords; guarantee extra.simplefin hash * refactor(simplefin): extract relink service; enforce manual relink UI; tighten composite match; migration 7.2 * add provider date parser; refactor rake; move view queries; partial resilience * run balances-only import in background job. make update flow enqueue balances-only job * persists across all update redirects and initialize used_manual_ids to prevent NameError in relink candidate computation. * SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint * Fixed failed test after rebase. * scan_ruby fix * Calming the rabbit: Fix AccountProvider linking when accounts change Drop the legacy unique index instead of duplicating it Fix dynamic constant assignment Use fixtures consistently; avoid rescue for control flow. Replace bare rescue with explicit exception class. Move business logic out of the view. Critical: Transaction boundary excludes recompute phase, risking data loss. Inconsistency between documentation and implementation for zero-error case. Refactor to use the compute_unlinked_count helper for consistency. Fix cleanup task default: it deletes by default. Move sync stats computation to controller to avoid N+1 queries. Consolidate duplicate sync query. Clarify the intent of setting flash notice on the error path. Fix Date/Time comparison in should_be_inactive?. Move stats retrieval logic to controller. Remove duplicate Sync summary section. Remove the unnecessary sleep statement; use Capybara's built-in waiting. Add label wrappers for accessibility and consistency. * FIX SimpleFIN new account modal Now new account properly loads as a Modal, instead of new page. Fixes also form showing dashboard instead of settings page. * Remove SimpleFin legacy UI components, migrate schema, and refine linking behavior. # Conflicts: # app/helpers/settings_helper.rb * Extract SimpleFin-related logic to `prepare_show_context` helper and refactor for consistency. Adjust conditional checks and ensure controller variables are properly initialized. * Remove unused SimpleFin maps from prepare_show_context; select IDs to avoid N+1 Replace Tailwind bg-green-500 with semantic bg-success in _simplefin_panel/_provider_form Add f.label :setup_token in simplefin_items/new for a11y Remove duplicate require in AccountsControllerSimplefinCtaTest * Remove unnecessary blank lines * Reduce unnecessary changes This reduces the diff against main * Simplefin Account Setup: Display in modal This fixes an issue with the `X` dismiss button in the top right corner * Removed unnecessary comment. * removed unnecessary function. * fixed broken links * Removed unnecessary file * changed to database query * set to use UTC and gaurd against null * set dry_run=true * Fixed comment * Changed to use a database-level query * matched test name to test behavior. * Eliminate code duplication and Time.zone dependency * make final summary surface failures * lint fix * Revised timezone comment. better handle missing selectors. * sanitized LIKE wildcards * Fixed SimpleFin import to avoid “Currency can’t be blank” validation failures when providers return an empty currency string. * Added helper methods for admin and self-hosted checks * Specify exception types in rescue clauses. * Refined logic to determine transaction dates for credit accounts. * Refined stats calculation for `total_accounts` to track the maximum unique accounts per run instead of accumulating totals. * Moved `unlink_all!` logic to `SimplefinItem::Unlinking` concern and deprecated `SimplefinItem::Unlinker`. Updated related references. * Refined legacy unlinking logic, improved `current_holdings` formatting, and added ENV-based overrides for self-hosted checks. * Enhanced `unlink_all!` with explicit error handling, improved transaction safety, and refined ENV-based self-hosted checks. Adjusted exception types and cleaned up private method handling. * Improved currency assignment logic by adding fallback to `current_account` and `family` currencies. * Enhanced error tracking during SimpleFin account imports by adding categorized error buckets, limiting stored errors to the last 5, and improving `stats` calculations. * typo fix * Didn't realize rabbit was still mad... Refactored SimpleFin error handling and CTA logic: centralized duplicate detection and relink visibility into controller, improved task counters, adjusted redirect notices, and fixed form indexing. * Dang rabbit never stops... Centralized SimpleFin maps logic into `MapsHelper` concern and integrated it into relevant controllers and rake tasks. Optimized queries, reduced redundancy, and improved unlinked counts and manual account checks with batch processing. Adjusted task arguments for clarity. * Persistent rabbit. Optimized SimpleFin maps logic by implementing batch queries for manual account and unlinked count checks, reducing N+1 issues. Improved clarity of rake task argument descriptions and error messages for better usability. * Lost a commit somehow, resolved here. Refactored transaction extra details logic by introducing `build_transaction_extra_details` helper to improve clarity, reusability, and reduce view complexity. Enhanced rake tasks with strict dry-run validation and better error handling. Updated schema to allow nullable `merchant_id` and added conditional unique indexes for recurring transactions. * Refactored sensitive data redaction in `simplefin_unlink` task for recursive handling, optimized SQL sanitization in `simplefin_holdings_backfill`, improved error handling in `transactions_helper`, and streamlined day change calculation logic in `Holding` model. * Lint fix * Removed per PR comments. * Also removing per PR comment. * git commit -m "SimpleFIN polish: preserve #manual-accounts wrapper, unify \"manual\" scope, and correct unlinked counts - Preserve #manual-accounts wrapper: switch non-empty updates to turbo_stream.update and background broadcast_update_to; keep empty-path replace to render <div id=\"manual-accounts\"></div> - Unify definition of manual accounts via Account.visible_manual (visible + legacy-nil + no AccountProvider); reuse in controllers, jobs, and helper - Correct setup/unlinked counts: SimplefinItem::Syncer#finalize_setup_counts and maps now consider AccountProvider links (legacy account AND provider must be absent) Deleted: - app/models/simplefin_item/relink_service.rb - app/controllers/concerns/simplefin_items/relink_helpers.rb - app/javascript/controllers/auto_relink_controller.js - app/views/simplefin_items/_relink_modal.html.erb - app/views/simplefin_items/manual_relink.html.erb - app/views/simplefin_items/relink.html.erb - test/services/simplefin_item/relink_service_test.rb Refs: PR #318 unified link/unlink; PR #267 SimpleFIN; follow-up to fix wrapper ID loss and counting drift." * Extend unlinked account check to include "Investment" type * set SimpleFIN item for `balances`, remove redundant unpacking, and improve holdings task error * SimpleFIN: add `errors` action + modal; do not reintroduce legacy relink actions; removed dead helper * FIX simpleFIN linking * Add delay back, tests benefit from it * Put cache back in * Remove empty `rake` task * Small spelling fixes. --------- Signed-off-by: soky srm <sokysrm@gmail.com> Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: sokie <sokysrm@gmail.com> Co-authored-by: Dylan Corrales <deathcamel58@gmail.com> |
||
|
Juan José Mata
|
dfd467ccb5 |
Add "Reset account" followed by sample data preload (#163)
* Add reset with sample data option on profile settings * No need for "member" user in preload * Cleanup/shorten copy |
||
|
Zach Gollwitzer
|
0329a5f211 |
Data exports (#2517)
* Import / export UI * Data exports * Lint fixes, brakeman update |
||
|
Zach Gollwitzer
|
1aae00f586 |
perf(transactions): add kind to Transaction model and remove expensive Transfer joins in aggregations (#2388)
* add kind to transaction model * Basic transfer creator * Fix method naming conflict * Creator form pattern * Remove stale methods * Tweak migration * Remove BaseQuery, write entire query in each class for clarity * Query optimizations * Remove unused exchange rate query lines * Remove temporary cache-warming strategy * Fix test * Update transaction search * Decouple transactions endpoint from IncomeStatement * Clean up transactions controller * Update cursor rules * Cleanup comments, logic in search * Fix totals logic on transactions view * Fix pagination * Optimize search totals query * Default to last 30 days on transactions page if no filters * Decouple transactions list from transfer details * Revert transfer route * Migration reset * Bundle update * Fix matching logic, tests * Remove unused code |
||
|
Zach Gollwitzer
|
854a21993a | Handle ITEM_NOT_FOUND errors on Plaid deletions | ||
|
Zach Gollwitzer
|
e4ee06c9f6 |
Security resolver and health checker (#2281)
* Setup health check * Security health checker cron * Use resolver throughout codebase * Use resolver for trade builder * Add security health checks to schedule * Handle no provider * Lint fixes |
||
|
Zach Gollwitzer
|
f82f77466a | Fix Sentry context for security details exception | ||
|
Zach Gollwitzer
|
9f13b5bb83 |
Handle stale syncs (#2257)
* Handle stale syncs * Use `visible` sync logic in sidebar groups |
||
|
Zach Gollwitzer
|
10f255a9a9 |
Clarify backend data pipeline naming concepts (importers, processors, materializers, calculators, and syncers) (#2255)
* Rename MarketDataSyncer to MarketDataImporter * Materializers * Importers * More reference replacements |
||
|
Zach Gollwitzer
|
6dc1d22672 |
Market data sync refinements (#2252)
* Exchange rate syncer implementation * Security price syncer * Fix issues with provider API * Add back prod schedule * Add back price and exchange rate syncs to account syncs * Remove unused stock_exchanges table |
||
|
Zach Gollwitzer
|
10dd9e061a |
Improve account sync performance, handle concurrent market data syncing (#2236)
* PlaidConnectable concern * Remove bad abstraction * Put sync implementations in own concerns * Sync strategies * Move sync orchestration to Sync class * Clean up sync class, add state machine * Basic market data sync cron * Fix price sync * Improve sync window column names, add timestamps * 30 day syncs by default * Clean up market data methods * Report high duplicate sync counts to Sentry * Add sync states throughout app * account tab session * Persistent account tab selections * Remove manual sleep * Add migration to clear stale syncs on self hosted apps * Tweak sync states * Sync completion event broadcasts * Fix timezones in tests * Cleanup * More cleanup * Plaid item UI broadcasts for sync * Fix account ID namespace conflict * Sync broadcasters * Smoother account sync refreshes * Remove test sync delay |
||
|
Zach Gollwitzer
|
a51c4d2cba |
New onboarding, trials, Stripe integration (#2185)
* New onboarding, trials, Stripe integration * Fix tests * Lint fixes * Fix subscription endpoints |
||
|
Zach Gollwitzer
|
297a695d0f |
Transaction rules engine V1 (#1900)
* Domain model sketch * Scaffold out rules domain * Migrations * Remove existing data enrichment for clean slate * Sketch out business logic and basic tests * Simplify rule scope building and action executions * Get generator working again * Basic implementation + tests * Remove manual merchant management (rules will replace) * Revert "Remove manual merchant management (rules will replace)" This reverts commit 83dcbd9ff0aa7bbee211796b71aa48b71df5e57e. * Family and Provider merchants model * Fix brakeman warnings * Fix notification loader * Update notification position * Add Rule action and condition registries * Rule form with compound conditions and tests * Split out notification types, add CTA type * Rules form builder and Stimulus controller * Clean up rule registry domain * Clean up rules stimulus controller * CTA message for rule when user changes transaction category * Fix tests * Lint updates * Centralize notifications in Notifiable concern * Implement category rule prompts with auto backoff and option to disable * Fix layout bug caused by merge conflict * Initialize rule with correct action for category CTA * Add rule deletions, get rules working * Complete dynamic rule form, split Stimulus controllers by resource * Fix failing tests * Change test password to avoid chromium conflicts * Update integration tests * Centralize all test password references * Add re-apply rule action * Rule confirm modal * Run migrations * Trigger rule notification after inline category updates * Clean up rule styles * Basic attribute locking for rules * Apply attribute locks on user edits * Log data enrichments, only apply rules to unlocked attributes * Fix merge errors * Additional merge conflict fixes * Form UI improvements, ignore attribute locks on manual rule application * Batch AI auto-categorization of transactions * Auto merchant detection, ai enrichment in batches * Fix Plaid merchant assignments * Plaid category matching * Cleanup 1 * Test cleanup * Remove stale route * Fix desktop chat UI issues * Fix mobile nav styling issues |
||
|
Zach Gollwitzer
|
8648f11413 |
Sync hierarchy updates (#2087)
* Add parent sync orchestration * Pass sync object to dependents |
||
|
Zach Gollwitzer
|
1061aacb0f | Set AI queue |