Commit Graph

664 Commits

Author SHA1 Message Date
minion1227
dd707ec91b fix(invitations): allow re-inviting after an unaccepted invite expires (#2543)
* fix(invitations): allow re-inviting after an unaccepted invite expires

An expired, never-accepted invitation still occupies the partial unique
index `index_invitations_on_email_and_family_id_pending` (predicate
`accepted_at IS NULL`), but the `pending` scope excludes it via the
`expires_at > now` clause. The duplicate-guard validation therefore
passes and the INSERT collides with the index, raising
ActiveRecord::RecordNotUnique — surfaced to the admin as a 500 with no
way to re-invite that address through the UI.

Remove the stale expired row inside the create transaction
(before_create) so the unique slot is freed and the re-invite succeeds.
It runs only after validations pass, so a still-pending (non-expired)
invitation can never be removed here. Add a controller-level rescue of
ActiveRecord::RecordNotUnique as a safety net against a concurrent
double-submit race.

Closes #2535

* refactor(invitations): scope RecordNotUnique rescue to the save

The method-level `rescue ActiveRecord::RecordNotUnique` wrapped the whole
create action, so any future write after @invitation.save (e.g.
accept_for) would silently be reported as a generic invite failure.
Extract save_invitation to scope the rescue to the save alone, and add a
regression test that drives the raced unique-index path directly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 08:59:54 +02:00
Orange🍊
94d1e954e1 fix(reports): hide tax-advantaged transactions in breakdown (#2549) 2026-07-08 08:51:22 +02:00
sentry[bot]
7e1713b831 fix(pwa): harden service worker render to explicitly use js format (#2629)
Co-authored-by: sentry[bot] <39604003+sentry[bot]@users.noreply.github.com>
2026-07-08 08:31:51 +02:00
DataEnginr
5935e56a8b Merge branch 'main' into Transfer-charges 2026-07-01 05:50:51 +00:00
Neko
1f036fc526 feat: add preference to disable modal close on outside click (#2226)
* Add a user preference under Settings → Appearance that prevents modals
  from closing when clicking outside them. Useful to avoid accidentally
  losing unsaved form data.

  - Add `disable_modal_click_outside?` helper to User model (JSONB prefs)
  - DS::Dialog reads the user preference as default when not explicitly set
    (existing callers passing disable_click_outside: true/false are unaffected)
  - Wire up save in AppearancesController
  - Add toggle in the Modals section of the Appearance settings page
  - Add i18n strings

* fix(i18n): move modal keys to appearances.show namespace in 7 locales

   The modal translation keys (modals_title, modals_subtitle,
   disable_modal_click_outside_title, disable_modal_click_outside_description)
   were under settings.preferences.show but the view calls t(".modals_title")
   from settings/appearances/show.html.erb. Moved them to
   settings.appearances.show in de, es, nb, nl, ro, tr, and zh-TW.

* refactor(ds): decouple Dialog from Current.user via defaults_provider

---------

Co-authored-by: neko <neko@nixos>
2026-07-01 01:50:20 +02:00
Shibu M
8ccea36517 Merge branch 'we-promise:main' into Transfer-charges 2026-06-30 11:55:59 +05:30
sentry[bot]
24ba7ab6a0 fix(pwa): harden manifest render to explicitly use json format (#2508)
* fix(pwa): harden manifest render to explicitly use json format

* Fix RuboCop spacing in PWA manifest render

---------

Co-authored-by: sentry[bot] <39604003+sentry[bot]@users.noreply.github.com>
Co-authored-by: sure-admin <sure-admin@splashblot.com>
2026-06-30 07:51:51 +02:00
Guillem Arias Fauste
c5ca0431c9 feat(goals): investment-backed goals (Phase 2) (#2491)
* feat(goals): earmark a portion of an account toward a goal

Goals currently count each linked account's whole balance, so an account
shared across goals double-counts and one account can't fund several goals
in distinct slices. Add a per-account earmark — the "GoalBacking" the v1
model already foreshadowed (goal.rb).

- goal_accounts.allocated_amount (nullable). NULL = "dedicate the whole
  balance" (the v1 default: no backfill, existing goals unchanged); a set
  amount reserves a fixed slice.
- Goal#current_balance is now the single chokepoint computing each account's
  backing under a family-wide shared pool: fixed earmarks take their slice,
  an unallocated link takes the remainder, and when fixed earmarks exceed the
  balance every slice is scaled down pro-rata so the goals' shares can never
  sum past the account balance (no double-counting).
- Account#free_to_earmark / #goal_earmarked_total (mirror Budget's
  available_to_allocate) back a soft, non-blocking over-allocation hint.
- GoalsController threads a goal[allocations] hash through create/update.

Phase 1 of the goals earmarking work; investment-backed goals follow.

* feat(goals): earmark UI on the goal form + backing-aware funding breakdown

- Goal form: a per-account "earmark amount" input (blank = whole balance)
  next to each funding-account checkbox, prefilled from the saved
  allocation on edit.
- Goal#account_backing exposes a single linked account's share so the
  funding-accounts breakdown shows each account's earmarked contribution
  and percent instead of its whole balance — keeping the show page
  consistent with the (now allocation-aware) progress ring.
- English strings for the earmark controls and the "earmarked of balance"
  breakdown line.

* fix(goals): address review on the earmark shared-pool math

- Overdrawn (<= 0 balance) accounts now back nothing on both the fixed and
  whole-balance paths. The fixed path previously produced negative backing and
  let a goal claim money the account doesn't hold.
- An archived goal reads its OWN earmark from its own goal_accounts instead of
  the shared pool (which excludes archived goals), so it no longer mis-reports
  the whole account balance for itself.
- goals#index injects one family-wide earmark pool into every card
  (Goal.pooled_allocations_for) instead of querying once per goal (N+1), and
  preloads goal_accounts.
- The projection chart scales its whole-account historical series by the
  backing ratio so the saved line meets current_balance at "today" rather than
  dropping off a cliff for earmarked goals.
- Honest comments: free_to_earmark no longer claims a form warning that doesn't
  exist yet; pace documents its deliberate whole-account basis.

* fix(goals): widen the earmark input so the 'Whole balance' placeholder isn't clipped

* fix(goals): address review on #2490

- autosave: true on goal_accounts so earmark edits to already-linked accounts
  persist through goal.save! (Rails only auto-saves newly built children, so
  changing/clearing an existing earmark was silently dropped). + test.
- Reset the balance/progress memos on AASM transitions, not just the status
  memos, so a same-instance render after complete!/archive! isn't stale. + test.
- backing_ratio is 0 (not 1) when the linked-account total is non-positive, so
  the projection saved series ends at 0 to match the forced-zero current_balance.
- Localize the funding-row subtype label via goals.form.subtypes.*.
- Add the earmark strings to zh-CN (the maintained second locale; goals has no
  ca locale, so Catalan keeps falling back to en like the rest of goals).

* feat(goals): investment-backed goals (Phase 2)

Goals can now be funded by investment accounts, not just depository.

- Relax linked_accounts_must_be_depository -> _must_be_fundable
  (depository || investment); the funding picker + counts include investment
  accounts.
- Add goals.progress_basis ('balance' | 'contributions', default 'balance').
  Investment-backed goals default to 'contributions' so a market swing doesn't
  move the goal: current_balance = value - cumulative market gain
  (Sum of balances.net_market_flows); depository accounts have zero
  net_market_flows, so they're unchanged. Goal#market_value_money shows what
  it's worth today next to the contributed figure on the show page.
- Pledge false-match guard: investment accounts never use manual_save /
  valuation-delta matching (a market move isn't a deposit) - they resolve on
  transfer (cash-inflow) entries only. Guarded in both
  Account#default_pledge_kind and GoalPledge#matches?.
- Add a `reopen` AASM event (completed -> active) + route/action/menu item so a
  manually-completed goal whose value later dips can be reopened.

Stacked on the earmarking branch (#2490). Full suite green; +6 goal tests.

* fix(goals): address Phase 2 review — allocation-aware contributions, N+1, basis-on-update

- Contributions basis now goes through the same earmark/shared-pool logic as
  the balance basis: backing_balance_for -> backing_share_for(account, base),
  where base is the live balance (balance basis) or net contributions
  (contributions basis). Earmarks are respected and shared accounts no longer
  double-count on contributions goals; market_value_money stays consistent.
- Batch the per-account net_market_flows sum (Goal.market_flows_for) and inject
  it on index like pooled_allocations, killing the N+1 for contributions goals.
- Default the basis on update too (not just create), so adding an investment
  account to an existing depository goal flips it to contributions instead of
  silently tracking market value.
- Fix the stale reconciliation_manager comment (renamed validation) and the
  orphaned zh-CN must_be_depository key.

* fix(goals): address review on #2491

- before_save (not before_validation) for the progress_basis default, so a goal
  can be inspected via valid? without its basis flipping as a side effect (jjmata).
- Pledge copy keys off default_pledge_kind, not manual?, so a manual investment
  account — which pledges via transfer — shows the transfer prompt instead of the
  "update your manual balance" flow (codex). pledge_action_label_key and the
  pledge modal's per-account helper flag both use it.
- Add the Phase 2 strings (reopen success/invalid_transition, show.reopen,
  ring.market_value) to zh-CN.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-06-30 07:26:23 +02:00
Shibu M
35f395b39f Merge branch 'we-promise:main' into Transfer-charges 2026-06-30 10:44:23 +05:30
Guillem Arias Fauste
1b403d64e5 feat(goals): earmark a portion of an account toward a goal (Phase 1) (#2490)
* feat(goals): earmark a portion of an account toward a goal

Goals currently count each linked account's whole balance, so an account
shared across goals double-counts and one account can't fund several goals
in distinct slices. Add a per-account earmark — the "GoalBacking" the v1
model already foreshadowed (goal.rb).

- goal_accounts.allocated_amount (nullable). NULL = "dedicate the whole
  balance" (the v1 default: no backfill, existing goals unchanged); a set
  amount reserves a fixed slice.
- Goal#current_balance is now the single chokepoint computing each account's
  backing under a family-wide shared pool: fixed earmarks take their slice,
  an unallocated link takes the remainder, and when fixed earmarks exceed the
  balance every slice is scaled down pro-rata so the goals' shares can never
  sum past the account balance (no double-counting).
- Account#free_to_earmark / #goal_earmarked_total (mirror Budget's
  available_to_allocate) back a soft, non-blocking over-allocation hint.
- GoalsController threads a goal[allocations] hash through create/update.

Phase 1 of the goals earmarking work; investment-backed goals follow.

* feat(goals): earmark UI on the goal form + backing-aware funding breakdown

- Goal form: a per-account "earmark amount" input (blank = whole balance)
  next to each funding-account checkbox, prefilled from the saved
  allocation on edit.
- Goal#account_backing exposes a single linked account's share so the
  funding-accounts breakdown shows each account's earmarked contribution
  and percent instead of its whole balance — keeping the show page
  consistent with the (now allocation-aware) progress ring.
- English strings for the earmark controls and the "earmarked of balance"
  breakdown line.

* fix(goals): address review on the earmark shared-pool math

- Overdrawn (<= 0 balance) accounts now back nothing on both the fixed and
  whole-balance paths. The fixed path previously produced negative backing and
  let a goal claim money the account doesn't hold.
- An archived goal reads its OWN earmark from its own goal_accounts instead of
  the shared pool (which excludes archived goals), so it no longer mis-reports
  the whole account balance for itself.
- goals#index injects one family-wide earmark pool into every card
  (Goal.pooled_allocations_for) instead of querying once per goal (N+1), and
  preloads goal_accounts.
- The projection chart scales its whole-account historical series by the
  backing ratio so the saved line meets current_balance at "today" rather than
  dropping off a cliff for earmarked goals.
- Honest comments: free_to_earmark no longer claims a form warning that doesn't
  exist yet; pace documents its deliberate whole-account basis.

* fix(goals): widen the earmark input so the 'Whole balance' placeholder isn't clipped

* fix(goals): address review on #2490

- autosave: true on goal_accounts so earmark edits to already-linked accounts
  persist through goal.save! (Rails only auto-saves newly built children, so
  changing/clearing an existing earmark was silently dropped). + test.
- Reset the balance/progress memos on AASM transitions, not just the status
  memos, so a same-instance render after complete!/archive! isn't stale. + test.
- backing_ratio is 0 (not 1) when the linked-account total is non-positive, so
  the projection saved series ends at 0 to match the forced-zero current_balance.
- Localize the funding-row subtype label via goals.form.subtypes.*.
- Add the earmark strings to zh-CN (the maintained second locale; goals has no
  ca locale, so Catalan keeps falling back to en like the rest of goals).

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-06-30 06:55:29 +02:00
Shibu M
9481a41aa9 Merge branch 'we-promise:main' into Transfer-charges 2026-06-29 11:45:02 +05:30
HairyHook
8bc20cb9a5 Fixes issue #2415 - Subcategories are not alphabetically ordered like Categories (#2429)
* Update categories_controller.rb

First change required to fix subcategories not sorted alphabetically.

Signed-off-by: HairyHook <63165721+HairyHook@users.noreply.github.com>

* Subcategory ordering fix 1 in category.rb

Signed-off-by: HairyHook <63165721+HairyHook@users.noreply.github.com>

* Update categories_controller.rb

Signed-off-by: HairyHook <63165721+HairyHook@users.noreply.github.com>

---------

Signed-off-by: HairyHook <63165721+HairyHook@users.noreply.github.com>
2026-06-29 00:37:59 +02:00
Juan José Mata
b3f70c8951 feat:Add SnapTrade OAuth device flow (#2523)
* Add SnapTrade OAuth connection flow

* Restore SnapTrade brokerage portal links

* Guard SnapTrade OAuth setup completion

* Move SnapTrade OAuth start to POST

* Use one SnapTrade item in provider panel

* Fix SnapTrade OAuth controller tests

* Fix SnapTrade OAuth drawer completion redirect

* Update SnapTrade limits message.

* Restrict SnapTrade OAuth scopes
2026-06-29 00:30:03 +02:00
Shibu M
3e32b7e3f3 Merge branch 'we-promise:main' into Transfer-charges 2026-06-29 03:00:25 +05:30
Orange🍊
30780a5961 refactor(api): scope controllers through current_resource_owner (#2414)
Follow-up to #2405. Replace remaining API controller reads of
Current.user, Current.family, and Current.session with
current_resource_owner. Add an architecture guard test to prevent
regression.

Scope is limited to the Current sweep only:
- Revert balance_sheet user-scoping (moves to account-auth PR B).
- Revert provider_connections DebugLogEntry logging (separate PR).
- Remove UsersController#destroy attempt to destroy unsaved API session.
2026-06-28 21:59:44 +02:00
DataEnginr
e75f2a0c78 Fix fee display consistency, derive fees from entries, clean schema churn
- Show principal-only transfer amounts on both sides with separate fee and total lines (fixes inconsistent gross/net convention)
- Derive displayed fee amounts from fee_transactions entries (single source of truth) instead of stored columns
- Remove stored source_fee_amount/destination_fee_amount columns from transfers table
- Add foreign key for transactions.transfer_id -> transfers.id (replaces invalid CHECK subquery)
- Move destination fee line inside destination side div for consistent layout
- Remove orphaned view_fee_transaction locale keys from 7 locale files
- Rebuild schema.rb from origin/main to eliminate unrelated column reordering churn
2026-06-28 19:47:53 +00:00
DataEnginr
1b21c4dd7b Store fees as separate expense transactions with principal-only entries
Entries now hold principal only (no fee baked into amounts). Fee transactions created as standard kind with Fees category. Transfer#amount_abs returns principal from new amount column. Update handler recomputes entries and fee transactions on edit. Remove dead source_principal/destination_principal helpers. Schema regenerated cleanly with only transfer fee columns.
2026-06-28 18:39:28 +00:00
Shibu M
eccf050521 Merge branch 'we-promise:main' into Transfer-charges 2026-06-28 22:31:14 +05:30
DataEnginr
945a08b0c2 Merge remote-tracking branch 'upstream/main' into feature/exclude-from-reports
# Conflicts:
#	db/schema.rb
2026-06-27 17:44:14 +00:00
Shibu M
354ece074d Merge branch 'we-promise:main' into Transfer-charges 2026-06-27 23:09:05 +05:30
Juan José Mata
b416618558 Add SnapTrade OAuth device flow (#2494)
* Add SnapTrade OAuth device flow

* Add SnapTrade OAuth device flow

### Motivation
- Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint.
- Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling.

### Description
- Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers.
- Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly.
- Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity.
- Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses.
- Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI.
- Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence.

### Testing
- Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed.
- Ran `bin/rubocop` against modified files and no offenses were reported.

* Add SnapTrade OAuth device flow

### Motivation
- Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint.
- Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling.

### Description
- Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers.
- Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly.
- Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity.
- Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses.
- Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI.
- Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence.

### Testing
- Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed.
- Ran `bin/rubocop` against modified files and no offenses were reported.

* Add SnapTrade OAuth device flow

### Motivation
- Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint.
- Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling.

### Description
- Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers.
- Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly.
- Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity.
- Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses.
- Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI.
- Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence.

### Testing
- Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed.
- Ran `bin/rubocop` against modified files and no offenses were reported.

* Add SnapTrade OAuth device flow

### Motivation
- Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint.
- Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling.

### Description
- Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers.
- Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly.
- Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity.
- Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses.
- Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI.
- Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence.

### Testing
- Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed.
- Ran `bin/rubocop` against modified files and no offenses were reported.

* Add SnapTrade OAuth device flow (provider, model, controller, routes, migration, tests)

### Motivation

- Add support for SnapTrade OAuth 2.0 device authorization flow so users can authorize SnapTrade via device codes in addition to the existing portal flow.
- Persist OAuth token metadata on `SnaptradeItem` for subsequent polling and usage by the provider and UI.

### Description

- Implemented OAuth device flow in the provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token`, plus HTTP helper methods `oauth_connection`, `oauth_client_id`, and `parse_oauth_response` in `Provider::Snaptrade`.
- Added controller endpoints `start_oauth_device_flow` and `complete_oauth_device_flow` in `SnaptradeItemsController` with robust error handling and helpers `oauth_error_payload` and `parse_oauth_error_body` to surface OAuth error fields.
- Extended `SnaptradeItem` with encrypted columns for `oauth_access_token`, `oauth_refresh_token`, token metadata, `oauth_token_active?`, and model methods `start_oauth_device_flow` and `complete_oauth_device_flow!` to store token metadata.
- Added migration `AddOauthDeviceFlowToSnaptradeItems` and updated `db/schema.rb` to include the new columns, registered a new initializer `config/initializers/snaptrade.rb` to read `SNAPTRADE_OAUTH_CLIENT_ID`, and updated `.env*.example` files to document `SNAPTRADE_OAUTH_CLIENT_ID`.
- Exposed new routes `start_oauth_device_flow` and `complete_oauth_device_flow` for `snaptrade_items`.

### Testing

- Added unit tests for provider OAuth behavior in `test/models/provider/snaptrade_oauth_test.rb`, model token persistence in `test/models/snaptrade_item_oauth_test.rb`, and controller error propagation in `test/controllers/snaptrade_items_controller_test.rb`.
- Ran the test suite with `bin/rails test` and the full test run (including the new SnapTrade OAuth tests) passed.

* Add SnapTrade OAuth device flow (start/poll), store tokens, and tests

### Motivation
- Add support for SnapTrade OAuth Device Authorization flow so administrators can start device authorization and poll for tokens without exposing provider internals.
- Persist OAuth token metadata on `SnaptradeItem` so the app can reuse and surface token state for SnapTrade integrations.
- Improve error handling and sanitization for OAuth API errors returned by SnapTrade to avoid leaking upstream internals.

### Description
- Introduces new environment examples and initializer: adds `SNAPTRADE_OAUTH_CLIENT_ID` to `.env.local.example`/`.env.test.example` and configures `Rails.configuration.x.snaptrade.oauth_client_id` in `config/initializers/snaptrade.rb`.
- Extends `Provider::Snaptrade` with OAuth device flow support, adding discovery URL, device grant constant, Faraday `oauth_connection`, and methods `oauth_authorization_server_metadata`, `start_device_authorization`, `poll_device_token`, `oauth_client_id`, and `parse_oauth_response`, plus improved retry and API error wrapping.
- Adds DB migration and schema changes to store OAuth fields on `snaptrade_items` (`oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, `oauth_token_expires_at`) and marks those attributes encrypted when ActiveRecord encryption is enabled.
- Adds `SnaptradeItem` helpers `start_oauth_device_flow`, `complete_oauth_device_flow!`, and `oauth_token_active?` to start/poll and persist token metadata.
- Adds controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` with sanitized error responses and helper methods `oauth_error_payload` and `parse_oauth_error_body`, and wires new member routes in `config/routes.rb`.
- Updates `SnaptradeItemsController` before_action lists to include the new actions and adds user-facing error message helper `start_oauth_device_flow_error_message`.

### Testing
- Added unit tests `test/models/provider/snaptrade_oauth_test.rb` to validate discovery, device authorization request, token polling, missing client ID handling, and OAuth error propagation, and all assertions passed.
- Added `test/models/snaptrade_item_oauth_test.rb` to assert `complete_oauth_device_flow!` stores tokens and expiry and that `oauth_token_active?` reports correctly, and the test passed.
- Extended `test/controllers/snaptrade_items_controller_test.rb` with controller-level tests confirming sanitized OAuth error payloads and behavior for start/complete endpoints, and these controller tests passed.

* Add SnapTrade OAuth device flow support and token storage

### Motivation

- Add support for the OAuth 2.0 device authorization flow for SnapTrade so users can link brokerages via a device-code flow without traditional browser-based client redirects.
- Persist OAuth token metadata on the SnaptradeItem so tokens can be reused and expiration tracked.
- Surface and sanitize provider error payloads to callers while avoiding leakage of internal configuration details.

### Description

- Added new DB columns and a migration (`oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, `oauth_token_expires_at`) and updated `db/schema.rb` to reflect the change.
- Encrypted the new token fields on `SnaptradeItem` and added `oauth_token_active?`, `start_oauth_device_flow`, and `complete_oauth_device_flow!` helpers to the model.
- Implemented OAuth logic in `Provider::Snaptrade` including discovery (`OAUTH_DISCOVERY_URL`), `start_device_authorization`, `poll_device_token`, request parsing, retry handling, and a small Faraday connection wrapper; added configuration accessors for `oauth_client_id` via `config.x.snaptrade`.
- Added controller endpoints `start_oauth_device_flow` and `complete_oauth_device_flow` with safe error handling and helpers to format OAuth error payloads, and registered new member routes for `snaptrade_items`.
- Added an initializer to load `SNAPTRADE_OAUTH_CLIENT_ID` into `Rails.configuration.x.snaptrade`, and updated `.env.local.example` and `.env.test.example` to include the new env var.
- Added unit tests for provider OAuth behavior (`test/models/provider/snaptrade_oauth_test.rb`), SnaptradeItem token persistence (`test/models/snaptrade_item_oauth_test.rb`), and controller error handling (`test/controllers/snaptrade_items_controller_test.rb`), and updated controller tests accordingly.

### Testing

- Ran provider-level tests in `Provider::SnaptradeOauthTest` to validate discovery, device authorization, token polling, and error handling, which passed.
- Ran model tests in `SnaptradeItemOauthTest` to verify token metadata is stored and `oauth_token_active?` works, which passed.
- Ran controller tests in `SnaptradeItemsControllerTest` that exercise the start/complete endpoints and error sanitization, which passed.

* Add SnapTrade OAuth device-flow support and token storage

### Motivation

- Add support for SnapTrade OAuth device authorization so users can perform device-code based OAuth without embedding secrets in the browser.
- Persist OAuth token metadata on `SnaptradeItem` and expose programmatic start/complete endpoints while avoiding leaking provider internals on errors.
- Make OAuth client ID configurable via environment or credentials for Sure deployments.

### Description

- Introduce `SNAPTRADE_OAUTH_CLIENT_ID` to `.env` examples and add `config.x.snaptrade.oauth_client_id` initializer for configuration.
- Add migration `AddOauthDeviceFlowToSnaptradeItems` and update `schema.rb` to add `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at` to `snaptrade_items`.
- Persist and encrypt new token fields in `SnaptradeItem` and add helper methods `oauth_token_active?`, `start_oauth_device_flow`, and `complete_oauth_device_flow!`.
- Extend `Provider::Snaptrade` with OAuth discovery, device authorization (`start_device_authorization`), token polling (`poll_device_token`), Faraday-based `oauth_connection`, parsing and error-wrapping logic, and retry handling.
- Add controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` to `SnaptradeItemsController`, plus helper methods to format OAuth error payloads and user-facing error messages.
- Wire up new routes (`post :start_oauth_device_flow`, `post :complete_oauth_device_flow`) and add `config/initializers/snaptrade.rb`.
- Add comprehensive tests for the OAuth flow and controller error handling and update a system test stub to avoid provider leakage during UI tests.

### Testing

- Added `Provider::SnaptradeOauthTest` which stubs discovery, device authorization and token endpoints and asserts request payloads and responses; tests passed.
- Added `SnaptradeItemOauthTest` which verifies token metadata persistence and `oauth_token_active?`; test passed.
- Extended `SnaptradeItemsControllerTest` with scenarios for successful and failing device-flow completion and start flow error handling; tests passed.
- Ran the affected system test changes in `TradesTest` (provider stubbing and modal behavior); the updated tests passed.
2026-06-27 06:43:12 +02:00
Shibu M
846ece9d01 Merge branch 'we-promise:main' into Transfer-charges 2026-06-27 06:32:41 +05:30
DataEnginr
42a075a6db Address all PR review comments 2026-06-27 00:57:03 +00:00
DataEnginr
9723dc9e98 resolved the issues raised post pr 2026-06-27 00:57:03 +00:00
panther
eb0866a678 Add exclude_from_reports option to accounts
Adds a toggle to mark accounts as excluded from all financial reports
while keeping them active and visible individually.

- Migration: add exclude_from_reports boolean column to accounts
- Model: included_in_reports scope
- BalanceSheet: filter excluded from ClassificationGroup and AccountGroup totals,
  HistoricalAccountScope, AccountRow flag
- IncomeStatement: exclude via SQL fragments in Totals, FamilyStats, CategoryStats
- InvestmentStatement/Budget: chain included_in_reports scope
- ReportsController: filter in breakdown view, export queries, trades
- AccountsController: toggle_exclude_from_reports action + route
- UI: DS::Toggle in account form, eye-off indicator in sidebar, menu toggle items
- Locale: all labels in en.yml
- Tests: model scope, controller toggle, income statement/balance sheet filtering
- 5070 tests pass (0 failures, 0 regressions)
2026-06-27 00:57:03 +00:00
Will Wilson
9487e6cbfb Allow multiple active API keys per user (#2077)
* feat(api): allow multiple active API keys per user

Previously a user could hold only one active API key; creating a new one
silently revoked the existing key, breaking any app using it. Allow
multiple named active keys instead.

- Drop the one-active-key-per-source validation; add name uniqueness
  among the user's active visible keys (revoked names are reusable,
  same name allowed across users).
- Rewrite Settings::ApiKeysController as a RESTful collection
  (index/show/new/create/destroy); create no longer revokes existing
  keys, and key lookup is scoped to the current user's active keys.
- resource :api_key -> resources :api_keys.
- Add an API keys list view with per-key revoke and an empty state;
  rewrite the show page for a single key.
- Update i18n (remove single-key copy, pluralise nav label) and tests.

* refactor(api): address review on multiple API keys

- Remove unreachable destroy branches (cannot_revoke is guarded by the
  .visible 404; revoke! raises rather than returning false) and document
  that .visible is the demo-key revocation guard.
- Delete the orphaned created.html.erb / created.turbo_stream.erb
  templates (no action renders them) and their unused locale keys.
- Extract shared partials (_scope_badges, _status_indicator, _key_meta,
  _key_reveal, _usage) to de-duplicate the index and show views; unify
  the active-status indicator on the standard dot.
- Carry forward the @container / @lg:flex-row / min-w-0 responsive
  fixes from #2079 into the shared key-reveal partial.

* test(api): cover newly-created API key confirmation render

* refactor(api): harden demo-key guard and address review nits

- Document the demo-key revocation guard on ApiKey's `visible` scope (the
  authoritative spot) and add a model test locking the invariant that
  `.visible` excludes the demo monitoring key.
- _scope_badges: use an i18n lookup with a humanize fallback instead of
  bypassing translation for unknown scopes.
- _key_reveal: drop the hard-coded `id` from the shared partial; the
  system test now locates the key via its data-clipboard-target.

* refactor(api-keys): migrate hand-rolled badges to DS::Pill

Replace raw span elements in scope badges and status indicator
with DS::Pill to align with the design system migration convention.

* fix(loans): opening anchor now uses current balance, not original principal

When creating a loan manually, the opening anchor valuation was being
set to `initial_balance` (the original loan principal) instead of
`account.balance` (the current outstanding balance). After the sync
job ran, `account.balance` was overwritten to match the anchor,
making every manually-created loan show its original principal as
the current balance.

Fix by always using `account.balance` for the opening anchor in
`create_and_sync`, and reading `Loan#original_balance` from the
`loans.initial_balance` column directly (with a fallback to
`first_valuation_amount` for provider-synced loans that may not
have the column populated).

* fix(api-keys): strip accidental loan changes; rescue revoke! failures

The fix(loans) commit was accidentally committed into this branch.
Remove the loan-related changes from account.rb, loan.rb, and both
test files, restoring them to their pre-loan-commit state.

Also fix the destroy action: revoke! uses update! internally which
raises ActiveRecord::RecordInvalid on failure rather than returning
false. Add rescue for RecordInvalid and RecordNotDestroyed so failures
produce a flash alert instead of a 500. Re-adds the revoke_failed
locale key that was dropped from settings.api_keys.destroy.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-06-26 06:47:38 +02:00
Guillem Arias Fauste
169fd139b5 fix(chat): surface and recover from undelivered assistant responses (#2436)
* fix(chat): surface and recover from undelivered assistant responses

When the background worker that runs AssistantResponseJob is down or not
polling the high_priority queue, the eager "pending" assistant message had no
safeguard: chat hung on "Thinking…" forever with no error, timeout, or retry,
and the LLM provider was never even called.

Add three layers of resilience:

- Client watchdog (chat_controller.js): a pending "Thinking…" bubble that waits
  past a threshold (default 90s) with no response asks the server to fail it.
  Keyed on the pending marker — which disappears the instant a real response
  streams — so a slow-but-working response is never falsely timed out.

- Server failure capture (Chat#handle_undelivered_response! +
  MessagesController#report_timeout): clears the dead bubble, records a friendly
  "the assistant didn't respond" error with Retry, and writes a DebugLogEntry so
  support can see it in /settings/debug.

- Worker liveness signal (BackgroundJobHealth + warning banner): reads Sidekiq's
  process/queue state directly from Redis in the web process — so a down worker
  is detectable even though the worker is the thing that's broken — and warns
  self-hosted users when no worker polls high_priority or it's badly backed up.
  Fails open so a Sidekiq/Redis blip never blocks chat.

Tests: Chat model (3), MessagesController (2), BackgroundJobHealth (5).

* fix(chat): address review — server-side timeout guard + watchdog retry safety

- Chat#handle_undelivered_response!: gate the state change behind a row lock and
  a server-side minimum age (UNDELIVERED_RESPONSE_TIMEOUT = 60s). The browser
  watchdog is untrusted, so re-read the row under lock and only fail a bubble
  that is still pending AND has genuinely waited past the timeout — never racing
  a worker that is finishing a slow response. (Codex P2 + CodeRabbit)
- chat_controller.js: only mark a report URL as reported on response.ok (fetch
  resolves on HTTP 4xx/5xx, rejecting only on network errors), and guard
  concurrent duplicate POSTs with an in-flight set, so a failed report retries
  instead of stranding the bubble. (CodeRabbit)
- _worker_health_warning: drop the unused `chat:` local + its render arg. (CodeRabbit)
2026-06-25 05:44:44 +02:00
Shibu M
d058bcbdcc Merge branch 'we-promise:main' into Transfer-charges 2026-06-21 01:29:31 +05:30
Jake
dc2a565b6a feat(up): add Up Bank (AU) provider integration (#2391)
* feat(up): add Up Bank (AU) provider integration

Adds Up Bank as a per-family, token-based bank sync provider, modelled on
the existing Akahu integration. Up uses a JSON:API REST API with a personal
access token (Bearer), cursor pagination via links.next, and returns both
HELD (pending) and SETTLED transactions from one endpoint.

New:
- Provider::Up client (JSON:API unwrap, links.next pagination, retries,
  typed errors, /util/ping) + Provider::UpAdapter (Factory-registered,
  Depository + Loan).
- UpItem / UpAccount models with Provided, Unlinking, Syncer,
  SyncCompleteEvent, Importer, Processor, Transactions::Processor, and
  UpEntry::Processor (amount sign flip, HELD->pending, foreignAmount FX,
  merchant from description, stale-pending pruning).
- Family::UpConnectable, UpItemsController, routes, settings panel + connect
  flow views, accounts index wiring, initializer, en locale, and model tests.

Core wiring:
- "up" added to Transaction::PENDING_PROVIDERS, the three pending-match SQL
  blocks in Account::ProviderImportAdapter, Provider::Metadata::REGISTRY,
  ProviderMerchant/DataEnrichment source enums, ProviderConnectionStatus,
  settings provider panels, and financial data reset.

Migration create_up_items_and_accounts must be run before use. No external
API endpoints added (no OpenAPI changes).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(up): dump up tables to schema and make since filter TZ-safe

The feature commit added the up_items/up_accounts migration but never
re-dumped db/schema.rb, leaving the schema version and tables stale.
Add the two table definitions and foreign keys and bump the schema
version so a fresh DB load matches the migration.

Also format a bare Date `since` as UTC midnight instead of the server's
local zone, so `filter[since]` is deterministic regardless of where the
app runs (previously shifted by the local UTC offset).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(up): address code review feedback

Behavior/correctness:
- Persist skipped accounts via a new up_accounts.ignored flag and a
  needs_setup scope, so skipped accounts stop resurfacing as "needs
  setup" on every sync. Linking clears the flag.
- destroy now checks unlink_all! per-account results and aborts deletion
  (alert) if any unlink failed, instead of swallowing failures.
- render_provider_panel_error redirect uses :see_other (was an invalid
  4xx redirect status).
- Up provider adapter falls back to item institution name/url when
  institution_metadata is absent (early return previously blocked it).

Resilience/security:
- fetch_all_resources guards against an API repeating the same
  links.next cursor (Set#add?), preventing infinite pagination.
- HTTP client validates absolute URLs (from links.next) against Up's
  HTTPS host before sending the bearer token, preventing credential
  leakage to untrusted hosts.

Diagnostics:
- Route provider sync/import failures through DebugLogEntry.capture
  (controller, UpItem, syncer, unlinking) with family/account context.
  Low-level HTTP client and currency-normalization warnings keep
  Rails.logger to match existing provider conventions.

Data integrity:
- up_accounts.name and currency are NOT NULL (align with model presence
  validations); account_id stays nullable (allow_nil uniqueness).

Forms:
- select_existing_account radio is required; controller guards a blank/
  unknown up_account_id with a friendly alert instead of RecordNotFound.

Tests:
- Add UpAccount needs_setup scope test, pagination loop guard test,
  untrusted-host rejection test; tighten filter[since] assertion to the
  exact UTC timestamp.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(up): address second-round review feedback

- Capture sync/import failures via DebugLogEntry so swallowed errors in
  account/transaction fetching and transaction processing surface in
  /settings/debug instead of only Rails.logger.
- Gate UP_DEBUG_RAW raw payload dump to local envs to avoid leaking PII
  (merchant names, amounts, account IDs) in managed/production logs.
- Collapse linked/unlinked/total account counts into one memoized query
  instead of 3 separate COUNTs per rendered item.
- Rename "Set Up Up Accounts" locale title to "Link Up Accounts".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(up): add method docstrings and align failed_result keys

Add docstrings to all Up provider source files (controller, models,
providers, concerns) to satisfy the 80% docstring coverage threshold.

Third-round review: failed_result now mirrors import's result shape
(accounts_updated/created/failed, transactions_imported/failed) instead
of the stale accounts_imported key, so failure results stay consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 20:33:09 +02:00
Shibu M
10b0a687e3 Merge branch 'we-promise:main' into Transfer-charges 2026-06-20 11:06:01 +05:30
Orange🍊
6e5f35b306 fix(api): prevent API auth from inheriting impersonation (#2405)
Build fresh API session contexts instead of reusing persisted web sessions that may carry impersonation state.

Reject deactivated report export API key owners and strengthen regression coverage for API key, OAuth, and report export authentication paths.
2026-06-19 17:14:43 +02:00
Artem Danilov
9de9a23ce2 feat(prices): add T-Invest (T-Bank) securities + brand-logo provider (#2408)
Adds Provider::TinkoffInvest, a token-based securities provider built on the
public T-Invest REST gateway (invest-public-api.tinkoff.ru/rest). It serves
prices for Russian instruments (shares, ETF/БПИФ, bonds) and, crucially, brand
logos via the T-Invest CDN — the authoritative logo source for MOEX
instruments, which ISS (MoexPublic) does not provide.

- Registry: register `tinkoff_invest` under the :securities concept; token via
  ENV TINKOFF_INVEST_API_KEY or encrypted Setting.tinkoff_invest_api_key.
- Logos independent of the price provider: Security#import_brand_logo consults
  T-Invest for a logo whenever a token is configured (after the price-provider
  metadata fetch, so it never short-circuits website_url backfill). Gated on
  token presence, not the securities checklist.
- display_logo_url: with no website domain, a stored provider logo (T-Invest)
  now beats the ticker-only Brandfetch lettermark; when a domain exists,
  Brandfetch still wins (unchanged).
- MoexPublic no longer reports moex.com as the issuer website — it's the
  exchange, not the issuer, and would make Brandfetch render the exchange logo
  for every instrument and shadow the real brand logo.
- Prices: GetCandles (daily, paged) + GetLastPrices; Quotation units+nano/1e9;
  bonds priced as percent-of-par x nominal (missing nominal raises, not 0).
- Settings: encrypted token field (always shown) + provider checkbox + en locale.
- Tests for search/info/logo-url/prices/bond/incomplete-candle and display logic.

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-19 17:07:09 +02:00
maverick
f3c2a3122d Merge branch 'we-promise:main' into Transfer-charges 2026-06-16 11:57:07 +05:30
Andrew B
6945b5a296 feat(security): warn when ActiveRecord encryption is not configured (#2362)
* feat(security): warn when ActiveRecord encryption is not configured

Self-hosted instances without explicit ACTIVE_RECORD_ENCRYPTION_* keys (or Rails credentials) store sensitive columns - API keys, provider/bank tokens, the MFA (TOTP) secret, and PII - unencrypted at rest. The app boots and works normally so this plaintext at rest state is easy to miss.

Change: Make it visible:
  - log a clear startup warning (config/initializers/encryption_warning.rb)
  - show a warning banner on /settings/security when encryption is unconfigured

* refactor(security): apply review feedback on encryption warning

- list the three ACTIVE_RECORD_ENCRYPTION_* keys in the banner, rendered via the DS::Alert content block to match the log
- drop the redundant respond_to?(:self_hosted?) guard in the initializer so it matches the controller check
- add a managed-mode test asserting the banner is hidden
2026-06-16 08:11:03 +02:00
Guillem Arias Fauste
c29380ce57 feat(dashboard): masonry packing + per-widget size controls (#2328)
* feat(dashboard): masonry packing + per-widget size controls

In two-column mode the dashboard used a row-based CSS grid, so cards
stretched to equal row height and left dead space (e.g. the Net Worth
chart padded out to match the tall Balance Sheet table). Replace the
row-based layout with masonry packing and add per-widget size guardrails.

- Masonry: CSS grid with computed row-spans + grid-auto-flow: dense, driven
  by a dashboard-masonry Stimulus controller (ResizeObserver + turbo:frame-load).
  The DOM stays a single flat list, so drag/keyboard reorder is unaffected.
  Active only in multi-column mode; single column falls back to normal flow.
- Internal sizing: the net worth chart height is now driven by a
  --dash-widget-h CSS var (fixes an inert flex-1) so the card no longer pads
  out below the chart.
- Guardrails: per-widget layout metadata (col_span, grow, min_height,
  width_toggle) in PagesController, with per-user overrides persisted under
  preferences["dashboard_section_layout"], deep-merged so width and height coexist.
- Size menu: a hover control on size-capable cards — Width (Half/Full) for the
  cashflow sankey and net worth chart, Height (Compact/Auto/Tall) for grow
  widgets. The sankey defaults to full width.

Adds model + controller tests for preference persistence and i18n keys.

* refactor(dashboard): redesign size menu with segmented controls

The size menu used a plain radio list and a diagonal maximize-2 trigger that
collided with the cashflow sankey's modal-expand button. Replace it with a
layout-config popover: a sliders-horizontal trigger plus two labeled axis
groups (Width, Height), each rendered as a DS::SegmentedControl with the
active option filled. Clearer, more compact, and it reads as a card-layout
control. The widget-size controller now mirrors the segmented control's
active-class + aria-pressed contract.

* feat(dashboard): expose width toggle on balance sheet and investments

Tables benefit from horizontal room, so give Balance Sheet and Investments
the same Width (Half/Full) control as the sankey and net worth chart. Height
presets stay chart-only — a table sized to a fixed height would just add
whitespace or force scrolling. The Outflows donut is intentionally left out
(full width is mostly whitespace for a donut).

* fix(dashboard): address review feedback on size controls

- Only apply the full-width col-span and show the Width control when the
  two-column layout is enabled. A full widget previously leaked
  2xl:col-span-2 into the single-column grid, creating an implicit second
  column at 2xl widths and breaking the single-column preference. This also
  keeps the Width control coherent with the Appearance two-column setting
  (it now appears only where it does something). [Codex]
- Stop size-menu keydowns from bubbling to the section reorder handler, so
  keyboard users can open the menu and pick options without entering
  grab/reorder mode. [Codex]
- Harden preferences params: ignore a malformed (non-hash)
  dashboard_section_layout / collapsed_sections instead of raising a 500,
  and require section_order to be an array. [CodeRabbit]
- Localize the dashboard sections aria-label. [CodeRabbit]

* chore(settings): mention per-widget size controls in two-column copy

Surface the new per-widget width/height controls in the Appearance
"Two-column layout" description so the capability is discoverable.

* test(dashboard): assert non-mutation for malformed layout input

Addresses review feedback: asserting assert_nil made the test depend on
the fixture happening to have no dashboard height for net_worth_chart.
Capture the pre-PATCH value and assert it is unchanged, so the test
stays valid (malformed input ignored) even if the fixture later gets a
default height.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Guillem Arias <guillem.arias@col.vueling.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-06-15 20:34:34 +02:00
maverick
333cfe02fb Merge branch 'main' into Transfer-charges 2026-06-13 17:55:41 +05:30
thebandit
64b4c0fee4 Add support for dividend, deposit, withdrawal, and interest trade types to Trades API (#1761)
* Update trades api with support for additional types

* rubocop fixes

* fix missing amount validation for interest type

* define missing schema reference

* fix test api_headers to use display_key per guidelines

* expand test coverage

* replaced duplicate JSON response blocks with helper method

* Add DB assertions to linked transfer test and fix invalid date test

* update brakeman.ignore fingerpint for refactored code

* Update the Brakeman ignore note to document validation for newly permitted keys

* fix API key auth in Minitest test to follow correct pattern

* update required in trades rswag spec to match the minimum fields that apply to all types

* extract dividend handling from build_investment_trade_params to dedicated method

* adjust response format to use the existing jbuilder views for Transfers and Transactions

* normalize type before passing to create form

* validate amount as a positive numeric value + tests

* rubocop fixes

* Add missing Trades API test coverage and docs

- Add Minitest tests for withdrawal (422, transfer linking), interest
  (explicit ticker), and dividend update
- Add rswag 401/403/404/422 response docs for create, update, destroy
- Regenerate docs/api/openapi.yaml

* Update Security.find line reference in brakeman.ignore note

* Mark TransactionResponse account_type as nullable in rswag docs
2026-06-13 11:55:16 +02:00
Shibu M
fe6b2f9872 merged with main 2026-06-12 09:13:24 +00:00
Will Wilson
2075c8c41c feat(mcp): OAuth 2.1 auth for MCP — connect Claude.ai with your Sure login (#2234)
* feat(mcp): add OAuth well-known discovery endpoints (RFC 8414 + RFC 9728)

Serves /.well-known/oauth-protected-resource (RFC 9728) and
/.well-known/oauth-authorization-server (RFC 8414) so MCP clients
can auto-discover the authorization server. Both endpoints are
unauthenticated and respect APP_URL for reverse-proxy deployments.

* feat(mcp): add dynamic client registration endpoint (RFC 7591)

POST /register creates a public Doorkeeper::Application on demand so
MCP clients (e.g. Claude.ai) can self-register without manual setup.
Validates redirect_uris (including blank entries), falls back to
"MCP Client" name, returns no client_secret (public client, PKCE only).
Rate-limited to 10 registrations/min/IP via Rack::Attack.

* feat(mcp): authenticate via Doorkeeper OAuth2, keep MCP_API_TOKEN as fallback

MCP endpoint now accepts OAuth2 Bearer tokens issued by Doorkeeper.
Falls back to the existing MCP_API_TOKEN env-var flow so self-hosted
deployments are not broken. Requires MCP_OAUTH_ENABLED or MCP_API_TOKEN
to be set — the endpoint returns 503 otherwise.

- OauthBase concern provides APP_URL-aware configured_base_url (trailing
  slash stripped to prevent double-slash URLs)
- Bearer scheme parsed case-insensitively (RFC 7235)
- Only read_write scope accepted — read scope would allow mutating tools
  (CreateGoal, ImportBankStatement), so read-only tokens are rejected
- Deactivated users rejected even with a valid Doorkeeper token
- WWW-Authenticate header on 401 points to RFC 9728 resource metadata
- SHA-256 digest used for constant-time env-var comparison
- Rack::Attack throttle added for POST /register
- Routes wired: /.well-known/*, /register, use_doorkeeper

* fix(mcp): disable Turbo on OAuth consent form for external redirect URIs

Turbo was intercepting the authorization form POST and XHR-fetching
the redirect_uri (e.g. https://claude.ai/api/mcp/auth_callback),
which CORS blocks. Extend the existing turbo_disabled guard to cover
any redirect_uri that doesn't originate from the app itself.

* feat(mcp): add Settings::McpController with connected clients view

- Settings > MCP page (under Advanced) shows the MCP server URL with
  copy button and step-by-step instructions for connecting Claude.ai
- Lists active non-mobile OAuth tokens with app name and revoke action;
  mobile device tokens are excluded to prevent accidental disconnection
- Removes the MCP_OAUTH_ENABLED env-var gate — OAuth auth is always
  available since Doorkeeper handles consent; MCP_API_TOKEN remains
  as a self-hosted fallback

* fix(mcp): remove client_credentials from grant_types_supported metadata

Only authorization_code is supported by the registration endpoint.
Advertising client_credentials was misleading — a client that reads
the metadata and attempts that flow would get an application with the
wrong grant type.
2026-06-11 16:19:58 +02:00
maverick
17babb19ba Merge branch 'main' into Transfer-charges 2026-06-10 10:55:06 +05:30
Guillem Arias Fauste
c375b8bf5c feat(ai): self-host settings UI for Anthropic provider (5/5) (#1987)
* feat(ai): add Anthropic provider with chat parity (1/5)

Introduces Provider::Anthropic alongside Provider::Openai, implementing
the LlmConcept chat_response contract over the official anthropic Ruby
SDK. Batch ops, PDF, and RAG land in follow-up PRs.

- Provider::Anthropic uses Messages API for sync and streaming responses
- ChatConfig builds requests with ephemeral prompt-cache markers on the
  system prompt and the last tool definition
- MessageFormatter reconstructs multi-turn history (text + tool_use +
  tool_result blocks) from raw Message records, including the paired
  user-role tool_result turn Anthropic requires after every tool_use
- ChatParser maps Anthropic Message into the shared ChatResponse Data
- Registry, Setting, User, Chat default model wired for ANTHROPIC_*
  envs and Setting.anthropic_*; LLM_PROVIDER selects between providers
- Responder forwards raw conversation_history (Array<Message>) so
  providers without hosted conversation state can rebuild context
- OpenAI provider accepts and ignores the new kwarg (no behavior change)

Tests cover provider init, model gating, MessageFormatter for all turn
shapes, ChatConfig request building (max_tokens, system cache, tool
conversion), ChatParser for text / tool_use / mixed blocks, Registry
discovery, and mocked chat_response success / error / function_request
paths. Live VCR cassettes recorded in a follow-up with a real key.

Stacked PRs: 2/5 batch ops + cost ledger, 3/5 PDF, 4/5 pgvector RAG,
5/5 settings UI + disclosure.

* fix(ai): address PR review on Anthropic provider foundation

Surface fixes raised by Codex + CodeRabbit on PR 1/5:

- Provider::Anthropic#chat_response now accepts (and ignores) a
  `messages:` kwarg. Assistant::Responder passes both `messages:`
  (OpenAI-shape) and `conversation_history:` (raw Message records) for
  cross-provider parity, so the previous signature raised
  ArgumentError on the first chat turn through the Anthropic provider.
- Provider::Anthropic#supports_model? bypasses the `claude` prefix
  gate when a custom base_url is configured, mirroring the OpenAI
  provider. Bedrock-shaped IDs like
  `anthropic.claude-sonnet-4-5-20250929-v1:0` and
  `claude-opus-4@20250514` are otherwise rejected by
  Assistant::Provided#get_model_provider and the chat dies.
- Setting.anthropic_access_token is now in
  EncryptedSettingFields::ENCRYPTED_FIELDS so the Anthropic API key
  is encrypted at rest like every other provider secret. Previously
  plaintext while siblings (openai_access_token, twelve_data_api_key,
  external_assistant_token) were ciphertext.
- Chat.default_model falls back to whichever provider is actually
  configured. Previously, with LLM_PROVIDER=anthropic but no
  Anthropic credentials, the default model resolved to a Claude ID
  that no registered provider supported, so chats failed even when
  OpenAI was fully configured. Adds Provider::{Anthropic,Openai}#configured?
  class methods for the readable callsite.
- Provider::Anthropic.effective_model uses
  `ENV["ANTHROPIC_MODEL"].presence || Setting.anthropic_model` so the
  Setting lookup is only performed when the env var is absent — the
  previous `ENV.fetch(KEY, default)` evaluated the default arg
  eagerly on every call.
- Provider::Anthropic::ChatConfig#anthropic_input_schema strips both
  `:strict` and `"strict"` keys so JSON-decoded schemas with string
  keys cannot leak the OpenAI-only flag through to Anthropic.

Test coverage added: supports_model? bypass on custom endpoints,
chat_response messages: kwarg compatibility, default_model fallback
in the three credential combinations, configured? against ENV +
Setting, strict-flag stripping for both key types, and a
`Setting.expects(:anthropic_model).never` assertion proving the
ENV-precedence test now exercises the lazy path.

All 4365 tests pass (1 pre-existing libvips env error unrelated).

* test(chat): make default_model tests resilient to ENV model overrides

CodeRabbit flagged on PR review: the new default_model tests asserted
against Provider::*::DEFAULT_MODEL, but Chat.default_model actually
returns Provider::*.effective_model.presence (which reads
OPENAI_MODEL / ANTHROPIC_MODEL from the environment). With either env
var set, the tests would fail intermittently even though routing was
correct.

- New default_model tests now assert against the provider's
  effective_model directly, so they verify the routing decision
  (which provider's value wins) without coupling to the constant.
- Pre-existing "creates with default model" assertions had the same
  brittleness; switch them to compare against Chat.default_model so
  the chosen model is whatever the env / Setting cascade resolves to.

Verified by running `ANTHROPIC_MODEL=claude-haiku-4-5 OPENAI_MODEL=gpt-4o
bin/rails test test/models/chat_test.rb` — 16 runs, 0 failures
(previously 2 pre-existing failures + 0 from the new tests).

* fix(ai): address local review on Anthropic foundation

- Provider::Anthropic#supports_pdf_processing? bypasses prefix gate for
  custom endpoints, mirroring supports_model?
- Provider::Anthropic#initialize raises Error when custom_endpoint? AND
  model.blank?, parity with Provider::Openai
- stream_chat_response captures partial usage on mid-stream errors and
  records it via the new on_partial callback so chat_response can skip
  the duplicate error row in the outer rescue
- safe_accumulated_message swallows the secondary failure when the SDK
  cannot reconstruct a snapshot
- langfuse_client memoizes properly (||= instead of =) so repeated calls
  don't churn Langfuse instances
- MessageFormatter sorts tool_calls by created_at then id so the
  message array is deterministic across replays; skips tool_calls
  missing both provider_call_id and provider_id rather than sending
  `id: nil` and getting rejected by Anthropic
- Setting.anthropic_access_token default falls back through
  ENV["ANTHROPIC_API_KEY"].presence (was missing .presence, so an
  empty-string env value bled through)
- User#openai_configured? / #anthropic_configured? delegate to the
  Provider::* class methods — single source of truth
- Assistant::Responder renames the OpenAI-shape history builder
  conversation_history → openai_messages_payload so the kwarg name
  matches the local method name (messages: openai_messages_payload,
  conversation_history: chat_message_records)
- Assistant::Builtin stale-history comment updated to reference both
  builders

Adds a streaming chat_response test using ad-hoc subclasses of the
SDK event types so the case/when dispatch matches via is_a? without
stubbing class-level === behavior.

* test(ai): add Anthropic tool_use round-trip + multi-tool turn coverage

Addresses @jjmata's "worth confirming" note on PR #1983: tool-use turns
from prior assistant messages must round-trip correctly when retrieved
from the database.

- New `ChatParser → ToolCall::Function → MessageFormatter` test walks
  the full path: Anthropic response with a tool_use block →
  ChatFunctionRequest → ToolCall::Function.from_function_request →
  persisted on the AssistantMessage → MessageFormatter rebuild on the
  next turn. Asserts the original `tool_use.id` is preserved end-to-end
  as both `tool_use.id` and the paired `tool_result.tool_use_id`, and
  that the original `input` hash and serialized result content survive.
- New multi-tool assistant turn test confirms two tool_use blocks on a
  single assistant message render as two tool_use blocks followed by
  two paired tool_result blocks in a single user-role follow-up,
  matching Anthropic's required alternation.

Both tests exercise the existing PR1 code without behavior changes.

* test(ai): require "ostruct" explicitly in Anthropic provider tests

OpenStruct is moving out of Ruby's default load path (warning in 3.4+,
removed in 3.5+). Tests work today because ActiveSupport transitively
loads it, but that's incidental. Match the existing convention in
test/controllers/settings/hostings_controller_test.rb which explicitly
requires ostruct for the same reason.

* fix(ai): sanitize Langfuse warn logs, normalize tool_use.input, dedup history fetch

Addresses three open CodeRabbit findings on PR #1983.

- Provider::Anthropic Langfuse rescue branches no longer include
  `e.full_message` in `Rails.logger.warn`. `full_message` bundles the
  backtrace + cause chain and on some SDK error types includes the
  serialized request/response payload (prompt, model output). Logs
  now report `#{e.class}: #{e.message}` only. Three sites:
  create_langfuse_trace, log_langfuse_generation, upsert_langfuse_trace.
  Note: Provider::Openai has the same pattern (copy-pasted source) —
  harmonization deferred to a follow-up cleanup PR; this commit fixes
  only the Anthropic provider to keep PR scope tight.

- MessageFormatter#parse_arguments now coerces any non-Hash parsed
  result to `{}`. Anthropic's Messages API requires `tool_use.input`
  to be a JSON object (map); a stored ToolCall::Function record whose
  arguments parse to a scalar, bool, or array (corrupt row, legacy
  data, cross-provider bleed) would otherwise produce a payload the
  API rejects. Normal flow stores Hash arguments end-to-end so the
  fix is defensive — adds 2 tests covering scalar/array JSON strings
  and non-String non-Hash inputs.

- Assistant::Responder dedups the chat-history fetch. The previous
  layout fired two near-identical `chat.messages.where(...).includes(
  :tool_calls).ordered` queries per LLM turn (one for the OpenAI-shape
  payload, one for the raw-records kwarg). A new memoized
  `complete_chat_messages` fetches once; `chat_message_records` filters
  out the current message via `Array#reject`, `openai_messages_payload`
  iterates the cached array unchanged. One SQL query per turn instead
  of two. Memoization scope = single Responder instance (per LLM call),
  so cache invalidation is not a concern.

All 4370 tests pass (1 pre-existing libvips env error unrelated).
Rubocop + brakeman clean.

* fix(ci): replace sk-ant- prefixed test placeholders

Pipelock secret scanner pattern-matches `sk-ant-*` as a real Anthropic
API key and fails the PR security-scan check. Test stubs and
ClimateControl env values used `sk-ant-test`, `sk-ant-from-setting`,
`sk-ant-x`, `sk-ant-y` as obvious placeholders, but the scanner does
not care about value entropy.

Switched to `fake-anthropic-key-*` / `fake-token-*` strings so the
scanner stops flagging them. No production code touched, no behavior
change — Provider::Anthropic still accepts any non-blank token.

* feat(ai): add Anthropic batch ops + LLM cost ledger (2/5)

Implements auto_categorize, auto_detect_merchants, and
enhance_provider_merchants on Provider::Anthropic via forced tool calls,
plus the cost-ledger plumbing they need.

- Provider::Anthropic::AutoCategorizer, AutoMerchantDetector,
  ProviderMerchantEnhancer each define a single output tool whose
  input_schema mirrors the desired output, then force the model to call
  it via tool_choice: { type: "tool", name: ..., disable_parallel_tool_use: true }.
  Anthropic guarantees the tool_use.input matches the schema, so there
  is no JSON parsing fragility, no <think> tag stripping, and no
  json_object/json_schema fallback ladders.
- Concerns::UsageRecorder mirrors the OpenAI sibling but persists
  cache_creation_input_tokens / cache_read_input_tokens to dedicated
  columns instead of metadata.
- Migration adds cache_creation_tokens, cache_read_tokens (nullable
  integers) to llm_usages. OpenAI rows leave them null.
- LlmUsage::PRICING gains Claude 4.x rows (opus-4-7 $15/$75, sonnet-4-6
  $3/$15, haiku-4-5 $1/$5 per MTok). infer_provider returns "anthropic"
  for claude-* via the existing exact/prefix lookup.
- Provider::Anthropic#chat_response now persists cache columns directly
  rather than stashing them in metadata.
- 25-transaction batch cap mirrors the OpenAI provider so the cost
  ledger sees the same shape regardless of which provider ran a batch.

Tests cover the forced-tool-call path, null/None normalization,
case-insensitive merchant matching, the missing-tool_use error path,
and Anthropic-specific pricing + provider inference on LlmUsage.

Stacked on #1983 (PR 1/5). 3/5 PDF + vision next.

* fix(ai): attribute Bedrock model IDs to anthropic + clean nil enum

- LlmUsage.infer_provider now returns "anthropic" for Bedrock /
  Vertex shaped IDs (anthropic.* and anthropic/*), so cost-ledger
  filtering by provider stays correct even when no per-MTok rate is
  stored. Previously these IDs fell through to the "openai" default.
- AutoCategorizer drops the redundant nil sentinel from the
  category_name enum — the union type [string, null] already permits
  null, and some JSON Schema validators reject nil literals inside
  enum arrays.

* test(ai): require "ostruct" in Anthropic batch op tests

Same rationale as the PR1 ostruct fix — explicit require so the tests
don't depend on ActiveSupport's transitive load when Ruby 3.5+ removes
OpenStruct from the default load path.

* feat(ai): Anthropic native PDF processing (3/5)

Implements process_pdf and extract_bank_statement on Provider::Anthropic
using the native `document` content block — no rasterization, no text
pre-extraction.

- Provider::Anthropic::PdfProcessor classifies the document, summarizes
  it, and extracts statement metadata via a forced report_document_analysis
  tool whose input_schema mirrors the existing Provider::Openai output
  (document_type from Import::DOCUMENT_TYPES, summary, extracted_data).
- Provider::Anthropic::BankStatementExtractor returns the same
  { transactions, period, account_holder, account_number, bank_name,
  opening_balance, closing_balance } shape via report_bank_statement so
  downstream pdf_import code is provider-agnostic.
- Both attach the PDF as
  { type: "document", source: { type: "base64", media_type: "application/pdf", data: <b64> } }
  — Claude 3.5+ / 4.x accept this natively (up to 32MB / 100 pages).
  No pdf-reader, no pdftoppm, no chunking for typical statements.
- supports_pdf_processing? (introduced in PR 1) already returns true for
  claude-* models, gating process_pdf with a clear error otherwise.
- Cost ledger rows are persisted via the shared UsageRecorder concern,
  including cache_creation/cache_read tokens.

Tests verify the document block shape, tool_choice forcing, normalized
document_type for unknown classifications, transaction normalization
(date / amount / reference → notes), and the missing-tool_use error
path. Blank pdf_content raises before any client call.

Stacked on #1984 (PR 2/5). 4/5 pgvector RAG next.

* fix(ai): guard PDF size + surface bank-statement truncation

- PdfProcessor and BankStatementExtractor raise upfront when
  pdf_content.bytesize exceeds MAX_PDF_BYTES (32 MB, matching
  Anthropic's hard limit). Previously a 100 MB PDF would be
  base64-encoded (~133 MB) and packed into the JSON body before
  the API rejected it — peak heap ~270 MB per Sidekiq worker.
- BankStatementExtractor inspects response.stop_reason; when the
  model hit max_tokens it logs a warning and flags result[:truncated]
  so downstream callers know the transaction list may be incomplete.
- ISO date pattern added to statement_period_start/end schema in
  PdfProcessor so the model can't return "March 2026" — Anthropic
  enforces the regex via the tool's input_schema.

Tests cover the size guard (raises before any client.messages call),
truncated-result flagging, and the warning log path.

* test(ai): require "ostruct" in Anthropic PDF tests

Match the explicit ostruct require added in PR1/PR2 — same Ruby 3.5+
load-path reason.

* feat(ai): default Anthropic installs to pgvector RAG (4/5)

The provider-agnostic vector store stack (VectorStore::Pgvector + the
Embeddable concern) already shipped to main. This PR closes the
Anthropic loop:

- VectorStore::Registry.adapter_name now returns :pgvector when
  Setting.llm_provider == "anthropic" and no explicit
  VECTOR_STORE_PROVIDER override is set. Anthropic has no hosted vector
  store, so falling back to the local pgvector adapter is the only
  correct default. Explicit VECTOR_STORE_PROVIDER still wins.
- SearchFamilyFiles surfaces a longer message when no adapter is wired
  up — calling out pgvector + EMBEDDING_URI_BASE as the supported
  Anthropic-only path so the user is not stuck with an "OpenAI required"
  hint that is no longer accurate.

The Embeddable concern already pulls embeddings from
EMBEDDING_URI_BASE / EMBEDDING_ACCESS_TOKEN (with OpenAI as fallback),
so Anthropic installs point this at Voyage AI, a local Ollama instance,
or OpenAI embeddings — independent of the chat provider.

Tests cover the new default routing, the existing OpenAI default
staying intact, and explicit VECTOR_STORE_PROVIDER overriding the
Anthropic default.

Stacked on #1985 (PR 3/5). 5/5 settings UI + retention disclosure next.

* feat(ai): self-host settings UI for Anthropic provider (5/5)

Adds the Anthropic panel and the install-wide LLM provider selector to
the self-hosting settings page, plus a shared data-retention
disclosure that covers both OpenAI and Anthropic.

- New _llm_provider_selector partial: select for Setting.llm_provider
  (openai | anthropic), respects the LLM_PROVIDER env var (disables the
  control + shows the "configured through environment variables" hint
  when set, mirroring the existing OpenAI panel behaviour), and renders
  a compact data-handling block with one-line retention statements for
  each provider.
- New _anthropic_settings partial mirrors _openai_settings exactly:
  password-field for the API key with **** redaction, optional
  base_url (for AWS Bedrock / GCP Vertex), optional default model. All
  three fields disable when their ENV var is set.
- show.html.erb renders provider selector + OpenAI panel + Anthropic
  panel under the same "General" section so users can configure either
  (or both) without switching pages.
- Settings::HostingsController#update now permits and persists
  anthropic_access_token (ignoring the **** placeholder, same pattern
  as OpenAI), anthropic_base_url, anthropic_model, and llm_provider
  (validated against %w[openai anthropic]). On Setting::ValidationError
  the rescue branch preserves anthropic_base_url / anthropic_model
  input so the form re-renders with the user's typed values intact —
  parity with the issue #1824 fix for OpenAI.
- Locale keys added under settings.hostings.{llm_provider_selector,
  anthropic_settings}.

Tests cover token update + placeholder redaction, base_url + model
update, llm_provider switch to anthropic, and rejection of unknown
provider values. The existing GET render test still passes, exercising
all three new partials.

Closes the 5/5 Anthropic series stacked on #1986.

* fix(ai): valid Tailwind token + base_url URL validation

- Data-handling block in _llm_provider_selector swaps the invalid
  bg-surface-secondary token for bg-container-inset, matching the
  inset-card pattern used elsewhere in sure-design-system/components.css.
  bg-surface-secondary is not defined anywhere in the design system —
  Tailwind treated it as a no-op, so the block rendered with no
  background contrast.
- Settings::HostingsController validates anthropic_base_url as a
  URI::HTTP (catches https too) and raises Setting::ValidationError
  with a localized message when the input is not parseable.
  Previously any string was persisted, surfacing as an opaque
  connection error at request time instead of an immediate UX failure.
- Blank base_url now clears the setting (was already the case but
  exercised explicitly in tests now).

* fix(ci): replace sk-ant- prefixed token in hostings controller test

Same pipelock secret-scan trigger as PR1 fix on registry/anthropic
tests. The sk-ant-* prefix is matched verbatim by the scanner
regardless of value entropy.

* fix(ai): provision pgvector table when it is the default store

#1986 makes pgvector the default vector store for Anthropic installs, but
CreateVectorStoreChunks only ran when VECTOR_STORE_PROVIDER=pgvector was set
explicitly — so a fresh Anthropic-only install migrated without the
vector_store_chunks table and failed on uploads/searches.

Add VectorStore::Registry.pgvector_effective? as the single source of truth
for "is pgvector active?" (explicit env OR the Anthropic default), and a new
idempotent migration that enables the extension + creates the table whenever
pgvector is effective and the table is missing — covering fresh and
already-migrated installs without drift. Addresses Codex P1.

* fix(ai): provision pgvector table for Anthropic-default installs

Migration gated on raw VECTOR_STORE_PROVIDER==pgvector, so an
Anthropic-default install (which selects pgvector implicitly via
Setting.llm_provider without setting VECTOR_STORE_PROVIDER) skipped
table creation and failed later on a missing vector_store_chunks
relation. Route through VectorStore::Registry.pgvector_effective? —
the single source of truth already shared by the adapter selection.

Addresses Codex P1 review finding.

* fix(ai): provision pgvector chunks table on schema-load installs

The ensure-migration only helps db:migrate upgraders. Fresh installs go
through bin/docker-entrypoint's db:prepare, which loads schema.rb (the
conditional table can't be dumped there — it needs the vector extension)
and marks every migration applied without running it. An Anthropic-only
fresh install therefore selected the pgvector adapter but had no table,
failing with raw PG errors on first upload or search.

Two layers close it:

- VectorStore::Pgvector#ensure_schema! provisions the table idempotently
  on first use (mirrors CreateVectorStoreChunks; memoized; failures wrap
  in VectorStore::Error, which with_response turns into a clean failed
  response).
- VectorStore::Registry#build_pgvector now gates on
  VectorStore::Pgvector.available? (table exists, or extension present),
  so installs whose Postgres lacks pgvector entirely degrade to the
  assistant's provider_not_configured message instead of raising
  mid-chat.

Also resolves the schema.rb version conflict against main (keep the
branch's 2026_06_01_120000, on top of main's current tables).

* fix(ai): address review nitpicks on pgvector provisioning

- Registry: update the adapter doc comment to mention the
  Anthropic-to-pgvector default alongside the openai fallback.
- ensure_schema!: guard the DDL with if_not_exists instead of a Mutex.
  Adapter instances are built per call and never shared across threads,
  so the realistic race is two processes (web + Sidekiq) provisioning
  concurrently; IF NOT EXISTS makes the loser a no-op where a Mutex
  would only serialize threads inside one process.

* fix(ai): address review on Anthropic settings UI

- Require an Anthropic model when a custom base URL is saved, mirroring the
  OpenAI branch. Auto-submit-on-blur could persist a base URL with no model,
  making Provider::Anthropic raise "Model is required..." on every LLM call.
- Narrow the LLM provider selector copy: only chat honors Setting.llm_provider;
  categorization, merchant detection and PDF processing still always use OpenAI.
  Stop advertising provider switching for those flows until they are wired.
- Reset global Setting.* in test teardown to prevent state leakage, and add a
  test covering the new base-URL-requires-model validation.

* feat(ds): conditional LLM provider settings + merged copy

The self-hosting AI section showed both providers' credential blocks at once
and duplicated near-identical copy. Tidy it:

- Replace the provider <select> with a DS::SegmentedControl driving a new
  provider-settings Stimulus controller: only the active provider's panel is
  shown; switching reveals the other instantly and persists Setting.llm_provider.
- Merge the two byte-identical data-retention lines into one provider-neutral
  Data handling note.
- Scope the token-budget copy to OpenAI-compatible calls (read only by
  Provider::Openai) and add an inline 'add a key to activate' hint when the
  active provider is unconfigured.

UI-only; no provider behavior change.

* feat(ds): responsive LLM provider picker (tabs >=sm, select on mobile)

The segmented tabs overflow a phone viewport once there are 3+ providers
(measured: 4 labels want ~409px in a 319px column at 390px wide). Below sm,
fall back to a native <select> -- which doubles as the submitted field -- while
keeping the segmented tabs at sm and up.

Both controls bind to the same provider-settings Stimulus controller (the
select reads its value, the tabs read data-provider), so adding a 3rd/4th
provider scales on mobile with no layout math.

* fix(hostings): sanitize llm provider selector

* test(hostings): avoid brittle provider hint assertion

---------

Co-authored-by: sure-admin <sure-admin@splashblot.com>
2026-06-09 23:00:04 +02:00
Shibu M
ecf19ed1f9 Merge remote-tracking branch 'upstream/main' into Transfer-charges 2026-06-08 15:30:15 +00:00
ghost
a0f5e56668 perf(transactions): preload new form options (#2189)
* perf(transactions): preload new form options

* refactor(transactions): reuse new form account scope
2026-06-06 17:11:01 +02:00
Blaž Dular
94422955f8 feat(merchants): add raw data import (csv) for merchants (#1992)
* feat(merchants): add csv import endpoint for merchants

* docs: update endpoint docs

* fix(merchant): recommended ai fixes
2026-06-06 16:33:32 +02:00
Juan José Mata
172301f875 Fix SSO provider settings updates (#2210) 2026-06-06 09:14:50 +02:00
Shibu M
e73006af54 Add transfer fee support for bank charges on account-to-account transfers 2026-06-05 11:06:13 +00:00
Tobias Rahloff
fe47c918bb feat(enable_banking): support MFA/decoupled banks and harden session handling (#2174)
Decoupled/MFA banks (e.g. VR Bank in Holstein) were hard-blocked because the
authorize flow aborted whenever auth_methods[0] was DECOUPLED. Enable Banking's
hosted /auth page actually coordinates decoupled SCA and redirects back with a
code, so route these banks through it instead:

- Provider#start_authorization accepts and forwards an auth_method param
- EnableBankingItem#select_auth_method picks the best method
  (REDIRECT > DECOUPLED > EMBEDDED), filtering by psu_type and skipping hidden
  methods
- Shared begin_authorization! re-fetches ASPSP metadata on each authorize and
  reauthorize, so the method is always re-derived (no persistence required)
- Remove the DECOUPLED block in the controller

Also stop the integration from constantly reporting "session expired":

- Only a session-level GET /sessions 401/404 flips the connection to
  requires_update; per-account 401/404 are retried and no longer kill the
  whole connection
- Reconcile session_expires_at from the API's access.valid_until on every sync
- Treat an expired session as a graceful requires_update state instead of
  raising a bare error

No schema changes. Adds covering tests.
2026-06-04 19:53:52 +02:00
Guillem Arias Fauste
6a89efb9c9 fix(goals): default goal currency so it survives a failed create (#2171)
When a new goal failed server-side validation (e.g. no funding account
selected), the currency was never assigned — `goal_params` doesn't permit
`:currency` and it was only derived from the first linked account. On the
re-rendered form the money field fell back to its USD default, so a CHF
(or any non-USD) family saw their target amount silently reset to dollars,
and a spurious "Currency must be filled" error appeared on a field the
user can't even set.

Always default the currency to the first linked account's currency when
present, otherwise the family's primary currency. Currency is now never
blank, so the derived-field error never surfaces and the money field keeps
the family currency across a failed submit.

Fixes we-promise/sure#2170
2026-06-04 11:57:16 +02:00
Guillem Arias Fauste
0d32f7507c fix(goals): scope funding-account picker to the current user's accessible accounts (#2172)
* fix(goals): scope funding-account picker to the current user's accessible accounts

The new/edit goal funding picker and the linkable-account count queried
`Current.family.accounts`, so it listed (and would link/fund from) every
depository account in the family — including accounts owned by other
members that aren't shared with the current user. Switch the three
queries (index count, lookup, picker list) to
`Current.user.accessible_accounts`, matching the access boundary used
elsewhere. Adds controller tests covering the new-form picker and the
create path rejecting a non-accessible same-family account.

Fixes #2168

* fix(goals): preserve inaccessible linked accounts on goal edit

The funding picker only renders Current.user.accessible_accounts, so a
family goal linked to another member's private account renders no
checkbox for it. On update, sync_linked_accounts! treated that omission
as an intentional removal and destroyed the link the editor could not
see. Restrict unlinking to the editor's accessible accounts so links
outside their access are preserved. Adds a regression test.
2026-06-04 11:52:28 +02:00
ghost
6e04c6927d feat(imports): add SureImport session batches (#1785)
* feat(imports): add SureImport session batches

Add first-class SureImport sessions for ordered multi-file NDJSON imports.

Persist source mappings across chunks, make session/chunk processing idempotent, expose progress readback, and keep existing single-file import behavior compatible.

Includes the devcontainer libvips runtime dependency needed by ActiveStorage variant tests.

Addresses #1610.

Related to #1458.

* fix(imports): avoid scanner-like API key test data

* test(imports): assert skipped balances are not persisted

* fix(imports): harden session publish retries

Validate expected import chunk sequences exactly before publish, and restore session state with error details when enqueueing the publish job fails.

* fix(imports): close session retry edge cases

Backfill expected chunk counts after client-session insert races and enqueue import-session jobs after the status transition commits. Persist a safe enqueue failure body so API readback does not expose raw queue errors.

* fix(imports): address session publish review gaps

Remove dead transaction external-id assignment, harden session publish retry/sync behavior, align session chunk status docs, and add regression coverage for partial retries and safe enqueue error readback.

* fix(imports): include sessions in family reset

Clear import sessions through the family reset job so chunk imports and source mappings do not survive a reset.

Expose import session and source mapping counts in the reset status response and regenerated OpenAPI schema so polling reflects the full reset surface.

* test(imports): cover split import mapping invariants

* test(imports): cover session verification invariants

* fix(imports): scope SureImport session reimports

* Tighten SureImport session batching

* fix(imports): export rule source ids for sessions

* test(imports): stabilize rule id export assertion

* test(imports): restore reset status session fixture
2026-06-04 11:48:44 +02:00