mirror of
https://github.com/we-promise/sure.git
synced 2026-07-12 21:05:20 +00:00
401cd6ab08ff06a9e8af00a503a6270f429cd556
532 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Artem Danilov
|
92456936fb |
fix(accounts): persist subtype when it is assigned before accountable_type (#2432)
#2356 build the accountable inside Account#subtype= so a top-level subtype survives create. That fix assumes accountable_type is already set when subtype= runs, but the real controller path violates it: strong-params permit preserves filter order, and account_params lists :subtype before :accountable_type. So on create subtype= runs while accountable_type (and accountable_class) is still blank, the build is skipped, and the chosen subtype is silently dropped — the account renders with the type's fallback label (e.g. a Depository shows 'Cash' instead of 'Savings'). The existing regression test only covered the accountable_type-first order, so it never caught this. Make the writer order-independent: when subtype arrives before the type, stash it and apply it from an accountable_type= override once the type is known. Add regression tests for the permit order and for create_and_sync. |
||
|
Juan José Mata
|
b3f70c8951 |
feat:Add SnapTrade OAuth device flow (#2523)
* Add SnapTrade OAuth connection flow * Restore SnapTrade brokerage portal links * Guard SnapTrade OAuth setup completion * Move SnapTrade OAuth start to POST * Use one SnapTrade item in provider panel * Fix SnapTrade OAuth controller tests * Fix SnapTrade OAuth drawer completion redirect * Update SnapTrade limits message. * Restrict SnapTrade OAuth scopes |
||
|
DataEnginr
|
2c18987a2d | Merge upstream/main into feature/exclude-from-reports | ||
|
Orange🍊
|
d637cd2f75 |
fix(imports): support QIF dd mmm yyyy date format (#2500)
* Fix QIF import for dd mmm yyyy dates (#2498) American Express QIF exports use dd mmm yyyy D-fields (e.g. D26 Jan 2026). Import previously failed with "Unable to detect date format" for two reasons: 1. QifParser.normalize_qif_date stripped all internal whitespace, turning 26 Jan 2026 into the unparseable 26Jan2026. For month-name dates the space IS the separator, so collapse multiples to a single space instead of removing them. Numeric dates keep the existing strip-all behavior. 2. Family::DATE_FORMATS had no candidate mapping to dd mmm yyyy, so detect_date_format could not match it. Add "%d %b %Y" (DD MMM YYYY). Extend the 2-digit-year expansion separator class to include space so month-name dates with 2-digit years also normalize (26 Jan 26 -> 26 Jan 2026). Covered by normalize/parse/detect and Amex-style row-generation regression tests in qif_import_test.rb. * test(qif): cover month-name 2-digit year normalization & parse Addresses CodeRabbit nitpick on #2500: the production change extends the 2-digit-year expansion regex separator class to include a space so month-name dates like "26 Jan 26" normalize to "26 Jan 2026". Add normalize_qif_date and parse_qif_date regressions for that branch. |
||
|
Josh
|
d4b12d7f7a |
Fix SimpleFIN partial auth reconnect status (#2509)
* Fix SimpleFIN partial auth reconnect status * Address SimpleFIN review feedback |
||
|
DataEnginr
|
945a08b0c2 |
Merge remote-tracking branch 'upstream/main' into feature/exclude-from-reports
# Conflicts: # db/schema.rb |
||
|
Juan José Mata
|
b416618558 |
Add SnapTrade OAuth device flow (#2494)
* Add SnapTrade OAuth device flow * Add SnapTrade OAuth device flow ### Motivation - Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint. - Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling. ### Description - Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers. - Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly. - Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity. - Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses. - Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI. - Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence. ### Testing - Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed. - Ran `bin/rubocop` against modified files and no offenses were reported. * Add SnapTrade OAuth device flow ### Motivation - Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint. - Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling. ### Description - Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers. - Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly. - Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity. - Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses. - Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI. - Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence. ### Testing - Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed. - Ran `bin/rubocop` against modified files and no offenses were reported. * Add SnapTrade OAuth device flow ### Motivation - Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint. - Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling. ### Description - Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers. - Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly. - Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity. - Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses. - Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI. - Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence. ### Testing - Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed. - Ran `bin/rubocop` against modified files and no offenses were reported. * Add SnapTrade OAuth device flow ### Motivation - Integrate SnapTrade device-code OAuth so administrators can start a device authorization flow and poll for tokens using SnapTrade's well-known OAuth metadata endpoint. - Persist and encrypt device-flow token material on `SnaptradeItem` to support long-lived API calls and future refresh handling. ### Description - Add OAuth discovery and device-code support to the SnapTrade provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token` helper methods and shared `oauth_connection` / `parse_oauth_response` helpers. - Persist OAuth device-flow tokens on `snaptrade_items` via a migration that adds `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at`, and update `db/schema.rb` accordingly. - Extend `SnaptradeItem` to encrypt stored `oauth_access_token` and `oauth_refresh_token` when ActiveRecord encryption is configured and add `oauth_token_active?` to check token validity. - Add high-level `start_oauth_device_flow` and `complete_oauth_device_flow!` helpers to the `SnaptradeItem::Provided` concern to start authorization and persist token responses. - Expose admin-only member routes and JSON controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` on `SnaptradeItemsController` to drive the device flow from the UI. - Add focused Minitest coverage: `test/models/provider/snaptrade_oauth_test.rb` for provider requests and error handling and `test/models/snaptrade_item_oauth_test.rb` for token persistence. ### Testing - Ran the targeted test suite: `bin/rails test test/models/provider/snaptrade_oauth_test.rb test/models/snaptrade_item_oauth_test.rb test/controllers/snaptrade_items_controller_test.rb`, and all tests passed. - Ran `bin/rubocop` against modified files and no offenses were reported. * Add SnapTrade OAuth device flow (provider, model, controller, routes, migration, tests) ### Motivation - Add support for SnapTrade OAuth 2.0 device authorization flow so users can authorize SnapTrade via device codes in addition to the existing portal flow. - Persist OAuth token metadata on `SnaptradeItem` for subsequent polling and usage by the provider and UI. ### Description - Implemented OAuth device flow in the provider with `oauth_authorization_server_metadata`, `start_device_authorization`, and `poll_device_token`, plus HTTP helper methods `oauth_connection`, `oauth_client_id`, and `parse_oauth_response` in `Provider::Snaptrade`. - Added controller endpoints `start_oauth_device_flow` and `complete_oauth_device_flow` in `SnaptradeItemsController` with robust error handling and helpers `oauth_error_payload` and `parse_oauth_error_body` to surface OAuth error fields. - Extended `SnaptradeItem` with encrypted columns for `oauth_access_token`, `oauth_refresh_token`, token metadata, `oauth_token_active?`, and model methods `start_oauth_device_flow` and `complete_oauth_device_flow!` to store token metadata. - Added migration `AddOauthDeviceFlowToSnaptradeItems` and updated `db/schema.rb` to include the new columns, registered a new initializer `config/initializers/snaptrade.rb` to read `SNAPTRADE_OAUTH_CLIENT_ID`, and updated `.env*.example` files to document `SNAPTRADE_OAUTH_CLIENT_ID`. - Exposed new routes `start_oauth_device_flow` and `complete_oauth_device_flow` for `snaptrade_items`. ### Testing - Added unit tests for provider OAuth behavior in `test/models/provider/snaptrade_oauth_test.rb`, model token persistence in `test/models/snaptrade_item_oauth_test.rb`, and controller error propagation in `test/controllers/snaptrade_items_controller_test.rb`. - Ran the test suite with `bin/rails test` and the full test run (including the new SnapTrade OAuth tests) passed. * Add SnapTrade OAuth device flow (start/poll), store tokens, and tests ### Motivation - Add support for SnapTrade OAuth Device Authorization flow so administrators can start device authorization and poll for tokens without exposing provider internals. - Persist OAuth token metadata on `SnaptradeItem` so the app can reuse and surface token state for SnapTrade integrations. - Improve error handling and sanitization for OAuth API errors returned by SnapTrade to avoid leaking upstream internals. ### Description - Introduces new environment examples and initializer: adds `SNAPTRADE_OAUTH_CLIENT_ID` to `.env.local.example`/`.env.test.example` and configures `Rails.configuration.x.snaptrade.oauth_client_id` in `config/initializers/snaptrade.rb`. - Extends `Provider::Snaptrade` with OAuth device flow support, adding discovery URL, device grant constant, Faraday `oauth_connection`, and methods `oauth_authorization_server_metadata`, `start_device_authorization`, `poll_device_token`, `oauth_client_id`, and `parse_oauth_response`, plus improved retry and API error wrapping. - Adds DB migration and schema changes to store OAuth fields on `snaptrade_items` (`oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, `oauth_token_expires_at`) and marks those attributes encrypted when ActiveRecord encryption is enabled. - Adds `SnaptradeItem` helpers `start_oauth_device_flow`, `complete_oauth_device_flow!`, and `oauth_token_active?` to start/poll and persist token metadata. - Adds controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` with sanitized error responses and helper methods `oauth_error_payload` and `parse_oauth_error_body`, and wires new member routes in `config/routes.rb`. - Updates `SnaptradeItemsController` before_action lists to include the new actions and adds user-facing error message helper `start_oauth_device_flow_error_message`. ### Testing - Added unit tests `test/models/provider/snaptrade_oauth_test.rb` to validate discovery, device authorization request, token polling, missing client ID handling, and OAuth error propagation, and all assertions passed. - Added `test/models/snaptrade_item_oauth_test.rb` to assert `complete_oauth_device_flow!` stores tokens and expiry and that `oauth_token_active?` reports correctly, and the test passed. - Extended `test/controllers/snaptrade_items_controller_test.rb` with controller-level tests confirming sanitized OAuth error payloads and behavior for start/complete endpoints, and these controller tests passed. * Add SnapTrade OAuth device flow support and token storage ### Motivation - Add support for the OAuth 2.0 device authorization flow for SnapTrade so users can link brokerages via a device-code flow without traditional browser-based client redirects. - Persist OAuth token metadata on the SnaptradeItem so tokens can be reused and expiration tracked. - Surface and sanitize provider error payloads to callers while avoiding leakage of internal configuration details. ### Description - Added new DB columns and a migration (`oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, `oauth_token_expires_at`) and updated `db/schema.rb` to reflect the change. - Encrypted the new token fields on `SnaptradeItem` and added `oauth_token_active?`, `start_oauth_device_flow`, and `complete_oauth_device_flow!` helpers to the model. - Implemented OAuth logic in `Provider::Snaptrade` including discovery (`OAUTH_DISCOVERY_URL`), `start_device_authorization`, `poll_device_token`, request parsing, retry handling, and a small Faraday connection wrapper; added configuration accessors for `oauth_client_id` via `config.x.snaptrade`. - Added controller endpoints `start_oauth_device_flow` and `complete_oauth_device_flow` with safe error handling and helpers to format OAuth error payloads, and registered new member routes for `snaptrade_items`. - Added an initializer to load `SNAPTRADE_OAUTH_CLIENT_ID` into `Rails.configuration.x.snaptrade`, and updated `.env.local.example` and `.env.test.example` to include the new env var. - Added unit tests for provider OAuth behavior (`test/models/provider/snaptrade_oauth_test.rb`), SnaptradeItem token persistence (`test/models/snaptrade_item_oauth_test.rb`), and controller error handling (`test/controllers/snaptrade_items_controller_test.rb`), and updated controller tests accordingly. ### Testing - Ran provider-level tests in `Provider::SnaptradeOauthTest` to validate discovery, device authorization, token polling, and error handling, which passed. - Ran model tests in `SnaptradeItemOauthTest` to verify token metadata is stored and `oauth_token_active?` works, which passed. - Ran controller tests in `SnaptradeItemsControllerTest` that exercise the start/complete endpoints and error sanitization, which passed. * Add SnapTrade OAuth device-flow support and token storage ### Motivation - Add support for SnapTrade OAuth device authorization so users can perform device-code based OAuth without embedding secrets in the browser. - Persist OAuth token metadata on `SnaptradeItem` and expose programmatic start/complete endpoints while avoiding leaking provider internals on errors. - Make OAuth client ID configurable via environment or credentials for Sure deployments. ### Description - Introduce `SNAPTRADE_OAUTH_CLIENT_ID` to `.env` examples and add `config.x.snaptrade.oauth_client_id` initializer for configuration. - Add migration `AddOauthDeviceFlowToSnaptradeItems` and update `schema.rb` to add `oauth_access_token`, `oauth_refresh_token`, `oauth_token_type`, `oauth_scope`, and `oauth_token_expires_at` to `snaptrade_items`. - Persist and encrypt new token fields in `SnaptradeItem` and add helper methods `oauth_token_active?`, `start_oauth_device_flow`, and `complete_oauth_device_flow!`. - Extend `Provider::Snaptrade` with OAuth discovery, device authorization (`start_device_authorization`), token polling (`poll_device_token`), Faraday-based `oauth_connection`, parsing and error-wrapping logic, and retry handling. - Add controller actions `start_oauth_device_flow` and `complete_oauth_device_flow` to `SnaptradeItemsController`, plus helper methods to format OAuth error payloads and user-facing error messages. - Wire up new routes (`post :start_oauth_device_flow`, `post :complete_oauth_device_flow`) and add `config/initializers/snaptrade.rb`. - Add comprehensive tests for the OAuth flow and controller error handling and update a system test stub to avoid provider leakage during UI tests. ### Testing - Added `Provider::SnaptradeOauthTest` which stubs discovery, device authorization and token endpoints and asserts request payloads and responses; tests passed. - Added `SnaptradeItemOauthTest` which verifies token metadata persistence and `oauth_token_active?`; test passed. - Extended `SnaptradeItemsControllerTest` with scenarios for successful and failing device-flow completion and start flow error handling; tests passed. - Ran the affected system test changes in `TradesTest` (provider stubbing and modal behavior); the updated tests passed. |
||
|
Markus Laaksonen
|
112b09f425 |
fix(sync): store EnableBanking credit card debt as balance instead of available credit (#2459)
* fix(sync): store EnableBanking credit card debt as balance instead of available credit Previously, the EnableBanking processor forcibly overrode the primary account balance for credit cards to be the available credit (credit_limit - debt) rather than the actual outstanding debt. Since credit cards are modeled as Liability accounts, this caused the balance sheet (net worth) to treat available credit mathematically as a debt. This PR aligns the EnableBanking processor with the Plaid and SimpleFIN processors by storing the absolute debt as the account balance, while tracking the available credit via accountable metadata. Fixes #2458 * docs(sync): update EnableBanking credit card processor documentation Updates the inline processor comments to reflect the new behavior introduced by the previous commit, clarifying that outstanding debt is stored sequentially as the primary balance rather than the UX available credit overriding it. * refactor(sync): clarify balance and available credit calculations in EnableBanking processor Refactors the debt polarity assignments to clarify why liability balances are strictly parsed as absolute positive numbers. Replaces the implicit ordering dependency between the '.abs' conversion and the 'available_credit' math with an explicit 'outstanding_debt' variable to prevent regression by future maintainers. * style: remove trailing whitespace in EnableBanking processor |
||
|
panther
|
eb0866a678 |
Add exclude_from_reports option to accounts
Adds a toggle to mark accounts as excluded from all financial reports while keeping them active and visible individually. - Migration: add exclude_from_reports boolean column to accounts - Model: included_in_reports scope - BalanceSheet: filter excluded from ClassificationGroup and AccountGroup totals, HistoricalAccountScope, AccountRow flag - IncomeStatement: exclude via SQL fragments in Totals, FamilyStats, CategoryStats - InvestmentStatement/Budget: chain included_in_reports scope - ReportsController: filter in breakdown view, export queries, trades - AccountsController: toggle_exclude_from_reports action + route - UI: DS::Toggle in account form, eye-off indicator in sidebar, menu toggle items - Locale: all labels in en.yml - Tests: model scope, controller toggle, income statement/balance sheet filtering - 5070 tests pass (0 failures, 0 regressions) |
||
|
Artem Danilov
|
9532c158dd |
fix(balances): materialize entries dated before the opening anchor (#2434)
Co-authored-by: Tolaria <zeolite-fairing-0k@icloud.com> |
||
|
Will Wilson
|
9487e6cbfb |
Allow multiple active API keys per user (#2077)
* feat(api): allow multiple active API keys per user Previously a user could hold only one active API key; creating a new one silently revoked the existing key, breaking any app using it. Allow multiple named active keys instead. - Drop the one-active-key-per-source validation; add name uniqueness among the user's active visible keys (revoked names are reusable, same name allowed across users). - Rewrite Settings::ApiKeysController as a RESTful collection (index/show/new/create/destroy); create no longer revokes existing keys, and key lookup is scoped to the current user's active keys. - resource :api_key -> resources :api_keys. - Add an API keys list view with per-key revoke and an empty state; rewrite the show page for a single key. - Update i18n (remove single-key copy, pluralise nav label) and tests. * refactor(api): address review on multiple API keys - Remove unreachable destroy branches (cannot_revoke is guarded by the .visible 404; revoke! raises rather than returning false) and document that .visible is the demo-key revocation guard. - Delete the orphaned created.html.erb / created.turbo_stream.erb templates (no action renders them) and their unused locale keys. - Extract shared partials (_scope_badges, _status_indicator, _key_meta, _key_reveal, _usage) to de-duplicate the index and show views; unify the active-status indicator on the standard dot. - Carry forward the @container / @lg:flex-row / min-w-0 responsive fixes from #2079 into the shared key-reveal partial. * test(api): cover newly-created API key confirmation render * refactor(api): harden demo-key guard and address review nits - Document the demo-key revocation guard on ApiKey's `visible` scope (the authoritative spot) and add a model test locking the invariant that `.visible` excludes the demo monitoring key. - _scope_badges: use an i18n lookup with a humanize fallback instead of bypassing translation for unknown scopes. - _key_reveal: drop the hard-coded `id` from the shared partial; the system test now locates the key via its data-clipboard-target. * refactor(api-keys): migrate hand-rolled badges to DS::Pill Replace raw span elements in scope badges and status indicator with DS::Pill to align with the design system migration convention. * fix(loans): opening anchor now uses current balance, not original principal When creating a loan manually, the opening anchor valuation was being set to `initial_balance` (the original loan principal) instead of `account.balance` (the current outstanding balance). After the sync job ran, `account.balance` was overwritten to match the anchor, making every manually-created loan show its original principal as the current balance. Fix by always using `account.balance` for the opening anchor in `create_and_sync`, and reading `Loan#original_balance` from the `loans.initial_balance` column directly (with a fallback to `first_valuation_amount` for provider-synced loans that may not have the column populated). * fix(api-keys): strip accidental loan changes; rescue revoke! failures The fix(loans) commit was accidentally committed into this branch. Remove the loan-related changes from account.rb, loan.rb, and both test files, restoring them to their pre-loan-commit state. Also fix the destroy action: revoke! uses update! internally which raises ActiveRecord::RecordInvalid on failure rather than returning false. Add rescue for RecordInvalid and RecordNotDestroyed so failures produce a flash alert instead of a 500. Re-adds the revoke_failed locale key that was dropped from settings.api_keys.destroy. --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
Frank Li
|
b88deb3545 |
fix: use ES256 instead of EdDSA for Coinbase CDP JWT signing (#1888)
* fix: use ES256 instead of EdDSA for Coinbase CDP JWT signing Coinbase Developer Platform issues EC P-256 keys, not Ed25519. The previous EdDSA implementation would always return Unauthorized for any key generated via portal.cdp.coinbase.com. Switch to ES256 (ECDSA SHA-256) using OpenSSL, and convert the DER-encoded signature to the raw r||s format required by the JWT spec. API Secret field now accepts the full PEM private key directly (including BEGIN/END headers) as provided by Coinbase. * fix(coinbase): address PR review comments - Normalize escaped \n in PEM key before parsing (fixes pasting directly from the Coinbase CDP JSON download file where newlines are \n literals) - Extract parse_ec_private_key helper with docstring explaining both accepted PEM formats - Fix double-space style nit on encoded_header assignment - Remove ed25519 gem from Gemfile (no longer used after ES256 migration) - Add Provider::CoinbaseTest covering: JWT 3-part structure, ES256 alg header, kid claim, CDP payload claims, 64-byte raw r||s signature, escaped-newline key acceptance, and cryptographic signature verification * test(coinbase): fix base64url padding in JWT test assertions * chore: update Gemfile.lock to remove ed25519 gem * Sample credential in test * Comments field, not real key use * test(coinbase): generate EC key dynamically; drop scanner exclusions Generate a fresh P-256 key per test run with OpenSSL::PKey::EC.generate instead of hardcoding a PEM private key. This removes committed key material and the need to exclude the provider and test files from the pipelock secret scan, so those exclusions are removed too. Addresses review findings on hardcoded test key + scan exclusions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Juan José Mata <jjmata@jjmata.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Guillem Arias Fauste
|
169fd139b5 |
fix(chat): surface and recover from undelivered assistant responses (#2436)
* fix(chat): surface and recover from undelivered assistant responses When the background worker that runs AssistantResponseJob is down or not polling the high_priority queue, the eager "pending" assistant message had no safeguard: chat hung on "Thinking…" forever with no error, timeout, or retry, and the LLM provider was never even called. Add three layers of resilience: - Client watchdog (chat_controller.js): a pending "Thinking…" bubble that waits past a threshold (default 90s) with no response asks the server to fail it. Keyed on the pending marker — which disappears the instant a real response streams — so a slow-but-working response is never falsely timed out. - Server failure capture (Chat#handle_undelivered_response! + MessagesController#report_timeout): clears the dead bubble, records a friendly "the assistant didn't respond" error with Retry, and writes a DebugLogEntry so support can see it in /settings/debug. - Worker liveness signal (BackgroundJobHealth + warning banner): reads Sidekiq's process/queue state directly from Redis in the web process — so a down worker is detectable even though the worker is the thing that's broken — and warns self-hosted users when no worker polls high_priority or it's badly backed up. Fails open so a Sidekiq/Redis blip never blocks chat. Tests: Chat model (3), MessagesController (2), BackgroundJobHealth (5). * fix(chat): address review — server-side timeout guard + watchdog retry safety - Chat#handle_undelivered_response!: gate the state change behind a row lock and a server-side minimum age (UNDELIVERED_RESPONSE_TIMEOUT = 60s). The browser watchdog is untrusted, so re-read the row under lock and only fail a bubble that is still pending AND has genuinely waited past the timeout — never racing a worker that is finishing a slow response. (Codex P2 + CodeRabbit) - chat_controller.js: only mark a report URL as reported on response.ok (fetch resolves on HTTP 4xx/5xx, rejecting only on network errors), and guard concurrent duplicate POSTs with an in-flight set, so a failed report retries instead of stranding the bubble. (CodeRabbit) - _worker_health_warning: drop the unused `chat:` local + its render arg. (CodeRabbit) |
||
|
0xmarty_
|
dcf6260a4e |
Fix SimpleFIN epoch-string balance dates (#2397)
Co-authored-by: 0xmarty_ <17337626+marty0x@users.noreply.github.com> |
||
|
Blaž Dular
|
974f1d8e44 |
feat(mcp): add get/create/update tools for tags and categories (#2374)
* feat(mcp): add get/create/update tools for tags and categories * test(mcp): add tests for tag and category assistant functions * fix(mcp): harden category/tag assistant functions against bad tool input * refactor(mcp): use UuidFormat.valid? instead of local UUID_PATTERN * chore(mcp): remove account_filter component files that don't belong in this branch |
||
|
Artem Danilov
|
c65f4828dd |
fix(tinkoff): resolve the tradeable listing and price bonds via BondBy (#2413)
* fix(tinkoff): resolve the tradeable listing and price bonds via BondBy Validated against the live T-Invest API and fixed three correctness issues in Provider::TinkoffInvest: - FindInstrument returns several listings per ticker (e.g. SBER on TQBR plus dark/non-API boards 37M/SPEQ); only the apiTradeAvailableFlag listing has live prices. resolve_short now ranks tradeable-first (then requested-MIC, then exact ticker/ISIN) instead of taking the first match, so GetLastPrices/GetCandles return data. Search now surfaces only tradeable instruments. - find_instruments no longer hard-filters on apiTradeAvailableFlag, so a qualified-investor instrument Tinkoff lists but can't API-trade still resolves (for logos) and otherwise falls back to another price provider. - Bond nominal comes from BondBy (the generic GetInstrumentBy omits it), returning the current amortized nominal so percent-of-par converts correctly. Verified end-to-end: SBER 313.65, T 282.76, LQDT 2.019, and the SFO Split bond (amortized nominal 417.71) all price; logos resolve to real CDN PNGs. * fix(tinkoff): honor requested MIC first, guard amortizing bonds, strip suffix Address review feedback on the resolution/pricing path: - resolve_short now ranks a requested-MIC match BEFORE tradeability, so a security is never priced off another exchange's listing (e.g. a MISX security no longer picks up a tradeable XSPX board's price); tradeability and exact ticker/ISIN remain secondary tie-breaks. - Amortizing bonds: BondBy returns only the current nominal, so applying it to historical percent-of-par closes would underprice them. For amortizing bonds we now return only the live price and skip the candle history; fixed-par bonds and equities keep full history. bond_info exposes nominal + amortizationFlag. - Strip exchange suffixes (.ME/.MOEX/.MISX/.MCX) before querying T-Invest, which only knows the bare SECID — so a stored "T.MOEX" ticker resolves to "T". * fix(tinkoff): don't cache nil resolution results (skip_nil) so a transient empty response isn't locked in for the 24h TTL |
||
|
ghost
|
56000d7834 |
feat(mobile): add SureCard primitive and migrate account cards (#2370)
* feat(mobile): add SureCard primitive and migrate account cards SureCard mirrors the web card chrome — `bg-container` + a hairline border + rounded corners + the subtle DS shadow (`shadowXs`, from the scale shipped in #2349) — resolved from the active SureColors palette so it stays in lockstep with `sure.tokens.json` in light and dark. An optional `onTap` gives a flat ink response clipped to the card radius. Migrates AccountCard off the Material `Card` to `SureCard`. Refs #2235. * fix(coinstats): deterministic wallet batch order in bulk fetches The bulk balance/transaction fetches built the "blockchain:address" param in linked-account query order, which isn't stable across runs — so the batched param (and its mocked expectations) varied seed to seed, intermittently failing CoinstatsItem::ImporterTest and red-flagging unrelated PRs. Sort the wallets before joining so the batch order is deterministic, and align the test expectations to the sorted order. * fix(mobile): tokenize swipe-reveal radius + cover SureCard dark theme Address PR review: - The account-card swipe-reveal background still used a hardcoded radius (12) that mismatched SureCard's radiusLg (10) once the card was migrated — tokenize it so the reveal corners line up during the swipe. - Parameterize the SureCard chrome test over light + dark, since the card is brightness-aware (catches palette regressions in either mode). * test(mobile): assert SureCard onTap ink is clipped to the card radius Address review nit: the onTap test claimed the ink is clipped to the card but only checked the InkWell exists. Now it asserts the InkWell's borderRadius equals SureTokens.radiusLg (tester.widget already enforces a single InkWell). |
||
|
Jake
|
dc2a565b6a |
feat(up): add Up Bank (AU) provider integration (#2391)
* feat(up): add Up Bank (AU) provider integration Adds Up Bank as a per-family, token-based bank sync provider, modelled on the existing Akahu integration. Up uses a JSON:API REST API with a personal access token (Bearer), cursor pagination via links.next, and returns both HELD (pending) and SETTLED transactions from one endpoint. New: - Provider::Up client (JSON:API unwrap, links.next pagination, retries, typed errors, /util/ping) + Provider::UpAdapter (Factory-registered, Depository + Loan). - UpItem / UpAccount models with Provided, Unlinking, Syncer, SyncCompleteEvent, Importer, Processor, Transactions::Processor, and UpEntry::Processor (amount sign flip, HELD->pending, foreignAmount FX, merchant from description, stale-pending pruning). - Family::UpConnectable, UpItemsController, routes, settings panel + connect flow views, accounts index wiring, initializer, en locale, and model tests. Core wiring: - "up" added to Transaction::PENDING_PROVIDERS, the three pending-match SQL blocks in Account::ProviderImportAdapter, Provider::Metadata::REGISTRY, ProviderMerchant/DataEnrichment source enums, ProviderConnectionStatus, settings provider panels, and financial data reset. Migration create_up_items_and_accounts must be run before use. No external API endpoints added (no OpenAPI changes). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(up): dump up tables to schema and make since filter TZ-safe The feature commit added the up_items/up_accounts migration but never re-dumped db/schema.rb, leaving the schema version and tables stale. Add the two table definitions and foreign keys and bump the schema version so a fresh DB load matches the migration. Also format a bare Date `since` as UTC midnight instead of the server's local zone, so `filter[since]` is deterministic regardless of where the app runs (previously shifted by the local UTC offset). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(up): address code review feedback Behavior/correctness: - Persist skipped accounts via a new up_accounts.ignored flag and a needs_setup scope, so skipped accounts stop resurfacing as "needs setup" on every sync. Linking clears the flag. - destroy now checks unlink_all! per-account results and aborts deletion (alert) if any unlink failed, instead of swallowing failures. - render_provider_panel_error redirect uses :see_other (was an invalid 4xx redirect status). - Up provider adapter falls back to item institution name/url when institution_metadata is absent (early return previously blocked it). Resilience/security: - fetch_all_resources guards against an API repeating the same links.next cursor (Set#add?), preventing infinite pagination. - HTTP client validates absolute URLs (from links.next) against Up's HTTPS host before sending the bearer token, preventing credential leakage to untrusted hosts. Diagnostics: - Route provider sync/import failures through DebugLogEntry.capture (controller, UpItem, syncer, unlinking) with family/account context. Low-level HTTP client and currency-normalization warnings keep Rails.logger to match existing provider conventions. Data integrity: - up_accounts.name and currency are NOT NULL (align with model presence validations); account_id stays nullable (allow_nil uniqueness). Forms: - select_existing_account radio is required; controller guards a blank/ unknown up_account_id with a friendly alert instead of RecordNotFound. Tests: - Add UpAccount needs_setup scope test, pagination loop guard test, untrusted-host rejection test; tighten filter[since] assertion to the exact UTC timestamp. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(up): address second-round review feedback - Capture sync/import failures via DebugLogEntry so swallowed errors in account/transaction fetching and transaction processing surface in /settings/debug instead of only Rails.logger. - Gate UP_DEBUG_RAW raw payload dump to local envs to avoid leaking PII (merchant names, amounts, account IDs) in managed/production logs. - Collapse linked/unlinked/total account counts into one memoized query instead of 3 separate COUNTs per rendered item. - Rename "Set Up Up Accounts" locale title to "Link Up Accounts". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(up): add method docstrings and align failed_result keys Add docstrings to all Up provider source files (controller, models, providers, concerns) to satisfy the 80% docstring coverage threshold. Third-round review: failed_result now mirrors import's result shape (accounts_updated/created/failed, transactions_imported/failed) instead of the stale accounts_imported key, so failure results stay consistent. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Artem Danilov
|
9de9a23ce2 |
feat(prices): add T-Invest (T-Bank) securities + brand-logo provider (#2408)
Adds Provider::TinkoffInvest, a token-based securities provider built on the public T-Invest REST gateway (invest-public-api.tinkoff.ru/rest). It serves prices for Russian instruments (shares, ETF/БПИФ, bonds) and, crucially, brand logos via the T-Invest CDN — the authoritative logo source for MOEX instruments, which ISS (MoexPublic) does not provide. - Registry: register `tinkoff_invest` under the :securities concept; token via ENV TINKOFF_INVEST_API_KEY or encrypted Setting.tinkoff_invest_api_key. - Logos independent of the price provider: Security#import_brand_logo consults T-Invest for a logo whenever a token is configured (after the price-provider metadata fetch, so it never short-circuits website_url backfill). Gated on token presence, not the securities checklist. - display_logo_url: with no website domain, a stored provider logo (T-Invest) now beats the ticker-only Brandfetch lettermark; when a domain exists, Brandfetch still wins (unchanged). - MoexPublic no longer reports moex.com as the issuer website — it's the exchange, not the issuer, and would make Brandfetch render the exchange logo for every instrument and shadow the real brand logo. - Prices: GetCandles (daily, paged) + GetLastPrices; Quotation units+nano/1e9; bonds priced as percent-of-par x nominal (missing nominal raises, not 0). - Settings: encrypted token field (always shown) + provider checkbox + en locale. - Tests for search/info/logo-url/prices/bond/incomplete-candle and display logic. Co-authored-by: Claude <noreply@anthropic.com> |
||
|
ghost
|
6ae9779567 |
perf(reports): avoid residual category lazy loads (#2255)
* perf(reports): avoid residual category lazy loads * test(reports): reuse SQL query capture helper Move the reports SQL capture regression helper into test/support and document the category loading and budget reload choices called out in review. |
||
|
ghost
|
894d705e83 |
perf(dashboard): streamline investment activity totals (#2257)
* perf(dashboard): streamline investment activity totals * fix(dashboard): skip empty investment totals cache writes Short-circuit empty investment account totals before cache fetch and move the new SQL query capture regression helper into test/support. * refactor(dashboard): address review on investment totals query - Drop defensive ArgumentError guards in InvestmentStatement::Totals#initialize. The sole caller (totals_query) always passes correct types, and the empty_result early-return already handles the zero-accounts case. - Remove the redundant accounts JOIN and its family_id/status filters from the aggregation SQL. account_ids is already scoped to the family's visible (draft/active) investment accounts, so the join back to accounts is unnecessary work in the query plan. Drop the now-unused family_id param. - Add a regression assertion that the aggregate no longer joins accounts. |
||
|
ghost
|
5811a8df15 |
perf(recurring): batch recurring transaction identification (#2239)
* perf(recurring): batch recurring transaction identification * fix(recurring): clamp manual month-end matching * test(recurring): guard amount-scoped lookup cache |
||
|
Artem Danilov
|
b57bb938da |
feat(prices): add Moscow Exchange (MOEX ISS) securities + FX provider (#2394)
* feat(prices): add Moscow Exchange (MOEX ISS) securities + FX provider Add Provider::MoexPublic, a keyless provider built on the free MOEX ISS API (https://iss.moex.com/iss), modeled on Provider::BinancePublic. Securities: shares, funds/ETF/БПИФ (e.g. LQDT), and bonds (OFZ + corporate). Bonds are priced clean — LAST% × FACEVALUE / 100 in the instrument currency, with per-row FACEVALUE for amortizing issues; NKD/accrued coupon excluded. Exchange rates: also implements ExchangeRateConcept for RUB↔{USD,EUR,CNY} via selt TOM instruments (USD000UTSTOM/EUR_RUB__TOM/CNYRUB_TOM); the selt quote is X/RUB, inverted for RUB→X, nil for non-RUB-crossed pairs. Details: - Board/engine resolution via the ISS primary-board flag with a hardcoded priority fallback (TQBR, TQTF, TQOB, TQCB, …). - Instrument currency from CURRENCYID/FACEUNIT (handles USD/CNY eurobonds & FX funds), normalizing legacy SUR/RUR → RUB; default RUB. - Full history via from/till + start= pagination; current price fallback chain LAST → MARKETPRICE → LCURRENTPRICE → LCLOSEPRICE → PREVPRICE → latest history close. - Bare SECID identity, exchange_operating_mic=MISX, country_code=nil (wildcard like Binance); search accepts .ME/.MOEX/.MISX/.MCX aliases and ISIN. - RateLimitable throttling, SslConfigurable, Faraday retry/timeouts; all public methods wrapped in with_provider_response. Wired into Provider::Registry for both :securities and :exchange_rates, the hosting provider-selection UI, locales, and config/exchanges.yml (MISX). Docker-tested (devcontainer, Ruby 3.4.9): 29 new tests green, full provider suite + i18n green, rubocop clean; smoke-tested against live ISS (SBER price, OFZ clean price, USD/RUB FX). * fix(moex): address review — FX weekend lookback, dead branch, translated hints - fetch_exchange_rate now fetches a 10-day lookback window (not just the exact day) so a weekend/holiday request resolves to the prior trading day's close, matching Yahoo's behavior (Codex P2). - Remove dead identical if/else branches in history_row_price (CodeRabbit). - Translate moex_public_hint into ca/fr/hu/vi/zh-CN instead of English copy (CodeRabbit). - Add a test covering the FX prior-trading-day lookback. * fix(moex): guard ISS date parsing; doc TQTE in board priority Address maintainer review (jjmata): - parse_iss_date wraps Date.parse so a malformed ISS TRADEDATE skips just that row (with a contextual log warning) instead of failing the whole history/FX fetch. Used in history_row_price and fx_history. - Add TQTE to the BOARD_PRIORITY doc comment (it was in the constant but missing from the comment). - Add a test covering the unparseable-date skip. |
||
|
Artem Danilov
|
fffcaf2345 |
fix(prices): resolve dashed crypto tickers (BTC-USD, TRX-USD) via BinancePublic (#2372)
* fix(prices): resolve dashed crypto tickers (BTC-USD, TRX-USD) via BinancePublic
Provider::BinancePublic is the dedicated crypto price provider, but its
search_securities only matched bare base assets ("BTC") or unseparated
Binance pairs ("BTCUSDT"/"BTCUSD"). The canonical "<BASE>-USD" form that
Yahoo emits and users paste (e.g. "TRX-USD", "USDT-USD") never matched, so
those holdings fell through to an unpriced offline security whenever the
stock provider didn't also return the coin.
Collapse the base/quote separator in the search query (and strip it in
parse_ticker) so "BTC-USD" / "TRX/USDT" are treated like "BTCUSD" and
resolve to live Binance pricing. Stablecoins pasted as "USDT-USD" resolve
to the synthetic USD price via their stripped base.
* fix(prices): only synthesize stablecoin search for USD quote
Address review: gate the synthetic stablecoin result on a USD quote (bare
coin or "<coin>USD" form). A non-USD quote like "USDTEUR" / "USDT-EUR"
has a real Binance pair and now falls through to normal matching instead of
being replaced by the USD synthetic. Drops the now-unused base_asset_query
helper. Adds a USDT-EUR regression test.
|
||
|
Augusto Xavier
|
844cba2fd3 |
fix(imports): normalize CSV upload encoding when validations are skipped (#2299)
Import::UploadsController#update persists CSV uploads with save!(validate: false), which skips the before_validation :ensure_utf8_encoding callback. Non-UTF-8 files (e.g. ISO-8859-1 / Windows-1252 exports from Brazilian banks) were written to the text column as-is and rejected by Postgres with PG::CharacterNotInRepertoire, surfacing as a 500 during upload. Also register ensure_utf8_encoding on before_save so the existing normalization (rchardet detection + Latin-1/Windows fallback) still runs when validations are skipped. The callback is idempotent and no-ops on valid UTF-8, so the validated path is unchanged. Fixes #2294 |
||
|
Artem Danilov
|
e342ac4ad1 |
fix(accounts): persist subtype when creating an account (#2356)
The Account#subtype= writer delegates to accountable&.subtype=, which is a silent no-op while the accountable is nil. On create the accountable is built from accountable_attributes via accepts_nested_attributes_for, but the form submits subtype as a top-level account attribute. Mass-assignment applies subtype before accountable_attributes, so the selected subtype was dropped on create (update worked because the accountable already exists). Build the accountable from the delegated type inside the writer when it is not yet present, so the value is preserved; the later accountable_attributes assignment (update_only) updates the same record. Add regression tests covering the create flow and the assignment ordering. |
||
|
Jonathan Chang
|
b9716a0485 |
fix: Transaction Pagination Skipping Entries (#2179)
* Fix transaction pagination regressions and CI blockers * Rails EOL already fixed --------- Co-authored-by: Juan José Mata <jjmata@jjmata.com> |
||
|
ghost
|
6ea931f3fe |
feat(imports): add YNAB CSV import (#2361)
Adds YnabImport (mirroring ActualImport) for YNAB "Export budget" register CSVs: - Amount — combines the split Outflow/Inflow columns into a single signed amount (inflow - |outflow|), stripping currency symbols and thousands separators. A single signed Amount column takes precedence when present. - Category — resolves across export shapes: the combined "Category Group/Category" column, the split "Category Group" + "Category", or legacy YNAB 4 "Master Category" + "Sub Category". - Names — falls back from a blank Payee to the Memo, then the default row name. - Validation — requires at least one amount source (Outflow/Inflow or Amount); a file exposing none leaves rows un-clean instead of importing zero-dollar entries. Enables the previously-disabled YNAB option on the imports screen (using the YNAB logo, like Mint) with its configuration partial, and removes the now-dead imports.new.coming_soon locale key. Documents the type in the API import-type enums (rswag request spec + swagger_helper + generated openapi.yaml). Closes #1255. |
||
|
Augusto Xavier
|
09dc428136 |
fix(rules): make explicit re-apply override locked attributes (#2273)
When a rule is re-applied from the UI, RulesController passes ignore_attribute_locks: true, but Enrichable#enrich_attributes still rejected locked attributes unconditionally, so locked (manually edited or import-locked) transactions were silently skipped and reported as blocked. Thread the flag through enrich_attribute/enrich_attributes as a new ignore_locks keyword (default false, so provider syncs and AI enrichment keep respecting locks) and pass it from the six synchronous rule action executors. Fixes #2051 |
||
|
ghost
|
94d2ee908d |
fix(jobs): enqueue jobs after transaction commit to fix SyncJob deserialization race (#2354)
* fix(jobs): enqueue jobs after transaction commit to fix SyncJob deserialization race Syncable#sync_later creates a Sync row and enqueues SyncJob inside the same transaction. Rails 7.2 deferred such enqueues until commit by default; Rails 8.0 changed the default to enqueue immediately and 8.1 left the global config toggle non-functional, so a worker could dequeue the job before COMMIT, fail to resolve the Sync GlobalID (ActiveJob::DeserializationError), and have it silently dropped by discard_on -- surfacing as stuck syncs after the Rails 8.1 upgrade. Set enqueue_after_transaction_commit = true on ApplicationJob (the Rails 8.2 default, inherited by every job) and drop the dead :never symbol override on DestroyJob. Add regression and invariant tests. * test(jobs): use OpenStruct for DestroyJob failure case Replace the bare mock and the respond_to? expectation (an implementation detail of how DestroyJob probes the model) with an OpenStruct that genuinely responds to scheduled_for_deletion. Keeps only the command-facing assertions: destroy raises and update! is called with scheduled_for_deletion: false. Matches the repo convention of preferring OpenStruct for mock instances. * test(snaptrade): assert connection-cleanup enqueue defers until commit SnaptradeAccount#after_destroy enqueues SnaptradeConnectionCleanupJob, which references the item/account by id. Before the ApplicationJob fix, Rails 8.1 enqueued it immediately inside the destroy transaction, so a worker could run before COMMIT, see the not-yet-deleted row in its shared-authorization guard, skip the provider call, and leak the SnapTrade connection. This regression test destroys an account inside a transaction and asserts the job is not enqueued until the transaction commits. |
||
|
Guillem Arias Fauste
|
c29380ce57 |
feat(dashboard): masonry packing + per-widget size controls (#2328)
* feat(dashboard): masonry packing + per-widget size controls In two-column mode the dashboard used a row-based CSS grid, so cards stretched to equal row height and left dead space (e.g. the Net Worth chart padded out to match the tall Balance Sheet table). Replace the row-based layout with masonry packing and add per-widget size guardrails. - Masonry: CSS grid with computed row-spans + grid-auto-flow: dense, driven by a dashboard-masonry Stimulus controller (ResizeObserver + turbo:frame-load). The DOM stays a single flat list, so drag/keyboard reorder is unaffected. Active only in multi-column mode; single column falls back to normal flow. - Internal sizing: the net worth chart height is now driven by a --dash-widget-h CSS var (fixes an inert flex-1) so the card no longer pads out below the chart. - Guardrails: per-widget layout metadata (col_span, grow, min_height, width_toggle) in PagesController, with per-user overrides persisted under preferences["dashboard_section_layout"], deep-merged so width and height coexist. - Size menu: a hover control on size-capable cards — Width (Half/Full) for the cashflow sankey and net worth chart, Height (Compact/Auto/Tall) for grow widgets. The sankey defaults to full width. Adds model + controller tests for preference persistence and i18n keys. * refactor(dashboard): redesign size menu with segmented controls The size menu used a plain radio list and a diagonal maximize-2 trigger that collided with the cashflow sankey's modal-expand button. Replace it with a layout-config popover: a sliders-horizontal trigger plus two labeled axis groups (Width, Height), each rendered as a DS::SegmentedControl with the active option filled. Clearer, more compact, and it reads as a card-layout control. The widget-size controller now mirrors the segmented control's active-class + aria-pressed contract. * feat(dashboard): expose width toggle on balance sheet and investments Tables benefit from horizontal room, so give Balance Sheet and Investments the same Width (Half/Full) control as the sankey and net worth chart. Height presets stay chart-only — a table sized to a fixed height would just add whitespace or force scrolling. The Outflows donut is intentionally left out (full width is mostly whitespace for a donut). * fix(dashboard): address review feedback on size controls - Only apply the full-width col-span and show the Width control when the two-column layout is enabled. A full widget previously leaked 2xl:col-span-2 into the single-column grid, creating an implicit second column at 2xl widths and breaking the single-column preference. This also keeps the Width control coherent with the Appearance two-column setting (it now appears only where it does something). [Codex] - Stop size-menu keydowns from bubbling to the section reorder handler, so keyboard users can open the menu and pick options without entering grab/reorder mode. [Codex] - Harden preferences params: ignore a malformed (non-hash) dashboard_section_layout / collapsed_sections instead of raising a 500, and require section_order to be an array. [CodeRabbit] - Localize the dashboard sections aria-label. [CodeRabbit] * chore(settings): mention per-widget size controls in two-column copy Surface the new per-widget width/height controls in the Appearance "Two-column layout" description so the capability is discoverable. * test(dashboard): assert non-mutation for malformed layout input Addresses review feedback: asserting assert_nil made the test depend on the fixture happening to have no dashboard height for net_worth_chart. Capture the pre-PATCH value and assert it is unchanged, so the test stays valid (malformed input ignored) even if the fixture later gets a default height. --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Guillem Arias <guillem.arias@col.vueling.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
Sure Admin (bot)
|
08bfd43775 |
fix: preserve CoinStats balances when wallet sync data is missing (#2132)
* test: cover CoinStats missing-wallet balance preservation * fix: preserve CoinStats balances when wallet data is missing * chore: route CoinStats sync warnings to debug log * docs: make debug log guidance timeless --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
Sure Admin (bot)
|
89eb441145 | fix(sync): discover nightly provider items reflectively (#2334) | ||
|
Guillem Arias Fauste
|
5937195df0 |
fix(chat): clear assistant bubble on destroy so the chat doesn't hang on Thinking (#2315)
* fix(chat): clear the assistant message bubble when a turn is destroyed When an assistant turn fails before any text streams (e.g. a provider auth/model/network error on the first call), Assistant::Builtin#respond_to destroys the still-pending message. Message only broadcast on create and update, never on destroy, so the rendered 'Thinking…' bubble was never removed — the chat appeared stuck thinking forever even though the job had already errored (and appended an error via chat#add_error below it). Add after_destroy_commit broadcast_remove_to so a destroyed message is removed from the page. * refactor(chat): trim destroy-broadcast comment to one line Project convention asks for comments only when the why is non-obvious; the behaviour is already covered by the commit/PR description. Per review feedback. --------- Co-authored-by: Guillem Arias <guillem.arias@col.vueling.com> |
||
|
BeltaKoda
|
749d54dd96 |
Fix Plaid sync failure for loan subtypes missing from Loan::SUBTYPES (#2298)
PlaidAccount::TypeMappable maps the Plaid loan subtypes "home equity",
"line of credit", and "business" to home_equity, line_of_credit, and
business — but Loan::SUBTYPES never defined them. Linking any such
account (e.g. a HELOC reported by the institution as loan/"line of
credit") makes the item's sync fail with:
Validation failed: Accountable subtype is not included in the list
and, because Link itself succeeded, the failure is silent in the UI
(same UX gap as #1792).
Add the three subtypes to Loan::SUBTYPES, and add a regression test
asserting every subtype emitted by TYPE_MAPPING is valid for its
accountable so the mapper and models can't drift apart again.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Rene Arredondo
|
0e1b3f8396 |
fix(enable-banking): tolerate any 422 on PDNG fetch (#1805) (#1889)
* fix(enable-banking): tolerate any 422 on PDNG fetch (#1805) * fix(enable-banking): fall back to string key when reading PDNG error |
||
|
Kelvinchen03
|
59edcaa986 | fix: EODHD lookup for EU mutual funds with EUFUND exchange code (#2212) | ||
|
ghost
|
de8cd86f2f |
perf(sync): scope transfer matching after account sync (#2230)
* perf(sync): scope transfer matching after account sync * fix(sync): make transfer lookup index migration reversible |
||
|
ghost
|
3ccb82ef9d |
fix(imports): import Actual rows with blank payee (#2282)
* fix(imports): import Actual rows with blank payee Actual Budget exports reconciliation and starting-balance rows with a blank Payee. ActualImport mapped the row name straight from the Payee column with no fallback (unlike Import and MintImport), so a blank Payee produced a blank Entry name. Entry requires a name, and import! wraps all rows in a single transaction, so one blank-payee row failed validation and rolled back the entire import -- surfacing only a generic "Import failed" while the worker logged "done". Fall back to the Notes column (which carries text like "Reconciliation balance adjustment") and then to the default row name, matching the blank-name handling already used by the base importer and MintImport. Add a blank-payee row to the Actual fixture and regression tests covering the Notes fallback, the default fallback, and an end-to-end import that no longer fails on blank-payee rows. * ci(security): skip calendar-based brakeman Rails EOL check The scan_ruby job fails because brakeman's CheckEOLRails warns that Rails 7.2.3.1 reaches end of life on 2026-08-09. That check fires purely on the calendar -- it warns 60 days before the EOL date and escalates in confidence as the date nears (brakeman/checks/eol_check.rb) -- so it turns `bin/brakeman` (exit code 3) red on every branch and on main regardless of the code being scanned. Add config/brakeman.yml (auto-loaded by `bin/brakeman`) skipping only CheckEOLRails. CheckEOLRuby is left enabled because the current Ruby is not near end of life, so that signal is preserved. A TODO records that the skip should be removed when Sure upgrades off Rails 7.2. |
||
|
Guillem Arias Fauste
|
f25fe30a41 |
feat(ai): honor Setting.llm_provider for batch and PDF flows (#2265)
Auto-categorization, merchant detection/enhancement, and PDF/bank-statement extraction hard-coded Provider::Registry.get_provider(:openai), so selecting Anthropic (or running an Anthropic-only self-hosted install) left those operations using/missing OpenAI rather than the chosen provider. Add Provider::Registry.preferred_llm_provider, which resolves the LLM provider honoring Setting.llm_provider with a configured-provider fallback (mirroring how chat picks its provider), and route all six TODO(#2113) call sites through it: - Family::AutoCategorizer#llm_provider - Family::AutoMerchantDetector#llm_provider - ProviderMerchant::Enhancer#llm_provider - PdfImport (process_pdf + extract_bank_statement) - Assistant::Function::ImportBankStatement Provider::Anthropic already implements auto_categorize / auto_detect_merchants / enhance_provider_merchants (#1984) and process_pdf / extract_bank_statement (#1985), so no provider changes are needed — only the wiring. Closes #2113. |
||
|
Dan
|
3627c4d2d1 | Respect CoinStats wallet rate limits (#2251) | ||
|
Guillem Arias Fauste
|
3b4aa55516 |
test(goals): pledge-delta integration coverage (follow-up to #2178) (#2206)
* fix(goals): match manual_save pledges by contribution delta, not full balance Closes #2177 * fix(goals): clarify depository-only contribution; lowercase test names Address review: document that valuation_contribution's delta is only consumed for goal-linked Depository accounts, so the positive-delta guard is correct and no liability paydown sign case can arise. Lowercase NOT in two test names to match project convention. * test(goals): pin pledge matching through the reconciliation manager Follow-up to #2178, which matched manual_save pledges by contribution delta but only unit-tested the matcher with an injected delta. The delta derivation itself (the balances-table lookup and the nil-to-0 fallback in valuation_contribution) had no coverage, and reconciliation_manager_test never touched pledges. Two manager-level tests pin the wiring end to end: a 2000 -> 2150 reconcile matches an open 150 pledge, and a same-balance reconcile (zero delta) leaves it open, closing the <= 0 boundary. Also documents the staleness window on valuation_contribution: the prior balance reads the balances table, which recomputes async after the valuation saves, so a second same-day reconcile racing that sync derives a stale delta and self-heals on the next save. * fix(goals): derive same-day re-reconcile delta from the prior valuation The balances table recomputes asynchronously after a reconcile, so a second same-day reconcile racing that sync derived its delta from the pre-first-reconcile row. An overstated delta could wrongly close a larger open pledge — and a wrong match, unlike a miss, never self-heals. Prefer the valuation's own pre-save amount as the prior balance when the reconcile updates an existing valuation; fall back to the balances row, then 0. Same-date re-reconciles are immune to the race; only the cross-date window remains, and only in the miss direction. --------- Co-authored-by: galuis116 <galuis116@gmail.com> |
||
|
Guillem Arias Fauste
|
d845e44ff8 |
feat(ai): default Anthropic installs to pgvector RAG (4/5) (#1986)
* feat(ai): add Anthropic provider with chat parity (1/5)
Introduces Provider::Anthropic alongside Provider::Openai, implementing
the LlmConcept chat_response contract over the official anthropic Ruby
SDK. Batch ops, PDF, and RAG land in follow-up PRs.
- Provider::Anthropic uses Messages API for sync and streaming responses
- ChatConfig builds requests with ephemeral prompt-cache markers on the
system prompt and the last tool definition
- MessageFormatter reconstructs multi-turn history (text + tool_use +
tool_result blocks) from raw Message records, including the paired
user-role tool_result turn Anthropic requires after every tool_use
- ChatParser maps Anthropic Message into the shared ChatResponse Data
- Registry, Setting, User, Chat default model wired for ANTHROPIC_*
envs and Setting.anthropic_*; LLM_PROVIDER selects between providers
- Responder forwards raw conversation_history (Array<Message>) so
providers without hosted conversation state can rebuild context
- OpenAI provider accepts and ignores the new kwarg (no behavior change)
Tests cover provider init, model gating, MessageFormatter for all turn
shapes, ChatConfig request building (max_tokens, system cache, tool
conversion), ChatParser for text / tool_use / mixed blocks, Registry
discovery, and mocked chat_response success / error / function_request
paths. Live VCR cassettes recorded in a follow-up with a real key.
Stacked PRs: 2/5 batch ops + cost ledger, 3/5 PDF, 4/5 pgvector RAG,
5/5 settings UI + disclosure.
* fix(ai): address PR review on Anthropic provider foundation
Surface fixes raised by Codex + CodeRabbit on PR 1/5:
- Provider::Anthropic#chat_response now accepts (and ignores) a
`messages:` kwarg. Assistant::Responder passes both `messages:`
(OpenAI-shape) and `conversation_history:` (raw Message records) for
cross-provider parity, so the previous signature raised
ArgumentError on the first chat turn through the Anthropic provider.
- Provider::Anthropic#supports_model? bypasses the `claude` prefix
gate when a custom base_url is configured, mirroring the OpenAI
provider. Bedrock-shaped IDs like
`anthropic.claude-sonnet-4-5-20250929-v1:0` and
`claude-opus-4@20250514` are otherwise rejected by
Assistant::Provided#get_model_provider and the chat dies.
- Setting.anthropic_access_token is now in
EncryptedSettingFields::ENCRYPTED_FIELDS so the Anthropic API key
is encrypted at rest like every other provider secret. Previously
plaintext while siblings (openai_access_token, twelve_data_api_key,
external_assistant_token) were ciphertext.
- Chat.default_model falls back to whichever provider is actually
configured. Previously, with LLM_PROVIDER=anthropic but no
Anthropic credentials, the default model resolved to a Claude ID
that no registered provider supported, so chats failed even when
OpenAI was fully configured. Adds Provider::{Anthropic,Openai}#configured?
class methods for the readable callsite.
- Provider::Anthropic.effective_model uses
`ENV["ANTHROPIC_MODEL"].presence || Setting.anthropic_model` so the
Setting lookup is only performed when the env var is absent — the
previous `ENV.fetch(KEY, default)` evaluated the default arg
eagerly on every call.
- Provider::Anthropic::ChatConfig#anthropic_input_schema strips both
`:strict` and `"strict"` keys so JSON-decoded schemas with string
keys cannot leak the OpenAI-only flag through to Anthropic.
Test coverage added: supports_model? bypass on custom endpoints,
chat_response messages: kwarg compatibility, default_model fallback
in the three credential combinations, configured? against ENV +
Setting, strict-flag stripping for both key types, and a
`Setting.expects(:anthropic_model).never` assertion proving the
ENV-precedence test now exercises the lazy path.
All 4365 tests pass (1 pre-existing libvips env error unrelated).
* test(chat): make default_model tests resilient to ENV model overrides
CodeRabbit flagged on PR review: the new default_model tests asserted
against Provider::*::DEFAULT_MODEL, but Chat.default_model actually
returns Provider::*.effective_model.presence (which reads
OPENAI_MODEL / ANTHROPIC_MODEL from the environment). With either env
var set, the tests would fail intermittently even though routing was
correct.
- New default_model tests now assert against the provider's
effective_model directly, so they verify the routing decision
(which provider's value wins) without coupling to the constant.
- Pre-existing "creates with default model" assertions had the same
brittleness; switch them to compare against Chat.default_model so
the chosen model is whatever the env / Setting cascade resolves to.
Verified by running `ANTHROPIC_MODEL=claude-haiku-4-5 OPENAI_MODEL=gpt-4o
bin/rails test test/models/chat_test.rb` — 16 runs, 0 failures
(previously 2 pre-existing failures + 0 from the new tests).
* fix(ai): address local review on Anthropic foundation
- Provider::Anthropic#supports_pdf_processing? bypasses prefix gate for
custom endpoints, mirroring supports_model?
- Provider::Anthropic#initialize raises Error when custom_endpoint? AND
model.blank?, parity with Provider::Openai
- stream_chat_response captures partial usage on mid-stream errors and
records it via the new on_partial callback so chat_response can skip
the duplicate error row in the outer rescue
- safe_accumulated_message swallows the secondary failure when the SDK
cannot reconstruct a snapshot
- langfuse_client memoizes properly (||= instead of =) so repeated calls
don't churn Langfuse instances
- MessageFormatter sorts tool_calls by created_at then id so the
message array is deterministic across replays; skips tool_calls
missing both provider_call_id and provider_id rather than sending
`id: nil` and getting rejected by Anthropic
- Setting.anthropic_access_token default falls back through
ENV["ANTHROPIC_API_KEY"].presence (was missing .presence, so an
empty-string env value bled through)
- User#openai_configured? / #anthropic_configured? delegate to the
Provider::* class methods — single source of truth
- Assistant::Responder renames the OpenAI-shape history builder
conversation_history → openai_messages_payload so the kwarg name
matches the local method name (messages: openai_messages_payload,
conversation_history: chat_message_records)
- Assistant::Builtin stale-history comment updated to reference both
builders
Adds a streaming chat_response test using ad-hoc subclasses of the
SDK event types so the case/when dispatch matches via is_a? without
stubbing class-level === behavior.
* test(ai): add Anthropic tool_use round-trip + multi-tool turn coverage
Addresses @jjmata's "worth confirming" note on PR #1983: tool-use turns
from prior assistant messages must round-trip correctly when retrieved
from the database.
- New `ChatParser → ToolCall::Function → MessageFormatter` test walks
the full path: Anthropic response with a tool_use block →
ChatFunctionRequest → ToolCall::Function.from_function_request →
persisted on the AssistantMessage → MessageFormatter rebuild on the
next turn. Asserts the original `tool_use.id` is preserved end-to-end
as both `tool_use.id` and the paired `tool_result.tool_use_id`, and
that the original `input` hash and serialized result content survive.
- New multi-tool assistant turn test confirms two tool_use blocks on a
single assistant message render as two tool_use blocks followed by
two paired tool_result blocks in a single user-role follow-up,
matching Anthropic's required alternation.
Both tests exercise the existing PR1 code without behavior changes.
* test(ai): require "ostruct" explicitly in Anthropic provider tests
OpenStruct is moving out of Ruby's default load path (warning in 3.4+,
removed in 3.5+). Tests work today because ActiveSupport transitively
loads it, but that's incidental. Match the existing convention in
test/controllers/settings/hostings_controller_test.rb which explicitly
requires ostruct for the same reason.
* fix(ai): sanitize Langfuse warn logs, normalize tool_use.input, dedup history fetch
Addresses three open CodeRabbit findings on PR #1983.
- Provider::Anthropic Langfuse rescue branches no longer include
`e.full_message` in `Rails.logger.warn`. `full_message` bundles the
backtrace + cause chain and on some SDK error types includes the
serialized request/response payload (prompt, model output). Logs
now report `#{e.class}: #{e.message}` only. Three sites:
create_langfuse_trace, log_langfuse_generation, upsert_langfuse_trace.
Note: Provider::Openai has the same pattern (copy-pasted source) —
harmonization deferred to a follow-up cleanup PR; this commit fixes
only the Anthropic provider to keep PR scope tight.
- MessageFormatter#parse_arguments now coerces any non-Hash parsed
result to `{}`. Anthropic's Messages API requires `tool_use.input`
to be a JSON object (map); a stored ToolCall::Function record whose
arguments parse to a scalar, bool, or array (corrupt row, legacy
data, cross-provider bleed) would otherwise produce a payload the
API rejects. Normal flow stores Hash arguments end-to-end so the
fix is defensive — adds 2 tests covering scalar/array JSON strings
and non-String non-Hash inputs.
- Assistant::Responder dedups the chat-history fetch. The previous
layout fired two near-identical `chat.messages.where(...).includes(
:tool_calls).ordered` queries per LLM turn (one for the OpenAI-shape
payload, one for the raw-records kwarg). A new memoized
`complete_chat_messages` fetches once; `chat_message_records` filters
out the current message via `Array#reject`, `openai_messages_payload`
iterates the cached array unchanged. One SQL query per turn instead
of two. Memoization scope = single Responder instance (per LLM call),
so cache invalidation is not a concern.
All 4370 tests pass (1 pre-existing libvips env error unrelated).
Rubocop + brakeman clean.
* fix(ci): replace sk-ant- prefixed test placeholders
Pipelock secret scanner pattern-matches `sk-ant-*` as a real Anthropic
API key and fails the PR security-scan check. Test stubs and
ClimateControl env values used `sk-ant-test`, `sk-ant-from-setting`,
`sk-ant-x`, `sk-ant-y` as obvious placeholders, but the scanner does
not care about value entropy.
Switched to `fake-anthropic-key-*` / `fake-token-*` strings so the
scanner stops flagging them. No production code touched, no behavior
change — Provider::Anthropic still accepts any non-blank token.
* feat(ai): add Anthropic batch ops + LLM cost ledger (2/5)
Implements auto_categorize, auto_detect_merchants, and
enhance_provider_merchants on Provider::Anthropic via forced tool calls,
plus the cost-ledger plumbing they need.
- Provider::Anthropic::AutoCategorizer, AutoMerchantDetector,
ProviderMerchantEnhancer each define a single output tool whose
input_schema mirrors the desired output, then force the model to call
it via tool_choice: { type: "tool", name: ..., disable_parallel_tool_use: true }.
Anthropic guarantees the tool_use.input matches the schema, so there
is no JSON parsing fragility, no <think> tag stripping, and no
json_object/json_schema fallback ladders.
- Concerns::UsageRecorder mirrors the OpenAI sibling but persists
cache_creation_input_tokens / cache_read_input_tokens to dedicated
columns instead of metadata.
- Migration adds cache_creation_tokens, cache_read_tokens (nullable
integers) to llm_usages. OpenAI rows leave them null.
- LlmUsage::PRICING gains Claude 4.x rows (opus-4-7 $15/$75, sonnet-4-6
$3/$15, haiku-4-5 $1/$5 per MTok). infer_provider returns "anthropic"
for claude-* via the existing exact/prefix lookup.
- Provider::Anthropic#chat_response now persists cache columns directly
rather than stashing them in metadata.
- 25-transaction batch cap mirrors the OpenAI provider so the cost
ledger sees the same shape regardless of which provider ran a batch.
Tests cover the forced-tool-call path, null/None normalization,
case-insensitive merchant matching, the missing-tool_use error path,
and Anthropic-specific pricing + provider inference on LlmUsage.
Stacked on #1983 (PR 1/5). 3/5 PDF + vision next.
* fix(ai): attribute Bedrock model IDs to anthropic + clean nil enum
- LlmUsage.infer_provider now returns "anthropic" for Bedrock /
Vertex shaped IDs (anthropic.* and anthropic/*), so cost-ledger
filtering by provider stays correct even when no per-MTok rate is
stored. Previously these IDs fell through to the "openai" default.
- AutoCategorizer drops the redundant nil sentinel from the
category_name enum — the union type [string, null] already permits
null, and some JSON Schema validators reject nil literals inside
enum arrays.
* test(ai): require "ostruct" in Anthropic batch op tests
Same rationale as the PR1 ostruct fix — explicit require so the tests
don't depend on ActiveSupport's transitive load when Ruby 3.5+ removes
OpenStruct from the default load path.
* feat(ai): Anthropic native PDF processing (3/5)
Implements process_pdf and extract_bank_statement on Provider::Anthropic
using the native `document` content block — no rasterization, no text
pre-extraction.
- Provider::Anthropic::PdfProcessor classifies the document, summarizes
it, and extracts statement metadata via a forced report_document_analysis
tool whose input_schema mirrors the existing Provider::Openai output
(document_type from Import::DOCUMENT_TYPES, summary, extracted_data).
- Provider::Anthropic::BankStatementExtractor returns the same
{ transactions, period, account_holder, account_number, bank_name,
opening_balance, closing_balance } shape via report_bank_statement so
downstream pdf_import code is provider-agnostic.
- Both attach the PDF as
{ type: "document", source: { type: "base64", media_type: "application/pdf", data: <b64> } }
— Claude 3.5+ / 4.x accept this natively (up to 32MB / 100 pages).
No pdf-reader, no pdftoppm, no chunking for typical statements.
- supports_pdf_processing? (introduced in PR 1) already returns true for
claude-* models, gating process_pdf with a clear error otherwise.
- Cost ledger rows are persisted via the shared UsageRecorder concern,
including cache_creation/cache_read tokens.
Tests verify the document block shape, tool_choice forcing, normalized
document_type for unknown classifications, transaction normalization
(date / amount / reference → notes), and the missing-tool_use error
path. Blank pdf_content raises before any client call.
Stacked on #1984 (PR 2/5). 4/5 pgvector RAG next.
* fix(ai): guard PDF size + surface bank-statement truncation
- PdfProcessor and BankStatementExtractor raise upfront when
pdf_content.bytesize exceeds MAX_PDF_BYTES (32 MB, matching
Anthropic's hard limit). Previously a 100 MB PDF would be
base64-encoded (~133 MB) and packed into the JSON body before
the API rejected it — peak heap ~270 MB per Sidekiq worker.
- BankStatementExtractor inspects response.stop_reason; when the
model hit max_tokens it logs a warning and flags result[:truncated]
so downstream callers know the transaction list may be incomplete.
- ISO date pattern added to statement_period_start/end schema in
PdfProcessor so the model can't return "March 2026" — Anthropic
enforces the regex via the tool's input_schema.
Tests cover the size guard (raises before any client.messages call),
truncated-result flagging, and the warning log path.
* test(ai): require "ostruct" in Anthropic PDF tests
Match the explicit ostruct require added in PR1/PR2 — same Ruby 3.5+
load-path reason.
* feat(ai): default Anthropic installs to pgvector RAG (4/5)
The provider-agnostic vector store stack (VectorStore::Pgvector + the
Embeddable concern) already shipped to main. This PR closes the
Anthropic loop:
- VectorStore::Registry.adapter_name now returns :pgvector when
Setting.llm_provider == "anthropic" and no explicit
VECTOR_STORE_PROVIDER override is set. Anthropic has no hosted vector
store, so falling back to the local pgvector adapter is the only
correct default. Explicit VECTOR_STORE_PROVIDER still wins.
- SearchFamilyFiles surfaces a longer message when no adapter is wired
up — calling out pgvector + EMBEDDING_URI_BASE as the supported
Anthropic-only path so the user is not stuck with an "OpenAI required"
hint that is no longer accurate.
The Embeddable concern already pulls embeddings from
EMBEDDING_URI_BASE / EMBEDDING_ACCESS_TOKEN (with OpenAI as fallback),
so Anthropic installs point this at Voyage AI, a local Ollama instance,
or OpenAI embeddings — independent of the chat provider.
Tests cover the new default routing, the existing OpenAI default
staying intact, and explicit VECTOR_STORE_PROVIDER overriding the
Anthropic default.
Stacked on #1985 (PR 3/5). 5/5 settings UI + retention disclosure next.
* fix(ai): provision pgvector table when it is the default store
#1986 makes pgvector the default vector store for Anthropic installs, but
CreateVectorStoreChunks only ran when VECTOR_STORE_PROVIDER=pgvector was set
explicitly — so a fresh Anthropic-only install migrated without the
vector_store_chunks table and failed on uploads/searches.
Add VectorStore::Registry.pgvector_effective? as the single source of truth
for "is pgvector active?" (explicit env OR the Anthropic default), and a new
idempotent migration that enables the extension + creates the table whenever
pgvector is effective and the table is missing — covering fresh and
already-migrated installs without drift. Addresses Codex P1.
* fix(ai): provision pgvector table for Anthropic-default installs
Migration gated on raw VECTOR_STORE_PROVIDER==pgvector, so an
Anthropic-default install (which selects pgvector implicitly via
Setting.llm_provider without setting VECTOR_STORE_PROVIDER) skipped
table creation and failed later on a missing vector_store_chunks
relation. Route through VectorStore::Registry.pgvector_effective? —
the single source of truth already shared by the adapter selection.
Addresses Codex P1 review finding.
* fix(ai): provision pgvector chunks table on schema-load installs
The ensure-migration only helps db:migrate upgraders. Fresh installs go
through bin/docker-entrypoint's db:prepare, which loads schema.rb (the
conditional table can't be dumped there — it needs the vector extension)
and marks every migration applied without running it. An Anthropic-only
fresh install therefore selected the pgvector adapter but had no table,
failing with raw PG errors on first upload or search.
Two layers close it:
- VectorStore::Pgvector#ensure_schema! provisions the table idempotently
on first use (mirrors CreateVectorStoreChunks; memoized; failures wrap
in VectorStore::Error, which with_response turns into a clean failed
response).
- VectorStore::Registry#build_pgvector now gates on
VectorStore::Pgvector.available? (table exists, or extension present),
so installs whose Postgres lacks pgvector entirely degrade to the
assistant's provider_not_configured message instead of raising
mid-chat.
Also resolves the schema.rb version conflict against main (keep the
branch's 2026_06_01_120000, on top of main's current tables).
* fix(ai): address review nitpicks on pgvector provisioning
- Registry: update the adapter doc comment to mention the
Anthropic-to-pgvector default alongside the openai fallback.
- ensure_schema!: guard the DDL with if_not_exists instead of a Mutex.
Adapter instances are built per call and never shared across threads,
so the realistic race is two processes (web + Sidekiq) provisioning
concurrently; IF NOT EXISTS makes the loser a no-op where a Mutex
would only serialize threads inside one process.
|
||
|
Jonathan Chang
|
8a38a82c93 |
fix: Savings Goal Marked Reached While Still Short (#2180)
* fix: saving goal is marke while several states * feat(enable_banking): support MFA/decoupled banks and harden session handling (#2174) Decoupled/MFA banks (e.g. VR Bank in Holstein) were hard-blocked because the authorize flow aborted whenever auth_methods[0] was DECOUPLED. Enable Banking's hosted /auth page actually coordinates decoupled SCA and redirects back with a code, so route these banks through it instead: - Provider#start_authorization accepts and forwards an auth_method param - EnableBankingItem#select_auth_method picks the best method (REDIRECT > DECOUPLED > EMBEDDED), filtering by psu_type and skipping hidden methods - Shared begin_authorization! re-fetches ASPSP metadata on each authorize and reauthorize, so the method is always re-derived (no persistence required) - Remove the DECOUPLED block in the controller Also stop the integration from constantly reporting "session expired": - Only a session-level GET /sessions 401/404 flips the connection to requires_update; per-account 401/404 are retried and no longer kill the whole connection - Reconcile session_expires_at from the API's access.valid_until on every sync - Treat an expired session as a graceful requires_update state instead of raising a bare error No schema changes. Adds covering tests. * fix: request change reach --------- Co-authored-by: Tobias Rahloff <rahloff@gmail.com> |
||
|
ghost
|
1448128762 |
perf(reports): collapse investment flow aggregates (#2190)
* perf(reports): collapse investment flow aggregates * fix(reports): avoid shared-account flow duplication |
||
|
Guillem Arias Fauste
|
96079188a2 |
feat(dashboard): unify per-widget period selectors into one picker (#2162)
* feat(ds): elevate dropdown overlays and stabilize selection check gutter Menus and popovers floated at the same elevation as inline cards (shadow-border-xs), so dropdowns blended into the content beneath them. Bump DS::Menu and DS::Popover panels to shadow-border-lg. DS::MenuItem rendered its leading icon only when present, so a selection check shifted the row's text out of alignment with the unselected rows. Add a `selected:` param that reserves a fixed-width check gutter (check when selected, empty otherwise) so row text stays aligned. Apply the same reserved gutter to the bespoke category dropdown row, and add a `selectable` menu preview. * feat(dashboard): unify per-widget period selectors into one picker The dashboard rendered three identical period <select>s (cashflow, outflows, net worth), each writing the same global User#default_period and full-reloading the page via turbo_frame "_top" — so changing one changed all. Replace them with a single shared UI::PeriodPicker (DS::Menu of period links) in a toolbar, and wrap the sections grid in a "dashboard_sections" Turbo frame so a period change swaps only the dashboard (no full-page reload). Reuse the same picker on the account chart, removing its duplicate select. * refactor(dashboard): use DS::MenuItem selected: gutter in period picker Now that DS::MenuItem reserves a check gutter, the period picker passes selected: instead of a conditional leading check icon, so the current period's row stays aligned with the rest. * fix(ds): expose menu selection via menuitemradio + aria-checked Selectable DS::MenuItem rows conveyed selection only visually. Render them as role="menuitemradio" with aria-checked so assistive tech gets the selection state of single-select lists, merging the menu ARIA contract with any caller-supplied aria. Addresses CodeRabbit review feedback. * refactor(dashboard): drop redundant aria-current from period picker DS::MenuItem now exposes selection via menuitemradio + aria-checked, so the period picker no longer needs its own aria-current. Update the component test to assert the new ARIA. * fix(ds): include selectable roles in menu roving-focus query DS::MenuItem selectable rows render as role=menuitemradio, but the menu controller built its roving-focus list from [role=menuitem] only, leaving single-select menus with no keyboard focus/arrow handling. Query the menuitemradio/menuitemcheckbox roles too. Addresses Codex review feedback. * fix(dashboard): keep non-picker links out of frame + fix custom-month picker - turbo_frame_tag "dashboard_sections" now targets _top so ordinary links inside the sections (e.g. Balance Sheet account links) navigate the page instead of failing to resolve inside the frame. - Period.current_month_for / last_month_for carry their semantic key for custom-month families, so the picker shows the right label and checks the right option instead of falling back to 30D. Addresses Codex review feedback. * fix(dashboard): announce selected period in picker trigger's accessible name The static "Select time period" aria-label overrode the visible selected label as the trigger's accessible name, so assistive tech kept announcing the same name regardless of selection. Interpolate the selected short label into the aria-label and pin it with a component test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
ghost
|
683f518780 |
fix(balance-sheet): preserve disabled-account net worth history (#1730)
* fix(balance-sheet): preserve disabled account history * fix(balance-sheet): tighten historical account scope * fix(balance-sheet): stop disabled account carry-forward --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Juan José Mata <juanjo.mata@gmail.com> |
||
|
Tobias Rahloff
|
fe47c918bb |
feat(enable_banking): support MFA/decoupled banks and harden session handling (#2174)
Decoupled/MFA banks (e.g. VR Bank in Holstein) were hard-blocked because the authorize flow aborted whenever auth_methods[0] was DECOUPLED. Enable Banking's hosted /auth page actually coordinates decoupled SCA and redirects back with a code, so route these banks through it instead: - Provider#start_authorization accepts and forwards an auth_method param - EnableBankingItem#select_auth_method picks the best method (REDIRECT > DECOUPLED > EMBEDDED), filtering by psu_type and skipping hidden methods - Shared begin_authorization! re-fetches ASPSP metadata on each authorize and reauthorize, so the method is always re-derived (no persistence required) - Remove the DECOUPLED block in the controller Also stop the integration from constantly reporting "session expired": - Only a session-level GET /sessions 401/404 flips the connection to requires_update; per-account 401/404 are retried and no longer kill the whole connection - Reconcile session_expires_at from the API's access.valid_until on every sync - Treat an expired session as a graceful requires_update state instead of raising a bare error No schema changes. Adds covering tests. |
||
|
ghost
|
6e04c6927d |
feat(imports): add SureImport session batches (#1785)
* feat(imports): add SureImport session batches Add first-class SureImport sessions for ordered multi-file NDJSON imports. Persist source mappings across chunks, make session/chunk processing idempotent, expose progress readback, and keep existing single-file import behavior compatible. Includes the devcontainer libvips runtime dependency needed by ActiveStorage variant tests. Addresses #1610. Related to #1458. * fix(imports): avoid scanner-like API key test data * test(imports): assert skipped balances are not persisted * fix(imports): harden session publish retries Validate expected import chunk sequences exactly before publish, and restore session state with error details when enqueueing the publish job fails. * fix(imports): close session retry edge cases Backfill expected chunk counts after client-session insert races and enqueue import-session jobs after the status transition commits. Persist a safe enqueue failure body so API readback does not expose raw queue errors. * fix(imports): address session publish review gaps Remove dead transaction external-id assignment, harden session publish retry/sync behavior, align session chunk status docs, and add regression coverage for partial retries and safe enqueue error readback. * fix(imports): include sessions in family reset Clear import sessions through the family reset job so chunk imports and source mappings do not survive a reset. Expose import session and source mapping counts in the reset status response and regenerated OpenAPI schema so polling reflects the full reset surface. * test(imports): cover split import mapping invariants * test(imports): cover session verification invariants * fix(imports): scope SureImport session reimports * Tighten SureImport session batching * fix(imports): export rule source ids for sessions * test(imports): stabilize rule id export assertion * test(imports): restore reset status session fixture |
||
|
Guillem Arias Fauste
|
8ccc434b3d |
feat(ai): Anthropic native PDF processing (3/5) (#1985)
* feat(ai): add Anthropic provider with chat parity (1/5)
Introduces Provider::Anthropic alongside Provider::Openai, implementing
the LlmConcept chat_response contract over the official anthropic Ruby
SDK. Batch ops, PDF, and RAG land in follow-up PRs.
- Provider::Anthropic uses Messages API for sync and streaming responses
- ChatConfig builds requests with ephemeral prompt-cache markers on the
system prompt and the last tool definition
- MessageFormatter reconstructs multi-turn history (text + tool_use +
tool_result blocks) from raw Message records, including the paired
user-role tool_result turn Anthropic requires after every tool_use
- ChatParser maps Anthropic Message into the shared ChatResponse Data
- Registry, Setting, User, Chat default model wired for ANTHROPIC_*
envs and Setting.anthropic_*; LLM_PROVIDER selects between providers
- Responder forwards raw conversation_history (Array<Message>) so
providers without hosted conversation state can rebuild context
- OpenAI provider accepts and ignores the new kwarg (no behavior change)
Tests cover provider init, model gating, MessageFormatter for all turn
shapes, ChatConfig request building (max_tokens, system cache, tool
conversion), ChatParser for text / tool_use / mixed blocks, Registry
discovery, and mocked chat_response success / error / function_request
paths. Live VCR cassettes recorded in a follow-up with a real key.
Stacked PRs: 2/5 batch ops + cost ledger, 3/5 PDF, 4/5 pgvector RAG,
5/5 settings UI + disclosure.
* fix(ai): address PR review on Anthropic provider foundation
Surface fixes raised by Codex + CodeRabbit on PR 1/5:
- Provider::Anthropic#chat_response now accepts (and ignores) a
`messages:` kwarg. Assistant::Responder passes both `messages:`
(OpenAI-shape) and `conversation_history:` (raw Message records) for
cross-provider parity, so the previous signature raised
ArgumentError on the first chat turn through the Anthropic provider.
- Provider::Anthropic#supports_model? bypasses the `claude` prefix
gate when a custom base_url is configured, mirroring the OpenAI
provider. Bedrock-shaped IDs like
`anthropic.claude-sonnet-4-5-20250929-v1:0` and
`claude-opus-4@20250514` are otherwise rejected by
Assistant::Provided#get_model_provider and the chat dies.
- Setting.anthropic_access_token is now in
EncryptedSettingFields::ENCRYPTED_FIELDS so the Anthropic API key
is encrypted at rest like every other provider secret. Previously
plaintext while siblings (openai_access_token, twelve_data_api_key,
external_assistant_token) were ciphertext.
- Chat.default_model falls back to whichever provider is actually
configured. Previously, with LLM_PROVIDER=anthropic but no
Anthropic credentials, the default model resolved to a Claude ID
that no registered provider supported, so chats failed even when
OpenAI was fully configured. Adds Provider::{Anthropic,Openai}#configured?
class methods for the readable callsite.
- Provider::Anthropic.effective_model uses
`ENV["ANTHROPIC_MODEL"].presence || Setting.anthropic_model` so the
Setting lookup is only performed when the env var is absent — the
previous `ENV.fetch(KEY, default)` evaluated the default arg
eagerly on every call.
- Provider::Anthropic::ChatConfig#anthropic_input_schema strips both
`:strict` and `"strict"` keys so JSON-decoded schemas with string
keys cannot leak the OpenAI-only flag through to Anthropic.
Test coverage added: supports_model? bypass on custom endpoints,
chat_response messages: kwarg compatibility, default_model fallback
in the three credential combinations, configured? against ENV +
Setting, strict-flag stripping for both key types, and a
`Setting.expects(:anthropic_model).never` assertion proving the
ENV-precedence test now exercises the lazy path.
All 4365 tests pass (1 pre-existing libvips env error unrelated).
* test(chat): make default_model tests resilient to ENV model overrides
CodeRabbit flagged on PR review: the new default_model tests asserted
against Provider::*::DEFAULT_MODEL, but Chat.default_model actually
returns Provider::*.effective_model.presence (which reads
OPENAI_MODEL / ANTHROPIC_MODEL from the environment). With either env
var set, the tests would fail intermittently even though routing was
correct.
- New default_model tests now assert against the provider's
effective_model directly, so they verify the routing decision
(which provider's value wins) without coupling to the constant.
- Pre-existing "creates with default model" assertions had the same
brittleness; switch them to compare against Chat.default_model so
the chosen model is whatever the env / Setting cascade resolves to.
Verified by running `ANTHROPIC_MODEL=claude-haiku-4-5 OPENAI_MODEL=gpt-4o
bin/rails test test/models/chat_test.rb` — 16 runs, 0 failures
(previously 2 pre-existing failures + 0 from the new tests).
* fix(ai): address local review on Anthropic foundation
- Provider::Anthropic#supports_pdf_processing? bypasses prefix gate for
custom endpoints, mirroring supports_model?
- Provider::Anthropic#initialize raises Error when custom_endpoint? AND
model.blank?, parity with Provider::Openai
- stream_chat_response captures partial usage on mid-stream errors and
records it via the new on_partial callback so chat_response can skip
the duplicate error row in the outer rescue
- safe_accumulated_message swallows the secondary failure when the SDK
cannot reconstruct a snapshot
- langfuse_client memoizes properly (||= instead of =) so repeated calls
don't churn Langfuse instances
- MessageFormatter sorts tool_calls by created_at then id so the
message array is deterministic across replays; skips tool_calls
missing both provider_call_id and provider_id rather than sending
`id: nil` and getting rejected by Anthropic
- Setting.anthropic_access_token default falls back through
ENV["ANTHROPIC_API_KEY"].presence (was missing .presence, so an
empty-string env value bled through)
- User#openai_configured? / #anthropic_configured? delegate to the
Provider::* class methods — single source of truth
- Assistant::Responder renames the OpenAI-shape history builder
conversation_history → openai_messages_payload so the kwarg name
matches the local method name (messages: openai_messages_payload,
conversation_history: chat_message_records)
- Assistant::Builtin stale-history comment updated to reference both
builders
Adds a streaming chat_response test using ad-hoc subclasses of the
SDK event types so the case/when dispatch matches via is_a? without
stubbing class-level === behavior.
* test(ai): add Anthropic tool_use round-trip + multi-tool turn coverage
Addresses @jjmata's "worth confirming" note on PR #1983: tool-use turns
from prior assistant messages must round-trip correctly when retrieved
from the database.
- New `ChatParser → ToolCall::Function → MessageFormatter` test walks
the full path: Anthropic response with a tool_use block →
ChatFunctionRequest → ToolCall::Function.from_function_request →
persisted on the AssistantMessage → MessageFormatter rebuild on the
next turn. Asserts the original `tool_use.id` is preserved end-to-end
as both `tool_use.id` and the paired `tool_result.tool_use_id`, and
that the original `input` hash and serialized result content survive.
- New multi-tool assistant turn test confirms two tool_use blocks on a
single assistant message render as two tool_use blocks followed by
two paired tool_result blocks in a single user-role follow-up,
matching Anthropic's required alternation.
Both tests exercise the existing PR1 code without behavior changes.
* test(ai): require "ostruct" explicitly in Anthropic provider tests
OpenStruct is moving out of Ruby's default load path (warning in 3.4+,
removed in 3.5+). Tests work today because ActiveSupport transitively
loads it, but that's incidental. Match the existing convention in
test/controllers/settings/hostings_controller_test.rb which explicitly
requires ostruct for the same reason.
* fix(ai): sanitize Langfuse warn logs, normalize tool_use.input, dedup history fetch
Addresses three open CodeRabbit findings on PR #1983.
- Provider::Anthropic Langfuse rescue branches no longer include
`e.full_message` in `Rails.logger.warn`. `full_message` bundles the
backtrace + cause chain and on some SDK error types includes the
serialized request/response payload (prompt, model output). Logs
now report `#{e.class}: #{e.message}` only. Three sites:
create_langfuse_trace, log_langfuse_generation, upsert_langfuse_trace.
Note: Provider::Openai has the same pattern (copy-pasted source) —
harmonization deferred to a follow-up cleanup PR; this commit fixes
only the Anthropic provider to keep PR scope tight.
- MessageFormatter#parse_arguments now coerces any non-Hash parsed
result to `{}`. Anthropic's Messages API requires `tool_use.input`
to be a JSON object (map); a stored ToolCall::Function record whose
arguments parse to a scalar, bool, or array (corrupt row, legacy
data, cross-provider bleed) would otherwise produce a payload the
API rejects. Normal flow stores Hash arguments end-to-end so the
fix is defensive — adds 2 tests covering scalar/array JSON strings
and non-String non-Hash inputs.
- Assistant::Responder dedups the chat-history fetch. The previous
layout fired two near-identical `chat.messages.where(...).includes(
:tool_calls).ordered` queries per LLM turn (one for the OpenAI-shape
payload, one for the raw-records kwarg). A new memoized
`complete_chat_messages` fetches once; `chat_message_records` filters
out the current message via `Array#reject`, `openai_messages_payload`
iterates the cached array unchanged. One SQL query per turn instead
of two. Memoization scope = single Responder instance (per LLM call),
so cache invalidation is not a concern.
All 4370 tests pass (1 pre-existing libvips env error unrelated).
Rubocop + brakeman clean.
* fix(ci): replace sk-ant- prefixed test placeholders
Pipelock secret scanner pattern-matches `sk-ant-*` as a real Anthropic
API key and fails the PR security-scan check. Test stubs and
ClimateControl env values used `sk-ant-test`, `sk-ant-from-setting`,
`sk-ant-x`, `sk-ant-y` as obvious placeholders, but the scanner does
not care about value entropy.
Switched to `fake-anthropic-key-*` / `fake-token-*` strings so the
scanner stops flagging them. No production code touched, no behavior
change — Provider::Anthropic still accepts any non-blank token.
* feat(ai): add Anthropic batch ops + LLM cost ledger (2/5)
Implements auto_categorize, auto_detect_merchants, and
enhance_provider_merchants on Provider::Anthropic via forced tool calls,
plus the cost-ledger plumbing they need.
- Provider::Anthropic::AutoCategorizer, AutoMerchantDetector,
ProviderMerchantEnhancer each define a single output tool whose
input_schema mirrors the desired output, then force the model to call
it via tool_choice: { type: "tool", name: ..., disable_parallel_tool_use: true }.
Anthropic guarantees the tool_use.input matches the schema, so there
is no JSON parsing fragility, no <think> tag stripping, and no
json_object/json_schema fallback ladders.
- Concerns::UsageRecorder mirrors the OpenAI sibling but persists
cache_creation_input_tokens / cache_read_input_tokens to dedicated
columns instead of metadata.
- Migration adds cache_creation_tokens, cache_read_tokens (nullable
integers) to llm_usages. OpenAI rows leave them null.
- LlmUsage::PRICING gains Claude 4.x rows (opus-4-7 $15/$75, sonnet-4-6
$3/$15, haiku-4-5 $1/$5 per MTok). infer_provider returns "anthropic"
for claude-* via the existing exact/prefix lookup.
- Provider::Anthropic#chat_response now persists cache columns directly
rather than stashing them in metadata.
- 25-transaction batch cap mirrors the OpenAI provider so the cost
ledger sees the same shape regardless of which provider ran a batch.
Tests cover the forced-tool-call path, null/None normalization,
case-insensitive merchant matching, the missing-tool_use error path,
and Anthropic-specific pricing + provider inference on LlmUsage.
Stacked on #1983 (PR 1/5). 3/5 PDF + vision next.
* fix(ai): attribute Bedrock model IDs to anthropic + clean nil enum
- LlmUsage.infer_provider now returns "anthropic" for Bedrock /
Vertex shaped IDs (anthropic.* and anthropic/*), so cost-ledger
filtering by provider stays correct even when no per-MTok rate is
stored. Previously these IDs fell through to the "openai" default.
- AutoCategorizer drops the redundant nil sentinel from the
category_name enum — the union type [string, null] already permits
null, and some JSON Schema validators reject nil literals inside
enum arrays.
* test(ai): require "ostruct" in Anthropic batch op tests
Same rationale as the PR1 ostruct fix — explicit require so the tests
don't depend on ActiveSupport's transitive load when Ruby 3.5+ removes
OpenStruct from the default load path.
* feat(ai): Anthropic native PDF processing (3/5)
Implements process_pdf and extract_bank_statement on Provider::Anthropic
using the native `document` content block — no rasterization, no text
pre-extraction.
- Provider::Anthropic::PdfProcessor classifies the document, summarizes
it, and extracts statement metadata via a forced report_document_analysis
tool whose input_schema mirrors the existing Provider::Openai output
(document_type from Import::DOCUMENT_TYPES, summary, extracted_data).
- Provider::Anthropic::BankStatementExtractor returns the same
{ transactions, period, account_holder, account_number, bank_name,
opening_balance, closing_balance } shape via report_bank_statement so
downstream pdf_import code is provider-agnostic.
- Both attach the PDF as
{ type: "document", source: { type: "base64", media_type: "application/pdf", data: <b64> } }
— Claude 3.5+ / 4.x accept this natively (up to 32MB / 100 pages).
No pdf-reader, no pdftoppm, no chunking for typical statements.
- supports_pdf_processing? (introduced in PR 1) already returns true for
claude-* models, gating process_pdf with a clear error otherwise.
- Cost ledger rows are persisted via the shared UsageRecorder concern,
including cache_creation/cache_read tokens.
Tests verify the document block shape, tool_choice forcing, normalized
document_type for unknown classifications, transaction normalization
(date / amount / reference → notes), and the missing-tool_use error
path. Blank pdf_content raises before any client call.
Stacked on #1984 (PR 2/5). 4/5 pgvector RAG next.
* fix(ai): guard PDF size + surface bank-statement truncation
- PdfProcessor and BankStatementExtractor raise upfront when
pdf_content.bytesize exceeds MAX_PDF_BYTES (32 MB, matching
Anthropic's hard limit). Previously a 100 MB PDF would be
base64-encoded (~133 MB) and packed into the JSON body before
the API rejected it — peak heap ~270 MB per Sidekiq worker.
- BankStatementExtractor inspects response.stop_reason; when the
model hit max_tokens it logs a warning and flags result[:truncated]
so downstream callers know the transaction list may be incomplete.
- ISO date pattern added to statement_period_start/end schema in
PdfProcessor so the model can't return "March 2026" — Anthropic
enforces the regex via the tool's input_schema.
Tests cover the size guard (raises before any client.messages call),
truncated-result flagging, and the warning log path.
* test(ai): require "ostruct" in Anthropic PDF tests
Match the explicit ostruct require added in PR1/PR2 — same Ruby 3.5+
load-path reason.
* fix(llm-usage): include Anthropic cache tokens in estimated_cost
calculate_cost only priced prompt + completion tokens, so estimated_cost
under-reported every cached call — the cache_creation/cache_read columns this PR
added were tracked but never billed. Verified against the Anthropic dashboard: a
cached chat turn billed $0.05 but the ledger recorded $0.038; the gap was exactly
the unpriced cache tokens.
Price them relative to the input rate (Anthropic: cache write 1.25x, read 0.1x)
and thread the cache counts from both recorders (chat + batch). OpenAI rows leave
the columns null (treated as 0), so they're unaffected. Ledger now reproduces the
dashboard ($0.054 for the test turn).
* chore(ai): guard chat usage double-record; flag deferred Anthropic batch wiring
- Hardening: guard the success-path record_llm_usage with
`unless partial_usage_recorded` so a future change that emits partial usage on
a normal stream can't silently double-bill (the symptom investigated in the
#1984 review). No behavior change today — on_partial only fires from the
mid-stream-error rescue, which re-raises past this line.
- Notice: the family auto-categorize / merchant-detect / merchant-enhance flows
still hardcode get_provider(:openai). Provider::Anthropic now implements those
batch ops but they aren't wired into the family flows yet — documented with
TODOs at each site for the follow-up.
* chore(ai): point family-flow TODOs at tracking issue #2113
* chore(ai): flag deferred Anthropic PDF wiring (TODO #2113)
The PDF import + bank-statement-extract flows hardcode get_provider(:openai).
Provider::Anthropic implements process_pdf / extract_bank_statement (this PR)
but they aren't wired into these paths yet — documented with TODOs at each
site. Tracked in #2113 (broadened to cover batch ops + PDF).
* chore(anthropic-pdf): drop redundant strip_heredoc; document no-dedup
- The squiggly heredoc (<<~) already strips indentation, so the trailing
.strip_heredoc was a no-op in both PDF extractors.
- Document why BankStatementExtractor intentionally does NOT deduplicate (unlike
the OpenAI extractor): we send the whole PDF as one native document block, so
there are no overlapping-chunk artifacts to dedupe, and deduping would wrongly
merge legitimate same-day, same-amount transactions.
* fix(anthropic): cap PDF bytes below the base64-encoded request limit
Anthropic's 32 MB limit is on the Messages request body, and the PDF is sent
base64-encoded (~4/3 larger) alongside the JSON envelope, so a 32 MB raw PDF
encodes to ~42 MB and is rejected. Cap the raw bytes at 3/4 of the request
budget minus a 1 MB envelope reserve (~23 MiB). Addresses Codex review on #1985.
|