* Reorganize import UI with Financial Tools / Raw Data tabs
Split the flat list of import sources into two tabbed sections using
DS::Tabs: "Financial Tools" (Mint, Quicken/QIF, YNAB coming soon) and
"Raw Data" (transactions, investments, accounts, categories, rules,
documents). This prepares for adding more tool-specific importers
without cluttering the list.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* Fix import controller test to account for YNAB coming soon entry
The new YNAB "coming soon" disabled entry adds a 5th aria-disabled
element to the import dialog.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* Fix system tests to click Raw Data tab before selecting import type
Transaction, trade, and account imports are now under the Raw Data tab
and need an explicit tab click before the buttons are visible.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* feat: Add bulk import for NDJSON export files
Implements an import flow that accepts the full all.ndjson file from data exports,
allowing users to restore their complete data including:
- Accounts with accountable types
- Categories with parent relationships
- Tags and merchants
- Transactions with category, merchant, and tag references
- Trades with securities
- Valuations
- Budgets and budget categories
- Rules with conditions and actions (including compound conditions)
Key changes:
- Add BulkImport model extending Import base class
- Add Family::DataImporter to handle NDJSON parsing and import logic
- Update imports controller and views to support NDJSON workflow
- Skip configuration/mapping steps for structured NDJSON imports
- Add i18n translations for bulk import UI
- Add tests for BulkImport and DataImporter
* fix: Fix category import and test query issues
- Add default lucide_icon ("shapes") for categories when not provided
- Fix valuation test to use proper ActiveRecord joins syntax
* Linter errors
* fix: Add default color for tags when not provided in import
* fix: Add default kind for transactions when not provided in import
* Fix test
* Fix tests
* Fix remaining merge conflicts from PR 766 cherry-pick
Resolve conflict markers in test fixtures and clean up BulkImport
entry in new.html.erb to use the _import_option partial consistently.
https://claude.ai/code/session_01BM4SBWNhATqoKTEvy3qTS3
* Import Sure `.ndjson`
* Remove `.ndjson` import from raw data
* Fix support for Sure "bulk" import from old branch
* Linter
* Fix CI test
* Fix more CI tests
* Fix tests
* Fix tests / move PDF import to first tab
* Remove redundant title
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add AI merchant enhancement and dedup
* Enhancements
Add error if job is already running
add note that we also merge merchants
* Allow updating provider website
* Review fixes
* Update provider_merchant.rb
* Linter and fixes
* FIX transaction quick menu modal
* Initial split transaction support
* Add support to unsplit and edit split
* Update show.html.erb
* FIX address reviews
* Improve UX
* Update show.html.erb
* Reviews
* Update edit.html.erb
* Add parent category to dialog
* Update en.yml
* Add UI indication to totals
* FIX ui update
* Add category select like rest of app
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Add conditional migration for vector_store_chunks table
Creates the pgvector-backed chunks table when VECTOR_STORE_PROVIDER=pgvector.
Enables the vector extension, adds store_id/file_id indexes, and uses
vector(1024) column type for embeddings.
* Add VectorStore::Embeddable concern for text extraction and embedding
Shared concern providing extract_text (PDF via pdf-reader, plain-text as-is),
paragraph-boundary chunking (~2000 chars, ~200 overlap), and embed/embed_batch
via OpenAI-compatible /v1/embeddings endpoint using Faraday. Configurable via
EMBEDDING_MODEL, EMBEDDING_URI_BASE, with fallback to OPENAI_* env vars.
* Implement VectorStore::Pgvector adapter with raw SQL
Replaces the stub with a full implementation using
ActiveRecord::Base.connection with parameterized binds. Supports
create_store, delete_store, upload_file (extract+chunk+embed+insert),
remove_file, and cosine-similarity search via the <=> operator.
* Add registry test for pgvector adapter selection
* Configure pgvector in compose.example.ai.yml
Switch db image to pgvector/pgvector:pg16, add VECTOR_STORE_PROVIDER,
EMBEDDING_MODEL, and EMBEDDING_DIMENSIONS env vars, and include
nomic-embed-text in Ollama's pre-loaded models.
* Update pgvector docs from scaffolded to ready
Document env vars, embedding model setup, pgvector Docker image
requirement, and Ollama pull instructions.
* Address PR review feedback
- Migration: remove env guard, use pgvector_available? check so it runs
on plain Postgres (CI) but creates the table on pgvector-capable servers.
Add NOT NULL constraints on content/embedding/metadata, unique index on
(store_id, file_id, chunk_index).
- Pgvector adapter: wrap chunk inserts in a DB transaction to prevent
partial file writes. Override supported_extensions to match formats
that extract_text can actually parse.
- Embeddable: add hard_split fallback for paragraphs exceeding CHUNK_SIZE
to avoid overflowing embedding model token limits.
* Bump schema version to include vector_store_chunks migration
CI uses db:schema:load which checks the version — without this bump,
the migration is detected as pending and tests fail to start.
* Update 20260316120000_create_vector_store_chunks.rb
---------
Co-authored-by: sokiee <sokysrm@gmail.com>
DeveloperMessage was a debug-only STI subclass of Message that was never
created by any production code. Remove the model, view partial, test,
and fixtures, and simplify Chat#conversation_messages accordingly.
https://claude.ai/code/session_012pm5HKGKFs1tpAsvXMr4Tp
Co-authored-by: Claude <noreply@anthropic.com>
* feat(balance): incremental ForwardCalculator — only recalculate from changed date forward
When a Sync record carries a window_start_date, ForwardCalculator now
seeds its starting balances from the persisted DB balance for
window_start_date - 1, then iterates only from window_start_date to
calc_end_date. This avoids recomputing every daily balance on a
long-lived account when a single transaction changes.
Key changes:
- Account::Syncer passes sync.window_start_date to Balance::Materializer
- Balance::Materializer accepts window_start_date and forwards it to
ForwardCalculator; purge_stale_balances uses opening_anchor_date as the
lower bound in incremental mode so pre-window balances are not deleted
- Balance::ForwardCalculator accepts window_start_date; resolve_starting_balances
loads end_cash_balance/end_non_cash_balance from the prior DB record and
falls back to full recalculation when no prior record exists
- Tests added for incremental correctness, fallback behaviour, and purge safety
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
# Conflicts:
# app/models/balance/materializer.rb
* Enhance fallback logic on ForwardCalculator and Materializer
* fix(balance): address CodeRabbit review issues on incremental ForwardCalculator
- materializer.rb: handle empty sorted_balances in incremental mode by still
purging stale tail balances beyond window_start_date - 1, preventing orphaned
future rows when a transaction is deleted and the recalc window produces no rows
- materializer_test.rb: stub incremental? alongside calculate in the incremental
sync test so the guard in ForwardCalculator#incremental? doesn't raise when
@fell_back is nil (never set because calculate was stubbed out)
- materializer_test.rb: correct window_start_date in the fallback test from
3.days.ago to 2.days.ago so window_start_date - 1 hits a date with no
persisted balance, correctly triggering full recalculation instead of
accidentally seeding from the stale wrong_pre_window balance
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(balance): multi-currency fallback to full recalculation and add corresponding tests
* address coderabbit comment about test
* Make the foreign-currency precondition explicit in the test setup.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: Add default account for manual transaction entries (#1061)
Allow users to designate a default account that auto-selects
in the transaction creation form. Also consolidates account list
actions (edit, link/unlink, enable/disable) into a meatball menu.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* - handle context menu width on mobile
- restrict default account to depository types only
- added FR, ES and DE i18n files
* - Add credit card accounts can also be used as default
- Moved logic into controller
* Scope context menu max-width to accounts menu only
- decouples the width constraint from the shared DS::Menu component by introducing an optional max_width param
* fix ci test and address issues raised by coderabbit and codex
* Address CodeRabbit review feedback
- Use .present? for institution_name guards to avoid empty UI artifacts
- Align "Set default" menu visibility with actual preselection eligibility
(active + unlinked + supports_default?) to prevent drift between UI and model
- Keep disabled star visible when account is already default but now ineligible
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add eligible_for_transaction_default? predicate to Account model
Consolidates active + unlinked + supports_default? checks into a single
shared predicate used by the controller, view, and user model guard,
preventing a direct PATCH from bypassing UI eligibility rules.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Added "Unset default" option
Added negative test for default account
Removed duplicated logic for account.eligible_for_transaction_default
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(transaction): add support for file attachments using Active Storage
* feat(attachments): implement transaction attachments with upload, show, and delete functionality
* feat(attachments): enhance attachment upload functionality to support multiple files and improved error handling
* feat(attachments): add attachment upload form and display functionality in transaction views
* feat(attachments): implement attachment validation for count, size, and content type; enhance upload form with validation hints
* fix(attachments): use correct UI components
* feat(attachments): Implement Turbo Stream responses for creating and deleting transaction attachments.
* fix(attachments): include auth in activestorage controller
* test(attachments): add test coverage for turbostream and auth
* feat(attachments): extract strings to i18n
* fix(attachments): ensure only newly added attachments are purged when transaction validation fails.
* fix(attachments): validate attachment params
* refactor(attachments): use stimulus declarative actions
* fix(attachments): add auth for other representations
* refactor(attachments): use Browse component for attachment uploads
* fix(attachments): reject empty values on attachment upload
* fix(attachments): hide the upload form if reached max uploads
* fix(attachments): correctly purge only newly added attachments on upload failure
* fix(attachments): ensure attachment count limit is respected within a transaction lock
* fix(attachments): update attachment parameter handling to avoid `ParameterMissing` errors.
* fix(components): adjust icon_only logic for buttonish
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Feat: Add QIF (Quicken Interchange Format) import functionality
- Add the ability to import QIF files for users coming from Quicken
- Includes categories and tags
- Comprehensive tests for QifImport, including parsing, row generation, and import functionality.
- Ensure handling of hierarchical categories (ex "Home:Home Improvement" is imported as Parent:Child)
* Fix QIF import issues raised in code review
- Fix two-digit year windowing in QIF date parser (e.g. '99 → 1999, not 2099)
- Fix ArgumentError from invalid `undef: :raise` encoding option
- Nil-safe `leaf_category_name` with blank guard and `.to_s` coercion
- Memoize `qif_account_type` to avoid re-parsing the full QIF file
- Add strong parameters (`selection_params`) to QifCategorySelectionsController
- Wrap all mutations in DB transactions in uploads and category-selections controllers
- Skip unchanged tag rows (only write rows where tags actually differ)
- Replace hardcoded strings with i18n keys across QIF views and nav
- Fix potentially colliding checkbox/label IDs in category selection view
- Improve keyboard accessibility: use semantic `<label>` for file picker area
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix QIF import test count and Brakeman mass assignment warning
- Update ImportsControllerTest to expect 4 disabled import options (was 3),
accounting for the new QIF import type added in this branch
- Remove :account_id from upload_params permit list; it was never accessed
through strong params (always via params.dig with Current.family scope),
so this resolves the Brakeman high-confidence mass assignment warning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix: QIF import security, safety, and i18n issues raised in code review
- Added french, spanish and german translations for newly added i18n keys
- Replace params.dig(:import, :account_id) with a proper strong-params
accessor (import_account_id) in UploadsController to satisfy Rails
parameter filtering requirements
- Guard ImportsController#show against QIF imports reaching the publish
screen before a file has been uploaded, preventing an unrescued error
on publish
- Gate the QIF "Clean" nav step link on import.uploaded? to prevent
routing to CleansController with an unconfigured import (which would
raise "Unknown import type: QifImport" via ImportsHelper)
- Replace hard-coded "txn" pluralize calls in the category/tag selection
view with t(".txn_count") and add pluralization keys to the locale file
- Localize all hard-coded strings in the QIF upload section of
uploads/show.html.erb and add corresponding en.yml keys
- Convert the CSV upload drop zone from a clickable <div> (JS-only) to
a semantic <label> element, making it keyboard-accessible without
JavaScript
* Fix: missing translations keys
* Add icon mapping and random color assignment to new categories
* fix a lint issue
* Add a warning about splits and some plumbing for future support.
Updated locales.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add post-trial inactive family cleanup with data archival
Families that expire their trial without subscribing now get cleaned up
daily. Empty families (no accounts) are destroyed immediately after a
14-day grace period. Families with meaningful data (12+ transactions,
some recent) get their data exported as NDJSON/ZIP to an ArchivedExport
record before deletion, downloadable via a token-based URL for 90 days.
- Add InactiveFamilyCleanerJob (scheduled daily at 4 AM, managed mode only)
- Add ArchivedExport model with token-based downloads
- Add inactive_trial_for_cleanup scope and requires_data_archive? to Family
- Extend DataCleanerJob to purge expired archived exports
- Add ArchivedExportsController for unauthenticated token downloads
https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ
* Fix Brakeman redirect warning in ArchivedExportsController
Use rails_blob_path instead of redirecting directly to the ActiveStorage
attachment, which avoids the allow_other_host: true open redirect.
https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ
* Update schema.rb with archived_exports table
Add the archived_exports table definition to schema.rb to match
the pending migration, unblocking CI tests.
https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ
* Fix broken CI tests for ArchivedExports and InactiveFamilyCleaner
- ArchivedExportsController 404 test: use assert_response :not_found
instead of assert_raises since Rails rescues RecordNotFound in
integration tests and returns a 404 response.
- InactiveFamilyCleanerJob test: remove assert_no_difference on
Family.count since the inactive_trial fixture gets cleaned up by
the job. The test intent is to verify the active family survives,
which is checked by assert Family.exists?.
https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ
* Wrap ArchivedExport creation in a transaction
Ensure the ArchivedExport record and its file attachment succeed
atomically. If the attach fails, the transaction rolls back so no
orphaned record is left without an export file.
https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ
* Store only a digest of the download token for ArchivedExport
Replace plaintext download_token column with download_token_digest
(SHA-256 hex). The raw token is generated via SecureRandom on create,
exposed transiently via attr_reader for use in emails/logs, and only
its digest is persisted. Lookup uses find_by_download_token! which
digests the incoming token before querying.
https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ
* Remove raw download token from cleanup job logs
Log a truncated digest prefix instead of the raw token, which is the
sole credential for the unauthenticated download endpoint.
https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ
* Fix empty assert_no_difference block in cleaner job test
Wrap the perform_now call with both assertions so the
ArchivedExport.count check actually exercises the job.
https://claude.ai/code/session_01LR3Vo83R5s5SczYe6T33dQ
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Make categories global
This solves us A LOT of cash flow and budgeting problems.
* Update schema.rb
* Update auto_categorizer.rb
* Update income_statement.rb
* FIX budget sub-categories
* FIX sub-categories and tests
* Add 2 step migration
* Enforce one pending invitation per email across all families
Users can only belong to one family, so allowing the same email to have
pending invitations from multiple families leads to ambiguous behavior.
Add a `no_other_pending_invitation` validation on create to prevent this.
Accepted and expired invitations from other families are not blocked.
Fixes#1172https://claude.ai/code/session_016fGqgha18jP48dhznm6k4m
* Normalize email before validation and use case-insensitive lookup
When ActiveRecord encryption is not configured, the email column stores
raw values preserving original casing. The prior validation used a direct
equality match which would miss case variants (e.g. Case@Test.com vs
case@test.com), leaving a gap in the cross-family uniqueness guarantee.
Fix by:
1. Adding a normalize_email callback that downcases/strips email before
validation, so all new records store lowercase consistently.
2. Using LOWER() in the SQL query for non-encrypted deployments to catch
any pre-existing mixed-case records.
https://claude.ai/code/session_016fGqgha18jP48dhznm6k4m
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Check for pending invitations before creating new Family during SSO account creation
When a user signs in via Google SSO and doesn't have an account yet, the
system now checks for pending invitations before creating a new Family.
If an invitation exists, the user joins the invited family instead.
- OidcAccountsController: check Invitation.pending in link/create_user
- API AuthController: check pending invitations in sso_create_account
- SessionsController: pass has_pending_invitation to mobile SSO callback
- Web view: show "Accept Invitation" button when invitation exists
- Flutter: show "Accept Invitation" tab/button when invitation pending
https://claude.ai/code/session_019Tr6edJa496V1ErGmsbqFU
* Fix external assistant tests: clear Settings cache to prevent test pollution
The tests relied solely on with_env_overrides to clear configuration, but
rails-settings-cached may retain stale Setting values across tests when
the cache isn't explicitly invalidated. Ensure both ENV vars AND Setting
values are cleared with Setting.clear_cache before assertions.
https://claude.ai/code/session_019Tr6edJa496V1ErGmsbqFU
---------
Co-authored-by: Claude <noreply@anthropic.com>
Both Uncategorized and Other Investments are synthetic categories with
id=nil. When expense_totals_by_category indexes by category.id, Other
Investments overwrites Uncategorized at the nil key, causing uncategorized
actual spending to always return 0.
Use category.name as fallback key (id || name) to differentiate the two
synthetic categories in all hash builders and lookup sites.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add new Date field when creating a new Account
* Fix german translation
* Update app/controllers/concerns/accountable_resource.rb
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Michel Roegl-Brunner <73236783+michelroegl-brunner@users.noreply.github.com>
* Add missing opening_balance:date to update_params
* Change label text
---------
Signed-off-by: Michel Roegl-Brunner <73236783+michelroegl-brunner@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
The `subcategories` method queries `WHERE parent_id = category.id`, but
for the synthetic uncategorized budget category, `category.id` is nil.
This caused `WHERE parent_id IS NULL` to match ALL top-level categories,
making them appear as subcategories of uncategorized. This inflated
actual_spending and produced a large negative available_to_spend.
Add a nil guard on category.id to return an empty relation for synthetic
categories.
Fixes#819
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Add budget rollover: copy from previous month
When navigating to an uninitialized budget month, show a prompt
offering to copy amounts from the most recent initialized budget.
Copies budgeted_spending, expected_income, and all matching category
allocations. Also fixes over-allocation warning showing on uninitialized
budgets.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Redirect copy_previous to categories wizard for review
Matches the normal budget setup flow (edit → categories → show)
so users can review/tweak copied allocations before confirming.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Address code review: eager-load categories, guard against overwrite
- Add .includes(:budget_categories) to most_recent_initialized_budget
to avoid N+1 when copy_from! iterates source categories
- Guard copy_previous action against overwriting already-initialized
budgets (prevents crafted POST from clobbering existing data)
- Add i18n key for already_initialized flash message
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Add invariant guards to copy_from! for defensive safety
Validate that source budget belongs to the same family and precedes
the target budget before copying. Protects against misuse from
other callers beyond the controller.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Fix button overflow on small screens in copy previous prompt
Stack buttons vertically on mobile, side-by-side on sm+ breakpoint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(helm): add Pipelock ConfigMap, scanning config, and consolidate compose
- Add ConfigMap template rendering DLP, response scanning, MCP input/tool
scanning, and forward proxy settings from values
- Mount ConfigMap as /etc/pipelock/pipelock.yaml volume in deployment
- Add checksum/config annotation for automatic pod restart on config change
- Gate HTTPS_PROXY/HTTP_PROXY env injection on forwardProxy.enabled (skip
in MCP-only mode)
- Use hasKey for all boolean values to prevent Helm default swallowing false
- Single source of truth for ports (forwardProxy.port/mcpProxy.port)
- Pipelock-specific imagePullSecrets with fallback to app secrets
- Merge standalone compose.example.pipelock.yml into compose.example.ai.yml
- Add pipelock.example.yaml for Docker Compose users
- Add exclude-paths to CI workflow for locale file false positives
* Add external assistant support (OpenAI-compatible SSE proxy)
Allow self-hosted instances to delegate chat to an external AI agent
via an OpenAI-compatible streaming endpoint. Configurable per-family
through Settings UI or ASSISTANT_TYPE env override.
- Assistant::External::Client: SSE streaming HTTP client (no new gems)
- Settings UI with type selector, env lock indicator, config status
- Helm chart and Docker Compose env var support
- 45 tests covering client, config, routing, controller, integration
* Add session key routing, email allowlist, and config plumbing
Route to the actual OpenClaw session via x-openclaw-session-key header
instead of creating isolated sessions. Gate external assistant access
behind an email allowlist (EXTERNAL_ASSISTANT_ALLOWED_EMAILS env var).
Plumb session_key and allowedEmails through Helm chart, compose, and
env template.
* Add HTTPS_PROXY support to External::Client for Pipelock integration
Net::HTTP does not auto-read HTTPS_PROXY/HTTP_PROXY env vars (unlike
Faraday). Explicitly resolve proxy from environment in build_http so
outbound traffic to the external assistant routes through Pipelock's
forward proxy when enabled. Respects NO_PROXY for internal hosts.
* Add UI fields for external assistant config (Setting-backed with env fallback)
Follow the same pattern as OpenAI settings: database-backed Setting
fields with env var defaults. Self-hosters can now configure the
external assistant URL, token, and agent ID from the browser
(Settings > Self-Hosting > AI Assistant) instead of requiring env vars.
Fields disable when the corresponding env var is set.
* Improve external assistant UI labels and add help text
Change placeholder to generic OpenAI-compatible URL pattern. Add help
text under each field explaining where the values come from: URL from
agent provider, token for authentication, agent ID for multi-agent
routing.
* Add external assistant docs and fix URL help text
Add External AI Assistant section to docs/hosting/ai.md covering setup
(UI and env vars), how it works, Pipelock security scanning, access
control, and Docker Compose example. Drop "chat completions" jargon
from URL help text.
* Harden external assistant: retry logic, disconnect UI, error handling, and test coverage
- Add retry with backoff for transient network errors (no retry after streaming starts)
- Add disconnect button with confirmation modal in self-hosting settings
- Narrow rescue scope with fallback logging for unexpected errors
- Safe cleanup of partial responses on stream interruption
- Gate ai_available? on family assistant_type instead of OR-ing all providers
- Truncate conversation history to last 20 messages
- Proxy-aware HTTP client with NO_PROXY support
- Sanitize protocol to use generic headers (X-Agent-Id, X-Session-Key)
- Full test coverage for streaming, retries, proxy routing, config, and disconnect
* Exclude external assistant client from Pipelock scan-diff
False positive: `@token` instance variable flagged as "Credential in URL".
Temporary workaround until Pipelock supports inline suppression.
* Address review feedback: NO_PROXY boundary fix, SSE done flag, design tokens
- Fix NO_PROXY matching to require domain boundary (exact match or .suffix),
case-insensitive. Prevents badexample.com matching example.com.
- Add done flag to SSE streaming so read_body stops after [DONE]
- Move MAX_CONVERSATION_MESSAGES to class level
- Use bg-success/bg-destructive design tokens for status indicators
- Add rationale comment for pipelock scan exclusion
- Update docs last-updated date
* Address second round of review feedback
- Allowlist email comparison is now case-insensitive and nil-safe
- Cap SSE buffer at 1 MB to prevent memory blowup from malformed streams
- Don't expose upstream HTTP response body in user-facing errors (log it instead)
- Fix frozen string warning on buffer initialization
- Fix "builtin" typo in docs (should be "built-in")
* Protect completed responses from cleanup, sanitize error messages
- Don't destroy a fully streamed assistant message if post-stream
metadata update fails (only cleanup partial responses)
- Log raw connection/HTTP errors internally, show generic messages
to users to avoid leaking network/proxy details
- Update test assertions for new error message wording
* Fix SSE content guard and NO_PROXY test correctness
Use nil check instead of present? for SSE delta content to preserve
whitespace-only chunks (newlines, spaces) that can occur in code output.
Fix NO_PROXY test to use HTTP_PROXY matching the http:// client URL so
the proxy resolution and NO_PROXY bypass logic are actually exercised.
* Forward proxy credentials to Net::HTTP
Pass proxy_uri.user and proxy_uri.password to Net::HTTP.new so
authenticated proxies (http://user:pass@host:port) work correctly.
Without this, credentials parsed from the proxy URL were silently
dropped. Nil values are safe as positional args when no creds exist.
* Update pipelock integration to v0.3.1 with full scanning config
Bump Helm image tag from 0.2.7 to 0.3.1. Add missing security
sections to both the Helm ConfigMap and compose example config:
mcp_tool_policy, mcp_session_binding, and tool_chain_detection.
These protect the /mcp endpoint against tool injection, session
hijacking, and multi-step exfiltration chains.
Add version and mode fields to config files. Enable include_defaults
for DLP and response scanning to merge user patterns with the 35
built-in patterns. Remove redundant --mode CLI flag from the Helm
deployment template since mode is now in the config file.
* Fix: use cookie/crumb auth in healthy? chart endpoint check
The health check was calling /v8/finance/chart/AAPL via the plain
unauthenticated client. Yahoo Finance requires cookie + crumb
authentication on the chart endpoint, so the health check would
fail even when credentials are valid. Updated healthy? to use
fetch_cookie_and_crumb + authenticated_client, consistent with
fetch_security_prices and fetch_chart_data.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix: add cookie/crumb auth to all /v8/finance/chart/ calls
fetch_security_prices and fetch_chart_data (used for exchange rates)
were calling the chart endpoint without cookie/crumb authentication,
inconsistent with healthy? and fetch_security_info. Added auth to both,
including the same retry-on-Unauthorized pattern already used in
fetch_security_info.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Update user-agent strings in yahoo_finance.rb
Updated user-agent strings to reflect current browser versions
Signed-off-by: Serge L <serge@souritech.ca>
* Fix: Add stale-crumb retry to healthy? and fetch_chart_data
Yahoo Finance returns 200 OK with {"chart":{"error":{"code":"Unauthorized"}}}
when a cached crumb expires server-side. Both healthy? and fetch_chart_data
now mirror the retry pattern already in fetch_security_prices: detect the
Unauthorized body, clear the crumb cache, fetch fresh credentials, and
retry the request once. Adds a test for the healthy? retry path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Refactor: Extract fetch_authenticated_chart helper to DRY crumb retry logic
The cookie/crumb fetch + stale-crumb retry pattern was duplicated across
healthy?, fetch_security_prices, and fetch_chart_data. Extract it into a
single private fetch_authenticated_chart(symbol, params) helper that
centralizes the retry logic; all three call sites now delegate to it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix: Catch JSON::ParserError in fetch_chart_data rescue clause
After moving JSON.parse inside fetch_authenticated_chart, a malformed
Yahoo response would throw JSON::ParserError through fetch_chart_data's
rescue Faraday::Error, breaking the inverse currency pair fallback.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix: Raise AuthenticationError if retry still returns Unauthorized
After refreshing the crumb and retrying, if Yahoo still returns an
Unauthorized error body the helper now raises AuthenticationError instead
of silently returning the error payload. This prevents callers from
misinterpreting a persistent auth failure as missing chart data.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix: Raise AuthenticationError after failed retry in fetch_security_info
Mirrors the same post-retry Unauthorized check added to fetch_authenticated_chart.
Without this, a persistent auth failure on the quoteSummary endpoint would
surface as a generic "No security info found" error instead of an AuthenticationError.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Signed-off-by: Serge L <serge@souritech.ca>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace direct trace.update calls with client trace upserts so OpenAI provider is compatible with langfuse-ruby 0.1.6 behavior. Add richer warning logs that include full exception details for trace creation, trace upserts, and generation logging failures. Add tests for client-based trace upserts and detailed error logging.
* fix: crypto subtype not persisted by permitting :subtype in CryptosController
* Backfill crypto subtype for existig accounts so Trades API works
* fix: backfill only unlinked cryptos; use raw SQL in migration; deterministic redirect in test
* Update schema.rb for BackfillcryptoSubtypeForTrades migration
---------
Signed-off-by: dataCenter430 <161712630+dataCenter430@users.noreply.github.com>
Balance sheet totals and accountable type summaries used a SQL JOIN on
exchange_rates matching only today's date, which returned NULL (defaulting
to 1:1) when no rate existed for that exact date. This caused foreign
currency accounts to show incorrect totals.
Changes:
- Refactor BalanceSheet::AccountTotals to batch-fetch exchange rates via
ExchangeRate.rates_for, with provider fallback, instead of a SQL join
- Refactor Accountable.balance_money to use the same batch approach
- Add ExchangeRate.rates_for helper for deduplicated rate lookups
- Fix net worth chart query to fall back to the nearest future rate when
no historical rate exists for a given date
- Add composite index on accounts (family_id, status, accountable_type)
- Reuse nearest cached exchange rate within a 5-day lookback window
before calling the provider, preventing redundant API calls on
weekends and holidays when providers return prior-day rates
https://claude.ai/code/session_01GyssBJxQqdWnuYofQRjUu8
Co-authored-by: Claude <noreply@anthropic.com>
* fix: Include investment_contribution in transfer? check and protect transfer entries from sync
Transfer transactions with kind "investment_contribution" were not recognized
as transfers by the UI, causing missing +/- indicators, "Transfer" labels,
and showing regular transaction forms instead of transfer details.
Also adds user_modified: true to entries created via TransferMatchesController
and SetAsTransferOrPayment rule action to protect them from provider sync
overwrites, matching the existing behavior in Transfer::Creator.
https://claude.ai/code/session_019BZ5Z1aqKSK3cRdR81P5Jg
* fix: Centralize transfer/budget kind constants for consistent investment_contribution handling
Define TRANSFER_KINDS and BUDGET_EXCLUDED_KINDS on Transaction to eliminate
hard-coded kind lists scattered across filters, rules, and analytics code.
investment_contribution is now consistently treated as a transfer in search
filters, rule conditions, and UI display (via TRANSFER_KINDS), while budget
analytics correctly continue treating it as an expense (via BUDGET_EXCLUDED_KINDS).
https://claude.ai/code/session_019BZ5Z1aqKSK3cRdR81P5Jg
* fix: Update tests for consistent investment_contribution as transfer kind
- search_test: loan_payment is now in TRANSFER_KINDS, so uncategorized
filter correctly excludes it (same as funds_movement/cc_payment)
- condition_test: investment_contribution is now a transfer kind, so it
matches the transfer filter rather than expense filter
https://claude.ai/code/session_019BZ5Z1aqKSK3cRdR81P5Jg
* fix: Eliminate SQL injection warnings in Transaction::Search
Replace string-interpolated SQL with parameterized queries:
- totals: use sanitize_sql_array with ? placeholders
- apply_category_filter: pass TRANSFER_KINDS as bind parameter
- apply_type_filter: use where(kind:)/where.not(kind:) and
parameterized IN (?) for compound OR conditions
- Remove unused transfer_kinds_sql helper
https://claude.ai/code/session_019BZ5Z1aqKSK3cRdR81P5Jg
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: locale-dependent category duplication bug
* fix: use family locale for investment contributions category to prevent duplicates and handle legacy data
* Remove v* tag trigger from flutter-build to fix double-runs
publish.yml already calls flutter-build via workflow_call on v* tags,
so the direct push trigger was causing duplicate workflow runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor mobile release asset flow
* fix: category uniqueness and workflow issues
* fix: fix test issue
* fix: solve test issue
* fix: resolve legacy problem
* fix: solve lint test issue
* fix: revert unrelated changes
---------
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Handle empty compound conditions on rules index
* fix: avoid contradictory rule condition summary on /rules
* refactor: move rules condition display logic from view to model
* fix: localize rule title fallback and preload conditions in rules index
* Add family moniker selection and dynamic UI labels
Introduce a Family moniker persisted in the database with allowed values Family/Group, add required onboarding selection for it, and thread moniker-aware copy through key user-facing views and locales. Also add helper methods and tests for onboarding form presence and family moniker behavior.
* Small copy edits/change moniker question order
* Conditional Group/Family onboarding flow fixes
* Fix label
* Grouping of fields
* Profile Info page Group/Family changes
* Only admins can change Group/Family moniker
* Repetitive defaults
* Moniker in Account model
* Moniker in User model
* Auth fix
* Sure product is also a moniker
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
* Add SearchFamilyImportedFiles assistant function with vector store support
Implement per-Family document search using OpenAI vector stores, allowing
the AI assistant to search through uploaded financial documents (tax returns,
statements, contracts, etc.). The architecture is modular with a provider-
agnostic VectorStoreConcept interface so other RAG backends can be added.
Key components:
- Assistant::Function::SearchFamilyImportedFiles - tool callable from any LLM
- Provider::VectorStoreConcept - abstract vector store interface
- Provider::Openai vector store methods (create, upload, search, delete)
- Family::VectorSearchable concern with document management
- FamilyDocument model for tracking uploaded files
- Migration adding vector_store_id to families and family_documents table
https://claude.ai/code/session_01TSkKc7a9Yu2ugm1RvSf4dh
* Extract VectorStore adapter layer for swappable backends
Replace the Provider::VectorStoreConcept mixin with a standalone adapter
architecture under VectorStore::. This cleanly separates vector store
concerns from the LLM provider and makes it trivial to swap backends.
Components:
- VectorStore::Base — abstract interface (create/delete/upload/remove/search)
- VectorStore::Openai — uses ruby-openai gem's native vector_stores.search
- VectorStore::Pgvector — skeleton for local pgvector + embedding model
- VectorStore::Qdrant — skeleton for Qdrant vector DB
- VectorStore::Registry — resolves adapter from VECTOR_STORE_PROVIDER env
- VectorStore::Response — success/failure wrapper (like Provider::Response)
Consumers updated to go through VectorStore.adapter:
- Family::VectorSearchable
- Assistant::Function::SearchFamilyImportedFiles
- FamilyDocument
Removed: Provider::VectorStoreConcept, vector store methods from Provider::Openai
https://claude.ai/code/session_01TSkKc7a9Yu2ugm1RvSf4dh
* Add Vector Store configuration docs to ai.md
Documents how to configure the document search feature, covering all
three supported backends (OpenAI, pgvector, Qdrant), environment
variables, Docker Compose examples, supported file types, and privacy
considerations.
https://claude.ai/code/session_01TSkKc7a9Yu2ugm1RvSf4dh
* No need to specify `imported` in code
* Missed a couple more places
* Tiny reordering for the human OCD
* Update app/models/assistant/function/search_family_files.rb
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
* PR comments
* More PR comments
---------
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* First cut of a simplified "intro" UI layout
* Linter
* Add guest role and intro-only access
* Fix guest role UI defaults (#940)
Use enum predicate to avoid missing role helper.
* Remove legacy user role mapping (#941)
Drop the unused user role references in role normalization
and SSO role mapping forms to avoid implying a role that
never existed.
Refs: #0
* Remove role normalization (#942)
Remove role normalization
Roles are now stored directly without legacy mappings.
* Revert role mapping logic
* Remove `normalize_role_settings`
* Remove unnecessary migration
* Make `member` the default
* Broken `.erb`
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
* Add Indexa Capital provider scaffold
Generate Indexa Capital provider scaffolding and align credential fields with the API authentication requirements.
* Fix PR 926 lint and schema CI failures
* Implement Indexa Capital provider with real API integration
- Rewrite all broken view templates (were meta-ERB from code generator)
- Create missing select_accounts.html.erb template
- Implement real API calls: list_accounts via /users/me, get_holdings
via /accounts/{number}/fiscal-results, get_account_balance via
/accounts/{number}/performance
- Add API token auth support (stored token > env token > credentials)
- Add api_token column with encryption support
- Redesign settings panel: API token prominent, credentials collapsible
- Fix account balances display using performance endpoint portfolios
- Fix accounts index empty-state guard missing indexa_capital_items
- Simplify activities fetch job (no activities API endpoint exists)
- Fix i18n interpolation (%%{ -> %{) throughout locale file
* Add tests for Indexa Capital provider integration
- IndexaCapitalItemTest: validations, credentials, scopes, sync status
- IndexaCapitalAccountTest: upsert, holdings, account provider linking
- Provider::IndexaCapitalTest: auth modes, API stubs, error handling
- IndexaCapitalItemsControllerTest: CRUD, setup, linking, authorization
- Fixtures for items (token + credentials) and accounts (mutual + pension)
52 tests, 98 assertions, 0 failures
* Address code review feedback from PR #933
- Fix zero balance bug: use `nil?` instead of `present?` so 0 is stored
- Fix has_indexa_capital_credentials? to check api_token (was ignored)
- Fix build_provider to delegate to Provided concern (was ignoring token)
- Fix IndexaCapital section outside encryption_error guard in settings
- Add account_number sanitization to prevent path traversal in API URLs
- Replace all skipped processor tests with real working tests
- Add zero-balance and path-traversal test coverage
61 tests, 107 assertions, 0 failures
* Address code review round 2: credentials validation, RuboCop, test quality
- Fix RuboCop SpaceInsideArrayLiteralBrackets in credentials check
- Chain where.not calls so all three username/document/password must be present
- Require all three credentials (||) instead of any one (&&) in validate_configuration!
- Move attr_reader to private to avoid exposing credentials publicly
- Parse dates with Date.parse in extract_balance for robustness
- Remove stale TODO and Crypto from supported_account_types
- Order build_provider query deterministically by created_at
- Replace no-op holdings assertion with meaningful assert_difference
* Address code review round 3: JSON parse safety and test precision
- Rescue JSON::ParserError on 2xx responses for clearer error messages
- Fix weak balance assertion: set balance to 0 before processing, assert
expected value (27093.01 = sum of holdings amounts)
* Include Indexa Capital in automatic family sync
Add indexa_capital_items to Family::Syncer#child_syncables so balances
and holdings refresh on daily auto-sync and login sync, not only on
manual sync button clicks.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* feat: Protect demo monitoring API key from deletion
- Add DEMO_MONITORING_KEY constant to ApiKey model
- Add `demo_monitoring_key?` method to identify the monitoring key
- Add `visible` scope to exclude monitoring key from UI queries
- Update controller to use `visible` scope, hiding the monitoring key
- Prevent revocation of the monitoring key with explicit error handling
- Update Demo::Generator to use the shared constant
Users on the demo instance can still create their own API keys,
but cannot see or delete the monitoring key used for uptime checks.
https://claude.ai/code/session_01RQFsw39K7PB5kztboVdBdB
* Linter
* Protect demo monitoring API key from deletion
* Use monitoring source for demo API key
* Add test for demo monitoring revoke guard
* Disable Rack::Attack in test and development
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Auto-create mobile OAuth application when missing (#912)
Self-hosted users who set up their instance without running `db:seed`
(or reset their database) got "Record not found" on mobile login because
`MobileDevice.shared_oauth_application` used `find_by!` which raises
when the "Sure Mobile" Doorkeeper application does not exist.
Switch to `find_or_create_by!` so the record is created transparently
on first use, matching the attributes from the seed file.
* Nice Claude Code suggestion
---------
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* feat: add SSL_CA_FILE and SSL_VERIFY environment variables to support self-signed certificates in self-hosted environments
* fix: NoMethodError by defining SSL helper methods before configure block executes
* refactor: Refactor SessionsController to use shared SslConfigurable module and simplify SSL initializer redundant checks
* refactor: improve SSL configuration robustness and error detection accuracy
* fix:HTTParty SSL options, add file validation guards, prevent Tempfile GC, and redact URLs in error logs
* fix: Fix SSL concern indentation and stub Simplefin POST correctly in tests
* fix: normalize ssl_verify to always return boolean instead of nil
* fix: solve failing SimpleFin test
* refactor: trim unused error-handling code from SslConfigurable, replace Tempfile with fixed-path CA bundle, fix namespace pollution in initializers, and add unit tests for core SSL configuration and Langfuse CRL callback.
* fix: added require ileutils in the initializer and require ostruct in the test file.
* fix: solve autoload conflict that broke provider loading, validate all certs in PEM bundles, and add missing requires.
* Fix OIDC household invitation (issue #900)
- Auto-add existing user when inviting by email (no invite email sent)
- Accept page: choose 'Create account' or 'Sign in' (supports OIDC)
- Store invitation token in session on sign-in; accept after login (password,
OIDC, OIDC link, OIDC JIT, MFA)
- Invitation#accept_for!(user): add user to household and mark accepted
- Defensive guards: nil/blank user, token normalization, accept_for! return check
* Address PR review: rename accept_for! to accept_for, i18n OIDC notice, test fixes, stub Rails.application.config
* Fix flaky system test: assert only configure step, not flash message
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add customizable budget month start day (#253)
Allow users to set a custom month-to-date start date (1st-28th) for
budgeting and MTD calculations. Useful for users who want budget
periods aligned with their pay schedule (e.g., 25th to 24th).
Changes:
- Add month_start_day column to families table (default: 1)
- Add database check constraint for valid range (1-28)
- Add Family#uses_custom_month_start?, custom_month_start_for,
custom_month_end_for, current_custom_month_period helper methods
- Add Period.current_month_for(family), last_month_for(family) methods
- Update Budget model for custom month boundaries in find_or_bootstrap,
param_to_date, budget_date_valid?, current?, and name methods
- Add month_start_day setting to Settings > Preferences UI
- Add warning message when custom month start day is configured
- Add comprehensive tests with travel_to for date robustness
Fixes#253
* Add /api/v1/user endpoint for Flutter mobile app and PWA
Expose user preferences including month_start_day via API endpoint
following existing pattern for default_period. This allows Flutter
mobile app and PWA to read/update user preferences through a
consistent API contract.
Endpoints:
- GET /api/v1/user - Read user preferences including family settings
- PATCH /api/v1/user - Update user preferences
Response includes: id, email, first_name, last_name, default_period,
locale, and family settings (currency, timezone, date_format, country,
month_start_day).
* Update Periodable to use family-aware MTD periods
When users select 'current_month' or 'last_month' period filters on
dashboard/reports, now respects the family's custom month_start_day
setting instead of using static calendar month boundaries.
This ensures MTD filter on dashboard is consistent with how budgets
calculate their periods when custom month start day is configured.
* Fix param_to_date to correctly map budget params to custom periods
When a family uses a custom start day, the previous implementation
called custom_month_start_for on the 1st of the month, which incorrectly
shifted dates before the start day to the previous month.
Now we directly construct the date using family.month_start_day, so
'jan-2026' with month_start_day=25 correctly returns Jan 25, 2026
instead of Dec 25, 2025.
* Fix param_to_date and use Current pattern in API controller
- Fix param_to_date to directly construct date with family.month_start_day
instead of using custom_month_start_for which incorrectly shifted dates
- Replace current_user with Current.user/Current.family in API controller
to follow project convention used in other API v1 controllers
* Add i18n for budget name method
Use I18n.t for localizable budget period names to follow
project conventions for user-facing strings.
* Remove unused budget_end variable in budget_date_valid?
* Use Date.current for timezone consistency in Budget#current?
* Address PR review feedback
- Remove API users endpoint (mobile won't use yet)
- Remove user route from config/routes.rb
- Remove ai_summary/document_type schema bleed from pdf-import-ai branch
* Pass family to param_to_date for custom month logic
* Run migration to add month_start_day column to schema
* Schema regressions
---------
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Use dependent: :purge_later for user profile_image cleanup
This is a simpler alternative to PR #787's callback-based approach.
Instead of adding a custom callback and method, we use Rails' built-in
`dependent: :purge_later` option which is already used by FamilyExport
and other models in the codebase.
This single-line change ensures orphaned ActiveStorage attachments are
automatically purged when a user is destroyed, without the overhead of
querying all attachments manually.
https://claude.ai/code/session_01Np3deHEAJqCBfz3aY7c3Tk
* Add dependent: :purge_later to all ActiveStorage attachments
Extends the attachment cleanup from PR #787 to cover ALL models with
ActiveStorage attachments, not just User.profile_image.
Models updated:
- PdfImport.pdf_file - prevents orphaned PDF files from imports
- Account.logo - prevents orphaned account logos
- PlaidItem.logo, SimplefinItem.logo, SnaptradeItem.logo,
CoinstatsItem.logo, CoinbaseItem.logo, LunchflowItem.logo,
MercuryItem.logo, EnableBankingItem.logo - prevents orphaned
provider logos
This ensures that when a family is deleted (cascade from last user
purge), all associated storage files are properly cleaned up via
Rails' built-in dependent: :purge_later mechanism.
https://claude.ai/code/session_01Np3deHEAJqCBfz3aY7c3Tk
* Make sure `Provider` generator adds it
* Fix tests
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add import row generation from PDF extracted data
- Add generate_rows_from_extracted_data method to PdfImport
- Add import! method to create transactions from PDF rows
- Update ProcessPdfJob to generate rows after extraction
- Update configured?, cleaned?, publishable? for PDF workflow
- Add column_keys, required_column_keys, mapping_steps
- Set bank statements to pending status for user review
- Add tests for new functionality
Closes#844
* Add tests for BankStatementExtractor
- Test transaction extraction from PDF content
- Test deduplication across chunk boundaries
- Test amount normalization for various formats
- Test graceful handling of malformed JSON responses
- Test error handling for empty/nil PDF content
* Fix supports_pdf_processing? to validate effective model
The validation was always checking @default_model, but process_pdf
allows overriding the model via parameter. This could cause a
vision-capable override model to be rejected, or a non-vision-capable
override to pass validation only to fail during processing.
Changes:
- supports_pdf_processing? now accepts optional model parameter
- process_pdf passes effective model to validation
- Raise Provider::Openai::Error inside with_provider_response for
consistent error handling
Addresses review feedback from PR#808
* Fix insert_all! bug: explicitly set import_id
Rails insert_all! on associations does NOT auto-set the foreign key.
Added import_id explicitly and use Import::Row.insert_all! directly.
Also reload rows before counting to ensure accurate count.
* Fix pending status showing as processing for bank statements with rows
When bank statement PDF imports have extracted rows, show a 'Ready for Review'
screen with a link to the confirm path instead of the 'Processing' spinner.
This addresses the PR feedback that users couldn't reach the review flow even
though rows were created.
* Gate publishable? on account.present? to prevent import failure
PDF imports are created without an account, and import! raises if account
is missing. This prevents users from hitting publish and having the job fail.
* Wrap generate_rows_from_extracted_data in transaction for atomicity
- Clear rows and reset count even when no transactions extracted
- Use transaction block to prevent partial updates on failure
- Use mapped_rows.size instead of reload for count
* Localize transactions count string with i18n helper
* Add AccountMapping step for PDF imports when account is nil
PDF imports need account selection before publishing. This adds
Import::AccountMapping to mapping_steps when account is nil,
matching the behavior of TransactionImport and TradeImport.
Addresses PR#846 feedback about account selection for PDF imports.
* Only include CategoryMapping when rows have non-empty categories
PDF extraction doesn't extract categories from bank statements,
so the CategoryMapping step would show empty. Now we only include
CategoryMapping if rows actually have non-empty category values.
This prevents showing an empty mapping step for PDF imports.
* Fix PDF import UI flow and account selection
- Add direct account selection in PDF import UI instead of AccountMapping
- AccountMapping designed for CSV imports with multiple account values
- PDF imports need single account for all transactions
- Add update action and route for imports controller
- Fix controller to handle pdf_import param format from form_with
- Show Publish button when import is publishable (account set)
- Fix stepper nav: Upload/Configure/Clean non-clickable for PDF imports
- Redirect PDF imports from configuration step (auto-configured)
- Improve AI prompt to recognize M-PESA/mobile money as bank statements
- Fix migration ordering for import_rows table columns
* Add guard for invalid account_id in imports#update
Prevents silently clearing account when invalid ID is passed.
Returns error message instead of confusing 'Account saved' notice.
* Localize step names in import nav and add account guard
- Use t() helper for all step names (Upload, Configure, Clean, Map, Confirm)
- Add guard for invalid account_id in imports#update
- Prevents silently clearing account when invalid ID is passed
* Make category column migrations idempotent
Check if columns exist before adding to prevent duplicate column
errors when migrations are re-run with new timestamps.
* Add match_path for PDF import step highlighting
Fixes step detection when path is nil by using separate match_path
for current step highlighting while keeping links disabled.
* Rename category migrations and update to Rails 7.2
- Rename class to EnsureCategoryFieldsOnImportRows to avoid conflicts
- Rename class to EnsureCategoryIconOnImportRows
- Update migration version from 7.1 to 7.2 per guidelines
- Rename files to match class names
- Add match_path for PDF import step highlighting
* Use primary (black) style for Create Account and Save buttons
* Remove match_path from auto-completed PDF steps
Only step 4 (Confirm) needs match_path for active-step detection.
Steps 1-3 are purely informational and always complete.
* Add fallback for document type translation
Handles nil or unexpected document_type values gracefully.
Also removes match_path from auto-completed PDF steps.
* Use index-based step number for mobile indicator
Fixes 'Step 5 of 4' issue when Map step is dynamically removed.
* Fix hostings_controller_test: use blank? instead of nil
Setting returns empty string not nil for unset values.
* Localize step progress label and use design token
* Fix button styling: use design system Tailwind classes
btn--primary and btn--secondary CSS classes don't exist.
Use actual design system classes from DS::Buttonish.
* Fix CRLF line endings in tags_controller_test.rb
---------
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
