Balance sheet totals and accountable type summaries used a SQL JOIN on
exchange_rates matching only today's date, which returned NULL (defaulting
to 1:1) when no rate existed for that exact date. This caused foreign
currency accounts to show incorrect totals.
Changes:
- Refactor BalanceSheet::AccountTotals to batch-fetch exchange rates via
ExchangeRate.rates_for, with provider fallback, instead of a SQL join
- Refactor Accountable.balance_money to use the same batch approach
- Add ExchangeRate.rates_for helper for deduplicated rate lookups
- Fix net worth chart query to fall back to the nearest future rate when
no historical rate exists for a given date
- Add composite index on accounts (family_id, status, accountable_type)
- Reuse nearest cached exchange rate within a 5-day lookback window
before calling the provider, preventing redundant API calls on
weekends and holidays when providers return prior-day rates
https://claude.ai/code/session_01GyssBJxQqdWnuYofQRjUu8
Co-authored-by: Claude <noreply@anthropic.com>
* fix: Include investment_contribution in transfer? check and protect transfer entries from sync
Transfer transactions with kind "investment_contribution" were not recognized
as transfers by the UI, causing missing +/- indicators, "Transfer" labels,
and showing regular transaction forms instead of transfer details.
Also adds user_modified: true to entries created via TransferMatchesController
and SetAsTransferOrPayment rule action to protect them from provider sync
overwrites, matching the existing behavior in Transfer::Creator.
https://claude.ai/code/session_019BZ5Z1aqKSK3cRdR81P5Jg
* fix: Centralize transfer/budget kind constants for consistent investment_contribution handling
Define TRANSFER_KINDS and BUDGET_EXCLUDED_KINDS on Transaction to eliminate
hard-coded kind lists scattered across filters, rules, and analytics code.
investment_contribution is now consistently treated as a transfer in search
filters, rule conditions, and UI display (via TRANSFER_KINDS), while budget
analytics correctly continue treating it as an expense (via BUDGET_EXCLUDED_KINDS).
https://claude.ai/code/session_019BZ5Z1aqKSK3cRdR81P5Jg
* fix: Update tests for consistent investment_contribution as transfer kind
- search_test: loan_payment is now in TRANSFER_KINDS, so uncategorized
filter correctly excludes it (same as funds_movement/cc_payment)
- condition_test: investment_contribution is now a transfer kind, so it
matches the transfer filter rather than expense filter
https://claude.ai/code/session_019BZ5Z1aqKSK3cRdR81P5Jg
* fix: Eliminate SQL injection warnings in Transaction::Search
Replace string-interpolated SQL with parameterized queries:
- totals: use sanitize_sql_array with ? placeholders
- apply_category_filter: pass TRANSFER_KINDS as bind parameter
- apply_type_filter: use where(kind:)/where.not(kind:) and
parameterized IN (?) for compound OR conditions
- Remove unused transfer_kinds_sql helper
https://claude.ai/code/session_019BZ5Z1aqKSK3cRdR81P5Jg
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: locale-dependent category duplication bug
* fix: use family locale for investment contributions category to prevent duplicates and handle legacy data
* Remove v* tag trigger from flutter-build to fix double-runs
publish.yml already calls flutter-build via workflow_call on v* tags,
so the direct push trigger was causing duplicate workflow runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Refactor mobile release asset flow
* fix: category uniqueness and workflow issues
* fix: fix test issue
* fix: solve test issue
* fix: resolve legacy problem
* fix: solve lint test issue
* fix: revert unrelated changes
---------
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Handle empty compound conditions on rules index
* fix: avoid contradictory rule condition summary on /rules
* refactor: move rules condition display logic from view to model
* fix: localize rule title fallback and preload conditions in rules index
* Add family moniker selection and dynamic UI labels
Introduce a Family moniker persisted in the database with allowed values Family/Group, add required onboarding selection for it, and thread moniker-aware copy through key user-facing views and locales. Also add helper methods and tests for onboarding form presence and family moniker behavior.
* Small copy edits/change moniker question order
* Conditional Group/Family onboarding flow fixes
* Fix label
* Grouping of fields
* Profile Info page Group/Family changes
* Only admins can change Group/Family moniker
* Repetitive defaults
* Moniker in Account model
* Moniker in User model
* Auth fix
* Sure product is also a moniker
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
* Add SearchFamilyImportedFiles assistant function with vector store support
Implement per-Family document search using OpenAI vector stores, allowing
the AI assistant to search through uploaded financial documents (tax returns,
statements, contracts, etc.). The architecture is modular with a provider-
agnostic VectorStoreConcept interface so other RAG backends can be added.
Key components:
- Assistant::Function::SearchFamilyImportedFiles - tool callable from any LLM
- Provider::VectorStoreConcept - abstract vector store interface
- Provider::Openai vector store methods (create, upload, search, delete)
- Family::VectorSearchable concern with document management
- FamilyDocument model for tracking uploaded files
- Migration adding vector_store_id to families and family_documents table
https://claude.ai/code/session_01TSkKc7a9Yu2ugm1RvSf4dh
* Extract VectorStore adapter layer for swappable backends
Replace the Provider::VectorStoreConcept mixin with a standalone adapter
architecture under VectorStore::. This cleanly separates vector store
concerns from the LLM provider and makes it trivial to swap backends.
Components:
- VectorStore::Base — abstract interface (create/delete/upload/remove/search)
- VectorStore::Openai — uses ruby-openai gem's native vector_stores.search
- VectorStore::Pgvector — skeleton for local pgvector + embedding model
- VectorStore::Qdrant — skeleton for Qdrant vector DB
- VectorStore::Registry — resolves adapter from VECTOR_STORE_PROVIDER env
- VectorStore::Response — success/failure wrapper (like Provider::Response)
Consumers updated to go through VectorStore.adapter:
- Family::VectorSearchable
- Assistant::Function::SearchFamilyImportedFiles
- FamilyDocument
Removed: Provider::VectorStoreConcept, vector store methods from Provider::Openai
https://claude.ai/code/session_01TSkKc7a9Yu2ugm1RvSf4dh
* Add Vector Store configuration docs to ai.md
Documents how to configure the document search feature, covering all
three supported backends (OpenAI, pgvector, Qdrant), environment
variables, Docker Compose examples, supported file types, and privacy
considerations.
https://claude.ai/code/session_01TSkKc7a9Yu2ugm1RvSf4dh
* No need to specify `imported` in code
* Missed a couple more places
* Tiny reordering for the human OCD
* Update app/models/assistant/function/search_family_files.rb
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
* PR comments
* More PR comments
---------
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Include newer providers in automatic family sync
Coinbase, CoinStats, Mercury, and SnapTrade all implement Syncable
and have Syncer classes but were not listed in child_syncables,
meaning their data only refreshed on manual sync button clicks.
* refactor(syncer): Open/Closed principle for provider sync
- Adding new providers requires modifying child_syncables (violates O/C)
- plaid_items missing .active scope (bug: syncs deleted items)
- snaptrade_items can exist without user registration → fails on sync
- Scattered knowledge about 'ready to sync' logic
1. **Registry pattern**: SYNCABLE_ITEM_ASSOCIATIONS constant lists all
provider associations that participate in family sync
2. **Encapsulated sync-readiness**: Each item model defines its own
`syncable` scope that knows when it's ready for auto-sync:
- Most providers: `syncable = active` (not scheduled for deletion)
- SnapTrade: `syncable = active + user_registered` (has API creds)
3. **Single loop**: child_syncables iterates the registry, calling
`.syncable` on each association
- Adding a provider = add to registry + define syncable scope
- Each model owns its 'ready to sync' business logic
- Fixes plaid_items bug (now uses .active via .syncable)
- Fixes snaptrade auto-sync failures (filters unregistered items)
- Easy to extend with new conditions per provider
- family/syncer.rb: Registry + dynamic collection
- *_item.rb (7 files): Add `scope :syncable, -> { active }`
- snaptrade_item.rb: Add syncable with user_registered filter
* Fix rubocop bracket spacing in SnaptradeItem syncable scope
* First cut of a simplified "intro" UI layout
* Linter
* Add guest role and intro-only access
* Fix guest role UI defaults (#940)
Use enum predicate to avoid missing role helper.
* Remove legacy user role mapping (#941)
Drop the unused user role references in role normalization
and SSO role mapping forms to avoid implying a role that
never existed.
Refs: #0
* Remove role normalization (#942)
Remove role normalization
Roles are now stored directly without legacy mappings.
* Revert role mapping logic
* Remove `normalize_role_settings`
* Remove unnecessary migration
* Make `member` the default
* Broken `.erb`
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
* Add Indexa Capital provider scaffold
Generate Indexa Capital provider scaffolding and align credential fields with the API authentication requirements.
* Fix PR 926 lint and schema CI failures
* Implement Indexa Capital provider with real API integration
- Rewrite all broken view templates (were meta-ERB from code generator)
- Create missing select_accounts.html.erb template
- Implement real API calls: list_accounts via /users/me, get_holdings
via /accounts/{number}/fiscal-results, get_account_balance via
/accounts/{number}/performance
- Add API token auth support (stored token > env token > credentials)
- Add api_token column with encryption support
- Redesign settings panel: API token prominent, credentials collapsible
- Fix account balances display using performance endpoint portfolios
- Fix accounts index empty-state guard missing indexa_capital_items
- Simplify activities fetch job (no activities API endpoint exists)
- Fix i18n interpolation (%%{ -> %{) throughout locale file
* Add tests for Indexa Capital provider integration
- IndexaCapitalItemTest: validations, credentials, scopes, sync status
- IndexaCapitalAccountTest: upsert, holdings, account provider linking
- Provider::IndexaCapitalTest: auth modes, API stubs, error handling
- IndexaCapitalItemsControllerTest: CRUD, setup, linking, authorization
- Fixtures for items (token + credentials) and accounts (mutual + pension)
52 tests, 98 assertions, 0 failures
* Address code review feedback from PR #933
- Fix zero balance bug: use `nil?` instead of `present?` so 0 is stored
- Fix has_indexa_capital_credentials? to check api_token (was ignored)
- Fix build_provider to delegate to Provided concern (was ignoring token)
- Fix IndexaCapital section outside encryption_error guard in settings
- Add account_number sanitization to prevent path traversal in API URLs
- Replace all skipped processor tests with real working tests
- Add zero-balance and path-traversal test coverage
61 tests, 107 assertions, 0 failures
* Address code review round 2: credentials validation, RuboCop, test quality
- Fix RuboCop SpaceInsideArrayLiteralBrackets in credentials check
- Chain where.not calls so all three username/document/password must be present
- Require all three credentials (||) instead of any one (&&) in validate_configuration!
- Move attr_reader to private to avoid exposing credentials publicly
- Parse dates with Date.parse in extract_balance for robustness
- Remove stale TODO and Crypto from supported_account_types
- Order build_provider query deterministically by created_at
- Replace no-op holdings assertion with meaningful assert_difference
* Address code review round 3: JSON parse safety and test precision
- Rescue JSON::ParserError on 2xx responses for clearer error messages
- Fix weak balance assertion: set balance to 0 before processing, assert
expected value (27093.01 = sum of holdings amounts)
* Include Indexa Capital in automatic family sync
Add indexa_capital_items to Family::Syncer#child_syncables so balances
and holdings refresh on daily auto-sync and login sync, not only on
manual sync button clicks.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* feat: Protect demo monitoring API key from deletion
- Add DEMO_MONITORING_KEY constant to ApiKey model
- Add `demo_monitoring_key?` method to identify the monitoring key
- Add `visible` scope to exclude monitoring key from UI queries
- Update controller to use `visible` scope, hiding the monitoring key
- Prevent revocation of the monitoring key with explicit error handling
- Update Demo::Generator to use the shared constant
Users on the demo instance can still create their own API keys,
but cannot see or delete the monitoring key used for uptime checks.
https://claude.ai/code/session_01RQFsw39K7PB5kztboVdBdB
* Linter
* Protect demo monitoring API key from deletion
* Use monitoring source for demo API key
* Add test for demo monitoring revoke guard
* Disable Rack::Attack in test and development
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Auto-create mobile OAuth application when missing (#912)
Self-hosted users who set up their instance without running `db:seed`
(or reset their database) got "Record not found" on mobile login because
`MobileDevice.shared_oauth_application` used `find_by!` which raises
when the "Sure Mobile" Doorkeeper application does not exist.
Switch to `find_or_create_by!` so the record is created transparently
on first use, matching the attributes from the seed file.
* Nice Claude Code suggestion
---------
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* feat: add SSL_CA_FILE and SSL_VERIFY environment variables to support self-signed certificates in self-hosted environments
* fix: NoMethodError by defining SSL helper methods before configure block executes
* refactor: Refactor SessionsController to use shared SslConfigurable module and simplify SSL initializer redundant checks
* refactor: improve SSL configuration robustness and error detection accuracy
* fix:HTTParty SSL options, add file validation guards, prevent Tempfile GC, and redact URLs in error logs
* fix: Fix SSL concern indentation and stub Simplefin POST correctly in tests
* fix: normalize ssl_verify to always return boolean instead of nil
* fix: solve failing SimpleFin test
* refactor: trim unused error-handling code from SslConfigurable, replace Tempfile with fixed-path CA bundle, fix namespace pollution in initializers, and add unit tests for core SSL configuration and Langfuse CRL callback.
* fix: added require ileutils in the initializer and require ostruct in the test file.
* fix: solve autoload conflict that broke provider loading, validate all certs in PEM bundles, and add missing requires.
* Fix OIDC household invitation (issue #900)
- Auto-add existing user when inviting by email (no invite email sent)
- Accept page: choose 'Create account' or 'Sign in' (supports OIDC)
- Store invitation token in session on sign-in; accept after login (password,
OIDC, OIDC link, OIDC JIT, MFA)
- Invitation#accept_for!(user): add user to household and mark accepted
- Defensive guards: nil/blank user, token normalization, accept_for! return check
* Address PR review: rename accept_for! to accept_for, i18n OIDC notice, test fixes, stub Rails.application.config
* Fix flaky system test: assert only configure step, not flash message
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix: handle tags separately from entryable_attributes in bulk updates
Tags use a join table (taggings) rather than a direct column, which means
empty tag_ids clears all tags rather than meaning "no change". This caused
bulk category-only edits to accidentally clear existing tags.
This fix:
- Removes tag_ids from entryable_attributes in Entry.bulk_update!
- Adds update_tags parameter to explicitly control tag updates
- Uses params.key?(:tag_ids) in controller to detect explicit tag changes
- Preserves existing tags when tag_ids is not provided in the request
This is a cleaner architectural solution compared to tracking "touched"
state in the frontend, as it properly acknowledges the semantic difference
between column attributes and join table associations.
https://claude.ai/code/session_014CsmTwjteP4qJs6YZqCKnY
* fix: handle tags separately in API transaction updates
Apply the same pattern to the API endpoint: tags are now handled
separately from entryable_attributes to distinguish between "not
provided" (preserve existing tags) and "explicitly set to empty"
(clear all tags).
This allows API consumers to:
- Update other fields without affecting tags (omit tag_ids)
- Clear all tags (send tag_ids: [])
- Set specific tags (send tag_ids: [id1, id2])
https://claude.ai/code/session_014CsmTwjteP4qJs6YZqCKnY
* Proposed fix
* fix: improve tag handling in bulk updates for transactions
* fix: allow bulk edit to clear/preserve tags by omitting hidden multi-select field
* PR comments
* Dumb copy/paste error
* Linter
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* Add mobile SSO support to sessions controller
Add /auth/mobile/:provider route and mobile_sso_start action that
captures device params in session and renders an auto-submitting POST
form to OmniAuth (required by omniauth-rails_csrf_protection).
Modify openid_connect callback to detect mobile_sso session, issue
Doorkeeper tokens via MobileDevice, and redirect to sureapp://oauth/callback
with tokens. Handles MFA users and unlinked accounts with error redirects.
Validates provider name against configured SSO providers and device info
before proceeding.
* Add SSO auth flow to Flutter service and provider
Add buildSsoUrl() and handleSsoCallback() to AuthService for
constructing the mobile SSO URL and parsing tokens from the deep
link callback.
Add startSsoLogin() and handleSsoCallback() to AuthProvider for
launching browser-based SSO and processing the redirect.
* Register deep link listener for SSO callback
Listen for sureapp://oauth/* deep links via app_links package,
handling both cold start (getInitialLink) and warm (uriLinkStream)
scenarios. Routes callbacks to AuthProvider.handleSsoCallback().
* Add Google Sign-In button to Flutter login screen
Add "or" divider and outlined Google Sign-In button that triggers
browser-based SSO via startSsoLogin('google_oauth2').
Add app_links and url_launcher dependencies to pubspec.yaml.
* Fix mobile SSO failure handling to redirect back to app
When OmniAuth fails during mobile SSO flow, redirect to
sureapp://oauth/callback with the error instead of the web login page.
Cleans up mobile_sso session data on failure.
* Address PR review feedback for mobile SSO flow
- Use strong params for device info in mobile_sso_start
- Guard against nil session data in handle_mobile_sso_callback
- Add error handling for AppLinks initialization and stream
- Handle launchUrl false return value in SSO login
- Use user-friendly error messages instead of exposing exceptions
- Reject empty token strings in SSO callback validation
* Consolidate mobile device token logic into MobileDevice model
Extract duplicated device upsert and token issuance code from
AuthController and SessionsController into MobileDevice. Add
CALLBACK_URL constant and URL builder helpers to eliminate repeated
deep-link strings. Add mobile SSO integration tests covering the
full flow, MFA rejection, unlinked accounts, and failure handling.
* Fix CI: resolve Brakeman redirect warnings and rubocop empty line
Move mobile SSO redirect into a private controller method with an
inline string literal so Brakeman can statically verify the target.
Remove unused URL builder helpers from MobileDevice. Fix extra empty
line at end of AuthController class body.
* Use authorization code exchange for mobile SSO and add signup error handling
Replace passing plaintext tokens in mobile SSO redirect URLs with a
one-time authorization code pattern. Tokens are now stored server-side
in Rails.cache (5min TTL) and exchanged via a secure POST to
/api/v1/auth/sso_exchange. Also wraps device/token creation in the
signup action with error handling and sanitizes device error messages.
* Add error handling for login device registration and blank SSO code guard
* Address PR #860 review: fix SSO race condition, add OpenAPI spec, and cleanup
- Fix race condition in sso_exchange by checking Rails.cache.delete return
value to ensure only one request can consume an authorization code
- Use strong parameters (params.require) for sso_exchange code param
- Move inline HTML from mobile_sso_start to a proper view template
- Clear stale session[:mobile_sso] flag on web login paths to prevent
abandoned mobile flows from hijacking subsequent web SSO logins
- Add OpenAPI/rswag spec for all auth API endpoints
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix mobile SSO test to match authorization code exchange pattern
The test was asserting tokens directly in the callback URL, but the code
uses an authorization code exchange pattern. Updated to exchange the code
via the sso_exchange API endpoint. Also swaps in a MemoryStore for this
test since the test environment uses null_store which discards writes.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Refactor mobile OAuth to use single shared application
Replace per-device Doorkeeper::Application creation with a shared
"Sure Mobile" OAuth app. Device tracking uses mobile_device_id on
access tokens instead of oauth_application_id on mobile_devices.
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* Add customizable budget month start day (#253)
Allow users to set a custom month-to-date start date (1st-28th) for
budgeting and MTD calculations. Useful for users who want budget
periods aligned with their pay schedule (e.g., 25th to 24th).
Changes:
- Add month_start_day column to families table (default: 1)
- Add database check constraint for valid range (1-28)
- Add Family#uses_custom_month_start?, custom_month_start_for,
custom_month_end_for, current_custom_month_period helper methods
- Add Period.current_month_for(family), last_month_for(family) methods
- Update Budget model for custom month boundaries in find_or_bootstrap,
param_to_date, budget_date_valid?, current?, and name methods
- Add month_start_day setting to Settings > Preferences UI
- Add warning message when custom month start day is configured
- Add comprehensive tests with travel_to for date robustness
Fixes#253
* Add /api/v1/user endpoint for Flutter mobile app and PWA
Expose user preferences including month_start_day via API endpoint
following existing pattern for default_period. This allows Flutter
mobile app and PWA to read/update user preferences through a
consistent API contract.
Endpoints:
- GET /api/v1/user - Read user preferences including family settings
- PATCH /api/v1/user - Update user preferences
Response includes: id, email, first_name, last_name, default_period,
locale, and family settings (currency, timezone, date_format, country,
month_start_day).
* Update Periodable to use family-aware MTD periods
When users select 'current_month' or 'last_month' period filters on
dashboard/reports, now respects the family's custom month_start_day
setting instead of using static calendar month boundaries.
This ensures MTD filter on dashboard is consistent with how budgets
calculate their periods when custom month start day is configured.
* Fix param_to_date to correctly map budget params to custom periods
When a family uses a custom start day, the previous implementation
called custom_month_start_for on the 1st of the month, which incorrectly
shifted dates before the start day to the previous month.
Now we directly construct the date using family.month_start_day, so
'jan-2026' with month_start_day=25 correctly returns Jan 25, 2026
instead of Dec 25, 2025.
* Fix param_to_date and use Current pattern in API controller
- Fix param_to_date to directly construct date with family.month_start_day
instead of using custom_month_start_for which incorrectly shifted dates
- Replace current_user with Current.user/Current.family in API controller
to follow project convention used in other API v1 controllers
* Add i18n for budget name method
Use I18n.t for localizable budget period names to follow
project conventions for user-facing strings.
* Remove unused budget_end variable in budget_date_valid?
* Use Date.current for timezone consistency in Budget#current?
* Address PR review feedback
- Remove API users endpoint (mobile won't use yet)
- Remove user route from config/routes.rb
- Remove ai_summary/document_type schema bleed from pdf-import-ai branch
* Pass family to param_to_date for custom month logic
* Run migration to add month_start_day column to schema
* Schema regressions
---------
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Use dependent: :purge_later for user profile_image cleanup
This is a simpler alternative to PR #787's callback-based approach.
Instead of adding a custom callback and method, we use Rails' built-in
`dependent: :purge_later` option which is already used by FamilyExport
and other models in the codebase.
This single-line change ensures orphaned ActiveStorage attachments are
automatically purged when a user is destroyed, without the overhead of
querying all attachments manually.
https://claude.ai/code/session_01Np3deHEAJqCBfz3aY7c3Tk
* Add dependent: :purge_later to all ActiveStorage attachments
Extends the attachment cleanup from PR #787 to cover ALL models with
ActiveStorage attachments, not just User.profile_image.
Models updated:
- PdfImport.pdf_file - prevents orphaned PDF files from imports
- Account.logo - prevents orphaned account logos
- PlaidItem.logo, SimplefinItem.logo, SnaptradeItem.logo,
CoinstatsItem.logo, CoinbaseItem.logo, LunchflowItem.logo,
MercuryItem.logo, EnableBankingItem.logo - prevents orphaned
provider logos
This ensures that when a family is deleted (cascade from last user
purge), all associated storage files are properly cleaned up via
Rails' built-in dependent: :purge_later mechanism.
https://claude.ai/code/session_01Np3deHEAJqCBfz3aY7c3Tk
* Make sure `Provider` generator adds it
* Fix tests
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add import row generation from PDF extracted data
- Add generate_rows_from_extracted_data method to PdfImport
- Add import! method to create transactions from PDF rows
- Update ProcessPdfJob to generate rows after extraction
- Update configured?, cleaned?, publishable? for PDF workflow
- Add column_keys, required_column_keys, mapping_steps
- Set bank statements to pending status for user review
- Add tests for new functionality
Closes#844
* Add tests for BankStatementExtractor
- Test transaction extraction from PDF content
- Test deduplication across chunk boundaries
- Test amount normalization for various formats
- Test graceful handling of malformed JSON responses
- Test error handling for empty/nil PDF content
* Fix supports_pdf_processing? to validate effective model
The validation was always checking @default_model, but process_pdf
allows overriding the model via parameter. This could cause a
vision-capable override model to be rejected, or a non-vision-capable
override to pass validation only to fail during processing.
Changes:
- supports_pdf_processing? now accepts optional model parameter
- process_pdf passes effective model to validation
- Raise Provider::Openai::Error inside with_provider_response for
consistent error handling
Addresses review feedback from PR#808
* Fix insert_all! bug: explicitly set import_id
Rails insert_all! on associations does NOT auto-set the foreign key.
Added import_id explicitly and use Import::Row.insert_all! directly.
Also reload rows before counting to ensure accurate count.
* Fix pending status showing as processing for bank statements with rows
When bank statement PDF imports have extracted rows, show a 'Ready for Review'
screen with a link to the confirm path instead of the 'Processing' spinner.
This addresses the PR feedback that users couldn't reach the review flow even
though rows were created.
* Gate publishable? on account.present? to prevent import failure
PDF imports are created without an account, and import! raises if account
is missing. This prevents users from hitting publish and having the job fail.
* Wrap generate_rows_from_extracted_data in transaction for atomicity
- Clear rows and reset count even when no transactions extracted
- Use transaction block to prevent partial updates on failure
- Use mapped_rows.size instead of reload for count
* Localize transactions count string with i18n helper
* Add AccountMapping step for PDF imports when account is nil
PDF imports need account selection before publishing. This adds
Import::AccountMapping to mapping_steps when account is nil,
matching the behavior of TransactionImport and TradeImport.
Addresses PR#846 feedback about account selection for PDF imports.
* Only include CategoryMapping when rows have non-empty categories
PDF extraction doesn't extract categories from bank statements,
so the CategoryMapping step would show empty. Now we only include
CategoryMapping if rows actually have non-empty category values.
This prevents showing an empty mapping step for PDF imports.
* Fix PDF import UI flow and account selection
- Add direct account selection in PDF import UI instead of AccountMapping
- AccountMapping designed for CSV imports with multiple account values
- PDF imports need single account for all transactions
- Add update action and route for imports controller
- Fix controller to handle pdf_import param format from form_with
- Show Publish button when import is publishable (account set)
- Fix stepper nav: Upload/Configure/Clean non-clickable for PDF imports
- Redirect PDF imports from configuration step (auto-configured)
- Improve AI prompt to recognize M-PESA/mobile money as bank statements
- Fix migration ordering for import_rows table columns
* Add guard for invalid account_id in imports#update
Prevents silently clearing account when invalid ID is passed.
Returns error message instead of confusing 'Account saved' notice.
* Localize step names in import nav and add account guard
- Use t() helper for all step names (Upload, Configure, Clean, Map, Confirm)
- Add guard for invalid account_id in imports#update
- Prevents silently clearing account when invalid ID is passed
* Make category column migrations idempotent
Check if columns exist before adding to prevent duplicate column
errors when migrations are re-run with new timestamps.
* Add match_path for PDF import step highlighting
Fixes step detection when path is nil by using separate match_path
for current step highlighting while keeping links disabled.
* Rename category migrations and update to Rails 7.2
- Rename class to EnsureCategoryFieldsOnImportRows to avoid conflicts
- Rename class to EnsureCategoryIconOnImportRows
- Update migration version from 7.1 to 7.2 per guidelines
- Rename files to match class names
- Add match_path for PDF import step highlighting
* Use primary (black) style for Create Account and Save buttons
* Remove match_path from auto-completed PDF steps
Only step 4 (Confirm) needs match_path for active-step detection.
Steps 1-3 are purely informational and always complete.
* Add fallback for document type translation
Handles nil or unexpected document_type values gracefully.
Also removes match_path from auto-completed PDF steps.
* Use index-based step number for mobile indicator
Fixes 'Step 5 of 4' issue when Map step is dynamically removed.
* Fix hostings_controller_test: use blank? instead of nil
Setting returns empty string not nil for unset values.
* Localize step progress label and use design token
* Fix button styling: use design system Tailwind classes
btn--primary and btn--secondary CSS classes don't exist.
Use actual design system classes from DS::Buttonish.
* Fix CRLF line endings in tags_controller_test.rb
---------
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
* fix: lunchflow parity with simplefin/plaid pending behaviour
* fix: don't suggest duplicate if both entries are pending
* refactor: reuse the same external_id for re-synced pending transactions
* chore: replace illogical duplicate collision test with multiple sync test
* fix: prevent duplicates when users edit pending lunchflow transactions
* chore: add test for preventing duplicates when users edit pending lunchflow transactions
* fix: normalise extra hash keys for pending detection
* test: Add tests for uncategorized filter across all locales
Adds two tests to catch the bug where filtering for "Uncategorized"
transactions fails when the user's locale is not English:
1. Tests that filtering with the locale-specific uncategorized name
works correctly in all SUPPORTED_LOCALES
2. Tests that filtering with the English "Uncategorized" parameter
works regardless of the current locale (catches the French bug)
https://claude.ai/code/session_01JcKj4776k5Es8Cscbm4kUo
* fix: Fix uncategorized filter for French, Catalan, and Dutch locales
The uncategorized filter was failing when the URL parameter contained
"Uncategorized" (English) but the user's locale was different. This
affected 3 locales with non-English translations:
- French: "Non catégorisé"
- Catalan: "Sense categoria"
- Dutch: "Ongecategoriseerd"
The fix adds Category.all_uncategorized_names which returns all possible
uncategorized name translations across supported locales, and updates
the search filter to check against all variants instead of just the
current locale's translation.
https://claude.ai/code/session_01JcKj4776k5Es8Cscbm4kUo
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Fix budget category totals to net refunds against expenses
Budget spending calculations now subtract refunds (negative transactions
classified as income) from expense totals in the same category. Previously,
refunds were excluded entirely, causing budgets to show gross spending
instead of net spending.
Fixes#314
* Handle missing git binary in commit_sha initializer
Rescues Errno::ENOENT when git is not installed, falling back to
BUILD_COMMIT_SHA env var or "unknown". Fixes crash in Docker
development containers that lack git.
* Revert "Handle missing git binary in commit_sha initializer"
This reverts commit 7e58458faa.
* Subtract uncategorized refunds from overall budget spending
Uncategorized refunds were not being netted against actual_spending
because the synthetic uncategorized category has no persisted ID and
wasn't matched by the budget_categories ID set. Now checks for
category.uncategorized? in addition to the ID lookup.
* perf: optimize budget category actual spending calculation
* feat: Add PDF import with AI-powered document analysis
This enhances the import functionality to support PDF files with AI-powered
document analysis. When a PDF is uploaded, it is processed by AI to:
- Identify the document type (bank statement, credit card statement, etc.)
- Generate a summary of the document contents
- Extract key metadata (institution, dates, balances, transaction count)
After processing, an email is sent to the user asking for next steps.
Key changes:
- Add PdfImport model for handling PDF document imports
- Add Provider::Openai::PdfProcessor for AI document analysis
- Add ProcessPdfJob for async PDF processing
- Add PdfImportMailer for user notification emails
- Update imports controller to detect and handle PDF uploads
- Add PDF import option to the new import page
- Add i18n translations for all new strings
- Add comprehensive tests for the new functionality
* Add bank statement import with AI extraction
- Create ImportBankStatement assistant function for MCP
- Add BankStatementExtractor with chunked processing for small context windows
- Register function in assistant configurable
- Make PdfImport#pdf_file_content public for extractor access
- Increase OpenAI request timeout to 600s for slow local models
- Increase DB connection pool to 20 for concurrent operations
Tested with M-Pesa bank statement via remote Ollama (qwen3:8b):
- Successfully extracted 18 transactions
- Generated CSV and created TransactionImport
- Works with 3000 char chunks for small context windows
* Add pdf-reader gem dependency
The BankStatementExtractor uses PDF::Reader to parse bank statement
PDFs, but the gem was not properly declared in the Gemfile. This would
cause NameError in production when processing bank statements.
Added pdf-reader ~> 2.12 to Gemfile dependencies.
* Fix transaction deduplication to preserve legitimate duplicates
The previous deduplication logic removed ALL duplicate transactions based
on [date, amount, name], which would drop legitimate same-day duplicates
like multiple ATM withdrawals or card authorizations.
Changed to only deduplicate transactions that appear in consecutive chunks
(chunking artifacts) while preserving all legitimate duplicates within the
same chunk or non-adjacent chunks.
* Refactor bank statement extraction to use public provider method
Address code review feedback:
- Add public extract_bank_statement method to Provider::Openai
- Remove direct access to private client via send(:client)
- Update ImportBankStatement to use new public method
- Add require 'set' to BankStatementExtractor
- Remove PII-sensitive content from error logs
- Add defensive check for nil response.error
- Handle oversized PDF pages in chunking logic
- Remove unused process_native and process_generic methods
- Update email copy to reflect feature availability
- Add guard for nil document_type in email template
- Document pdf-reader gem rationale in Gemfile
Tested with both OpenAI (gpt-4o) and Ollama (qwen3:8b):
- OpenAI: 49 transactions extracted in 30s
- Ollama: 40 transactions extracted in 368s
- All encapsulation and error handling working correctly
* Update schema.rb with ai_summary and document_type columns
* Address PR #808 review comments
- Rename :csv_file to :import_file across controllers/views/tests
- Add PDF test fixture (sample_bank_statement.pdf)
- Add supports_pdf_processing? method for graceful degradation
- Revert unrelated database.yml pool change (600->3)
- Remove month_start_day schema bleed from other PR
- Fix PdfProcessor: use .strip instead of .strip_heredoc
- Add server-side PDF magic byte validation
- Conditionally show PDF import option when AI provider available
- Fix ProcessPdfJob: sanitize errors, handle update failure
- Move pdf_file attachment from Import to PdfImport
- Document deduplication logic limitations
- Fix ImportBankStatement: catch specific exceptions only
- Remove unnecessary require 'set'
- Remove dead json_schema method from PdfProcessor
- Reduce default OpenAI timeout from 600s to 60s
- Fix nil guard in text mailer template
- Add require 'csv' to ImportBankStatement
- Remove Gemfile pdf-reader comment
* Fix RuboCop indentation in ProcessPdfJob
* Refactor PDF import check to use model predicate method
Replace is_a?(PdfImport) type check with requires_csv_workflow? predicate
that leverages STI inheritance for cleaner controller logic.
* Fix missing 'unknown' locale key and schema version mismatch
- Add 'unknown: Unknown Document' to document_types locale
- Fix schema version to match latest migration (2026_01_24_180211)
* Document OPENAI_REQUEST_TIMEOUT env variable
Added to .env.local.example and docs/hosting/ai.md
* Rename ALLOWED_MIME_TYPES to ALLOWED_CSV_MIME_TYPES for clarity
* Add comment explaining requires_csv_workflow? predicate
* Remove redundant required_column_keys from PdfImport
Base class already returns [] by default
* Add ENV toggle to disable PDF processing for non-vision endpoints
OPENAI_SUPPORTS_PDF_PROCESSING=false can be used for OpenAI-compatible
endpoints (e.g., Ollama) that don't support vision/PDF processing.
* Wire up transaction extraction for PDF bank statements
- Add extracted_data JSONB column to imports
- Add extract_transactions method to PdfImport
- Call extraction in ProcessPdfJob for bank statements
- Store transactions in extracted_data for later review
* Fix ProcessPdfJob retry logic, sanitize and localize errors
- Allow retries after partial success (classification ok, extraction failed)
- Log sanitized error message instead of raw message to avoid data leakage
- Use i18n for user-facing error messages
* Add vision-capable model validation for PDF processing
* Fix drag-and-drop test to use correct field name csv_file
* Schema bleedover from another branch
* Fix drag-drop import form field name to match controller
* Add vision capability guard to process_pdf method
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* Preserve existing demo data by default
Add SKIP_CLEAR environment variable to demo_data rake tasks.
Defaults to true (preserving existing data). Set SKIP_CLEAR=0
to wipe data before generating new demo data.
https://claude.ai/code/session_01GcoMc2SH3czPrbeGkHbmpE
* Add deterministic instatus.com API key for demo data
Create a read-only API key named "instatus.com" with a fixed value
when generating demo data. This allows uptime monitoring tools to
use a hardcoded API key that doesn't change between demo data runs.
The key is idempotent - if it already exists, it will be reused.
https://claude.ai/code/session_01GcoMc2SH3czPrbeGkHbmpE
* OK to name instatus to a point
* Remove all Instatus references
* Rename to create_monitoring_api_key! and scope lookup to admin_user
- Rename create_instatus_api_key! to create_monitoring_api_key! (snake_case)
- Scope API key lookup to admin_user instead of global ApiKey lookup
- Each family's admin now has their own monitoring API key
https://claude.ai/code/session_01GcoMc2SH3czPrbeGkHbmpE
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add warning for TwelveData plan-restricted tickers
Fixes#800
- Add Security::PlanRestrictionTracker concern using Rails cache
- Detect plan upgrade errors during Security::Price::Importer sync
- Display amber warning on /settings/hosting with affected tickers
- Include unit tests for the new functionality
* Scope plan restriction cache by provider
Addresses review feedback:
- Cache key now includes provider name to support multiple data providers
- Methods now require provider parameter for proper scoping
- Added tests for provider-scoped restrictions
- Added documentation explaining instance-level API key architecture
* Fix RuboCop array bracket spacing
* Fix empty array bracket spacing
* Move plan upgrade detection to Provider::TwelveData
* Fix provider scoping tests to use direct cache writes
---------
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
* fix: Handle uncategorized transactions filter correctly
When filtering for 'Uncategorized' transactions, the filter was not working
because 'Uncategorized' is a virtual category (Category.uncategorized returns
a non-persisted Category object) and does not exist in the database.
The filter was attempting to match 'categories.name IN (Uncategorized)' which
returned zero results.
This fix removes 'Uncategorized' from the category names array before querying
the database, allowing the existing 'category_id IS NULL' condition to work
correctly.
Fixes filtering for uncategorized transactions while maintaining backward
compatibility with all other category filters.
* test: Add comprehensive tests for Uncategorized filter
- Test filtering for only uncategorized transactions
- Test combining uncategorized with real categories
- Test excluding uncategorized when not in filter
- Ensures fix prevents regression
* refactor: Use Category.uncategorized.name for i18n support
- Replace hard-coded 'Uncategorized' string with Category.uncategorized.name
- Conditionally build SQL query based on include_uncategorized flag
- Avoid adding category_id IS NULL clause when not needed
- Update tests to use Category.uncategorized.name for consistency
- Cleaner logic: only include uncategorized condition when requested
Addresses code review feedback on i18n support and query optimization.
* test: Fix travel category fixture error
Create travel category dynamically instead of using non-existent fixture
* style: Fix rubocop spacing in array brackets
---------
Co-authored-by: Charsel <charsel@charsel.com>
* Add transaction type condition filter for rules
Add ability to filter rules by transaction type (income, expense, transfer).
This allows users to create rules that differentiate between transactions
with the same name but different types.
- Add Rule::ConditionFilter::TransactionType with select dropdown
- Register in TransactionResource condition_filters
- Add tests for income, expense, and transfer filtering
Closes#373
* Address PR review feedback for transaction type filter
- Fix income filter to exclude transfers and investment_contribution
- Fix expense filter to include investment_contribution regardless of sign
- Add i18n for option and operator labels
- Add tests for edge cases (transfer inflows, investment contributions)
Logic now matches Transaction::Search#apply_type_filter for consistency.
- Move family-scoped queries to models via family_scope class method
- Remove hardcoded model names from Enrichable concern
- Replace inline rescue with proper respond_to? check
- Add count tracking for better logging
- Preserve user locks: Only unlock attributes where current value still matches
what AI set. If user changed the value, they took ownership.
- Add nil guard clause for family parameter in ClearAiCacheJob
- Add partial failure handling so one model's failure doesn't block the other
Add ClearAiCacheJob for async cache clearing with low priority. Extend Enrichable concern with clear_ai_cache methods to unlock AI-enriched attributes and delete AI enrichment records. Trigger automatic cache clearing when OpenAI model setting changes.
* Use Accept-Language for unauthenticated locale
* Add per-user locale overrides
* Fix test
* Use more than the top `accept-language` entry
* Localization of string
* feat: add protection indicator to entries and unlock functionality
- Introduced protection indicator component rendering on hover and in detail views.
- Added support to unlock entries, clearing protection flags (`user_modified`, `import_locked`, and locked attributes).
- Updated routes, controllers, and models to enable unlock functionality for trades and transactions.
- Refactored views and localized content to support the new feature.
- Added relevant tests for unlocking functionality and attribute handling.
* feat: improve sync protection and turbo stream updates for entries
- Added tests for turbo stream updates reflecting protection indicators.
- Ensured user-modified entries lock specific attributes to prevent overwrites.
- Updated controllers to mark entries as user-modified and reload for accurate rendering.
- Enhanced protection indicator rendering using turbo frames.
- Applied consistent lock state handling across trades and transactions.
* Address PR review comments for protection indicator
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* refactor: rename `raw_investments_payload` to `raw_holdings_payload`
- Update references and models to use consistent naming.
- Adjust migrations, tests, and encryption setup accordingly.
* fix: improve safety when accessing raw_holdings_payload keys
- Use `dig` with safe navigation to prevent potential nil errors.
- Add support for decryption from the old column name `raw_investments_payload`.
- Adjust related methods and calculations for consistency.
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* Initial sec
* Update PII fields
* FIX add tests
* FIX safely read plaintext data on rake backfill
* Update user.rb
* FIX tests
* encryption_ready? block
* Test conditional to encryption on
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
