* Fix budget category totals to net refunds against expenses
Budget spending calculations now subtract refunds (negative transactions
classified as income) from expense totals in the same category. Previously,
refunds were excluded entirely, causing budgets to show gross spending
instead of net spending.
Fixes#314
* Handle missing git binary in commit_sha initializer
Rescues Errno::ENOENT when git is not installed, falling back to
BUILD_COMMIT_SHA env var or "unknown". Fixes crash in Docker
development containers that lack git.
* Revert "Handle missing git binary in commit_sha initializer"
This reverts commit 7e58458faa.
* Subtract uncategorized refunds from overall budget spending
Uncategorized refunds were not being netted against actual_spending
because the synthetic uncategorized category has no persisted ID and
wasn't matched by the budget_categories ID set. Now checks for
category.uncategorized? in addition to the ID lookup.
* perf: optimize budget category actual spending calculation
* feat: Add PDF import with AI-powered document analysis
This enhances the import functionality to support PDF files with AI-powered
document analysis. When a PDF is uploaded, it is processed by AI to:
- Identify the document type (bank statement, credit card statement, etc.)
- Generate a summary of the document contents
- Extract key metadata (institution, dates, balances, transaction count)
After processing, an email is sent to the user asking for next steps.
Key changes:
- Add PdfImport model for handling PDF document imports
- Add Provider::Openai::PdfProcessor for AI document analysis
- Add ProcessPdfJob for async PDF processing
- Add PdfImportMailer for user notification emails
- Update imports controller to detect and handle PDF uploads
- Add PDF import option to the new import page
- Add i18n translations for all new strings
- Add comprehensive tests for the new functionality
* Add bank statement import with AI extraction
- Create ImportBankStatement assistant function for MCP
- Add BankStatementExtractor with chunked processing for small context windows
- Register function in assistant configurable
- Make PdfImport#pdf_file_content public for extractor access
- Increase OpenAI request timeout to 600s for slow local models
- Increase DB connection pool to 20 for concurrent operations
Tested with M-Pesa bank statement via remote Ollama (qwen3:8b):
- Successfully extracted 18 transactions
- Generated CSV and created TransactionImport
- Works with 3000 char chunks for small context windows
* Add pdf-reader gem dependency
The BankStatementExtractor uses PDF::Reader to parse bank statement
PDFs, but the gem was not properly declared in the Gemfile. This would
cause NameError in production when processing bank statements.
Added pdf-reader ~> 2.12 to Gemfile dependencies.
* Fix transaction deduplication to preserve legitimate duplicates
The previous deduplication logic removed ALL duplicate transactions based
on [date, amount, name], which would drop legitimate same-day duplicates
like multiple ATM withdrawals or card authorizations.
Changed to only deduplicate transactions that appear in consecutive chunks
(chunking artifacts) while preserving all legitimate duplicates within the
same chunk or non-adjacent chunks.
* Refactor bank statement extraction to use public provider method
Address code review feedback:
- Add public extract_bank_statement method to Provider::Openai
- Remove direct access to private client via send(:client)
- Update ImportBankStatement to use new public method
- Add require 'set' to BankStatementExtractor
- Remove PII-sensitive content from error logs
- Add defensive check for nil response.error
- Handle oversized PDF pages in chunking logic
- Remove unused process_native and process_generic methods
- Update email copy to reflect feature availability
- Add guard for nil document_type in email template
- Document pdf-reader gem rationale in Gemfile
Tested with both OpenAI (gpt-4o) and Ollama (qwen3:8b):
- OpenAI: 49 transactions extracted in 30s
- Ollama: 40 transactions extracted in 368s
- All encapsulation and error handling working correctly
* Update schema.rb with ai_summary and document_type columns
* Address PR #808 review comments
- Rename :csv_file to :import_file across controllers/views/tests
- Add PDF test fixture (sample_bank_statement.pdf)
- Add supports_pdf_processing? method for graceful degradation
- Revert unrelated database.yml pool change (600->3)
- Remove month_start_day schema bleed from other PR
- Fix PdfProcessor: use .strip instead of .strip_heredoc
- Add server-side PDF magic byte validation
- Conditionally show PDF import option when AI provider available
- Fix ProcessPdfJob: sanitize errors, handle update failure
- Move pdf_file attachment from Import to PdfImport
- Document deduplication logic limitations
- Fix ImportBankStatement: catch specific exceptions only
- Remove unnecessary require 'set'
- Remove dead json_schema method from PdfProcessor
- Reduce default OpenAI timeout from 600s to 60s
- Fix nil guard in text mailer template
- Add require 'csv' to ImportBankStatement
- Remove Gemfile pdf-reader comment
* Fix RuboCop indentation in ProcessPdfJob
* Refactor PDF import check to use model predicate method
Replace is_a?(PdfImport) type check with requires_csv_workflow? predicate
that leverages STI inheritance for cleaner controller logic.
* Fix missing 'unknown' locale key and schema version mismatch
- Add 'unknown: Unknown Document' to document_types locale
- Fix schema version to match latest migration (2026_01_24_180211)
* Document OPENAI_REQUEST_TIMEOUT env variable
Added to .env.local.example and docs/hosting/ai.md
* Rename ALLOWED_MIME_TYPES to ALLOWED_CSV_MIME_TYPES for clarity
* Add comment explaining requires_csv_workflow? predicate
* Remove redundant required_column_keys from PdfImport
Base class already returns [] by default
* Add ENV toggle to disable PDF processing for non-vision endpoints
OPENAI_SUPPORTS_PDF_PROCESSING=false can be used for OpenAI-compatible
endpoints (e.g., Ollama) that don't support vision/PDF processing.
* Wire up transaction extraction for PDF bank statements
- Add extracted_data JSONB column to imports
- Add extract_transactions method to PdfImport
- Call extraction in ProcessPdfJob for bank statements
- Store transactions in extracted_data for later review
* Fix ProcessPdfJob retry logic, sanitize and localize errors
- Allow retries after partial success (classification ok, extraction failed)
- Log sanitized error message instead of raw message to avoid data leakage
- Use i18n for user-facing error messages
* Add vision-capable model validation for PDF processing
* Fix drag-and-drop test to use correct field name csv_file
* Schema bleedover from another branch
* Fix drag-drop import form field name to match controller
* Add vision capability guard to process_pdf method
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* feat: add valuations API endpoints for managing account reconciliations
* refactor: formatting
* fix: make account extraction clearer
* feat: validation and error handling improvements
* feat: transaction
* feat: error handling
* Add API documentation LLM context
* Make it easier for people
* feat: transaction in creation
* feat: add OpenAPI spec for Valuations API
* fix: update notes validation to check for key presence
* Prevent double render
* All other docs use `apiKeyAuth`
* More `apiKeyAuth`
* Remove testing assertions from API doc specs
* fix: correct valuation entry references
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Preserve existing demo data by default
Add SKIP_CLEAR environment variable to demo_data rake tasks.
Defaults to true (preserving existing data). Set SKIP_CLEAR=0
to wipe data before generating new demo data.
https://claude.ai/code/session_01GcoMc2SH3czPrbeGkHbmpE
* Add deterministic instatus.com API key for demo data
Create a read-only API key named "instatus.com" with a fixed value
when generating demo data. This allows uptime monitoring tools to
use a hardcoded API key that doesn't change between demo data runs.
The key is idempotent - if it already exists, it will be reused.
https://claude.ai/code/session_01GcoMc2SH3czPrbeGkHbmpE
* OK to name instatus to a point
* Remove all Instatus references
* Rename to create_monitoring_api_key! and scope lookup to admin_user
- Rename create_instatus_api_key! to create_monitoring_api_key! (snake_case)
- Scope API key lookup to admin_user instead of global ApiKey lookup
- Each family's admin now has their own monitoring API key
https://claude.ai/code/session_01GcoMc2SH3czPrbeGkHbmpE
---------
Co-authored-by: Claude <noreply@anthropic.com>
* pwa(cleanup): enforce LF, head meta + icons, manifest orientation, remove static webmanifest
* pwa(cleanup): add gitattributes, head meta/icons, manifest orientation; remove static manifest; small nav & dashboard fixes
* pwa(cleanup): improve transaction drawer header layout with inline close button
* fix: address PR review feedback
- Add dom_id to transaction header for Turbo Stream updates (Codex)
- Add pending badge next to date when transaction is pending (CodeRabbit)
- Make close button keyboard-focusable by removing tabindex=-1 (CodeRabbit)
- Fix settings nav horizontal scroll with flex-nowrap space-x-1 (CodeRabbit)
* fix: localize 'Linked with Plaid' tooltip string (CodeRabbit)
* Update .gitattributes
Better comment smh
* fix: align transaction/transfer dialog icons and update transfer drawer pattern
- Fix icon alignment in transaction header (items-center instead of items-start)
- Make transfer/linked icons consistent size and color
- Update transfers/show.html.erb to use frame: drawer with hide_close_icon pattern
- Match transfer dialog header layout with transaction details
* fix: enhance header layout
This in the transaction and transfer views, with consistent icon placement
* fix: remove fixed height from HTML document class
basically a regression issue pretty sure
* fix: update dialog rendering to use 'frame' and hide close icon in headers
* fix: update transaction type tabs layout for improved responsiveness
* fix: conditionally render transaction type tabs based on account type
* Add warning for TwelveData plan-restricted tickers
Fixes#800
- Add Security::PlanRestrictionTracker concern using Rails cache
- Detect plan upgrade errors during Security::Price::Importer sync
- Display amber warning on /settings/hosting with affected tickers
- Include unit tests for the new functionality
* Scope plan restriction cache by provider
Addresses review feedback:
- Cache key now includes provider name to support multiple data providers
- Methods now require provider parameter for proper scoping
- Added tests for provider-scoped restrictions
- Added documentation explaining instance-level API key architecture
* Fix RuboCop array bracket spacing
* Fix empty array bracket spacing
* Move plan upgrade detection to Provider::TwelveData
* Fix provider scoping tests to use direct cache writes
---------
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
* feat: Move upcoming transactions in a dedicated tab
* Adjust formatting
* feat: adjust visibility on mobile
* feat: change translation label
* feat: show only upcoming transactions expected in the next 10 days
* feat: show upcoming transactions tab only when option enabled
* feat: render empty partial when there are no recurring transactions
* feat: align icon sizing and spacing between transactions and upcoming sections
* feat: add missing localitazion labels
* fix: move filter on upcoming transactions in controller
* fix: add missing localitazion labels
* fix: Handle uncategorized transactions filter correctly
When filtering for 'Uncategorized' transactions, the filter was not working
because 'Uncategorized' is a virtual category (Category.uncategorized returns
a non-persisted Category object) and does not exist in the database.
The filter was attempting to match 'categories.name IN (Uncategorized)' which
returned zero results.
This fix removes 'Uncategorized' from the category names array before querying
the database, allowing the existing 'category_id IS NULL' condition to work
correctly.
Fixes filtering for uncategorized transactions while maintaining backward
compatibility with all other category filters.
* test: Add comprehensive tests for Uncategorized filter
- Test filtering for only uncategorized transactions
- Test combining uncategorized with real categories
- Test excluding uncategorized when not in filter
- Ensures fix prevents regression
* refactor: Use Category.uncategorized.name for i18n support
- Replace hard-coded 'Uncategorized' string with Category.uncategorized.name
- Conditionally build SQL query based on include_uncategorized flag
- Avoid adding category_id IS NULL clause when not needed
- Update tests to use Category.uncategorized.name for consistency
- Cleaner logic: only include uncategorized condition when requested
Addresses code review feedback on i18n support and query optimization.
* test: Fix travel category fixture error
Create travel category dynamically instead of using non-existent fixture
* style: Fix rubocop spacing in array brackets
---------
Co-authored-by: Charsel <charsel@charsel.com>
* Add transaction type condition filter for rules
Add ability to filter rules by transaction type (income, expense, transfer).
This allows users to create rules that differentiate between transactions
with the same name but different types.
- Add Rule::ConditionFilter::TransactionType with select dropdown
- Register in TransactionResource condition_filters
- Add tests for income, expense, and transfer filtering
Closes#373
* Address PR review feedback for transaction type filter
- Fix income filter to exclude transfers and investment_contribution
- Fix expense filter to include investment_contribution regardless of sign
- Add i18n for option and operator labels
- Add tests for edge cases (transfer inflows, investment contributions)
Logic now matches Transaction::Search#apply_type_filter for consistency.
- Move family-scoped queries to models via family_scope class method
- Remove hardcoded model names from Enrichable concern
- Replace inline rescue with proper respond_to? check
- Add count tracking for better logging
- Preserve user locks: Only unlock attributes where current value still matches
what AI set. If user changed the value, they took ownership.
- Add nil guard clause for family parameter in ClearAiCacheJob
- Add partial failure handling so one model's failure doesn't block the other
Add ClearAiCacheJob for async cache clearing with low priority. Extend Enrichable concern with clear_ai_cache methods to unlock AI-enriched attributes and delete AI enrichment records. Trigger automatic cache clearing when OpenAI model setting changes.
* Use Accept-Language for unauthenticated locale
* Add per-user locale overrides
* Fix test
* Use more than the top `accept-language` entry
* Localization of string
* feat: add protection indicator to entries and unlock functionality
- Introduced protection indicator component rendering on hover and in detail views.
- Added support to unlock entries, clearing protection flags (`user_modified`, `import_locked`, and locked attributes).
- Updated routes, controllers, and models to enable unlock functionality for trades and transactions.
- Refactored views and localized content to support the new feature.
- Added relevant tests for unlocking functionality and attribute handling.
* feat: improve sync protection and turbo stream updates for entries
- Added tests for turbo stream updates reflecting protection indicators.
- Ensured user-modified entries lock specific attributes to prevent overwrites.
- Updated controllers to mark entries as user-modified and reload for accurate rendering.
- Enhanced protection indicator rendering using turbo frames.
- Applied consistent lock state handling across trades and transactions.
* Address PR review comments for protection indicator
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* feat: display shared budget as a badge
* fix: reduce margin on badge
* fix: adjust mobile spacing
* Consistent opacity
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
---------
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* refactor: rename `raw_investments_payload` to `raw_holdings_payload`
- Update references and models to use consistent naming.
- Adjust migrations, tests, and encryption setup accordingly.
* fix: improve safety when accessing raw_holdings_payload keys
- Use `dig` with safe navigation to prevent potential nil errors.
- Add support for decryption from the old column name `raw_investments_payload`.
- Adjust related methods and calculations for consistency.
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* feat: add auto-open functionality for collapsible sections and streamline unlinked account handling
- Introduce `auto-open` Stimulus controller to auto-expand <details> elements based on URL params.
- Update all settings sections and panels to support the new `auto_open_param` for seamless navigation.
- Improve unlinked account logic for Coinbase, SimpleFIN, and SnapTrade, ensuring consistent and optimized handling.
- Refactor sync warnings and badges for better readability and user experience.
- Extend localization for additional menu items, warnings, and setup prompts.
* fix: improve error handling and safe HTML usage in Coinbase and settings components
- Log warning for unhandled exceptions in Coinbase unlinked account count fallback.
- Escape `auto_open_param` in settings section for safe HTML injection.
- Clean up URL params in `auto-open` controller after auto-expansion.
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* fix: Enable scrolling on login page for small mobile devices
Changed auth layout containers from `h-full` to `min-h-full` and added
`overflow-y-auto` to allow content to scroll when it exceeds viewport
height on small mobile phones.
* Real fix
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Initial sec
* Update PII fields
* FIX add tests
* FIX safely read plaintext data on rake backfill
* Update user.rb
* FIX tests
* encryption_ready? block
* Test conditional to encryption on
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* fix: Handle missing encryption keys gracefully on providers page
When Active Record encryption keys are not configured, the settings/providers
page would crash with an unhandled exception. This change catches the
ActiveRecord::Encryption::Errors::Configuration error and displays a
friendly error message instead, explaining that encryption credentials
need to be configured before using sync providers.
https://claude.ai/code/session_015nPsLWkr12i5ok5bwLtA7p
* Simplify rescue block
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: Show cancellation message when subscription is pending cancellation
When a subscription is cancelled via Stripe, the UI incorrectly showed
"Your contribution continues on..." instead of reflecting the cancellation
status. This fix adds tracking of `cancel_at_period_end` from Stripe webhooks
and displays "Your contribution ends on..." when a subscription has been
cancelled but is still active until the billing period ends.
https://claude.ai/code/session_01Y8ELTdK1k9o315iSq43TRN
* chore: Update schema.rb with cancel_at_period_end column
https://claude.ai/code/session_01Y8ELTdK1k9o315iSq43TRN
* Schema version
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add security remapping support to holdings
- Introduced `provider_security` tracking for holdings with schema updates.
- Implemented security remap/reset workflows in `Holding` model and UI.
- Updated routes, controllers, and tests to support new functionality.
- Enhanced client-side interaction with Stimulus controller for remapping.
# Conflicts:
# app/components/UI/account/activity_feed.html.erb
# db/schema.rb
* Refactor "New transaction" to "New activity" across UI and tests
- Updated localized strings, button labels, and ARIA attributes.
- Improved error handling in holdings' current price display.
- Scoped fallback queries in `provider_import_adapter` to prevent overwrites.
- Added safeguard for offline securities in price fetching logic.
* Update security remapping to merge holdings on collision by deleting duplicates
- Removed error handling for collisions in `remap_security!`.
- Added logic to merge holdings by deleting duplicates on conflicting dates.
- Modified associated test to validate merging behavior.
* Update security remapping to merge holdings on collision by combining qty and amount
- Modified `remap_security!` to merge holdings by summing `qty` and `amount` on conflicting dates.
- Adjusted logic to calculate `price` for merged holdings.
- Updated test to validate new merge behavior.
* Improve DOM handling in Turbo redirect action & enhance holdings merge logic
- Updated Turbo's custom `redirect` action to use the "replace" option for cleaner DOM updates without clearing the cache.
- Enhanced holdings merge logic to calculate weighted average cost basis during security remapping, ensuring more accurate cost_basis updates.
* Track provider_security_id during security updates to support reset workflows
* Fix provider tracking: guard nil ticker lookups and preserve merge attrs
- Guard fallback 1b lookup when security.ticker is blank to avoid matching NULL tickers
- Preserve external_id, provider_security_id, account_provider_id during collision merge
* Fix schema.rb version after merge (includes tax_treatment migration)
* fix: Rename migration to run after schema version
The migration 20260117000001 was skipped in CI because it had a timestamp
earlier than the schema version (2026_01_17_200000). CI loads schema.rb
directly and only runs migrations with versions after the schema version.
Renamed to 20260119000001 so it runs correctly.
* Update schema: remove Coinbase tables, add new fields and indexes
* Update schema: add back `tax_treatment` field with default value "taxable"
* Improve Turbo redirect action: use "replace" to avoid form submission in history
* Lock merged holdings to prevent provider overwrites and fix activity feed template indentation
* Refactor holdings transfer logic: enforce currency checks during collisions and enhance merge handling
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Hide payment contribution options from demo and manually created users
Demo data users and manually created users don't have stripe_customer_id
set on their family, so they should not see payment/contribution options.
Changes:
- Add can_manage_subscription? method to Family::Subscribeable that checks
for presence of stripe_customer_id
- Guard Settings::PaymentsController to return 403 for users without
stripe_customer_id
- Guard SubscriptionsController#show action (Stripe portal redirect) for
users without stripe_customer_id
- Update settings navigation to hide the payment link when
stripe_customer_id is not present
- Add tests for the new behavior
* Fix broken test
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix: replace invalid redirect("about:blank") with proper controller actions
The privacy and terms routes were using redirect("about:blank") which is
invalid because about:blank is a browser-specific pseudo URL, not a valid
HTTP redirect target. This fix replaces them with proper controller actions
that render placeholder pages.
Changes:
- Add privacy and terms actions to PagesController with skip_authentication
- Create privacy.html.erb and terms.html.erb view templates
- Add i18n translations for the new pages
- Update routes to use pages#privacy and pages#terms
https://claude.ai/code/session_01RL36dMda1o6LXGsnGnTJZu
* Make legal routes configurable
---------
Co-authored-by: Claude <noreply@anthropic.com>
Wrap export list in turbo_frame_tag with conditional polling attributes.
When exports are pending/processing, page polls every 3 seconds for updates.
Add turbo_frame: _top to download/delete buttons for proper frame handling.
* Add files via upload
Signed-off-by: Jose <39016041+jospaquim@users.noreply.github.com>
* Add merchants and tags resources to routes
Signed-off-by: Jose <39016041+jospaquim@users.noreply.github.com>
* update
* update spaces
* fix: Apply CodeRabbit suggestions and add YARD documentation
* docs: Add API documentation for merchants and tags endpoints
* fix: Address CodeRabbit feedback on documentation
* fix: Use authorize_scope! instead of ensure_read_scope
* test(api): Add request specs for merchants and tags endpoints
* test(api): Add request specs for merchants and tags endpoints
* test(api): Convert specs to Minitest format in test/
* fix: Correct indentation for private methods
* fix: merchant and tag test
* Enhance tag tests for family scope and access
Added tests to ensure tags from other families are not returned and that attempts to access them return 404.
Signed-off-by: Jose <39016041+jospaquim@users.noreply.github.com>
* Enhance merchants controller tests for family scope
Added tests to ensure that merchants from other families are not returned in the index action and that accessing a merchant from another family returns a 404 error.
Signed-off-by: Jose <39016041+jospaquim@users.noreply.github.com>
* Fix test/implementation
* Remove old token test code
* Improve test
---------
Signed-off-by: Jose <39016041+jospaquim@users.noreply.github.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* feat: implement expandable view for cashflow sankey chart
* refactor: migrate cashflow dialog sizing to tailwind utilities
* refactor: declarative draggable restore on cashflow dialog close
* refactor: localized title and use Tailwind utilities
* refactor: update dialog interaction especially on mobile
* refactor: add global expand text to localization
* fix: restore draggable immediately after dialog close
* Whitespace noise
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Add SnapTrade connection management with lazy-loading and deletion functionality.
* Refactor lazy-load controller to simplify event handling and enhance loading state management; improve SnapTrade deletion logic with additional safeguards and logging.
* Improve SnapTrade connection error handling and centralize unknown brokerage message using i18n.
* Centralize SnapTrade connection default name and missing authorization ID messages using i18n.
* Enhance SnapTrade connection deletion logic with improved error handling, i18n support for API deletion failures, and consistent Turbo Stream responses.
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* Expand option activity type support in `activities_processor` and update i18n hint for SnapTrade account loading.
* Add `SELL_SIDE_TYPES` constant to streamline sell-side activity handling in `activities_processor`.
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
