* Fix budget category totals to net refunds against expenses
Budget spending calculations now subtract refunds (negative transactions
classified as income) from expense totals in the same category. Previously,
refunds were excluded entirely, causing budgets to show gross spending
instead of net spending.
Fixes#314
* Handle missing git binary in commit_sha initializer
Rescues Errno::ENOENT when git is not installed, falling back to
BUILD_COMMIT_SHA env var or "unknown". Fixes crash in Docker
development containers that lack git.
* Revert "Handle missing git binary in commit_sha initializer"
This reverts commit 7e58458faa.
* Subtract uncategorized refunds from overall budget spending
Uncategorized refunds were not being netted against actual_spending
because the synthetic uncategorized category has no persisted ID and
wasn't matched by the budget_categories ID set. Now checks for
category.uncategorized? in addition to the ID lookup.
* perf: optimize budget category actual spending calculation
* feat: Add PDF import with AI-powered document analysis
This enhances the import functionality to support PDF files with AI-powered
document analysis. When a PDF is uploaded, it is processed by AI to:
- Identify the document type (bank statement, credit card statement, etc.)
- Generate a summary of the document contents
- Extract key metadata (institution, dates, balances, transaction count)
After processing, an email is sent to the user asking for next steps.
Key changes:
- Add PdfImport model for handling PDF document imports
- Add Provider::Openai::PdfProcessor for AI document analysis
- Add ProcessPdfJob for async PDF processing
- Add PdfImportMailer for user notification emails
- Update imports controller to detect and handle PDF uploads
- Add PDF import option to the new import page
- Add i18n translations for all new strings
- Add comprehensive tests for the new functionality
* Add bank statement import with AI extraction
- Create ImportBankStatement assistant function for MCP
- Add BankStatementExtractor with chunked processing for small context windows
- Register function in assistant configurable
- Make PdfImport#pdf_file_content public for extractor access
- Increase OpenAI request timeout to 600s for slow local models
- Increase DB connection pool to 20 for concurrent operations
Tested with M-Pesa bank statement via remote Ollama (qwen3:8b):
- Successfully extracted 18 transactions
- Generated CSV and created TransactionImport
- Works with 3000 char chunks for small context windows
* Add pdf-reader gem dependency
The BankStatementExtractor uses PDF::Reader to parse bank statement
PDFs, but the gem was not properly declared in the Gemfile. This would
cause NameError in production when processing bank statements.
Added pdf-reader ~> 2.12 to Gemfile dependencies.
* Fix transaction deduplication to preserve legitimate duplicates
The previous deduplication logic removed ALL duplicate transactions based
on [date, amount, name], which would drop legitimate same-day duplicates
like multiple ATM withdrawals or card authorizations.
Changed to only deduplicate transactions that appear in consecutive chunks
(chunking artifacts) while preserving all legitimate duplicates within the
same chunk or non-adjacent chunks.
* Refactor bank statement extraction to use public provider method
Address code review feedback:
- Add public extract_bank_statement method to Provider::Openai
- Remove direct access to private client via send(:client)
- Update ImportBankStatement to use new public method
- Add require 'set' to BankStatementExtractor
- Remove PII-sensitive content from error logs
- Add defensive check for nil response.error
- Handle oversized PDF pages in chunking logic
- Remove unused process_native and process_generic methods
- Update email copy to reflect feature availability
- Add guard for nil document_type in email template
- Document pdf-reader gem rationale in Gemfile
Tested with both OpenAI (gpt-4o) and Ollama (qwen3:8b):
- OpenAI: 49 transactions extracted in 30s
- Ollama: 40 transactions extracted in 368s
- All encapsulation and error handling working correctly
* Update schema.rb with ai_summary and document_type columns
* Address PR #808 review comments
- Rename :csv_file to :import_file across controllers/views/tests
- Add PDF test fixture (sample_bank_statement.pdf)
- Add supports_pdf_processing? method for graceful degradation
- Revert unrelated database.yml pool change (600->3)
- Remove month_start_day schema bleed from other PR
- Fix PdfProcessor: use .strip instead of .strip_heredoc
- Add server-side PDF magic byte validation
- Conditionally show PDF import option when AI provider available
- Fix ProcessPdfJob: sanitize errors, handle update failure
- Move pdf_file attachment from Import to PdfImport
- Document deduplication logic limitations
- Fix ImportBankStatement: catch specific exceptions only
- Remove unnecessary require 'set'
- Remove dead json_schema method from PdfProcessor
- Reduce default OpenAI timeout from 600s to 60s
- Fix nil guard in text mailer template
- Add require 'csv' to ImportBankStatement
- Remove Gemfile pdf-reader comment
* Fix RuboCop indentation in ProcessPdfJob
* Refactor PDF import check to use model predicate method
Replace is_a?(PdfImport) type check with requires_csv_workflow? predicate
that leverages STI inheritance for cleaner controller logic.
* Fix missing 'unknown' locale key and schema version mismatch
- Add 'unknown: Unknown Document' to document_types locale
- Fix schema version to match latest migration (2026_01_24_180211)
* Document OPENAI_REQUEST_TIMEOUT env variable
Added to .env.local.example and docs/hosting/ai.md
* Rename ALLOWED_MIME_TYPES to ALLOWED_CSV_MIME_TYPES for clarity
* Add comment explaining requires_csv_workflow? predicate
* Remove redundant required_column_keys from PdfImport
Base class already returns [] by default
* Add ENV toggle to disable PDF processing for non-vision endpoints
OPENAI_SUPPORTS_PDF_PROCESSING=false can be used for OpenAI-compatible
endpoints (e.g., Ollama) that don't support vision/PDF processing.
* Wire up transaction extraction for PDF bank statements
- Add extracted_data JSONB column to imports
- Add extract_transactions method to PdfImport
- Call extraction in ProcessPdfJob for bank statements
- Store transactions in extracted_data for later review
* Fix ProcessPdfJob retry logic, sanitize and localize errors
- Allow retries after partial success (classification ok, extraction failed)
- Log sanitized error message instead of raw message to avoid data leakage
- Use i18n for user-facing error messages
* Add vision-capable model validation for PDF processing
* Fix drag-and-drop test to use correct field name csv_file
* Schema bleedover from another branch
* Fix drag-drop import form field name to match controller
* Add vision capability guard to process_pdf method
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* Add warning for TwelveData plan-restricted tickers
Fixes#800
- Add Security::PlanRestrictionTracker concern using Rails cache
- Detect plan upgrade errors during Security::Price::Importer sync
- Display amber warning on /settings/hosting with affected tickers
- Include unit tests for the new functionality
* Scope plan restriction cache by provider
Addresses review feedback:
- Cache key now includes provider name to support multiple data providers
- Methods now require provider parameter for proper scoping
- Added tests for provider-scoped restrictions
- Added documentation explaining instance-level API key architecture
* Fix RuboCop array bracket spacing
* Fix empty array bracket spacing
* Move plan upgrade detection to Provider::TwelveData
* Fix provider scoping tests to use direct cache writes
---------
Co-authored-by: mkdev11 <jaysmth689+github@users.noreply.github.com>
* fix: Handle uncategorized transactions filter correctly
When filtering for 'Uncategorized' transactions, the filter was not working
because 'Uncategorized' is a virtual category (Category.uncategorized returns
a non-persisted Category object) and does not exist in the database.
The filter was attempting to match 'categories.name IN (Uncategorized)' which
returned zero results.
This fix removes 'Uncategorized' from the category names array before querying
the database, allowing the existing 'category_id IS NULL' condition to work
correctly.
Fixes filtering for uncategorized transactions while maintaining backward
compatibility with all other category filters.
* test: Add comprehensive tests for Uncategorized filter
- Test filtering for only uncategorized transactions
- Test combining uncategorized with real categories
- Test excluding uncategorized when not in filter
- Ensures fix prevents regression
* refactor: Use Category.uncategorized.name for i18n support
- Replace hard-coded 'Uncategorized' string with Category.uncategorized.name
- Conditionally build SQL query based on include_uncategorized flag
- Avoid adding category_id IS NULL clause when not needed
- Update tests to use Category.uncategorized.name for consistency
- Cleaner logic: only include uncategorized condition when requested
Addresses code review feedback on i18n support and query optimization.
* test: Fix travel category fixture error
Create travel category dynamically instead of using non-existent fixture
* style: Fix rubocop spacing in array brackets
---------
Co-authored-by: Charsel <charsel@charsel.com>
* Add transaction type condition filter for rules
Add ability to filter rules by transaction type (income, expense, transfer).
This allows users to create rules that differentiate between transactions
with the same name but different types.
- Add Rule::ConditionFilter::TransactionType with select dropdown
- Register in TransactionResource condition_filters
- Add tests for income, expense, and transfer filtering
Closes#373
* Address PR review feedback for transaction type filter
- Fix income filter to exclude transfers and investment_contribution
- Fix expense filter to include investment_contribution regardless of sign
- Add i18n for option and operator labels
- Add tests for edge cases (transfer inflows, investment contributions)
Logic now matches Transaction::Search#apply_type_filter for consistency.
* refactor: rename `raw_investments_payload` to `raw_holdings_payload`
- Update references and models to use consistent naming.
- Adjust migrations, tests, and encryption setup accordingly.
* fix: improve safety when accessing raw_holdings_payload keys
- Use `dig` with safe navigation to prevent potential nil errors.
- Add support for decryption from the old column name `raw_investments_payload`.
- Adjust related methods and calculations for consistency.
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* Initial sec
* Update PII fields
* FIX add tests
* FIX safely read plaintext data on rake backfill
* Update user.rb
* FIX tests
* encryption_ready? block
* Test conditional to encryption on
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* fix: Show cancellation message when subscription is pending cancellation
When a subscription is cancelled via Stripe, the UI incorrectly showed
"Your contribution continues on..." instead of reflecting the cancellation
status. This fix adds tracking of `cancel_at_period_end` from Stripe webhooks
and displays "Your contribution ends on..." when a subscription has been
cancelled but is still active until the billing period ends.
https://claude.ai/code/session_01Y8ELTdK1k9o315iSq43TRN
* chore: Update schema.rb with cancel_at_period_end column
https://claude.ai/code/session_01Y8ELTdK1k9o315iSq43TRN
* Schema version
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add security remapping support to holdings
- Introduced `provider_security` tracking for holdings with schema updates.
- Implemented security remap/reset workflows in `Holding` model and UI.
- Updated routes, controllers, and tests to support new functionality.
- Enhanced client-side interaction with Stimulus controller for remapping.
# Conflicts:
# app/components/UI/account/activity_feed.html.erb
# db/schema.rb
* Refactor "New transaction" to "New activity" across UI and tests
- Updated localized strings, button labels, and ARIA attributes.
- Improved error handling in holdings' current price display.
- Scoped fallback queries in `provider_import_adapter` to prevent overwrites.
- Added safeguard for offline securities in price fetching logic.
* Update security remapping to merge holdings on collision by deleting duplicates
- Removed error handling for collisions in `remap_security!`.
- Added logic to merge holdings by deleting duplicates on conflicting dates.
- Modified associated test to validate merging behavior.
* Update security remapping to merge holdings on collision by combining qty and amount
- Modified `remap_security!` to merge holdings by summing `qty` and `amount` on conflicting dates.
- Adjusted logic to calculate `price` for merged holdings.
- Updated test to validate new merge behavior.
* Improve DOM handling in Turbo redirect action & enhance holdings merge logic
- Updated Turbo's custom `redirect` action to use the "replace" option for cleaner DOM updates without clearing the cache.
- Enhanced holdings merge logic to calculate weighted average cost basis during security remapping, ensuring more accurate cost_basis updates.
* Track provider_security_id during security updates to support reset workflows
* Fix provider tracking: guard nil ticker lookups and preserve merge attrs
- Guard fallback 1b lookup when security.ticker is blank to avoid matching NULL tickers
- Preserve external_id, provider_security_id, account_provider_id during collision merge
* Fix schema.rb version after merge (includes tax_treatment migration)
* fix: Rename migration to run after schema version
The migration 20260117000001 was skipped in CI because it had a timestamp
earlier than the schema version (2026_01_17_200000). CI loads schema.rb
directly and only runs migrations with versions after the schema version.
Renamed to 20260119000001 so it runs correctly.
* Update schema: remove Coinbase tables, add new fields and indexes
* Update schema: add back `tax_treatment` field with default value "taxable"
* Improve Turbo redirect action: use "replace" to avoid form submission in history
* Lock merged holdings to prevent provider overwrites and fix activity feed template indentation
* Refactor holdings transfer logic: enforce currency checks during collisions and enhance merge handling
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Hide payment contribution options from demo and manually created users
Demo data users and manually created users don't have stripe_customer_id
set on their family, so they should not see payment/contribution options.
Changes:
- Add can_manage_subscription? method to Family::Subscribeable that checks
for presence of stripe_customer_id
- Guard Settings::PaymentsController to return 403 for users without
stripe_customer_id
- Guard SubscriptionsController#show action (Stripe portal redirect) for
users without stripe_customer_id
- Update settings navigation to hide the payment link when
stripe_customer_id is not present
- Add tests for the new behavior
* Fix broken test
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Introduce SnapTrade integration with models, migrations, views, and activity processing logic.
* Refactor SnapTrade activities processing: improve activity fetching flow, handle pending states, and update UI elements for enhanced user feedback.
* Update Brakeman ignore file to include intentional redirect for SnapTrade OAuth portal.
* Refactor SnapTrade models, views, and processing logic: add currency extraction helper, improve pending state handling, optimize migration checks, and enhance user feedback in UI.
* Remove encryption for SnapTrade `snaptrade_user_id`, as it is an identifier, not a secret.
* Introduce `SnaptradeConnectionCleanupJob` to asynchronously handle SnapTrade connection cleanup and improve i18n for SnapTrade item status messages.
* Update SnapTrade encryption: make `snaptrade_user_secret` non-deterministic to enhance security.
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* **Add Coinbase integration with item and account management**
- Creates migrations for `coinbase_items` and `coinbase_accounts`.
- Adds models, controllers, views, and background tasks to support account linking, syncing, and transaction handling.
- Implements Coinbase API client and adapter for seamless integration.
- Supports ActiveRecord encryption for secure credential storage.
- Adds UI components for provider setup, account management, and synchronization.
* Localize Coinbase-related UI strings, refine account linking for security, and add timeouts to Coinbase API requests.
* Localize Coinbase account handling to support native currencies (USD, EUR, GBP, etc.) across balances, trades, holdings, and transactions.
* Improve Coinbase processing with timezone-safe parsing, native currency support, and immediate holdings updates.
* Improve trend percentage formatting and enhance race condition handling for Coinbase account linking.
* Fix log message wording for orphan cleanup
* Ensure `selected_accounts` parameter is sanitized by rejecting blank entries.
* Add tests for Coinbase integration: account, item, and controller coverage
- Adds unit tests for `CoinbaseAccount` and `CoinbaseItem` models.
- Adds integration tests for `CoinbaseItemsController`.
- Introduces Stimulus `select-all` controller for UI checkbox handling.
- Localizes UI strings and logging for Coinbase integration.
* Update test fixtures to use consistent placeholder API keys and secrets
* Refine `coinbase_item` tests to ensure deterministic ordering and improve scope assertions.
* Integrate `SyncStats::Collector` into Coinbase syncer to streamline statistics collection and enhance consistency.
* Localize Coinbase sync status messages and improve sync summary test coverage.
* Update `CoinbaseItem` encryption: use deterministic encryption for `api_key` and standard for `api_secret`.
* fix schema drift
* Beta labels to lower expectations
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Implement a setting to retrieve high res logos
* Update _brand_fetch_settings.html.erb
* Add fallback for stock tickers also to use Brandfetch
* Update security.rb
* Update toggle logic for high-res logos setting
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
* Update security.rb
* Update security.rb
---------
Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
* Add tax treatment support for accounts, investments, and cryptos
* Replace hardcoded region labels with I18n translations
* Add I18n support for subtype labels with fallback to hardcoded values
* fixed schema
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* feat: Support multiple crypto wallets with same token
Allows users to import multiple wallets containing the same
cryptocurrency (e.g., ETH on different wallet addresses).
Changes:
- Add wallet_address column to coinstats_accounts
- Update uniqueness validation to include wallet_address
- Extract and store wallet address in WalletLinker
- Add composite unique index on [item_id, account_id, wallet_address]
- Add tests for multi-wallet support and backwards compatibility
Users can now have:
- ETH (0xAAA...) → "Ethereum (0xAA...AA)"
- ETH (0xBBB...) → "Ethereum (0xBB...BB)"
Backwards compatible: existing accounts with wallet_address: nil
continue to work.
* style: Fix array bracket spacing in migration
* chore: Update schema.rb with wallet_address column and index
Add the missing wallet_address column and composite unique index
to db/schema.rb for CI compatibility with db:schema:load
* test: Add test for wallet deletion with same token different addresses
Verifies that deleting one wallet does not affect other wallets
that share the same token but have different addresses.
Addresses review comment from @EthanC via @jjmata
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
* Implement dynamic role assignment for new family creators.
Introduced `User.role_for_new_family_creator` to assign `super_admin` to the first user of an instance and a configurable fallback role (e.g., `admin`) to subsequent users. Updated controllers and tests accordingly.
* Update default fallback role for family creators to admin.
---------
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
* Add `investment_activity_label` to trades and enhance activity label handling
- Introduced `investment_activity_label` column to the `trades` table with a migration.
- Backfilled existing `trades` with activity labels based on quantity (`Buy`, `Sell`, or `Other`).
- Replaced `category_id` in trades with `investment_activity_label` for better alignment with transaction labels.
- Updated views and controllers to display and manage activity labels for trades.
- Added localized badge components for displaying and editing labels dynamically.
- Enhanced `PlaidAccount::Investments::TransactionsProcessor` to assign and process activity labels automatically.
- Added investment flows section to reports for tracking contributions and withdrawals.
- Refactored related tests and models for consistency and to ensure proper validation and filtering.
* Improve handling of `investment_activity_label`, trade type, and security selection in trades and transactions
- Refined label assignment logic in `trades_controller` to default to `Buy`/`Sell` based on transaction nature.
- Simplified security selection in `transactions_controller` by resolving via unique IDs or custom tickers.
- Streamlined UI for trade and transaction forms by updating dropdown options and label text.
- Enabled quick-edit badges to open `convert_to_trade` modal when applicable, enhancing flexibility.
- Adjusted tests and views to align with updated workflows and ensure consistent behavior.
* Improve handling of `investment_activity_label`, trade type, and security selection in trades and transactions
- Refined label assignment logic in `trades_controller` to default to `Buy`/`Sell` based on transaction nature.
- Simplified security selection in `transactions_controller` by resolving via unique IDs or custom tickers.
- Streamlined UI for trade and transaction forms by updating dropdown options and label text.
- Enabled quick-edit badges to open `convert_to_trade` modal when applicable, enhancing flexibility.
- Adjusted tests and views to align with updated workflows and ensure consistent behavior.
* Improve handling of `investment_activity_label`, trade type, and security selection in trades and transactions
- Refined label assignment logic in `trades_controller` to default to `Buy`/`Sell` based on transaction nature.
- Simplified security selection in `transactions_controller` by resolving via unique IDs or custom tickers.
- Streamlined UI for trade and transaction forms by updating dropdown options and label text.
- Enabled quick-edit badges to open `convert_to_trade` modal when applicable, enhancing flexibility.
- Adjusted tests and views to align with updated workflows and ensure consistent behavior.
* Add safeguard for `dropdownTarget` existence in quick edit controller
- Prevent errors by ensuring `dropdownTarget` is present before toggling its visibility.
* Fix undefined method 'category' for Trade on mobile view
Trade model uses investment_activity_label, not category. The upstream
merge introduced a call to trade.category which doesn't exist. Use the
activity label badge on mobile instead.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Fix activity label logic for zero/blank quantity and sell inference
- Return `nil` for blank or zero quantity in `investment_activity_label_for`.
- Correct `is_sell` logic to use the amount’s sign properly in `transactions_controller`.
* Fix i18n key paths in transactions controller for convert_to_trade
- Update flash message translations to use full i18n paths.
- Use `BigDecimal` for quantity and price calculations to improve precision.
---------
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: Preserve transaction tags during rule application
When rules set tags, they now ADD to existing tags instead of replacing
them. This fixes issue #518 where tags were being removed during bank sync.
The root cause was that SetTransactionTags called enrich_attribute with
just the single tag from the rule, which replaced all existing tags.
Now it merges the new tag with existing tags using .uniq to prevent
duplicates.
This preserves:
- User-applied tags that shouldn't be overwritten by rules
- Tags from other rules when multiple rules match the same transaction
- Tags set during previous syncs
* fix: Add nil guard for tag in SetTransactionTags
Return early with 0 if the tag is not found, preventing NoMethodError
when find_by_id returns nil. This matches the pattern used in
SetTransactionMerchant.
---------
Co-authored-by: Claude <noreply@anthropic.com>
- Added a new test to validate how dormant credit cards with zero balance and negative available balance are processed.
- Updated processor logic to ensure `current_balance` takes precedence when explicitly set to zero, preventing incorrect usage of `available_balance`.
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
- Updated `PLAID_TYPE_TO_LABEL` in `TransactionsProcessor` to consolidate labels ("Cancel" and "Cash" now mapped to "Other").
- Adjusted `label_from_plaid_type` to return "Other" as the default fallback.
- Enhanced tests to include additional valid activity labels and ensure label consistency.
- Minor fixes to locale keys for transaction views.
- Removed `exclude_from_cashflow` attribute across models, controllers, and views.
- Updated queries to rely solely on the `excluded` flag for filtering transactions and entries.
- Simplified migration by consolidating `exclude_from_cashflow` functionality into the existing `excluded` toggle.
- Refactored related tests to remove outdated logic and ensured compatibility with the updated implementation.
- Deleted the `InvestmentActivityDetector` and associated tests.
- Removed rake tasks for backfilling and clearing investment activity labels.
- Simplified transaction processing in `SimplefinEntry::Processor` by removing inferred activity label logic.
- Added new rule `SetInvestmentActivityLabel` for setting labels using rules.
- Updated `Rule::Registry::TransactionResource` to include the new rule executor.
- Introduced `InvestmentActivityDetector` to mark internal investment activity as excluded from cashflow and assign appropriate labels.
- Added `exclude_from_cashflow` flag to `entries` and `investment_activity_label` to `transactions` with migrations.
- Implemented rake tasks to backfill and clear investment activity labels.
- Updated `PlaidAccount::Investments::TransactionsProcessor` to map Plaid transaction types to labels.
- Included comprehensive test coverage for new functionality.
* Print initial impl
* Try to keep the bigger section together
* /* Tufte-inspired Print Report Styles */
* styling
* I8n
* Move print styling out.
* FIX unrelated test ordering
on line 53 - import.rows.first doesn't guarantee ordering. Without an explicit ORDER BY, the database may return rows in any order.
* Update print-report.css
* Update print.html.erb
* pass data to view
* Update index.html.erb
* Fix ERB helpers
* Update reports_helper.rb
* Add cost basis tracking and management to holdings
- Added migration to introduce `cost_basis_source` and `cost_basis_locked` fields to `holdings`.
- Implemented backfill for existing holdings to set `cost_basis_source` based on heuristics.
- Introduced `Holding::CostBasisReconciler` to manage cost basis resolution logic.
- Added user interface components for editing and locking cost basis in holdings.
- Updated `materializer` to integrate reconciliation logic and respect locked holdings.
- Extended tests for cost basis-related workflows to ensure accuracy and reliability.
* Fix cost basis calculation in holdings controller
- Ensure `cost_basis` is converted to decimal for accurate arithmetic.
- Fix conditional check to properly validate positive `cost_basis`.
* Improve cost basis validation and error handling in holdings controller
- Allow zero as a valid cost basis for gifted/inherited shares.
- Add error handling with user feedback for invalid cost basis values.
---------
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
- Refactored `persist_holdings` to separate and conditionally upsert holdings with and without cost_basis.
- Updated `avg_cost` logic to treat 0 cost_basis as unknown and return nil when cost_basis cannot be determined.
- Modified trend and investment calculation to exclude holdings with unknown cost_basis.
- Adjusted `average_cost` formatting to handle nil values in API responses and views.
- Added comprehensive tests to ensure cost_basis preservation and fallback behavior.
- Localized `unknown` label for display when cost_basis is unavailable.
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
* Add pending transaction handling and duplicate reconciliation logic
- Implemented logic to exclude pending transactions from budgets and analytics calculations.
- Introduced mechanisms for reconciling pending transactions with posted versions.
- Added duplicate detection with support for merging or dismissing matches.
- Updated transaction search filters to include a `status_filter` for pending/confirmed transactions.
- Introduced UI elements for reviewing and resolving duplicates.
- Enhanced `ProviderSyncSummary` with stats for reconciled and stale pending transactions.
* Refactor translation handling and enhance transaction and sync logic
- Moved hardcoded strings to locale files for improved translation support.
- Refined styling for duplicate transaction indicators and sync summaries.
- Improved logic for excluding stale pending transactions and updating timestamps on batch exclusion.
- Added unique IDs to status filters for better element targeting in UI.
- Optimized database queries to avoid N+1 issues in stale pending calculations.
* Add sync settings and enhance pending transaction handling
- Introduced a new "Sync Settings" section in hosting settings with UI to toggle inclusion of pending transactions.
- Updated handling of pending transactions with improved inference logic for `posted=0` and `transacted_at` in processors.
- Added priority order for pending transaction inclusion: explicit argument > environment variable > runtime configurable setting.
- Refactored settings and controllers to store updated sync preferences.
* Refactor sync settings and pending transaction reconciliation
- Extracted logic for pending transaction reconciliation, stale exclusion, and unmatched tracking into dedicated methods for better maintainability.
- Updated sync settings to infer defaults from multiple provider environment variables (`SIMPLEFIN_INCLUDE_PENDING`, `PLAID_INCLUDE_PENDING`).
- Refined UI and messaging to handle multi-provider configurations in sync settings.
# Conflicts:
# app/models/simplefin_item/importer.rb
* Debounce transaction reconciliation during imports
- Added per-run reconciliation debouncing to prevent repeated scans for the same account during chunked history imports.
- Trimmed size of reconciliation stats to retain recent details only.
- Introduced error tracking for reconciliation steps to improve UI visibility of issues.
* Apply ABS() in pending transaction queries and improve error handling
- Updated pending transaction logic to use ABS() for consistent handling of negative amounts.
- Adjusted amount bounds calculations to ensure accuracy for both positive and negative values.
- Refined exception handling in `merge_duplicate` to log failures and update user alert.
- Replaced `Date.today` with `Date.current` in tests to ensure timezone consistency.
- Minor optimization to avoid COUNT queries by loading limited records directly.
* Improve error handling in duplicate suggestion and dismissal logic
- Added exception handling for `store_duplicate_suggestion` to log failures and prevent crashes during fuzzy/low-confidence matches.
- Enhanced `dismiss_duplicate` action to handle `ActiveRecord::RecordInvalid` and display appropriate user alerts.
---------
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
* Add liability balance normalization logic with comprehensive tests
- Updated `SimplefinAccount::Processor` to normalize liability balances based on observed values, ensuring correct handling of debts and overpayments.
- Enhanced `SimplefinItem::Importer` to apply similar normalization rules during imports, improving consistency.
- Added multiple test cases in `SimplefinAccountProcessorTest` to validate edge cases for liabilities and mixed-sign scenarios.
- Introduced helper methods (`to_decimal`, `same_sign?`) to simplify numeric operations in normalization logic.
* Add overpayment detection for liabilities with heuristic-based classification
- Introduced `SimplefinAccount::Liabilities::OverpaymentAnalyzer` to classify liability balances as credit, debt, or unknown using transaction history.
- Updated `SimplefinAccount::Processor` and `SimplefinItem::Importer` to integrate heuristic-based balance normalization with fallback logic for ambiguous cases.
- Added comprehensive unit tests in `OverpaymentAnalyzerTest` to validate classification logic and edge cases.
- Enhanced logging and observability around classification results and fallback scenarios.
* Refactor liability handling for better fallback consistency
- Updated `sticky_key` method in `OverpaymentAnalyzer` to handle missing `@sfa.id` with a default value.
- Enhanced `SimplefinAccount::Processor` to use `with_indifferent_access` for `raw_payload` and `org_data`, improving robustness in liability type inference.
* Extract numeric helper methods into `SimplefinNumericHelpers` concern and apply across models
- Moved `to_decimal` and `same_sign?` methods into a new `SimplefinNumericHelpers` concern for reuse.
- Updated `OverpaymentAnalyzer`, `Processor`, and `Importer` to include the concern and remove redundant method definitions.
- Added empty fixtures for `simplefin_accounts` and `simplefin_items` to ensure test isolation.
- Refactored `OverpaymentAnalyzerTest` to reduce fixture dependencies and ensure cleanup of created records.
* Refactor overpayment detection logic for clarity and fallback consistency
- Simplified `enabled?` method in `OverpaymentAnalyzer` for clearer precedence order (Setting > ENV > default).
- Added `parse_bool` helper to streamline boolean parsing.
- Enhanced error handling with detailed logging for transaction gathering failures.
- Improved `sticky_key` method to use a temporary object ID fallback when `@sfa.id` is missing.
---------
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
* Implement API v1 Imports controller
- Add Api::V1::ImportsController with index, show, and create actions
- Add Jbuilder views for index and show
- Add integration tests
- Implement row generation logic in create action
- Update routes
* Validate import account belongs to family
- Add validation to Import model to ensure account belongs to the same family
- Add regression test case in Api::V1::ImportsControllerTest
* updating docs to be more detailed
* Rescue StandardError instead of bare rescue in ImportsController
* Optimize Imports API and fix documentation
- Implement rows_count counter cache for Imports
- Preload rows in Api::V1::ImportsController#show
- Update documentation to show correct OAuth scopes
* Fix formatting in ImportsControllerTest
* Permit all import parameters and fix unknown attribute error
* Restore API routes for auth, chats, and messages
* removing pr summary
* Fix trailing whitespace and configured? test failure
- Update Import#configured? to use rows_count for performance and consistency
- Mock rows_count in TransactionImportTest
- Fix trailing whitespace in migration
* Harden security and fix mass assignment in ImportsController
- Handle type and account_id explicitly in create action
- Rename import_params to import_config_params for clarity
- Validate type against Import::TYPES
* Fix MintImport rows_count update and migration whitespace
- Update MintImport#generate_rows_from_csv to update rows_count counter cache
- Fix trailing whitespace and final newline in AddRowsCountToImports migration
* Implement full-screen Drag and Drop CSV import on Transactions page
- Add DragAndDropImport Stimulus controller listening on document
- Add full-screen overlay with icon and text to Transactions index
- Update ImportsController to handle direct file uploads via create action
- Add system test for drag and drop functionality
* Implement Drag and Drop CSV upload on Import Upload page
- Add drag-and-drop-import controller to import/uploads/show
- Add full-screen overlay to import/uploads/show
- Annotate upload form and input with drag-and-drop targets
- Add PR_SUMMARY.md
* removing pr summary
* Add file validation to ImportsController
- Validate file size (max 10MB) and MIME type in create action
- Prevent memory exhaustion and invalid file processing
- Defined MAX_CSV_SIZE and ALLOWED_MIME_TYPES in Import model
* Refactor dragLeave logic with counter pattern to prevent flickering
* Extract shared drag-and-drop overlay partial
- Create app/views/imports/_drag_drop_overlay.html.erb
- Update transactions/index and import/uploads/show to use the partial
- Reduce code duplication in views
* Update Brakeman and harden ImportsController security
- Update brakeman to 7.1.2
- Explicitly handle type assignment in ImportsController#create to avoid mass assignment
- Remove :type from permitted import parameters
* Fix trailing whitespace in DragAndDropImportTest
* Don't commit LLM comments as file
* FIX add api validation
---------
Co-authored-by: Carlos Adames <cj@Carloss-MacBook-Air.local>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
Co-authored-by: sokie <sokysrm@gmail.com>
* FIX issue with stock price retrieval on weekend
* make weekend provisional and increase lookback
* FIX query error
* fix gap fill
The bug: When a price is provisional but the provider doesn't return a new value (weekends), we fall back to the existing DB value instead of gap-filling from Friday's correct price.
* Update importer.rb
Align provider fetch to use PROVISIONAL_LOOKBACK_DAYS for consistency. In the DB fallback, derive currency from provider_prices or db_prices and filter the query accordingly.
* Update 20260110122603_mark_suspicious_prices_provisional.rb
* Delete db/migrate/20260110122603_mark_suspicious_prices_provisional.rb
Signed-off-by: soky srm <sokysrm@gmail.com>
* Update importer.rb
* FIX tests
* FIX last tests
* Update importer_test.rb
The test doesn't properly force effective_start_date to skip old dates because there are many missing dates between the old date and recent dates. Let me fix it to properly test the subset processing scenario.
---------
Signed-off-by: soky srm <sokysrm@gmail.com>
