Commit Graph

67 Commits

Author SHA1 Message Date
BitToby
ba6e286b41 feat: add SSL_CA_FILE and SSL_VERIFY environment variables to support… (#894)
* feat: add SSL_CA_FILE and SSL_VERIFY environment variables to support self-signed certificates in self-hosted environments

* fix: NoMethodError by defining SSL helper methods before configure block executes

* refactor: Refactor SessionsController to use shared SslConfigurable module and simplify SSL initializer redundant checks

* refactor: improve SSL configuration robustness and error detection accuracy

* fix:HTTParty SSL options, add file validation guards, prevent Tempfile GC, and redact URLs in error logs

* fix:  Fix SSL concern indentation and stub Simplefin POST correctly in tests

* fix: normalize ssl_verify to always return boolean instead of nil

* fix: solve failing SimpleFin test

* refactor:  trim unused error-handling code from SslConfigurable, replace Tempfile with fixed-path CA bundle, fix namespace pollution in initializers, and add unit tests for core SSL configuration and Langfuse CRL callback.

* fix: added require ileutils in the initializer and require ostruct in the test file.

* fix: solve autoload conflict that broke provider loading, validate all certs in PEM bundles, and add missing requires.
2026-02-06 18:04:03 +01:00
Juan José Mata
946d0cb3ef Merge branch 'main' into feature/llm-cache-reset
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-27 08:50:01 +01:00
eureka928
23e9749ed1 Refactor clear_ai_cache to use family_scope pattern
- Move family-scoped queries to models via family_scope class method
- Remove hardcoded model names from Enrichable concern
- Replace inline rescue with proper respond_to? check
- Add count tracking for better logging
2026-01-26 10:17:28 +01:00
eureka928
029d09685e Address PR review feedback
- Preserve user locks: Only unlock attributes where current value still matches
  what AI set. If user changed the value, they took ownership.
- Add nil guard clause for family parameter in ClearAiCacheJob
- Add partial failure handling so one model's failure doesn't block the other
2026-01-26 10:14:06 +01:00
eureka928
ed8185cf2b Optimize clear_ai_cache to batch unlock attributes
Replace N individual update calls with single update_column for better performance.
2026-01-26 09:50:05 +01:00
eureka928
46ab1d8373 Add AI cache clearing infrastructure
Add ClearAiCacheJob for async cache clearing with low priority. Extend Enrichable concern with clear_ai_cache methods to unlock AI-enriched attributes and delete AI enrichment records. Trigger automatic cache clearing when OpenAI model setting changes.
2026-01-26 09:46:20 +01:00
soky srm
696ff0966b Initial security fixes (#461)
* Initial sec

* Update PII fields

* FIX add tests

* FIX safely read plaintext data on rake backfill

* Update user.rb

* FIX tests

* encryption_ready? block

* Test conditional to encryption on

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-23 22:05:28 +01:00
AdamWHY2K
3f5fff27ea feat: process pending transactions from lunchflow (#731)
* feat(config): add Lunchflow runtime configuration flags

* feat(api): add include_pending parameter to Lunchflow API

* feat(processor): add pending metadata support to Lunchflow processor

* feat(processor): generate temporary IDs for pending transactions

* feat(importer): integrate pending transaction support in sync

* fix(importer): improve deduplication for transactions without IDs

* feat(model): add Lunchflow pending support to Transaction scopes

* test: add Lunchflow processor pending metadata tests

* docs: update AGENTS.md for Lunchflow pending support

* chore: remove unused variable

* fix: simplify key check

* fix: dotenv-linter key order

* fix: avoid collapsing distinct pending transactions

* fix: prevent unbounded raw payload growth for blank IDs
2026-01-23 00:53:24 +01:00
LPW
a83f70425f Add SnapTrade brokerage integration with full trade history support (#737)
* Introduce SnapTrade integration with models, migrations, views, and activity processing logic.

* Refactor SnapTrade activities processing: improve activity fetching flow, handle pending states, and update UI elements for enhanced user feedback.

* Update Brakeman ignore file to include intentional redirect for SnapTrade OAuth portal.

* Refactor SnapTrade models, views, and processing logic: add currency extraction helper, improve pending state handling, optimize migration checks, and enhance user feedback in UI.

* Remove encryption for SnapTrade `snaptrade_user_id`, as it is an identifier, not a secret.

* Introduce `SnaptradeConnectionCleanupJob` to asynchronously handle SnapTrade connection cleanup and improve i18n for SnapTrade item status messages.

* Update SnapTrade encryption: make `snaptrade_user_secret` non-deterministic to enhance security.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-22 20:52:49 +01:00
LPW
64dc5c2fb8 Add tax treatment classification for investment accounts with international support (#693)
* Add tax treatment support for accounts, investments, and cryptos

* Replace hardcoded region labels with I18n translations

* Add I18n support for subtype labels with fallback to hardcoded values

* fixed schema

---------

Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
2026-01-18 17:29:02 +01:00
LPW
c391ba2b23 Harden SimpleFIN sync: protect user data, fix stuck syncs, optimize API calls (#671)
* Implement entry protection flags for sync overwrites

- Added `user_modified` and `import_locked` flags to `entries` table to prevent provider sync from overwriting user-edited and imported data.
- Introduced backfill migration to mark existing entries based on conditions.
- Enhanced sync and processing logic to respect protection flags, track skipped entries, and log detailed stats.
- Updated UI to display skipped/protected entries and reasons in sync summaries.

* Localize error details summary text and adjust `sync_account_later` method placement

* Restored schema.rb

---------

Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
2026-01-16 12:34:06 +01:00
LPW
78aa064bb0 Add overpayment detection for SimpleFIN liabilities (default ON) with heuristic-based classification and robust fallbacks (#412)
* Add liability balance normalization logic with comprehensive tests

- Updated `SimplefinAccount::Processor` to normalize liability balances based on observed values, ensuring correct handling of debts and overpayments.
- Enhanced `SimplefinItem::Importer` to apply similar normalization rules during imports, improving consistency.
- Added multiple test cases in `SimplefinAccountProcessorTest` to validate edge cases for liabilities and mixed-sign scenarios.
- Introduced helper methods (`to_decimal`, `same_sign?`) to simplify numeric operations in normalization logic.

* Add overpayment detection for liabilities with heuristic-based classification

- Introduced `SimplefinAccount::Liabilities::OverpaymentAnalyzer` to classify liability balances as credit, debt, or unknown using transaction history.
- Updated `SimplefinAccount::Processor` and `SimplefinItem::Importer` to integrate heuristic-based balance normalization with fallback logic for ambiguous cases.
- Added comprehensive unit tests in `OverpaymentAnalyzerTest` to validate classification logic and edge cases.
- Enhanced logging and observability around classification results and fallback scenarios.

* Refactor liability handling for better fallback consistency

- Updated `sticky_key` method in `OverpaymentAnalyzer` to handle missing `@sfa.id` with a default value.
- Enhanced `SimplefinAccount::Processor` to use `with_indifferent_access` for `raw_payload` and `org_data`, improving robustness in liability type inference.

* Extract numeric helper methods into `SimplefinNumericHelpers` concern and apply across models

- Moved `to_decimal` and `same_sign?` methods into a new `SimplefinNumericHelpers` concern for reuse.
- Updated `OverpaymentAnalyzer`, `Processor`, and `Importer` to include the concern and remove redundant method definitions.
- Added empty fixtures for `simplefin_accounts` and `simplefin_items` to ensure test isolation.
- Refactored `OverpaymentAnalyzerTest` to reduce fixture dependencies and ensure cleanup of created records.

* Refactor overpayment detection logic for clarity and fallback consistency

- Simplified `enabled?` method in `OverpaymentAnalyzer` for clearer precedence order (Setting > ENV > default).
- Added `parse_bool` helper to streamline boolean parsing.
- Enhanced error handling with detailed logging for transaction gathering failures.
- Improved `sticky_key` method to use a temporary object ID fallback when `@sfa.id` is missing.

---------

Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
2026-01-10 17:24:23 +01:00
LPW
140ea78b0e Add global sync summary component for all providers (#588)
* Add shared sync statistics collection and provider sync summary UI

- Introduced `SyncStats::Collector` concern to centralize sync statistics logic, including account, transaction, holdings, and health stats collection.
- Added collapsible `ProviderSyncSummary` component for displaying sync summaries across providers.
- Updated syncers (e.g., `LunchflowItem::Syncer`) to use the shared collector methods for consistent stats calculation.
- Added rake tasks under `dev:sync_stats` for testing and development purposes, including fake stats generation with optional issues.
- Enhanced provider-specific views to include sync summaries using the new shared component.

* Refactor `ProviderSyncSummary` to improve maintainability

- Extracted `severity_color_class` to simplify severity-to-CSS mapping.
- Replaced `holdings_label` with `holdings_label_key` for streamlined localization.
- Updated locale file to separate `found` and `processed` translations for clarity.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-09 19:26:37 +01:00
soky srm
d185c6161c FIX providers invalid currency handling (#589) 2026-01-09 11:54:38 +01:00
Ethan
3b4ab735b0 Add (beta) CoinStats Crypto Wallet Integration with Balance and Transaction Syncing (#512)
* Feat(CoinStats): Scaffold implementation, not yet functional

* Feat(CoinStats): Implement crypto wallet balance and transactions

* Feat(CoinStats): Add tests, Minor improvements

* Feat(CoinStats): Utilize bulk fetch API endpoints

* Feat(CoinStats): Migrate strings to i8n

* Feat(CoinStats): Fix error handling in wallet link modal

* Feat(CoinStats): Implement hourly provider sync job

* Feat(CoinStats): Generate docstrings

* Fix(CoinStats): Validate API Key on provider update

* Fix(Providers): Safely handle race condition in merchance creation

* Fix(CoinStats): Don't catch system signals in account processor

* Fix(CoinStats): Preload before iterating accounts

* Fix(CoinStats): Add no opener / referrer to API dashboard link

* Fix(CoinStats): Use strict matching for symbols

* Fix(CoinStats): Remove dead code in transactions importer

* Fix(CoinStats): Avoid transaction fallback ID collisions

* Fix(CoinStats): Improve Blockchains fetch error handling

* Fix(CoinStats): Enforce NOT NULL constraint for API Key schema

* Fix(CoinStats): Migrate sync status strings to i8n

* Fix(CoinStats): Use class name rather than hardcoded string

* Fix(CoinStats): Use account currency rather than hardcoded USD

* Fix(CoinStats): Migrate from standalone to Provider class

* Fix(CoinStats): Fix test failures due to string changes
2026-01-07 15:59:04 +01:00
Juan José Mata
bf90cad9a0 Add Recent Runs visibility for rule executions (#376)
* Add Recent Runs visibility for rule executions

Adds a comprehensive tracking system for rule execution history with the following features:

- Creates RuleRun model to track execution metadata:
  * Date/time of execution
  * Execution type (manual/scheduled)
  * Success/failure status
  * Rule reference
  * Transaction counts (processed and modified)
  * Error messages for failed runs

- Updates RuleJob to automatically record execution results:
  * Captures transaction processing statistics
  * Handles success/failure states
  * Stores error details for debugging

- Adds "Recent Runs" section to rules index page:
  * Paginated display (20 runs per page)
  * Columnar layout similar to LLM usage page
  * Visual status indicators (success/failed badges)
  * Error tooltips for failed runs
  * Responsive design with design system tokens

- Includes i18n translations for all user-facing strings

This provides users with visibility into rule execution history, making it easier to debug issues and monitor rule performance.

* Update schema.rb with rule_runs table definition

* Linter noise

* Separate transaction counts into Queued, Processed, and Modified

Previously, the code eagerly reported transactions as "processed" when they
were only queued for processing. This commit separates the counts into three
distinct metrics:

- Transactions Queued: Count of transactions matching the rule's filter
  conditions before any processing begins
- Transactions Processed: Count of transactions that were actually processed
  and modified by the rule actions
- Transactions Modified: Count of transactions that had their values changed
  (currently same as Processed, but allows for future differentiation)

Changes:
- Add transactions_queued column to rule_runs table
- Update RuleJob to track all three counts separately
- Update action executors to return count of modified transactions
- Update Rule#apply to aggregate modification counts from actions
- Add transactions_queued label to locales
- Update Recent Runs view to display new column
- Add validation for transactions_queued in RuleRun model

The tracking now correctly reports:
1. How many transactions matched the filter (queued)
2. How many were actually modified (processed/modified)
3. Distinguishes between matching and modifying transactions

* Add Pending status to track async rule execution progress

Introduced a new "pending" status for rule runs to properly track async
AI operations. The system now:

- Tracks pending async jobs with a counter that decrements as jobs complete
- Updates transactions_modified incrementally as each job finishes
- Only counts transactions that were actually modified (not just queued)
- Displays pending status with yellow badge in the UI
- Automatically transitions from pending to success when all jobs complete

This provides better visibility into long-running AI categorization and
merchant detection operations, showing real-time progress as Sidekiq
processes the batches.

* Fix migration version to 7.2 as per project standards

* Consolidate rule_runs migrations into single migration file

Merged three separate migrations (create, add_transactions_queued,
add_pending_jobs_count) into a single CreateRuleRuns migration.
This provides better clarity and maintains a clean migration history.

Changes:
- Updated CreateRuleRuns migration to include all columns upfront
- Removed redundant add_column migrations
- Updated schema version to 2025_11_24_000000

* Linter and test fixes

* Space optimization

* LLM l10n is better than no l10n

* Fix implementation for tags/AI rules

* Fix tests

* Use batch_size

* Consider jobs "unknown" status sometimes

* Rabbit suggestion

* Rescue block for RuleRun.create!

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-07 16:30:02 +01:00
Alessio Cappa
35f4b11872 fix: Fix Enable Banking sync date determination 2025-11-30 17:58:29 +01:00
soky srm
e8f935bc6f Remove plaid initialiser (#317)
* Remove plaid initialiser

The initializer can be safely removed because:
  - Config is lazily loaded via Provider::Registry
  - reload_configuration is called after settings updates
  - All calling code handles nil configs gracefully
  - Initial nil state is fine - config loads on first use

* Fix for missing config

* Actually don't pollute application.rb

* Add currency loading for balances

* Fix race condition on lazy load

* Allow loans to be imported in lunch flow also

* Fix currency processor
2025-11-12 16:01:19 +01:00
Himmelschmidt
4cd737b5d9 Fix SimpleFin investment holdings and comprehensive integration improvements (#104)
* Remove skipped account functionality from SimpleFin

- Remove "Skip - don't add" option from account setup
- Simplify account setup flow to require all accounts be assigned types
- Update controller logic to handle all accounts without skipping
- Redirect to accounts page instead of SimpleFin items page
- Add I18N message support with t(".success")

* Simplify SimpleFin sync logic by removing skipped accounts

- Remove skipped account filtering from syncer
- All unlinked accounts now block sync until setup is complete
- Remove skipped account UI elements from setup view
- Streamline sync flow without complex skipped/non-skipped logic

* Fix cash balance calculation for SimpleFin investment accounts

- Update SimplefinAccount::Processor to recalculate balances during sync
- Properly calculate cash_balance for investment accounts using BalanceCalculator
- Handle negative balances for credit cards and loans correctly
- Ensure account balance and cash balance are updated from latest SimpleFin data

* Add I18N translations and edit view for SimpleFin

- Add comprehensive English translations for SimpleFin UI
- Create edit view for SimpleFin re-authentication
- Support status messages, errors, and user feedback
- Match translation structure with Plaid integration

* Add specialized SimpleFin data processors

- Add investment processors for transactions, holdings, and balance calculation
- Add liability processors for credit cards and loans
- Add transaction processor for standard account transactions
- Add account importer for SimpleFin account data
- Organize processors by account type for maintainable architecture

* Add loading button controller for SimpleFin forms

- Add Stimulus controller to show loading state during form submission
- Disable button and show loading text to prevent double submissions
- Improve user experience during SimpleFin account setup

* Add SimpleFin edit and update routes

- Add edit and update actions to SimpleFin items routes
- Enable re-authentication flow for expired SimpleFin connections
- Match route structure with Plaid items for consistency

* Add institution metadata fields to SimpleFin items

- Add institution_id, institution_name, institution_domain fields
- Add institution_url, institution_color for UI customization
- Add raw_institution_payload for complete institution data storage
- Enable better institution organization and display

* Enhance SimpleFin item with institution support and metadata

- Add institution summary and connected institutions methods
- Store and retrieve institution metadata from SimpleFin API
- Add institution-aware import functionality
- Support multiple institutions per SimpleFin connection

* Fix account creation and Plaid provider issues

- Fix cash balance calculation in Account.create_from_simplefin_account
- Add nil check for plaid_provider in remove_plaid_item method
- Ensure proper balance handling for investment accounts during creation

* Improve sync parent relationship handling

- Add parent sync assignment for existing syncs when parent_sync is provided
- Ensure sync hierarchy is maintained when expanding sync windows
- Fix sync relationship consistency in nested sync operations

* Update SimpleFin item view with enhanced UI

- Improve SimpleFin connection display and status information
- Add better visual styling and user feedback
- Match UI consistency with Plaid item views

* Update database schema with institution fields

- Add institution metadata fields to simplefin_items table
- Support institution tracking and organization features

* Update SimpleFin tests for new functionality

- Update controller tests for edit/update actions and removed skip functionality
- Update model tests for institution metadata and enhanced features
- Ensure test coverage for SimpleFin improvements

* Add migration to remove old institution fields

* Fix linting issues

* Apply suggestion from @coderabbitai[bot]

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Himmelschmidt <46351743+Himmelschmidt@users.noreply.github.com>

* Fix 2 failing tests

* Wrap SimpleFin account transfer in database transaction

* Make loading button controller more reusable

- Add loadingText Stimulus value for configurable loading text
- Remove unused originalText variable
- Update view to pass loading text via data attribute

* Remove unused SimplefinAccount::Importer class

This class was added in the PR but is never called anywhere in the codebase.
The actual SimpleFin account processing is handled by SimplefinAccount::Processor
and its specialized sub-processors.

* Fix SimpleFin account transfer bug during token updates

- Call import_latest_simplefin_data before account transfer to ensure
  new SimpleFin accounts exist with account_id populated
- Prevents silent failure where accounts become orphaned when
  refreshing expired SimpleFin tokens

* Fix SimpleFin error handling to render correct template and use i18n

- Update render_error method to accept context parameter for template selection
- Fix update action to render :edit template instead of :new on errors
- Replace hardcoded error messages with localized strings using t() calls
- Add comprehensive error message keys to SimpleFin locale file

* Improve loading button accessibility and HTML semantics

- Add aria-disabled and aria-busy attributes for screen readers
- Use semantic span elements instead of divs for button content
- Add aria-hidden to decorative spinner element

* Fix SimpleFin SSL verification to use OpenSSL constant

* Remove HTTParty streaming to prevent empty response body and PII logging

* Use BigDecimal zero for consistent numeric types in balance calculator

* Add investment account guard to holdings processor

* Remove duplicate balance normalization from SimpleFin loan processor

* Fix critical account deletion bug in SimpleFin token update

* Fix linting issues in SimpleFin controller tests

* Replace hardcoded colors with design system tokens in SimpleFin views

* Gate investment processors to Investment accounts only

Prevents investment processors from running on non-Investment account types,
matching the pattern used by liability processors.

* Localize hardcoded strings in SimpleFin edit form

* Adding the error message to a hover state.

* Use only 1 month for sync_start_date, new account restriction?

* Harden investment cash_balance calculation with error handling

- Add try/catch around SimplefinAccount::Investments::BalanceCalculator
- Fallback to zero on calculation errors or nil results
- Log warning with error details for debugging
- Prevents nil cash_balance that could cause downstream issues

* Fix SimpleFin institution fields migration and add DB constraints

- Remove destructive migration that dropped existing institution fields
- Add only new fields (institution_domain, institution_color)
- Add indexes on institution fields for query performance
- Add NOT NULL constraints on required fields (institution_id, institution_name)
- Fix schema jsonb consistency for raw_institution_payload

* Improve SimpleFin holdings error handling and BigDecimal consistency

* Add security attribute to external link in SimpleFin edit form

* Improve SimpleFin sync timing and add user-configurable date range

- Fix initial sync timing issue: change from 1 month to 7 days default lookback
- Add user-selectable sync start date in account setup UI
- Implement chunked historical sync that respects user-selected date range
- Add sync_start_date field to SimpleFin items
- Handle new accounts on existing connections with same date picker

This addresses SimpleFin API limitations and gives users control over
how much transaction history to sync during initial setup.

* Fix SimpleFin sync status to show detailed results instead of "Never synced"

- Modify sync completion logic to always complete even with unlinked accounts
- Add sync_stats column to track account sync statistics during sync process
- Update sync status display to show "X synced, Y need setup" instead of "Never synced"
- Store detailed sync statistics (total, linked, unlinked accounts) in sync record
- Add sync_status_summary method to provide meaningful status text
- Remove early return that prevented sync completion when accounts need setup

Resolves issue where successful account syncing still showed "Never synced" status.

* Fix Transaction persistence before Entry creation in SimpleFin processor

Persist Transaction with create! instead of new to ensure it has an ID before
creating Entry that references it as entryable. Prevents foreign key errors.

* Fix indifferent access for SimpleFin institution data extraction

The accounts_snapshot parameter comes from JSON with string keys, but the
code was accessing with symbol keys which could silently fail. Convert to
indifferent access to handle both string and symbol keys properly.

* Localize hardcoded deletion in progress string

Replace hardcoded "(deletion in progress...)" text with I18n translation
to maintain consistency with the rest of the view.

* Fix SimpleFin item update test to properly verify rebind/destroy behavior

The test now creates a different SimplefinItem instance and mocks
create_simplefin_item! to return it, ensuring the controller operates
on a new record instead of the same instance. This properly exercises
the rebind/destroy logic and verifies the original item is scheduled
for deletion.

* Fix potential transaction data loss in SimpleFin importer

Prevent wiping stored transactions when API omits transaction data by only
updating raw_transactions_payload when transactions are actually present
in the response, preserving existing transaction data when API doesn't
include transactions key.

* Fix SimpleFin sync chunking and enforce 3-year historical limit

- Fix SimpleFin's actual API limit from 365 days to 60 days per request
- Implement proper backward-walking chunked sync for historical data
- Enforce 3-year maximum historical data limit (60 days × 22 requests)
- Update date picker to reflect 3-year limit and better defaults
- Add comprehensive logging for debugging sync issues

* Add dedicated raw_holdings_payload storage for SimpleFin accounts

- Add raw_holdings_payload column to simplefin_accounts table
- Separates holdings data from general account data for cleaner processing
- Follows same pattern as raw_transactions_payload for consistency
- Enables proper SimpleFin holdings processing pipeline

* Enhance SimpleFin holdings storage with external ID tracking

- Add external_id and cost_basis columns to holdings table
- Update holdings processor to use external_id for precise matching
- Capture all available SimpleFin holdings data (shares, market_value, cost_basis, etc.)
- Use SimpleFin holding ID as external_id with "simplefin_" prefix
- Calculate price from market_value/shares when available
- Store raw holdings payload in simplefin_accounts for complete data retention

This enables better holdings tracking than composite key approach and ensures
we capture all SimpleFin data even if not immediately used in our models.

* Simplify SimpleFin transaction enrichment

- Add MerchantDetector that uses payee field directly for merchant creation
- Enhance SimpleFin entry name generation combining payee + description
- Remove transaction processor category matching logic
- Create dedicated SimpleFin entry processor

Uses SimpleFin's clean payee data without unnecessary filtering.

* Add source field to ProviderMerchant and fix data enrichment

- Add source field to ProviderMerchant model for provider-specific merchant tracking
- Fix DataEnrichment to handle string transaction IDs correctly with to_i conversion

Enables per-provider merchant deduplication and fixes transaction lookups.

* Fix SimpleFin controller parameter handling

- Convert string account_ids to integers for proper account lookup
- Ensure account selection works correctly with form submissions

Fixes account filtering when setting up SimpleFin sync.

* Fix linting issues - auto-corrected whitespace and formatting

* Derive institution domain from URL when missing in SimpleFin items

* Fix render_error to preserve persisted record for edit context

* Add unique index to prevent duplicate holdings

* Fix potential NameError in holdings processor rescue block

* Guard against missing SimpleFin transaction IDs

* Fix SimpleFin amount parsing error handling

Re-raise ArgumentError instead of silently returning BigDecimal("0")
to prevent misleading $0 transactions from invalid amount data.

* Fix SimpleFin chunked import data loss bug

Merge transaction arrays instead of overwriting to prevent data loss
during chunked imports. Preserve most recent holdings data only.

* Add external_id uniqueness validation to Holding model

* Fix holdings cost_basis precision and add external_id unique constraint

* Fix SimpleFin test mock expectations and remove debug statements

- Fixed SimplefinItemsControllerTest by properly mocking Provider::Simplefin
  instead of over-mocking the create_simplefin_item! method
- Removed DEBUG puts statements from SimplefinItem::Importer

* Fix linting issues - auto-corrected whitespace and formatting

---------

Signed-off-by: Himmelschmidt <46351743+Himmelschmidt@users.noreply.github.com>
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-10-22 19:51:24 +02:00
Zach Gollwitzer
c1d98fe73b Start and end balance anchors for historical account balances (#2455)
* Add kind field to valuation

* Fix schema conflict

* Add kind to valuation

* Scaffold opening balance manager

* Opening balance manager implementation

* Update account import to use opening balance manager + tests

* Update account to use opening balance manager

* Fix test assertions, usage of current balance manager

* Lint fixes

* Add Opening Balance manager, add tests to forward calculator

* Add credit card to "all cash" designation

* Simplify valuation model

* Add current balance manager with tests

* Add current balance logic to reverse calculator and plaid sync

* Tweaks to initial calc logic

* Ledger testing helper, tweak assertions for reverse calculator

* Update test assertions

* Extract balance transformer, simplify calculators

* Algo simplifications

* Final tweaks to calculators

* Cleanup

* Fix error, propagate sync errors up to parent

* Update migration script, valuation naming
2025-07-15 11:42:41 -04:00
Zach Gollwitzer
662f2c04ce Multi-step account forms + clearer balance editing (#2427)
* Initial multi-step property form

* Improve form structure, add optional tooltip help icons to form fields

* Add basic inline alert component

* Clean up and improve property form lifecycle

* Implement Account status concept

* Lint fixes

* Remove whitespace

* Balance editing, scope updates for account

* Passing tests

* Fix brakeman warning

* Remove stale columns

* data constraint tweaks

* Redundant property
2025-07-03 09:33:07 -04:00
Zach Gollwitzer
10ce2c8e23 Balance sheet cache layer, non-blocking sync UI (#2356)
* Balance sheet cache layer with cache-busting

* Update family cache timestamps during Sync

* Less blocking sync loaders

* Consolidate family data caching key logic

* Fix turbo stream broadcasts

* Remove dev delay

* Add back account group sorting
2025-06-10 18:20:06 -04:00
Zach Gollwitzer
03a146222d Plaid sync domain improvements (#2267)
Breaks our Plaid sync process out into more manageable classes. Notably, this moves the sync process to a distinct, 2-step flow:

1. Import stage - we first make API calls and import Plaid data to "mirror" tables
2. Processing stage - read the raw data, apply business rules, build internal domain models and sync balances

This provides several benefits:

- Plaid syncs can now be "replayed" without fetching API data again
- Mirror tables provide better audit and debugging capabilities
- Eliminates the "all or nothing" sync behavior that is currently in place, which is brittle
2025-05-23 18:58:22 -04:00
Zach Gollwitzer
94a807c3c9 Encapsulate enrichment actions, add tests 2025-05-20 11:33:35 -04:00
Zach Gollwitzer
137219c121 Fix attribute locking namespace conflict, duplicate syncs 2025-05-19 16:39:31 -04:00
Zach Gollwitzer
29a8ac9d8a Tweak exception logging, sync stale behavior 2025-05-18 10:19:15 -04:00
Zach Gollwitzer
9f13b5bb83 Handle stale syncs (#2257)
* Handle stale syncs

* Use `visible` sync logic in sidebar groups
2025-05-17 18:28:21 -04:00
Zach Gollwitzer
10dd9e061a Improve account sync performance, handle concurrent market data syncing (#2236)
* PlaidConnectable concern

* Remove bad abstraction

* Put sync implementations in own concerns

* Sync strategies

* Move sync orchestration to Sync class

* Clean up sync class, add state machine

* Basic market data sync cron

* Fix price sync

* Improve sync window column names, add timestamps

* 30 day syncs by default

* Clean up market data methods

* Report high duplicate sync counts to Sentry

* Add sync states throughout app

* account tab session

* Persistent account tab selections

* Remove manual sleep

* Add migration to clear stale syncs on self hosted apps

* Tweak sync states

* Sync completion event broadcasts

* Fix timezones in tests

* Cleanup

* More cleanup

* Plaid item UI broadcasts for sync

* Fix account ID namespace conflict

* Sync broadcasters

* Smoother account sync refreshes

* Remove test sync delay
2025-05-15 10:19:56 -04:00
Zach Gollwitzer
bcb47a9d29 Fix auto sync trigger logic and add tests 2025-05-13 16:14:29 -04:00
Zach Gollwitzer
ab2cec55e7 Propagate child sync errors up to parent, fix sync status (#2232)
* Propagate child sync errors up to parent, fix sync status

* Remove testing error
2025-05-09 14:56:49 -04:00
Alex Hatzenbuhler
47aeaf8cea Add nice formatting for subtypes on account list (#2138)
* Add nice formatting for subtypes on account list

* Fix rubocop linting errors

* Implement better mapping

* Fix rubocop linting

* Add short and long versions of subtypes

* Simplify subtype reference

Co-authored-by: Zach Gollwitzer <zach.gollwitzer@gmail.com>
Signed-off-by: Alex Hatzenbuhler <hatz@hey.com>

* Simplify reference logic, add a small test

* Fix test

* Fix tests

---------

Signed-off-by: Alex Hatzenbuhler <hatz@hey.com>
Co-authored-by: Zach Gollwitzer <zach.gollwitzer@gmail.com>
2025-04-22 14:10:50 -04:00
Zach Gollwitzer
297a695d0f Transaction rules engine V1 (#1900)
* Domain model sketch

* Scaffold out rules domain

* Migrations

* Remove existing data enrichment for clean slate

* Sketch out business logic and basic tests

* Simplify rule scope building and action executions

* Get generator working again

* Basic implementation + tests

* Remove manual merchant management (rules will replace)

* Revert "Remove manual merchant management (rules will replace)"

This reverts commit 83dcbd9ff0aa7bbee211796b71aa48b71df5e57e.

* Family and Provider merchants model

* Fix brakeman warnings

* Fix notification loader

* Update notification position

* Add Rule action and condition registries

* Rule form with compound conditions and tests

* Split out notification types, add CTA type

* Rules form builder and Stimulus controller

* Clean up rule registry domain

* Clean up rules stimulus controller

* CTA message for rule when user changes transaction category

* Fix tests

* Lint updates

* Centralize notifications in Notifiable concern

* Implement category rule prompts with auto backoff and option to disable

* Fix layout bug caused by merge conflict

* Initialize rule with correct action for category CTA

* Add rule deletions, get rules working

* Complete dynamic rule form, split Stimulus controllers by resource

* Fix failing tests

* Change test password to avoid chromium conflicts

* Update integration tests

* Centralize all test password references

* Add re-apply rule action

* Rule confirm modal

* Run migrations

* Trigger rule notification after inline category updates

* Clean up rule styles

* Basic attribute locking for rules

* Apply attribute locks on user edits

* Log data enrichments, only apply rules to unlocked attributes

* Fix merge errors

* Additional merge conflict fixes

* Form UI improvements, ignore attribute locks on manual rule application

* Batch AI auto-categorization of transactions

* Auto merchant detection, ai enrichment in batches

* Fix Plaid merchant assignments

* Plaid category matching

* Cleanup 1

* Test cleanup

* Remove stale route

* Fix desktop chat UI issues

* Fix mobile nav styling issues
2025-04-18 11:39:58 -04:00
Zach Gollwitzer
8648f11413 Sync hierarchy updates (#2087)
* Add parent sync orchestration

* Pass sync object to dependents
2025-04-11 12:13:46 -04:00
Zach Gollwitzer
f65b93a352 Data provider simplification, tests, and documentation (#1997)
* Ignore env.test from source control

* Simplification of providers interface

* Synth tests

* Update money to use new find rates method

* Remove unused issues code

* Additional issue feature removals

* Update price data fetching and tests

* Update documentation for providers

* Security test fixes

* Fix self host test

* Update synth usage data access

* Remove AI pr schema changes
2025-03-17 11:54:53 -04:00
Zach Gollwitzer
fa0248056d Show UI warning to user when they need provider data but have not setup Synth yet (#1926)
* Simplify provider concerns

* Update tests

* Add UI warning for missing Synth key if family requires external data
2025-02-28 11:35:10 -05:00
Zach Gollwitzer
d75be2282b New Design System + Codebase Refresh (#1823)
Since the very first 0.1.0-alpha.1 release, we've been moving quickly to add new features to the Maybe app. In doing so, some parts of the codebase have become outdated, unnecessary, or overly-complex as a natural result of this feature prioritization.

Now that "core" Maybe is complete, we're moving into a second phase of development where we'll be working hard to improve the accuracy of existing features and build additional features on top of "core". This PR is a quick overhaul of the existing codebase aimed to:

- Establish the brand new and simplified dashboard view (pictured above)
- Establish and move towards the conventions introduced in Cursor rules and project design overview #1788
- Consolidate layouts and improve the performance of layout queries
- Organize the core models of the Maybe domain (i.e. Account::Entry, Account::Transaction, etc.) and break out specific traits of each model into dedicated concerns for better readability
- Remove stale / dead code from codebase
- Remove overly complex code paths in favor of simpler ones
2025-02-21 11:57:59 -05:00
Zach Gollwitzer
5eb5ec7aef fix: Plaid webhook verification (#1824)
* Fix Plaid webhook verification

* Fix client creation in webhook controller
2025-02-07 10:35:42 -05:00
Zach Gollwitzer
53f4b32c33 Fix EU plaid flow (#1761)
* Fix EU plaid flow

* Fix failing tests
2025-01-31 17:04:26 -05:00
Josh Pigford
4bf72506d5 Initial pass at Plaid EU (#1555)
* Initial pass at Plaid EU

* Add EU support to Plaid Items

* Lint

* Temp fix for rubocop isseus

* Merge cleanup

* Pass in region and get tests passing

* Use absolute path for translation

---------

Signed-off-by: Josh Pigford <josh@joshpigford.com>
2025-01-31 12:13:58 -06:00
Zach Gollwitzer
a4d10097d5 Preserve pagination on entry updates (#1563)
* Preserve pagination on entry updates

* Test fix
2024-12-20 11:24:46 -05:00
Zach Gollwitzer
fe199f2357 Add account data enrichment (#1532)
* Add data enrichment

* Make data enrichment optional for self-hosters

* Add categories to data enrichment

* Only update category and merchant if nil

* Fix name overrides

* Lint fixes
2024-12-13 17:22:27 -05:00
Zach Gollwitzer
49c353e10c Plaid portfolio sync algorithm and calculation improvements (#1526)
* Start tests rework

* Cash balance on schema

* Add reverse syncer

* Reverse balance sync with holdings

* Reverse holdings sync

* Reverse holdings sync should work with only trade entries

* Consolidate brokerage cash

* Add forward sync option

* Update new balance info after syncs

* Intraday balance calculator and sync fixes

* Show only balance for trade entries

* Tests passing

* Update Gemfile.lock

* Cleanup, performance improvements

* Remove account reloads for reliable sync outputs

* Simplify valuation view logic

* Special handling for Plaid cash holding
2024-12-10 17:41:20 -05:00
Zach Gollwitzer
c3248cd796 Improve account transaction, trade, and valuation editing and sync experience (#1506)
* Consolidate entry controller logic

* Transaction builder

* Update trades controller to use new params

* Load account charts in turbo frames, fix PG overflow

* Consolidate tests

* Tests passing

* Remove unused code

* Add client side trade form validations
2024-11-27 16:01:50 -05:00
Zach Gollwitzer
6996a225ba Add post-sync UI stream updates (#1482)
* Add post-sync UI stream updates

* Fix stream channel id
2024-11-20 16:46:06 -05:00
Zach Gollwitzer
e641cfccd4 Add post-sync hook (#1479) 2024-11-20 11:01:52 -05:00
Zach Gollwitzer
cbba2ba675 Basic Plaid Integration (#1433)
* Basic plaid data model and linking

* Remove institutions, add plaid items

* Improve schema and Plaid provider

* Add webhook verification sketch

* Webhook verification

* Item accounts and balances sync setup

* Provide test encryption keys

* Fix test

* Only provide encryption keys in prod

* Try defining keys in test env

* Consolidate account sync logic

* Add back plaid account initialization

* Plaid transaction sync

* Sync UI overhaul for Plaid

* Add liability and investment syncing

* Handle investment webhooks and process current day holdings

* Remove logs

* Remove "all" period select for performance

* fix amount calc

* Remove todo comment

* Coming soon for investment historical data

* Document Plaid configuration

* Listen for holding updates
2024-11-15 13:49:37 -05:00
Zach Gollwitzer
65db49273c Account Activity View + Account Forms (#1406)
* Remove balance mode, sketch out refactor

* Activity view checkpoint

* Entry partials, checkpoint

* Finish txn partial

* Give entries context when editing for different turbo responses

* Calculate change of balance for each entry

* Account tabs consolidation

* Translations, linting, brakeman updates

* Account actions concern

* Finalize forms, get account system tests passing

* Get tests passing

* Lint, rubocop, schema updates

* Improve routing and stream responses

* Fix broken routes

* Add import option for adding accounts

* Fix system test

* Fix test specificity

* Fix sparklines

* Improve account redirects
2024-11-04 20:27:31 -05:00
Josh Pigford
e7f09e6f71 Groundwork for security info (#1396)
* Groundwork for security info

* Lint
2024-10-30 18:08:19 -04:00
Zach Gollwitzer
bf695972e4 Remove missing prices issue (#1390) 2024-10-29 14:55:46 -04:00
Zach Gollwitzer
3c0fdd84ee Fix mode bug 2024-10-18 18:25:17 -04:00