Commit Graph

90 Commits

Author SHA1 Message Date
LPW
ee6afb48fd Add encryption support to provider account models (#815)
* Enable encryption for raw payloads in account models.

* Add backfill support for Snaptrade, Coinbase, Coinstats, and Mercury accounts.
2026-02-04 23:40:01 +01:00
Juan José Mata
0fb9d60ee6 Use dependent: :purge_later for ActiveRecord attachments (#882)
* Use dependent: :purge_later for user profile_image cleanup

This is a simpler alternative to PR #787's callback-based approach.
Instead of adding a custom callback and method, we use Rails' built-in
`dependent: :purge_later` option which is already used by FamilyExport
and other models in the codebase.

This single-line change ensures orphaned ActiveStorage attachments are
automatically purged when a user is destroyed, without the overhead of
querying all attachments manually.

https://claude.ai/code/session_01Np3deHEAJqCBfz3aY7c3Tk

* Add dependent: :purge_later to all ActiveStorage attachments

Extends the attachment cleanup from PR #787 to cover ALL models with
ActiveStorage attachments, not just User.profile_image.

Models updated:
- PdfImport.pdf_file - prevents orphaned PDF files from imports
- Account.logo - prevents orphaned account logos
- PlaidItem.logo, SimplefinItem.logo, SnaptradeItem.logo,
  CoinstatsItem.logo, CoinbaseItem.logo, LunchflowItem.logo,
  MercuryItem.logo, EnableBankingItem.logo - prevents orphaned
  provider logos

This ensures that when a family is deleted (cascade from last user
purge), all associated storage files are properly cleaned up via
Rails' built-in dependent: :purge_later mechanism.

https://claude.ai/code/session_01Np3deHEAJqCBfz3aY7c3Tk

* Make sure `Provider` generator adds it

* Fix tests

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-02-03 15:45:25 +01:00
Juan José Mata
02cd84568e Add deterministic API key for uptime monitoring (#834)
* Preserve existing demo data by default

Add SKIP_CLEAR environment variable to demo_data rake tasks.
Defaults to true (preserving existing data). Set SKIP_CLEAR=0
to wipe data before generating new demo data.

https://claude.ai/code/session_01GcoMc2SH3czPrbeGkHbmpE

* Add deterministic instatus.com API key for demo data

Create a read-only API key named "instatus.com" with a fixed value
when generating demo data. This allows uptime monitoring tools to
use a hardcoded API key that doesn't change between demo data runs.

The key is idempotent - if it already exists, it will be reused.

https://claude.ai/code/session_01GcoMc2SH3czPrbeGkHbmpE

* OK to name instatus to a point

* Remove all Instatus references

* Rename to create_monitoring_api_key! and scope lookup to admin_user

- Rename create_instatus_api_key! to create_monitoring_api_key! (snake_case)
- Scope API key lookup to admin_user instead of global ApiKey lookup
- Each family's admin now has their own monitoring API key

https://claude.ai/code/session_01GcoMc2SH3czPrbeGkHbmpE

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-30 12:29:46 +01:00
LPW
d98711d4ea Rename raw_investments_payload to raw_holdings_payload for Plaid accounts (#760)
* refactor: rename `raw_investments_payload` to `raw_holdings_payload`

- Update references and models to use consistent naming.
- Adjust migrations, tests, and encryption setup accordingly.

* fix: improve safety when accessing raw_holdings_payload keys

- Use `dig` with safe navigation to prevent potential nil errors.
- Add support for decryption from the old column name `raw_investments_payload`.
- Adjust related methods and calculations for consistency.

---------

Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
2026-01-24 11:16:26 +01:00
soky srm
696ff0966b Initial security fixes (#461)
* Initial sec

* Update PII fields

* FIX add tests

* FIX safely read plaintext data on rake backfill

* Update user.rb

* FIX tests

* encryption_ready? block

* Test conditional to encryption on

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-23 22:05:28 +01:00
luckyPipewrench
4909532a08 fix: address PR review feedback for generator templates
- Fix invalid Tailwind class bg-gray -> bg-gray-400 in panel template
- Fix count_holdings to only count linked accounts for consistency
2026-01-23 12:50:34 -05:00
luckyPipewrench
9caee4ec36 Remove Testprovider support and related references for streamlined provider handling. 2026-01-23 11:52:20 -05:00
luckyPipewrench
b8ffe06974 Add banking support to family generator, including transactions processor, SDK updates, and related templates. Streamline logic for handling provider types. 2026-01-23 11:31:57 -05:00
luckyPipewrench
3382c07194 Refactor family generator templates for streamlined labels, error handling, and account setup logic. 2026-01-22 22:19:38 -05:00
luckyPipewrench
17693f0418 Refactor family generator to centralize concern creation, improve SDK support, and add tests, views, jobs, and sync logic. 2026-01-22 22:07:22 -05:00
soky srm
179552657c Mercury integration (#723)
* Initial mercury impl

* FIX both mercury and generator class

* Finish mercury integration and provider generator

* Fix schema

* Fix linter and tags

* Update routes.rb

* Avoid schema drift

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-22 20:37:07 +01:00
Juan José Mata
051197137c Fix Polish translation template error (and increase testing/fix other locales as well) (#734)
* Add Polish locale support for money formatting

The Money::Formatting module handles locale-specific currency formatting
for French, German, Spanish, Italian, and Portuguese (Brazil), but was
missing support for Polish locale. This caused formatting issues when
users with Polish locale viewed account valuations.

- Add Polish locale handling to locale_options method
- Polish formatting uses: space as thousands delimiter, comma as
  decimal separator, symbol after number ("%n %u" format)
- Add test coverage for Polish locale formatting

* Add locale support for all supported locales in Money::Formatting

Extend the Money::Formatting module to handle all locales from
SUPPORTED_LOCALES to prevent template errors when users select
different languages.

Added locale-specific formatting for:
- Turkish (tr): dot delimiter, comma separator, symbol after number
- Norwegian Bokmål (nb): space delimiter, comma separator, symbol after
- Catalan (ca): dot delimiter, comma separator, symbol after number
- Romanian (ro): dot delimiter, comma separator, symbol after number
- Dutch (nl): dot delimiter, comma separator, symbol before number

Also improved Dutch handling to work with all currencies (not just EUR).

Added comprehensive tests for:
- All newly supported locales
- Chinese (zh-CN, zh-TW) which use default English-style formatting
- A test that verifies all SUPPORTED_LOCALES can format without errors

* Fix broken money formatting tests

- Fix Chinese Traditional locale test: TWD currency uses "TW$" symbol
  (prefixed with first 2 chars of ISO code to distinguish from USD)
- Fix all supported locales test: replace assert_nothing_raised (which
  doesn't accept message argument in Minitest) with explicit assertions

* Refactor Money::Formatting to consolidate locale patterns

Group locales by their formatting patterns into constants to reduce
repetition and make it easier to add new locales:

- EUROPEAN_SYMBOL_AFTER: de, es, it, tr, ca, ro
  (dot delimiter, comma separator, symbol after)
- SPACE_DELIMITER_SYMBOL_AFTER: pl, nb
  (space delimiter, comma separator, symbol after)
- EUROPEAN_SYMBOL_BEFORE: nl, pt-BR
  (dot delimiter, comma separator, symbol before)

French locale remains separate due to its unique non-breaking space usage.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-22 14:27:12 +01:00
foXaCe
917101853b fix: Locale-aware currency formatting (#677)
* fix: Locale-aware currency formatting

Add locale-specific formatting for money display:
- French (fr): symbol after number with non-breaking space (1 000,12 €)
- German (de): symbol after number (1.000,12 €)
- Spanish (es): symbol after number (1.000,12 €)
- Italian (it): symbol after number (1.000,12 €)
- Portuguese-Brazil (pt-BR): symbol before with space (R$ 1.000,12)

This follows international conventions where most European languages
place the currency symbol after the number, unlike English.

* fix: Address CodeRabbit review comments

- Use non-breaking spaces (NBSP) for French locale formatting
- Add nil guard in locale_options to prevent NoMethodError
- Add test coverage for Portuguese (Brazil) locale

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-16 23:55:51 +01:00
Juan José Mata
ababbb99d0 Is sample_data or demo_data? 2026-01-13 15:53:10 +00:00
soky srm
064833621e Merge pull request #538 from luckyPipewrench/sso-upgrades
Multi-provider SSO with admin UI and SAML support
2026-01-12 15:38:59 +01:00
LPW
3658e812a8 Add pending transaction handling and duplicate reconciliation logic (#602)
* Add pending transaction handling and duplicate reconciliation logic

- Implemented logic to exclude pending transactions from budgets and analytics calculations.
- Introduced mechanisms for reconciling pending transactions with posted versions.
- Added duplicate detection with support for merging or dismissing matches.
- Updated transaction search filters to include a `status_filter` for pending/confirmed transactions.
- Introduced UI elements for reviewing and resolving duplicates.
- Enhanced `ProviderSyncSummary` with stats for reconciled and stale pending transactions.

* Refactor translation handling and enhance transaction and sync logic

- Moved hardcoded strings to locale files for improved translation support.
- Refined styling for duplicate transaction indicators and sync summaries.
- Improved logic for excluding stale pending transactions and updating timestamps on batch exclusion.
- Added unique IDs to status filters for better element targeting in UI.
- Optimized database queries to avoid N+1 issues in stale pending calculations.

* Add sync settings and enhance pending transaction handling

- Introduced a new "Sync Settings" section in hosting settings with UI to toggle inclusion of pending transactions.
- Updated handling of pending transactions with improved inference logic for `posted=0` and `transacted_at` in processors.
- Added priority order for pending transaction inclusion: explicit argument > environment variable > runtime configurable setting.
- Refactored settings and controllers to store updated sync preferences.

* Refactor sync settings and pending transaction reconciliation

- Extracted logic for pending transaction reconciliation, stale exclusion, and unmatched tracking into dedicated methods for better maintainability.
- Updated sync settings to infer defaults from multiple provider environment variables (`SIMPLEFIN_INCLUDE_PENDING`, `PLAID_INCLUDE_PENDING`).
- Refined UI and messaging to handle multi-provider configurations in sync settings.

# Conflicts:
#	app/models/simplefin_item/importer.rb

* Debounce transaction reconciliation during imports

- Added per-run reconciliation debouncing to prevent repeated scans for the same account during chunked history imports.
- Trimmed size of reconciliation stats to retain recent details only.
- Introduced error tracking for reconciliation steps to improve UI visibility of issues.

* Apply ABS() in pending transaction queries and improve error handling

- Updated pending transaction logic to use ABS() for consistent handling of negative amounts.
- Adjusted amount bounds calculations to ensure accuracy for both positive and negative values.
- Refined exception handling in `merge_duplicate` to log failures and update user alert.
- Replaced `Date.today` with `Date.current` in tests to ensure timezone consistency.
- Minor optimization to avoid COUNT queries by loading limited records directly.

* Improve error handling in duplicate suggestion and dismissal logic

- Added exception handling for `store_duplicate_suggestion` to log failures and prevent crashes during fuzzy/low-confidence matches.
- Enhanced `dismiss_duplicate` action to handle `ActiveRecord::RecordInvalid` and display appropriate user alerts.

---------

Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
2026-01-10 20:11:00 +01:00
Josh Waldrep
238fa8e0ca Merge remote-tracking branch 'upstream/main' into sso-upgrades
# Conflicts:
#	app/views/simplefin_items/_simplefin_item.html.erb
#	db/schema.rb
2026-01-10 11:57:23 -05:00
LPW
140ea78b0e Add global sync summary component for all providers (#588)
* Add shared sync statistics collection and provider sync summary UI

- Introduced `SyncStats::Collector` concern to centralize sync statistics logic, including account, transaction, holdings, and health stats collection.
- Added collapsible `ProviderSyncSummary` component for displaying sync summaries across providers.
- Updated syncers (e.g., `LunchflowItem::Syncer`) to use the shared collector methods for consistent stats calculation.
- Added rake tasks under `dev:sync_stats` for testing and development purposes, including fake stats generation with optional issues.
- Enhanced provider-specific views to include sync summaries using the new shared component.

* Refactor `ProviderSyncSummary` to improve maintainability

- Extracted `severity_color_class` to simplify severity-to-CSS mapping.
- Replaced `holdings_label` with `holdings_label_key` for streamlined localization.
- Updated locale file to separate `found` and `processed` translations for clarity.

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-09 19:26:37 +01:00
soky srm
e37c03d1d4 Implement Run all rules (#582) 2026-01-08 15:20:14 +01:00
Josh Waldrep
14993d871c feat: comprehensive SSO/OIDC upgrade with enterprise features
Multi-provider SSO support:
   - Database-backed SSO provider management with admin UI
   - Support for OpenID Connect, Google OAuth2, GitHub, and SAML 2.0
   - Flipper feature flag (db_sso_providers) for dynamic provider loading
   - ProviderLoader service for YAML or database configuration

   Admin functionality:
   - Admin::SsoProvidersController for CRUD operations
   - Admin::UsersController for super_admin role management
   - Pundit policies for authorization
   - Test connection endpoint for validating provider config

   User provisioning improvements:
   - JIT (just-in-time) account creation with configurable default role
   - Changed default JIT role from admin to member (security)
   - User attribute sync on each SSO login
   - Group/role mapping from IdP claims

   SSO identity management:
   - Settings::SsoIdentitiesController for users to manage connected accounts
   - Issuer validation for OIDC identities
   - Unlink protection when no password set

   Audit logging:
   - SsoAuditLog model tracking login, logout, link, unlink, JIT creation
   - Captures IP address, user agent, and metadata

   Advanced OIDC features:
   - Custom scopes per provider
   - Configurable prompt parameter (login, consent, select_account, none)
   - RP-initiated logout (federated logout to IdP)
   - id_token storage for logout

   SAML 2.0 support:
   - omniauth-saml gem integration
   - IdP metadata URL or manual configuration
   - Certificate and fingerprint validation
   - NameID format configuration
2026-01-03 17:56:42 -05:00
Juan José Mata
204315b70b No Rswag in prod 2025-12-22 11:30:38 +00:00
Juan José Mata
ea8e6a1106 Improve Langfuse setup check via rake 2025-12-20 00:08:10 +00:00
Juan José Mata
9d54719007 Add RSwag coverage for /chat and /transactions API endpoints (#210)
* Add RSwag coverage for chat API

* Linter

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Juan José Mata <jjmata@jjmata.com>

* Add transaction rswag

* FIX linter

---------

Signed-off-by: Juan José Mata <jjmata@jjmata.com>
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: sokie <sokysrm@gmail.com>
2025-12-17 14:14:17 +01:00
LPW
101b9dac95 Refactor transaction merging logic to prefer accurate posted dates and improve deduplication (#449)
- Updated `date_utils.rb` to return `nil` for timestamps equal to 0.
- Enhanced `SimplefinItem::Importer` to merge transactions by prioritizing non-pending records with valid posted timestamps.
- Introduced a comparator and key-builder for resolving transaction conflicts systematically.

Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
2025-12-13 09:13:02 +01:00
soky srm
5d6c1bc280 Provider generator (#364)
* Move provider config to family

* Update schema.rb

* Add provier generator

* Add table creation also

* FIX generator namespace

* Add support for global providers also

* Remove over-engineered stuff

* FIX parser

* FIX linter

* Some generator fixes

* Update generator with fixes

* Update item_model.rb.tt

* Add missing linkable concern

* Add missing routes

* Update adapter.rb.tt

* Update connectable_concern.rb.tt

* Update unlinking_concern.rb.tt

* Update family_generator.rb

* Update family_generator.rb

* Delete .claude/settings.local.json

Signed-off-by: soky srm <sokysrm@gmail.com>

* Move docs under API related folder

* Rename Rails generator doc

* Light edits to LLM generated doc

* Small Lunch Flow config panel regressions.

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-12-08 22:52:30 +01:00
soky srm
88952e4714 Small llms improvements (#400)
* Initial implementation

* FIX keys

* Add langfuse evals support

* FIX trace upload

* Delete .claude/settings.local.json

Signed-off-by: soky srm <sokysrm@gmail.com>

* Update client.rb

* Small LLMs improvements

* Keep batch size normal

* Update categorizer

* FIX json mode

* Add reasonable alternative to matching

* FIX thinking blocks for llms

* Implement json mode support with AUTO mode

* Make auto default for everyone

* FIX linter

* Address review

* Allow export manual categories

* FIX user export

* FIX oneshot example pollution

* Update categorization_golden_v1.yml

* Update categorization_golden_v1.yml

* Trim to 100 items

* Update auto_categorizer.rb

* FIX for auto retry in auto mode

* Separate the Eval Logic from the Auto-Categorizer

The expected_null_count parameter conflates eval-specific logic with production categorization logic.

* Force json mode on evals

* Introduce a more mixed dataset

150 items, performance from a local model:

By Difficulty:
  easy: 93.22% accuracy (55/59)
  medium: 93.33% accuracy (42/45)
  hard: 92.86% accuracy (26/28)
  edge_case: 100.0% accuracy (18/18)

* Improve datasets

Remove Data leakage from prompts

* Create eval runs as "pending"

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-12-07 18:11:34 +01:00
Josh Waldrep
5de091b14c Add tests for cross-provider holding isolation and enhance fallback logic
- Introduced `Account::ProviderImportAdapterCrossProviderTest` to validate no cross-provider claiming of holdings.
- Updated `ProviderImportAdapter` to scope fallback matching by `account_provider_id`.
- Added early conflict guard and rescue for unique index violations during imports.
- Simplified rake task usage feedback.
2025-11-22 15:24:12 -05:00
Josh Waldrep
195dd18a52 Add tests and refactor for Simplefin holdings import and processing
- Introduced tests for importer post-import logic and `SimplefinHoldingsApplyJob`.
- Refactored `ProviderImportAdapter` to improve holding resolution strategy.
- Added handling of investment and crypto holdings in importer with debounce logic for job enqueuing.
- Updated rake task to use `SimplefinHoldingsApplyJob` for holding materialization.
2025-11-22 14:41:21 -05:00
soky srm
be0b20dfd9 Lunchflow settings family (#363)
* Move provider config to family

* remove global settings

* Remove turbo auto  submit

* Fix flash location

* Fix mssing syncer for lunchflow

* Update schema.rb

* FIX tests and encryption config

* FIX make rabbit happy

* FIX run migration in SQL

* FIX turbo frame modal

* Branding fixes

* FIX rabbit

* OCD with product names

* More OCD

* No other console.log|warn in codebase

---------

Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2025-11-22 02:14:29 +01:00
LPW
61eb611529 Simplefin enhancements v2 (#267)
* SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint

# Conflicts:
#	db/schema.rb

# Conflicts:
#	app/controllers/simplefin_items_controller.rb

* fix testing

* fix linting

* xfix linting x2

* Review PR #267 on we-promise/sure (SimpleFin enhancements v2). Address all 15 actionable CodeRabbit comments: Add UUID validations in rakes (e.g., simplefin_unlink), swap Ruby pattern matching/loops for efficient DB queries (e.g., where LOWER(name) LIKE ?), generate docstrings for low-coverage areas (31%), consolidate routes for simplefin_items, move view logic to helpers (e.g., format_transaction_extra), strengthen tests with exact assertions/fixtures for dedup/relink failures. Also, check for overlaps with merged #262 (merchants fix): Ensure merchant creation in simplefin_entry/processor.rb aligns with new payee-based flow and MD5 IDs; add tests for edge cases like empty payees or over-merging pendings. Prioritize security (PII redaction in logs, no hardcoded secrets).

* SimpleFin: address CodeRabbit comments (batch 1)

- Consolidate simplefin_items routes under a single resources block; keep URLs stable
- Replace inline JS with Stimulus auto-relink controller; auto-load relink modal via global modal frame
- Improve a11y in relink modal by wrapping rows in labels
- Harden unlink rake: default dry_run=true, UUID validation, redact PII in outputs, clearer errors
- Backfill rake: default dry_run=true, UUID validation; groundwork for per-SFA counters
- Fix-was-merged rake: default dry_run=true, UUID validation; clearer outputs
- Idempotent transfer auto-match (find_or_create_by! + RecordNotUnique rescue)
- Extract SimpleFin error tooltip assembly into helper and use it in view

RuboCop: maintain 2-space indentation, spaces inside array brackets, spaces after commas, and no redundant returns

* Linter noise

* removed filed commited by mistake.

* manual relink flow and tighten composite matching

* enforce manual relink UI; fix adapter keywords; guarantee extra.simplefin hash

* refactor(simplefin): extract relink service; enforce manual relink UI; tighten composite match; migration 7.2

* add provider date parser; refactor rake; move view queries; partial resilience

* run balances-only import in background job. make update flow enqueue balances-only job

* persists across all update redirects and initialize
used_manual_ids to prevent NameError in relink candidate computation.

* SimpleFin: metadata + merge fixes; holdings (incl. crypto) + Day Change; Sync Summary; ops rakes; lint

* Fixed failed test after rebase.

* scan_ruby fix

* Calming the rabbit:
Fix AccountProvider linking when accounts change
Drop the legacy unique index instead of duplicating it
Fix dynamic constant assignment
Use fixtures consistently; avoid rescue for control flow.
Replace bare rescue with explicit exception class.
Move business logic out of the view.
Critical: Transaction boundary excludes recompute phase, risking data loss.
Inconsistency between documentation and implementation for zero-error case.
Refactor to use the compute_unlinked_count helper for consistency.
Fix cleanup task default: it deletes by default.
Move sync stats computation to controller to avoid N+1 queries.
Consolidate duplicate sync query.
Clarify the intent of setting flash notice on the error path.
Fix Date/Time comparison in should_be_inactive?.
Move stats retrieval logic to controller.
Remove duplicate Sync summary section.
Remove the unnecessary sleep statement; use Capybara's built-in waiting.
Add label wrappers for accessibility and consistency.

* FIX SimpleFIN new account modal

Now new account properly loads as a Modal, instead of new page.
Fixes also form showing dashboard instead of settings page.

* Remove SimpleFin legacy UI components, migrate schema, and refine linking behavior.

# Conflicts:
#	app/helpers/settings_helper.rb

* Extract SimpleFin-related logic to `prepare_show_context` helper and refactor for consistency. Adjust conditional checks and ensure controller variables are properly initialized.

* Remove unused SimpleFin maps from prepare_show_context; select IDs to avoid N+1
Replace Tailwind bg-green-500 with semantic bg-success in _simplefin_panel/_provider_form
Add f.label :setup_token in simplefin_items/new for a11y
Remove duplicate require in AccountsControllerSimplefinCtaTest

* Remove unnecessary blank lines

* Reduce unnecessary changes

This reduces the diff against main

* Simplefin Account Setup: Display in modal

This fixes an issue with the `X` dismiss button in the top right corner

* Removed unnecessary comment.

* removed unnecessary function.

* fixed broken links

* Removed unnecessary file

* changed to database query

* set to use UTC and gaurd against null

* set dry_run=true

* Fixed comment

* Changed to use a database-level query

* matched test name to test behavior.

* Eliminate code duplication and Time.zone dependency

* make final summary surface failures

* lint fix

* Revised timezone comment. better handle missing selectors.

* sanitized LIKE wildcards

* Fixed SimpleFin import to avoid “Currency can’t be blank” validation failures when providers return an empty currency string.

* Added helper methods for admin and self-hosted checks

* Specify exception types in rescue clauses.

* Refined logic to determine transaction dates for credit accounts.

* Refined stats calculation for `total_accounts` to track the maximum unique accounts per run instead of accumulating totals.

* Moved `unlink_all!` logic to `SimplefinItem::Unlinking` concern and deprecated `SimplefinItem::Unlinker`. Updated related references.

* Refined legacy unlinking logic, improved `current_holdings` formatting, and added ENV-based overrides for self-hosted checks.

* Enhanced `unlink_all!` with explicit error handling, improved transaction safety, and refined ENV-based self-hosted checks. Adjusted exception types and cleaned up private method handling.

* Improved currency assignment logic by adding fallback to `current_account` and `family` currencies.

* Enhanced error tracking during SimpleFin account imports by adding categorized error buckets, limiting stored errors to the last 5, and improving `stats` calculations.

* typo fix

* Didn't realize rabbit was still mad...
Refactored SimpleFin error handling and CTA logic: centralized duplicate detection and relink visibility into controller, improved task counters, adjusted redirect notices, and fixed form indexing.

* Dang rabbit never stops... Centralized SimpleFin maps logic into `MapsHelper` concern and integrated it into relevant controllers and rake tasks. Optimized queries, reduced redundancy, and improved unlinked counts and manual account checks with batch processing. Adjusted task arguments for clarity.

* Persistent rabbit. Optimized SimpleFin maps logic by implementing batch queries for manual account and unlinked count checks, reducing N+1 issues. Improved clarity of rake task argument descriptions and error messages for better usability.

* Lost a commit somehow, resolved here. Refactored transaction extra details logic by introducing `build_transaction_extra_details` helper to improve clarity, reusability, and reduce view complexity. Enhanced rake tasks with strict dry-run validation and better error handling. Updated schema to allow nullable `merchant_id` and added conditional unique indexes for recurring transactions.

* Refactored sensitive data redaction in `simplefin_unlink` task for recursive handling, optimized SQL sanitization in `simplefin_holdings_backfill`, improved error handling in `transactions_helper`, and streamlined day change calculation logic in `Holding` model.

* Lint fix

* Removed per PR comments.

* Also removing per PR comment.

* git commit -m "SimpleFIN polish: preserve #manual-accounts wrapper, unify \"manual\" scope, and correct unlinked counts
- Preserve #manual-accounts wrapper: switch non-empty updates to turbo_stream.update and background broadcast_update_to; keep empty-path replace to render <div id=\"manual-accounts\"></div>
- Unify definition of manual accounts via Account.visible_manual (visible + legacy-nil + no AccountProvider); reuse in controllers, jobs, and helper
- Correct setup/unlinked counts: SimplefinItem::Syncer#finalize_setup_counts and maps now consider AccountProvider links (legacy account AND provider must be absent)
Deleted:
- app/models/simplefin_item/relink_service.rb
- app/controllers/concerns/simplefin_items/relink_helpers.rb
- app/javascript/controllers/auto_relink_controller.js
- app/views/simplefin_items/_relink_modal.html.erb
- app/views/simplefin_items/manual_relink.html.erb
- app/views/simplefin_items/relink.html.erb
- test/services/simplefin_item/relink_service_test.rb
Refs: PR #318 unified link/unlink; PR #267 SimpleFIN; follow-up to fix wrapper ID loss and counting drift."

* Extend unlinked account check to include "Investment" type

* set SimpleFIN item for `balances`, remove redundant unpacking, and improve holdings task error

* SimpleFIN: add `errors` action + modal; do not reintroduce legacy relink actions; removed dead helper

* FIX simpleFIN linking

* Add delay back, tests benefit from it

* Put cache back in

* Remove empty `rake` task

* Small spelling fixes.

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
Co-authored-by: Josh Waldrep <joshua.waldrep5+github@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: sokie <sokysrm@gmail.com>
Co-authored-by: Dylan Corrales <deathcamel58@gmail.com>
2025-11-17 21:51:37 +01:00
soky srm
e8f935bc6f Remove plaid initialiser (#317)
* Remove plaid initialiser

The initializer can be safely removed because:
  - Config is lazily loaded via Provider::Registry
  - reload_configuration is called after settings updates
  - All calling code handles nil configs gracefully
  - Initial nil state is fine - config loads on first use

* Fix for missing config

* Actually don't pollute application.rb

* Add currency loading for balances

* Fix race condition on lazy load

* Allow loans to be imported in lunch flow also

* Fix currency processor
2025-11-12 16:01:19 +01:00
luckyPipewrench
fea228d03e Simplefin sync improvements (#240)
* Fix syncing issues with new connections and accounts..

- Keep SimpleFin institution metadata strictly per account (`simplefin_accounts.org_data`).
- Relax `simplefin_items` institution constraints to allow creating items before org data exists.
- Remove code that copied the first account’s `org` onto `simplefin_items`.

* Improve Simplefin Sync
•
SimpleFin: family “Sync” includes SimpleFin items; importer does unbounded discovery (with pending=1 fallback) before windowed fetch, for both regular and first syncs.
•
Stop populating item‑level institution fields; keep institution metadata per account.
•
Relax NOT NULL on item institution fields.
•
Post‑sync dashboard broadcasts are now guarded (UI cannot fail the job).
•
Show a friendly “daily refresh limit” banner on the SimpleFin card when the latest sync is rate‑limited.
•
Add bin/rails sure:simplefin:debug[ITEM_ID] to print latest sync, snapshot account count, simplefin_accounts count, and unlinked list.

* Fixed double‑quoted strings, spacing around array brackets and commas

* chore: ignore local .junie files

* - Broadcast error logs now include full backtraces
- SimpleFin discovery logic deduplicated fixed
- app/models/simplefin_item/importer.rb
--Added a concise docstring for perform_account_discovery describing purpose, steps, and side‑effects.
--Added a docstring for fetch_accounts_data describing params and return value.
2025-10-26 15:50:45 +01:00
Juan José Mata
5706280dd7 More rebranding changes (#159)
* Replace Maybe for Sure in select code areas

* Make sure passwords are consistent

* Remove (admin|member) from demo data first name

* Database and schema names finally to `sure`

* Fix broken test

* Another (benchmarking) database name to `sure_*`

* More rebranding to Sure

* Missed this Maybe mention in the same page

* Random nitpicks and more Maybes

* Demo data accounts and more Maybes

* Test data account updates

* Impersonation test accounts

* Consistency with `compose.example.yml`
2025-09-24 00:19:51 +02:00
Juan José Mata
099425d240 First quick Sure rebrand (#74)
* First cut of smallest rebrand, pending icons

* Leave SQL schema tokens/user names the same for now

* First batch of logos

* Release notes/what's new

* /releases missing

* redirect_uri for sureapp://

* Padded logo

* Test the correct /releases URL

* Missed a few mobile URIs

* Some icons/asssets from /website/ repo

* Seed/sample data user @sure.local now

* New screenshot

* Want to keep their legal "boilerplate" from the upstream repo
2025-08-05 23:35:01 +02:00
Juan José Mata
54bc37a651 Remove Synth Finance references (#47)
* Remove Synth Finance integration

* Linter noise

* Fix failing (old) test, use it for Twelve Data

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
2025-08-02 00:28:55 +02:00
Zach Gollwitzer
da2045dbd8 Additional cache columns on balances for activity view breakdowns (#2505)
* Initial schema iteration

* Add new balance components

* Add existing data migrator to backfill components

* Update calculator test assertions for new balance components

* Update flow assertions for forward calculator

* Update reverse calculator flows assumptions

* Forward calculator tests passing

* Get all calculator tests passing

* Assert flows factor
2025-07-23 10:06:25 -04:00
Zach Gollwitzer
c1d98fe73b Start and end balance anchors for historical account balances (#2455)
* Add kind field to valuation

* Fix schema conflict

* Add kind to valuation

* Scaffold opening balance manager

* Opening balance manager implementation

* Update account import to use opening balance manager + tests

* Update account to use opening balance manager

* Fix test assertions, usage of current balance manager

* Lint fixes

* Add Opening Balance manager, add tests to forward calculator

* Add credit card to "all cash" designation

* Simplify valuation model

* Add current balance manager with tests

* Add current balance logic to reverse calculator and plaid sync

* Tweaks to initial calc logic

* Ledger testing helper, tweak assertions for reverse calculator

* Update test assertions

* Extract balance transformer, simplify calculators

* Algo simplifications

* Final tweaks to calculators

* Cleanup

* Fix error, propagate sync errors up to parent

* Update migration script, valuation naming
2025-07-15 11:42:41 -04:00
Zach Gollwitzer
fe8aebe920 Don't raise on invalid demo data 2025-07-10 16:01:47 -04:00
Zach Gollwitzer
1aae00f586 perf(transactions): add kind to Transaction model and remove expensive Transfer joins in aggregations (#2388)
* add kind to transaction model

* Basic transfer creator

* Fix method naming conflict

* Creator form pattern

* Remove stale methods

* Tweak migration

* Remove BaseQuery, write entire query in each class for clarity

* Query optimizations

* Remove unused exchange rate query lines

* Remove temporary cache-warming strategy

* Fix test

* Update transaction search

* Decouple transactions endpoint from IncomeStatement

* Clean up transactions controller

* Update cursor rules

* Cleanup comments, logic in search

* Fix totals logic on transactions view

* Fix pagination

* Optimize search totals query

* Default to last 30 days on transactions page if no filters

* Decouple transactions list from transfer details

* Revert transfer route

* Migration reset

* Bundle update

* Fix matching logic, tests

* Remove unused code
2025-06-20 13:31:58 -04:00
Zach Gollwitzer
a5f1677f60 perf(income statement): cache income statement queries (#2371)
* Leftover cleanup from prior PR

* Benchmark convenience task

* Change default warm benchmark time

* Cache income statement queries

* Fix private method access
2025-06-15 10:09:46 -04:00
Zach Gollwitzer
84b2426e54 Benchmarking setup (#2366)
* Benchmarking setup

* Get demo data working in benchmark scenario

* Finalize default demo scenario

* Finalize benchmarking setup
2025-06-14 11:53:53 -04:00
Zach Gollwitzer
5a4c955522 Realistic demo data for performance testing (#2361)
* Realistic demo data for performance testing

* Add note about performance testing

* Fix bugbot issues

* More realistic account values
2025-06-11 18:48:39 -04:00
Zach Gollwitzer
fcdc42760d Tweak dup securities data migration 2025-05-22 16:02:34 -04:00
Zach Gollwitzer
fe24117c50 Stronger security unique index and data migration
Note to self hosters:

If you started self hosting prior to this commit, you may have duplicate securities in your database.

This is usually not a problem, but if you'd like to clean things up, you can run the data migration
by opening a terminal on the machine you're hosting with and running:

```sh
rake data_migration:migrate_duplicate_securities
```
2025-05-22 15:15:07 -04:00
Zach Gollwitzer
df8e22afe9 Stripe tasks 2025-05-13 08:56:32 -04:00
Zach Gollwitzer
a51c4d2cba New onboarding, trials, Stripe integration (#2185)
* New onboarding, trials, Stripe integration

* Fix tests

* Lint fixes

* Fix subscription endpoints
2025-05-01 16:47:14 -04:00
Zach Gollwitzer
90a9546f32 Pre-launch design sync with Figma spec (#2154)
* Add lookbook + viewcomponent, organize design system file

* Build menu component

* Button updates

* More button fixes

* Replace all menus with new ViewComponent

* Checkpoint: fix tests, all buttons and menus converted

* Split into Link and Button components for clarity

* Button cleanup

* Simplify custom confirmation configuration in views

* Finalize button, link component API

* Add toggle field to custom form builder + Component

* Basic tabs component

* Custom tabs, convert all menu / tab instances in app

* Gem updates

* Centralized icon helper

* Update all icon usage to central helper

* Lint fixes

* Centralize all disclosure instances

* Dialog replacements

* Consolidation of all dialog styles

* Test fixes

* Fix app layout issues, move to component with slots

* Layout simplification

* Flakey test fix

* Fix dashboard mobile issues

* Finalize homepage

* Lint fixes

* Fix shadows and borders in dark mode

* Fix tests

* Remove stale class

* Fix filled icon logic

* Move transparent? to public interface
2025-04-30 18:14:22 -04:00
Zach Gollwitzer
e657c40d19 Account:: namespace simplifications and cleanup (#2110)
* Flatten Holding model

* Flatten balance model

* Entries domain renames

* Fix valuations reference

* Fix trades stream

* Fix brakeman warnings

* Fix tests

* Replace existing entryable type references in DB
2025-04-14 11:40:34 -04:00
Zach Gollwitzer
939244bd3e Use faraday retry, move retry logic to concrete provider level (#2042) 2025-04-01 08:41:49 -04:00
Zach Gollwitzer
f65b93a352 Data provider simplification, tests, and documentation (#1997)
* Ignore env.test from source control

* Simplification of providers interface

* Synth tests

* Update money to use new find rates method

* Remove unused issues code

* Additional issue feature removals

* Update price data fetching and tests

* Update documentation for providers

* Security test fixes

* Fix self host test

* Update synth usage data access

* Remove AI pr schema changes
2025-03-17 11:54:53 -04:00