Commit Graph

25 Commits

Author SHA1 Message Date
LPW
c504ba9b99 Add security remapping for holdings with sync protection (#692)
* Add security remapping support to holdings

- Introduced `provider_security` tracking for holdings with schema updates.
- Implemented security remap/reset workflows in `Holding` model and UI.
- Updated routes, controllers, and tests to support new functionality.
- Enhanced client-side interaction with Stimulus controller for remapping.

# Conflicts:
#	app/components/UI/account/activity_feed.html.erb
#	db/schema.rb

* Refactor "New transaction" to "New activity" across UI and tests

- Updated localized strings, button labels, and ARIA attributes.
- Improved error handling in holdings' current price display.
- Scoped fallback queries in `provider_import_adapter` to prevent overwrites.
- Added safeguard for offline securities in price fetching logic.

* Update security remapping to merge holdings on collision by deleting duplicates

- Removed error handling for collisions in `remap_security!`.
- Added logic to merge holdings by deleting duplicates on conflicting dates.
- Modified associated test to validate merging behavior.

* Update security remapping to merge holdings on collision by combining qty and amount

- Modified `remap_security!` to merge holdings by summing `qty` and `amount` on conflicting dates.
- Adjusted logic to calculate `price` for merged holdings.
- Updated test to validate new merge behavior.

* Improve DOM handling in Turbo redirect action & enhance holdings merge logic

- Updated Turbo's custom `redirect` action to use the "replace" option for cleaner DOM updates without clearing the cache.
- Enhanced holdings merge logic to calculate weighted average cost basis during security remapping, ensuring more accurate cost_basis updates.

* Track provider_security_id during security updates to support reset workflows

* Fix provider tracking: guard nil ticker lookups and preserve merge attrs

- Guard fallback 1b lookup when security.ticker is blank to avoid matching NULL tickers
- Preserve external_id, provider_security_id, account_provider_id during collision merge

* Fix schema.rb version after merge (includes tax_treatment migration)

* fix: Rename migration to run after schema version

The migration 20260117000001 was skipped in CI because it had a timestamp
earlier than the schema version (2026_01_17_200000). CI loads schema.rb
directly and only runs migrations with versions after the schema version.

Renamed to 20260119000001 so it runs correctly.

* Update schema: remove Coinbase tables, add new fields and indexes

* Update schema: add back `tax_treatment` field with default value "taxable"

* Improve Turbo redirect action: use "replace" to avoid form submission in history

* Lock merged holdings to prevent provider overwrites and fix activity feed template indentation

* Refactor holdings transfer logic: enforce currency checks during collisions and enhance merge handling

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-23 12:54:55 +01:00
LPW
237035c8d4 Improve convert-to-trade security selection with search-first UX (#703)
* Enhance security handling logic:
- Prioritize user's country in sorting securities and country codes.
- Add comprehensive mapping for MIC codes to user-friendly exchange names.
- Revamp combobox to consistently pull from a provider when available.
- Improve handling of custom ticker and exchange input fields.

* Localize securities combobox display and exchange labels.

---------

Co-authored-by: luckyPipewrench <luckypipewrench@proton.me>
2026-01-19 15:49:51 +01:00
soky srm
76dc91377c Merchants improvements (#594)
* FIX logos

* Implement merchant mods

* FIX confirm issue

* FIX linter

* Add recently seen merchants to re-add if needed

* Update merge.html.erb

* FIX do security check

* Add error handling for update failures.
2026-01-09 19:38:04 +01:00
soky srm
0d798c522a Add support for provider selection (#346) 2025-11-17 18:13:20 +01:00
Vincent Teo
5bdefe6e63 feat: Add Twelve Data provider for exchange rates and securities (#2)
* feat: Add Twelve Data provider for exchange rates and securities

* test: fix hosting controller test, linting

* fix: add countries gem to handle country codes in Twelve Data provider

* fix: allow security search combobox to have no logo

* refactor: update Twelve Data provider use time series endpoint

* fix: set twelve data as default provider
2025-07-31 23:31:37 +02:00
Kenrick Tandrian
cea49d5038 fix(models): use self.id (#2410) 2025-06-24 11:15:28 -04:00
Zach Gollwitzer
e4ee06c9f6 Security resolver and health checker (#2281)
* Setup health check

* Security health checker cron

* Use resolver throughout codebase

* Use resolver for trade builder

* Add security health checks to schedule

* Handle no provider

* Lint fixes
2025-05-22 12:43:24 -04:00
Josh Pigford
a7438e5c78 Add country_code attribute to Security model and related classes
* Update Security model to include country_code in the data definition.
* Modify Provider::SecurityConcept to define country_code for security.
* Enhance Provider::Synth to extract country_code from security data.
* Update Security::Provided to include country_code when creating security instances.
* Adjust Security::SynthComboboxOption to add country_code as an attribute.
* Revise combobox_security partial to conditionally display country flag and code.
2025-05-22 09:45:08 -05:00
Zach Gollwitzer
f82f77466a Fix Sentry context for security details exception 2025-05-18 10:52:35 -04:00
Zach Gollwitzer
74c7b0941d More exception logging tweaks 2025-05-18 10:27:46 -04:00
Zach Gollwitzer
10f255a9a9 Clarify backend data pipeline naming concepts (importers, processors, materializers, calculators, and syncers) (#2255)
* Rename MarketDataSyncer to MarketDataImporter

* Materializers

* Importers

* More reference replacements
2025-05-17 16:37:16 -04:00
Zach Gollwitzer
6dc1d22672 Market data sync refinements (#2252)
* Exchange rate syncer implementation

* Security price syncer

* Fix issues with provider API

* Add back prod schedule

* Add back price and exchange rate syncs to account syncs

* Remove unused stock_exchanges table
2025-05-16 14:17:56 -04:00
Zach Gollwitzer
10dd9e061a Improve account sync performance, handle concurrent market data syncing (#2236)
* PlaidConnectable concern

* Remove bad abstraction

* Put sync implementations in own concerns

* Sync strategies

* Move sync orchestration to Sync class

* Clean up sync class, add state machine

* Basic market data sync cron

* Fix price sync

* Improve sync window column names, add timestamps

* 30 day syncs by default

* Clean up market data methods

* Report high duplicate sync counts to Sentry

* Add sync states throughout app

* account tab session

* Persistent account tab selections

* Remove manual sleep

* Add migration to clear stale syncs on self hosted apps

* Tweak sync states

* Sync completion event broadcasts

* Fix timezones in tests

* Cleanup

* More cleanup

* Plaid item UI broadcasts for sync

* Fix account ID namespace conflict

* Sync broadcasters

* Smoother account sync refreshes

* Remove test sync delay
2025-05-15 10:19:56 -04:00
Zach Gollwitzer
a268c5a563 Revert "Add env to toggle provider price syncs"
This reverts commit 0006b6f6ca.
2025-05-09 17:47:35 -04:00
Zach Gollwitzer
0006b6f6ca Add env to toggle provider price syncs 2025-05-09 16:59:23 -04:00
Zach Gollwitzer
48a07d6158 Revert batch upserting 2025-05-09 16:42:44 -04:00
Zach Gollwitzer
5d798fe0a0 Remove retry logic from security upsert 2025-05-09 16:31:16 -04:00
Zach Gollwitzer
f07c41821e Add warn log for security price upsert retries 2025-05-09 16:05:10 -04:00
Zach Gollwitzer
7605b0221d Batch upsert security prices on sync 2025-05-09 15:56:48 -04:00
Joseph Ho
d86ccd36b6 provider: Ensure data provider exist before fetching for price. (#2045) 2025-04-04 12:14:28 -04:00
Zach Gollwitzer
29f445d75e Fix security search 2025-03-28 17:34:29 -04:00
Zach Gollwitzer
2f6b11c18f Personal finance AI (v1) (#2022)
* AI sidebar

* Add chat and message models with associations

* Implement AI chat functionality with sidebar and messaging system

- Add chat and messages controllers
- Create chat and message views
- Implement chat-related routes
- Add message broadcasting and user interactions
- Update application layout to support chat sidebar
- Enhance user model with initials method

* Refactor AI sidebar with enhanced chat menu and interactions

- Update sidebar layout with dynamic width and improved responsiveness
- Add new chat menu Stimulus controller for toggling between chat and chat list views
- Improve chat list display with recent chats and empty state
- Extract AI avatar to a partial for reusability
- Enhance message display and interaction styling
- Add more contextual buttons and interaction hints

* Improve chat scroll behavior and message styling

- Refactor chat scroll functionality with Stimulus controller
- Optimize message scrolling in chat views
- Update message styling for better visual hierarchy
- Enhance chat container layout with flex and auto-scroll
- Simplify message rendering across different chat views

* Extract AI avatar to a shared partial for consistent styling

- Refactor AI avatar rendering across chat views
- Replace hardcoded avatar markup with a reusable partial
- Simplify avatar display in chats and messages views

* Update sidebar controller to handle right panel width dynamically

- Add conditional width class for right sidebar panel
- Ensure consistent sidebar toggle behavior for both left and right panels
- Use specific width class for right panel (w-[375px])

* Refactor chat form and AI greeting with flexible partials

- Extract message form to a reusable partial with dynamic context support
- Create flexible AI greeting partial for consistent welcome messages
- Simplify chat and sidebar views by leveraging new partials
- Add support for different form scenarios (chat, new chat, sidebar)
- Improve code modularity and reduce duplication

* Add chat clearing functionality with dynamic menu options

- Implement clear chat action in ChatsController
- Add clear chat route to support clearing messages
- Update AI sidebar with dropdown menu for chat actions
- Preserve system message when clearing chat
- Enhance chat interaction with new menu options

* Add frontmatter to project structure documentation

- Create initial frontmatter for structure.mdc file
- Include description and configuration options
- Prepare for potential dynamic documentation rendering

* Update general project rules with additional guidelines

- Add rule for using `Current.family` instead of `current_family`
- Include new guidelines for testing, API routes, and solution approach
- Expand project-specific rules for more consistent development practices

* Add OpenAI gem and AI-friendly data representations

- Add `ruby-openai` gem for AI integration
- Implement `to_ai_readable_hash` methods in BalanceSheet and IncomeStatement
- Include Promptable module in both models
- Add savings rate calculation method in IncomeStatement
- Prepare financial models for AI-powered insights and interactions

* Enhance AI Financial Assistant with Advanced Querying and Debugging Capabilities

- Implement comprehensive AI financial query system with function-based interactions
- Add detailed debug logging for AI responses and function calls
- Extend BalanceSheet and IncomeStatement models with AI-friendly methods
- Create robust error handling and fallback mechanisms for AI queries
- Update chat and message views to support debug mode and enhanced rendering
- Add AI query routes and initial test coverage for financial assistant

* Refactor AI sidebar and chat layout with improved structure and comments

- Remove inline AI chat from application layout
- Enhance AI sidebar with more semantic HTML structure
- Add descriptive comments to clarify different sections of chat view
- Improve flex layout and scrolling behavior in chat messages container
- Optimize message rendering with more explicit class names and structure

* Add Markdown rendering support for AI chat messages

- Implement `markdown` helper method in ApplicationHelper using Redcarpet
- Update message view to render AI messages with Markdown formatting
- Add comprehensive Markdown rendering options (tables, code blocks, links)
- Enhance AI Financial Assistant prompt to encourage Markdown usage
- Remove commented Markdown CSS in Tailwind application stylesheet

* Missing comma

* Enhance AI response processing with chat history context

* Improve AI debug logging with payload size limits and internal message flag

* Enhance AI chat interaction with improved thinking indicator and scrolling behavior

* Add AI consent and enable/disable functionality for AI chat

* Upgrade Biome and refactor JavaScript template literals

- Update @biomejs/biome to latest version with caret (^) notation
- Refactor AI query and chat controllers to use template literals
- Standardize npm scripts formatting in package.json

* Add beta testing usage note to AI consent modal

* Update test fixtures and configurations for AI chat functionality

- Add family association to chat fixtures and tests
- Set consistent password digest for test users
- Enable AI for test users
- Add OpenAI access token for test environment
- Update chat and user model tests to include family context

* Simplify data model and get tests passing

* Remove structure.mdc from version control

* Integrate AI chat styles into existing prose pattern

* Match Figma design spec, implement Turbo frames and actions for chats controller

* AI rules refresh

* Consolidate Stimulus controllers, thinking state, controllers, and views

* Naming, domain alignment

* Reset migrations

* Improve data model to support tool calls and message types

* Tool calling tests and fixtures

* Tool call implementation and test

* Get assistant test working again

* Test updates

* Process tool calls within provider

* Chat UI back to working state again

* Remove stale code

* Tests passing

* Update openai class naming to avoid conflicts

* Reconfigure test env

* Rebuild gemfile

* Fix naming conflicts for ChatResponse

* Message styles

* Use OpenAI conversation state management

* Assistant function base implementation

* Add back thinking messages, clean up error handling for chat

* Fix sync error when security price has bad data from provider

* Add balance sheet function to assistant

* Add better function calling error visibility

* Add income statement function

* Simplify and clean up "thinking" interactions with Turbo frames

* Remove stale data definitions from functions

* Ensure VCR fixtures working with latest code

* basic stream implementation

* Get streaming working

* Make AI sidebar wider when left sidebar is collapsed

* Get tests working with streaming responses

* Centralize provider error handling

* Provider data boundaries

---------

Co-authored-by: Josh Pigford <josh@joshpigford.com>
2025-03-28 13:08:22 -04:00
Zach Gollwitzer
f65b93a352 Data provider simplification, tests, and documentation (#1997)
* Ignore env.test from source control

* Simplification of providers interface

* Synth tests

* Update money to use new find rates method

* Remove unused issues code

* Additional issue feature removals

* Update price data fetching and tests

* Update documentation for providers

* Security test fixes

* Fix self host test

* Update synth usage data access

* Remove AI pr schema changes
2025-03-17 11:54:53 -04:00
Zach Gollwitzer
cf59fe45e7 Fix ticker filling when Synth is connected (#1950) 2025-03-05 09:30:47 -05:00
Zach Gollwitzer
fa0248056d Show UI warning to user when they need provider data but have not setup Synth yet (#1926)
* Simplify provider concerns

* Update tests

* Add UI warning for missing Synth key if family requires external data
2025-02-28 11:35:10 -05:00