Commit Graph

15 Commits

Author SHA1 Message Date
soky srm
5750e69acf Provider investment fixes (#600)
* FIX issue with stock price retrieval on weekend

* make weekend provisional and increase lookback

* FIX query error

* fix gap fill

The bug: When a price is provisional but the provider doesn't return a new value (weekends), we fall back to the existing DB value instead of gap-filling from Friday's correct price.

* Update importer.rb

Align provider fetch to use PROVISIONAL_LOOKBACK_DAYS for consistency. In the DB fallback, derive currency from provider_prices or db_prices and filter the query accordingly.

* Update 20260110122603_mark_suspicious_prices_provisional.rb

* Delete db/migrate/20260110122603_mark_suspicious_prices_provisional.rb

Signed-off-by: soky srm <sokysrm@gmail.com>

* Update importer.rb

* FIX tests

* FIX last tests

* Update importer_test.rb

The test doesn't properly force effective_start_date to skip old dates because there are many missing dates between the old date and recent dates. Let me fix it to properly test the subset processing scenario.

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
2026-01-10 15:43:07 +01:00
soky srm
4dfd2913c7 Investment prices fixes (#559)
* Fix investments retrieval

     Problem Summary

     Stock prices for securities like European stocks become stale because:
     1. sync_all_accounts runs at 2:22 UTC (before European markets open)
     2. Provider doesn't have today's price yet, so importer gap-fills with LOCF (yesterday's price)
     3. Later import_market_data at 22:00 UTC sees all prices exist and skips fetching
     4. Real closing price is never retrieved

     Solution Overview

     Add a provisional boolean column to mark gap-filled prices that should be re-fetched.

* Update schema.rb

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-07 16:16:01 +01:00
soky srm
0290ed636f FIX holdings and exchange rate (#345)
* FIX holdings and exchange rate

- Fix holdings not showing total return instead showing day change %
- Fix exchange rate saving 0 for holidays, use LOCF - Last Observation Carried Forward

* Fix failing test
2025-11-17 15:18:20 +01:00
Zach Gollwitzer
ab5bce3462 Fix provider guards for start price 2025-05-19 15:19:41 -04:00
Zach Gollwitzer
29a8ac9d8a Tweak exception logging, sync stale behavior 2025-05-18 10:19:15 -04:00
Zach Gollwitzer
10f255a9a9 Clarify backend data pipeline naming concepts (importers, processors, materializers, calculators, and syncers) (#2255)
* Rename MarketDataSyncer to MarketDataImporter

* Materializers

* Importers

* More reference replacements
2025-05-17 16:37:16 -04:00
Zach Gollwitzer
35d1447494 Adjust Sentry missing price and rate to warning level 2025-05-16 14:25:35 -04:00
Zach Gollwitzer
6dc1d22672 Market data sync refinements (#2252)
* Exchange rate syncer implementation

* Security price syncer

* Fix issues with provider API

* Add back prod schedule

* Add back price and exchange rate syncs to account syncs

* Remove unused stock_exchanges table
2025-05-16 14:17:56 -04:00
Zach Gollwitzer
f65b93a352 Data provider simplification, tests, and documentation (#1997)
* Ignore env.test from source control

* Simplification of providers interface

* Synth tests

* Update money to use new find rates method

* Remove unused issues code

* Additional issue feature removals

* Update price data fetching and tests

* Update documentation for providers

* Security test fixes

* Fix self host test

* Update synth usage data access

* Remove AI pr schema changes
2025-03-17 11:54:53 -04:00
Zach Gollwitzer
5f8a3c9f50 Search securities with correct exchange mic 2025-03-07 17:48:26 -05:00
Zach Gollwitzer
fa0248056d Show UI warning to user when they need provider data but have not setup Synth yet (#1926)
* Simplify provider concerns

* Update tests

* Add UI warning for missing Synth key if family requires external data
2025-02-28 11:35:10 -05:00
Josh Pigford
68d7cb5de6 Enhance security information retrieval and handling (#1826)
* Enhance security information retrieval and handling

- Add support for operating MIC codes in security info fetching
- Update security uniqueness validation to handle unknown securities
- Improve security creation and update logic in Plaid investment sync
- Update combobox and view components to handle operating MIC codes
- Add unknown flag for securities with incomplete information

* Update schema.rb

* Refactor the need for mic codes

* Don't fetch prices unless a security has the necessary mic code

* Deduplication

* Lint

* Update Securities and Plaid Investment Sync

- Modify PlaidInvestmentSync to return plaid_security for USD cash
- Add non-null constraint to Securities ticker column
- Update Securities fixture to use exchange_operating_mic instead of exchange_mic

---------

Signed-off-by: Josh Pigford <josh@joshpigford.com>
2025-02-11 10:40:30 -06:00
Josh Pigford
490f44589e First pass at security price reference (#1388)
* First pass at security price reference

* Data cleanup

* Synth security fetching does better with a mic_code

* Update test suite

😭

* Update schema.rb

* Update generator.rb
2024-10-29 15:37:59 -04:00
Zach Gollwitzer
3399b74849 Handle market holidays during holding sync (#1292)
* Handle market holidays during holding sync

* Use informal holidays instead of custom override
2024-10-10 18:02:12 -04:00
Zach Gollwitzer
453a54e5e6 Add security prices provider (Synth integration) (#1039)
* User tickers as primary lookup symbol instead of isin

* Add security price provider

* Fetch security prices in bulk to improve sync performance

* Fetch prices in bulk, better mocking for tests
2024-08-01 19:43:23 -04:00