Commit Graph

4 Commits

Author SHA1 Message Date
soky srm
5750e69acf Provider investment fixes (#600)
* FIX issue with stock price retrieval on weekend

* make weekend provisional and increase lookback

* FIX query error

* fix gap fill

The bug: When a price is provisional but the provider doesn't return a new value (weekends), we fall back to the existing DB value instead of gap-filling from Friday's correct price.

* Update importer.rb

Align provider fetch to use PROVISIONAL_LOOKBACK_DAYS for consistency. In the DB fallback, derive currency from provider_prices or db_prices and filter the query accordingly.

* Update 20260110122603_mark_suspicious_prices_provisional.rb

* Delete db/migrate/20260110122603_mark_suspicious_prices_provisional.rb

Signed-off-by: soky srm <sokysrm@gmail.com>

* Update importer.rb

* FIX tests

* FIX last tests

* Update importer_test.rb

The test doesn't properly force effective_start_date to skip old dates because there are many missing dates between the old date and recent dates. Let me fix it to properly test the subset processing scenario.

---------

Signed-off-by: soky srm <sokysrm@gmail.com>
2026-01-10 15:43:07 +01:00
soky srm
4dfd2913c7 Investment prices fixes (#559)
* Fix investments retrieval

     Problem Summary

     Stock prices for securities like European stocks become stale because:
     1. sync_all_accounts runs at 2:22 UTC (before European markets open)
     2. Provider doesn't have today's price yet, so importer gap-fills with LOCF (yesterday's price)
     3. Later import_market_data at 22:00 UTC sees all prices exist and skips fetching
     4. Real closing price is never retrieved

     Solution Overview

     Add a provisional boolean column to mark gap-filled prices that should be re-fetched.

* Update schema.rb

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
2026-01-07 16:16:01 +01:00
Zach Gollwitzer
10f255a9a9 Clarify backend data pipeline naming concepts (importers, processors, materializers, calculators, and syncers) (#2255)
* Rename MarketDataSyncer to MarketDataImporter

* Materializers

* Importers

* More reference replacements
2025-05-17 16:37:16 -04:00
Zach Gollwitzer
6dc1d22672 Market data sync refinements (#2252)
* Exchange rate syncer implementation

* Security price syncer

* Fix issues with provider API

* Add back prod schedule

* Add back price and exchange rate syncs to account syncs

* Remove unused stock_exchanges table
2025-05-16 14:17:56 -04:00