mirror of
https://github.com/we-promise/sure.git
synced 2026-08-05 00:22:17 +00:00
* Add RentCast and Realie AVM providers for property valuation Adds Automated Valuation Model (AVM) provider support so self-hosted users can create property accounts from a US address lookup instead of entering details manually: - Provider::Rentcast and Provider::Realie clients, each fetching the property record (type, year built, square footage) and value estimate in a single API request, registered under a new :property_valuations registry concept - API key fields (encrypted, ENV-overridable) with monthly usage display in Settings > Self-Hosting under "Property Valuation Providers" - New property flow: when a provider key is configured, the method selector offers "Add via RentCast/Realie" alongside manual entry; the lookup form reuses the manual flow's localized address fields and creates the account active with fetched attributes, valuation as balance, and the address saved - SyncPropertyValuationsJob refreshes linked property valuations once a day (never hourly) via config/schedule.yml; each provider enforces its monthly request cap (RentCast 50, Realie 25) with a calendar-month counter shared between creation lookups and refreshes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Allow ENV override of AVM provider monthly request caps The RentCast (50/month) and Realie (25/month) budgets default to the free tier limits but can now be raised for paid plans via RENTCAST_MAX_REQUESTS_PER_MONTH / REALIE_MAX_REQUESTS_PER_MONTH, mirroring the AlphaVantage and Tiingo request limit overrides. The settings descriptions and usage display reflect the effective cap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address review feedback: durable request counters, safer refresh job - Move monthly AVM request counters from Rails.cache to a new provider_request_counts table with atomic upsert increments, so the hard budget caps survive cache eviction, restarts, and Redis flushes - SyncPropertyValuationsJob: only mark a property synced when the balance update succeeds (wrapped in a transaction), process stalest valuations first so a tight budget goes where it matters, and report failures via DebugLogEntry.capture instead of Rails.logger - Realie: reject lookups whose returned city/ZIP contradict the entered address (street+state queries can match the wrong city); document that numeric use codes are unpublished and leave the subtype unset - Add Faraday open/request timeouts to both provider clients - Localize all user-facing provider error messages (config/locales/models/provider/en.yml) - Add a check constraint restricting properties.avm_provider to known providers - Use the min-h-80 scale token instead of min-h-[320px] - Tests: exact-argument expectations on the lookup stub, provider stubs in the no-provider test, RentCast/Realie API key update tests, ProviderRequestCount unit tests, failed-balance regression test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Stub AVM provider registry lookups in settings system test Settings::HostingsController#show now resolves the RentCast and Realie providers for usage display; the system test's partial get_provider stubbing treats the new lookups as unexpected invocations without these. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address second-round review: candidate scan, provider reuse, shared row partial - Realie: when the address lookup returns multiple candidates, pick the first one consistent with the entered city/ZIP instead of judging only the first array element; the location-mismatch error now only fires when no candidate matches - SyncPropertyValuationsJob: resolve one provider instance per key for the whole run, so the per-instance request throttle actually spaces requests across properties instead of resetting on each iteration - Extract the AVM method selector row into a shared partial so the manual and provider options render one shape Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add retry middleware and sync index from review feedback - Both AVM clients now retry transient connection failures (same Faraday retry config as Tiingo) so a network blip doesn't burn one of the tight monthly budget's requests - Partial index on properties(avm_provider, avm_last_synced_on) backing the daily sync job's filter and stalest-first ordering Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Key the AVM sync index on avm_last_synced_on The daily job orders by avm_last_synced_on ASC NULLS FIRST across all AVM-linked properties, so leading the partial index with avm_provider prevented it from serving the sort. Rekeyed on avm_last_synced_on with matching null ordering; the partial predicate still covers the filter. Amended in place since the migration is unmerged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Map Realie numeric use codes; skip incomplete addresses in sync job - Realie documents its numeric use codes at docs.realie.ai/api-reference/feature-key (earlier skip reason was wrong — the table exists). Map the residential/land/agricultural codes to Property subtypes, leaving codes without a subtype equivalent (e.g. 1006 mobile/manufactured) unset like the RentCast mapping. - The daily job now skips properties whose address is missing a street or state before resolving a provider, logging a warn-level DebugLogEntry, so a malformed address can't burn a monthly-budget request every day. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Guard currency mismatches in refresh; reset AVM keys in test teardown - The daily job now skips (with a warn-level DebugLogEntry) properties whose account currency no longer matches the provider's valuation currency, checked via the concept's valuation_currency before spending a request — writing a USD valuation into a re-currencied account would corrupt the balance - Hostings controller test teardown now clears rentcast_api_key and realie_api_key alongside the other cached global settings Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Validate AVM lookup inputs before spending a provider request The form marks name and address fields required, but a forged or JS-less submission bypasses that and would burn one of the tight monthly-budget requests on a lookup that can't produce a property. Property::AvmImport now validates name and the full address locally (localized error) before calling the provider. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add preview-and-confirm step to the AVM property lookup RentCast's AVM endpoint is location-based: a plausible but nonexistent address still geocodes and returns an area-derived estimate with no property record behind it. Instead of silently creating an account from that, the lookup now shows what the provider returned — type, year built, area (each "Unknown" when missing), and the estimated value — with an explicit notice when no property record was found, and only the user's confirmation creates the account. The confirm step reuses the fetched data via the form (sanitized server-side), so the flow still costs exactly one provider request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Require a complete US address before daily valuation refresh The job's guard only required street and state, but a later-blanked city or ZIP silently disables the Realie wrong-city check (blank entered fields count as "no mismatch"), so a refresh could accept another property's valuation. The guard now mirrors the import-time completeness check and also skips addresses edited to a non-US country. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Sign the AVM preview payload; blur valuation in privacy mode - The confirm step now rebuilds everything from a signed, 1-hour message-verifier token generated at lookup time, proving the lookup (and its counted provider request) actually ran — a direct confirm POST with fabricated data can no longer create provider-linked properties that the daily refresh job would then spend quota on. Forged/expired tokens re-render the lookup form with a clear error. - The preview's market value now carries the privacy-sensitive class so privacy mode blurs it like other account amounts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Treat zero Realie valuations as absent; bind preview token to family Review follow-ups from PR #2727: - Realie returns modelValue: 0 when it can't produce an AVM estimate; zero now falls back to the assessed market value instead of syncing a $0 balance, and an all-zero record errors out. - Monetary values now parse via BigDecimal(value.to_s) rather than Float#to_d, avoiding binary float rounding artifacts in balances. - The signed AVM preview token's purpose is scoped to the family that ran the lookup, so a leaked token can't be replayed cross-family. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Refresh AVM property valuations monthly instead of daily Per @jjmata's review on #2727: valuations change slowly and providers enforce tight monthly request caps, so a daily cron adds no value. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
217 lines
5.8 KiB
Ruby
217 lines
5.8 KiB
Ruby
class Provider::Registry
|
|
include ActiveModel::Validations
|
|
|
|
Error = Class.new(StandardError)
|
|
|
|
CONCEPTS = %i[exchange_rates securities llm property_valuations]
|
|
|
|
validates :concept, inclusion: { in: CONCEPTS }
|
|
|
|
class << self
|
|
def for_concept(concept)
|
|
new(concept.to_sym)
|
|
end
|
|
|
|
def get_provider(name)
|
|
send(name)
|
|
rescue NoMethodError
|
|
raise Error.new("Provider '#{name}' not found in registry")
|
|
end
|
|
|
|
# Resolves the LLM provider for batch/PDF flows, honoring Setting.llm_provider
|
|
# the way chat does: prefer the configured provider, but fall back to whichever
|
|
# one actually has credentials so an install that swaps providers (or has only
|
|
# one configured) keeps working. Returns nil when neither is configured —
|
|
# callers guard on that.
|
|
def preferred_llm_provider
|
|
order = Setting.llm_provider == "anthropic" ? %i[anthropic openai] : %i[openai anthropic]
|
|
order.each do |name|
|
|
provider = get_provider(name)
|
|
return provider if provider
|
|
end
|
|
nil
|
|
end
|
|
|
|
def plaid_provider_for_region(region)
|
|
region.to_sym == :us ? plaid_us : plaid_eu
|
|
end
|
|
|
|
private
|
|
def stripe
|
|
secret_key = ENV["STRIPE_SECRET_KEY"]
|
|
webhook_secret = ENV["STRIPE_WEBHOOK_SECRET"]
|
|
|
|
return nil unless secret_key.present? && webhook_secret.present?
|
|
|
|
Provider::Stripe.new(secret_key:, webhook_secret:)
|
|
end
|
|
|
|
def twelve_data
|
|
api_key = ENV["TWELVE_DATA_API_KEY"].presence || Setting.twelve_data_api_key
|
|
|
|
return nil unless api_key.present?
|
|
|
|
Provider::TwelveData.new(api_key)
|
|
end
|
|
|
|
def plaid_us
|
|
Provider::PlaidAdapter.ensure_configuration_loaded
|
|
config = Rails.application.config.plaid
|
|
|
|
return nil unless config.present?
|
|
|
|
Provider::Plaid.new(config, region: :us)
|
|
end
|
|
|
|
def plaid_eu
|
|
Provider::PlaidEuAdapter.ensure_configuration_loaded
|
|
config = Rails.application.config.plaid_eu
|
|
|
|
return nil unless config.present?
|
|
|
|
Provider::Plaid.new(config, region: :eu)
|
|
end
|
|
|
|
def github
|
|
Provider::Github.new
|
|
end
|
|
|
|
def openai
|
|
access_token = ENV["OPENAI_ACCESS_TOKEN"].presence || Setting.openai_access_token
|
|
|
|
return nil unless access_token.present?
|
|
|
|
uri_base = ENV["OPENAI_URI_BASE"].presence || Setting.openai_uri_base
|
|
model = ENV["OPENAI_MODEL"].presence || Setting.openai_model
|
|
|
|
if uri_base.present? && model.blank?
|
|
Rails.logger.error("Custom OpenAI provider configured without a model; please set OPENAI_MODEL or Setting.openai_model")
|
|
return nil
|
|
end
|
|
|
|
Provider::Openai.new(access_token, uri_base: uri_base, model: model)
|
|
end
|
|
|
|
def anthropic
|
|
access_token = ENV["ANTHROPIC_ACCESS_TOKEN"].presence ||
|
|
ENV["ANTHROPIC_API_KEY"].presence ||
|
|
Setting.anthropic_access_token
|
|
|
|
return nil unless access_token.present?
|
|
|
|
base_url = ENV["ANTHROPIC_BASE_URL"].presence || Setting.anthropic_base_url
|
|
model = ENV["ANTHROPIC_MODEL"].presence || Setting.anthropic_model
|
|
|
|
Provider::Anthropic.new(access_token, base_url: base_url, model: model)
|
|
end
|
|
|
|
def yahoo_finance
|
|
Provider::YahooFinance.new
|
|
end
|
|
|
|
def tiingo
|
|
api_key = ENV["TIINGO_API_KEY"].presence || Setting.tiingo_api_key # pipelock:ignore
|
|
|
|
return nil unless api_key.present?
|
|
|
|
Provider::Tiingo.new(api_key)
|
|
end
|
|
|
|
def eodhd
|
|
api_key = ENV["EODHD_API_KEY"].presence || Setting.eodhd_api_key # pipelock:ignore
|
|
|
|
return nil unless api_key.present?
|
|
|
|
Provider::Eodhd.new(api_key)
|
|
end
|
|
|
|
def alpha_vantage
|
|
api_key = ENV["ALPHA_VANTAGE_API_KEY"].presence || Setting.alpha_vantage_api_key # pipelock:ignore
|
|
|
|
return nil unless api_key.present?
|
|
|
|
Provider::AlphaVantage.new(api_key)
|
|
end
|
|
|
|
def mfapi
|
|
Provider::Mfapi.new
|
|
end
|
|
|
|
def binance_public
|
|
Provider::BinancePublic.new
|
|
end
|
|
|
|
def moex_public
|
|
Provider::MoexPublic.new
|
|
end
|
|
|
|
def frankfurter
|
|
Provider::Frankfurter.new
|
|
end
|
|
|
|
def tinkoff_invest
|
|
api_key = ENV["TINKOFF_INVEST_API_KEY"].presence || Setting.tinkoff_invest_api_key # pipelock:ignore
|
|
|
|
return nil unless api_key.present?
|
|
|
|
Provider::TinkoffInvest.new(api_key)
|
|
end
|
|
|
|
def rentcast
|
|
api_key = ENV["RENTCAST_API_KEY"].presence || Setting.rentcast_api_key # pipelock:ignore
|
|
|
|
return nil unless api_key.present?
|
|
|
|
Provider::Rentcast.new(api_key)
|
|
end
|
|
|
|
def realie
|
|
api_key = ENV["REALIE_API_KEY"].presence || Setting.realie_api_key # pipelock:ignore
|
|
|
|
return nil unless api_key.present?
|
|
|
|
Provider::Realie.new(api_key)
|
|
end
|
|
end
|
|
|
|
def initialize(concept)
|
|
@concept = concept
|
|
validate!
|
|
end
|
|
|
|
def providers
|
|
available_providers.map { |p| self.class.send(p) }.compact
|
|
end
|
|
|
|
# Returns the list of provider key names (symbols) registered for this concept.
|
|
def provider_keys
|
|
available_providers
|
|
end
|
|
|
|
def get_provider(name)
|
|
provider_method = available_providers.find { |p| p == name.to_sym }
|
|
|
|
raise Error.new("Provider '#{name}' not found for concept: #{concept}") unless provider_method.present?
|
|
|
|
self.class.send(provider_method)
|
|
end
|
|
|
|
private
|
|
attr_reader :concept
|
|
|
|
def available_providers
|
|
case concept
|
|
when :exchange_rates
|
|
%i[twelve_data yahoo_finance moex_public frankfurter]
|
|
when :securities
|
|
%i[twelve_data yahoo_finance tiingo eodhd alpha_vantage mfapi binance_public moex_public tinkoff_invest]
|
|
when :llm
|
|
%i[openai anthropic]
|
|
when :property_valuations
|
|
%i[rentcast realie]
|
|
else
|
|
%i[plaid_us plaid_eu github openai anthropic]
|
|
end
|
|
end
|
|
end
|