mirror of
https://github.com/we-promise/sure.git
synced 2026-08-05 16:42:18 +00:00
* Add RentCast and Realie AVM providers for property valuation Adds Automated Valuation Model (AVM) provider support so self-hosted users can create property accounts from a US address lookup instead of entering details manually: - Provider::Rentcast and Provider::Realie clients, each fetching the property record (type, year built, square footage) and value estimate in a single API request, registered under a new :property_valuations registry concept - API key fields (encrypted, ENV-overridable) with monthly usage display in Settings > Self-Hosting under "Property Valuation Providers" - New property flow: when a provider key is configured, the method selector offers "Add via RentCast/Realie" alongside manual entry; the lookup form reuses the manual flow's localized address fields and creates the account active with fetched attributes, valuation as balance, and the address saved - SyncPropertyValuationsJob refreshes linked property valuations once a day (never hourly) via config/schedule.yml; each provider enforces its monthly request cap (RentCast 50, Realie 25) with a calendar-month counter shared between creation lookups and refreshes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Allow ENV override of AVM provider monthly request caps The RentCast (50/month) and Realie (25/month) budgets default to the free tier limits but can now be raised for paid plans via RENTCAST_MAX_REQUESTS_PER_MONTH / REALIE_MAX_REQUESTS_PER_MONTH, mirroring the AlphaVantage and Tiingo request limit overrides. The settings descriptions and usage display reflect the effective cap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address review feedback: durable request counters, safer refresh job - Move monthly AVM request counters from Rails.cache to a new provider_request_counts table with atomic upsert increments, so the hard budget caps survive cache eviction, restarts, and Redis flushes - SyncPropertyValuationsJob: only mark a property synced when the balance update succeeds (wrapped in a transaction), process stalest valuations first so a tight budget goes where it matters, and report failures via DebugLogEntry.capture instead of Rails.logger - Realie: reject lookups whose returned city/ZIP contradict the entered address (street+state queries can match the wrong city); document that numeric use codes are unpublished and leave the subtype unset - Add Faraday open/request timeouts to both provider clients - Localize all user-facing provider error messages (config/locales/models/provider/en.yml) - Add a check constraint restricting properties.avm_provider to known providers - Use the min-h-80 scale token instead of min-h-[320px] - Tests: exact-argument expectations on the lookup stub, provider stubs in the no-provider test, RentCast/Realie API key update tests, ProviderRequestCount unit tests, failed-balance regression test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Stub AVM provider registry lookups in settings system test Settings::HostingsController#show now resolves the RentCast and Realie providers for usage display; the system test's partial get_provider stubbing treats the new lookups as unexpected invocations without these. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address second-round review: candidate scan, provider reuse, shared row partial - Realie: when the address lookup returns multiple candidates, pick the first one consistent with the entered city/ZIP instead of judging only the first array element; the location-mismatch error now only fires when no candidate matches - SyncPropertyValuationsJob: resolve one provider instance per key for the whole run, so the per-instance request throttle actually spaces requests across properties instead of resetting on each iteration - Extract the AVM method selector row into a shared partial so the manual and provider options render one shape Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add retry middleware and sync index from review feedback - Both AVM clients now retry transient connection failures (same Faraday retry config as Tiingo) so a network blip doesn't burn one of the tight monthly budget's requests - Partial index on properties(avm_provider, avm_last_synced_on) backing the daily sync job's filter and stalest-first ordering Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Key the AVM sync index on avm_last_synced_on The daily job orders by avm_last_synced_on ASC NULLS FIRST across all AVM-linked properties, so leading the partial index with avm_provider prevented it from serving the sort. Rekeyed on avm_last_synced_on with matching null ordering; the partial predicate still covers the filter. Amended in place since the migration is unmerged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Map Realie numeric use codes; skip incomplete addresses in sync job - Realie documents its numeric use codes at docs.realie.ai/api-reference/feature-key (earlier skip reason was wrong — the table exists). Map the residential/land/agricultural codes to Property subtypes, leaving codes without a subtype equivalent (e.g. 1006 mobile/manufactured) unset like the RentCast mapping. - The daily job now skips properties whose address is missing a street or state before resolving a provider, logging a warn-level DebugLogEntry, so a malformed address can't burn a monthly-budget request every day. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Guard currency mismatches in refresh; reset AVM keys in test teardown - The daily job now skips (with a warn-level DebugLogEntry) properties whose account currency no longer matches the provider's valuation currency, checked via the concept's valuation_currency before spending a request — writing a USD valuation into a re-currencied account would corrupt the balance - Hostings controller test teardown now clears rentcast_api_key and realie_api_key alongside the other cached global settings Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Validate AVM lookup inputs before spending a provider request The form marks name and address fields required, but a forged or JS-less submission bypasses that and would burn one of the tight monthly-budget requests on a lookup that can't produce a property. Property::AvmImport now validates name and the full address locally (localized error) before calling the provider. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add preview-and-confirm step to the AVM property lookup RentCast's AVM endpoint is location-based: a plausible but nonexistent address still geocodes and returns an area-derived estimate with no property record behind it. Instead of silently creating an account from that, the lookup now shows what the provider returned — type, year built, area (each "Unknown" when missing), and the estimated value — with an explicit notice when no property record was found, and only the user's confirmation creates the account. The confirm step reuses the fetched data via the form (sanitized server-side), so the flow still costs exactly one provider request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Require a complete US address before daily valuation refresh The job's guard only required street and state, but a later-blanked city or ZIP silently disables the Realie wrong-city check (blank entered fields count as "no mismatch"), so a refresh could accept another property's valuation. The guard now mirrors the import-time completeness check and also skips addresses edited to a non-US country. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Sign the AVM preview payload; blur valuation in privacy mode - The confirm step now rebuilds everything from a signed, 1-hour message-verifier token generated at lookup time, proving the lookup (and its counted provider request) actually ran — a direct confirm POST with fabricated data can no longer create provider-linked properties that the daily refresh job would then spend quota on. Forged/expired tokens re-render the lookup form with a clear error. - The preview's market value now carries the privacy-sensitive class so privacy mode blurs it like other account amounts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Treat zero Realie valuations as absent; bind preview token to family Review follow-ups from PR #2727: - Realie returns modelValue: 0 when it can't produce an AVM estimate; zero now falls back to the assessed market value instead of syncing a $0 balance, and an all-zero record errors out. - Monetary values now parse via BigDecimal(value.to_s) rather than Float#to_d, avoiding binary float rounding artifacts in balances. - The signed AVM preview token's purpose is scoped to the family that ran the lookup, so a leaked token can't be replayed cross-family. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Refresh AVM property valuations monthly instead of daily Per @jjmata's review on #2727: valuations change slowly and providers enforce tight monthly request caps, so a daily cron adds no value. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
160 lines
6.5 KiB
Ruby
160 lines
6.5 KiB
Ruby
class Provider::Realie < Provider
|
|
include PropertyValuationConcept, RateLimitable
|
|
extend SslConfigurable
|
|
|
|
# Subclass so errors caught in this provider are raised as Provider::Realie::Error
|
|
Error = Class.new(Provider::Error)
|
|
RateLimitError = Class.new(Error)
|
|
|
|
# Minimum delay between requests to avoid rate limiting (in seconds)
|
|
MIN_REQUEST_INTERVAL = 1.0
|
|
|
|
# Maximum API requests per month (Realie free tier limit).
|
|
# Override with REALIE_MAX_REQUESTS_PER_MONTH for paid plans.
|
|
MAX_REQUESTS_PER_MONTH = 25
|
|
|
|
# Realie's documented numeric use codes to Property::SUBTYPES keys
|
|
# (https://docs.realie.ai/api-reference/feature-key). Codes without a
|
|
# sensible subtype equivalent (e.g. 1006 mobile/manufactured, commercial
|
|
# codes) are left unmapped, matching the RentCast mapping's behavior.
|
|
NUMERIC_USE_CODE_MAP = {
|
|
"1001" => "single_family_home", # Single family residential
|
|
"1008" => "single_family_home", # Rural/agricultural residence
|
|
"1022" => "single_family_home", # SFR with ADU
|
|
"1024" => "single_family_home", # Multiple SFRs on one parcel
|
|
"1002" => "townhouse", # Townhouse (residential)
|
|
"1004" => "condominium", # Condominium unit (residential)
|
|
"1124" => "condominium", # Condominium building (residential)
|
|
"1101" => "multi_family_home", # Duplex
|
|
"1102" => "multi_family_home", # Triplex
|
|
"1103" => "multi_family_home", # Quadruplex
|
|
"1104" => "apartment", # Apartment house (5+ units)
|
|
"1105" => "apartment", # Apartment house (100+ units)
|
|
"7001" => "agri_land", # Farm
|
|
"7002" => "agri_land", # Ranch
|
|
"8001" => "plot", # Residential vacant land
|
|
"8002" => "plot", # Commercial vacant land
|
|
"8008" => "plot" # Rural/agricultural vacant land
|
|
}.freeze
|
|
|
|
def initialize(api_key)
|
|
@api_key = api_key # pipelock:ignore
|
|
end
|
|
|
|
# A single address lookup returns the full property record, including the
|
|
# Realie model (AVM) value, so no separate valuation request is needed.
|
|
def fetch_property_valuation(line1:, locality: nil, region: nil, postal_code: nil)
|
|
with_provider_response do
|
|
throttle_request
|
|
record_monthly_request!
|
|
|
|
# The address lookup endpoint accepts only the street address and
|
|
# 2-letter state code — filtering by city additionally requires a
|
|
# county, which the property address form doesn't collect.
|
|
response = client.get("#{base_url}/api/public/property/address/") do |req|
|
|
req.params["address"] = line1.to_s.strip
|
|
req.params["state"] = region.to_s.strip.upcase
|
|
end
|
|
|
|
parsed = JSON.parse(response.body)
|
|
records = parsed["property"]
|
|
records = [ records ] unless records.is_a?(Array)
|
|
records = records.reject(&:blank?)
|
|
raise Error.new(I18n.t("providers.realie.errors.no_property")) if records.empty?
|
|
|
|
# A street+state query can return several candidates across different
|
|
# cities; pick the first one consistent with the entered city/ZIP.
|
|
record = records.find { |candidate| location_match?(candidate, locality: locality, postal_code: postal_code) }
|
|
raise Error.new(I18n.t("providers.realie.errors.location_mismatch")) if record.nil?
|
|
|
|
# Realie returns modelValue: 0 when it couldn't produce an AVM
|
|
# estimate, so zero counts as absent and falls back to the assessed
|
|
# total market value.
|
|
valuation = [ record["modelValue"], record["totalMarketValue"] ].find { |value| value.present? && value.to_d.positive? }
|
|
raise Error.new(I18n.t("providers.realie.errors.no_valuation")) if valuation.nil?
|
|
|
|
PropertyValuation.new(
|
|
valuation: BigDecimal(valuation.to_s),
|
|
currency: "USD",
|
|
property_type: subtype_for_use_code(record["useCode"]),
|
|
year_built: record["yearBuilt"],
|
|
area_value: record["buildingArea"],
|
|
area_unit: "sqft"
|
|
)
|
|
end
|
|
end
|
|
|
|
private
|
|
attr_reader :api_key
|
|
|
|
# The address lookup matches on street + state only (filtering by city
|
|
# additionally requires a county, which isn't collected), so a common
|
|
# street name can resolve to properties in other cities. A candidate
|
|
# matches unless its returned city/ZIP contradict what the user entered.
|
|
def location_match?(record, locality:, postal_code:)
|
|
returned_city = record["city"].to_s.strip
|
|
returned_zip = record["zipCode"].to_s.strip.first(5)
|
|
entered_city = locality.to_s.strip
|
|
entered_zip = postal_code.to_s.strip.first(5)
|
|
|
|
city_mismatch = returned_city.present? && entered_city.present? && !returned_city.casecmp?(entered_city)
|
|
zip_mismatch = returned_zip.present? && entered_zip.present? && returned_zip != entered_zip
|
|
|
|
!(city_mismatch || zip_mismatch)
|
|
end
|
|
|
|
# Use codes arrive either as Realie's documented numeric codes (mapped
|
|
# exactly above) or as free-form parcel descriptions, matched on
|
|
# keywords. Unmatched codes leave the subtype unset rather than guessing.
|
|
def subtype_for_use_code(use_code)
|
|
value = use_code.to_s.strip.downcase
|
|
return nil if value.blank?
|
|
|
|
return NUMERIC_USE_CODE_MAP[value] if value.match?(/\A\d+\z/)
|
|
|
|
case value
|
|
when /single family|sfr/ then "single_family_home"
|
|
when /multi.?family|duplex|triplex/ then "multi_family_home"
|
|
when /condo/ then "condominium"
|
|
when /town.?(house|home)/ then "townhouse"
|
|
when /apartment/ then "apartment"
|
|
when /agricultur|farm/ then "agri_land"
|
|
when /commercial|retail|office|industrial/ then "commercial"
|
|
when /vacant|land|lot/ then "plot"
|
|
end
|
|
end
|
|
|
|
def default_error_transformer(error)
|
|
case error
|
|
when Faraday::ResourceNotFound
|
|
Error.new(I18n.t("providers.realie.errors.not_found"))
|
|
else
|
|
super
|
|
end
|
|
end
|
|
|
|
def base_url
|
|
ENV["REALIE_URL"] || "https://app.realie.ai"
|
|
end
|
|
|
|
def client
|
|
@client ||= Faraday.new(url: base_url, ssl: self.class.faraday_ssl_options) do |faraday|
|
|
# Retry transient connection failures so a network blip doesn't burn
|
|
# one of the tight monthly budget's requests
|
|
faraday.request(:retry, {
|
|
max: 3,
|
|
interval: 1.0,
|
|
interval_randomness: 0.5,
|
|
backoff_factor: 2,
|
|
exceptions: Faraday::Retry::Middleware::DEFAULT_EXCEPTIONS + [ Faraday::ConnectionFailed ]
|
|
})
|
|
faraday.request :json
|
|
faraday.response :raise_error
|
|
faraday.options.timeout = 10
|
|
faraday.options.open_timeout = 5
|
|
faraday.headers["Authorization"] = api_key
|
|
faraday.headers["Accept"] = "application/json"
|
|
end
|
|
end
|
|
end
|