Files
sure/app/controllers/properties_controller.rb
kianrafiee d267288a45 Add RentCast & Realie integration for automatic property data and valuations (#2727)
* Add RentCast and Realie AVM providers for property valuation

Adds Automated Valuation Model (AVM) provider support so self-hosted
users can create property accounts from a US address lookup instead of
entering details manually:

- Provider::Rentcast and Provider::Realie clients, each fetching the
  property record (type, year built, square footage) and value estimate
  in a single API request, registered under a new :property_valuations
  registry concept
- API key fields (encrypted, ENV-overridable) with monthly usage display
  in Settings > Self-Hosting under "Property Valuation Providers"
- New property flow: when a provider key is configured, the method
  selector offers "Add via RentCast/Realie" alongside manual entry;
  the lookup form reuses the manual flow's localized address fields
  and creates the account active with fetched attributes, valuation as
  balance, and the address saved
- SyncPropertyValuationsJob refreshes linked property valuations once a
  day (never hourly) via config/schedule.yml; each provider enforces its
  monthly request cap (RentCast 50, Realie 25) with a calendar-month
  counter shared between creation lookups and refreshes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Allow ENV override of AVM provider monthly request caps

The RentCast (50/month) and Realie (25/month) budgets default to the
free tier limits but can now be raised for paid plans via
RENTCAST_MAX_REQUESTS_PER_MONTH / REALIE_MAX_REQUESTS_PER_MONTH,
mirroring the AlphaVantage and Tiingo request limit overrides. The
settings descriptions and usage display reflect the effective cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address review feedback: durable request counters, safer refresh job

- Move monthly AVM request counters from Rails.cache to a new
  provider_request_counts table with atomic upsert increments, so the
  hard budget caps survive cache eviction, restarts, and Redis flushes
- SyncPropertyValuationsJob: only mark a property synced when the
  balance update succeeds (wrapped in a transaction), process stalest
  valuations first so a tight budget goes where it matters, and report
  failures via DebugLogEntry.capture instead of Rails.logger
- Realie: reject lookups whose returned city/ZIP contradict the entered
  address (street+state queries can match the wrong city); document that
  numeric use codes are unpublished and leave the subtype unset
- Add Faraday open/request timeouts to both provider clients
- Localize all user-facing provider error messages
  (config/locales/models/provider/en.yml)
- Add a check constraint restricting properties.avm_provider to known
  providers
- Use the min-h-80 scale token instead of min-h-[320px]
- Tests: exact-argument expectations on the lookup stub, provider stubs
  in the no-provider test, RentCast/Realie API key update tests,
  ProviderRequestCount unit tests, failed-balance regression test

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Stub AVM provider registry lookups in settings system test

Settings::HostingsController#show now resolves the RentCast and Realie
providers for usage display; the system test's partial get_provider
stubbing treats the new lookups as unexpected invocations without these.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address second-round review: candidate scan, provider reuse, shared row partial

- Realie: when the address lookup returns multiple candidates, pick the
  first one consistent with the entered city/ZIP instead of judging only
  the first array element; the location-mismatch error now only fires
  when no candidate matches
- SyncPropertyValuationsJob: resolve one provider instance per key for
  the whole run, so the per-instance request throttle actually spaces
  requests across properties instead of resetting on each iteration
- Extract the AVM method selector row into a shared partial so the
  manual and provider options render one shape

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add retry middleware and sync index from review feedback

- Both AVM clients now retry transient connection failures (same
  Faraday retry config as Tiingo) so a network blip doesn't burn one of
  the tight monthly budget's requests
- Partial index on properties(avm_provider, avm_last_synced_on) backing
  the daily sync job's filter and stalest-first ordering

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Key the AVM sync index on avm_last_synced_on

The daily job orders by avm_last_synced_on ASC NULLS FIRST across all
AVM-linked properties, so leading the partial index with avm_provider
prevented it from serving the sort. Rekeyed on avm_last_synced_on with
matching null ordering; the partial predicate still covers the filter.
Amended in place since the migration is unmerged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Map Realie numeric use codes; skip incomplete addresses in sync job

- Realie documents its numeric use codes at
  docs.realie.ai/api-reference/feature-key (earlier skip reason was
  wrong — the table exists). Map the residential/land/agricultural codes
  to Property subtypes, leaving codes without a subtype equivalent
  (e.g. 1006 mobile/manufactured) unset like the RentCast mapping.
- The daily job now skips properties whose address is missing a street
  or state before resolving a provider, logging a warn-level
  DebugLogEntry, so a malformed address can't burn a monthly-budget
  request every day.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Guard currency mismatches in refresh; reset AVM keys in test teardown

- The daily job now skips (with a warn-level DebugLogEntry) properties
  whose account currency no longer matches the provider's valuation
  currency, checked via the concept's valuation_currency before spending
  a request — writing a USD valuation into a re-currencied account would
  corrupt the balance
- Hostings controller test teardown now clears rentcast_api_key and
  realie_api_key alongside the other cached global settings

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Validate AVM lookup inputs before spending a provider request

The form marks name and address fields required, but a forged or
JS-less submission bypasses that and would burn one of the tight
monthly-budget requests on a lookup that can't produce a property.
Property::AvmImport now validates name and the full address locally
(localized error) before calling the provider.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add preview-and-confirm step to the AVM property lookup

RentCast's AVM endpoint is location-based: a plausible but nonexistent
address still geocodes and returns an area-derived estimate with no
property record behind it. Instead of silently creating an account from
that, the lookup now shows what the provider returned — type, year
built, area (each "Unknown" when missing), and the estimated value —
with an explicit notice when no property record was found, and only the
user's confirmation creates the account.

The confirm step reuses the fetched data via the form (sanitized
server-side), so the flow still costs exactly one provider request.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Require a complete US address before daily valuation refresh

The job's guard only required street and state, but a later-blanked
city or ZIP silently disables the Realie wrong-city check (blank
entered fields count as "no mismatch"), so a refresh could accept
another property's valuation. The guard now mirrors the import-time
completeness check and also skips addresses edited to a non-US country.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Sign the AVM preview payload; blur valuation in privacy mode

- The confirm step now rebuilds everything from a signed, 1-hour
  message-verifier token generated at lookup time, proving the lookup
  (and its counted provider request) actually ran — a direct confirm
  POST with fabricated data can no longer create provider-linked
  properties that the daily refresh job would then spend quota on.
  Forged/expired tokens re-render the lookup form with a clear error.
- The preview's market value now carries the privacy-sensitive class so
  privacy mode blurs it like other account amounts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Treat zero Realie valuations as absent; bind preview token to family

Review follow-ups from PR #2727:
- Realie returns modelValue: 0 when it can't produce an AVM estimate;
  zero now falls back to the assessed market value instead of syncing
  a $0 balance, and an all-zero record errors out.
- Monetary values now parse via BigDecimal(value.to_s) rather than
  Float#to_d, avoiding binary float rounding artifacts in balances.
- The signed AVM preview token's purpose is scoped to the family that
  ran the lookup, so a leaked token can't be replayed cross-family.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Refresh AVM property valuations monthly instead of daily

Per @jjmata's review on #2727: valuations change slowly and providers
enforce tight monthly request caps, so a daily cron adds no value.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 04:47:58 +02:00

249 lines
8.7 KiB
Ruby

class PropertiesController < ApplicationController
include AccountableResource, StreamExtensions
before_action :set_property, only: [ :balances, :address, :update_balances, :update_address ]
before_action :require_property_write_permission!, only: [ :update_balances, :update_address ]
def new
@account = Current.family.accounts.build(accountable: Property.new)
@avm_providers = configured_avm_providers
end
def create
return create_via_avm_provider if params[:avm_provider].present?
@account = Current.family.accounts.create!(
property_params.merge(
balance: 0,
status: "draft",
owner: Current.user,
currency: property_params[:currency].presence || Current.family.currency
)
)
@account.auto_share_with_family! if Current.family.share_all_by_default?
redirect_to balances_property_path(@account)
end
def update
if @account.update(property_params)
@success_message = "Property details updated successfully."
if @account.active?
render :edit
else
redirect_to balances_property_path(@account)
end
else
@error_message = "Unable to update property details."
render :edit, status: :unprocessable_entity
end
end
def edit
end
def balances
end
def update_balances
result = nil
Account.transaction do
@account.update!(currency: balance_params[:currency]) if balance_params[:currency].present?
result = @account.set_current_balance(balance_params[:balance].to_d)
raise ActiveRecord::Rollback unless result.success?
end
if result&.success?
@success_message = "Balance updated successfully."
if @account.active?
render :balances
else
redirect_to address_property_path(@account)
end
else
@error_message = result&.error_message
render :balances, status: :unprocessable_entity
end
end
def address
@property = @account.property
@property.address ||= Address.new
end
def update_address
if @account.property.update(address_params)
if @account.draft?
@account.activate!
# The property setup wizard (create → balances → address) is multi-step,
# so the original `?return_to=` only survives in the session (captured by
# StoreLocation), not as a threaded form param. Honor it on completion so
# flows like the savings-goals "Add an account" CTA land back where they
# started instead of on the account page. Sanitized + consumed: the
# turbo_stream branch below isn't covered by Rails' redirect host-guard,
# so an unsafe value must not reach stream_redirect_to.
return_path = safe_return_to(session.delete(:return_to)) || account_path(@account)
respond_to do |format|
format.html { redirect_to return_path }
format.turbo_stream { stream_redirect_to return_path }
end
else
@success_message = "Address updated successfully."
render :address
end
else
@error_message = "Unable to update address. Please check the required fields."
render :address, status: :unprocessable_entity
end
end
private
def create_via_avm_provider
@avm_providers = configured_avm_providers
provider_key = params[:avm_provider].to_s
unless @avm_providers.map(&:to_s).include?(provider_key)
redirect_to new_property_path, alert: t("providers.property_valuation.not_configured") and return
end
if params[:avm_step] == "confirm"
# The signed token proves the lookup step actually ran (and spent its
# provider request) — a direct confirm POST with fabricated data
# would otherwise create provider-linked properties that the daily
# refresh job then spends quota on. Everything is rebuilt from the
# token payload, so nothing user-editable is trusted.
payload = verify_avm_preview_token!
importer = Property::AvmImport.new(
family: Current.family,
owner: Current.user,
provider_key: payload["provider_key"],
name: payload["name"],
address_attributes: payload["address"].symbolize_keys
)
@account = importer.create_account(avm_data_from_payload(payload))
# The provider lookup fills in what the manual wizard's balance and
# address steps would have collected, so the account is complete —
# land on the account page (or the flow that initiated the wizard).
return_path = safe_return_to(session.delete(:return_to)) || account_path(@account)
respond_to do |format|
format.html { redirect_to return_path }
format.turbo_stream { stream_redirect_to return_path }
end
else
# Step 1: spend one provider request, then show the fetched data for
# review before anything is created.
importer = Property::AvmImport.new(
family: Current.family,
owner: Current.user,
provider_key: provider_key,
name: params.dig(:account, :name),
address_attributes: avm_address_params.to_h.symbolize_keys
)
@avm_preview = importer.lookup
@avm_preview_token = avm_preview_verifier.generate(
{
"provider_key" => provider_key,
"name" => params.dig(:account, :name),
"address" => avm_address_params.to_h,
"data" => @avm_preview.to_h.transform_values(&:to_s)
},
expires_in: 1.hour,
purpose: avm_preview_token_purpose
)
@avm_provider_key = provider_key
@account = Current.family.accounts.build(name: params.dig(:account, :name), accountable: Property.new)
@avm_address = Address.new(avm_address_params.to_h)
render :new
end
rescue Property::AvmImport::Error => error
@avm_provider_key = provider_key
@error_message = error.message
@account = Current.family.accounts.build(name: params.dig(:account, :name), accountable: Property.new)
# The confirm step doesn't resubmit the address fields, so build
# defensively — an expired token re-renders an empty lookup form.
@avm_address = Address.new(params.dig(:account, :address)&.permit(:line1, :locality, :region, :postal_code)&.to_h || {})
render :new, status: :unprocessable_entity
end
def avm_address_params
params.require(:account).require(:address).permit(:line1, :locality, :region, :postal_code)
end
def avm_preview_verifier
Rails.application.message_verifier(:avm_preview)
end
# Scoping the token's purpose to the family means a token minted in one
# family's session can't be replayed to seed a property in another —
# a purpose mismatch fails verification just like a bad signature.
def avm_preview_token_purpose
"avm_preview/family/#{Current.family.id}"
end
def verify_avm_preview_token!
avm_preview_verifier.verify(params[:avm_preview_token].to_s, purpose: avm_preview_token_purpose)
rescue ActiveSupport::MessageVerifier::InvalidSignature
raise Property::AvmImport::Error.new(t("providers.property_valuation.preview_expired"))
end
# Rebuilds the valuation data from the signed preview payload (values
# were stringified for serialization; blanks were nil).
def avm_data_from_payload(payload)
raw = payload["data"] || {}
Provider::PropertyValuationConcept::PropertyValuation.new(
valuation: raw["valuation"].to_d,
currency: raw["currency"].presence || "USD",
property_type: raw["property_type"].presence,
year_built: raw["year_built"].presence&.to_i,
area_value: raw["area_value"].presence&.to_i,
area_unit: raw["area_unit"].presence || "sqft"
)
end
def configured_avm_providers
registry = Provider::Registry.for_concept(:property_valuations)
registry.provider_keys.select { |key| registry.get_provider(key).present? }
end
def balance_params
params.require(:account).permit(:balance, :currency)
end
def address_params
params.require(:property)
.permit(address_attributes: [ :line1, :line2, :locality, :region, :country, :postal_code ])
end
def property_params
params.require(:account)
.permit(
:name,
:currency,
:accountable_type,
:institution_name,
:institution_domain,
:notes,
accountable_attributes: [ :id, :subtype, :year_built, :area_unit, :area_value ]
)
end
def set_property
@account = accessible_accounts.find(params[:id])
@property = @account.property
end
def require_property_write_permission!
require_account_permission!(@account)
end
end