Files
sure/test/controllers/properties_controller_test.rb
kianrafiee d267288a45 Add RentCast & Realie integration for automatic property data and valuations (#2727)
* Add RentCast and Realie AVM providers for property valuation

Adds Automated Valuation Model (AVM) provider support so self-hosted
users can create property accounts from a US address lookup instead of
entering details manually:

- Provider::Rentcast and Provider::Realie clients, each fetching the
  property record (type, year built, square footage) and value estimate
  in a single API request, registered under a new :property_valuations
  registry concept
- API key fields (encrypted, ENV-overridable) with monthly usage display
  in Settings > Self-Hosting under "Property Valuation Providers"
- New property flow: when a provider key is configured, the method
  selector offers "Add via RentCast/Realie" alongside manual entry;
  the lookup form reuses the manual flow's localized address fields
  and creates the account active with fetched attributes, valuation as
  balance, and the address saved
- SyncPropertyValuationsJob refreshes linked property valuations once a
  day (never hourly) via config/schedule.yml; each provider enforces its
  monthly request cap (RentCast 50, Realie 25) with a calendar-month
  counter shared between creation lookups and refreshes

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Allow ENV override of AVM provider monthly request caps

The RentCast (50/month) and Realie (25/month) budgets default to the
free tier limits but can now be raised for paid plans via
RENTCAST_MAX_REQUESTS_PER_MONTH / REALIE_MAX_REQUESTS_PER_MONTH,
mirroring the AlphaVantage and Tiingo request limit overrides. The
settings descriptions and usage display reflect the effective cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address review feedback: durable request counters, safer refresh job

- Move monthly AVM request counters from Rails.cache to a new
  provider_request_counts table with atomic upsert increments, so the
  hard budget caps survive cache eviction, restarts, and Redis flushes
- SyncPropertyValuationsJob: only mark a property synced when the
  balance update succeeds (wrapped in a transaction), process stalest
  valuations first so a tight budget goes where it matters, and report
  failures via DebugLogEntry.capture instead of Rails.logger
- Realie: reject lookups whose returned city/ZIP contradict the entered
  address (street+state queries can match the wrong city); document that
  numeric use codes are unpublished and leave the subtype unset
- Add Faraday open/request timeouts to both provider clients
- Localize all user-facing provider error messages
  (config/locales/models/provider/en.yml)
- Add a check constraint restricting properties.avm_provider to known
  providers
- Use the min-h-80 scale token instead of min-h-[320px]
- Tests: exact-argument expectations on the lookup stub, provider stubs
  in the no-provider test, RentCast/Realie API key update tests,
  ProviderRequestCount unit tests, failed-balance regression test

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Stub AVM provider registry lookups in settings system test

Settings::HostingsController#show now resolves the RentCast and Realie
providers for usage display; the system test's partial get_provider
stubbing treats the new lookups as unexpected invocations without these.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address second-round review: candidate scan, provider reuse, shared row partial

- Realie: when the address lookup returns multiple candidates, pick the
  first one consistent with the entered city/ZIP instead of judging only
  the first array element; the location-mismatch error now only fires
  when no candidate matches
- SyncPropertyValuationsJob: resolve one provider instance per key for
  the whole run, so the per-instance request throttle actually spaces
  requests across properties instead of resetting on each iteration
- Extract the AVM method selector row into a shared partial so the
  manual and provider options render one shape

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add retry middleware and sync index from review feedback

- Both AVM clients now retry transient connection failures (same
  Faraday retry config as Tiingo) so a network blip doesn't burn one of
  the tight monthly budget's requests
- Partial index on properties(avm_provider, avm_last_synced_on) backing
  the daily sync job's filter and stalest-first ordering

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Key the AVM sync index on avm_last_synced_on

The daily job orders by avm_last_synced_on ASC NULLS FIRST across all
AVM-linked properties, so leading the partial index with avm_provider
prevented it from serving the sort. Rekeyed on avm_last_synced_on with
matching null ordering; the partial predicate still covers the filter.
Amended in place since the migration is unmerged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Map Realie numeric use codes; skip incomplete addresses in sync job

- Realie documents its numeric use codes at
  docs.realie.ai/api-reference/feature-key (earlier skip reason was
  wrong — the table exists). Map the residential/land/agricultural codes
  to Property subtypes, leaving codes without a subtype equivalent
  (e.g. 1006 mobile/manufactured) unset like the RentCast mapping.
- The daily job now skips properties whose address is missing a street
  or state before resolving a provider, logging a warn-level
  DebugLogEntry, so a malformed address can't burn a monthly-budget
  request every day.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Guard currency mismatches in refresh; reset AVM keys in test teardown

- The daily job now skips (with a warn-level DebugLogEntry) properties
  whose account currency no longer matches the provider's valuation
  currency, checked via the concept's valuation_currency before spending
  a request — writing a USD valuation into a re-currencied account would
  corrupt the balance
- Hostings controller test teardown now clears rentcast_api_key and
  realie_api_key alongside the other cached global settings

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Validate AVM lookup inputs before spending a provider request

The form marks name and address fields required, but a forged or
JS-less submission bypasses that and would burn one of the tight
monthly-budget requests on a lookup that can't produce a property.
Property::AvmImport now validates name and the full address locally
(localized error) before calling the provider.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Add preview-and-confirm step to the AVM property lookup

RentCast's AVM endpoint is location-based: a plausible but nonexistent
address still geocodes and returns an area-derived estimate with no
property record behind it. Instead of silently creating an account from
that, the lookup now shows what the provider returned — type, year
built, area (each "Unknown" when missing), and the estimated value —
with an explicit notice when no property record was found, and only the
user's confirmation creates the account.

The confirm step reuses the fetched data via the form (sanitized
server-side), so the flow still costs exactly one provider request.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Require a complete US address before daily valuation refresh

The job's guard only required street and state, but a later-blanked
city or ZIP silently disables the Realie wrong-city check (blank
entered fields count as "no mismatch"), so a refresh could accept
another property's valuation. The guard now mirrors the import-time
completeness check and also skips addresses edited to a non-US country.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Sign the AVM preview payload; blur valuation in privacy mode

- The confirm step now rebuilds everything from a signed, 1-hour
  message-verifier token generated at lookup time, proving the lookup
  (and its counted provider request) actually ran — a direct confirm
  POST with fabricated data can no longer create provider-linked
  properties that the daily refresh job would then spend quota on.
  Forged/expired tokens re-render the lookup form with a clear error.
- The preview's market value now carries the privacy-sensitive class so
  privacy mode blurs it like other account amounts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Treat zero Realie valuations as absent; bind preview token to family

Review follow-ups from PR #2727:
- Realie returns modelValue: 0 when it can't produce an AVM estimate;
  zero now falls back to the assessed market value instead of syncing
  a $0 balance, and an all-zero record errors out.
- Monetary values now parse via BigDecimal(value.to_s) rather than
  Float#to_d, avoiding binary float rounding artifacts in balances.
- The signed AVM preview token's purpose is scoped to the family that
  ran the lookup, so a leaked token can't be replayed cross-family.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Refresh AVM property valuations monthly instead of daily

Per @jjmata's review on #2727: valuations change slowly and providers
enforce tight monthly request caps, so a daily cron adds no value.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 04:47:58 +02:00

541 lines
16 KiB
Ruby

require "test_helper"
class PropertiesControllerTest < ActionDispatch::IntegrationTest
include AccountableResourceInterfaceTest
setup do
sign_in @user = users(:family_admin)
@account = accounts(:property)
end
test "creates property in draft status and redirects to balances step" do
assert_difference -> { Account.count } => 1 do
post properties_path, params: {
account: {
name: "New Property",
subtype: "house",
currency: "EUR",
institution_name: "Property Lender",
institution_domain: "propertylender.example",
notes: "Property notes",
accountable_type: "Property",
accountable_attributes: {
year_built: 1990,
area_value: 1200,
area_unit: "sqft"
}
}
}
end
created_account = Account.order(:created_at).last
assert created_account.accountable.is_a?(Property)
assert_equal "draft", created_account.status
assert_equal 0, created_account.balance
assert_equal "EUR", created_account.currency
assert_equal "Property Lender", created_account[:institution_name]
assert_equal "propertylender.example", created_account[:institution_domain]
assert_equal "Property notes", created_account[:notes]
assert_equal 1990, created_account.accountable.year_built
assert_equal 1200, created_account.accountable.area_value
assert_equal "sqft", created_account.accountable.area_unit
assert_redirected_to balances_property_path(created_account)
end
test "updates property overview" do
assert_no_difference [ "Account.count", "Property.count" ] do
patch property_path(@account), params: {
account: {
name: "Updated Property",
institution_name: "Updated Lender",
institution_domain: "updatedlender.example",
notes: "Updated property notes",
accountable_attributes: {
id: @account.accountable.id,
subtype: "condominium"
}
}
}
end
@account.reload
assert_equal "Updated Property", @account.name
assert_equal "condominium", @account.subtype
assert_equal "Updated Lender", @account[:institution_name]
assert_equal "updatedlender.example", @account[:institution_domain]
assert_equal "Updated property notes", @account[:notes]
# If account is active, it renders edit view; otherwise redirects to balances
if @account.active?
assert_response :success
else
assert_redirected_to balances_property_path(@account)
end
end
# Tab view tests
test "shows balances tab" do
get balances_property_path(@account)
assert_response :success
end
test "shows address tab" do
get address_property_path(@account)
assert_response :success
end
# Tab update tests
test "updates balances tab" do
original_balance = @account.balance
patch update_balances_property_path(@account), params: {
account: {
balance: 600000,
currency: "EUR"
}
}
@account.reload
assert_equal 600000, @account.balance
assert_equal "EUR", @account.currency
# If account is active, it renders balances view; otherwise redirects to address
if @account.active?
assert_response :success
else
assert_redirected_to address_property_path(@account)
end
end
test "updates address tab" do
patch update_address_property_path(@account), params: {
property: {
address_attributes: {
line1: "456 New Street",
locality: "San Francisco",
region: "CA",
country: "US",
postal_code: "94102"
}
}
}
@account.reload
assert_equal "456 New Street", @account.accountable.address.line1
assert_equal "San Francisco", @account.accountable.address.locality
# If account is draft, it activates and redirects; otherwise renders address
if @account.draft?
assert_redirected_to account_path(@account)
else
assert_response :success
end
end
test "balances update handles validation errors" do
Account.any_instance.stubs(:set_current_balance).returns(OpenStruct.new(success?: false, error_message: "Invalid balance"))
patch update_balances_property_path(@account), params: {
account: {
balance: 600000,
currency: "EUR"
}
}
assert_response :unprocessable_entity
end
test "address update handles validation errors" do
Property.any_instance.stubs(:update).returns(false)
patch update_address_property_path(@account), params: {
property: {
address_attributes: {
line1: "123 Test St"
}
}
}
assert_response :unprocessable_entity
end
test "address update activates draft account" do
# Create a draft property account
draft_account = Account.create!(
family: @user.family,
name: "Draft Property",
accountable: Property.new,
status: "draft",
balance: 500000,
currency: "USD"
)
assert draft_account.draft?
patch update_address_property_path(draft_account), params: {
property: {
address_attributes: {
line1: "789 Activate St",
locality: "New York",
region: "NY",
country: "US",
postal_code: "10001"
}
}
}
draft_account.reload
assert draft_account.active?
assert_redirected_to account_path(draft_account)
end
test "address update on draft account honors stored return_to over the account page" do
draft_account = Account.create!(
family: @user.family,
name: "Draft Property RT",
accountable: Property.new,
status: "draft",
balance: 500000,
currency: "USD"
)
# The property wizard (create → balances → address) doesn't thread return_to
# as a form param, so StoreLocation's session value is the only carrier.
get new_account_path(return_to: transactions_path)
patch update_address_property_path(draft_account), params: {
property: {
address_attributes: {
line1: "789 Activate St",
locality: "New York",
region: "NY",
country: "US",
postal_code: "10001"
}
}
}
draft_account.reload
assert draft_account.active?
assert_redirected_to transactions_path
end
test "address update ignores an external stored return_to (open-redirect guard)" do
draft_account = Account.create!(
family: @user.family,
name: "Draft Property Evil",
accountable: Property.new,
status: "draft",
balance: 500000,
currency: "USD"
)
# A hostile ?return_to is rejected at store time, so the wizard falls back
# to the account page rather than stream-redirecting off-site.
get new_account_path(return_to: "https://evil.example/phish")
patch update_address_property_path(draft_account), params: {
property: {
address_attributes: {
line1: "1 Safe St", locality: "NYC", region: "NY", country: "US", postal_code: "10001"
}
}
}
draft_account.reload
assert draft_account.active?
assert_redirected_to account_path(draft_account)
end
test "address update tolerates a non-String stored return_to without raising" do
draft_account = Account.create!(
family: @user.family,
name: "Draft Property Array",
accountable: Property.new,
status: "draft",
balance: 500000,
currency: "USD"
)
# `?return_to[]=foo` makes params[:return_to] an Array; safe_return_to must
# reject it via the is_a?(String) guard instead of raising NoMethodError.
get new_account_path("return_to" => [ "/transactions" ])
patch update_address_property_path(draft_account), params: {
property: {
address_attributes: {
line1: "1 Safe St", locality: "NYC", region: "NY", country: "US", postal_code: "10001"
}
}
}
draft_account.reload
assert draft_account.active?
assert_redirected_to account_path(draft_account)
end
# AVM provider lookup tests
def stub_avm_provider(response)
provider = mock
provider.stubs(:fetch_property_valuation).returns(response)
Provider::Registry.stubs(:rentcast).returns(provider)
provider
end
def successful_avm_response
Provider::Response.new(
success?: true,
data: Provider::PropertyValuationConcept::PropertyValuation.new(
valuation: 356_000,
currency: "USD",
property_type: "single_family_home",
year_built: 1973,
area_value: 1878,
area_unit: "sqft"
),
error: nil
)
end
test "new shows method selector when an AVM provider is configured" do
stub_avm_provider(successful_avm_response)
get new_property_path(step: "method_select")
assert_response :success
assert_match "Add via RentCast", response.body
end
test "new skips method selector when no AVM provider is configured" do
Provider::Registry.stubs(:rentcast).returns(nil)
Provider::Registry.stubs(:realie).returns(nil)
get new_property_path(step: "method_select")
assert_response :success
assert_match "Enter property manually", response.body
end
test "new renders AVM lookup form for a configured provider" do
stub_avm_provider(successful_avm_response)
get new_property_path(method: "rentcast")
assert_response :success
assert_match "Add property via RentCast", response.body
end
test "AVM lookup renders a preview without creating an account" do
provider = mock
provider.expects(:fetch_property_valuation).with(
line1: "5500 Grand Lake Dr",
locality: "San Antonio",
region: "TX",
postal_code: "78244"
).returns(successful_avm_response)
Provider::Registry.stubs(:rentcast).returns(provider)
assert_no_difference "Account.count" do
post properties_path, params: {
avm_provider: "rentcast",
account: {
name: "AVM Home",
accountable_type: "Property",
address: {
line1: "5500 Grand Lake Dr",
locality: "San Antonio",
region: "TX",
postal_code: "78244"
}
}
}
end
assert_response :success
assert_match "Confirm property details", response.body
assert_match "1973", response.body
assert_match "1878", response.body
end
test "AVM preview warns when no property record was found" do
no_record = Provider::Response.new(
success?: true,
data: Provider::PropertyValuationConcept::PropertyValuation.new(
valuation: 450_000, currency: "USD", property_type: nil, year_built: nil, area_value: nil, area_unit: "sqft"
),
error: nil
)
stub_avm_provider(no_record)
post properties_path, params: {
avm_provider: "rentcast",
account: {
name: "AVM Home",
accountable_type: "Property",
address: { line1: "1 Ghost St", locality: "San Antonio", region: "TX", postal_code: "78244" }
}
}
assert_response :success
assert_match "couldn&#39;t find a property record", response.body
assert_match "Unknown", response.body
end
def signed_preview_token(family: @user.family)
Rails.application.message_verifier(:avm_preview).generate(
{
"provider_key" => "rentcast",
"name" => "AVM Home",
"address" => {
"line1" => "5500 Grand Lake Dr",
"locality" => "San Antonio",
"region" => "TX",
"postal_code" => "78244"
},
"data" => {
"valuation" => "356000.0",
"currency" => "USD",
"property_type" => "single_family_home",
"year_built" => "1973",
"area_value" => "1878",
"area_unit" => "sqft"
}
},
expires_in: 1.hour,
purpose: "avm_preview/family/#{family.id}"
)
end
test "AVM confirm creates the account from the signed preview token without a provider call" do
provider = mock
provider.expects(:fetch_property_valuation).never
Provider::Registry.stubs(:rentcast).returns(provider)
assert_difference -> { Account.count } => 1 do
post properties_path, params: {
avm_provider: "rentcast",
avm_step: "confirm",
avm_preview_token: signed_preview_token,
account: { accountable_type: "Property" }
}
end
created_account = Account.order(:created_at).last
assert_redirected_to account_path(created_account)
assert created_account.active?
assert_equal "AVM Home", created_account.name
assert_equal 356_000, created_account.balance
assert_equal "USD", created_account.currency
property = created_account.accountable
assert_equal "rentcast", property.avm_provider
assert_equal Date.current, property.avm_last_synced_on
assert_equal "single_family_home", property.subtype
assert_equal 1973, property.year_built
assert_equal 1878, property.area_value
assert_equal "sqft", property.area_unit
address = property.address
assert_equal "5500 Grand Lake Dr", address.line1
assert_equal "San Antonio", address.locality
assert_equal "TX", address.region
assert_equal "78244", address.postal_code
assert_equal "US", address.country
end
test "re-renders lookup form when the AVM provider returns an error" do
stub_avm_provider(
Provider::Response.new(
success?: false,
data: nil,
error: Provider::Rentcast::Error.new("RentCast could not find a property matching this address")
)
)
assert_no_difference "Account.count" do
post properties_path, params: {
avm_provider: "rentcast",
account: {
name: "AVM Home",
accountable_type: "Property",
address: { line1: "1 Nowhere Ln", locality: "Nowhere", region: "TX", postal_code: "00000" }
}
}
end
assert_response :unprocessable_entity
assert_match "could not find a property", response.body
end
test "rejects AVM confirm without a valid signed preview token" do
provider = mock
provider.expects(:fetch_property_valuation).never
Provider::Registry.stubs(:rentcast).returns(provider)
assert_no_difference "Account.count" do
post properties_path, params: {
avm_provider: "rentcast",
avm_step: "confirm",
avm_preview_token: "forged-token",
account: { accountable_type: "Property" }
}
end
assert_response :unprocessable_entity
assert_match "preview has expired", response.body
end
test "rejects an AVM preview token minted for another family" do
provider = mock
provider.expects(:fetch_property_valuation).never
Provider::Registry.stubs(:rentcast).returns(provider)
assert_no_difference "Account.count" do
post properties_path, params: {
avm_provider: "rentcast",
avm_step: "confirm",
avm_preview_token: signed_preview_token(family: families(:empty)),
account: { accountable_type: "Property" }
}
end
assert_response :unprocessable_entity
assert_match "preview has expired", response.body
end
test "rejects incomplete AVM submissions before calling the provider" do
provider = mock
provider.expects(:fetch_property_valuation).never
Provider::Registry.stubs(:rentcast).returns(provider)
assert_no_difference "Account.count" do
post properties_path, params: {
avm_provider: "rentcast",
account: {
name: "",
accountable_type: "Property",
address: { line1: "5500 Grand Lake Dr", locality: "", region: "TX", postal_code: "78244" }
}
}
end
assert_response :unprocessable_entity
assert_match "complete US address", response.body
end
test "rejects AVM creation for unconfigured providers" do
assert_no_difference "Account.count" do
post properties_path, params: {
avm_provider: "realie",
account: {
name: "AVM Home",
accountable_type: "Property",
address: { line1: "123 Main Street", locality: "LA", region: "CA", postal_code: "90001" }
}
}
end
assert_redirected_to new_property_path
end
end