mirror of
https://github.com/we-promise/sure.git
synced 2026-07-27 12:12:13 +00:00
* Add RentCast and Realie AVM providers for property valuation Adds Automated Valuation Model (AVM) provider support so self-hosted users can create property accounts from a US address lookup instead of entering details manually: - Provider::Rentcast and Provider::Realie clients, each fetching the property record (type, year built, square footage) and value estimate in a single API request, registered under a new :property_valuations registry concept - API key fields (encrypted, ENV-overridable) with monthly usage display in Settings > Self-Hosting under "Property Valuation Providers" - New property flow: when a provider key is configured, the method selector offers "Add via RentCast/Realie" alongside manual entry; the lookup form reuses the manual flow's localized address fields and creates the account active with fetched attributes, valuation as balance, and the address saved - SyncPropertyValuationsJob refreshes linked property valuations once a day (never hourly) via config/schedule.yml; each provider enforces its monthly request cap (RentCast 50, Realie 25) with a calendar-month counter shared between creation lookups and refreshes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Allow ENV override of AVM provider monthly request caps The RentCast (50/month) and Realie (25/month) budgets default to the free tier limits but can now be raised for paid plans via RENTCAST_MAX_REQUESTS_PER_MONTH / REALIE_MAX_REQUESTS_PER_MONTH, mirroring the AlphaVantage and Tiingo request limit overrides. The settings descriptions and usage display reflect the effective cap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address review feedback: durable request counters, safer refresh job - Move monthly AVM request counters from Rails.cache to a new provider_request_counts table with atomic upsert increments, so the hard budget caps survive cache eviction, restarts, and Redis flushes - SyncPropertyValuationsJob: only mark a property synced when the balance update succeeds (wrapped in a transaction), process stalest valuations first so a tight budget goes where it matters, and report failures via DebugLogEntry.capture instead of Rails.logger - Realie: reject lookups whose returned city/ZIP contradict the entered address (street+state queries can match the wrong city); document that numeric use codes are unpublished and leave the subtype unset - Add Faraday open/request timeouts to both provider clients - Localize all user-facing provider error messages (config/locales/models/provider/en.yml) - Add a check constraint restricting properties.avm_provider to known providers - Use the min-h-80 scale token instead of min-h-[320px] - Tests: exact-argument expectations on the lookup stub, provider stubs in the no-provider test, RentCast/Realie API key update tests, ProviderRequestCount unit tests, failed-balance regression test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Stub AVM provider registry lookups in settings system test Settings::HostingsController#show now resolves the RentCast and Realie providers for usage display; the system test's partial get_provider stubbing treats the new lookups as unexpected invocations without these. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address second-round review: candidate scan, provider reuse, shared row partial - Realie: when the address lookup returns multiple candidates, pick the first one consistent with the entered city/ZIP instead of judging only the first array element; the location-mismatch error now only fires when no candidate matches - SyncPropertyValuationsJob: resolve one provider instance per key for the whole run, so the per-instance request throttle actually spaces requests across properties instead of resetting on each iteration - Extract the AVM method selector row into a shared partial so the manual and provider options render one shape Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add retry middleware and sync index from review feedback - Both AVM clients now retry transient connection failures (same Faraday retry config as Tiingo) so a network blip doesn't burn one of the tight monthly budget's requests - Partial index on properties(avm_provider, avm_last_synced_on) backing the daily sync job's filter and stalest-first ordering Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Key the AVM sync index on avm_last_synced_on The daily job orders by avm_last_synced_on ASC NULLS FIRST across all AVM-linked properties, so leading the partial index with avm_provider prevented it from serving the sort. Rekeyed on avm_last_synced_on with matching null ordering; the partial predicate still covers the filter. Amended in place since the migration is unmerged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Map Realie numeric use codes; skip incomplete addresses in sync job - Realie documents its numeric use codes at docs.realie.ai/api-reference/feature-key (earlier skip reason was wrong — the table exists). Map the residential/land/agricultural codes to Property subtypes, leaving codes without a subtype equivalent (e.g. 1006 mobile/manufactured) unset like the RentCast mapping. - The daily job now skips properties whose address is missing a street or state before resolving a provider, logging a warn-level DebugLogEntry, so a malformed address can't burn a monthly-budget request every day. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Guard currency mismatches in refresh; reset AVM keys in test teardown - The daily job now skips (with a warn-level DebugLogEntry) properties whose account currency no longer matches the provider's valuation currency, checked via the concept's valuation_currency before spending a request — writing a USD valuation into a re-currencied account would corrupt the balance - Hostings controller test teardown now clears rentcast_api_key and realie_api_key alongside the other cached global settings Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Validate AVM lookup inputs before spending a provider request The form marks name and address fields required, but a forged or JS-less submission bypasses that and would burn one of the tight monthly-budget requests on a lookup that can't produce a property. Property::AvmImport now validates name and the full address locally (localized error) before calling the provider. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add preview-and-confirm step to the AVM property lookup RentCast's AVM endpoint is location-based: a plausible but nonexistent address still geocodes and returns an area-derived estimate with no property record behind it. Instead of silently creating an account from that, the lookup now shows what the provider returned — type, year built, area (each "Unknown" when missing), and the estimated value — with an explicit notice when no property record was found, and only the user's confirmation creates the account. The confirm step reuses the fetched data via the form (sanitized server-side), so the flow still costs exactly one provider request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Require a complete US address before daily valuation refresh The job's guard only required street and state, but a later-blanked city or ZIP silently disables the Realie wrong-city check (blank entered fields count as "no mismatch"), so a refresh could accept another property's valuation. The guard now mirrors the import-time completeness check and also skips addresses edited to a non-US country. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Sign the AVM preview payload; blur valuation in privacy mode - The confirm step now rebuilds everything from a signed, 1-hour message-verifier token generated at lookup time, proving the lookup (and its counted provider request) actually ran — a direct confirm POST with fabricated data can no longer create provider-linked properties that the daily refresh job would then spend quota on. Forged/expired tokens re-render the lookup form with a clear error. - The preview's market value now carries the privacy-sensitive class so privacy mode blurs it like other account amounts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Treat zero Realie valuations as absent; bind preview token to family Review follow-ups from PR #2727: - Realie returns modelValue: 0 when it can't produce an AVM estimate; zero now falls back to the assessed market value instead of syncing a $0 balance, and an all-zero record errors out. - Monetary values now parse via BigDecimal(value.to_s) rather than Float#to_d, avoiding binary float rounding artifacts in balances. - The signed AVM preview token's purpose is scoped to the family that ran the lookup, so a leaked token can't be replayed cross-family. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Refresh AVM property valuations monthly instead of daily Per @jjmata's review on #2727: valuations change slowly and providers enforce tight monthly request caps, so a daily cron adds no value. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
329 lines
14 KiB
Ruby
329 lines
14 KiB
Ruby
class Settings::HostingsController < ApplicationController
|
|
layout "settings"
|
|
|
|
# Minimum accepted value for each configurable LLM budget field. Mirrors the
|
|
# `min:` attribute on the form inputs in `_openai_settings.html.erb` so the
|
|
# controller rejects what the browser-side validator would reject.
|
|
LLM_BUDGET_MINIMUMS = {
|
|
llm_context_window: 256,
|
|
llm_max_response_tokens: 64,
|
|
llm_max_items_per_call: 1
|
|
}.freeze
|
|
|
|
guard_feature unless: -> { self_hosted? }
|
|
|
|
before_action :ensure_admin, only: [ :update, :clear_cache, :disconnect_external_assistant ]
|
|
before_action :ensure_super_admin_for_onboarding, only: :update
|
|
|
|
def show
|
|
@breadcrumbs = [
|
|
[ t("breadcrumbs.home"), root_path ],
|
|
[ t("breadcrumbs.self_hosting"), nil ]
|
|
]
|
|
|
|
# Determine which providers are currently selected
|
|
exchange_rate_provider = ENV["EXCHANGE_RATE_PROVIDER"].presence || Setting.exchange_rate_provider
|
|
enabled_securities = Setting.enabled_securities_providers
|
|
|
|
# Show provider settings if used for FX or enabled for securities
|
|
@show_twelve_data_settings = exchange_rate_provider == "twelve_data" || enabled_securities.include?("twelve_data")
|
|
@show_yahoo_finance_settings = exchange_rate_provider == "yahoo_finance" || enabled_securities.include?("yahoo_finance")
|
|
@show_tiingo_settings = enabled_securities.include?("tiingo")
|
|
@show_eodhd_settings = enabled_securities.include?("eodhd")
|
|
@show_alpha_vantage_settings = enabled_securities.include?("alpha_vantage")
|
|
# T-Invest doubles as a brand-logo source consulted regardless of the price
|
|
# provider, so its token is useful even when it's not enabled for prices.
|
|
# Always surface the token field, decoupled from the securities checklist.
|
|
@show_tinkoff_invest_settings = true
|
|
|
|
# Only fetch provider data if we're showing the section
|
|
if @show_twelve_data_settings
|
|
twelve_data_provider = Provider::Registry.get_provider(:twelve_data)
|
|
@twelve_data_usage = twelve_data_provider&.usage
|
|
@plan_restricted_securities = Current.family.securities_with_plan_restrictions(provider: "TwelveData")
|
|
end
|
|
|
|
if @show_yahoo_finance_settings
|
|
@yahoo_finance_provider = Provider::Registry.get_provider(:yahoo_finance)
|
|
@yahoo_finance_health_status = @yahoo_finance_provider&.health_status || :unknown
|
|
end
|
|
|
|
# Property valuation (AVM) providers — usage is shown against their tight
|
|
# monthly request caps when a key is configured
|
|
@rentcast_usage = Provider::Registry.get_provider(:rentcast)&.usage
|
|
@realie_usage = Provider::Registry.get_provider(:realie)&.usage
|
|
end
|
|
|
|
def update
|
|
if hosting_params.key?(:onboarding_state)
|
|
onboarding_state = hosting_params[:onboarding_state].to_s
|
|
Setting.onboarding_state = onboarding_state
|
|
end
|
|
|
|
if hosting_params.key?(:require_email_confirmation)
|
|
Setting.require_email_confirmation = hosting_params[:require_email_confirmation]
|
|
end
|
|
|
|
if hosting_params.key?(:invite_only_default_family_id)
|
|
value = hosting_params[:invite_only_default_family_id].presence
|
|
Setting.invite_only_default_family_id = value
|
|
end
|
|
|
|
if hosting_params.key?(:brand_fetch_client_id)
|
|
Setting.brand_fetch_client_id = hosting_params[:brand_fetch_client_id]
|
|
end
|
|
|
|
if hosting_params.key?(:brand_fetch_high_res_logos)
|
|
Setting.brand_fetch_high_res_logos = hosting_params[:brand_fetch_high_res_logos] == "1"
|
|
end
|
|
|
|
update_encrypted_setting(:twelve_data_api_key)
|
|
|
|
if hosting_params.key?(:exchange_rate_provider)
|
|
Setting.exchange_rate_provider = hosting_params[:exchange_rate_provider]
|
|
end
|
|
|
|
if hosting_params.key?(:securities_provider)
|
|
Setting.securities_provider = hosting_params[:securities_provider]
|
|
end
|
|
|
|
if hosting_params.key?(:securities_providers)
|
|
new_providers = Array(hosting_params[:securities_providers]).reject(&:blank?) & Security.valid_price_providers
|
|
old_providers = Setting.enabled_securities_providers
|
|
|
|
Setting.securities_providers = new_providers.join(",")
|
|
|
|
# Clear the legacy singular setting so the fallback in
|
|
# enabled_securities_providers doesn't re-enable a provider
|
|
# the user just unchecked.
|
|
Setting.securities_provider = nil if new_providers.empty?
|
|
|
|
# Mark securities linked to removed providers as offline so they aren't
|
|
# silently queried against an incompatible fallback provider (e.g. MFAPI
|
|
# scheme codes sent to TwelveData). The price_provider is preserved so
|
|
# provider_status can report :provider_unavailable.
|
|
removed = old_providers - new_providers
|
|
removed.each do |removed_provider|
|
|
Security.where(price_provider: removed_provider, offline: false)
|
|
.in_batches.update_all(offline: true, offline_reason: "provider_disabled")
|
|
end
|
|
|
|
# Bring securities back online when their provider is re-enabled — but only
|
|
# those that were taken offline by a provider toggle, not by health checks.
|
|
added = new_providers - old_providers
|
|
added.each do |added_provider|
|
|
Security.where(price_provider: added_provider, offline: true, offline_reason: "provider_disabled")
|
|
.in_batches.update_all(offline: false, offline_reason: nil, failed_fetch_count: 0, failed_fetch_at: nil)
|
|
end
|
|
end
|
|
|
|
update_encrypted_setting(:tiingo_api_key)
|
|
update_encrypted_setting(:eodhd_api_key)
|
|
update_encrypted_setting(:alpha_vantage_api_key)
|
|
update_encrypted_setting(:tinkoff_invest_api_key)
|
|
update_encrypted_setting(:rentcast_api_key)
|
|
update_encrypted_setting(:realie_api_key)
|
|
|
|
if hosting_params.key?(:syncs_include_pending)
|
|
Setting.syncs_include_pending = hosting_params[:syncs_include_pending] == "1"
|
|
end
|
|
|
|
sync_settings_changed = false
|
|
|
|
if hosting_params.key?(:auto_sync_enabled)
|
|
Setting.auto_sync_enabled = hosting_params[:auto_sync_enabled] == "1"
|
|
sync_settings_changed = true
|
|
end
|
|
|
|
if hosting_params.key?(:auto_sync_time)
|
|
time_value = hosting_params[:auto_sync_time]
|
|
unless Setting.valid_auto_sync_time?(time_value)
|
|
flash[:alert] = t(".invalid_sync_time")
|
|
return redirect_to settings_hosting_path
|
|
end
|
|
|
|
Setting.auto_sync_time = time_value
|
|
Setting.auto_sync_timezone = current_user_timezone
|
|
sync_settings_changed = true
|
|
end
|
|
|
|
if sync_settings_changed
|
|
sync_auto_sync_scheduler!
|
|
end
|
|
|
|
if hosting_params.key?(:openai_access_token)
|
|
token_param = hosting_params[:openai_access_token].to_s.strip
|
|
# Ignore blanks and redaction placeholders to prevent accidental overwrite
|
|
unless token_param.blank? || token_param == "********"
|
|
Setting.openai_access_token = token_param
|
|
end
|
|
end
|
|
|
|
# Validate OpenAI configuration before updating
|
|
if hosting_params.key?(:openai_uri_base) || hosting_params.key?(:openai_model)
|
|
Setting.validate_openai_config!(
|
|
uri_base: hosting_params[:openai_uri_base],
|
|
model: hosting_params[:openai_model]
|
|
)
|
|
end
|
|
|
|
if hosting_params.key?(:openai_uri_base)
|
|
Setting.openai_uri_base = hosting_params[:openai_uri_base]
|
|
end
|
|
|
|
if hosting_params.key?(:openai_model)
|
|
Setting.openai_model = hosting_params[:openai_model]
|
|
end
|
|
|
|
if hosting_params.key?(:openai_json_mode)
|
|
Setting.openai_json_mode = hosting_params[:openai_json_mode].presence
|
|
end
|
|
|
|
if hosting_params.key?(:anthropic_access_token)
|
|
token_param = hosting_params[:anthropic_access_token].to_s.strip
|
|
unless token_param.blank? || token_param == "********"
|
|
Setting.anthropic_access_token = token_param
|
|
end
|
|
end
|
|
|
|
if hosting_params.key?(:anthropic_base_url)
|
|
raw_base_url = hosting_params[:anthropic_base_url].to_s.strip
|
|
if raw_base_url.blank?
|
|
Setting.anthropic_base_url = nil
|
|
else
|
|
parsed = URI.parse(raw_base_url) rescue nil
|
|
unless parsed.is_a?(URI::HTTP)
|
|
raise Setting::ValidationError, t(".invalid_anthropic_base_url")
|
|
end
|
|
# A custom Anthropic-compatible endpoint requires a model — Provider::Anthropic
|
|
# raises without one. Validate the pair together (mirrors the OpenAI branch), using
|
|
# the submitted model when present so a blanked model field is caught too.
|
|
effective_model =
|
|
if hosting_params.key?(:anthropic_model)
|
|
hosting_params[:anthropic_model].to_s.strip
|
|
else
|
|
Setting.anthropic_model.to_s.strip
|
|
end
|
|
if effective_model.blank?
|
|
raise Setting::ValidationError, t(".anthropic_model_required_for_base_url")
|
|
end
|
|
Setting.anthropic_base_url = raw_base_url
|
|
end
|
|
end
|
|
|
|
if hosting_params.key?(:anthropic_model)
|
|
Setting.anthropic_model = hosting_params[:anthropic_model].presence
|
|
end
|
|
|
|
if hosting_params.key?(:llm_provider)
|
|
provider = hosting_params[:llm_provider].to_s
|
|
if %w[openai anthropic].include?(provider)
|
|
Setting.llm_provider = provider
|
|
end
|
|
end
|
|
|
|
LLM_BUDGET_MINIMUMS.each do |key, minimum|
|
|
next unless hosting_params.key?(key)
|
|
raw = hosting_params[key].to_s.strip
|
|
if raw.blank?
|
|
Setting.public_send("#{key}=", nil)
|
|
next
|
|
end
|
|
parsed = Integer(raw, 10) rescue nil
|
|
if parsed.nil? || parsed < minimum
|
|
label = t("settings.hostings.openai_settings.#{key}_label")
|
|
raise Setting::ValidationError, t(".invalid_llm_budget", field: label, minimum: minimum)
|
|
end
|
|
Setting.public_send("#{key}=", parsed)
|
|
end
|
|
|
|
if hosting_params.key?(:external_assistant_url)
|
|
Setting.external_assistant_url = hosting_params[:external_assistant_url]
|
|
end
|
|
|
|
if hosting_params.key?(:external_assistant_token)
|
|
token_param = hosting_params[:external_assistant_token].to_s.strip
|
|
unless token_param.blank? || token_param == "********"
|
|
Setting.external_assistant_token = token_param
|
|
end
|
|
end
|
|
|
|
if hosting_params.key?(:external_assistant_agent_id)
|
|
Setting.external_assistant_agent_id = hosting_params[:external_assistant_agent_id]
|
|
end
|
|
|
|
update_assistant_type
|
|
|
|
redirect_to settings_hosting_path, notice: t(".success")
|
|
rescue Setting::ValidationError => error
|
|
# Preserve user-submitted OpenAI config so the form re-renders with their
|
|
# input intact (issue #1824). The form auto-submits on blur, so a partial
|
|
# entry (e.g. URI base before model) hits validation and would otherwise
|
|
# be wiped because the view reads from the unchanged Setting.* values.
|
|
@openai_uri_base_input = hosting_params[:openai_uri_base] if hosting_params.key?(:openai_uri_base)
|
|
@openai_model_input = hosting_params[:openai_model] if hosting_params.key?(:openai_model)
|
|
@anthropic_base_url_input = hosting_params[:anthropic_base_url] if hosting_params.key?(:anthropic_base_url)
|
|
@anthropic_model_input = hosting_params[:anthropic_model] if hosting_params.key?(:anthropic_model)
|
|
flash.now[:alert] = error.message
|
|
render :show, status: :unprocessable_entity
|
|
end
|
|
|
|
def clear_cache
|
|
DataCacheClearJob.perform_later(Current.family)
|
|
redirect_to settings_hosting_path, notice: t(".cache_cleared")
|
|
end
|
|
|
|
def disconnect_external_assistant
|
|
Setting.external_assistant_url = nil
|
|
Setting.external_assistant_token = nil
|
|
Setting.external_assistant_agent_id = nil
|
|
Current.family.update!(assistant_type: "builtin") unless ENV["ASSISTANT_TYPE"].present?
|
|
redirect_to settings_hosting_path, notice: t(".external_assistant_disconnected")
|
|
rescue => e
|
|
Rails.logger.error("[External Assistant] Disconnect failed: #{e.message}")
|
|
redirect_to settings_hosting_path, alert: t("settings.hostings.update.failure")
|
|
end
|
|
|
|
private
|
|
def hosting_params
|
|
return ActionController::Parameters.new unless params.key?(:setting)
|
|
params.require(:setting).permit(:onboarding_state, :require_email_confirmation, :invite_only_default_family_id, :brand_fetch_client_id, :brand_fetch_high_res_logos, :twelve_data_api_key, :tiingo_api_key, :eodhd_api_key, :alpha_vantage_api_key, :tinkoff_invest_api_key, :rentcast_api_key, :realie_api_key, :openai_access_token, :openai_uri_base, :openai_model, :openai_json_mode, :anthropic_access_token, :anthropic_base_url, :anthropic_model, :llm_provider, :llm_context_window, :llm_max_response_tokens, :llm_max_items_per_call, :exchange_rate_provider, :securities_provider, :syncs_include_pending, :auto_sync_enabled, :auto_sync_time, :external_assistant_url, :external_assistant_token, :external_assistant_agent_id, securities_providers: [])
|
|
end
|
|
|
|
def update_assistant_type
|
|
return unless params[:family].present? && params[:family][:assistant_type].present?
|
|
return if ENV["ASSISTANT_TYPE"].present?
|
|
|
|
assistant_type = params[:family][:assistant_type]
|
|
Current.family.update!(assistant_type: assistant_type) if Family::ASSISTANT_TYPES.include?(assistant_type)
|
|
end
|
|
|
|
def ensure_admin
|
|
redirect_to settings_hosting_path, alert: t(".not_authorized") unless Current.user.admin?
|
|
end
|
|
|
|
def ensure_super_admin_for_onboarding
|
|
onboarding_params = %i[onboarding_state invite_only_default_family_id]
|
|
return unless onboarding_params.any? { |p| hosting_params.key?(p) }
|
|
redirect_to settings_hosting_path, alert: t(".not_authorized") unless Current.user.super_admin?
|
|
end
|
|
|
|
def sync_auto_sync_scheduler!
|
|
AutoSyncScheduler.sync!
|
|
rescue StandardError => error
|
|
Rails.logger.error("[AutoSyncScheduler] Failed to sync scheduler: #{error.message}")
|
|
Rails.logger.error(error.backtrace.join("\n"))
|
|
flash[:alert] = t(".scheduler_sync_failed")
|
|
end
|
|
|
|
def update_encrypted_setting(param_key)
|
|
return unless hosting_params.key?(param_key)
|
|
value = hosting_params[param_key].to_s.strip
|
|
Setting.public_send(:"#{param_key}=", value) unless value.blank? || value == "********"
|
|
end
|
|
|
|
def current_user_timezone
|
|
Current.family&.timezone.presence || "UTC"
|
|
end
|
|
end
|