Files
sure/app/controllers/settings/hostings_controller.rb
LPW 84bfe5b7ab Add external AI assistant with Pipelock security proxy (#1069)
* feat(helm): add Pipelock ConfigMap, scanning config, and consolidate compose

- Add ConfigMap template rendering DLP, response scanning, MCP input/tool
  scanning, and forward proxy settings from values
- Mount ConfigMap as /etc/pipelock/pipelock.yaml volume in deployment
- Add checksum/config annotation for automatic pod restart on config change
- Gate HTTPS_PROXY/HTTP_PROXY env injection on forwardProxy.enabled (skip
  in MCP-only mode)
- Use hasKey for all boolean values to prevent Helm default swallowing false
- Single source of truth for ports (forwardProxy.port/mcpProxy.port)
- Pipelock-specific imagePullSecrets with fallback to app secrets
- Merge standalone compose.example.pipelock.yml into compose.example.ai.yml
- Add pipelock.example.yaml for Docker Compose users
- Add exclude-paths to CI workflow for locale file false positives

* Add external assistant support (OpenAI-compatible SSE proxy)

Allow self-hosted instances to delegate chat to an external AI agent
via an OpenAI-compatible streaming endpoint. Configurable per-family
through Settings UI or ASSISTANT_TYPE env override.

- Assistant::External::Client: SSE streaming HTTP client (no new gems)
- Settings UI with type selector, env lock indicator, config status
- Helm chart and Docker Compose env var support
- 45 tests covering client, config, routing, controller, integration

* Add session key routing, email allowlist, and config plumbing

Route to the actual OpenClaw session via x-openclaw-session-key header
instead of creating isolated sessions. Gate external assistant access
behind an email allowlist (EXTERNAL_ASSISTANT_ALLOWED_EMAILS env var).
Plumb session_key and allowedEmails through Helm chart, compose, and
env template.

* Add HTTPS_PROXY support to External::Client for Pipelock integration

Net::HTTP does not auto-read HTTPS_PROXY/HTTP_PROXY env vars (unlike
Faraday). Explicitly resolve proxy from environment in build_http so
outbound traffic to the external assistant routes through Pipelock's
forward proxy when enabled. Respects NO_PROXY for internal hosts.

* Add UI fields for external assistant config (Setting-backed with env fallback)

Follow the same pattern as OpenAI settings: database-backed Setting
fields with env var defaults. Self-hosters can now configure the
external assistant URL, token, and agent ID from the browser
(Settings > Self-Hosting > AI Assistant) instead of requiring env vars.
Fields disable when the corresponding env var is set.

* Improve external assistant UI labels and add help text

Change placeholder to generic OpenAI-compatible URL pattern. Add help
text under each field explaining where the values come from: URL from
agent provider, token for authentication, agent ID for multi-agent
routing.

* Add external assistant docs and fix URL help text

Add External AI Assistant section to docs/hosting/ai.md covering setup
(UI and env vars), how it works, Pipelock security scanning, access
control, and Docker Compose example. Drop "chat completions" jargon
from URL help text.

* Harden external assistant: retry logic, disconnect UI, error handling, and test coverage

- Add retry with backoff for transient network errors (no retry after streaming starts)
- Add disconnect button with confirmation modal in self-hosting settings
- Narrow rescue scope with fallback logging for unexpected errors
- Safe cleanup of partial responses on stream interruption
- Gate ai_available? on family assistant_type instead of OR-ing all providers
- Truncate conversation history to last 20 messages
- Proxy-aware HTTP client with NO_PROXY support
- Sanitize protocol to use generic headers (X-Agent-Id, X-Session-Key)
- Full test coverage for streaming, retries, proxy routing, config, and disconnect

* Exclude external assistant client from Pipelock scan-diff

False positive: `@token` instance variable flagged as "Credential in URL".
Temporary workaround until Pipelock supports inline suppression.

* Address review feedback: NO_PROXY boundary fix, SSE done flag, design tokens

- Fix NO_PROXY matching to require domain boundary (exact match or .suffix),
  case-insensitive. Prevents badexample.com matching example.com.
- Add done flag to SSE streaming so read_body stops after [DONE]
- Move MAX_CONVERSATION_MESSAGES to class level
- Use bg-success/bg-destructive design tokens for status indicators
- Add rationale comment for pipelock scan exclusion
- Update docs last-updated date

* Address second round of review feedback

- Allowlist email comparison is now case-insensitive and nil-safe
- Cap SSE buffer at 1 MB to prevent memory blowup from malformed streams
- Don't expose upstream HTTP response body in user-facing errors (log it instead)
- Fix frozen string warning on buffer initialization
- Fix "builtin" typo in docs (should be "built-in")

* Protect completed responses from cleanup, sanitize error messages

- Don't destroy a fully streamed assistant message if post-stream
  metadata update fails (only cleanup partial responses)
- Log raw connection/HTTP errors internally, show generic messages
  to users to avoid leaking network/proxy details
- Update test assertions for new error message wording

* Fix SSE content guard and NO_PROXY test correctness

Use nil check instead of present? for SSE delta content to preserve
whitespace-only chunks (newlines, spaces) that can occur in code output.

Fix NO_PROXY test to use HTTP_PROXY matching the http:// client URL so
the proxy resolution and NO_PROXY bypass logic are actually exercised.

* Forward proxy credentials to Net::HTTP

Pass proxy_uri.user and proxy_uri.password to Net::HTTP.new so
authenticated proxies (http://user:pass@host:port) work correctly.
Without this, credentials parsed from the proxy URL were silently
dropped. Nil values are safe as positional args when no creds exist.

* Update pipelock integration to v0.3.1 with full scanning config

Bump Helm image tag from 0.2.7 to 0.3.1. Add missing security
sections to both the Helm ConfigMap and compose example config:
mcp_tool_policy, mcp_session_binding, and tool_chain_detection.
These protect the /mcp endpoint against tool injection, session
hijacking, and multi-step exfiltration chains.

Add version and mode fields to config files. Enable include_defaults
for DLP and response scanning to merge user patterns with the 35
built-in patterns. Remove redundant --mode CLI flag from the Helm
deployment template since mode is now in the config file.
2026-03-03 15:47:51 +01:00

190 lines
6.9 KiB
Ruby

class Settings::HostingsController < ApplicationController
layout "settings"
guard_feature unless: -> { self_hosted? }
before_action :ensure_admin, only: [ :update, :clear_cache, :disconnect_external_assistant ]
def show
@breadcrumbs = [
[ "Home", root_path ],
[ "Self-Hosting", nil ]
]
# Determine which providers are currently selected
exchange_rate_provider = ENV["EXCHANGE_RATE_PROVIDER"].presence || Setting.exchange_rate_provider
securities_provider = ENV["SECURITIES_PROVIDER"].presence || Setting.securities_provider
# Show Twelve Data settings if either provider is set to twelve_data
@show_twelve_data_settings = exchange_rate_provider == "twelve_data" || securities_provider == "twelve_data"
# Show Yahoo Finance settings if either provider is set to yahoo_finance
@show_yahoo_finance_settings = exchange_rate_provider == "yahoo_finance" || securities_provider == "yahoo_finance"
# Only fetch provider data if we're showing the section
if @show_twelve_data_settings
twelve_data_provider = Provider::Registry.get_provider(:twelve_data)
@twelve_data_usage = twelve_data_provider&.usage
@plan_restricted_securities = Current.family.securities_with_plan_restrictions(provider: "TwelveData")
end
if @show_yahoo_finance_settings
@yahoo_finance_provider = Provider::Registry.get_provider(:yahoo_finance)
end
end
def update
if hosting_params.key?(:onboarding_state)
onboarding_state = hosting_params[:onboarding_state].to_s
Setting.onboarding_state = onboarding_state
end
if hosting_params.key?(:require_email_confirmation)
Setting.require_email_confirmation = hosting_params[:require_email_confirmation]
end
if hosting_params.key?(:brand_fetch_client_id)
Setting.brand_fetch_client_id = hosting_params[:brand_fetch_client_id]
end
if hosting_params.key?(:brand_fetch_high_res_logos)
Setting.brand_fetch_high_res_logos = hosting_params[:brand_fetch_high_res_logos] == "1"
end
if hosting_params.key?(:twelve_data_api_key)
Setting.twelve_data_api_key = hosting_params[:twelve_data_api_key]
end
if hosting_params.key?(:exchange_rate_provider)
Setting.exchange_rate_provider = hosting_params[:exchange_rate_provider]
end
if hosting_params.key?(:securities_provider)
Setting.securities_provider = hosting_params[:securities_provider]
end
if hosting_params.key?(:syncs_include_pending)
Setting.syncs_include_pending = hosting_params[:syncs_include_pending] == "1"
end
sync_settings_changed = false
if hosting_params.key?(:auto_sync_enabled)
Setting.auto_sync_enabled = hosting_params[:auto_sync_enabled] == "1"
sync_settings_changed = true
end
if hosting_params.key?(:auto_sync_time)
time_value = hosting_params[:auto_sync_time]
unless Setting.valid_auto_sync_time?(time_value)
flash[:alert] = t(".invalid_sync_time")
return redirect_to settings_hosting_path
end
Setting.auto_sync_time = time_value
Setting.auto_sync_timezone = current_user_timezone
sync_settings_changed = true
end
if sync_settings_changed
sync_auto_sync_scheduler!
end
if hosting_params.key?(:openai_access_token)
token_param = hosting_params[:openai_access_token].to_s.strip
# Ignore blanks and redaction placeholders to prevent accidental overwrite
unless token_param.blank? || token_param == "********"
Setting.openai_access_token = token_param
end
end
# Validate OpenAI configuration before updating
if hosting_params.key?(:openai_uri_base) || hosting_params.key?(:openai_model)
Setting.validate_openai_config!(
uri_base: hosting_params[:openai_uri_base],
model: hosting_params[:openai_model]
)
end
if hosting_params.key?(:openai_uri_base)
Setting.openai_uri_base = hosting_params[:openai_uri_base]
end
if hosting_params.key?(:openai_model)
Setting.openai_model = hosting_params[:openai_model]
end
if hosting_params.key?(:openai_json_mode)
Setting.openai_json_mode = hosting_params[:openai_json_mode].presence
end
if hosting_params.key?(:external_assistant_url)
Setting.external_assistant_url = hosting_params[:external_assistant_url]
end
if hosting_params.key?(:external_assistant_token)
token_param = hosting_params[:external_assistant_token].to_s.strip
unless token_param.blank? || token_param == "********"
Setting.external_assistant_token = token_param
end
end
if hosting_params.key?(:external_assistant_agent_id)
Setting.external_assistant_agent_id = hosting_params[:external_assistant_agent_id]
end
update_assistant_type
redirect_to settings_hosting_path, notice: t(".success")
rescue Setting::ValidationError => error
flash.now[:alert] = error.message
render :show, status: :unprocessable_entity
end
def clear_cache
DataCacheClearJob.perform_later(Current.family)
redirect_to settings_hosting_path, notice: t(".cache_cleared")
end
def disconnect_external_assistant
Setting.external_assistant_url = nil
Setting.external_assistant_token = nil
Setting.external_assistant_agent_id = nil
Current.family.update!(assistant_type: "builtin") unless ENV["ASSISTANT_TYPE"].present?
redirect_to settings_hosting_path, notice: t(".external_assistant_disconnected")
rescue => e
Rails.logger.error("[External Assistant] Disconnect failed: #{e.message}")
redirect_to settings_hosting_path, alert: t("settings.hostings.update.failure")
end
private
def hosting_params
return ActionController::Parameters.new unless params.key?(:setting)
params.require(:setting).permit(:onboarding_state, :require_email_confirmation, :brand_fetch_client_id, :brand_fetch_high_res_logos, :twelve_data_api_key, :openai_access_token, :openai_uri_base, :openai_model, :openai_json_mode, :exchange_rate_provider, :securities_provider, :syncs_include_pending, :auto_sync_enabled, :auto_sync_time, :external_assistant_url, :external_assistant_token, :external_assistant_agent_id)
end
def update_assistant_type
return unless params[:family].present? && params[:family][:assistant_type].present?
return if ENV["ASSISTANT_TYPE"].present?
assistant_type = params[:family][:assistant_type]
Current.family.update!(assistant_type: assistant_type) if Family::ASSISTANT_TYPES.include?(assistant_type)
end
def ensure_admin
redirect_to settings_hosting_path, alert: t(".not_authorized") unless Current.user.admin?
end
def sync_auto_sync_scheduler!
AutoSyncScheduler.sync!
rescue StandardError => error
Rails.logger.error("[AutoSyncScheduler] Failed to sync scheduler: #{error.message}")
Rails.logger.error(error.backtrace.join("\n"))
flash[:alert] = t(".scheduler_sync_failed")
end
def current_user_timezone
Current.family&.timezone.presence || "UTC"
end
end