Files
sure/test/controllers/settings/hostings_controller_test.rb
LPW 84bfe5b7ab Add external AI assistant with Pipelock security proxy (#1069)
* feat(helm): add Pipelock ConfigMap, scanning config, and consolidate compose

- Add ConfigMap template rendering DLP, response scanning, MCP input/tool
  scanning, and forward proxy settings from values
- Mount ConfigMap as /etc/pipelock/pipelock.yaml volume in deployment
- Add checksum/config annotation for automatic pod restart on config change
- Gate HTTPS_PROXY/HTTP_PROXY env injection on forwardProxy.enabled (skip
  in MCP-only mode)
- Use hasKey for all boolean values to prevent Helm default swallowing false
- Single source of truth for ports (forwardProxy.port/mcpProxy.port)
- Pipelock-specific imagePullSecrets with fallback to app secrets
- Merge standalone compose.example.pipelock.yml into compose.example.ai.yml
- Add pipelock.example.yaml for Docker Compose users
- Add exclude-paths to CI workflow for locale file false positives

* Add external assistant support (OpenAI-compatible SSE proxy)

Allow self-hosted instances to delegate chat to an external AI agent
via an OpenAI-compatible streaming endpoint. Configurable per-family
through Settings UI or ASSISTANT_TYPE env override.

- Assistant::External::Client: SSE streaming HTTP client (no new gems)
- Settings UI with type selector, env lock indicator, config status
- Helm chart and Docker Compose env var support
- 45 tests covering client, config, routing, controller, integration

* Add session key routing, email allowlist, and config plumbing

Route to the actual OpenClaw session via x-openclaw-session-key header
instead of creating isolated sessions. Gate external assistant access
behind an email allowlist (EXTERNAL_ASSISTANT_ALLOWED_EMAILS env var).
Plumb session_key and allowedEmails through Helm chart, compose, and
env template.

* Add HTTPS_PROXY support to External::Client for Pipelock integration

Net::HTTP does not auto-read HTTPS_PROXY/HTTP_PROXY env vars (unlike
Faraday). Explicitly resolve proxy from environment in build_http so
outbound traffic to the external assistant routes through Pipelock's
forward proxy when enabled. Respects NO_PROXY for internal hosts.

* Add UI fields for external assistant config (Setting-backed with env fallback)

Follow the same pattern as OpenAI settings: database-backed Setting
fields with env var defaults. Self-hosters can now configure the
external assistant URL, token, and agent ID from the browser
(Settings > Self-Hosting > AI Assistant) instead of requiring env vars.
Fields disable when the corresponding env var is set.

* Improve external assistant UI labels and add help text

Change placeholder to generic OpenAI-compatible URL pattern. Add help
text under each field explaining where the values come from: URL from
agent provider, token for authentication, agent ID for multi-agent
routing.

* Add external assistant docs and fix URL help text

Add External AI Assistant section to docs/hosting/ai.md covering setup
(UI and env vars), how it works, Pipelock security scanning, access
control, and Docker Compose example. Drop "chat completions" jargon
from URL help text.

* Harden external assistant: retry logic, disconnect UI, error handling, and test coverage

- Add retry with backoff for transient network errors (no retry after streaming starts)
- Add disconnect button with confirmation modal in self-hosting settings
- Narrow rescue scope with fallback logging for unexpected errors
- Safe cleanup of partial responses on stream interruption
- Gate ai_available? on family assistant_type instead of OR-ing all providers
- Truncate conversation history to last 20 messages
- Proxy-aware HTTP client with NO_PROXY support
- Sanitize protocol to use generic headers (X-Agent-Id, X-Session-Key)
- Full test coverage for streaming, retries, proxy routing, config, and disconnect

* Exclude external assistant client from Pipelock scan-diff

False positive: `@token` instance variable flagged as "Credential in URL".
Temporary workaround until Pipelock supports inline suppression.

* Address review feedback: NO_PROXY boundary fix, SSE done flag, design tokens

- Fix NO_PROXY matching to require domain boundary (exact match or .suffix),
  case-insensitive. Prevents badexample.com matching example.com.
- Add done flag to SSE streaming so read_body stops after [DONE]
- Move MAX_CONVERSATION_MESSAGES to class level
- Use bg-success/bg-destructive design tokens for status indicators
- Add rationale comment for pipelock scan exclusion
- Update docs last-updated date

* Address second round of review feedback

- Allowlist email comparison is now case-insensitive and nil-safe
- Cap SSE buffer at 1 MB to prevent memory blowup from malformed streams
- Don't expose upstream HTTP response body in user-facing errors (log it instead)
- Fix frozen string warning on buffer initialization
- Fix "builtin" typo in docs (should be "built-in")

* Protect completed responses from cleanup, sanitize error messages

- Don't destroy a fully streamed assistant message if post-stream
  metadata update fails (only cleanup partial responses)
- Log raw connection/HTTP errors internally, show generic messages
  to users to avoid leaking network/proxy details
- Update test assertions for new error message wording

* Fix SSE content guard and NO_PROXY test correctness

Use nil check instead of present? for SSE delta content to preserve
whitespace-only chunks (newlines, spaces) that can occur in code output.

Fix NO_PROXY test to use HTTP_PROXY matching the http:// client URL so
the proxy resolution and NO_PROXY bypass logic are actually exercised.

* Forward proxy credentials to Net::HTTP

Pass proxy_uri.user and proxy_uri.password to Net::HTTP.new so
authenticated proxies (http://user:pass@host:port) work correctly.
Without this, credentials parsed from the proxy URL were silently
dropped. Nil values are safe as positional args when no creds exist.

* Update pipelock integration to v0.3.1 with full scanning config

Bump Helm image tag from 0.2.7 to 0.3.1. Add missing security
sections to both the Helm ConfigMap and compose example config:
mcp_tool_policy, mcp_session_binding, and tool_chain_detection.
These protect the /mcp endpoint against tool injection, session
hijacking, and multi-step exfiltration chains.

Add version and mode fields to config files. Enable include_defaults
for DLP and response scanning to merge user patterns with the 35
built-in patterns. Remove redundant --mode CLI flag from the Helm
deployment template since mode is now in the config file.
2026-03-03 15:47:51 +01:00

244 lines
8.0 KiB
Ruby

require "test_helper"
require "ostruct"
class Settings::HostingsControllerTest < ActionDispatch::IntegrationTest
include ProviderTestHelper
setup do
sign_in users(:family_admin)
@provider = mock
Provider::Registry.stubs(:get_provider).with(:twelve_data).returns(@provider)
@provider.stubs(:healthy?).returns(true)
Provider::Registry.stubs(:get_provider).with(:yahoo_finance).returns(@provider)
@provider.stubs(:usage).returns(provider_success_response(
OpenStruct.new(
used: 10,
limit: 100,
utilization: 10,
plan: "free",
)
))
end
test "cannot edit when self hosting is disabled" do
@provider.stubs(:usage).returns(@usage_response)
Rails.configuration.stubs(:app_mode).returns("managed".inquiry)
get settings_hosting_url
assert_response :forbidden
patch settings_hosting_url, params: { setting: { onboarding_state: "invite_only" } }
assert_response :forbidden
end
test "should get edit when self hosting is enabled" do
@provider.expects(:usage).returns(@usage_response)
with_self_hosting do
get settings_hosting_url
assert_response :success
end
end
test "can update settings when self hosting is enabled" do
with_self_hosting do
patch settings_hosting_url, params: { setting: { twelve_data_api_key: "1234567890" } }
assert_equal "1234567890", Setting.twelve_data_api_key
end
end
test "can update onboarding state when self hosting is enabled" do
with_self_hosting do
patch settings_hosting_url, params: { setting: { onboarding_state: "invite_only" } }
assert_equal "invite_only", Setting.onboarding_state
assert Setting.require_invite_for_signup
patch settings_hosting_url, params: { setting: { onboarding_state: "closed" } }
assert_equal "closed", Setting.onboarding_state
refute Setting.require_invite_for_signup
end
end
test "can update openai access token when self hosting is enabled" do
with_self_hosting do
patch settings_hosting_url, params: { setting: { openai_access_token: "token" } }
assert_equal "token", Setting.openai_access_token
end
end
test "can update openai uri base and model together when self hosting is enabled" do
with_self_hosting do
patch settings_hosting_url, params: { setting: { openai_uri_base: "https://api.example.com/v1", openai_model: "gpt-4" } }
assert_equal "https://api.example.com/v1", Setting.openai_uri_base
assert_equal "gpt-4", Setting.openai_model
end
end
test "cannot update openai uri base without model when self hosting is enabled" do
with_self_hosting do
Setting.openai_model = ""
patch settings_hosting_url, params: { setting: { openai_uri_base: "https://api.example.com/v1" } }
assert_response :unprocessable_entity
assert_match(/OpenAI model is required/, flash[:alert])
assert Setting.openai_uri_base.blank?, "Expected openai_uri_base to remain blank after failed validation"
end
end
test "can update openai model alone when self hosting is enabled" do
with_self_hosting do
patch settings_hosting_url, params: { setting: { openai_model: "gpt-4" } }
assert_equal "gpt-4", Setting.openai_model
end
end
test "cannot clear openai model when custom uri base is set" do
with_self_hosting do
Setting.openai_uri_base = "https://api.example.com/v1"
Setting.openai_model = "gpt-4"
patch settings_hosting_url, params: { setting: { openai_model: "" } }
assert_response :unprocessable_entity
assert_match(/OpenAI model is required/, flash[:alert])
assert_equal "gpt-4", Setting.openai_model
end
end
test "can clear data cache when self hosting is enabled" do
account = accounts(:investment)
holding = account.holdings.first
exchange_rate = exchange_rates(:one)
security_price = holding.security.prices.first
account_balance = account.balances.create!(date: Date.current, balance: 1000, currency: "USD")
with_self_hosting do
perform_enqueued_jobs(only: DataCacheClearJob) do
delete clear_cache_settings_hosting_url
end
end
assert_redirected_to settings_hosting_url
assert_equal I18n.t("settings.hostings.clear_cache.cache_cleared"), flash[:notice]
assert_not ExchangeRate.exists?(exchange_rate.id)
assert_not Security::Price.exists?(security_price.id)
assert_not Holding.exists?(holding.id)
assert_not Balance.exists?(account_balance.id)
end
test "can update assistant type to external" do
with_self_hosting do
assert_equal "builtin", users(:family_admin).family.assistant_type
patch settings_hosting_url, params: { family: { assistant_type: "external" } }
assert_redirected_to settings_hosting_url
assert_equal "external", users(:family_admin).family.reload.assistant_type
end
end
test "ignores invalid assistant type values" do
with_self_hosting do
patch settings_hosting_url, params: { family: { assistant_type: "hacked" } }
assert_redirected_to settings_hosting_url
assert_equal "builtin", users(:family_admin).family.reload.assistant_type
end
end
test "ignores assistant type update when ASSISTANT_TYPE env is set" do
with_self_hosting do
with_env_overrides("ASSISTANT_TYPE" => "external") do
patch settings_hosting_url, params: { family: { assistant_type: "external" } }
assert_redirected_to settings_hosting_url
# DB value should NOT change when env override is active
assert_equal "builtin", users(:family_admin).family.reload.assistant_type
end
end
end
test "can update external assistant settings" do
with_self_hosting do
patch settings_hosting_url, params: { setting: {
external_assistant_url: "https://agent.example.com/v1/chat",
external_assistant_token: "my-secret-token",
external_assistant_agent_id: "finance-bot"
} }
assert_redirected_to settings_hosting_url
assert_equal "https://agent.example.com/v1/chat", Setting.external_assistant_url
assert_equal "my-secret-token", Setting.external_assistant_token
assert_equal "finance-bot", Setting.external_assistant_agent_id
end
ensure
Setting.external_assistant_url = nil
Setting.external_assistant_token = nil
Setting.external_assistant_agent_id = nil
end
test "does not overwrite token with masked placeholder" do
with_self_hosting do
Setting.external_assistant_token = "real-secret"
patch settings_hosting_url, params: { setting: { external_assistant_token: "********" } }
assert_equal "real-secret", Setting.external_assistant_token
end
ensure
Setting.external_assistant_token = nil
end
test "disconnect external assistant clears settings and resets type" do
with_self_hosting do
Setting.external_assistant_url = "https://agent.example.com/v1/chat"
Setting.external_assistant_token = "token"
Setting.external_assistant_agent_id = "finance-bot"
users(:family_admin).family.update!(assistant_type: "external")
delete disconnect_external_assistant_settings_hosting_url
assert_redirected_to settings_hosting_url
assert_not Assistant::External.configured?
assert_equal "builtin", users(:family_admin).family.reload.assistant_type
end
ensure
Setting.external_assistant_url = nil
Setting.external_assistant_token = nil
Setting.external_assistant_agent_id = nil
end
test "disconnect external assistant requires admin" do
with_self_hosting do
sign_in users(:family_member)
delete disconnect_external_assistant_settings_hosting_url
assert_redirected_to settings_hosting_url
assert_equal I18n.t("settings.hostings.not_authorized"), flash[:alert]
end
end
test "can clear data only when admin" do
with_self_hosting do
sign_in users(:family_member)
assert_no_enqueued_jobs do
delete clear_cache_settings_hosting_url
end
assert_redirected_to settings_hosting_url
assert_equal I18n.t("settings.hostings.not_authorized"), flash[:alert]
end
end
end