Files
sure/compose.example.ai.yml
LPW 84bfe5b7ab Add external AI assistant with Pipelock security proxy (#1069)
* feat(helm): add Pipelock ConfigMap, scanning config, and consolidate compose

- Add ConfigMap template rendering DLP, response scanning, MCP input/tool
  scanning, and forward proxy settings from values
- Mount ConfigMap as /etc/pipelock/pipelock.yaml volume in deployment
- Add checksum/config annotation for automatic pod restart on config change
- Gate HTTPS_PROXY/HTTP_PROXY env injection on forwardProxy.enabled (skip
  in MCP-only mode)
- Use hasKey for all boolean values to prevent Helm default swallowing false
- Single source of truth for ports (forwardProxy.port/mcpProxy.port)
- Pipelock-specific imagePullSecrets with fallback to app secrets
- Merge standalone compose.example.pipelock.yml into compose.example.ai.yml
- Add pipelock.example.yaml for Docker Compose users
- Add exclude-paths to CI workflow for locale file false positives

* Add external assistant support (OpenAI-compatible SSE proxy)

Allow self-hosted instances to delegate chat to an external AI agent
via an OpenAI-compatible streaming endpoint. Configurable per-family
through Settings UI or ASSISTANT_TYPE env override.

- Assistant::External::Client: SSE streaming HTTP client (no new gems)
- Settings UI with type selector, env lock indicator, config status
- Helm chart and Docker Compose env var support
- 45 tests covering client, config, routing, controller, integration

* Add session key routing, email allowlist, and config plumbing

Route to the actual OpenClaw session via x-openclaw-session-key header
instead of creating isolated sessions. Gate external assistant access
behind an email allowlist (EXTERNAL_ASSISTANT_ALLOWED_EMAILS env var).
Plumb session_key and allowedEmails through Helm chart, compose, and
env template.

* Add HTTPS_PROXY support to External::Client for Pipelock integration

Net::HTTP does not auto-read HTTPS_PROXY/HTTP_PROXY env vars (unlike
Faraday). Explicitly resolve proxy from environment in build_http so
outbound traffic to the external assistant routes through Pipelock's
forward proxy when enabled. Respects NO_PROXY for internal hosts.

* Add UI fields for external assistant config (Setting-backed with env fallback)

Follow the same pattern as OpenAI settings: database-backed Setting
fields with env var defaults. Self-hosters can now configure the
external assistant URL, token, and agent ID from the browser
(Settings > Self-Hosting > AI Assistant) instead of requiring env vars.
Fields disable when the corresponding env var is set.

* Improve external assistant UI labels and add help text

Change placeholder to generic OpenAI-compatible URL pattern. Add help
text under each field explaining where the values come from: URL from
agent provider, token for authentication, agent ID for multi-agent
routing.

* Add external assistant docs and fix URL help text

Add External AI Assistant section to docs/hosting/ai.md covering setup
(UI and env vars), how it works, Pipelock security scanning, access
control, and Docker Compose example. Drop "chat completions" jargon
from URL help text.

* Harden external assistant: retry logic, disconnect UI, error handling, and test coverage

- Add retry with backoff for transient network errors (no retry after streaming starts)
- Add disconnect button with confirmation modal in self-hosting settings
- Narrow rescue scope with fallback logging for unexpected errors
- Safe cleanup of partial responses on stream interruption
- Gate ai_available? on family assistant_type instead of OR-ing all providers
- Truncate conversation history to last 20 messages
- Proxy-aware HTTP client with NO_PROXY support
- Sanitize protocol to use generic headers (X-Agent-Id, X-Session-Key)
- Full test coverage for streaming, retries, proxy routing, config, and disconnect

* Exclude external assistant client from Pipelock scan-diff

False positive: `@token` instance variable flagged as "Credential in URL".
Temporary workaround until Pipelock supports inline suppression.

* Address review feedback: NO_PROXY boundary fix, SSE done flag, design tokens

- Fix NO_PROXY matching to require domain boundary (exact match or .suffix),
  case-insensitive. Prevents badexample.com matching example.com.
- Add done flag to SSE streaming so read_body stops after [DONE]
- Move MAX_CONVERSATION_MESSAGES to class level
- Use bg-success/bg-destructive design tokens for status indicators
- Add rationale comment for pipelock scan exclusion
- Update docs last-updated date

* Address second round of review feedback

- Allowlist email comparison is now case-insensitive and nil-safe
- Cap SSE buffer at 1 MB to prevent memory blowup from malformed streams
- Don't expose upstream HTTP response body in user-facing errors (log it instead)
- Fix frozen string warning on buffer initialization
- Fix "builtin" typo in docs (should be "built-in")

* Protect completed responses from cleanup, sanitize error messages

- Don't destroy a fully streamed assistant message if post-stream
  metadata update fails (only cleanup partial responses)
- Log raw connection/HTTP errors internally, show generic messages
  to users to avoid leaking network/proxy details
- Update test assertions for new error message wording

* Fix SSE content guard and NO_PROXY test correctness

Use nil check instead of present? for SSE delta content to preserve
whitespace-only chunks (newlines, spaces) that can occur in code output.

Fix NO_PROXY test to use HTTP_PROXY matching the http:// client URL so
the proxy resolution and NO_PROXY bypass logic are actually exercised.

* Forward proxy credentials to Net::HTTP

Pass proxy_uri.user and proxy_uri.password to Net::HTTP.new so
authenticated proxies (http://user:pass@host:port) work correctly.
Without this, credentials parsed from the proxy URL were silently
dropped. Nil values are safe as positional args when no creds exist.

* Update pipelock integration to v0.3.1 with full scanning config

Bump Helm image tag from 0.2.7 to 0.3.1. Add missing security
sections to both the Helm ConfigMap and compose example config:
mcp_tool_policy, mcp_session_binding, and tool_chain_detection.
These protect the /mcp endpoint against tool injection, session
hijacking, and multi-step exfiltration chains.

Add version and mode fields to config files. Enable include_defaults
for DLP and response scanning to merge user patterns with the 35
built-in patterns. Remove redundant --mode CLI flag from the Helm
deployment template since mode is now in the config file.
2026-03-03 15:47:51 +01:00

274 lines
9.0 KiB
YAML

# ===========================================================================
# Example Docker Compose file with Ollama (local LLM) and Pipelock (agent
# security proxy)
# ===========================================================================
#
# Purpose:
# --------
#
# This file extends the standard Sure setup with two optional capabilities:
#
# Pipelock — agent security proxy (always runs)
# - Forward proxy (port 8888): scans outbound HTTPS from Faraday-based
# clients (e.g. ruby-openai). NOT covered: SimpleFin, Coinbase, or
# anything using Net::HTTP/HTTParty directly. HTTPS_PROXY is
# cooperative; Docker Compose has no egress network policy.
# - MCP reverse proxy (port 8889): scans inbound AI traffic (DLP,
# prompt injection, tool poisoning, tool call policy). External AI
# clients should connect to Pipelock on port 8889 rather than
# directly to Sure's /mcp endpoint. Note: /mcp is still reachable
# on web port 3000 (auth token required); Pipelock adds scanning
# but Docker Compose cannot enforce network-level routing.
#
# Ollama + Open WebUI — local LLM inference (optional, --profile ai)
# - Only starts when you run: docker compose --profile ai up
#
# Setup:
# ------
#
# 1. Copy pipelock.example.yaml alongside this file (or customize it).
# 2. Read the full setup guide:
#
# https://github.com/we-promise/sure/blob/main/docs/hosting/docker.md
#
# Troubleshooting:
# ----------------
#
# If you run into problems, you should open a Discussion here:
#
# https://github.com/we-promise/sure/discussions/categories/general
#
x-db-env: &db_env
POSTGRES_USER: ${POSTGRES_USER:-sure_user}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-sure_password}
POSTGRES_DB: ${POSTGRES_DB:-sure_production}
x-rails-env: &rails_env
<<: *db_env
SECRET_KEY_BASE: ${SECRET_KEY_BASE:-a7523c3d0ae56415046ad8abae168d71074a79534a7062258f8d1d51ac2f76d3c3bc86d86b6b0b307df30d9a6a90a2066a3fa9e67c5e6f374dbd7dd4e0778e13}
SELF_HOSTED: "true"
RAILS_FORCE_SSL: "false"
RAILS_ASSUME_SSL: "false"
DB_HOST: db
DB_PORT: 5432
REDIS_URL: redis://redis:6379/1
# MCP server endpoint — enables /mcp for external AI assistants (e.g. Claude, GPT).
# Set both values to activate. MCP_USER_EMAIL must match an existing user's email.
# External AI clients should connect via Pipelock (port 8889) for scanning.
MCP_API_TOKEN: ${MCP_API_TOKEN:-}
MCP_USER_EMAIL: ${MCP_USER_EMAIL:-}
# Route outbound HTTPS through Pipelock for clients that respect HTTPS_PROXY.
# Covered: OpenAI API (ruby-openai/Faraday). NOT covered: SimpleFin, Coinbase (Net::HTTP).
HTTPS_PROXY: "http://pipelock:8888"
HTTP_PROXY: "http://pipelock:8888"
# Skip proxy for internal Docker network services (including ollama for local LLM calls)
NO_PROXY: "db,redis,pipelock,ollama,localhost,127.0.0.1"
AI_DEBUG_MODE: "true" # Useful for debugging, set to "false" in production
# Ollama using OpenAI API compatible endpoints
OPENAI_ACCESS_TOKEN: token-can-be-any-value-for-ollama
OPENAI_MODEL: llama3.1:8b # Note: Use tool-enabled model
OPENAI_URI_BASE: http://ollama:11434/v1
# NOTE: enabling OpenAI will incur costs when you use AI-related features in the app (chat, rules). Make sure you have set appropriate spend limits on your account before adding this.
# OPENAI_ACCESS_TOKEN: ${OPENAI_ACCESS_TOKEN}
# External AI Assistant — delegates chat to a remote AI agent (e.g., OpenClaw).
# The agent calls back to Sure's /mcp endpoint for financial data.
# Set EXTERNAL_ASSISTANT_URL + TOKEN to activate, then either set ASSISTANT_TYPE=external
# here (forces all families) or choose "External" in Settings > Self-Hosting > AI Assistant.
ASSISTANT_TYPE: ${ASSISTANT_TYPE:-}
EXTERNAL_ASSISTANT_URL: ${EXTERNAL_ASSISTANT_URL:-}
EXTERNAL_ASSISTANT_TOKEN: ${EXTERNAL_ASSISTANT_TOKEN:-}
EXTERNAL_ASSISTANT_AGENT_ID: ${EXTERNAL_ASSISTANT_AGENT_ID:-main}
EXTERNAL_ASSISTANT_SESSION_KEY: ${EXTERNAL_ASSISTANT_SESSION_KEY:-agent:main:main}
EXTERNAL_ASSISTANT_ALLOWED_EMAILS: ${EXTERNAL_ASSISTANT_ALLOWED_EMAILS:-}
services:
pipelock:
image: ghcr.io/luckypipewrench/pipelock:latest # pin to a specific version (e.g., :0.2.7) for production
container_name: pipelock
hostname: pipelock
restart: unless-stopped
volumes:
- ./pipelock.example.yaml:/etc/pipelock/pipelock.yaml:ro
command:
- "run"
- "--config"
- "/etc/pipelock/pipelock.yaml"
- "--listen"
- "0.0.0.0:8888"
- "--mcp-listen"
- "0.0.0.0:8889"
- "--mcp-upstream"
- "http://web:3000/mcp"
ports:
# MCP reverse proxy — external AI assistants connect here
- "${MCP_PROXY_PORT:-8889}:8889"
# Uncomment to expose forward proxy endpoints (/health, /metrics, /stats):
# - "8888:8888"
healthcheck:
test: ["CMD", "/pipelock", "healthcheck", "--addr", "127.0.0.1:8888"]
interval: 10s
timeout: 5s
retries: 3
start_period: 30s
networks:
- sure_net
# Note: You still have to download models manually using the ollama CLI or via Open WebUI
ollama:
profiles:
- ai
volumes:
- ollama:/root/.ollama
container_name: ollama
hostname: ollama
restart: unless-stopped
image: docker.io/ollama/ollama:latest
ports:
- "11434:11434"
environment:
- OLLAMA_KEEP_ALIVE=1h
- OLLAMA_MODELS=deepseek-r1:8b,llama3.1:8b # Pre-load model on startup, you can change this to your preferred model
networks:
- sure_net
# Recommended: Enable GPU support
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: all
# capabilities: [ gpu ]
ollama-webui:
profiles:
- ai
image: ghcr.io/open-webui/open-webui
container_name: ollama-webui
volumes:
- ollama-webui:/app/backend/data
depends_on:
- ollama
ports:
- "8080:8080"
environment: # https://docs.openwebui.com/getting-started/env-configuration#default_models
- OLLAMA_BASE_URLS=http://host.docker.internal:11434 #comma separated ollama hosts
- ENV=dev
- WEBUI_AUTH=False
- WEBUI_NAME=AI
- WEBUI_URL=http://localhost:8080
- WEBUI_SECRET_KEY=t0p-s3cr3t
- NO_PROXY=host.docker.internal
extra_hosts:
- host.docker.internal:host-gateway
restart: unless-stopped
networks:
- sure_net
web:
image: ghcr.io/we-promise/sure:stable
volumes:
- app-storage:/rails/storage
ports:
# Web UI for browser access. Note: /mcp is also reachable on this port,
# bypassing Pipelock's MCP scanning (auth token is still required).
# For hardened deployments, use `expose: [3000]` instead and front
# the web UI with a separate reverse proxy.
- ${PORT:-3000}:3000
restart: unless-stopped
environment:
<<: *rails_env
depends_on:
db:
condition: service_healthy
redis:
condition: service_healthy
pipelock: # Remove this block and unset HTTPS_PROXY/HTTP_PROXY to run without Pipelock
condition: service_healthy
dns:
- 8.8.8.8
- 1.1.1.1
networks:
- sure_net
worker:
image: ghcr.io/we-promise/sure:stable
command: bundle exec sidekiq
volumes:
- app-storage:/rails/storage
restart: unless-stopped
depends_on:
db:
condition: service_healthy
redis:
condition: service_healthy
pipelock: # Remove this block and unset HTTPS_PROXY/HTTP_PROXY to run without Pipelock
condition: service_healthy
dns:
- 8.8.8.8
- 1.1.1.1
environment:
<<: *rails_env
networks:
- sure_net
db:
image: postgres:16
restart: unless-stopped
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
<<: *db_env
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB" ]
interval: 5s
timeout: 5s
retries: 5
networks:
- sure_net
backup:
profiles:
- backup
image: prodrigestivill/postgres-backup-local
restart: unless-stopped
volumes:
- /opt/sure-data/backups:/backups # Change this path to your desired backup location on the host machine
environment:
- POSTGRES_HOST=db
- POSTGRES_DB=${POSTGRES_DB:-sure_production}
- POSTGRES_USER=${POSTGRES_USER:-sure_user}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-sure_password}
- SCHEDULE=@daily # Runs once a day at midnight
- BACKUP_KEEP_DAYS=7 # Keeps the last 7 days of backups
- BACKUP_KEEP_WEEKS=4 # Keeps 4 weekly backups
- BACKUP_KEEP_MONTHS=6 # Keeps 6 monthly backups
depends_on:
- db
networks:
- sure_net
redis:
image: redis:latest
restart: unless-stopped
volumes:
- redis-data:/data
healthcheck:
test: [ "CMD", "redis-cli", "ping" ]
interval: 5s
timeout: 5s
retries: 5
networks:
- sure_net
volumes:
app-storage:
postgres-data:
redis-data:
ollama:
ollama-webui:
networks:
sure_net:
driver: bridge