Files
sure/docs/hosting/docker.md
LPW a53a131c46 Add Pipelock operational templates, docs, and config hardening (#1102)
* feat(helm): add Pipelock ConfigMap, scanning config, and consolidate compose

- Add ConfigMap template rendering DLP, response scanning, MCP input/tool
  scanning, and forward proxy settings from values
- Mount ConfigMap as /etc/pipelock/pipelock.yaml volume in deployment
- Add checksum/config annotation for automatic pod restart on config change
- Gate HTTPS_PROXY/HTTP_PROXY env injection on forwardProxy.enabled (skip
  in MCP-only mode)
- Use hasKey for all boolean values to prevent Helm default swallowing false
- Single source of truth for ports (forwardProxy.port/mcpProxy.port)
- Pipelock-specific imagePullSecrets with fallback to app secrets
- Merge standalone compose.example.pipelock.yml into compose.example.ai.yml
- Add pipelock.example.yaml for Docker Compose users
- Add exclude-paths to CI workflow for locale file false positives

* Add external assistant support (OpenAI-compatible SSE proxy)

Allow self-hosted instances to delegate chat to an external AI agent
via an OpenAI-compatible streaming endpoint. Configurable per-family
through Settings UI or ASSISTANT_TYPE env override.

- Assistant::External::Client: SSE streaming HTTP client (no new gems)
- Settings UI with type selector, env lock indicator, config status
- Helm chart and Docker Compose env var support
- 45 tests covering client, config, routing, controller, integration

* Add session key routing, email allowlist, and config plumbing

Route to the actual OpenClaw session via x-openclaw-session-key header
instead of creating isolated sessions. Gate external assistant access
behind an email allowlist (EXTERNAL_ASSISTANT_ALLOWED_EMAILS env var).
Plumb session_key and allowedEmails through Helm chart, compose, and
env template.

* Add HTTPS_PROXY support to External::Client for Pipelock integration

Net::HTTP does not auto-read HTTPS_PROXY/HTTP_PROXY env vars (unlike
Faraday). Explicitly resolve proxy from environment in build_http so
outbound traffic to the external assistant routes through Pipelock's
forward proxy when enabled. Respects NO_PROXY for internal hosts.

* Add UI fields for external assistant config (Setting-backed with env fallback)

Follow the same pattern as OpenAI settings: database-backed Setting
fields with env var defaults. Self-hosters can now configure the
external assistant URL, token, and agent ID from the browser
(Settings > Self-Hosting > AI Assistant) instead of requiring env vars.
Fields disable when the corresponding env var is set.

* Improve external assistant UI labels and add help text

Change placeholder to generic OpenAI-compatible URL pattern. Add help
text under each field explaining where the values come from: URL from
agent provider, token for authentication, agent ID for multi-agent
routing.

* Add external assistant docs and fix URL help text

Add External AI Assistant section to docs/hosting/ai.md covering setup
(UI and env vars), how it works, Pipelock security scanning, access
control, and Docker Compose example. Drop "chat completions" jargon
from URL help text.

* Harden external assistant: retry logic, disconnect UI, error handling, and test coverage

- Add retry with backoff for transient network errors (no retry after streaming starts)
- Add disconnect button with confirmation modal in self-hosting settings
- Narrow rescue scope with fallback logging for unexpected errors
- Safe cleanup of partial responses on stream interruption
- Gate ai_available? on family assistant_type instead of OR-ing all providers
- Truncate conversation history to last 20 messages
- Proxy-aware HTTP client with NO_PROXY support
- Sanitize protocol to use generic headers (X-Agent-Id, X-Session-Key)
- Full test coverage for streaming, retries, proxy routing, config, and disconnect

* Exclude external assistant client from Pipelock scan-diff

False positive: `@token` instance variable flagged as "Credential in URL".
Temporary workaround until Pipelock supports inline suppression.

* Address review feedback: NO_PROXY boundary fix, SSE done flag, design tokens

- Fix NO_PROXY matching to require domain boundary (exact match or .suffix),
  case-insensitive. Prevents badexample.com matching example.com.
- Add done flag to SSE streaming so read_body stops after [DONE]
- Move MAX_CONVERSATION_MESSAGES to class level
- Use bg-success/bg-destructive design tokens for status indicators
- Add rationale comment for pipelock scan exclusion
- Update docs last-updated date

* Address second round of review feedback

- Allowlist email comparison is now case-insensitive and nil-safe
- Cap SSE buffer at 1 MB to prevent memory blowup from malformed streams
- Don't expose upstream HTTP response body in user-facing errors (log it instead)
- Fix frozen string warning on buffer initialization
- Fix "builtin" typo in docs (should be "built-in")

* Protect completed responses from cleanup, sanitize error messages

- Don't destroy a fully streamed assistant message if post-stream
  metadata update fails (only cleanup partial responses)
- Log raw connection/HTTP errors internally, show generic messages
  to users to avoid leaking network/proxy details
- Update test assertions for new error message wording

* Fix SSE content guard and NO_PROXY test correctness

Use nil check instead of present? for SSE delta content to preserve
whitespace-only chunks (newlines, spaces) that can occur in code output.

Fix NO_PROXY test to use HTTP_PROXY matching the http:// client URL so
the proxy resolution and NO_PROXY bypass logic are actually exercised.

* Forward proxy credentials to Net::HTTP

Pass proxy_uri.user and proxy_uri.password to Net::HTTP.new so
authenticated proxies (http://user:pass@host:port) work correctly.
Without this, credentials parsed from the proxy URL were silently
dropped. Nil values are safe as positional args when no creds exist.

* Update pipelock integration to v0.3.1 with full scanning config

Bump Helm image tag from 0.2.7 to 0.3.1. Add missing security
sections to both the Helm ConfigMap and compose example config:
mcp_tool_policy, mcp_session_binding, and tool_chain_detection.
These protect the /mcp endpoint against tool injection, session
hijacking, and multi-step exfiltration chains.

Add version and mode fields to config files. Enable include_defaults
for DLP and response scanning to merge user patterns with the 35
built-in patterns. Remove redundant --mode CLI flag from the Helm
deployment template since mode is now in the config file.

* Pipelock Helm hardening + docs for external assistant and pipelock

Helm templates:
- ServiceMonitor for Prometheus scraping on /metrics (proxy port)
- Ingress template for MCP reverse proxy (external AI agent access)
- PodDisruptionBudget with minAvailable/maxUnavailable mutual exclusion
- topologySpreadConstraints on Deployment
- Structured logging config (format, output, include_allowed/blocked)
- extraConfig escape hatch for additional pipelock.yaml sections
- requireForExternalAssistant guard (fails when assistant enabled without pipelock)
- Component label on Service metadata for ServiceMonitor targeting
- NOTES.txt pipelock section with health, access, security, metrics info
- Bump pipelock image tag 0.3.1 -> 0.3.2
- Fix: rename _asserts.tpl -> asserts.tpl (Helm skipped _ prefixed file)

Documentation:
- Helm chart README: full Pipelock section
- docs/hosting/pipelock.md: dedicated hosting guide (Docker + Kubernetes)
- docs/hosting/docker.md: AI features section (external assistant, pipelock)
- .env.example: external assistant and MCP env vars

Infra:
- Chart.lock pinning dependency versions
- .gitignore for vendored subchart tarballs

* Fix bot comments: quote ingress host, fix sidecar wording, add code block lang

* Fail fast when pipelock ingress enabled with empty hosts

* Fail fast when pipelock ingress host has empty paths

* Messed up the conflict merge

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-03-03 16:32:35 +01:00

10 KiB

Self Hosting Sure with Docker

This guide will help you setup, update, and maintain your self-hosted Sure application with Docker Compose. Docker Compose is the most popular and recommended way to self-host the Sure app.

Setup Guide

Follow the guide below to get your app running.

Step 1: Install Docker

Complete the following steps:

  1. Install Docker Engine by following the official guide
  2. Start the Docker service on your machine
  3. Verify that Docker is installed correctly and is running by opening up a terminal and running the following command:
# If Docker is setup correctly, this command will succeed
docker run hello-world

Step 2: Configure your Docker Compose file and environment

Create a directory for your app to run

Open your terminal and create a directory where your app will run. Below is an example command with a recommended directory:

# Create a directory on your computer for Docker files (name it whatever you like)
mkdir -p ~/docker-apps/sure

# Once created, navigate your current working directory to the new folder
cd ~/docker-apps/sure

Copy our sample Docker Compose file

Make sure you are in the directory you just created and run the following command:

# Download the sample compose.yml file from the GitHub repository
curl -o compose.yml https://raw.githubusercontent.com/we-promise/sure/main/compose.example.yml

This command will do the following:

  1. Fetch the sample docker compose file from our public Github repository
  2. Creates a file in your current directory called compose.yml with the contents of the example file

At this point, the only file in your current working directory should be compose.yml.

Step 3 (optional): Configure your environment

By default, our compose.example.yml file runs without any configuration.
That said, if you would like extra security (important if you're running outside of a local network), you can follow the steps below to set things up.

If you're running the app locally and don't care much about security, you can skip this step.

Create your environment file

In order to configure the app, you will need to create a file called .env, which is where Docker will read environment variables from.

To do this, you should get our .env.example as a starting point:

curl -o .env https://raw.githubusercontent.com/we-promise/sure/main/.env.example

Generate the app secret key

The app requires an environment variable called SECRET_KEY_BASE to run.

We will first need to generate this in the terminal. If you have openssl installed on your computer, you can generate it with the following command:

openssl rand -hex 64

Alternatively, you can generate a key without openssl or any external dependencies by pasting the following bash command in your terminal and running it:

head -c 64 /dev/urandom | od -An -tx1 | tr -d ' \n' && echo

Once you have generated a key, save it and move on to the next step.

Fill in your environment file

Open the file named .env that we created in a prior step using your favorite text editor.

Fill in this file with the following variables:

SECRET_KEY_BASE="replacemewiththegeneratedstringfromthepriorstep"
POSTGRES_PASSWORD="replacemewithyourdesireddatabasepassword"

Using HTTPS

Assuming you want to access your instance from the internet, you should have secured your URL address with an SSL certificate.
The Docker instance runs in plain HTTP and you need to tell it that you are redirecting your HTTPS stream to the HTTP one.
To do this, edit the compose.yml file and find the line stating:

RAILS_ASSUME_SSL: "false"

and change it to true

RAILS_ASSUME_SSL: "true"

Step 4: Run the app

You are now ready to run the app. Start with the following command to make sure everything is working:

docker compose up

This will pull our official Docker image and start the app. You will see logs in your terminal.

Open your browser, and navigate to http://localhost:3000.

If everything is working, you will see the Sure login screen.

Step 5: Create your account

The first time you run the app, you will need to register a new account by hitting "create your account" on the login page.

  1. Enter your email
  2. Enter a password

Step 6: Run the app in the background

Most self-hosting users will want the Sure app to run in the background on their computer so they can access it at all times. To do this, hit Ctrl+C to stop the running process, and then run the following command:

docker compose up -d

The -d flag will run Docker Compose in "detached" mode. To verify it is running, you can run the following command:

docker compose ls

Step 7: Enjoy!

Your app is now set up. You can visit it at http://localhost:3000 in your browser.

If you find bugs or have a feature request, be sure to read through our contributing guide here.

AI features, external assistant, and Pipelock

Sure ships with a separate compose file for AI-related features: compose.example.ai.yml. It adds:

  • Pipelock (always on): AI agent security proxy that scans outbound LLM calls and inbound MCP traffic
  • Ollama + Open WebUI (optional --profile ai): local LLM inference

Using the AI compose file

# Download both compose files
curl -o compose.yml https://raw.githubusercontent.com/we-promise/sure/main/compose.example.yml
curl -o compose.ai.yml https://raw.githubusercontent.com/we-promise/sure/main/compose.example.ai.yml
curl -o pipelock.example.yaml https://raw.githubusercontent.com/we-promise/sure/main/pipelock.example.yaml

# Run with Pipelock (no local LLM)
docker compose -f compose.ai.yml up -d

# Run with Pipelock + Ollama
docker compose -f compose.ai.yml --profile ai up -d

Setting up the external AI assistant

The external assistant delegates chat to a remote AI agent instead of calling LLMs directly. The agent calls back to Sure's /mcp endpoint for financial data (accounts, transactions, balance sheet).

  1. Set the MCP endpoint credentials in your .env:

    MCP_API_TOKEN=generate-a-random-token-here
    MCP_USER_EMAIL=your@email.com   # must match an existing Sure user
    
  2. Set the external assistant connection:

    EXTERNAL_ASSISTANT_URL=https://your-agent/v1/chat/completions
    EXTERNAL_ASSISTANT_TOKEN=your-agent-api-token
    
  3. Choose how to activate:

    • Per-family (UI): Go to Settings > Self-Hosting > AI Assistant, select "External"
    • Global (env): Set ASSISTANT_TYPE=external to force all families to use external

See docs/hosting/ai.md for full configuration details including agent ID, session keys, and email allowlisting.

Pipelock security proxy

Pipelock sits between Sure and external services, scanning AI traffic for:

  • Secret exfiltration (DLP): catches API keys, tokens, or personal data leaking in prompts
  • Prompt injection: detects attempts to override system instructions
  • Tool poisoning: validates MCP tool calls against known-safe patterns

When using compose.example.ai.yml, Pipelock is always running. External AI agents should connect to port 8889 (MCP reverse proxy) instead of directly to Sure's /mcp on port 3000.

For full Pipelock configuration, see docs/hosting/pipelock.md.

How to update your app

The mechanism that updates your self-hosted Sure app is the GHCR (Github Container Registry) Docker image that you see in the compose.yml file:

image: ghcr.io/we-promise/sure:latest

We recommend using one of the following images, but you can pin your app to whatever version you'd like (see packages):

  • ghcr.io/we-promise/sure:latest (latest alpha)
  • ghcr.io/we-promise/sure:stable (latest release)

By default, your app will NOT automatically update. To update your self-hosted app, run the following commands in your terminal:

cd ~/docker-apps/sure # Navigate to whatever directory you configured the app in
docker compose pull # This pulls the "latest" published image from GHCR
docker compose build # This rebuilds the app with updates
docker compose up --no-deps -d web worker # This restarts the app using the newest version

How to change which updates your app receives

If you'd like to pin the app to a specific version or tag, all you need to do is edit the compose.yml file:

image: ghcr.io/we-promise/sure:stable

After doing this, make sure and restart the app:

docker compose pull # This pulls the "latest" published image from GHCR
docker compose build # This rebuilds the app with updates
docker compose up --no-deps -d web worker # This restarts the app using the newest version

Troubleshooting

ActiveRecord::DatabaseConnectionError

If you are trying to get Sure started for the first time and run into database connection issues, it is likely because Docker has already initialized the Postgres database with a different default role (usually from a previous attempt to start the app).

If you run into this issue, you can optionally reset the database.

PLEASE NOTE: this will delete any existing data that you have in your Sure database, so proceed with caution. For first-time users of the app just trying to get started, you're generally safe to run the commands below.

By running the commands below, you will delete your existing Sure database and "reset" it.

docker compose down
docker volume rm sure_postgres-data # this is the name of the volume the DB is mounted to
docker compose up
docker compose exec db psql -U sure_user -d sure_development -c "SELECT 1;" # This will verify that the issue is fixed

Slow .csv import (processing rows taking longer than expected)

Importing comma-separated-value file(s) requires the sure-worker container to communicate with Redis. Check your worker logs for any unexpected errors, such as connection timeouts or Redis communication failures.