Documentation for review AI Assistant features, MCP and API additions (#1168)

* Create MCP server endpoint documentation * Add Assistant Architecture section to AI documentation * Add Users API documentation for account reset and delete endpoints * Document Pipelock CI security scanning in contributing guide * fix: correct scope and error codes in Users API documentation * Exclude `docs/hosting/ai.md` from Pipelock scan --------- Co-authored-by: askmanu[bot] <192355599+askmanu[bot]@users.noreply.github.com> Co-authored-by: Juan José Mata <jjmata@jjmata.com>
2026-04-19 03:54:08 +00:00 · 2026-03-16 17:24:28 +00:00
parent 5a43f123c2
commit a0b1029ba9
5 changed files with 697 additions and 0 deletions
--- a/docs/hosting/ai.md
+++ b/docs/hosting/ai.md
@@ -511,6 +511,238 @@ x-rails-env: &rails_env

 Or configure the assistant via the Settings UI after startup (MCP env vars are still required for callback).

+## Assistant Architecture
+
+Sure's AI assistant system uses a modular architecture that allows different assistant implementations to be plugged in based on configuration. This section explains the architecture for contributors who want to understand or extend the system.
+
+### Overview
+
+The assistant system evolved from a monolithic class to a module-based architecture with a registry pattern. This allows Sure to support multiple assistant types (builtin, external) and makes it easy to add new implementations.
+
+**Key benefits:**
+- **Extensible:** Add new assistant types without modifying existing code
+- **Configurable:** Choose assistant type per family or globally
+- **Isolated:** Each implementation has its own logic and dependencies
+- **Testable:** Implementations are independent and can be tested separately
+
+### Component Hierarchy
+
+#### `Assistant` Module
+
+The main entry point for all assistant operations. Located in `app/models/assistant.rb`.
+
+**Key methods:**
+
+| Method | Description |
+|--------|-------------|
+| `.for_chat(chat)` | Returns the appropriate assistant instance for a chat |
+| `.config_for(chat)` | Returns configuration for builtin assistants |
+| `.available_types` | Lists all registered assistant types |
+| `.function_classes` | Returns all available function/tool classes |
+
+**Example usage:**
+
+```ruby
+# Get an assistant for a chat
+assistant = Assistant.for_chat(chat)
+
+# Respond to a message
+assistant.respond_to(message)
+```
+
+#### `Assistant::Base`
+
+Abstract base class that all assistant implementations inherit from. Located in `app/models/assistant/base.rb`.
+
+**Contract:**
+- Must implement `respond_to(message)` instance method
+- Includes `Assistant::Broadcastable` for real-time updates
+- Receives the `chat` object in the initializer
+
+**Example implementation:**
+
+```ruby
+class Assistant::MyCustom < Assistant::Base
+  def respond_to(message)
+    # Your custom logic here
+    assistant_message = AssistantMessage.new(chat: chat, content: "Response")
+    assistant_message.save!
+  end
+end
+```
+
+#### `Assistant::Builtin`
+
+The default implementation that uses the configured OpenAI-compatible LLM provider. Located in `app/models/assistant/builtin.rb`.
+
+**Features:**
+- Uses `Assistant::Provided` for LLM provider selection
+- Uses `Assistant::Configurable` for system prompts and function configuration
+- Supports function calling via `Assistant::FunctionToolCaller`
+- Streams responses in real-time
+
+**Key methods:**
+
+| Method | Description |
+|--------|-------------|
+| `.for_chat(chat)` | Creates a new builtin assistant with config |
+| `#respond_to(message)` | Processes a message using the LLM |
+
+#### `Assistant::External`
+
+Implementation for delegating chat to a remote AI agent. Located in `app/models/assistant/external.rb`.
+
+**Features:**
+- Sends conversation to external agent via OpenAI-compatible API
+- Agent calls back to Sure's `/mcp` endpoint for financial data
+- Supports access control via email allowlist
+- Streams responses from the agent
+
+**Configuration:**
+
+```ruby
+config = Assistant::External.config
+# => #<struct url="...", token="...", agent_id="...", session_key="...">
+```
+
+### Registry Pattern
+
+The `Assistant` module uses a registry to map type names to implementation classes:
+
+```ruby
+REGISTRY = {
+  "builtin" => Assistant::Builtin,
+  "external" => Assistant::External
+}.freeze
+```
+
+**Type selection logic:**
+
+1. Check `ENV["ASSISTANT_TYPE"]` (global override)
+2. Check `chat.user.family.assistant_type` (per-family setting)
+3. Default to `"builtin"`
+
+**Example:**
+
+```ruby
+# Global override
+ENV["ASSISTANT_TYPE"] = "external"
+Assistant.for_chat(chat) # => Assistant::External instance
+
+# Per-family setting
+family.update(assistant_type: "external")
+Assistant.for_chat(chat) # => Assistant::External instance
+
+# Default
+Assistant.for_chat(chat) # => Assistant::Builtin instance
+```
+
+### Function Registry
+
+The `Assistant.function_classes` method centralizes all available financial tools:
+
+```ruby
+def self.function_classes
+  [
+    Function::GetTransactions,
+    Function::GetAccounts,
+    Function::GetHoldings,
+    Function::GetBalanceSheet,
+    Function::GetIncomeStatement,
+    Function::ImportBankStatement,
+    Function::SearchFamilyFiles
+  ]
+end
+```
+
+These functions are:
+- Used by builtin assistants for LLM function calling
+- Exposed via the MCP endpoint for external agents
+- Defined in `app/models/assistant/function/`
+
+### Adding a New Assistant Type
+
+To add a custom assistant implementation:
+
+#### 1. Create the implementation class
+
+```ruby
+# app/models/assistant/my_custom.rb
+class Assistant::MyCustom < Assistant::Base
+  class << self
+    def for_chat(chat)
+      new(chat)
+    end
+  end
+
+  def respond_to(message)
+    # Your implementation here
+    # Must create and save an AssistantMessage
+    assistant_message = AssistantMessage.new(
+      chat: chat,
+      content: "My custom response"
+    )
+    assistant_message.save!
+  end
+end
+```
+
+#### 2. Register the implementation
+
+```ruby
+# app/models/assistant.rb
+REGISTRY = {
+  "builtin" => Assistant::Builtin,
+  "external" => Assistant::External,
+  "my_custom" => Assistant::MyCustom
+}.freeze
+```
+
+#### 3. Add validation
+
+```ruby
+# app/models/family.rb
+ASSISTANT_TYPES = %w[builtin external my_custom].freeze
+```
+
+#### 4. Use the new type
+
+```bash
+# Global override
+ASSISTANT_TYPE=my_custom
+
+# Or set per-family in the database
+family.update(assistant_type: "my_custom")
+```
+
+### Integration Points
+
+#### Pipelock Integration
+
+For external assistants, Pipelock can scan traffic:
+- **Outbound:** Sure -> agent (via `HTTPS_PROXY`)
+- **Inbound:** Agent -> Sure /mcp (via MCP reverse proxy on port 8889)
+
+See the [External AI Assistant](#external-ai-assistant) and [Pipelock](pipelock.md) documentation for configuration.
+
+#### OpenClaw/WebSocket Support
+
+The `Assistant::External` implementation currently uses HTTP streaming. Future implementations could use WebSocket connections via OpenClaw or other gateways.
+
+**Example future implementation:**
+
+```ruby
+class Assistant::WebSocket < Assistant::Base
+  def respond_to(message)
+    # Connect via WebSocket
+    # Stream bidirectional communication
+    # Handle tool calls via MCP
+  end
+end
+```
+
+Register it in the `REGISTRY` and add to `Family::ASSISTANT_TYPES` to activate.
+
 ## AI Cache Management

 Sure caches AI-generated results (like auto-categorization and merchant detection) to avoid redundant API calls and costs. However, there are situations where you may want to clear this cache.
--- a/docs/hosting/mcp.md
+++ b/docs/hosting/mcp.md
@@ -0,0 +1,338 @@
+# MCP Server for External AI Assistants
+
+Sure includes a Model Context Protocol (MCP) server endpoint that allows external AI assistants like Claude Desktop, GPT agents, or custom AI clients to query your financial data.
+
+## What is MCP?
+
+[Model Context Protocol](https://modelcontextprotocol.io/) is a JSON-RPC 2.0 protocol that enables AI assistants to access structured data and tools from external applications. Instead of copying and pasting financial data into a chat window, your AI assistant can directly query Sure's data through a secure API.
+
+This is useful when:
+- You want to use an external AI assistant (Claude, GPT, custom agents) to analyze your Sure financial data
+- You prefer to keep your LLM provider separate from Sure
+- You're building custom AI agents that need access to financial tools
+
+## Prerequisites
+
+To enable the MCP endpoint, you need to set two environment variables:
+
+| Variable | Description | Example |
+|----------|-------------|---------|
+| `MCP_API_TOKEN` | Bearer token for authentication | `your-secret-token-here` |
+| `MCP_USER_EMAIL` | Email of the Sure user whose data the assistant can access | `user@example.com` |
+
+Both variables are **required**. The endpoint will not activate if either is missing.
+
+### Generating a secure token
+
+Generate a random token for `MCP_API_TOKEN`:
+
+```bash
+# macOS/Linux
+openssl rand -base64 32
+
+# Or use any secure password generator
+```
+
+### Choosing the user
+
+The `MCP_USER_EMAIL` must match an existing Sure user's email address. The AI assistant will have access to all financial data for that user's family.
+
+> [!CAUTION]
+> The AI assistant will have **read access to all financial data** for the specified user. Only set this for users you trust with your AI provider.
+
+## Configuration
+
+### Docker Compose
+
+Add the environment variables to your `compose.yml`:
+
+```yaml
+x-rails-env: &rails_env
+  MCP_API_TOKEN: your-secret-token-here
+  MCP_USER_EMAIL: user@example.com
+```
+
+Both `web` and `worker` services inherit this configuration.
+
+### Kubernetes (Helm)
+
+Add the variables to your `values.yaml` or set them via Secrets:
+
+```yaml
+env:
+  MCP_API_TOKEN: your-secret-token-here
+  MCP_USER_EMAIL: user@example.com
+```
+
+Or create a Secret and reference it:
+
+```yaml
+envFrom:
+  - secretRef:
+      name: sure-mcp-credentials
+```
+
+## Protocol Details
+
+The MCP endpoint is available at:
+
+```
+POST /mcp
+```
+
+### Authentication
+
+All requests must include the `MCP_API_TOKEN` as a Bearer token:
+
+```
+Authorization: Bearer <MCP_API_TOKEN>
+```
+
+### Supported Methods
+
+Sure implements the following JSON-RPC 2.0 methods:
+
+| Method | Description |
+|--------|-------------|
+| `initialize` | Protocol handshake, returns server info and capabilities |
+| `tools/list` | Lists available financial tools with schemas |
+| `tools/call` | Executes a tool with provided arguments |
+
+### Available Tools
+
+The MCP endpoint exposes these financial tools:
+
+| Tool | Description |
+|------|-------------|
+| `get_transactions` | Retrieve transaction history with filtering |
+| `get_accounts` | Get account information and balances |
+| `get_holdings` | Query investment holdings |
+| `get_balance_sheet` | Current financial position (assets, liabilities, net worth) |
+| `get_income_statement` | Income and expenses over a period |
+| `import_bank_statement` | Import bank statement data |
+| `search_family_files` | Search uploaded documents in the vault |
+
+These are the same tools used by Sure's builtin AI assistant.
+
+## Example Requests
+
+### Initialize
+
+Handshake to verify protocol version and capabilities:
+
+```bash
+curl -X POST https://your-sure-instance/mcp \
+  -H "Authorization: Bearer your-secret-token" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "jsonrpc": "2.0",
+    "id": 1,
+    "method": "initialize"
+  }'
+```
+
+Response:
+
+```json
+{
+  "jsonrpc": "2.0",
+  "id": 1,
+  "result": {
+    "protocolVersion": "2025-03-26",
+    "capabilities": {
+      "tools": {}
+    },
+    "serverInfo": {
+      "name": "sure",
+      "version": "1.0"
+    }
+  }
+}
+```
+
+### List Tools
+
+Get available tools with their schemas:
+
+```bash
+curl -X POST https://your-sure-instance/mcp \
+  -H "Authorization: Bearer your-secret-token" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "jsonrpc": "2.0",
+    "id": 2,
+    "method": "tools/list"
+  }'
+```
+
+Response includes tool names, descriptions, and JSON schemas for parameters.
+
+### Call a Tool
+
+Execute a tool to get transactions:
+
+```bash
+curl -X POST https://your-sure-instance/mcp \
+  -H "Authorization: Bearer your-secret-token" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "jsonrpc": "2.0",
+    "id": 3,
+    "method": "tools/call",
+    "params": {
+      "name": "get_transactions",
+      "arguments": {
+        "start_date": "2024-01-01",
+        "end_date": "2024-01-31"
+      }
+    }
+  }'
+```
+
+Response:
+
+```json
+{
+  "jsonrpc": "2.0",
+  "id": 3,
+  "result": {
+    "content": [
+      {
+        "type": "text",
+        "text": "[{\"id\":\"...\",\"amount\":-45.99,\"date\":\"2024-01-15\",\"name\":\"Coffee Shop\"}]"
+      }
+    ]
+  }
+}
+```
+
+## Security Considerations
+
+### Transient Session Isolation
+
+The MCP controller creates a **transient session** for each request. This prevents session state leaks that could expose other users' data if the Sure instance is using impersonation features.
+
+Each MCP request:
+1. Authenticates the token
+2. Loads the user specified in `MCP_USER_EMAIL`
+3. Creates a temporary session scoped to that user
+4. Executes the tool call
+5. Discards the session
+
+This ensures the AI assistant can only access data for the intended user.
+
+### Pipelock Security Scanning
+
+For production deployments, we recommend using [Pipelock](https://github.com/luckyPipewrench/pipelock) to scan MCP traffic for security threats.
+
+Pipelock provides:
+- **DLP scanning**: Detects secrets being exfiltrated through tool calls
+- **Prompt injection detection**: Identifies attempts to manipulate the AI
+- **Tool poisoning detection**: Prevents malicious tool call sequences
+- **Policy enforcement**: Block or warn on suspicious patterns
+
+See the [Pipelock documentation](pipelock.md) and the example configuration in `compose.example.pipelock.yml` for setup instructions.
+
+### Network Security
+
+The `/mcp` endpoint is exposed on the same port as the web UI (default 3000). For hardened deployments:
+
+**Docker Compose:**
+- The MCP endpoint is protected by the `MCP_API_TOKEN` but is reachable on port 3000
+- For additional security, use Pipelock's MCP reverse proxy (port 8889) which adds scanning
+- See `compose.example.ai.yml` for a Pipelock configuration
+
+**Kubernetes:**
+- Use NetworkPolicies to restrict access to the MCP endpoint
+- Route external agents through Pipelock's MCP reverse proxy
+- See the [Helm chart documentation](../../charts/sure/README.md) for Pipelock ingress setup
+
+## Production Deployment
+
+For a production-ready setup with security scanning:
+
+1. **Download the example configuration:**
+
+   ```bash
+   curl -o compose.ai.yml https://raw.githubusercontent.com/we-promise/sure/main/compose.example.ai.yml
+   curl -o pipelock.example.yaml https://raw.githubusercontent.com/we-promise/sure/main/pipelock.example.yaml
+   ```
+
+2. **Set your MCP credentials in `.env`:**
+
+   ```bash
+   MCP_API_TOKEN=your-secret-token
+   MCP_USER_EMAIL=user@example.com
+   ```
+
+3. **Start the stack:**
+
+   ```bash
+   docker compose -f compose.ai.yml up -d
+   ```
+
+4. **Connect your AI assistant to the Pipelock MCP proxy:**
+
+   ```
+   http://your-server:8889
+   ```
+
+The Pipelock proxy (port 8889) scans all MCP traffic before forwarding to Sure's `/mcp` endpoint.
+
+## Connecting AI Assistants
+
+### Claude Desktop
+
+Configure Claude Desktop to use Sure's MCP server:
+
+1. Open Claude Desktop settings
+2. Add a new MCP server
+3. Set the endpoint to `http://your-server:8889` (if using Pipelock) or `http://your-server:3000/mcp`
+4. Add the authorization header: `Authorization: Bearer your-secret-token`
+
+### Custom Agents
+
+Any AI agent that supports JSON-RPC 2.0 can connect to the MCP endpoint. The agent should:
+
+1. Send a POST request to `/mcp`
+2. Include the `Authorization: Bearer <token>` header
+3. Use the JSON-RPC 2.0 format for requests
+4. Handle the protocol methods: `initialize`, `tools/list`, `tools/call`
+
+## Troubleshooting
+
+### "MCP endpoint not configured" error
+
+**Symptom:** Requests return HTTP 503 with "MCP endpoint not configured"
+
+**Fix:** Ensure both `MCP_API_TOKEN` and `MCP_USER_EMAIL` are set as environment variables and restart Sure.
+
+### "unauthorized" error
+
+**Symptom:** Requests return HTTP 401 with "unauthorized"
+
+**Fix:** Verify the `Authorization` header contains the correct token: `Bearer <MCP_API_TOKEN>`
+
+### "MCP user not configured" error
+
+**Symptom:** Requests return HTTP 503 with "MCP user not configured"
+
+**Fix:** The `MCP_USER_EMAIL` does not match an existing user. Check that:
+- The email is correct
+- The user exists in the database
+- There are no typos or extra spaces
+
+### Pipelock connection refused
+
+**Symptom:** AI assistant cannot connect to Pipelock's MCP proxy (port 8889)
+
+**Fix:**
+1. Verify Pipelock is running: `docker compose ps pipelock`
+2. Check Pipelock health: `docker compose exec pipelock /pipelock healthcheck --addr 127.0.0.1:8888`
+3. Verify the port is exposed in your `compose.yml`
+
+## See Also
+
+- [External AI Assistant Configuration](ai.md#external-ai-assistant) - Configure Sure's chat to use an external agent
+- [Pipelock Security Proxy](pipelock.md) - Set up security scanning for MCP traffic
+- [Model Context Protocol Specification](https://modelcontextprotocol.io/) - Official MCP documentation