QT/sure
1
0
Fork 0
mirror of https://github.com/we-promise/sure.git synced 2026-04-18 11:34:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Documentation for review AI Assistant features, MCP and API additions (#1168)

Browse Source 
* Create MCP server endpoint documentation

* Add Assistant Architecture section to AI documentation

* Add Users API documentation for account reset and delete endpoints

* Document Pipelock CI security scanning in contributing guide

* fix: correct scope and error codes in Users API documentation

* Exclude `docs/hosting/ai.md` from Pipelock scan

---------

Co-authored-by: askmanu[bot] <192355599+askmanu[bot]@users.noreply.github.com>
Co-authored-by: Juan José Mata <jjmata@jjmata.com>
This commit is contained in:
Andrei Onel
2026-03-16 17:24:28 +00:00
committed by GitHub
parent 5a43f123c2
commit a0b1029ba9
5 changed files with 697 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.github/workflows/pipelock.yml vendored
View File

@@ -24,3 +24,4 @@ jobs:
test-vectors: 'false'
exclude-paths: |
config/locales/views/reports/
docs/hosting/ai.md

9
CONTRIBUTING.md
View File

@@ -40,3 +40,12 @@ To get setup for local development, you have two options:
7. Before requesting a review, please make sure that all [Github Checks](https://docs.github.com/en/rest/checks?apiVersion=2022-11-28) have passed and your branch is up-to-date with the `main` branch. After doing so, request a review and wait for a maintainer's approval.
All PRs should target the `main` branch.
### Automated Security Scanning
Every pull request to the `main` branch automatically runs a Pipelock security scan. This scan analyzes your PR diff for:
- Leaked secrets (API keys, tokens, credentials)
- Agent security risks (misconfigurations, exposed credentials, missing controls)
The scan runs as part of the CI pipeline and typically completes in ~30 seconds. If security issues are found, the CI check will fail. You don't need to configure anything—the security scanning is automatic and zero-configuration.

117
docs/api/users.md Normal file
View File

@@ -0,0 +1,117 @@
# Users API Documentation
The Users API allows external applications to manage user account data within Sure. The OpenAPI description is generated directly from executable request specs, ensuring it always reflects the behaviour of the running Rails application.
## Generated OpenAPI specification
- The source of truth for the documentation lives in [`spec/requests/api/v1/users_spec.rb`](../../spec/requests/api/v1/users_spec.rb). These specs authenticate against the Rails stack, exercise every user endpoint, and capture real response shapes.
- Regenerate the OpenAPI document with:
```sh
RAILS_ENV=test bundle exec rake rswag:specs:swaggerize
```
The task compiles the request specs and writes the result to [`docs/api/openapi.yaml`](openapi.yaml).
- Run just the documentation specs with:
```sh
bundle exec rspec spec/requests/api/v1/users_spec.rb
```
## Authentication requirements
All user endpoints require an OAuth2 access token or API key that grants the `read_write` scope.
## Available endpoints
| Endpoint | Scope | Description |
| --- | --- | --- |
| `DELETE /api/v1/users/reset` | `read_write` | Reset account data while preserving the user account. |
| `DELETE /api/v1/users/me` | `read_write` | Permanently delete the user account. |
Refer to the generated [`openapi.yaml`](openapi.yaml) for request/response schemas, reusable components (errors), and security definitions.
## Reset account
`DELETE /api/v1/users/reset`
Resets all financial data (accounts, categories, merchants, tags, transactions, etc.) for the current user's family while keeping the user account intact. The reset runs asynchronously in the background.
### Request
No request body required.
### Response
```json
{
"message": "Account reset has been initiated"
}
```
### Use cases
- Clear all financial data to start fresh
- Remove test data after initial setup
- Reset to a clean state for new imports
## Delete account
`DELETE /api/v1/users/me`
Permanently deactivates the current user account and all associated data. This action cannot be undone.
### Request
No request body required.
### Response
```json
{
"message": "Account has been deleted"
}
```
### Error responses
In addition to standard error codes (`unauthorized`, `insufficient_scope`), the delete endpoint may return:
**422 Unprocessable Entity**
```json
{
"error": "Failed to delete account",
"details": ["Cannot deactivate admin with other users"]
}
```
This occurs when the user cannot be deactivated (for example, an admin user with other active users in the family).
## Security considerations
- Both endpoints require the `read_write` scope. Read-only API keys cannot access these endpoints.
- Deactivated users cannot access these endpoints.
- The reset operation preserves the user account, allowing you to continue using Sure with a clean slate.
- The delete operation is permanent and removes the user account entirely.
## Error responses
Errors conform to the shared `ErrorResponse` schema in the OpenAPI document:
```json
{
"error": "error_code",
"message": "Human readable error message",
"details": ["Optional array of extra context"]
}
```
Common error codes include:
| Code | Description |
| --- | --- |
| `unauthorized` | Missing or invalid API key |
| `insufficient_scope` | API key lacks required `read_write` scope |
| `Failed to delete account` | Account deletion failed (see details field) |

232
docs/hosting/ai.md
View File

@@ -511,6 +511,238 @@ x-rails-env: &rails_env
Or configure the assistant via the Settings UI after startup (MCP env vars are still required for callback).
## Assistant Architecture
Sure's AI assistant system uses a modular architecture that allows different assistant implementations to be plugged in based on configuration. This section explains the architecture for contributors who want to understand or extend the system.
### Overview
The assistant system evolved from a monolithic class to a module-based architecture with a registry pattern. This allows Sure to support multiple assistant types (builtin, external) and makes it easy to add new implementations.
**Key benefits:**
- **Extensible:** Add new assistant types without modifying existing code
- **Configurable:** Choose assistant type per family or globally
- **Isolated:** Each implementation has its own logic and dependencies
- **Testable:** Implementations are independent and can be tested separately
### Component Hierarchy
#### `Assistant` Module
The main entry point for all assistant operations. Located in `app/models/assistant.rb`.
**Key methods:**
| Method | Description |
|--------|-------------|
| `.for_chat(chat)` | Returns the appropriate assistant instance for a chat |
| `.config_for(chat)` | Returns configuration for builtin assistants |
| `.available_types` | Lists all registered assistant types |
| `.function_classes` | Returns all available function/tool classes |
**Example usage:**
```ruby
# Get an assistant for a chat
assistant = Assistant.for_chat(chat)
# Respond to a message
assistant.respond_to(message)
```
#### `Assistant::Base`
Abstract base class that all assistant implementations inherit from. Located in `app/models/assistant/base.rb`.
**Contract:**
- Must implement `respond_to(message)` instance method
- Includes `Assistant::Broadcastable` for real-time updates
- Receives the `chat` object in the initializer
**Example implementation:**
```ruby
class Assistant::MyCustom < Assistant::Base
def respond_to(message)
# Your custom logic here
assistant_message = AssistantMessage.new(chat: chat, content: "Response")
assistant_message.save!
end
end
```
#### `Assistant::Builtin`
The default implementation that uses the configured OpenAI-compatible LLM provider. Located in `app/models/assistant/builtin.rb`.
**Features:**
- Uses `Assistant::Provided` for LLM provider selection
- Uses `Assistant::Configurable` for system prompts and function configuration
- Supports function calling via `Assistant::FunctionToolCaller`
- Streams responses in real-time
**Key methods:**
| Method | Description |
|--------|-------------|
| `.for_chat(chat)` | Creates a new builtin assistant with config |
| `#respond_to(message)` | Processes a message using the LLM |
#### `Assistant::External`
Implementation for delegating chat to a remote AI agent. Located in `app/models/assistant/external.rb`.
**Features:**
- Sends conversation to external agent via OpenAI-compatible API
- Agent calls back to Sure's `/mcp` endpoint for financial data
- Supports access control via email allowlist
- Streams responses from the agent
**Configuration:**
```ruby
config = Assistant::External.config
# => #<struct url="...", token="...", agent_id="...", session_key="...">
```
### Registry Pattern
The `Assistant` module uses a registry to map type names to implementation classes:
```ruby
REGISTRY = {
"builtin" => Assistant::Builtin,
"external" => Assistant::External
}.freeze
```
**Type selection logic:**
1. Check `ENV["ASSISTANT_TYPE"]` (global override)
2. Check `chat.user.family.assistant_type` (per-family setting)
3. Default to `"builtin"`
**Example:**
```ruby
# Global override
ENV["ASSISTANT_TYPE"] = "external"
Assistant.for_chat(chat) # => Assistant::External instance
# Per-family setting
family.update(assistant_type: "external")
Assistant.for_chat(chat) # => Assistant::External instance
# Default
Assistant.for_chat(chat) # => Assistant::Builtin instance
```
### Function Registry
The `Assistant.function_classes` method centralizes all available financial tools:
```ruby
def self.function_classes
[
Function::GetTransactions,
Function::GetAccounts,
Function::GetHoldings,
Function::GetBalanceSheet,
Function::GetIncomeStatement,
Function::ImportBankStatement,
Function::SearchFamilyFiles
]
end
```
These functions are:
- Used by builtin assistants for LLM function calling
- Exposed via the MCP endpoint for external agents
- Defined in `app/models/assistant/function/`
### Adding a New Assistant Type
To add a custom assistant implementation:
#### 1. Create the implementation class
```ruby
# app/models/assistant/my_custom.rb
class Assistant::MyCustom < Assistant::Base
class << self
def for_chat(chat)
new(chat)
end
end
def respond_to(message)
# Your implementation here
# Must create and save an AssistantMessage
assistant_message = AssistantMessage.new(
chat: chat,
content: "My custom response"
)
assistant_message.save!
end
end
```
#### 2. Register the implementation
```ruby
# app/models/assistant.rb
REGISTRY = {
"builtin" => Assistant::Builtin,
"external" => Assistant::External,
"my_custom" => Assistant::MyCustom
}.freeze
```
#### 3. Add validation
```ruby
# app/models/family.rb
ASSISTANT_TYPES = %w[builtin external my_custom].freeze
```
#### 4. Use the new type
```bash
# Global override
ASSISTANT_TYPE=my_custom
# Or set per-family in the database
family.update(assistant_type: "my_custom")
```
### Integration Points
#### Pipelock Integration
For external assistants, Pipelock can scan traffic:
- **Outbound:** Sure -> agent (via `HTTPS_PROXY`)
- **Inbound:** Agent -> Sure /mcp (via MCP reverse proxy on port 8889)
See the [External AI Assistant](#external-ai-assistant) and [Pipelock](pipelock.md) documentation for configuration.
#### OpenClaw/WebSocket Support
The `Assistant::External` implementation currently uses HTTP streaming. Future implementations could use WebSocket connections via OpenClaw or other gateways.
**Example future implementation:**
```ruby
class Assistant::WebSocket < Assistant::Base
def respond_to(message)
# Connect via WebSocket
# Stream bidirectional communication
# Handle tool calls via MCP
end
end
```
Register it in the `REGISTRY` and add to `Family::ASSISTANT_TYPES` to activate.
## AI Cache Management
Sure caches AI-generated results (like auto-categorization and merchant detection) to avoid redundant API calls and costs. However, there are situations where you may want to clear this cache.

338
docs/hosting/mcp.md Normal file
View File

@@ -0,0 +1,338 @@
# MCP Server for External AI Assistants
Sure includes a Model Context Protocol (MCP) server endpoint that allows external AI assistants like Claude Desktop, GPT agents, or custom AI clients to query your financial data.
## What is MCP?
[Model Context Protocol](https://modelcontextprotocol.io/) is a JSON-RPC 2.0 protocol that enables AI assistants to access structured data and tools from external applications. Instead of copying and pasting financial data into a chat window, your AI assistant can directly query Sure's data through a secure API.
This is useful when:
- You want to use an external AI assistant (Claude, GPT, custom agents) to analyze your Sure financial data
- You prefer to keep your LLM provider separate from Sure
- You're building custom AI agents that need access to financial tools
## Prerequisites
To enable the MCP endpoint, you need to set two environment variables:
| Variable | Description | Example |
|----------|-------------|---------|
| `MCP_API_TOKEN` | Bearer token for authentication | `your-secret-token-here` |
| `MCP_USER_EMAIL` | Email of the Sure user whose data the assistant can access | `user@example.com` |
Both variables are **required**. The endpoint will not activate if either is missing.
### Generating a secure token
Generate a random token for `MCP_API_TOKEN`:
```bash
# macOS/Linux
openssl rand -base64 32
# Or use any secure password generator
```
### Choosing the user
The `MCP_USER_EMAIL` must match an existing Sure user's email address. The AI assistant will have access to all financial data for that user's family.
> [!CAUTION]
> The AI assistant will have **read access to all financial data** for the specified user. Only set this for users you trust with your AI provider.
## Configuration
### Docker Compose
Add the environment variables to your `compose.yml`:
```yaml
x-rails-env: &rails_env
MCP_API_TOKEN: your-secret-token-here
MCP_USER_EMAIL: user@example.com
```
Both `web` and `worker` services inherit this configuration.
### Kubernetes (Helm)
Add the variables to your `values.yaml` or set them via Secrets:
```yaml
env:
MCP_API_TOKEN: your-secret-token-here
MCP_USER_EMAIL: user@example.com
```
Or create a Secret and reference it:
```yaml
envFrom:
- secretRef:
name: sure-mcp-credentials
```
## Protocol Details
The MCP endpoint is available at:
```
POST /mcp
```
### Authentication
All requests must include the `MCP_API_TOKEN` as a Bearer token:
```
Authorization: Bearer <MCP_API_TOKEN>
```
### Supported Methods
Sure implements the following JSON-RPC 2.0 methods:
| Method | Description |
|--------|-------------|
| `initialize` | Protocol handshake, returns server info and capabilities |
| `tools/list` | Lists available financial tools with schemas |
| `tools/call` | Executes a tool with provided arguments |
### Available Tools
The MCP endpoint exposes these financial tools:
| Tool | Description |
|------|-------------|
| `get_transactions` | Retrieve transaction history with filtering |
| `get_accounts` | Get account information and balances |
| `get_holdings` | Query investment holdings |
| `get_balance_sheet` | Current financial position (assets, liabilities, net worth) |
| `get_income_statement` | Income and expenses over a period |
| `import_bank_statement` | Import bank statement data |
| `search_family_files` | Search uploaded documents in the vault |
These are the same tools used by Sure's builtin AI assistant.
## Example Requests
### Initialize
Handshake to verify protocol version and capabilities:
```bash
curl -X POST https://your-sure-instance/mcp \
-H "Authorization: Bearer your-secret-token" \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 1,
"method": "initialize"
}'
```
Response:
```json
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"protocolVersion": "2025-03-26",
"capabilities": {
"tools": {}
},
"serverInfo": {
"name": "sure",
"version": "1.0"
}
}
}
```
### List Tools
Get available tools with their schemas:
```bash
curl -X POST https://your-sure-instance/mcp \
-H "Authorization: Bearer your-secret-token" \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 2,
"method": "tools/list"
}'
```
Response includes tool names, descriptions, and JSON schemas for parameters.
### Call a Tool
Execute a tool to get transactions:
```bash
curl -X POST https://your-sure-instance/mcp \
-H "Authorization: Bearer your-secret-token" \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"id": 3,
"method": "tools/call",
"params": {
"name": "get_transactions",
"arguments": {
"start_date": "2024-01-01",
"end_date": "2024-01-31"
}
}
}'
```
Response:
```json
{
"jsonrpc": "2.0",
"id": 3,
"result": {
"content": [
{
"type": "text",
"text": "[{\"id\":\"...\",\"amount\":-45.99,\"date\":\"2024-01-15\",\"name\":\"Coffee Shop\"}]"
}
]
}
}
```
## Security Considerations
### Transient Session Isolation
The MCP controller creates a **transient session** for each request. This prevents session state leaks that could expose other users' data if the Sure instance is using impersonation features.
Each MCP request:
1. Authenticates the token
2. Loads the user specified in `MCP_USER_EMAIL`
3. Creates a temporary session scoped to that user
4. Executes the tool call
5. Discards the session
This ensures the AI assistant can only access data for the intended user.
### Pipelock Security Scanning
For production deployments, we recommend using [Pipelock](https://github.com/luckyPipewrench/pipelock) to scan MCP traffic for security threats.
Pipelock provides:
- **DLP scanning**: Detects secrets being exfiltrated through tool calls
- **Prompt injection detection**: Identifies attempts to manipulate the AI
- **Tool poisoning detection**: Prevents malicious tool call sequences
- **Policy enforcement**: Block or warn on suspicious patterns
See the [Pipelock documentation](pipelock.md) and the example configuration in `compose.example.pipelock.yml` for setup instructions.
### Network Security
The `/mcp` endpoint is exposed on the same port as the web UI (default 3000). For hardened deployments:
**Docker Compose:**
- The MCP endpoint is protected by the `MCP_API_TOKEN` but is reachable on port 3000
- For additional security, use Pipelock's MCP reverse proxy (port 8889) which adds scanning
- See `compose.example.ai.yml` for a Pipelock configuration
**Kubernetes:**
- Use NetworkPolicies to restrict access to the MCP endpoint
- Route external agents through Pipelock's MCP reverse proxy
- See the [Helm chart documentation](../../charts/sure/README.md) for Pipelock ingress setup
## Production Deployment
For a production-ready setup with security scanning:
1. **Download the example configuration:**
```bash
curl -o compose.ai.yml https://raw.githubusercontent.com/we-promise/sure/main/compose.example.ai.yml
curl -o pipelock.example.yaml https://raw.githubusercontent.com/we-promise/sure/main/pipelock.example.yaml
```
2. **Set your MCP credentials in `.env`:**
```bash
MCP_API_TOKEN=your-secret-token
MCP_USER_EMAIL=user@example.com
```
3. **Start the stack:**
```bash
docker compose -f compose.ai.yml up -d
```
4. **Connect your AI assistant to the Pipelock MCP proxy:**
```
http://your-server:8889
```
The Pipelock proxy (port 8889) scans all MCP traffic before forwarding to Sure's `/mcp` endpoint.
## Connecting AI Assistants
### Claude Desktop
Configure Claude Desktop to use Sure's MCP server:
1. Open Claude Desktop settings
2. Add a new MCP server
3. Set the endpoint to `http://your-server:8889` (if using Pipelock) or `http://your-server:3000/mcp`
4. Add the authorization header: `Authorization: Bearer your-secret-token`
### Custom Agents
Any AI agent that supports JSON-RPC 2.0 can connect to the MCP endpoint. The agent should:
1. Send a POST request to `/mcp`
2. Include the `Authorization: Bearer <token>` header
3. Use the JSON-RPC 2.0 format for requests
4. Handle the protocol methods: `initialize`, `tools/list`, `tools/call`
## Troubleshooting
### "MCP endpoint not configured" error
**Symptom:** Requests return HTTP 503 with "MCP endpoint not configured"
**Fix:** Ensure both `MCP_API_TOKEN` and `MCP_USER_EMAIL` are set as environment variables and restart Sure.
### "unauthorized" error
**Symptom:** Requests return HTTP 401 with "unauthorized"
**Fix:** Verify the `Authorization` header contains the correct token: `Bearer <MCP_API_TOKEN>`
### "MCP user not configured" error
**Symptom:** Requests return HTTP 503 with "MCP user not configured"
**Fix:** The `MCP_USER_EMAIL` does not match an existing user. Check that:
- The email is correct
- The user exists in the database
- There are no typos or extra spaces
### Pipelock connection refused
**Symptom:** AI assistant cannot connect to Pipelock's MCP proxy (port 8889)
**Fix:**
1. Verify Pipelock is running: `docker compose ps pipelock`
2. Check Pipelock health: `docker compose exec pipelock /pipelock healthcheck --addr 127.0.0.1:8888`
3. Verify the port is exposed in your `compose.yml`
## See Also
- [External AI Assistant Configuration](ai.md#external-ai-assistant) - Configure Sure's chat to use an external agent
- [Pipelock Security Proxy](pipelock.md) - Set up security scanning for MCP traffic
- [Model Context Protocol Specification](https://modelcontextprotocol.io/) - Official MCP documentation