* Expose the Statement Vault to external agents over MCP A user wants to manage patrimonial history — a document-backed record of a family's wealth where every figure traces back to the statement it came from — by pointing an external agent harness at Sure. That model belongs in the harness, not in Sure: it needs numbered build deltas, golden tests and closed periods that a mutable Postgres row cannot provide. What Sure was missing was the seam. The Statement Vault already does most of the work — original bytes retained, SHA-256 dedup, period detection, account matching with a confidence score, reconciliation against ledger balances, and a month-by-month coverage map — but it is reachable only from the web UI. An agent could not archive a document, cite one, or check for gaps. Adds five preview MCP tools over what already exists, plus a citation grammar for values the agent writes: - upload_account_statement, list_account_statements, get_account_statement, get_statement_coverage - record_valuation, whose source citation is parsed rather than trusted: ["estimated: "] citation [" (grade: A|B|C)"]. An uncited or free-styled value is rejected at the write boundary instead of landing in the ledger looking authoritative. link and reject are deliberately not exposed. Attaching a statement to an account is the human's decision, and the vault UI is where it is made; the agent reports the suggested match and stops there. Assistant.function_classes now takes a user so preview tools stay out of the default surface. They are hidden from tools/list and not callable by name without the preference enabled, and the vault tools re-check the manager role and per-account permissions, since MCP calls never pass through a controller. Docs: the blueprint this implements, and a guide covering which side owns which layer, the vocabulary map between the two, the monthly runbook, and the gaps (non-user holders, non-statement documents, one value per date). No migrations, no API endpoints, no UI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * Address review feedback on the vault MCP tools Two non-blocking items from the review pass: Document why get_statement_coverage reads through accessible_by rather than writable_by. It reports which documents exist and writes nothing, so read access is the right bar — and tightening it would hide coverage gaps from people who can already see the figures those gaps sit behind. The comment exists so a future refactor doesn't "fix" it. Close the acknowledged verification gap with tests rather than a one-off manual check. The review noted that nothing proved a real vault payload serializes cleanly out through tools/call — vault responses are richer than the other tools' output, with nested account hashes, decimal balances, dates and a compacted hash. Two integration tests now drive the real /mcp endpoint end to end against a real AccountStatement: one listing it, one uploading bytes and reading back the SHA-256. Permanent regression coverage instead of a smoke test someone has to remember to repeat. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * docs(llm-guides): replace patrimonial blueprint with its final revision Swap the embedded early draft for the authoritative final revision of the wealth + tax modelling blueprint (MIT © 2026 diegomarino): - rename the domain vocabulary: patrimonial -> wealth, fiscal -> tax (tax_data/, the tax layer, tax_runner) - add §9.5 (the intel file: shape, generation, and the capture loop) - tighten worked examples down to placeholders - add the MIT header; keep the in-repo NOTE block (adapted to the new vocabulary) and the filename untouched so cross-links don't break * docs(llm-guides): align agent-harness guide with blueprint + fix reconcile semantics Follow the blueprint rename (patrimonial -> wealth, fiscal -> tax, fiscal_data/ -> tax_data/, "Phase 7 (fiscal layer)" -> "(the tax layer)") so the two docs stop disagreeing on vocabulary. Correct the reconciliation mapping, which conflated two different invariants: - blueprint reconcile-or-abort (§7 pass 3) is parse-integrity (parsed parts == the document's own printed total); Sure's reconciliation_checks is ledger agreement (statement balances vs the ledger). Sure has no parse-integrity check and never aborts. - opening_balance / closing_balance are user-entered, not auto-extracted, so over MCP reconciliation is "unavailable" until a human fills them. - tolerance differs: blueprint 1.00/account-period vs Sure's fixed 0.01. State in the ownership table, the invariants section, the vocabulary map and the monthly runbook that parse-integrity and the abort belong to the harness extractor. * Correct the vault tools' reconciliation claims and citation parsing Review findings from @diegomarino, all verified against the code before changing anything. The reconciliation claim was the serious one. get_account_statement told agents the checks were "the trustworthy part" and returned "the balances read off it" — but nothing reads balances off a document. MetadataDetector never touches them and create_from_prepared_upload! never sets them; they are user-editable fields in the Statement Vault UI. So a statement archived over MCP always came back with an empty check list, which an agent could easily read as "the document agrees with the ledger" when it means "nobody has entered the figures". The description now says so, and the payload carries a reconciliation_note spelling it out for anything reading only the JSON. Also noted that these checks are ledger agreement, not parse integrity: nothing here verifies a document's parts sum to its printed total. Provenance::Citation had two patterns disagreeing about spacing. GRADE_SUFFIX allowed "(grade:A)" but FORMAT required exactly one space, so that citation passed the pre-check and then parsed as ungraded with the grade swallowed into the text — silently discarding the reliability the caller supplied, which is the one thing this parser exists to prevent. list_account_statements downcases content_sha256 before querying. The column is constrained to lowercase hex, so uppercase input could never match, and an agent would read the empty result as "not archived" and upload a duplicate. Its period filters are renamed overlapping_from / overlapping_until, since they match on overlap and the old names claimed otherwise to anyone reading the schema without the descriptions. has_more now explains that there is no cursor and the way forward is a bigger limit or narrower filters. record_valuation no longer overwrites the entry's notes. Re-recording a date would destroy a note a person had written there. Nothing is removed now: an identical citation is a no-op, a changed one is appended, and the trail of what was cited when survives. Detecting "did this tool write that line?" is not possible — almost any prose parses as a valid ungraded citation — so the code does not guess. Minor: accept urlsafe base64 on upload, and explain in the code why record_valuation checks the account ACL rather than the vault manager role, so nobody "tightens" it into the wrong permission later. Tests cover each: the grade-spacing cases both ways, uppercase SHA lookup, overlap window boundaries, note preservation and no-stacking, the unavailable reconciliation note appearing and disappearing, and — per the review — that the download URL's signed id actually expires, rather than trusting the description's claim. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * Repair a bad merge in the MCP controller test The merge of main spliced the incoming `tools/call executes update_transaction` test into the middle of the upload round-trip test, before its closing `end`. That left the file one `end` short, so it did not parse — taking out both `ci / lint` (Lint/Syntax) and `ci / test_unit` (the whole file failed to load). Restores the missing `end`. Both tests are kept as their authors wrote them; nothing else changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * docs: use wealth history wording (#2885) * Stop the vault tools promising verification they don't perform Three findings from the automated review passes, all confirmed against the code before changing anything. The download URL was dead on arrival for the caller it was built for. Sure serves stored files through Active Storage controllers that config/initializers/active_storage_authorization.rb gates on `viewable_by?(Current.user)` — a signed-in browser session. An MCP client has a bearer token and no session, so following the URL would have redirected to sign-in. Removed it rather than leaving a link that cannot work, and the description now points at search_family_files or the vault UI. Coverage called a month `covered` when a document merely existed. An unreconciled statement is not mismatched, so it took the `covered` branch, and the payload carried nothing to correct the reading — the same "advertised verification that never happened" bug fixed last round in get_account_statement, in a second place. Months now carry their own reconciliation_status, and the description says covered means presence, not agreement. Listing filtered visibility after limiting. Beyond underfilling a page, with no cursor and a 100-row cap an accessible statement behind enough newer invisible ones was unreachable. Visibility now lives in the query, mirroring viewable_by? for a statement manager. Also: rescue unexpected upload failures into a tool error instead of a raw exception string, derive the documented size limit from MAX_FILE_SIZE, list every coverage status in mcp.md, and cover the failed-reconciliation and base64-normalisation branches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * Keep storage exception detail out of the MCP response The upload_failed message interpolated the exception text, which crosses out to an external agent. A storage failure can carry bucket names, object keys, paths or request details, so the agent now gets a fixed message and the exception stays in the server log. The test asserts the absence of detail rather than pinning the leaked string into the contract. Also fixes a test that did not test what it claimed: the urlsafe-base64 case used a fixture encoding to plain base64, so it exercised the padding branch and never the "-_" translation. It now uses content whose encoding contains both characters and asserts that up front. Renames "rejects content that decodes to zero bytes" to "rejects blank content", which is what it actually covers — Base64.strict_encode64("") is "", which is blank and returns before the decoder runs, so invalid_content is correct and empty_file is not reachable from this path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * docs: align wealth blueprint review feedback * Correct the harness runbook: parse before publishing The guide told an implementer to archive each document to Sure first and work from there. That strands them. Sure never returns a document's bytes over MCP — Active Storage serves stored files only to a signed-in browser session — and there is no text fallback either, because statements archived through upload_account_statement never enter the vector store, so search_family_files cannot see them. A statement in Sure is metadata to an agent and nothing more. That blocks exactly three blueprint steps, all of them operating on bank and broker statements: the extractors, the parts-vs-printed-total check, and the glyph decoder. Everything else it parses — tax returns, capital accounts, annual accounts — the harness already holds locally. So the order inverts: the harness ingests into its own vault, extracts there with the whole file in reach, and publishes to Sure afterwards. This restores principle 8 rather than bending it — the recurring pipeline reads from the canonical store, and treating Sure as canonical forced a re-fetch the architecture never sanctioned. Both sides hash the same bytes, so the SHA-256 verifies Sure holds the identical document without moving it. Writes down the two consequences: a statement uploaded straight into Sure's UI can be known but never parsed (reliability C or PENDING until a copy reaches the harness), and neither vault backs up the other. Also drops a stale tools-table row still advertising the 15-minute download URL removed earlier, corrects get_account_statement's description where it suggested search_family_files as a fallback it cannot be, and disambiguates "the vault" in the MCP tool table, which is what misled me in the first place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: diegomarino <diegomarino@users.noreply.github.com> Co-authored-by: Sure Admin (bot) <sure-admin@splashblot.com>
17 KiB
Wealth history with an external agent harness
How to run a provenance-first wealth model — one where every number walks back to the document it came from — on top of Sure, without putting that model inside Sure.
The model itself is specified by the wealth + tax blueprint. This guide is the seam: which half owns what, which MCP tool serves which layer of the blueprint, and which invariants Sure cannot enforce for you.
The two-install shape
Sure is the system of record. Accounts, values and statement documents live here, and the agent's job against Sure is to keep them clean, complete and cited.
The harness is your own repository, driven by an agent (OpenClaw, Claude
Code, anything that speaks MCP). It holds the wealth + tax memory: the position
catalog, the tax criteria, the numbered-delta compiler, the golden tests and
the compiled workbook. It reads and writes Sure over /mcp.
your repo (the harness) Sure
─────────────────────── ────
catalogs/ positions, criteria, FX ──MCP──► accounts, entries, valuations
tax_data/*.jsonl Statement Vault (documents + sha256)
data/YYYY-MM/*.csv (snapshot) ◄─MCP─── coverage + reconciliation checks
src/NNN_*.py → workbook.xlsx
tests/ goldens
Sure is deliberately not the compiler. The blueprint's §2 principle 1 — the workbook is a build output regenerated from source — only works if the source is versioned, diffable and immutable once closed. That is what your git repo is for.
Who owns which layer
| Blueprint layer | Owner | In Sure |
|---|---|---|
| L0/L1 entities, holders | Sure, partially | Family, User, accounts.owner_id. Holders that are not Sure users — a holding company, a trust — have no home here; keep them in the harness's entity registry. |
| L1 account registry | Sure | Account + its accountable. Off-bank positions model well as OtherAsset or Property accounts. |
| L1 positions, parameters, FX table, tax criteria | Harness | — |
| L2 wealth series | Sure is authoritative | Entry / Valuation, Holding. The harness snapshots it into data/YYYY-MM/*.csv so git diff and the goldens have something to bite on. |
L2 tax series — criterion, applicable, declared, legal_max |
Harness only | Sure has one value per account per date. "One value, one criterion" (§2 principle 6) has no representation here and should not be forced into one. |
| L3 vault — account statements | Split | Sure's Statement Vault is the shared archive: original bytes in storage, SHA-256 dedup, period detection, account matching, review queue. But it never hands bytes back over MCP, so the harness must keep its own copy of any statement it intends to extract from. See "The harness keeps the parseable master" below. |
| L3 vault — tax returns, annual accounts, capital accounts, contracts, minutes, email | Harness | The Statement Vault is statement-shaped and accepts PDF/CSV/XLSX only. Keep other primary sources in the harness's own git-ignored vault. |
_control.csv / reconcile-or-abort |
Split | Sure's get_account_statement reports ledger-agreement checks (statement balances vs the ledger, tolerance 0.01, report-only). The blueprint's parse-integrity check (parts vs the document's own printed total) and the abort have no counterpart in Sure — keep both in the harness extractor. |
Gap map / PENDING policy |
Sure | get_statement_coverage reports covered / missing / mismatched / ambiguous per month. |
| Numbered deltas, goldens, workbook, divergences report | Harness only | — |
Three invariants Sure cannot give you
- Closed periods are immutable (§2 principle 7). A Postgres row is mutable and keeps no history you can diff. Immutability lives in the harness snapshot and its commits.
- Golden tests (§14). Same reason: pin row counts and snapshot totals in the harness, against the snapshot, not against a live query.
- Reconcile-or-abort as a hard gate (§2 principle 3, §7 pass 3). Sure's
get_account_statementreports statement-vs-ledger agreement and never aborts; it does not check the blueprint's parse-integrity invariant (parsed parts summing to the document's own printed total), and its tolerance is a fixed 0.01, not the blueprint's 1.00-per-account-period gate. Both that check and the abort belong to the harness extractor. See the vocabulary map.
Treat Sure as a source you re-derive from, not as the archive of what you already derived.
The harness keeps the parseable master
Sure never returns a document's bytes over MCP, by design. Stored files are
served only to a signed-in browser session (Active Storage authorization checks
viewable_by?(Current.user)), and an MCP client holds a bearer token, not a
session. Nor is there a text fallback: statements archived through
upload_account_statement do not enter the vector store, so
search_family_files cannot see them either. To an agent, a statement in Sure is
metadata — identity, period, account, coverage, ledger reconciliation — and
nothing more.
That matters because the blueprint needs the bytes for three things, all of them operating on bank and broker statements: the §9 extractors, the §7 pass-3 parse-integrity check, and the §9.2 glyph decoder. Everything else the blueprint parses — tax returns, capital accounts, annual accounts — the harness already holds locally, per the ownership table.
So parse first, publish second. The extractor runs on the harness's own copy, where the bytes are; Sure receives the archived copy afterwards:
- The document lands in the harness's inbox.
- The harness's
vault_opsingests it into the harness vault — canonical name, SHA-256, manifest, human OK (§8.1–8.3). - The extractor parses it there, with the whole file: labelled subtotals, glyph decoding where needed, reconcile-or-abort against the printed total.
- The harness publishes a copy to Sure with
upload_account_statement. - The harness records values with
record_valuation, citing that SHA-256. - Sure supplies what the harness cannot: ledger reconciliation, the coverage map, the review and linking UI, and the household's shared archive.
This ordering restores §2 principle 8 rather than bending it. The principle says the recurring pipeline reads exclusively from the canonical store — and treating Sure as canonical would force a re-fetch the architecture never sanctioned. The harness vault is canonical; Sure is where you publish.
The SHA-256 is the join key, and it removes the need to move bytes at all.
Both sides hash the same file independently, so
list_account_statements(content_sha256: …) verifies that Sure holds the
identical document. That is §8.2's "same content = same hash regardless of name"
applied across a system boundary.
Two consequences worth stating plainly rather than discovering later:
- A statement someone uploads straight into Sure's web UI, which the harness
never saw, can be known but not parsed. You get its period, account,
coverage and ledger reconciliation; you cannot extract from it. Per §13 any
value derived from it is reliability C, or stays
PENDING, until a copy reaches the harness inbox. - Neither vault backs up the other. Sure cannot rebuild the harness's data layer, and the harness cannot rebuild Sure's archive.
The tools
These are preview features. Enable them per user in Settings → Preferences;
until then they do not appear in tools/list and calling one by name returns
"Unknown tool". The MCP user must also be an admin or member — the Statement
Vault is closed to guests, over MCP exactly as in the UI.
| Tool | Use it for |
|---|---|
upload_account_statement |
Ingest a statement (PDF/CSV/XLSX, ≤25 MB, base64). Returns the SHA-256. Re-uploading identical bytes returns the existing record with duplicate: true — dedup is free and idempotent, so a re-run is safe. |
list_account_statements |
The vault index and the review queue. Filter by account, period, review_status, or content_sha256 to check whether a document is already archived. |
get_account_statement |
One document: identity, the balances recorded for it (user-entered in Settings → Statement Vault, not auto-extracted — blank until filled), and the reconciliation checks against the ledger when balances exist. Returns no bytes and no link — see "The harness keeps the parseable master". |
get_statement_coverage |
The month-by-month gap map for an account: covered, missing, mismatched, ambiguous, duplicate, not_expected. |
record_valuation |
Write a value for a date, with a mandatory citation. |
search_family_files |
Semantic search inside uploaded documents (needs a vector store configured). Complements the vault: identity from list_account_statements, contents from here. |
get_accounts, get_holdings, get_balance_sheet, get_transactions |
Pulling L1/L2 into the harness snapshot. |
The citation grammar
record_valuation requires a source and parses it. This is the one place Sure
can enforce §2 principle 2 — never invent a datum — so it fails loud rather than
storing an uncited number:
source := ["estimated: "] citation [" (grade: A|B|C)"]
A— an official document for that exact date.B— derived with a document.C— a proxy or assumption: a standing TODO to re-derive from the real source.estimated:— interpolated or proxied rather than read off a document. Estimates must carry a grade.
Valid:
Private bank statement 2026-03-31, securities subtotal (grade: A)
estimated: linear interpolation over 2024-08 / 2024-12 anchors (grade: C)
Rejected: a blank citation, (grade: D), Estimated: with a capital E, and an
estimated: value with no grade. The citation is stored on the entry's notes,
so it travels with the value and shows in the UI.
What the agent does not get to do
link and reject are not exposed. Attaching a statement to an account is
the human sign-off step of §8.3, and §17 keeps external actions with the owner.
Sure already proposes a match — suggested_account with a confidence score,
computed at upload — and the user confirms it in Settings → Statement Vault.
Report the suggestion. Do not describe a suggested match as a link.
Vocabulary map
Reading the blueprint against this codebase:
| Blueprint | Sure |
|---|---|
{inbox} / staging inbox |
statements with review_status: "unmatched" |
| triage (mechanical-first) | AccountStatement::MetadataDetector — period, institution, last-4 from filename and contents |
| triage proposal | AccountStatement::AccountMatcher → suggested_account + match_confidence |
| dry-run manifest + human OK (§8.3) | the link/reject step in Settings → Statement Vault |
| sha256 index (§8.2) | account_statements.content_sha256, unique per family |
TOKEN_YYYY-MM-DD_Title.ext (§8.1) |
not enforced; the SHA-256 is the stable identifier, and filenames are free text |
_control.csv official total |
opening_balance / closing_balance — user-entered fields (Settings → Statement Vault), not auto-extracted on upload; blank until a human fills them, so reconciliation is unavailable over MCP until then |
| reconcile-or-abort (§7 pass 3) | no direct equivalent — reconciliation_checks gives statement-vs-ledger agreement (tolerance 0.01, report-only, matched / mismatched); it does not sum parsed parts against the document's printed total, and never aborts. Parse-integrity + abort stay harness-side |
gap policy PENDING (§13) |
coverage status missing |
| reliability grade (§13.4) | the (grade: A|B|C) suffix on record_valuation's source |
The monthly runbook, against Sure
Blueprint §15.2, rewritten:
- Ingest and extract, harness-side first. New documents land in the
harness's inbox, go through its own
vault_ops(canonical name, SHA-256, manifest, human OK), and are parsed there — reconcile-or-abort against the printed total, while the bytes are still in reach. Only then publish each one to Sure withupload_account_statement. Aduplicate: trueresponse means it was already archived: a normal outcome, not an error, and confirmation that both sides hold the same file. Publishing before extracting strands you — Sure will not hand the bytes back. - Hand off the match. Report each unmatched statement and its suggested account to the user; they confirm in Settings → Statement Vault. Do not proceed as if the link exists.
- Reconcile.
get_account_statementon each new statement. Reconciliation only runs once the statement'sopening_balance/closing_balanceare filled — these are user-entered in Settings → Statement Vault, not auto-extracted, so an agent-only pipeline getsreconciliation_status: "unavailable"here until a human enters them. When a check does run, amismatchedresult means the ledger and the document disagree — stop and report it. Never adjust the figure to make it agree. (This is statement-vs-ledger agreement; the blueprint's parts-vs-printed-total parse-integrity check and its abort are the harness extractor's job.) - Close the gaps.
get_statement_coverageper account for the year. Everymissingmonth is either a document to go find or an explicitPENDINGin the harness — never a silently interpolated number. - Value the off-bank positions.
record_valuationwith a citation, one per position that moved. - Snapshot. Pull L1/L2 into
data/YYYY-MM/*.csvwith the byte-stable writer. A no-op re-run must produce an emptygit diff. - Goldens. Run them. When a count moves, update the constant with its one-line justification.
- Build. Regenerate the workbook, recalc, confirm zero formula errors.
- Commit the snapshot, the golden update, and the working-memory sync in one commit naming the period and the reconciliation result.
Building the harness: the phases, remapped
Blueprint §20 assumes you build everything. Against Sure, several phases are already done:
- Phase 0 (map the sources) — unchanged, and still the phase people skip.
list_account_statementsandget_statement_coveragegive you the inventory for anything already in Sure. - Phase 1 (catalogs, schemas, skeleton) — build the harness's schemas and
the snapshot writer. The account registry comes from
get_accountsinstead of being hand-curated; keep the positions catalog local. - Phases 2–4 (extractors) — mostly replaced. Sure's provider syncs and
statement parsing already produce L2. The harness's "extractor" is a
pull_from_surestep writing the snapshot. Write extractors only for source families Sure does not handle. - Phase 5 (views) — unchanged, harness-side.
- Phase 6 (the vault) — partly already built. Sure gives you archival, SHA-256 dedup, the review queue and the coverage map for account statements, so don't rebuild those. You still need a harness-side vault for every document the extractors read — statements included, since Sure won't return their bytes — plus the non-statement sources it was never going to hold.
- Phase 7 (the tax layer) — entirely harness-side. Sure has no criterion dimension and should not grow one.
- Phase 8 (forensics, intel, agent memory) — entirely harness-side. Email
sweeps never touch Sure; facts they establish enter as an
upload_account_statement(if a document backs them) plus arecord_valuationciting it.
Known gaps
Worth knowing before you promise the user something:
- Non-user holders. Sure's holder concept is a
Userin aFamily. A holding company as a first-class holder needs the harness's own registry, with its Sure accounts mapped to it. - Non-statement documents. PDF/CSV/XLSX statements only. Tax returns and capital accounts can be uploaded if they fit those formats, but the vault will treat them as statements — period detection and account matching will produce noise. Prefer the harness's own vault for those.
- One value per account per date. Storing two criteria for the same position and date, exactly one applicable, is the harness's job.
- Statement periods are detected, not guaranteed.
MetadataDetectorreads them from the filename and contents; checkperiod_start_on/period_end_onon the upload response before relying on coverage.
See also
- The wealth + tax blueprint — the full spec.
- MCP Server for External AI Assistants — endpoint, authentication, tool list.
- External AI Assistant configuration — pointing OpenClaw at Sure.
- Gating a preview feature — the toggle these tools sit behind.