mirror of
https://github.com/we-promise/sure.git
synced 2026-09-02 13:21:17 +00:00
* Expose the Statement Vault to external agents over MCP A user wants to manage patrimonial history — a document-backed record of a family's wealth where every figure traces back to the statement it came from — by pointing an external agent harness at Sure. That model belongs in the harness, not in Sure: it needs numbered build deltas, golden tests and closed periods that a mutable Postgres row cannot provide. What Sure was missing was the seam. The Statement Vault already does most of the work — original bytes retained, SHA-256 dedup, period detection, account matching with a confidence score, reconciliation against ledger balances, and a month-by-month coverage map — but it is reachable only from the web UI. An agent could not archive a document, cite one, or check for gaps. Adds five preview MCP tools over what already exists, plus a citation grammar for values the agent writes: - upload_account_statement, list_account_statements, get_account_statement, get_statement_coverage - record_valuation, whose source citation is parsed rather than trusted: ["estimated: "] citation [" (grade: A|B|C)"]. An uncited or free-styled value is rejected at the write boundary instead of landing in the ledger looking authoritative. link and reject are deliberately not exposed. Attaching a statement to an account is the human's decision, and the vault UI is where it is made; the agent reports the suggested match and stops there. Assistant.function_classes now takes a user so preview tools stay out of the default surface. They are hidden from tools/list and not callable by name without the preference enabled, and the vault tools re-check the manager role and per-account permissions, since MCP calls never pass through a controller. Docs: the blueprint this implements, and a guide covering which side owns which layer, the vocabulary map between the two, the monthly runbook, and the gaps (non-user holders, non-statement documents, one value per date). No migrations, no API endpoints, no UI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * Address review feedback on the vault MCP tools Two non-blocking items from the review pass: Document why get_statement_coverage reads through accessible_by rather than writable_by. It reports which documents exist and writes nothing, so read access is the right bar — and tightening it would hide coverage gaps from people who can already see the figures those gaps sit behind. The comment exists so a future refactor doesn't "fix" it. Close the acknowledged verification gap with tests rather than a one-off manual check. The review noted that nothing proved a real vault payload serializes cleanly out through tools/call — vault responses are richer than the other tools' output, with nested account hashes, decimal balances, dates and a compacted hash. Two integration tests now drive the real /mcp endpoint end to end against a real AccountStatement: one listing it, one uploading bytes and reading back the SHA-256. Permanent regression coverage instead of a smoke test someone has to remember to repeat. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * docs(llm-guides): replace patrimonial blueprint with its final revision Swap the embedded early draft for the authoritative final revision of the wealth + tax modelling blueprint (MIT © 2026 diegomarino): - rename the domain vocabulary: patrimonial -> wealth, fiscal -> tax (tax_data/, the tax layer, tax_runner) - add §9.5 (the intel file: shape, generation, and the capture loop) - tighten worked examples down to placeholders - add the MIT header; keep the in-repo NOTE block (adapted to the new vocabulary) and the filename untouched so cross-links don't break * docs(llm-guides): align agent-harness guide with blueprint + fix reconcile semantics Follow the blueprint rename (patrimonial -> wealth, fiscal -> tax, fiscal_data/ -> tax_data/, "Phase 7 (fiscal layer)" -> "(the tax layer)") so the two docs stop disagreeing on vocabulary. Correct the reconciliation mapping, which conflated two different invariants: - blueprint reconcile-or-abort (§7 pass 3) is parse-integrity (parsed parts == the document's own printed total); Sure's reconciliation_checks is ledger agreement (statement balances vs the ledger). Sure has no parse-integrity check and never aborts. - opening_balance / closing_balance are user-entered, not auto-extracted, so over MCP reconciliation is "unavailable" until a human fills them. - tolerance differs: blueprint 1.00/account-period vs Sure's fixed 0.01. State in the ownership table, the invariants section, the vocabulary map and the monthly runbook that parse-integrity and the abort belong to the harness extractor. * Correct the vault tools' reconciliation claims and citation parsing Review findings from @diegomarino, all verified against the code before changing anything. The reconciliation claim was the serious one. get_account_statement told agents the checks were "the trustworthy part" and returned "the balances read off it" — but nothing reads balances off a document. MetadataDetector never touches them and create_from_prepared_upload! never sets them; they are user-editable fields in the Statement Vault UI. So a statement archived over MCP always came back with an empty check list, which an agent could easily read as "the document agrees with the ledger" when it means "nobody has entered the figures". The description now says so, and the payload carries a reconciliation_note spelling it out for anything reading only the JSON. Also noted that these checks are ledger agreement, not parse integrity: nothing here verifies a document's parts sum to its printed total. Provenance::Citation had two patterns disagreeing about spacing. GRADE_SUFFIX allowed "(grade:A)" but FORMAT required exactly one space, so that citation passed the pre-check and then parsed as ungraded with the grade swallowed into the text — silently discarding the reliability the caller supplied, which is the one thing this parser exists to prevent. list_account_statements downcases content_sha256 before querying. The column is constrained to lowercase hex, so uppercase input could never match, and an agent would read the empty result as "not archived" and upload a duplicate. Its period filters are renamed overlapping_from / overlapping_until, since they match on overlap and the old names claimed otherwise to anyone reading the schema without the descriptions. has_more now explains that there is no cursor and the way forward is a bigger limit or narrower filters. record_valuation no longer overwrites the entry's notes. Re-recording a date would destroy a note a person had written there. Nothing is removed now: an identical citation is a no-op, a changed one is appended, and the trail of what was cited when survives. Detecting "did this tool write that line?" is not possible — almost any prose parses as a valid ungraded citation — so the code does not guess. Minor: accept urlsafe base64 on upload, and explain in the code why record_valuation checks the account ACL rather than the vault manager role, so nobody "tightens" it into the wrong permission later. Tests cover each: the grade-spacing cases both ways, uppercase SHA lookup, overlap window boundaries, note preservation and no-stacking, the unavailable reconciliation note appearing and disappearing, and — per the review — that the download URL's signed id actually expires, rather than trusting the description's claim. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * Repair a bad merge in the MCP controller test The merge of main spliced the incoming `tools/call executes update_transaction` test into the middle of the upload round-trip test, before its closing `end`. That left the file one `end` short, so it did not parse — taking out both `ci / lint` (Lint/Syntax) and `ci / test_unit` (the whole file failed to load). Restores the missing `end`. Both tests are kept as their authors wrote them; nothing else changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * docs: use wealth history wording (#2885) * Stop the vault tools promising verification they don't perform Three findings from the automated review passes, all confirmed against the code before changing anything. The download URL was dead on arrival for the caller it was built for. Sure serves stored files through Active Storage controllers that config/initializers/active_storage_authorization.rb gates on `viewable_by?(Current.user)` — a signed-in browser session. An MCP client has a bearer token and no session, so following the URL would have redirected to sign-in. Removed it rather than leaving a link that cannot work, and the description now points at search_family_files or the vault UI. Coverage called a month `covered` when a document merely existed. An unreconciled statement is not mismatched, so it took the `covered` branch, and the payload carried nothing to correct the reading — the same "advertised verification that never happened" bug fixed last round in get_account_statement, in a second place. Months now carry their own reconciliation_status, and the description says covered means presence, not agreement. Listing filtered visibility after limiting. Beyond underfilling a page, with no cursor and a 100-row cap an accessible statement behind enough newer invisible ones was unreachable. Visibility now lives in the query, mirroring viewable_by? for a statement manager. Also: rescue unexpected upload failures into a tool error instead of a raw exception string, derive the documented size limit from MAX_FILE_SIZE, list every coverage status in mcp.md, and cover the failed-reconciliation and base64-normalisation branches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * Keep storage exception detail out of the MCP response The upload_failed message interpolated the exception text, which crosses out to an external agent. A storage failure can carry bucket names, object keys, paths or request details, so the agent now gets a fixed message and the exception stays in the server log. The test asserts the absence of detail rather than pinning the leaked string into the contract. Also fixes a test that did not test what it claimed: the urlsafe-base64 case used a fixture encoding to plain base64, so it exercised the padding branch and never the "-_" translation. It now uses content whose encoding contains both characters and asserts that up front. Renames "rejects content that decodes to zero bytes" to "rejects blank content", which is what it actually covers — Base64.strict_encode64("") is "", which is blank and returns before the decoder runs, so invalid_content is correct and empty_file is not reachable from this path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn * docs: align wealth blueprint review feedback * Correct the harness runbook: parse before publishing The guide told an implementer to archive each document to Sure first and work from there. That strands them. Sure never returns a document's bytes over MCP — Active Storage serves stored files only to a signed-in browser session — and there is no text fallback either, because statements archived through upload_account_statement never enter the vector store, so search_family_files cannot see them. A statement in Sure is metadata to an agent and nothing more. That blocks exactly three blueprint steps, all of them operating on bank and broker statements: the extractors, the parts-vs-printed-total check, and the glyph decoder. Everything else it parses — tax returns, capital accounts, annual accounts — the harness already holds locally. So the order inverts: the harness ingests into its own vault, extracts there with the whole file in reach, and publishes to Sure afterwards. This restores principle 8 rather than bending it — the recurring pipeline reads from the canonical store, and treating Sure as canonical forced a re-fetch the architecture never sanctioned. Both sides hash the same bytes, so the SHA-256 verifies Sure holds the identical document without moving it. Writes down the two consequences: a statement uploaded straight into Sure's UI can be known but never parsed (reliability C or PENDING until a copy reaches the harness), and neither vault backs up the other. Also drops a stale tools-table row still advertising the 15-minute download URL removed earlier, corrects get_account_statement's description where it suggested search_family_files as a fallback it cannot be, and disambiguates "the vault" in the MCP tool table, which is what misled me in the first place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn --------- Signed-off-by: Juan José Mata <juanjo.mata@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: diegomarino <diegomarino@users.noreply.github.com> Co-authored-by: Sure Admin (bot) <sure-admin@splashblot.com>
197 lines
7.2 KiB
Ruby
197 lines
7.2 KiB
Ruby
# frozen_string_literal: true
|
|
|
|
class Assistant::Function::UploadAccountStatement < Assistant::Function
|
|
include Assistant::Function::StatementVaultSupport
|
|
|
|
class << self
|
|
def name
|
|
"upload_account_statement"
|
|
end
|
|
|
|
def description
|
|
<<~INSTRUCTIONS
|
|
Store a statement document (PDF, CSV or XLSX) in the family's Statement Vault,
|
|
the canonical archive of primary-source financial documents.
|
|
|
|
The vault keeps the original bytes and indexes them by SHA-256, so uploading
|
|
the same file twice is safe: the existing statement is returned with
|
|
`duplicate: true` and nothing new is created.
|
|
|
|
On upload the vault reads the document's period, institution and account
|
|
hints, and proposes a matching account with a confidence score. It does NOT
|
|
link the statement to that account — linking is a human decision made in
|
|
Settings -> Statement Vault. Report the suggestion; don't claim the link.
|
|
|
|
Provide the file as base64 in `content_base64`. Maximum size is
|
|
#{AccountStatement::MAX_FILE_SIZE / 1.megabyte} MB.
|
|
|
|
Example:
|
|
|
|
```
|
|
upload_account_statement({
|
|
filename: "private-bank_2026-03-31_monthly-statement.pdf",
|
|
content_base64: "JVBERi0xLjcK..."
|
|
})
|
|
```
|
|
INSTRUCTIONS
|
|
end
|
|
end
|
|
|
|
def strict_mode?
|
|
false
|
|
end
|
|
|
|
def params_schema
|
|
build_schema(
|
|
required: %w[filename content_base64],
|
|
properties: {
|
|
filename: {
|
|
type: "string",
|
|
description: "Filename including extension. Must be one of: #{AccountStatement::ACCEPTED_FILE_EXTENSIONS.join(", ")}."
|
|
},
|
|
content_base64: {
|
|
type: "string",
|
|
description: "The document's raw bytes, base64-encoded."
|
|
},
|
|
account_id: {
|
|
type: "string",
|
|
description: "Optional account UUID to link the statement to on upload. Only pass this when the user has told you which account the document belongs to — otherwise omit it and let the vault suggest a match for the user to confirm."
|
|
}
|
|
}
|
|
)
|
|
end
|
|
|
|
def call(params = {})
|
|
return not_a_statement_manager unless statement_manager?
|
|
|
|
filename = params["filename"].to_s.strip
|
|
return error("filename_required", "Please provide a filename with an extension.") if filename.blank?
|
|
|
|
unless AccountStatement::ACCEPTED_FILE_EXTENSIONS.include?(File.extname(filename).downcase)
|
|
return error(
|
|
"unsupported_file_type",
|
|
"The Statement Vault accepts #{AccountStatement::ACCEPTED_FILE_EXTENSIONS.join(", ")} files only.",
|
|
accepted_extensions: AccountStatement::ACCEPTED_FILE_EXTENSIONS
|
|
)
|
|
end
|
|
|
|
content = decode_content(params["content_base64"])
|
|
return error("invalid_content", "content_base64 could not be decoded as base64.") if content.nil?
|
|
return error("empty_file", "The decoded file is empty.") if content.empty?
|
|
|
|
if content.bytesize > AccountStatement::MAX_FILE_SIZE
|
|
return error(
|
|
"file_too_large",
|
|
"The file is #{content.bytesize} bytes; the maximum is #{AccountStatement::MAX_FILE_SIZE} bytes."
|
|
)
|
|
end
|
|
|
|
account = nil
|
|
if params["account_id"].present?
|
|
account = family.accounts.writable_by(user).find_by(id: params["account_id"]) if valid_uuid?(params["account_id"])
|
|
|
|
unless account
|
|
return error(
|
|
"account_not_found",
|
|
"No writable account matched that account_id. Omit account_id to upload the statement unlinked and let the vault suggest a match."
|
|
)
|
|
end
|
|
end
|
|
|
|
prepared = AccountStatement.prepare_upload!(upload_for(content, filename))
|
|
statement = AccountStatement.create_from_prepared_upload!(family: family, account: account, prepared_upload: prepared)
|
|
|
|
{
|
|
success: true,
|
|
duplicate: false,
|
|
statement: statement_payload(statement),
|
|
message: "Stored #{statement.filename} in the Statement Vault."
|
|
}
|
|
rescue AccountStatement::DuplicateUploadError => e
|
|
duplicate_response(e.statement)
|
|
rescue AccountStatement::InvalidUploadError
|
|
error(
|
|
"invalid_file",
|
|
"The file failed validation: its contents don't match its extension, or it isn't a readable #{AccountStatement::ACCEPTED_FILE_EXTENSIONS.join("/")} document."
|
|
)
|
|
rescue ActiveRecord::RecordInvalid => e
|
|
error("validation_failed", e.record.errors.full_messages.join("; "))
|
|
rescue => e
|
|
# The shared upload path can raise from content sniffing, storage or a
|
|
# validation hook. The agent gets a fixed message, never the exception text:
|
|
# a storage failure can carry bucket names, object keys, paths or request
|
|
# details, and this response crosses out to an external client. Diagnostics
|
|
# stay in the server log.
|
|
Rails.logger.error("[UploadAccountStatement] #{e.class}: #{e.message}")
|
|
error("upload_failed", "The statement could not be stored due to an unexpected error. It has been logged for the administrator.")
|
|
end
|
|
|
|
private
|
|
# The existing copy may be filed against an account this user cannot see, so
|
|
# the dedup result is reported without the details that would disclose it.
|
|
def duplicate_response(statement)
|
|
if statement.viewable_by?(user)
|
|
{
|
|
success: true,
|
|
duplicate: true,
|
|
statement: statement_payload(statement),
|
|
message: "This document is already in the vault (same SHA-256). Returning the existing statement; nothing was created."
|
|
}
|
|
else
|
|
{
|
|
success: true,
|
|
duplicate: true,
|
|
statement: { content_sha256: statement.content_sha256 },
|
|
message: "This document is already in the vault, filed against an account this user cannot see. Nothing was created."
|
|
}
|
|
end
|
|
end
|
|
|
|
# Whitespace is stripped because agents routinely wrap long base64 across
|
|
# lines, and the urlsafe alphabet is translated to the standard one because
|
|
# they sometimes emit it. Decoding stays strict after that: Base64.decode64
|
|
# quietly discards characters it doesn't understand, which would archive
|
|
# corrupted bytes under a hash that looks perfectly legitimate.
|
|
def decode_content(value)
|
|
return nil if value.blank?
|
|
|
|
normalized = value.to_s.gsub(/\s+/, "").tr("-_", "+/")
|
|
normalized += "=" * ((4 - normalized.length % 4) % 4)
|
|
|
|
Base64.strict_decode64(normalized)
|
|
rescue ArgumentError
|
|
nil
|
|
end
|
|
|
|
# AccountStatement.prepare_upload! expects an uploaded-file-like object so it
|
|
# can stream, size-check and sniff the content type. Reusing it (rather than
|
|
# building a PreparedUpload by hand) keeps the MCP path under exactly the same
|
|
# validations as the web upload form. Content type is left nil on purpose so
|
|
# the vault sniffs it from the bytes rather than trusting the caller.
|
|
def upload_for(content, filename)
|
|
DecodedUpload.new(StringIO.new(content), filename)
|
|
end
|
|
|
|
class DecodedUpload
|
|
attr_reader :original_filename, :content_type
|
|
|
|
def initialize(io, filename, content_type = nil)
|
|
@io = io
|
|
@original_filename = filename
|
|
@content_type = content_type
|
|
end
|
|
|
|
def read(*args)
|
|
@io.read(*args)
|
|
end
|
|
|
|
def rewind
|
|
@io.rewind
|
|
end
|
|
|
|
def size
|
|
@io.size
|
|
end
|
|
end
|
|
end
|