Files
sure/app/models/assistant/function/upload_account_statement.rb
T
efb7cc3935 Tooling for the wealth + tax agent harness (#2848)
* Expose the Statement Vault to external agents over MCP

A user wants to manage patrimonial history — a document-backed record of a
family's wealth where every figure traces back to the statement it came from —
by pointing an external agent harness at Sure. That model belongs in the
harness, not in Sure: it needs numbered build deltas, golden tests and closed
periods that a mutable Postgres row cannot provide.

What Sure was missing was the seam. The Statement Vault already does most of
the work — original bytes retained, SHA-256 dedup, period detection, account
matching with a confidence score, reconciliation against ledger balances, and a
month-by-month coverage map — but it is reachable only from the web UI. An
agent could not archive a document, cite one, or check for gaps.

Adds five preview MCP tools over what already exists, plus a citation grammar
for values the agent writes:

- upload_account_statement, list_account_statements, get_account_statement,
  get_statement_coverage
- record_valuation, whose source citation is parsed rather than trusted:
  ["estimated: "] citation [" (grade: A|B|C)"]. An uncited or free-styled
  value is rejected at the write boundary instead of landing in the ledger
  looking authoritative.

link and reject are deliberately not exposed. Attaching a statement to an
account is the human's decision, and the vault UI is where it is made; the
agent reports the suggested match and stops there.

Assistant.function_classes now takes a user so preview tools stay out of the
default surface. They are hidden from tools/list and not callable by name
without the preference enabled, and the vault tools re-check the manager role
and per-account permissions, since MCP calls never pass through a controller.

Docs: the blueprint this implements, and a guide covering which side owns which
layer, the vocabulary map between the two, the monthly runbook, and the gaps
(non-user holders, non-statement documents, one value per date).

No migrations, no API endpoints, no UI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn

* Address review feedback on the vault MCP tools

Two non-blocking items from the review pass:

Document why get_statement_coverage reads through accessible_by rather than
writable_by. It reports which documents exist and writes nothing, so read
access is the right bar — and tightening it would hide coverage gaps from
people who can already see the figures those gaps sit behind. The comment
exists so a future refactor doesn't "fix" it.

Close the acknowledged verification gap with tests rather than a one-off
manual check. The review noted that nothing proved a real vault payload
serializes cleanly out through tools/call — vault responses are richer than
the other tools' output, with nested account hashes, decimal balances, dates
and a compacted hash. Two integration tests now drive the real /mcp endpoint
end to end against a real AccountStatement: one listing it, one uploading
bytes and reading back the SHA-256. Permanent regression coverage instead of
a smoke test someone has to remember to repeat.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn

* docs(llm-guides): replace patrimonial blueprint with its final revision

Swap the embedded early draft for the authoritative final revision of the
wealth + tax modelling blueprint (MIT © 2026 diegomarino):

- rename the domain vocabulary: patrimonial -> wealth, fiscal -> tax
  (tax_data/, the tax layer, tax_runner)
- add §9.5 (the intel file: shape, generation, and the capture loop)
- tighten worked examples down to placeholders
- add the MIT header; keep the in-repo NOTE block (adapted to the new
  vocabulary) and the filename untouched so cross-links don't break

* docs(llm-guides): align agent-harness guide with blueprint + fix reconcile semantics

Follow the blueprint rename (patrimonial -> wealth, fiscal -> tax,
fiscal_data/ -> tax_data/, "Phase 7 (fiscal layer)" -> "(the tax layer)")
so the two docs stop disagreeing on vocabulary.

Correct the reconciliation mapping, which conflated two different invariants:

- blueprint reconcile-or-abort (§7 pass 3) is parse-integrity (parsed parts
  == the document's own printed total); Sure's reconciliation_checks is
  ledger agreement (statement balances vs the ledger). Sure has no
  parse-integrity check and never aborts.
- opening_balance / closing_balance are user-entered, not auto-extracted, so
  over MCP reconciliation is "unavailable" until a human fills them.
- tolerance differs: blueprint 1.00/account-period vs Sure's fixed 0.01.

State in the ownership table, the invariants section, the vocabulary map and
the monthly runbook that parse-integrity and the abort belong to the harness
extractor.

* Correct the vault tools' reconciliation claims and citation parsing

Review findings from @diegomarino, all verified against the code before
changing anything.

The reconciliation claim was the serious one. get_account_statement told
agents the checks were "the trustworthy part" and returned "the balances read
off it" — but nothing reads balances off a document. MetadataDetector never
touches them and create_from_prepared_upload! never sets them; they are
user-editable fields in the Statement Vault UI. So a statement archived over
MCP always came back with an empty check list, which an agent could easily
read as "the document agrees with the ledger" when it means "nobody has
entered the figures". The description now says so, and the payload carries a
reconciliation_note spelling it out for anything reading only the JSON. Also
noted that these checks are ledger agreement, not parse integrity: nothing
here verifies a document's parts sum to its printed total.

Provenance::Citation had two patterns disagreeing about spacing. GRADE_SUFFIX
allowed "(grade:A)" but FORMAT required exactly one space, so that citation
passed the pre-check and then parsed as ungraded with the grade swallowed into
the text — silently discarding the reliability the caller supplied, which is
the one thing this parser exists to prevent.

list_account_statements downcases content_sha256 before querying. The column
is constrained to lowercase hex, so uppercase input could never match, and an
agent would read the empty result as "not archived" and upload a duplicate.
Its period filters are renamed overlapping_from / overlapping_until, since
they match on overlap and the old names claimed otherwise to anyone reading
the schema without the descriptions. has_more now explains that there is no
cursor and the way forward is a bigger limit or narrower filters.

record_valuation no longer overwrites the entry's notes. Re-recording a date
would destroy a note a person had written there. Nothing is removed now: an
identical citation is a no-op, a changed one is appended, and the trail of
what was cited when survives. Detecting "did this tool write that line?" is
not possible — almost any prose parses as a valid ungraded citation — so the
code does not guess.

Minor: accept urlsafe base64 on upload, and explain in the code why
record_valuation checks the account ACL rather than the vault manager role, so
nobody "tightens" it into the wrong permission later.

Tests cover each: the grade-spacing cases both ways, uppercase SHA lookup,
overlap window boundaries, note preservation and no-stacking, the unavailable
reconciliation note appearing and disappearing, and — per the review — that
the download URL's signed id actually expires, rather than trusting the
description's claim.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn

* Repair a bad merge in the MCP controller test

The merge of main spliced the incoming `tools/call executes
update_transaction` test into the middle of the upload round-trip test,
before its closing `end`. That left the file one `end` short, so it did
not parse — taking out both `ci / lint` (Lint/Syntax) and `ci / test_unit`
(the whole file failed to load).

Restores the missing `end`. Both tests are kept as their authors wrote
them; nothing else changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn

* docs: use wealth history wording (#2885)

* Stop the vault tools promising verification they don't perform

Three findings from the automated review passes, all confirmed against the
code before changing anything.

The download URL was dead on arrival for the caller it was built for. Sure
serves stored files through Active Storage controllers that
config/initializers/active_storage_authorization.rb gates on
`viewable_by?(Current.user)` — a signed-in browser session. An MCP client has
a bearer token and no session, so following the URL would have redirected to
sign-in. Removed it rather than leaving a link that cannot work, and the
description now points at search_family_files or the vault UI.

Coverage called a month `covered` when a document merely existed. An
unreconciled statement is not mismatched, so it took the `covered` branch, and
the payload carried nothing to correct the reading — the same "advertised
verification that never happened" bug fixed last round in
get_account_statement, in a second place. Months now carry their own
reconciliation_status, and the description says covered means presence, not
agreement.

Listing filtered visibility after limiting. Beyond underfilling a page, with
no cursor and a 100-row cap an accessible statement behind enough newer
invisible ones was unreachable. Visibility now lives in the query, mirroring
viewable_by? for a statement manager.

Also: rescue unexpected upload failures into a tool error instead of a raw
exception string, derive the documented size limit from MAX_FILE_SIZE, list
every coverage status in mcp.md, and cover the failed-reconciliation and
base64-normalisation branches.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn

* Keep storage exception detail out of the MCP response

The upload_failed message interpolated the exception text, which crosses out
to an external agent. A storage failure can carry bucket names, object keys,
paths or request details, so the agent now gets a fixed message and the
exception stays in the server log. The test asserts the absence of detail
rather than pinning the leaked string into the contract.

Also fixes a test that did not test what it claimed: the urlsafe-base64 case
used a fixture encoding to plain base64, so it exercised the padding branch
and never the "-_" translation. It now uses content whose encoding contains
both characters and asserts that up front.

Renames "rejects content that decodes to zero bytes" to "rejects blank
content", which is what it actually covers — Base64.strict_encode64("") is
"", which is blank and returns before the decoder runs, so invalid_content is
correct and empty_file is not reachable from this path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn

* docs: align wealth blueprint review feedback

* Correct the harness runbook: parse before publishing

The guide told an implementer to archive each document to Sure first and
work from there. That strands them. Sure never returns a document's bytes
over MCP — Active Storage serves stored files only to a signed-in browser
session — and there is no text fallback either, because statements archived
through upload_account_statement never enter the vector store, so
search_family_files cannot see them. A statement in Sure is metadata to an
agent and nothing more.

That blocks exactly three blueprint steps, all of them operating on bank and
broker statements: the extractors, the parts-vs-printed-total check, and the
glyph decoder. Everything else it parses — tax returns, capital accounts,
annual accounts — the harness already holds locally.

So the order inverts: the harness ingests into its own vault, extracts there
with the whole file in reach, and publishes to Sure afterwards. This restores
principle 8 rather than bending it — the recurring pipeline reads from the
canonical store, and treating Sure as canonical forced a re-fetch the
architecture never sanctioned. Both sides hash the same bytes, so the SHA-256
verifies Sure holds the identical document without moving it.

Writes down the two consequences: a statement uploaded straight into Sure's
UI can be known but never parsed (reliability C or PENDING until a copy
reaches the harness), and neither vault backs up the other.

Also drops a stale tools-table row still advertising the 15-minute download
URL removed earlier, corrects get_account_statement's description where it
suggested search_family_files as a fallback it cannot be, and disambiguates
"the vault" in the MCP tool table, which is what misled me in the first place.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JFDp9HhXDeswadu4cxFojn

---------

Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: diegomarino <diegomarino@users.noreply.github.com>
Co-authored-by: Sure Admin (bot) <sure-admin@splashblot.com>
2026-08-04 23:33:01 +02:00

197 lines
7.2 KiB
Ruby

# frozen_string_literal: true
class Assistant::Function::UploadAccountStatement < Assistant::Function
include Assistant::Function::StatementVaultSupport
class << self
def name
"upload_account_statement"
end
def description
<<~INSTRUCTIONS
Store a statement document (PDF, CSV or XLSX) in the family's Statement Vault,
the canonical archive of primary-source financial documents.
The vault keeps the original bytes and indexes them by SHA-256, so uploading
the same file twice is safe: the existing statement is returned with
`duplicate: true` and nothing new is created.
On upload the vault reads the document's period, institution and account
hints, and proposes a matching account with a confidence score. It does NOT
link the statement to that account linking is a human decision made in
Settings -> Statement Vault. Report the suggestion; don't claim the link.
Provide the file as base64 in `content_base64`. Maximum size is
#{AccountStatement::MAX_FILE_SIZE / 1.megabyte} MB.
Example:
```
upload_account_statement({
filename: "private-bank_2026-03-31_monthly-statement.pdf",
content_base64: "JVBERi0xLjcK..."
})
```
INSTRUCTIONS
end
end
def strict_mode?
false
end
def params_schema
build_schema(
required: %w[filename content_base64],
properties: {
filename: {
type: "string",
description: "Filename including extension. Must be one of: #{AccountStatement::ACCEPTED_FILE_EXTENSIONS.join(", ")}."
},
content_base64: {
type: "string",
description: "The document's raw bytes, base64-encoded."
},
account_id: {
type: "string",
description: "Optional account UUID to link the statement to on upload. Only pass this when the user has told you which account the document belongs to otherwise omit it and let the vault suggest a match for the user to confirm."
}
}
)
end
def call(params = {})
return not_a_statement_manager unless statement_manager?
filename = params["filename"].to_s.strip
return error("filename_required", "Please provide a filename with an extension.") if filename.blank?
unless AccountStatement::ACCEPTED_FILE_EXTENSIONS.include?(File.extname(filename).downcase)
return error(
"unsupported_file_type",
"The Statement Vault accepts #{AccountStatement::ACCEPTED_FILE_EXTENSIONS.join(", ")} files only.",
accepted_extensions: AccountStatement::ACCEPTED_FILE_EXTENSIONS
)
end
content = decode_content(params["content_base64"])
return error("invalid_content", "content_base64 could not be decoded as base64.") if content.nil?
return error("empty_file", "The decoded file is empty.") if content.empty?
if content.bytesize > AccountStatement::MAX_FILE_SIZE
return error(
"file_too_large",
"The file is #{content.bytesize} bytes; the maximum is #{AccountStatement::MAX_FILE_SIZE} bytes."
)
end
account = nil
if params["account_id"].present?
account = family.accounts.writable_by(user).find_by(id: params["account_id"]) if valid_uuid?(params["account_id"])
unless account
return error(
"account_not_found",
"No writable account matched that account_id. Omit account_id to upload the statement unlinked and let the vault suggest a match."
)
end
end
prepared = AccountStatement.prepare_upload!(upload_for(content, filename))
statement = AccountStatement.create_from_prepared_upload!(family: family, account: account, prepared_upload: prepared)
{
success: true,
duplicate: false,
statement: statement_payload(statement),
message: "Stored #{statement.filename} in the Statement Vault."
}
rescue AccountStatement::DuplicateUploadError => e
duplicate_response(e.statement)
rescue AccountStatement::InvalidUploadError
error(
"invalid_file",
"The file failed validation: its contents don't match its extension, or it isn't a readable #{AccountStatement::ACCEPTED_FILE_EXTENSIONS.join("/")} document."
)
rescue ActiveRecord::RecordInvalid => e
error("validation_failed", e.record.errors.full_messages.join("; "))
rescue => e
# The shared upload path can raise from content sniffing, storage or a
# validation hook. The agent gets a fixed message, never the exception text:
# a storage failure can carry bucket names, object keys, paths or request
# details, and this response crosses out to an external client. Diagnostics
# stay in the server log.
Rails.logger.error("[UploadAccountStatement] #{e.class}: #{e.message}")
error("upload_failed", "The statement could not be stored due to an unexpected error. It has been logged for the administrator.")
end
private
# The existing copy may be filed against an account this user cannot see, so
# the dedup result is reported without the details that would disclose it.
def duplicate_response(statement)
if statement.viewable_by?(user)
{
success: true,
duplicate: true,
statement: statement_payload(statement),
message: "This document is already in the vault (same SHA-256). Returning the existing statement; nothing was created."
}
else
{
success: true,
duplicate: true,
statement: { content_sha256: statement.content_sha256 },
message: "This document is already in the vault, filed against an account this user cannot see. Nothing was created."
}
end
end
# Whitespace is stripped because agents routinely wrap long base64 across
# lines, and the urlsafe alphabet is translated to the standard one because
# they sometimes emit it. Decoding stays strict after that: Base64.decode64
# quietly discards characters it doesn't understand, which would archive
# corrupted bytes under a hash that looks perfectly legitimate.
def decode_content(value)
return nil if value.blank?
normalized = value.to_s.gsub(/\s+/, "").tr("-_", "+/")
normalized += "=" * ((4 - normalized.length % 4) % 4)
Base64.strict_decode64(normalized)
rescue ArgumentError
nil
end
# AccountStatement.prepare_upload! expects an uploaded-file-like object so it
# can stream, size-check and sniff the content type. Reusing it (rather than
# building a PreparedUpload by hand) keeps the MCP path under exactly the same
# validations as the web upload form. Content type is left nil on purpose so
# the vault sniffs it from the bytes rather than trusting the caller.
def upload_for(content, filename)
DecodedUpload.new(StringIO.new(content), filename)
end
class DecodedUpload
attr_reader :original_filename, :content_type
def initialize(io, filename, content_type = nil)
@io = io
@original_filename = filename
@content_type = content_type
end
def read(*args)
@io.read(*args)
end
def rewind
@io.rewind
end
def size
@io.size
end
end
end