Files
sure/app/models/assistant/function_tool_caller.rb
T
Brandon 25a9011f14 feat(ai): analytical tool-set upgrade for the builtin assistant (#3064)
* fix(assistant): survive tool failures with error and hint results instead of aborting the turn

A tool exception used to raise FunctionExecutionError out of the responder
loop, turning the whole turn into a generic chat error banner. An unknown
tool name was worse: the rescue block itself crashed (fn.name on nil).

Tool failures now come back to the model as data ({error, hint}) so the
conversation survives and the model can retry once with corrected
arguments. The catch-all branch logs and tells the model not to retry.
FunctionExecutionError remains defined for API compatibility.

* fix(assistant): strict schemas declare every property as required

get_categories and get_tags declared an optional page property while
inheriting strict mode, which is invalid under strict function calling
(every property must be listed in required). Both now opt out of strict
mode like every other paginated tool, and gain a page_size param
(1..100) while being touched.

A registry-walking test asserts the invariant for every current and
future tool, preview tools included.

* fix(assistant): HistoryTrimmer always keeps the newest turn

Trimming iterates newest-first and stopped at the first group over
budget. When the newest group alone exceeded the budget, everything was
dropped, including the user message the model was being asked to
answer, and the provider received only the system prompt. The newest
group now always survives.

* perf(assistant): compact AI time series and make account history opt-in

get_accounts shipped a 5-year monthly series for every account on every
call, roughly 60 formatted money strings per account, which dominates
the tool payload for multi-account families and swamps small
self-hosted context windows. The series is now opt-in
(include_balance_series) and bounded by a named period (series_period,
default last_365_days).

to_ai_time_series states the currency once and emits numeric values
instead of formatting every point; the system prompt already tells the
model how to render currency.

get_accounts also now returns account ids (they are what other tools
accept as account_ids filters) and respects the visible scope, so
hidden accounts no longer leak into responses.

* refactor(assistant): id and name filters replace user-data enums in get_transactions

The schema inlined every account, category, merchant, and tag name as
enum values on every request. That grows without bound with family
data, defeats provider prompt caching (definitions change whenever a
name does), and is the pattern that made empty-enum pruning necessary
in the first place.

Filters are now plain string arrays documented as exact names from the
sibling get_* tools, which Transaction::Search already resolves
server-side, plus an account_ids UUID filter. New params: page_size
(1..100), sort_by amount, types (income/expense/transfer, the way to
exclude transfers), and statuses (pending/confirmed).

The three now-unused enum helpers are removed from the base class;
family_tag_names stays for update_tag, which still identifies tags by
name.

* feat(assistant): add get_merchants and get_recurring_transactions

Merchants were unreachable: names appeared nowhere and
update_transaction's merchant_id had no source of ids, making it
unusable. get_merchants lists id, exact name, and source, scoped
through available_merchants_for so merchants seen only in accounts
hidden from the user never leak.

Recurring transactions had a model, an Upcoming view, and no assistant
reach. get_recurring_transactions lists detected and manual recurring
items (status filter defaulting to active, optional
upcoming_within_days window) with per-currency totals of active
non-transfer items, answering subscription and upcoming-bill questions
directly instead of via transaction paging.

* feat(assistant): flexible periods on get_balance_sheet and trends on get_income_statement

get_balance_sheet was hard-wired to five years of monthly history with
no parameters, although Period supports arbitrary ranges and the chart
builder takes any interval. It now accepts a named period or custom
dates plus an interval, with a 400-point cap so a day-granularity
request over a decade returns an error instead of a giant series. The
default call is byte-compatible with the old shape. The balance sheet
object is also memoized; it was being constructed four times per call.

get_income_statement gains the analysis surface the assistant lacked:
group_by month for a monthly income/expenses/net series (capped at 36
buckets), compare_previous_period for an equal-length prior window
with absolute and percent deltas, and account_ids to scope totals to
specific accounts via IncomeStatement#totals_for. Category breakdowns
are family-wide by construction, so the account-filtered view omits
them and says why. Unknown or inaccessible account ids come back as a
soft failure naming the ids so the model can correct itself.

* feat(assistant): preview reads for insights and valuations

The Insights feed is generated nightly with pre-computed numbers, and
the chat assistant could not read a word of it. get_insights returns
the visible feed (type filter, acknowledged toggle, limit) without
marking anything read; an assistant read is not the user viewing the
feed. It sits in PREVIEW_FUNCTION_CLASSES because the feature itself is
preview-gated, which also keeps it off the default /mcp surface.

record_valuation was write-only: an agent recording provenance-cited
valuations had no way to audit what it wrote or find dates already
carrying a value. get_valuations lists valuation entries newest first
with kind and the citation notes, scoped to accessible visible
accounts.

* feat(assistant): cache-stable system prompt with session context

The prompt interpolated currency mid-text and the date near the end, so
no two requests shared a cacheable prefix, and it told the model
nothing about the family: not one account name, not a single category.
Models opened most conversations blind, either wandering through tools
or answering without data.

The prompt is now STATIC_INSTRUCTIONS, a frozen constant that is
byte-identical for every request (providers discount an
exactly-repeated prefix; tool definitions are also stable now that
schemas carry no user data), followed by a trailing Session context
block holding everything volatile: date, date format, currency details,
an account roster with balances, and category names.

The static half gains a request-classification rule (CHAT / LOOKUP /
ANALYSIS), a reuse-what-you-have rule with an explicit re-fetch
carve-out, specific-tool preference, and the error/hint retry-once
rule that pairs with the tool soft-fail contract.

Context stays cheap by construction: the roster collapses to per-type
counts beyond 25 accounts, categories to a count beyond 60 names, and
both collapse whenever the configured context window is under 4096
(the self-hosted default is 2048), via the new Assistant::TokenBudget
helper. Intro chats are untouched.

* feat(assistant): raise tool-round cap to 8 with a no-tools grace turn; instructions-aware history budget

Five rounds was tight for a tool surface that now supports real
analysis chains, and hitting the cap raised ToolCallLimitError, which
surfaced to the user as a dead chat with an error banner. The default
is now eight rounds (env override unchanged), and on the final
permitted round the follow-up request offers no tools, so the model
must answer in text with whatever it gathered. The limit error remains
as a defensive backstop.

The generic-path history budget reserved a flat 256 tokens for a
system prompt that already estimates well past that; the trimmer now
budgets against the actual instructions when available.

LLM_MAX_RESPONSE_TOKENS was reserved in budget math but never sent to
the provider. It is now sent (max_tokens on chat completions,
max_output_tokens on the Responses API) only when explicitly
configured via ENV or a stored Setting; stock installs keep today's
uncapped behavior.

* test(evals): chat golden v2 exercising the real prompt and registry

The eval runner scored a fiction: hardcoded instructions and four fake
permissive tool schemas, so a prompt or registry regression could
sail through green. It now runs STATIC_INSTRUCTIONS plus a fixed
synthetic session context and builds definitions from
Assistant.function_classes against a reference user (classes whose
schema cannot build are skipped with a log line, never faked).

chat_golden_v2 adds routing scenarios the upgrade cares about: CHAT
classification must use no tools, aggregates route to
get_income_statement / get_balance_sheet rather than transaction
paging, and the new analytical tools are selected with sensible
params. The dataset header documents the harness's single-shot
limitation.

* docs(ai,mcp): current tool tables, responder loop, prompt structure, timeout math

Both docs listed 7 tools against a registry of 19, in three separate
drift-prone copies. mcp.md now carries the canonical tables (default +
preview); ai.md links to them from the MCP section, keeps one grouped
functions list for the architecture chapter, and replaces its stale
hardcoded registry snippet with a pointer to assistant.rb.

The architecture section gains the contracts contributors need when
adding a function: the responder loop (rounds vs calls, cap 8, the
no-tools grace turn) and the error/hint soft-failure convention, plus
the prompt's static/session-context split and its collapse gates.
Timeout guidance is recomputed for the new default cap.

* fix(assistant): address automated review findings

Codex and CodeRabbit findings on the initial push, all verified before
changing anything:

- AI time series rounded every value to two decimals, which turns
  0.001 BTC into 0.0; values now round to the currency's own precision
  (BTC 8, CLF 4, OMR 3).
- get_income_statement validated account_ids against all visible
  accounts, but totals_for excludes hidden, excluded-from-reports and
  tax-advantaged accounts, so those ids produced silent zeros. Ids now
  validate against income_statement.eligible_accounts and the soft
  failure explains eligibility.
- get_recurring_transactions computed totals from the displayed rows,
  so past the 200-row cap the value labeled a total was partial. Totals
  now aggregate over the full filtered scope in SQL, and the response
  carries total_results and a truncated flag. The upcoming_within_days
  window also starts at today, matching its documentation; overdue
  items appear in unwindowed calls.
- get_valuations silently dropped a malformed date filter and presented
  unfiltered data as filtered; malformed dates now return invalid_date.
- get_balance_sheet returned a generic failure for a reversed custom
  range because Period's own validation raises past the Date::Error
  rescue; it now returns the structured invalid_date error.
- get_insights documents that its family-wide scope matches the web
  feed exactly (InsightsController serves Current.family.insights to
  every member), so the tool exposes nothing the /insights page does
  not already show the same user.
- Tests: limit clamp proven against more insights than the cap,
  Setting fallbacks stubbed in the provider budget tests, currency
  precision and reversed-range regression tests added.

* refactor(assistant): apply reviewer nitpicks

- order declares type alongside its enum, matching sort_by
- page-size clamp deduplicated into the base class (MAX_PAGE_SIZE +
  shared resolved_page_size); dead per-tool copies removed
- get_accounts preloads balance rows only when the series is requested
- get_income_statement validates the bucket count before running any
  aggregation work

Deliberately unchanged: the balance sheet's monthly_history key. The
default response shape stays byte-compatible for existing MCP
consumers, and the nested series already states its interval.

* fix(assistant): second-round review findings on get_valuations

- A reversed date range (start after end) now returns the structured
  invalid_date error instead of presenting an empty result as filtered
  data, matching get_balance_sheet's handling.
- Page numbers are normalized before pagination: Pagy raises on zero,
  negative or non-numeric pages. The fix lands as a shared
  resolved_page helper on the base class and applies to every
  paginated tool (categories, tags, merchants, transactions, holdings,
  valuations), since all shared the same page-or-1 pattern; schemas
  declare minimum: 1.

* fix(assistant): round series amounts as BigDecimal before Float conversion

Converting to Float first can perturb the value at the requested
precision; round the exact decimal, then convert for JSON.

* fix(assistant): address maintainer review findings

- get_accounts no longer fails the whole listing when one account's
  start date lies beyond the requested period (start_date derives from
  the first entry, which can be future-dated); that account simply has
  no series. The unrescued Period.custom was reachable exactly there.
- The balances preload is gone: the series goes through
  Balance::ChartSeriesBuilder, which runs its own query keyed by
  account ids, so the eager-loaded rows were loaded and discarded.
- Provider::Openai#context_window now delegates to
  Assistant::TokenBudget, removing the duplicated ENV > Setting >
  default precedence so prompt assembly and the provider can never
  disagree about the window.

* fix(ai): final no-tools round uses tool_choice none instead of dropping tools

Anthropic rejects requests whose messages contain tool_use blocks when
no tools are defined, so re-requesting with an empty tool list made the
final-round grace die in a provider 400 on Anthropic models. The final
round now sends the real tool definitions with tool_choice none, which
both providers accept, and the model answers in prose as intended.

* fix(assistant): scope every income statement read to the requesting user

get_income_statement validated account_ids against the user-scoped statement
but computed every total from an unscoped one. IncomeStatement falls back to
Current.user, which is nil in the assistant job and the MCP endpoint, so the
unscoped reads dropped the included_in_finances_for filter and reported
family-wide totals next to ids that had been checked against a narrower set.

Route all reads through one memoized user-scoped statement, the idiom
get_balance_sheet already uses. Also lets the per-instance memoization in
IncomeStatement apply across the eligibility check and the totals.

Adds a regression test that fails without the change, plus a companion test
asserting eligibility and totals agree on scope. Guard the strictness walk
against an empty registry so it cannot silently assert nothing.
2026-08-21 21:58:47 +02:00

67 lines
2.1 KiB
Ruby

class Assistant::FunctionToolCaller
Error = Class.new(StandardError)
FunctionExecutionError = Class.new(Error)
attr_reader :functions
def initialize(functions = [])
@functions = functions
end
def fulfill_requests(function_requests)
function_requests.map do |function_request|
result = execute(function_request)
ToolCall::Function.from_function_request(function_request, result)
end
end
def function_definitions
functions.map(&:to_definition)
end
private
# Tool failures come back as data instead of raising, so one bad call no
# longer aborts the whole turn. The hint steers the model toward a single
# corrected retry (the system prompt pairs it with a retry-once rule).
def execute(function_request)
fn = find_function(function_request)
if fn.nil?
return {
error: "Unknown tool: #{function_request.function_name}",
hint: "Only call tools from the provided list."
}
end
fn_args = JSON.parse(function_request.function_args.presence || "{}")
fn.call(fn_args)
rescue JSON::ParserError
{
error: "Arguments were not valid JSON",
hint: "Re-send #{function_request.function_name} with valid JSON arguments."
}
rescue ActiveRecord::RecordNotFound => e
{
error: e.message,
hint: "That record was not found. List valid options first (for example get_accounts or get_categories) and retry once with an exact match."
}
rescue Date::Error, ArgumentError, KeyError => e
{
error: e.message,
hint: "Check argument formats (dates are YYYY-MM-DD) and retry once with corrected arguments."
}
rescue => e
Rails.logger.error("Assistant tool #{fn.name} failed: #{e.class}: #{e.message}")
{
error: "#{fn.name} failed unexpectedly",
hint: "Do not retry with the same arguments. Answer with the data you already have and note the gap."
}
end
def find_function(function_request)
functions.find { |f| f.name == function_request.function_name }
end
end