* feat(ai-health): name the missing function calling behind an opaque chat error
The assistant reads accounts, transactions and holdings through function
calls, so every chat request carries a `tools` payload. A model without
function-calling support rejects it — OpenRouter answers a bare 404 — and
the operator sees only that status code, with nothing pointing at the
model. Both earlier attempts at this guessed from the chat-time error;
the AI status page already runs live probes, so let it answer the
question directly instead.
`AiHealth::Probe#function_calling` asks the configured model for one
trivial tool call the way the assistant asks for its own: chat
completions with `tools` for OpenAI-compatible endpoints, the Responses
API for hosted OpenAI, and `messages.create` with `tools` for Anthropic,
carrying the same strict schema `Provider::Openai` sends. Reading it
against the plain LLM probe is what makes the verdict sound rather than
a guess at 404s: plain chat passing while the same request with tools
fails means the model has no function calling; a response that carries
no tool call means the endpoint took the tools but the model ignored
them; both failing, or a timeout, stays an ordinary probe failure.
The AI status card gains a Function calling (tools) row, an alert
naming the fix for each of the two bad outcomes, and a failure reason.
The hosting settings model field now says the assistant needs a
tools-capable model and links super admins to the check.
Refs #830
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWCmRQ1ry26JnhA79s1ZKH
* fix(ai-health): only call a refusal a refusal, and probe the route chat takes
Two findings from review of the function-calling probe.
A tools request can fail for reasons that say nothing about tool support:
a 429, a 500, a dropped connection, an unreadable body. Reading any
non-timeout failure as `:unsupported` sent the operator hunting for a new
model over a transient blip. Only a 4xx the service chose to answer with
— excluding the ones that mean "not now" or "not you" — is a refusal of
the tools payload; everything else stays an ordinary probe failure. The
bare 404 from OpenRouter that this page exists to explain still reads as
missing function calling.
`Provider::Openai#supports_responses_endpoint?` is the real routing
decision and `OPENAI_SUPPORTS_RESPONSES_ENDPOINT` can flip it either way,
so choosing the API from "is the endpoint custom" could probe Chat
Completions while chat uses Responses, or the reverse — reporting on a
path the assistant never takes. Ask the provider instead, and cache the
two routes under separate keys.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWCmRQ1ry26JnhA79s1ZKH
* fix(ai-health): confirm a tools refusal before blaming the model
A client error on the tools request can mean "your tools payload" or "your
request, tools or not" — an invalid schema, a route the endpoint does not
serve, a model it will not run. Splitting those on the status code alone
still put a 422 from an endpoint contract on the model's account and told
the operator to go find another one.
The probe now confirms it: when the tools request comes back a client
error, it asks again with the tools taken off. Only if that lands is the
tools payload what was turned down, and the probe says so with its own
failure code — provider-agnostic, and no reading of error text for the
word "tool", which would only ever fit the provider it was written
against. Statuses that mean "not now" or "not you" (401, 402, 403, 408,
429) never get a second ask. `AiHealth` now just reports the probe's
verdict instead of inferring one from the status.
The troubleshooting fix no longer points at an OpenRouter free tier:
those providers commonly log prompts and completions for training, and
every assistant tool call carries accounts, transactions, and holdings.
It points at the model recommendations already in this doc, and says why
free tiers are the wrong place to look.
Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWCmRQ1ry26JnhA79s1ZKH
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Add synthetic PDF health checks
* Require exact marker in PDF health probes
* Report PDF health paths separately
* Refactor application code
* Remove unrelated schema dump changes
* Simplify synthetic PDF validation
---------
Signed-off-by: Juan José Mata <juanjo.mata@gmail.com>
* Add live AI checks to system health
Give super admins a dedicated AI status view with bounded liveness probes for LLMs, vector stores, pgvector, and embedding endpoints. Record sanitized failures in both the system debug log and Rails logger, and document the recommended local configuration.\n\nCloses #3145
* Fix AI health CI checks
* Address AI health review feedback
* Correct Ollama model preload guidance
* Distinguish OpenAI-compatible providers
* Make Ollama startup readiness explicit
* Recognize Cloudflare AI endpoints
