mirror of
https://github.com/we-promise/sure.git
synced 2026-09-05 06:41:08 +00:00
Juan José MataandClaude
f78303ebbe
* feat(ai-health): name the missing function calling behind an opaque chat error The assistant reads accounts, transactions and holdings through function calls, so every chat request carries a `tools` payload. A model without function-calling support rejects it — OpenRouter answers a bare 404 — and the operator sees only that status code, with nothing pointing at the model. Both earlier attempts at this guessed from the chat-time error; the AI status page already runs live probes, so let it answer the question directly instead. `AiHealth::Probe#function_calling` asks the configured model for one trivial tool call the way the assistant asks for its own: chat completions with `tools` for OpenAI-compatible endpoints, the Responses API for hosted OpenAI, and `messages.create` with `tools` for Anthropic, carrying the same strict schema `Provider::Openai` sends. Reading it against the plain LLM probe is what makes the verdict sound rather than a guess at 404s: plain chat passing while the same request with tools fails means the model has no function calling; a response that carries no tool call means the endpoint took the tools but the model ignored them; both failing, or a timeout, stays an ordinary probe failure. The AI status card gains a Function calling (tools) row, an alert naming the fix for each of the two bad outcomes, and a failure reason. The hosting settings model field now says the assistant needs a tools-capable model and links super admins to the check. Refs #830 Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DWCmRQ1ry26JnhA79s1ZKH * fix(ai-health): only call a refusal a refusal, and probe the route chat takes Two findings from review of the function-calling probe. A tools request can fail for reasons that say nothing about tool support: a 429, a 500, a dropped connection, an unreadable body. Reading any non-timeout failure as `:unsupported` sent the operator hunting for a new model over a transient blip. Only a 4xx the service chose to answer with — excluding the ones that mean "not now" or "not you" — is a refusal of the tools payload; everything else stays an ordinary probe failure. The bare 404 from OpenRouter that this page exists to explain still reads as missing function calling. `Provider::Openai#supports_responses_endpoint?` is the real routing decision and `OPENAI_SUPPORTS_RESPONSES_ENDPOINT` can flip it either way, so choosing the API from "is the endpoint custom" could probe Chat Completions while chat uses Responses, or the reverse — reporting on a path the assistant never takes. Ask the provider instead, and cache the two routes under separate keys. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DWCmRQ1ry26JnhA79s1ZKH * fix(ai-health): confirm a tools refusal before blaming the model A client error on the tools request can mean "your tools payload" or "your request, tools or not" — an invalid schema, a route the endpoint does not serve, a model it will not run. Splitting those on the status code alone still put a 422 from an endpoint contract on the model's account and told the operator to go find another one. The probe now confirms it: when the tools request comes back a client error, it asks again with the tools taken off. Only if that lands is the tools payload what was turned down, and the probe says so with its own failure code — provider-agnostic, and no reading of error text for the word "tool", which would only ever fit the provider it was written against. Statuses that mean "not now" or "not you" (401, 402, 403, 408, 429) never get a second ask. `AiHealth` now just reports the probe's verdict instead of inferring one from the status. The troubleshooting fix no longer points at an OpenRouter free tier: those providers commonly log prompts and completions for training, and every assistant tool call carries accounts, transactions, and holdings. It points at the model recommendations already in this doc, and says why free tiers are the wrong place to look. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DWCmRQ1ry26JnhA79s1ZKH --------- Co-authored-by: Claude <noreply@anthropic.com>
