Add function-calling probe to detect tool-use support (#3255)

* feat(ai-health): name the missing function calling behind an opaque chat error

The assistant reads accounts, transactions and holdings through function
calls, so every chat request carries a `tools` payload. A model without
function-calling support rejects it — OpenRouter answers a bare 404 — and
the operator sees only that status code, with nothing pointing at the
model. Both earlier attempts at this guessed from the chat-time error;
the AI status page already runs live probes, so let it answer the
question directly instead.

`AiHealth::Probe#function_calling` asks the configured model for one
trivial tool call the way the assistant asks for its own: chat
completions with `tools` for OpenAI-compatible endpoints, the Responses
API for hosted OpenAI, and `messages.create` with `tools` for Anthropic,
carrying the same strict schema `Provider::Openai` sends. Reading it
against the plain LLM probe is what makes the verdict sound rather than
a guess at 404s: plain chat passing while the same request with tools
fails means the model has no function calling; a response that carries
no tool call means the endpoint took the tools but the model ignored
them; both failing, or a timeout, stays an ordinary probe failure.

The AI status card gains a Function calling (tools) row, an alert
naming the fix for each of the two bad outcomes, and a failure reason.
The hosting settings model field now says the assistant needs a
tools-capable model and links super admins to the check.

Refs #830

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWCmRQ1ry26JnhA79s1ZKH

* fix(ai-health): only call a refusal a refusal, and probe the route chat takes

Two findings from review of the function-calling probe.

A tools request can fail for reasons that say nothing about tool support:
a 429, a 500, a dropped connection, an unreadable body. Reading any
non-timeout failure as `:unsupported` sent the operator hunting for a new
model over a transient blip. Only a 4xx the service chose to answer with
— excluding the ones that mean "not now" or "not you" — is a refusal of
the tools payload; everything else stays an ordinary probe failure. The
bare 404 from OpenRouter that this page exists to explain still reads as
missing function calling.

`Provider::Openai#supports_responses_endpoint?` is the real routing
decision and `OPENAI_SUPPORTS_RESPONSES_ENDPOINT` can flip it either way,
so choosing the API from "is the endpoint custom" could probe Chat
Completions while chat uses Responses, or the reverse — reporting on a
path the assistant never takes. Ask the provider instead, and cache the
two routes under separate keys.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWCmRQ1ry26JnhA79s1ZKH

* fix(ai-health): confirm a tools refusal before blaming the model

A client error on the tools request can mean "your tools payload" or "your
request, tools or not" — an invalid schema, a route the endpoint does not
serve, a model it will not run. Splitting those on the status code alone
still put a 422 from an endpoint contract on the model's account and told
the operator to go find another one.

The probe now confirms it: when the tools request comes back a client
error, it asks again with the tools taken off. Only if that lands is the
tools payload what was turned down, and the probe says so with its own
failure code — provider-agnostic, and no reading of error text for the
word "tool", which would only ever fit the provider it was written
against. Statuses that mean "not now" or "not you" (401, 402, 403, 408,
429) never get a second ask. `AiHealth` now just reports the probe's
verdict instead of inferring one from the status.

The troubleshooting fix no longer points at an OpenRouter free tier:
those providers commonly log prompts and completions for training, and
every assistant tool call carries accounts, transactions, and holdings.
It points at the model recommendations already in this doc, and says why
free tiers are the wrong place to look.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DWCmRQ1ry26JnhA79s1ZKH

---------

Co-authored-by: Claude <noreply@anthropic.com>

test/controllers/admin/system_health_controller_test.rb

@@ -3,7 +3,8 @@ require "test_helper"
 class Admin::SystemHealthControllerTest < ActionDispatch::IntegrationTest
   AI_ENVIRONMENT = %w[
     OPENAI_ACCESS_TOKEN OPENAI_URI_BASE OPENAI_MODEL OPENAI_REQUEST_TIMEOUT
-    OPENAI_SUPPORTS_PDF_PROCESSING ANTHROPIC_ACCESS_TOKEN ANTHROPIC_API_KEY
+    OPENAI_SUPPORTS_PDF_PROCESSING OPENAI_SUPPORTS_RESPONSES_ENDPOINT
+    ANTHROPIC_ACCESS_TOKEN ANTHROPIC_API_KEY
     ANTHROPIC_BASE_URL ANTHROPIC_MODEL ANTHROPIC_REQUEST_TIMEOUT
     VECTOR_STORE_PROVIDER EMBEDDING_URI_BASE EMBEDDING_MODEL
     EMBEDDING_DIMENSIONS EMBEDDING_ACCESS_TOKEN QDRANT_URL QDRANT_API_KEY
@@ -19,6 +20,7 @@ class Admin::SystemHealthControllerTest < ActionDispatch::IntegrationTest
     Setting.stubs(:anthropic_base_url).returns(nil)
     Setting.stubs(:anthropic_model).returns(nil)
     AiHealth::Probe.any_instance.stubs(:llm).returns(probe_result(:passing))
+    AiHealth::Probe.any_instance.stubs(:function_calling).returns(probe_result(:passing))
     AiHealth::Probe.any_instance.stubs(:pdf_text_extraction).returns(probe_result(:passing))
     AiHealth::Probe.any_instance.stubs(:pdf_vision_processing).returns(probe_result(:passing))
     AiHealth::Probe.any_instance.stubs(:openai_vector_store).returns(probe_result(:passing))
@@ -109,6 +111,7 @@ class Admin::SystemHealthControllerTest < ActionDispatch::IntegrationTest
     sign_in users(:sure_support_staff)
     stub_healthy_sidekiq
     AiHealth::Probe.any_instance.expects(:llm).never
+    AiHealth::Probe.any_instance.expects(:function_calling).never
     AiHealth::Probe.any_instance.expects(:pdf_text_extraction).never
     AiHealth::Probe.any_instance.expects(:pdf_vision_processing).never
     AiHealth::Probe.any_instance.expects(:openai_vector_store).never
@@ -156,6 +159,45 @@ class Admin::SystemHealthControllerTest < ActionDispatch::IntegrationTest
     assert_no_match(/local-token|uri-secret|query-secret/, response.body)
   end
 
+  test "AI status names the missing function-calling support behind an unhelpful chat error" do
+    sign_in users(:sure_support_staff)
+    stub_healthy_sidekiq
+    AiHealth::Probe.any_instance.stubs(:function_calling).returns(
+      probe_result(:failing, failure_code: :tools_refused, http_status: 404)
+    )
+
+    with_ai_environment(
+      "OPENAI_ACCESS_TOKEN" => "router-secret",
+      "OPENAI_URI_BASE" => "https://openrouter.ai/api/v1",
+      "OPENAI_MODEL" => "tngtech/deepseek-r1t2-chimera:free"
+    ) do
+      get admin_system_health_url(tab: "ai")
+    end
+
+    assert_response :success
+    assert_select "[data-testid='function-calling-status']", text: /Not supported by the effective provider/
+    assert_match(/The model does not support function calling/, response.body)
+    assert_match(/Function-calling failure reason/, response.body)
+    assert_no_match(/router-secret/, response.body)
+  end
+
+  test "AI status separates a model that ignores tools from one that cannot use them" do
+    sign_in users(:sure_support_staff)
+    stub_healthy_sidekiq
+    AiHealth::Probe.any_instance.stubs(:function_calling).returns(
+      probe_result(:failing, failure_code: :no_tool_call)
+    )
+
+    with_ai_environment("OPENAI_ACCESS_TOKEN" => "sk-secret-openai") do
+      get admin_system_health_url(tab: "ai")
+    end
+
+    assert_response :success
+    assert_select "[data-testid='function-calling-status']", text: /Tools accepted, but the model called none/
+    assert_match(/answered without calling the tool it was asked to call/, response.body)
+    assert_no_match(/The model does not support function calling/, response.body)
+  end
+
   test "AI status reports text and vision PDF probes separately" do
     sign_in users(:sure_support_staff)
     stub_healthy_sidekiq