docs(chat): size AI_RESPONSE_TIMEOUT for chained tool calls

The guidance assumed a tool-using turn costs two model calls. #2767 landed
after this branch was opened and made tool calls iterative: `Assistant::Responder`
now loops until `iteration > max_tool_call_iterations`, so a turn runs to
1 + ASSISTANT_MAX_TOOL_CALL_ITERATIONS calls — six by default — with tool
execution in between. At the default 60s per-call timeout that is up to 360s of
model time against a 90s watchdog.

Streaming does not rescue this either. `emit(:output_text)` only fires for a
response that carries text, and tool-call-only rounds carry none, so the bubble
stays on "Thinking…" through every round regardless of provider.

Documents ASSISTANT_MAX_TOOL_CALL_ITERATIONS as the cheaper lever: dropping it to
2 halves the worst case instead of demanding a half-hour timeout, at the cost of
failing long tool chains earlier with a clear limit error. compose.example.ai.yml
now shows that combination rather than a timeout sized for six calls it never had.

.env.example

@@ -41,11 +41,17 @@ OPENAI_URI_BASE=
 #
 # Keep this ABOVE OPENAI_REQUEST_TIMEOUT. This clock starts when the message is
 # queued and covers the whole turn, while OPENAI_REQUEST_TIMEOUT applies to each
-# HTTP call separately — a tool-using turn makes two of them. If this is the
-# lower of the two you get a generic "no response" instead of the specific
-# timeout error, and the job keeps burning tokens after the chat gave up.
+# HTTP call separately. A turn that chains tool calls can make up to
+# 1 + ASSISTANT_MAX_TOOL_CALL_ITERATIONS calls (6 by default) with tool execution
+# in between, so the worst case is several multiples of OPENAI_REQUEST_TIMEOUT.
+# If this is the lower of the two you get a generic "no response" instead of the
+# specific timeout error, and the job keeps burning tokens after the chat gave up.
 # Minimum 30; also settable on the Self-Hosting settings page.
 # AI_RESPONSE_TIMEOUT=90
+#
+# Lowering the tool-call cap is often the better lever on slow hardware: it
+# shrinks the worst case directly instead of requiring a very long timeout.
+# ASSISTANT_MAX_TOOL_CALL_ITERATIONS=5
 
 # Optional: OpenAI-compatible capability flags
 # OPENAI_REQUEST_TIMEOUT=60                  # HTTP timeout in seconds; raise for slow local models

.env.local.example

@@ -46,7 +46,8 @@ OPENAI_MODEL =
 
 # OpenAI-compatible capability flags (custom/self-hosted providers)
 # OPENAI_REQUEST_TIMEOUT = 60                # HTTP timeout in seconds; raise for slow local models
-# AI_RESPONSE_TIMEOUT = 90                   # Seconds the chat waits before "no response"; keep ABOVE OPENAI_REQUEST_TIMEOUT
+# AI_RESPONSE_TIMEOUT = 90                   # Whole-turn budget before "no response"; keep ABOVE OPENAI_REQUEST_TIMEOUT
+# ASSISTANT_MAX_TOOL_CALL_ITERATIONS = 5     # Chained tool calls per turn; a turn costs up to (1 + this) model calls
 # OPENAI_SUPPORTS_PDF_PROCESSING = true      # Set to false for endpoints without vision support
 # OPENAI_SUPPORTS_RESPONSES_ENDPOINT =       # true to force Responses API on custom providers
 # LLM_JSON_MODE =                            # auto | strict | json_object | none

compose.example.ai.yml

@@ -94,12 +94,17 @@ x-rails-env: &rails_env
   OPENAI_MODEL: llama3.1:8b # Note: Use tool-enabled model
   OPENAI_URI_BASE: http://ollama:11434/v1
   # Local models are not streamed, so the chat shows nothing until the whole reply
-  # is generated. Raise both together on slower hardware, keeping AI_RESPONSE_TIMEOUT
-  # the larger of the two — it covers the whole turn (queue time plus, for a
-  # tool-using turn, two OPENAI_REQUEST_TIMEOUT-bounded calls and the tool run
-  # between them), whereas OPENAI_REQUEST_TIMEOUT bounds each call on its own.
+  # is generated. OPENAI_REQUEST_TIMEOUT bounds each call to the model on its own,
+  # while AI_RESPONSE_TIMEOUT covers the whole turn — queue time plus every call a
+  # chained-tool turn makes, up to 1 + ASSISTANT_MAX_TOOL_CALL_ITERATIONS of them
+  # with tool execution in between — so keep it the larger of the two.
+  #
+  # The cap is lowered to 2 here rather than left at 5, so the worst case is three
+  # model calls instead of six. On hardware this slow that is a better trade than a
+  # half-hour watchdog, at the cost of failing longer tool chains earlier.
   OPENAI_REQUEST_TIMEOUT: ${OPENAI_REQUEST_TIMEOUT:-300}
-  AI_RESPONSE_TIMEOUT: ${AI_RESPONSE_TIMEOUT:-660}
+  ASSISTANT_MAX_TOOL_CALL_ITERATIONS: ${ASSISTANT_MAX_TOOL_CALL_ITERATIONS:-2}
+  AI_RESPONSE_TIMEOUT: ${AI_RESPONSE_TIMEOUT:-1000}
   # Vector store — pgvector keeps all data local (requires pgvector/pgvector Docker image for db)
   VECTOR_STORE_PROVIDER: pgvector
   EMBEDDING_MODEL: nomic-embed-text

config/locales/views/settings/hostings/en.yml

@@ -142,9 +142,9 @@ en:
         max_items_per_call_label: Max Items Per Batch (Optional)
         max_items_per_call_help: "Upper bound for auto-categorize / merchant detection batches. Default: 25. Larger batches are auto-sliced to fit the context window."
         ai_response_timeout_heading: Chat Response Timeout
-        ai_response_timeout_description: How long the chat waits for the assistant before showing a "no response" error. Raise this if you run a local model on slow hardware — responses from OpenAI-compatible providers are not streamed, so nothing appears until the whole reply is generated.
+        ai_response_timeout_description: How long the chat waits for the assistant before showing a "no response" error. Raise this if you run a local model on slow hardware — responses from OpenAI-compatible providers are not streamed, and tool-call rounds display nothing, so the chat can sit on "Thinking…" for the whole turn.
         ai_response_timeout_label: Response Timeout in Seconds (Optional)
-        ai_response_timeout_help: "Default: 90. Raise to 300+ for slow local models. Keep it above OPENAI_REQUEST_TIMEOUT — this covers the whole turn including queue time, while that limit applies to each call to the model separately."
+        ai_response_timeout_help: "Default: 90. Raise to 300+ for slow local models. Keep it above OPENAI_REQUEST_TIMEOUT — this covers the whole turn, including queue time and every call a chained-tool turn makes, while that limit applies to each call separately. Lowering ASSISTANT_MAX_TOOL_CALL_ITERATIONS reduces how large this needs to be."
         title: OpenAI
       yahoo_finance_settings:
         title: Yahoo Finance

docs/hosting/ai.md

@@ -223,11 +223,16 @@ LLM_CONTEXT_WINDOW=8192
 # Slow local models often need a longer HTTP timeout once the prompt budget issue is fixed.
 OPENAI_REQUEST_TIMEOUT=180
 
-# How long the chat waits before giving up and showing a "no response" error.
-# Responses from custom providers are not streamed, so nothing appears until the
-# whole reply is generated — this must cover the full generation, not just the
-# first token. Keep it above OPENAI_REQUEST_TIMEOUT (see below).
-AI_RESPONSE_TIMEOUT=400
+# Chained tool calls per turn. Each iteration is another call to the model, so
+# lowering this is the cheapest way to keep a turn inside the timeout below.
+ASSISTANT_MAX_TOOL_CALL_ITERATIONS=2
+
+# Whole-turn budget before the chat gives up and shows a "no response" error.
+# Responses from custom providers are not streamed and tool-call rounds display
+# nothing, so this must cover every call the turn makes — up to
+# 1 + ASSISTANT_MAX_TOOL_CALL_ITERATIONS of them — not just the first token.
+# Keep it above OPENAI_REQUEST_TIMEOUT (see below).
+AI_RESPONSE_TIMEOUT=600
 
 # Optional: enable debug logging in the AI chat
 AI_DEBUG_MODE=true 
@@ -239,7 +244,7 @@ AI_DEBUG_MODE=true
 - If you don't set a model, chats will fail with a validation error
 - Auto-categorization uses a conservative default `LLM_CONTEXT_WINDOW=2048`, so large category lists or schemas can exhaust the prompt budget before any transactions are sent
 - If requests start timing out after raising `LLM_CONTEXT_WINDOW`, increase `OPENAI_REQUEST_TIMEOUT` too; these are separate limits
-- Responses from custom providers are **not streamed** — the chat shows "Thinking…" until the entire reply is generated. If the chat errors while your model is clearly still working, raise `AI_RESPONSE_TIMEOUT`; `OPENAI_REQUEST_TIMEOUT` alone will not help. Keep `AI_RESPONSE_TIMEOUT` the larger of the two — it covers the whole turn, while `OPENAI_REQUEST_TIMEOUT` bounds each call to the model separately
+- Responses from custom providers are **not streamed** — the chat shows "Thinking…" until the entire reply is generated, and a turn that chains tool calls stays there through every round, since tool-call responses have no text to display. If the chat errors while your model is clearly still working, raise `AI_RESPONSE_TIMEOUT` or lower `ASSISTANT_MAX_TOOL_CALL_ITERATIONS`; `OPENAI_REQUEST_TIMEOUT` alone will not help. Keep `AI_RESPONSE_TIMEOUT` the largest of the three — it covers the whole turn, while `OPENAI_REQUEST_TIMEOUT` bounds each call to the model separately
 
 ### Docker Compose Example
 
@@ -1105,20 +1110,28 @@ Then restart both `web` and `worker` so the new env var is loaded. If you are us
 
 **Symptom:** The chat shows "Thinking…" for a while, then an error saying the assistant is not available — but the model does produce a reply and LLM Usage shows tokens were generated.
 
-**Cause:** Two separate limits, measured over different spans:
+**Cause:** Three settings interact here, measured over different spans:
 
 - `OPENAI_REQUEST_TIMEOUT` (default `60`) — applies to **each HTTP call** to the model, on its own.
+- `ASSISTANT_MAX_TOOL_CALL_ITERATIONS` (default `5`) — how many chained tool calls one turn may make. A turn costs up to `1 + this` model calls.
 - `AI_RESPONSE_TIMEOUT` (default `90`) — covers the **whole turn**, and its clock starts when the message is queued, so Sidekiq queue time counts against it.
 
-Responses from custom OpenAI-compatible providers are **not streamed**, so nothing appears in the chat until the entire reply is generated — the budget has to cover the full generation, not just the time to the first token. A tool-using turn spends it twice over: one call to decide the tool, the tool execution, then a second call to write the answer.
+Responses from custom OpenAI-compatible providers are **not streamed**, so nothing appears in the chat until the entire reply is generated. Worse, the assistant only shows text once a response actually contains some — a tool-call-only response produces nothing to display — so a turn that chains several tool calls sits on "Thinking…" through all of them. At the defaults the worst case is six sequential model calls plus five tool executions.
 
-**Fix:** Raise both, keeping `AI_RESPONSE_TIMEOUT` the **larger** of the two. It has to fit two `OPENAI_REQUEST_TIMEOUT`-bounded calls plus tool execution and queue wait, and if it is the smaller one you get a generic "no response" instead of the specific timeout error — while the job keeps running and burning tokens after the chat has given up.
+**Fix:** you have two levers, and the cheaper one is usually the tool-call cap.
+
+Lowering the cap shrinks the worst case directly. With `ASSISTANT_MAX_TOOL_CALL_ITERATIONS=2` a turn costs at most three model calls instead of six, so the timeout you need is halved. The trade-off is that genuinely long tool chains fail earlier, with a clear "exceeded the tool-call limit" error rather than a timeout.
 
 ```bash
 OPENAI_REQUEST_TIMEOUT=300
-AI_RESPONSE_TIMEOUT=660
+ASSISTANT_MAX_TOOL_CALL_ITERATIONS=2
+AI_RESPONSE_TIMEOUT=1000
 ```
 
+If you would rather keep the full five iterations, size `AI_RESPONSE_TIMEOUT` for `6 × OPENAI_REQUEST_TIMEOUT` plus tool execution and queue wait instead.
+
+Keep `AI_RESPONSE_TIMEOUT` the **largest** of these in every case. If it is smaller than the per-call limit you get a generic "no response" instead of the specific timeout error, and the job keeps running and burning tokens after the chat has given up.
+
 `AI_RESPONSE_TIMEOUT` can also be set at **Settings → Self-Hosting → OpenAI → Chat Response Timeout**, which takes effect without a restart. The environment variable wins if both are set. The minimum accepted value is `30`.
 
 Restart `web` and `worker` after changing the environment variables, and make sure your Docker Compose file forwards them into the containers.