mirror of
https://github.com/we-promise/sure.git
synced 2026-09-02 05:11:05 +00:00
* fix(chat): make the assistant response timeout configurable (#2893)
Self-hosted users running a local model report the chat failing with
"assistant not available" after 90 seconds even though the model
generates a reply and tokens are billed.
Three timeouts are involved and only one was configurable:
- OPENAI_REQUEST_TIMEOUT (60s) — already settable, not the blocker.
- The browser watchdog in chat_controller.js (90s) — hardcoded, and
this is what actually fires.
- Chat::UNDELIVERED_RESPONSE_TIMEOUT (60s) — a constant, so raising
the client value alone would not have helped.
The watchdog cannot be avoided by streaming here: custom
OpenAI-compatible providers route through generic_chat_response, which
forces synchronous calls, so nothing renders until the whole generation
finishes. Time-to-last-token has to beat the deadline.
Adds AI_RESPONSE_TIMEOUT (ENV > Setting > 90s default, floored at 30s),
exposed on the Self-Hosting settings page and passed to the Stimulus
controller at all three mount points — show, index and the sidebar in
the application layout, each of which declares data-controller="chat"
independently.
The server floor is derived from the same value but kept 10s below it.
report_timeout answers 200 whether or not it acted and the client only
retries on a non-ok response, so a floor at or above the client value
would let clock skew strand a pending bubble permanently.
Also guards AssistantMessage#append_text!. The watchdog runs in the web
process while the job holds its own copy of the message, so a job
finishing after the bubble was destroyed or demoted would silently
resurrect it alongside the error the user was already shown.
* fix(chat): let the watchdog retry when report_timeout declines
`report_timeout` answered 200 whether or not `handle_undelivered_response!`
acted. The Stimulus watchdog only stops retrying a URL once it sees a 2xx, so
a declined report was treated as final.
That stranded the bubble whenever the client's clock ran more than
SERVER_TIMEOUT_GRACE ahead of the server's: the watchdog posts at its own
timeout, the server sees a message younger than its floor and no-ops, and
nothing ever retries. The bubble spins forever with no error and no Retry.
Answering 409 instead lets the next 5s tick try again, so any amount of skew
costs retries rather than a stuck chat. The grace window stays as an
optimisation to keep those retries rare, not as the correctness mechanism.
* docs(chat): correct the AI_RESPONSE_TIMEOUT ordering guidance
The docs, locale string and examples all said to keep OPENAI_REQUEST_TIMEOUT at
or above AI_RESPONSE_TIMEOUT. That is backwards.
The two limits span different things. OPENAI_REQUEST_TIMEOUT bounds each HTTP
call to the model on its own; AI_RESPONSE_TIMEOUT covers the whole turn and its
clock starts when the message is queued, so it also absorbs Sidekiq queue time
and, for a tool-using turn, two model calls plus the tool run between them.
Keeping the chat timeout the larger of the two means a slow model surfaces the
specific HTTP timeout error rather than a generic "no response", and the job
stops instead of running on after the chat has given up. The shipped 60/90
defaults already had this ordering; only the guidance was wrong.
compose.example.ai.yml gets 300/660 so the Ollama example can actually complete
a tool-using turn.
* fix(chat): claim the pending bubble atomically before appending
append_text! read the row's status and then saved, leaving a window in which
the watchdog could demote the row to `failed` between the two. The late
content would then land on a bubble the user had already been told failed,
flipping it back to `complete`.
Replaces the read with a conditional UPDATE that only succeeds while the row is
still pending, so the check and the state change cannot be separated.
Uses a conditional UPDATE rather than with_lock because append_text! is called
once per chunk on the streaming path; a row lock and transaction per chunk would
be far more expensive. The claim only runs on the first append, since later ones
are no longer pending.
* test(chat): isolate AI_RESPONSE_TIMEOUT from the environment
Chat.response_timeout reads ENV ahead of Setting, so stubbing only the Setting
left the assertions at the mercy of the environment they run in. With
AI_RESPONSE_TIMEOUT=45 exported, five of these tests failed — the default,
floor and grace assertions were all silently measuring the env value.
Adds a with_setting_timeout helper that stubs the Setting and clears the
variable together, and switches the controller tests to stub
Chat.undelivered_response_timeout directly, since what they care about is the
resolved floor rather than how it was configured.
Both files now pass with or without AI_RESPONSE_TIMEOUT set.
* docs(chat): size AI_RESPONSE_TIMEOUT for chained tool calls
The guidance assumed a tool-using turn costs two model calls. #2767 landed
after this branch was opened and made tool calls iterative: `Assistant::Responder`
now loops until `iteration > max_tool_call_iterations`, so a turn runs to
1 + ASSISTANT_MAX_TOOL_CALL_ITERATIONS calls — six by default — with tool
execution in between. At the default 60s per-call timeout that is up to 360s of
model time against a 90s watchdog.
Streaming does not rescue this either. `emit(:output_text)` only fires for a
response that carries text, and tool-call-only rounds carry none, so the bubble
stays on "Thinking…" through every round regardless of provider.
Documents ASSISTANT_MAX_TOOL_CALL_ITERATIONS as the cheaper lever: dropping it to
2 halves the worst case instead of demanding a half-hour timeout, at the cost of
failing long tool chains earlier with a clear limit error. compose.example.ai.yml
now shows that combination rather than a timeout sized for six calls it never had.
* docs(chat): state the whole-turn timeout as a sum, not a maximum
The guidance said to keep AI_RESPONSE_TIMEOUT "above" or "the larger of"
OPENAI_REQUEST_TIMEOUT. That understates it: the watchdog covers the entire turn,
so the bound is
(1 + ASSISTANT_MAX_TOOL_CALL_ITERATIONS) * OPENAI_REQUEST_TIMEOUT
+ tool execution + queue wait
Merely exceeding the per-call limit can still leave the chat reporting failure
while the worker keeps going.
One phrasing was outright wrong: "keep AI_RESPONSE_TIMEOUT the largest of the
three" compared a duration against ASSISTANT_MAX_TOOL_CALL_ITERATIONS, which is a
count, not seconds.
Resizes the examples against the formula — compose.example.ai.yml 1000 -> 1200 and
the Ollama doc example 600 -> 720, both now showing the arithmetic — and states
plainly that the 90s default is sized for typical cloud latency rather than the
worst-case bound, with the formula being what matters once per-call latency
approaches the timeout.
* docs(chat): list the AI settings fields and tag the formula fence
The Settings UI walkthrough listed three of the eight fields on the AI Provider
form. JSON Mode, the three Token Budget fields and the new Chat Response Timeout
were all missing, so the timeout was only discoverable from the troubleshooting
section. Rewrites the list to follow the form's own grouping and uses the labels
the form actually renders.
Also tags the whole-turn formula fence as `text` (markdownlint MD040).
* fix(compose): forward ASSISTANT_MAX_TOOL_CALL_ITERATIONS in standard compose
This file enumerates container environment explicitly — there is no env_file — so
a variable absent from the x-rails-env anchor never reaches web or worker.
The tool-call cap was only named in a comment here, while the docs added in
3360dbf5 tell operators to lower it to keep a turn inside AI_RESPONSE_TIMEOUT.
Following that advice on this compose file silently changed nothing: the app kept
the default of 5 while the timeout was sized for 3 calls, which lands back on the
"no response" error this branch exists to fix.
Left with an empty default so the app's own default governs, matching
OPENAI_MODEL and LLM_CONTEXT_WINDOW above. compose.example.ai.yml already
forwarded it.
261 lines
9.2 KiB
Ruby
261 lines
9.2 KiB
Ruby
require "test_helper"
|
|
|
|
class ChatTest < ActiveSupport::TestCase
|
|
setup do
|
|
@user = users(:family_admin)
|
|
@assistant = mock
|
|
end
|
|
|
|
test "user sees all messages in debug mode" do
|
|
chat = chats(:one)
|
|
with_env_overrides AI_DEBUG_MODE: "true" do
|
|
assert_equal chat.messages.count, chat.conversation_messages.count
|
|
end
|
|
end
|
|
|
|
test "user sees assistant and user messages in normal mode" do
|
|
chat = chats(:one)
|
|
assert_equal 3, chat.conversation_messages.count
|
|
end
|
|
|
|
test "uses chat-scoped stream targets" do
|
|
first_chat = chats(:one)
|
|
second_chat = chats(:two)
|
|
|
|
assert_not_equal "messages", first_chat.messages_target
|
|
assert_not_equal "chat-error", first_chat.error_target
|
|
assert_not_equal first_chat.messages_target, second_chat.messages_target
|
|
assert_not_equal first_chat.error_target, second_chat.error_target
|
|
end
|
|
|
|
test "creates with initial message" do
|
|
prompt = "Test prompt"
|
|
|
|
assert_difference "@user.chats.count", 1 do
|
|
chat = @user.chats.start!(prompt, model: "gpt-4.1")
|
|
|
|
assert_equal 2, chat.messages.count
|
|
assert_equal 1, chat.messages.where(type: "UserMessage").count
|
|
assert_equal 1, chat.messages.where(type: "AssistantMessage", status: "pending").count
|
|
end
|
|
end
|
|
|
|
test "creates with default model when model is nil" do
|
|
prompt = "Test prompt"
|
|
|
|
assert_difference "@user.chats.count", 1 do
|
|
chat = @user.chats.start!(prompt, model: nil)
|
|
|
|
assert_equal 2, chat.messages.count
|
|
assert_equal Chat.default_model, chat.messages.find_by!(type: "UserMessage").ai_model
|
|
end
|
|
end
|
|
|
|
test "creates with default model when model is empty string" do
|
|
prompt = "Test prompt"
|
|
|
|
assert_difference "@user.chats.count", 1 do
|
|
chat = @user.chats.start!(prompt, model: "")
|
|
|
|
assert_equal 2, chat.messages.count
|
|
assert_equal Chat.default_model, chat.messages.find_by!(type: "UserMessage").ai_model
|
|
end
|
|
end
|
|
|
|
# These three tests assert routing (which provider's effective_model wins),
|
|
# not the constant value itself — the assertion side reads through
|
|
# Provider::*.effective_model so ENV overrides like ANTHROPIC_MODEL /
|
|
# OPENAI_MODEL don't make the tests flake.
|
|
test "default_model returns Anthropic's effective_model when LLM_PROVIDER=anthropic and Anthropic is configured" do
|
|
Provider::Anthropic.stubs(:configured?).returns(true)
|
|
Setting.stubs(:llm_provider).returns("anthropic")
|
|
|
|
assert_equal Provider::Anthropic.effective_model, Chat.default_model
|
|
end
|
|
|
|
test "default_model falls back to OpenAI's effective_model when Anthropic is preferred but unconfigured" do
|
|
Provider::Anthropic.stubs(:configured?).returns(false)
|
|
Provider::Openai.stubs(:configured?).returns(true)
|
|
Setting.stubs(:llm_provider).returns("anthropic")
|
|
|
|
assert_equal Provider::Openai.effective_model, Chat.default_model
|
|
end
|
|
|
|
test "default_model uses Anthropic's effective_model when OpenAI is unconfigured" do
|
|
Provider::Anthropic.stubs(:configured?).returns(true)
|
|
Provider::Openai.stubs(:configured?).returns(false)
|
|
Setting.stubs(:llm_provider).returns("openai")
|
|
|
|
assert_equal Provider::Anthropic.effective_model, Chat.default_model
|
|
end
|
|
|
|
test "creates with configured model when OPENAI_MODEL env is set" do
|
|
prompt = "Test prompt"
|
|
|
|
with_env_overrides OPENAI_MODEL: "custom-model" do
|
|
chat = @user.chats.start!(prompt, model: "")
|
|
|
|
assert_equal "custom-model", chat.messages.find_by!(type: "UserMessage").ai_model
|
|
end
|
|
end
|
|
|
|
test "returns nil presentable error message when no error is stored" do
|
|
chat = chats(:one)
|
|
|
|
chat.update!(error: nil)
|
|
|
|
assert_nil chat.presentable_error_message
|
|
end
|
|
|
|
test "surfaces a friendly rate limit error" do
|
|
chat = chats(:one)
|
|
|
|
chat.add_error(StandardError.new("OpenAI API error 429: rate limit exceeded"))
|
|
|
|
assert_equal I18n.t("chat.errors.rate_limited"), chat.presentable_error_message
|
|
assert_match "429", chat.technical_error_message
|
|
end
|
|
|
|
test "surfaces a friendly temporary provider error" do
|
|
chat = chats(:one)
|
|
|
|
chat.add_error(StandardError.new("OpenAI API error 503: service unavailable"))
|
|
|
|
assert_equal I18n.t("chat.errors.temporarily_unavailable"), chat.presentable_error_message
|
|
assert_match "503", chat.technical_error_message
|
|
end
|
|
|
|
test "surfaces a friendly auth configuration error" do
|
|
chat = chats(:one)
|
|
|
|
chat.add_error(StandardError.new("OpenAI API error: invalid api key"))
|
|
|
|
assert_equal I18n.t("chat.errors.misconfigured"), chat.presentable_error_message
|
|
assert_match "invalid api key", chat.technical_error_message
|
|
end
|
|
|
|
test "surfaces a friendly default error for unrecognized errors" do
|
|
chat = chats(:one)
|
|
|
|
chat.add_error(StandardError.new("something totally unknown happened"))
|
|
|
|
assert_equal I18n.t("chat.errors.default"), chat.presentable_error_message
|
|
end
|
|
|
|
test "falls back to a friendly message for legacy serialized errors" do
|
|
chat = chats(:one)
|
|
|
|
chat.update!(error: "OpenAI API error 429: rate limit exceeded".to_json)
|
|
|
|
assert_equal I18n.t("chat.errors.rate_limited"), chat.presentable_error_message
|
|
assert_equal "OpenAI API error 429: rate limit exceeded", chat.technical_error_message
|
|
end
|
|
|
|
test "handle_undelivered_response! clears a blank pending bubble and records an error + debug log" do
|
|
BackgroundJobHealth.stubs(:snapshot).returns({ healthy: false, workers: 0 })
|
|
BackgroundJobHealth.stubs(:summary).returns("workers=0")
|
|
|
|
chat = chats(:two)
|
|
pending = chat.messages.create!(type: "AssistantMessage", content: "", ai_model: "gpt-4.1", status: :pending, created_at: 5.minutes.ago)
|
|
|
|
assert_difference -> { DebugLogEntry.count } => 1, -> { Message.count } => -1 do
|
|
assert chat.handle_undelivered_response!(pending)
|
|
end
|
|
|
|
assert_not Message.exists?(pending.id)
|
|
assert_equal I18n.t("chat.errors.no_response"), chat.reload.presentable_error_message
|
|
end
|
|
|
|
test "handle_undelivered_response! demotes a partially-streamed pending bubble to failed" do
|
|
BackgroundJobHealth.stubs(:snapshot).returns({})
|
|
BackgroundJobHealth.stubs(:summary).returns("")
|
|
|
|
chat = chats(:two)
|
|
pending = chat.messages.create!(type: "AssistantMessage", content: "partial answer", ai_model: "gpt-4.1", status: :pending, created_at: 5.minutes.ago)
|
|
|
|
assert_no_difference -> { Message.count } do
|
|
assert chat.handle_undelivered_response!(pending)
|
|
end
|
|
|
|
assert_equal "failed", pending.reload.status
|
|
end
|
|
|
|
test "handle_undelivered_response! ignores a pending bubble younger than the server timeout" do
|
|
chat = chats(:two)
|
|
fresh = chat.messages.create!(type: "AssistantMessage", content: "", ai_model: "gpt-4.1", status: :pending, created_at: 5.seconds.ago)
|
|
|
|
assert_no_difference [ "DebugLogEntry.count", "Message.count" ] do
|
|
assert_not chat.handle_undelivered_response!(fresh)
|
|
end
|
|
|
|
assert fresh.reload.pending?
|
|
end
|
|
|
|
test "handle_undelivered_response! is a no-op for non-pending messages" do
|
|
chat = chats(:one)
|
|
complete = chat.messages.create!(type: "AssistantMessage", content: "done", ai_model: "gpt-4.1", status: :complete)
|
|
|
|
assert_no_difference [ "DebugLogEntry.count", "Message.count" ] do
|
|
assert_not chat.handle_undelivered_response!(complete)
|
|
end
|
|
end
|
|
|
|
# `Chat.response_timeout` reads ENV ahead of Setting, so every assertion about
|
|
# the Setting, the default or the floor has to clear AI_RESPONSE_TIMEOUT first —
|
|
# otherwise an environment that happens to define it silently decides the result.
|
|
def with_setting_timeout(value)
|
|
Setting.stubs(:ai_response_timeout).returns(value)
|
|
with_env_overrides("AI_RESPONSE_TIMEOUT" => nil) { yield }
|
|
end
|
|
|
|
test "response_timeout falls back to the default when unconfigured" do
|
|
with_setting_timeout(nil) do
|
|
assert_equal Chat::DEFAULT_RESPONSE_TIMEOUT, Chat.response_timeout
|
|
end
|
|
end
|
|
|
|
test "response_timeout prefers ENV over Setting" do
|
|
with_setting_timeout(120) do
|
|
assert_equal 120.seconds, Chat.response_timeout
|
|
end
|
|
|
|
Setting.stubs(:ai_response_timeout).returns(120)
|
|
with_env_overrides("AI_RESPONSE_TIMEOUT" => "300") do
|
|
assert_equal 300.seconds, Chat.response_timeout
|
|
end
|
|
end
|
|
|
|
test "response_timeout ignores non-positive values and enforces a floor" do
|
|
with_setting_timeout(0) do
|
|
assert_equal Chat::DEFAULT_RESPONSE_TIMEOUT, Chat.response_timeout
|
|
end
|
|
|
|
with_setting_timeout(5) do
|
|
assert_equal Chat::MIN_RESPONSE_TIMEOUT, Chat.response_timeout
|
|
end
|
|
end
|
|
|
|
# An early report is refused by the server, and `report_timeout` answers 409 so
|
|
# the watchdog retries. The grace window keeps those retries rare for a client
|
|
# whose clock is modestly ahead.
|
|
test "undelivered_response_timeout stays below the client timeout" do
|
|
with_setting_timeout(300) do
|
|
assert_equal 290.seconds, Chat.undelivered_response_timeout
|
|
assert Chat.undelivered_response_timeout < Chat.response_timeout
|
|
end
|
|
end
|
|
|
|
test "handle_undelivered_response! respects a raised timeout" do
|
|
chat = chats(:two)
|
|
pending = chat.messages.create!(type: "AssistantMessage", content: "", ai_model: "gpt-4.1", status: :pending, created_at: 5.minutes.ago)
|
|
|
|
with_setting_timeout(600) do
|
|
assert_no_difference [ "DebugLogEntry.count", "Message.count" ] do
|
|
assert_not chat.handle_undelivered_response!(pending)
|
|
end
|
|
end
|
|
|
|
assert pending.reload.pending?
|
|
end
|
|
end
|