Repository navigation
bug(cloud): Qwen3.8-27B generation returned HTTP 500 during bounded workshop evaluation #800
Description
During the ODSC request-routing workshop paired-answer collection, production SIE returned HTTP500 for one Qwen/Qwen3.8-27B-FP8 generation request. The caller used the public SDK default Cloud endpoint, four concurrent requests, and disabled automatic retries. The collector stopped after the failure and drained/retained its in-flight results.
Observed on2026-10-18, immediately before the collector finished at 2026-10-18T08:13:09.884650+00:00. SDK0.10.0 from public commit d680237f02c2b67fd9afd91f30b73a29cb18fae0. Retained error: ServerError, HTTP500 after4.193 seconds. No settled credit receipt was retained for this failed request; execution and billing status are therefore unknown. The collector retained only sanitized status metadata, so a backend request ID is unavailable.
Request parameters: temperature=0, max_tokens=3072, wait_for_capacity=False, max_oom_retries=0, provision_timeout_s=180, read_timeout_s=180. Model is the bare default profile, not the high-throughput batch variant.
System message:
Solve the mathematical problem carefully. Use at most 300 words of concise calculations, then put the final mathematical value in \boxed{...}. Avoid introductions, repetition, alternative approaches, and restating the problem. If uncertain, give your best final answer. Do not use a multiple-choice letter.
User message (public Easy2Hard-AMC eval row496, source revision55bc0d2fb10954151e669d2026b87fa896f2fa26):
Let $p(x) = x^3 + ax^2 + bx + c$, where $a$, $b$, and $c$ are complex numbers. Suppose that \[p(2009 + 9002\pi i) = p(2009) = p(9002) = 0\] What is the number of nonreal zeros of $x^{12} + ax^8 + bx^4 + c$?
Expected: successful generation, or a classified admission/capacity response where applicable. Please investigate the backend failure and whether it occurred before inference/settlement. This report establishes an observed transient failure, not that this question deterministically fails. One explicitly bounded reissue succeeded in14.181 seconds and returned a298-credit receipt. The original failure is retained; collection then resumed only previously unattempted questions. No answer quality was inspected to decide whether to repeat the failed call.
Related workshop PR: #795. No credentials or private prompts are included.
Activity
svonava commentedon Oct 18, 2026
Scope clarification: the priority is preventing this unexpected generation failure from recurring, with a concrete fix and regression coverage for the relevant interruption path. Reconciliation of the original failed call's charges is not required for acceptance.
Keep the original failure and the separately reported bounded successful reissue as evidence. A fix must preserve the generation client's protection against automatically replaying work that may already have executed.
svonava commentedon Oct 18, 2026
Related interruption hardening is now merged in #805. Python sync/async and TypeScript generation streams raise
SIEConnectionErrorwhen the response ends before an explicit or semantic completion marker, rather than accepting partial output as a successful result. The request is not replayed automatically.Regression coverage includes empty/comment-only/partial streams, multiple chat choices, malformed completion markers, cancellation, and error metadata. CodeRabbit approved the exact head and all 21 checks passed, including required CI and the real-model CPU workflow.
This closes a separately reproduced streaming failure mode; it does not establish the cause of the original non-streaming HTTP 500. Keeping #800 open for recurrence-prevention verification. Historical charge reconciliation is not an acceptance requirement.
svonava commentedon Oct 20, 2026
The gateway interruption mitigation is now deployed to production. The recorded production customer acceptance passed; this was checked from the acceptance result and accepted deployment record, not just the workflow status.
A separate bounded staging check of the default
Qwen/Qwen3.8-27B-FP8route also passed: one streaming request, a 16-token limit, HTTP 200, nonempty output, and a terminalfinish_reason="stop". No generation retry or replay was introduced.This supports closing the issue as mitigated. It does not uniquely attribute the original nonstreaming HTTP 500 or establish that every possible interruption is eliminated.
The separate SDK premature-EOF hardening in #805 remains merged. The deployment tested here uses public revision
27eecf055eee6ab921e5996525fd40bac69ea123, which predates #805, so this is not a delivery claim for that SDK change. Its rollout remains tracked in the existing release work.