Skip to content

Random non-responding requests invisible in the failure stats #198

Open
Bug
Open
Random non-responding requests invisible in the failure stats#198
Bug
Assignees
JaeYeonLee0621

Description

@JaeYeonLee0621
JaeYeonLee0621
opened on Sep 28, 2026

Describe the bug
After the vLLM version upgrade on 25/09, the model randomly times out for a few minutes.

These timeouts don't show up in the failed-request count. So the overall stats look fine even though users keep hitting real, repeated timeouts.

Possible reasons why it is not logged

  • Could be read timeout causes no logs (e.g. vllm not reachable)
  • Could be client side timeout causes no logs (OpenCode closes connection and we do not log it) ⭐

Activity

  1. JaeYeonLee0621
    self-assigned this
    on Sep 28, 2026

  2. JaeYeonLee0621 commented on Sep 28, 2026

    @JaeYeonLee0621
    JaeYeonLee0621
    on Sep 28, 2026
    ContributorAuthor

    The conceptual point

    • At start → None (state is "in progress / unknown").
    • If client closes → we should record 499 (client-closed) but right now my code sets 500
      there. 🆕
    • If upstream fails → 500.

    => Make none status code to 499 and test it shows up in the dashboard

  3. JaeYeonLee0621
    added the
    Bug
    issue type on Sep 28, 2026
  4. JaeYeonLee0621
    linked a pull request that will close this issue198: Track client-closed requests as 499 in usage logging #199on Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Bug

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions