fix(log): #0044 log-hygiene batch — artifact planner, ClientDisconnect, default-service setup probe (0.21.135) - #1051
Merged
Conversation
…99, setup probe (0.21.135) Three prod log-hygiene findings (kth-k8s:zippy-goat nightly review 2026-08-13). None is a real fault; each restates a benign/expected condition at a level that floods the retained kubectl-logs window or poisons the ERROR metric health thresholds key on. Same family as #8/#19/#20/#21. 1c. hypha/artifact.py::search — 6 query-planner narration lines (`search: stage parameter value`, `Adding stage filter condition`, `Adding condition to return ONLY …`) demoted INFO→DEBUG. search() is a hot listing path (~300k lines/24h); these are internal planner breadcrumbs with no steady-state operator value. 2. hypha/http_rpc.py::_handle_rpc_post — catch starlette ClientDisconnect specifically, BEFORE the broad `except Exception`. A client hanging up before/while its body arrives is a normal network event, not a server error; the broad arm logged it ERROR+traceback+500 (618/24h). Now DEBUG (no traceback) + 499 (no client left to answer). 3. hypha/core/workspace.py — default-service `setup` probe. The remote-service proxy (Munch/ObjectProxy) RAISES AttributeError('setup') when a default service defines no setup(); the old `if svc.setup:` inside a broad except logged a per-client ERROR whose detail interpolated to the useless bare string "setup" (~20/24h, unactionable). Now: getattr(svc,"setup",None) + callable() guard (no-setup → silent no-op), a narrower except around the resolve step, and every caught detail formatted `type(e).__name__: {e!r}` so a setup() that actually raises is diagnosable (names the real failure, not the attribute). workers/__init__.py:38 has the same shape but is left intentionally (worker services define setup; out of scope). Tests: tests/test_log_hygiene_0044.py (5, real, Docker-free/FakeRedis, no mocks). Reproduce-before-fix verified: stashing the 3 fixes makes the 3 regression guards fail, with the captured log showing the exact prod signature `ERROR workspace:… Failed to run setup for default service …: setup`. Version bumped 0.21.134→0.21.135 across the 10 packaging files (CLAUDE.md and .svamp/issues/0003.md mentions of 0.21.134 are #43 history, left as-is). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three prod log-hygiene findings from kth-k8s:zippy-goat's nightly review (2026-08-13). Prod runs 0.21.107, main ~0.21.134 — sub-findings 1a/1b were already fixed on main; 1c, 2, 3 are genuinely present and fixed here. None is a real fault — each restates a benign/expected condition at a level (INFO/ERROR) that either floods the retained
kubectl logsforensic window or poisons the ERROR metric that health thresholds key on. Same log-hygiene family as #8 / #19 / #20 / #21.1c —
hypha/artifact.py::searchquery-planner narration → DEBUG6 lines (
search: stage parameter value,Adding stage filter condition,Adding condition to return ONLY STAGED/COMMITTED …) fired at INFO on everysearch()— a hot listing path (~300k lines/24h in prod). Internal planner breadcrumbs, no steady-state operator value → demoted INFO→DEBUG.2 —
hypha/http_rpc.py::_handle_rpc_postcatchesClientDisconnectWhen a client hangs up before/while its request body arrives,
await request.body()raisesstarlette.requests.ClientDisconnect. The broadexcept Exceptionlogged it ERROR + full traceback + 500 (618/24h). It's a normal network event → caught specifically (before the broad arm), logged at DEBUG with no traceback, answered 499 (no client left to answer).3 —
hypha/core/workspace.pydefault-servicesetupprobe (the reusable code trap)The remote-service proxy (
get_remote_service→ Munch/ObjectProxy) RAISESAttributeError('setup')when a default service simply defines nosetup(). The oldif svc.setup:inside a broadexcept Exception as e: logger.error(f"… {e}")turned that benign, expected case into a per-client ERROR whose{e}interpolated to the useless bare string"setup"(~20/24h, unactionable by construction). Fix:setup = getattr(svc, "setup", None)+if callable(setup):(no-setup → silent no-op), a narrowerexceptaround the resolve step, and every caught detail reformattedf"{type(e).__name__}: {e!r}"so asetup()that actually raises is diagnosable — naming the real failure (RemoteException: RemoteException('RemoteError:boom-in-setup…')), never the attribute name.Tests
tests/test_log_hygiene_0044.py— 5 real tests, Docker-free (FakeRedis), no mocks. All conftestfastapi_server*fixtures aresubprocess.Popen(caplog can't reach them), so each path is driven in-process: a realArtifactController(store, s3_controller=None); realHTTPStreamingRPCServerroutes overhttpx.ASGITransportwith a valid Bearer token + monkeypatchedRequest.bodyraisingClientDisconnect; two real event-buscreate_rpcpeers registering adefaultservice (built-in registry key seeded so the real:default@block runs).Reproduce-before-fix verified: stashing the 3 fixes makes the 3 regression guards fail, with the captured log showing the exact prod signature
ERROR workspace:workspace.py:2401 Failed to run setup for default service …: setup.Version
Bumped 0.21.134 → 0.21.135 across the 10 packaging files. CLAUDE.md and
.svamp/issues/0003.mdmentions of0.21.134are #43 history and left as-is.🤖 Generated with Claude Code