Fix HTTP client, binary protocol, and graceful shutdown issues - #89
Conversation
NewHTTPClient did not assign the pathPrefix parameter, so clients could not reach servers configured with HTTPPathPrefix. Fetch and FetchMulti mutated the shared *url.URL in place, which was a data race under concurrent calls and leaked the query parameter set by FetchMulti into subsequent Fetch requests. Operate on a copy of the URL instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: fujiwara <fujiwara.shunichiro@gmail.com>
A negative n caused a panic by make() with a negative capacity, and n=0 returned an empty response. Reject n out of the range 1-1000 with 400 Bad Request. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: fujiwara <fujiwara.shunichiro@gmail.com>
- Respond to a failed binary GET with a binary error response instead of the text protocol "ERROR" line - Echo the requested opcode and opaque in error responses - Fix the short write check in writeBinaryError which compared against the length of the text error response - Reject binary requests with a body larger than 64KB to prevent a huge allocation by a malformed header Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: fujiwara <fujiwara.shunichiro@gmail.com>
The HTTP server passed the already-canceled context to Shutdown, so it returned immediately without draining in-flight requests. The gRPC server used Stop which kills in-flight RPCs. Use a new context with ShutdownTimeout for HTTP and GracefulStop with a timeout fallback for gRPC, and make RunHTTPServer/RunGRPCServer wait until the shutdown completes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: fujiwara <fujiwara.shunichiro@gmail.com>
Invalid values such as KATSUBUSHI_PORT=abc were silently ignored and the process started with the default value. Exit with an error instead, consistent with how flag.Parse handles invalid commandline values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: fujiwara <fujiwara.shunichiro@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR addresses several correctness and operational issues across the HTTP client/server, binary (memcached) protocol handling, graceful shutdown behavior, and configuration via environment variables.
Changes:
- Fix HTTP client path prefix handling and eliminate shared
url.URLmutation inFetch/FetchMulti; add related tests and validateGET /ids?n=range (1–1000). - Improve graceful shutdown behavior for HTTP and gRPC servers with a bounded drain timeout and completion waiting.
- Harden binary protocol error responses and add a request body size limit; add tests for error responses and oversized bodies; make invalid env var values fail fast.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents the n parameter constraints for GET /ids. |
| http.go | Adds graceful HTTP shutdown waiting, validates n, fixes HTTP client pathPrefix usage and URL mutation. |
| http_test.go | Adds tests for invalid n, pathPrefix handling, and URL isolation/concurrency behavior. |
| grpc.go | Adds graceful gRPC shutdown with timeout and waits for shutdown completion. |
| cmd/katsubushi/main.go | Makes invalid env var values for flags fail fast (exit code 2) via applyEnvToFlag. |
| cmd/katsubushi/main_test.go | Adds test coverage for invalid env var flag values. |
| binary_protocol.go | Adds binary request body size limit and makes binary error responses echo opcode/opaque. |
| binary_protocol_test.go | Adds tests for too-large binary request bodies and binary error response shape. |
| app.go | Introduces ShutdownTimeout constant used by HTTP/gRPC graceful shutdown paths. |
| app_test.go | Updates binary error response expectations to echo requested opcode. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if bodyLen < uint32(keyLen)+uint32(extraLen) { | ||
| return nil, fmt.Errorf("total body %d is too small. key length: %d, extra length %d", bodyLen, keyLen, extraLen) | ||
| } | ||
| if bodyLen > maxBinaryBodyLen { | ||
| return nil, fmt.Errorf("total body %d is too large. limit: %d", bodyLen, maxBinaryBodyLen) |
There was a problem hiding this comment.
Good catch — the overflow was reachable (keyLen=65535, extraLen=1 passes both length checks and panics on slicing, crashing the process since there is no recover in the connection goroutine). Fixed in 0d53ff5 by calculating the indexes in int, with a regression test that reproduces the panic on the old code. Kept keys over 250 bytes accepted since katsubushi ignores key contents.
keyLen + extraLen was calculated in uint16, so crafted headers with keyLen+extraLen >= 65536 wrapped around and caused a panic by slicing with out-of-range bounds, crashing the process. Calculate the indexes in int. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: fujiwara <fujiwara.shunichiro@gmail.com>
What
Fixes for issues found in a code review:
pathPrefix:NewHTTPClientdid not assign the parameter, so clients could not reach servers configured withHTTPPathPrefix.Fetch/FetchMultirewrote the shared*url.URLin place, a data race under concurrent calls, and then=query set byFetchMultileaked into subsequentFetchrequests. Now operates on a copy.GET /idsaccepted invalidn: a negativencaused a panic bymake()with a negative capacity, andn=0returned an empty response.nout of 1-1000 is now rejected with 400.ERRORline, which binary clients cannot parse. Error responses are now proper binary headers and echo the requested opcode/opaque. Also fixed a meaningless short write check.bodyLen=0xFFFFFFFFcould trigger a 4GiB allocation per request, andkeyLen+extraLenoverflowing uint16 caused a panic that crashed the process. Bodies are now limited to 64KB and the index math is done in int.Shutdown(returning immediately without draining), and the gRPC server usedStopwhich kills in-flight RPCs. Both now drain in-flight requests with a 10s timeout (ShutdownTimeout), andRunHTTPServer/RunGRPCServerwait until the shutdown completes.KATSUBUSHI_PORT=abcwere silently ignored. The process now exits with an error, consistent with howflag.Parsehandles invalid commandline values.Behavior changes
GET /ids?n=0returned 200 with an empty body, now returns 400.🤖 Generated with Claude Code