Skip to content

POST request bodies are dropped during forwarding (Transfer-Encoding: chunked rewrite sends empty body) #749

Closed

Description

@acepace

Proxify version:

v0.0.16

Current Behavior:

I'm sort of reopening #558 , because the problem isn't just in the log but rather the actual content leaving the proxy is truncated. #656 did not solve the actual proxy behavior.

Expected Behavior:

The upstream server should receive the original POST body intact — either preserved with the original Content-Length header, or properly chunked with the body content followed by the terminator

Steps To Reproduce:

For example,

proxify -http-addr 127.0.0.1:8888 -output /tmp/proxify-test.jsonl

HTTP server code

import sys
from http.server import HTTPServer, BaseHTTPRequestHandler


class DebugHandler(BaseHTTPRequestHandler):
    def _handle(self):
        content_length = int(self.headers.get("Content-Length", 0))
        body = self.rfile.read(content_length) if content_length else b""

        print(f"\n{'=' * 60}")
        print(f"{self.command} {self.path} {self.request_version}")
        for name, value in self.headers.items():
            print(f"  {name}: {value}")
        print(f"\nBody ({len(body)} bytes):")
        if body:
            print(f"  {body.decode(errors='replace')}")
        else:
            print("  <EMPTY — bug is present if a body was expected>")
        print(f"{'=' * 60}\n")
        sys.stdout.flush()

        self.send_response(200)
        self.send_header("Content-Type", "text/plain")
        self.end_headers()
        self.wfile.write(b"OK\n")

    do_GET = _handle
    do_POST = _handle
    do_PUT = _handle
    do_PATCH = _handle
    do_DELETE = _handle


if __name__ == "__main__":
    addr = ("127.0.0.1", 9999)
    print(f"Listening on {addr[0]}:{addr[1]} ...")
    HTTPServer(addr, DebugHandler).serve_forever()

Then running the following commands

curl  -X POST   -d "username=admin&password=secret"   http://127.0.0.1:9999/login
curl -x http://127.0.0.1:8888 -X POST   -d "username=admin&password=secret"   http://127.0.0.1:9999/login

Leads to the following distinct output

Good

============================================================
POST /login HTTP/1.1
  Host: 127.0.0.1:9999
  User-Agent: curl/8.5.0
  Accept: */*
  Content-Length: 30
  Content-Type: application/x-www-form-urlencoded

Body (30 bytes):
  username=admin&password=secret
============================================================

Bad

============================================================
POST /login HTTP/1.1
  Host: 127.0.0.1:9999
  User-Agent: curl/8.5.0
  Transfer-Encoding: chunked
  Accept: */*
  Accept-Encoding:
  Connection: close
  Content-Type: application/x-www-form-urlencoded
  Accept-Encoding: gzip

Body (0 bytes):
  <EMPTY — bug is present if a body was expected>
============================================================

127.0.0.1 - - [23/Feb/2026 15:16:20] "POST /login HTTP/1.1" 200 -

The logs themselves show all is good

─────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     │ File: proxify_logs/127.0.0.1:9999-a9f52d8ff10d9119.txt
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ POST http://127.0.0.1:9999/login HTTP/1.1
   2 │ Accept: */*
   3 │ Accept-Encoding:
   4 │ Connection: close
   5 │ Content-Length: 30
   6 │ Content-Type: application/x-www-form-urlencoded
   7 │ User-Agent: curl/8.5.0
   8 │
   9 │ username=admin&password=secret
  10 │ POST http://127.0.0.1:9999/login HTTP/1.1
  11 │ Accept: */*
  12 │ Accept-Encoding:
  13 │ Connection: close
  14 │ Content-Length: 30
  15 │ Content-Type: application/x-www-form-urlencoded
  16 │ User-Agent: curl/8.5.0
  17 │
  18 │ username=admin&password=secret
─────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
─────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
     │ File: proxify_logs/127.0.0.1:9999-aaa5bfe2bfa5acaf.txt
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ POST http://127.0.0.1:9999/login HTTP/1.1
   2 │ Accept: */*
   3 │ Accept-Encoding:
   4 │ Connection: close
   5 │ Content-Length: 30
   6 │ Content-Type: application/x-www-form-urlencoded
   7 │ User-Agent: curl/8.5.0
   8 │
   9 │ username=admin&password=secret
  10 │ POST http://127.0.0.1:9999/login HTTP/1.1
  11 │ Accept: */*
  12 │ Accept-Encoding:
  13 │ Connection: close
  14 │ Content-Length: 30
  15 │ Content-Type: application/x-www-form-urlencoded
  16 │ User-Agent: curl/8.5.0
  17 │
  18 │ username=admin&password=secret
─────┴────────────────────────────────────────────────────────

Activity

  1. added
    Type: BugInconsistencies or issues which will cause an issue or problem for users or implementors.
    on Feb 23, 2026
  2. github-actions commented on May 31, 2026

    @github-actions

    This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions!

  3. acepace commented on May 31, 2026

    @acepace
    Author

    Bug is still open as the fix hasn't been merged

  4. github-actions commented on Sep 6, 2026

    @github-actions

    This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions!

  5. beyondlong commented on Sep 8, 2026

    @beyondlong

    I investigated #749 and have a tested fix ready in a pull request. If the maintainers are willing to sponsor this work, I propose a $50 bounty for the completed fix.

    The patch:

    • preserves the incoming Content-Length instead of forcing every request to chunked transfer encoding;
    • gives the asynchronous logger an independent streaming body, so it no longer races the upstream transport for the same reader;
    • uses bounded-memory streaming rather than the unbounded io.ReadAll approach flagged on the earlier PR;
    • keeps forwarding successful if the logging reader stops early;
    • adds regression tests for the original POST-body failure, unknown-length bodies, and logger-reader failure.

    Verification completed on the current dev branch:

    • GOTOOLCHAIN=go1.25.0+auto go test -race ./...
    • GOTOOLCHAIN=go1.25.0+auto go vet ./...
    • GOTOOLCHAIN=go1.25.0+auto go build ./...
    • golangci-lint run --timeout 5m (0 issues)

    Implementation: #764

    The implementation is submitted so it can be reviewed on its technical merits. The bounty proposal is optional and only becomes payable if explicitly accepted by a maintainer.

    Disclosure: the implementation and tests were prepared with Codex assistance and reviewed and verified locally by the contributor.

  6. self-assigned this
    on Sep 13, 2026
  7. added 2 commits that reference this issue on Sep 13, 2026
    6e0513a
    0cefd1a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type: BugInconsistencies or issues which will cause an issue or problem for users or implementors.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions