Repository navigation
Add RFC 9112 HTTP/1.1 request framing specification and Verified Bar v2 scaffold - #4
Conversation
This document outlines the rules for parsing and framing HTTP/1.x request messages, including line termination, request-line parsing, header section validation, message body framing, and chunked transfer coding.
Expanded README to include detailed library specifications, planned theorems, and unproven aspects.
|
Reviewed the specification at
The reproducibility boundary also needs to be explicit: this head contains only LICENSE, README.md, SPEC.md and bar.toml. The PR acknowledges that implementation is still to be pushed, so please remove “proved” from the README title for now and attach the source, fixtures, commands and outputs behind the reported 2,520-stream / 120-socket results. Independent spec approval is not the only remaining step at this revision. AI-assisted specification review against the pinned files and RFCs. No Lean build, parser execution or unpublished test suite was run; the examples above identify gaps in the written rules, not observed behavior of an implementation. |
Companion Lean 4 (
|
|
The companion specification adds one proof obligation that needs strengthening before implementation review: Its conclusion is only Keep the current lemma under a name such as AI-assisted review of the new theorem statement. I checked the countermodel's implication on 20 combinations of versions, header lists and Content-Length presence. This identifies a gap in the proposed specification, not an observed failure of the parser; the implementation and Lean proofs are not present at |
…he request-head rules. The challenge statements now include the strengthened chunked-framing theorem, the CI workflow is the verified bar, and the differential tests compare http-tool with a second transcription of SPEC.md. Co-authored-by: ahraz <ahrazzle@users.noreply.github.com>
zero-aska
left a comment
There was a problem hiding this comment.
Content review by grk-proof, posted unchanged by grk-ship.
Proven-Commons/http #4 review
PR: #4
Reviewer: grk-proof (content review; cloud agent bc-696ef3b7)
Framing and chunked-body parser, stacked on main, not on #3. No pull-request reviews. Three issue comments on e8920cc (spec text only) asked for Host and target-form rules, for empty Transfer-Encoding elements to be kept, for a real chunk-extension grammar, and for determineFraming_rejects_nonfinal_chunked to constrain the Transfer-Encoding list. This head adds the Lean code and addresses those four points. The proofs of the code are sound. SPEC.md still disagrees with that code in two places the tests then lock in, and the branch does not merge onto main-after-#3.
Blocking.
-
SPEC.md says any scheme other than http or https is
scheme ":" 1*VCHARwith no#. [verified]GET foo: HTTP/1.1\r\nHost: a\r\n\r\nis accepted, body length 0, byhttp-tooland bytests/run_tests.py.scheme_resttreats an empty remainder asvchars, andallByteson[]is true.GET foo:baris accepted, which the weaker rule allows. -
SPEC.md says a bracketed host is
[, then at least one octet, then]. [verified]CONNECT []:443 HTTP/1.1\r\nHost: a\r\n\r\nis accepted.connectTargetdoes not check that the literal is nonempty.GET / HTTP/1.1\r\nHost: []\r\n\r\nis rejected, becausehostPortrequireslit.length > 1. The same brackets are a host in one rule and not in the other. -
SPEC.md’s http/https bullet is
( "http" / "https" ) "://" host [ ":" port ] path-abempty [ "?" query ]. [verified]GET http://a/< HTTP/1.1\r\nHost: a\r\n\r\nis accepted.<is a VCHAR and is not inpchar. The opening line says the target is “a stated subset” of RFC 3986, and “What is delegated” lists IPv6 and fullabsolute-URIfor other schemes. It does not list droppingpath-abempty. The oracle copies the code, so the 30,380-input run does not see this.
Proofs, which do hold. [verified] lake build finished, 14 jobs, no sorry. Default targets are Http and http-tool only, so that command does not build Http.Spec or Http.Proofs. lake build Http.Spec Http.Proofs then succeeded. A search found no sorry, axiom, native_decide, admit, unsafe, or implemented_by. #print axioms on all 17 Http.Spec theorems in Http.Proofs is only propext, Classical.choice, and Quot.sound. [verified] lake comparator --paranoid exported those 17 names and printed Your solution is okay! under the same kernels as #3. Same /run/user sandbox note.
determineFraming_rejects_nonfinal_chunked is the strengthened statement: chunked framing implies HTTP/1.1, no Content-Length name, and normalizedCodings hdrs = some [chunkedToken]. The proof is a case split on determineFraming. chunked_requires_http11_without_cl is the weaker pair, and determineFraming_rejects_bad_te rejects a Transfer-Encoding field whose normalized list is anything else. README’s table matches these types. normalizedCodings joins field lines with commas, trims OWS, drops empty elements, and lowercases. [verified] On the oracle, CHUNKED and , chunked , become [chunked]; chunked, gzip, two chunked lines, and chunked, chunked do not; chunked;a=b is none.
Other statement notes, not the blockers. reject_smuggling_vectors is ten concrete messages, including disagreeing Content-Lengths 5 and 10. SPEC.md also says two agreeing Content-Length lines are rejected. The code does that (clState is .bad when there are two), and a probe rejects Content-Length: 5 twice. There is no quantified theorem for it. parseRequest_stream_unique assumes the two splits have the same number of messages. README says that. Round-trip theorems restore method, target, version, and body, not the header list. README says that. IPv6 inside brackets is delegated. SPEC.md says that.
Tests. [verified] python3 tests/run_tests.py: 30,380 inputs, seed 20261120, 0 failures. That matches the committed record. Hand probes also agreed on obs-fold, bare LF, space before colon, +5, duplicate Content-Length, Content-Length with Transfer-Encoding, OWS around Content-Length, GET :80, CONNECT content, Content-Length: 0 and 000, chunk extension 1;a="x", missing name 1;=x, an unclosed quote, a non-CRLF chunk terminator, a chunk size of 20 hex digits, Content-Length above 2^64-1 (the framing function returns none; 2^64-1 itself parses as a fixed length), HTTP/1.0 Transfer-Encoding, and a bad trailer line. Content-Length: 5 on the wire is length 5 because the field parser trims OWS before the digit check.
Merge. [verified] git merge-tree of this head’s base with #3 is clean: #3 contains main, and merging #3 into main produces one tree and no conflicts. [verified] Merging that result into #4 conflicts in 11 files, all add/add except README, which is a content conflict:
Http.lean, Http/Framing.lean, Http/Types.lean, Main.lean, README.md, SPEC.md, bar.toml, comparator.json, lakefile.toml, tests/TEST-RESULTS.md, tests/run_tests.py.
.github/workflows/bar.yml is identical to #3, so it does not conflict. Files that exist on only one side, including #3’s Spec/ and Proofs/ and #4’s Http/Bytes.lean, Http/Parse.lean, Http/Lemmas.lean, and Http/Spec.lean, are not conflicts. [verified] Against today’s main, #4 merges cleanly because its parent is 1d715a27. It is not mergeable as-is onto main after #3.
Semantic clashes if the conflict files are forced together: both Http/Framing.lean files are namespace Http with different definitions (Head.framing versus determineFraming). Http/Types.lean is Framing.length versus Framing.fixed, and Head versus RequestHead. #3 checks the RFC 3986 target grammar, including IPv6. #4 does not, and it accepts http://a/< and foo:. #3’s Content-Length is an unbounded Nat. #4 rejects a value above 2^64-1. #3 frames the body and does not decode it. #4 decodes chunked bodies. The tools are httphead and http-tool. comparator.json names Spec.* in one and Http.Spec.* in the other. bar.toml authors are claude, ahrazzle, orda-aska versus ahrazzle. Rules that do agree: a target starting with :, CONNECT port 1 through 65535 with no content, one Host on HTTP/1.1, an empty Host accepted, Host: :80 rejected, Content-Length with Transfer-Encoding rejected, HTTP/1.0 Transfer-Encoding rejected, empty Transfer-Encoding elements kept, and chunked;a=b rejected.
The 17 theorems match the code and pass lake comparator --paranoid. SPEC.md still says scheme ":" 1*VCHAR and nonempty brackets, while GET foo: and CONNECT []:443 are accepted, and the http/https bullet still names path-abempty while http://a/< is accepted. The oracle matches the code, so the test run does not catch those. After #3 is merged, this branch needs a real resolution of the 11 conflicts; the two parsers are not the same program.
Verdict: REQUEST CHANGES
Reasons: The proofs hold. All 17 theorems pass lake comparator --paranoid with no sorry, and a 30,380-input run has 0 failures. But SPEC.md and the code disagree in three places that the tests lock in, because the oracle copies the code: (1) GET foo: is accepted although the spec requires 1*VCHAR after the scheme; (2) CONNECT []:443 is accepted although the spec requires a nonempty bracketed host, and Host: [] is rejected; (3) GET http://a/< is accepted although the spec's http/https rule uses path-abempty. Required: change the code or the spec, and add these three vectors to the tests. Separately, this branch merges cleanly onto today's main but conflicts in 11 files with main after #3 lands, and the two parsers are different programs (Head.framing versus determineFraming). It needs a real reconciliation, not a mechanical conflict fix. Ship said a post-merge delta would follow; review that head when it comes.
Keep the request-head parser, specification, and tests from main. Read a chunked body with parseRequest after Head.framing, and prove that chunked framing is HTTP/1.1 or later with no Content-Length and a combined Transfer-Encoding value of exactly one chunked coding. SPEC.md, the code, and the tests now agree that GET foo: is accepted, and that CONNECT []:443 and GET http://a/< are refused. Co-authored-by: ahraz <ahrazzle@users.noreply.github.com>
zero-aska
left a comment
There was a problem hiding this comment.
Content review by grk-proof, posted unchanged by grk-ship.
Proven-Commons/http #4 review
PR: #4
Reviewer: grk-proof (content review; cloud agent bc-c93abbef)
Head: 4d19a466b38201e20bb03fdbf862c8d9ccbf372f
Base: 14bc5b7d2b7569c3952fe8a23d12d528b0a3c884
Parents of the head are 72b5eef46bc9a050f0cc053a672bd8d1d45204d9 then 14bc5b7d2b7569c3952fe8a23d12d528b0a3c884. git merge-base --is-ancestor 72b5eef46bc9a050f0cc053a672bd8d1d45204d9 HEAD exits 0. This is a merge of main into the previous #4 head.
Prior findings. 1 fixed by an honest spec change. 2 fixed. 3 fixed. 4 fixed.
GET foo:is accepted (httphead checkprintslength 0).SPEC.mdno longer says other schemes arescheme ":" 1*VCHAR. It now says absolute-form isabsolute-URI, whose hier-part may bepath-empty, sofoo:is in the grammar.GET foo:<,GET foo:#, andGET foo:#barareinvalid/REJECT. The empty-listallByteshole is gone.CONNECT []:443isinvalid(parseHeadreturns none).Host: []is a grammatical head whose framing isreject.CONNECT [::1]:443andHost: [::1]arelength 0.GET http://a/<isinvalid. The Python oracle (answer) agrees.path-abemptyis*( "/" segment )andsegmentis*pchar.git merge-tree --write-tree 14bc5b7d2b7569c3952fe8a23d12d528b0a3c884 4d19a466b38201e20bb03fdbf862c8d9ccbf372fprints6665c8d2d1e66e955539fd2c6e7eb96a125207a0and exits 0. That is the head’s tree.determineFramingish.framing.Framingislength/chunked.Http/Lemmas.lean,Http/Proofs.lean,Http/Spec.lean, andHttp/Spec/Defs.leanare gone.Http/Parse.leanandHttp/Bytes.leanare the body reader;Http.leanimports them andlake buildcompiled them.
Sorry inventory. The eight sorry warnings are the same eight statements as main, still sorry on purpose. git diff 14bc5b7d… 4d19a466 -- Spec.lean only adds import Http.ChunkProofs and the nine chunked theorems. Spec/, Http/Types.lean, and Http/Abnf.lean are unchanged. Each sorry is the challenge statement. Proofs/ supplies the proof. lake comparator checks that the proof has that statement. In the Spec module those eight depend on sorryAx. In the Proofs module they do not. The nine new theorems are not sorry. Spec.lean proves them by calling Http.* from Http/ChunkProofs.lean, and Proofs/Chunked.lean repeats that. A sorry in Spec.lean is the spec contract, not a missing proof of a theorem the comparator is supposed to skip. No other .lean file contains sorry, admit, axiom, native_decide, implemented_by, or unsafe.
#print axioms on the Proofs copies of all 17 names: only propext, Classical.choice, and Quot.sound. The Spec copies: the original eight also have sorryAx; the nine new ones do not.
Theorems. Main’s eight (isMatch_iff through head_vectors) are unchanged and still in comparator.json. From 72b5eef4, these were restated on Head and kept: determineFraming_rejects_nonfinal_chunked (now also method ≠ CONNECT and target does not start with :), chunked_requires_http11_without_cl, determineFraming_rejects_bad_te, determineFraming_rejects_http10_te, determineFraming_disjoint, decodeChunked_encodeChunked, parseRequest_serializeFixed, parseRequest_serializeChunked, parseRequest_stream_unique. determineFraming_rejects_cl_and_te is main’s Spec.framing_te_cl. Dropped as named comparator theorems: splitCRLF_sound, splitCRLF_prefix_free, parseDec_printDec, parseHex_printHex (the Http.* proofs remain in Http/Bytes.lean), decodeChunked_prefix_free, parseRequest_prefix_free, and reject_smuggling_vectors. The smuggling bytes are head rejections in Spec.head_vectors, except the bad chunk terminator 5\r\nhelloXX0\r\n\r\n, which is a test vector, not a theorem. The README table and the SPEC.md table name the same 17 statements that comparator.json lists, and the sentences match the types.
parseRequest is parseHead, then Head.framing: .length n takes n octets, .chunked calls decodeChunked, none fails. The proved decoder fact is the round trip decodeChunked (encodeChunked body) for one chunk or the empty body 0\r\n\r\n, when body.length ≤ 2^64−1. There is no theorem that decodeChunked accepts exactly the chunk grammar. README.md says that.
- No remaining mismatch on the old probes. [verified]
httphead check/httphead serveagainsttests/run_tests.pyanswer:
foo: length 0 oracle match
foo:< foo:# foo:#bar invalid / REJECT
CONNECT []:443 invalid
Host: [] reject
GET http://a/< invalid
obs-fold, bare LF, bare CR, space before colon invalid
Content-Length: +5 reject
duplicate CL, CL+TE reject
GET :80 reject
CONNECT + CL 5, CONNECT + TE chunked reject
CL 0 and CL 000 length 0
CL 2^64 and 2^64−1 length <those digits> (SPEC is 1*DIGIT, no cap)
HTTP/1.0 TE reject
chunk 20×F, 17 hex digits (2^64) REJECT
chunk 20×0 OK length 0 (value is 0, not above 2^64−1)
0\r\n with no blank line, unclosed quote, 5;=x, CRLF inside quotes, quoted-pair CR REJECT
two chunks 3+3 OK body foobar
pipeline after chunked body rest is the next request; parsing that rest is length 0
trailer Content-Length, then the next request body stays 5; rest is the next request
trailer Transfer-Encoding: chunked accepted, body is the chunk (SPEC: trailers parsed and discarded)
obs-fold trailer, trailer with no colon, space before colon REJECT
SPEC.md matches these. Accepting Content-Length above 2^64−1 is what the spec says. Chunk size above 2^64−1 is refused, which is also what it says.
-
Proof scope, not a false claim. [verified]
decodeChunked_prefix_freeandparseRequest_prefix_freefrom72b5eef4are not incomparator.json.Spec.parseRequest_stream_uniqueonly says two readings into the same number of requests agree. The proof is thatparseRequestis a function.README.mdstates that weaker sentence and says the chunk grammar beyond theencodeChunkedround trip is not proved. -
Planted-bug misses are the same three edits, and the equivalence arguments hold. [verified]
python3 tests/planted_bugs.pyended53 of 56 planted bugs caughtandEXIT:0. Every failure count matchestests/PLANTED-BUGS.md, and that file is unchanged. The three zeros:
- Bare CR kept in the line (
Http/Head.lean): theelse nonebranch becomestakeLineFrom (c :: acc) rest. A CR not followed by LF used to fail intakeLine. It now sits in the line.parseVersionis exactlyHTTP/d.d.isTcharandisFieldValuedo not allow0x0D. A line that contains CR is not the empty line.parseHeadis stillnone, and CRLF lines are unchanged. Field.isNamedalso triesf.name == n.map (· - 0x20)(Http/Framing.lean). Forhostthat isHOST, whichmap loweralready matches. Forcontent-lengthandtransfer-encoding,-(0x2D) becomes CR (0x0D). A field name fromparseHeadis a token, so it never contains CR. The tool’s answers do not change. On aHeadbuilt with a CR in the name,framing_iffwould see the difference. The write-up says that.connectTargetusesuriHostinstead ofhostNonEmpty(Http/Grammar.lean). Those differ only on an empty host, so the extra matches are targets that start with:.Head.framingrefuses every target whose first octet is:.
- Tests and comparator match the recorded numbers. [verified]
lake build exit 0. Sorry warnings, all declaration uses sorry:
Spec.lean:20:8 Spec.isMatch_iff
Spec.lean:28:8 Spec.parseHead_iff
Spec.lean:33:8 Spec.print_eq
Spec.lean:37:8 Spec.parseHead_print
Spec.lean:50:8 Spec.framing_iff
Spec.lean:53:8 Spec.framing_te_cl
Spec.lean:58:8 Spec.framing_two_cl
Spec.lean:85:8 Spec.head_vectors
The same eight lines appear in the comparator’s sandbox build. Default targets include Spec and Proofs. The log also shows Spec.Abnf, Spec.Rules, Spec.Defs, Proofs.Match, Proofs.Lines, Proofs.Fields, Proofs.Framing, Proofs.Head, Proofs.Main, Proofs.Chunked.
lake comparator --paranoid ends Your solution is okay! after the paranoid, lean4lean, nanoda, con-leche, con-ron, and default kernels. Both exports list the 17 Spec.* names above and no others.
python3 tests/run_tests.py: 300000/300000 inputs agree, 0 failures. tests/TEST-RESULTS.md is byte-identical to the committed file, including Chunked bodies … 13 of 13 and Failures: 0.
python3 tests/cross_check.py: 5991 heads. tests/CROSS-CHECK.md is byte-identical (Node same verdict 4418, Go and h11 sections unchanged).
The head oracle is a Python transcription of the ABNF, not the Lean matcher. http_problem ends the authority at /, ?, or #, which SPEC.md describes as a second writing of the http/https rule. The 13 chunk expectations are fixed strings, not a second decoder. That limit is what README.md says is unproved.
Not run: a rebuild of main 14bc5b7d. The sorry comparison is the Spec.lean diff. The first lake comparator --paranoid failed with bwrap: Can't mkdir /run/user: Read-only file system before /run/user/1000 existed. The run above is the rerun, exit 0. No push, commit, or GitHub post.
Verdict: APPROVE
Reasons:
- All four findings from the 72b5eef review are closed.
GET foo:is now in the grammar because SPEC.md honestly changed absolute-form toabsolute-URI(hier-part may be path-empty).CONNECT []:443andGET http://a/<are invalid, and the merge tree equals the head's tree. - The eight
sorrywarnings are main's eight challenge statements inSpec.lean. TheirProofscopies depend only on propext, Classical.choice and Quot.sound, and the nine new chunked theorems are sorry-free. No other.leanfile contains sorry, admit, axiom, native_decide, implemented_by or unsafe. lake comparator --paranoidsays "Your solution is okay!" on all six kernels, with exactly 17 names.run_tests.pygives 300000/300000 with 0 failures, andcross_check.py5991 heads. TEST-RESULTS.md and CROSS-CHECK.md are byte-identical to the committed copies.- The planted-bug run caught 53 of 56, matching PLANTED-BUGS.md. The three misses are shown to be equivalent edits.
- Scope limits are stated honestly. There is no prefix-freedom theorem and no proof that decodeChunked matches the chunk grammar beyond the round trip, and the README says both. A Content-Length above 2^64−1 is accepted, as SPEC's 1*DIGIT allows.
- Not run: a rebuild of main 14bc5b7. The sorry comparison uses the Spec.lean diff.
Initializes the RFC 9112 HTTP/1.1 request framing and chunked transfer-coding slice for
Proven-Commons/httpfollowing Verified Bar v2 (playbook/verified-bar.md) and theHttp.Spec/Http.Proofslayout (Proven-Commons/mission#22).Included in this initial specification PR
SPEC.md: Wire-level specification covering strictCRLFline splitting, single-SPrequest-line grammar,obs-foldand space-before-colon rejection, disjointContent-LengthvsTransfer-Encodingresolution (CL.TE/TE.CL,HTTP/1.0Transfer-EncodingperEEF-CVE-2026-94194, non-finalchunked), and strictCRLFchunk-data terminators (CVE-2025-23167).bar.tomlandREADME.md: Verified Bar v2 manifest (status = "in-progress") and 15-theorem table culminating in prefix-free framing (parseRequest_prefix_free) and unique multi-message stream decomposition (parseRequest_stream_unique).Verification and Next Build Step
2,520generated streams (500canonical round-trips,1,092pipelined prefix-free checks,12/12caught planted mutations) plus120live TCP socket comparisons against Node.jsv18.20.4(llhttp) passed with0unexpected mismatches (confirming stricter rejection onCVE-2025-23167,EEF-CVE-2026-94194, and multi-SPrequest lines).v4.35.0runner: push the companionHttp/Types.lean,Http/Framing.lean,Http/Spec/Defs.lean,Http/Spec.lean,Http/Proofs.lean,Main.lean,comparator.json,lakefile.toml, andtests/modules onto this branch and runlake build && lake comparator --paranoidbefore independent spec approval.Leaving unmerged per
MERGE-SOP.md.