Skip to content

builder: explain why an unparsable PAX extended header fails the build - #2022

Open
gliush wants to merge 1 commit into
dragonflyoss:masterfrom
gliush:ivan/tarball-pax-newline
Open

builder: explain why an unparsable PAX extended header fails the build#2022
gliush wants to merge 1 commit into
dragonflyoss:masterfrom
gliush:ivan/tarball-pax-newline

Conversation

@gliush

@gliush gliush commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

nydus-image refuses a tarball whose PAX extended header holds a record with a newline in its value, and reports only that the tar crate complained:

tarball: failed to parse PaxExtension from tar header, malformed pax extension

The error now names the entry, and the code records why the build must end there rather than skip the record and carry on — the change this code invites, and must not get.

Skipping looks free: only SCHILY.xattr.* records are read there, and it would convert npm .bin shims, unconvertible today while docker pull handles them. It would also convert an attacker's symlink. tar 0.4.45 splits the header body on newlines before honouring each record's leading length, so the tail of a value is framed as a record and handed back as one. A value holding \n22 linkpath=/etc/evil\n makes Entry::link_name() return /etc/evil while the entry's ustar header says harmless, and a reader walking the records by their length — Go's archive/tar, so docker pull — sees no linkpath at all. path and size come from those same bytes.

Two of the three tests exist for that: one shows a spec-valid tarball is rejected on purpose, one shows what a build that only warned would write into the image.

The fix belongs in tar, walking records by their leading length. When it lands and the dependency is bumped, both tests fail and get updated to assert the real target is stored — which is when these images start converting.

Additional information

Tests:

  • test_malformed_pax_extension_is_rejected — the error names the entry.
  • test_pax_record_containing_newline_is_rejected — the npm .bin shim shape: a spec-valid body tar 0.4.45 cannot frame.
  • test_pax_record_smuggled_inside_a_value_cannot_reach_the_image — the smuggled linkpath. Against a warn-and-continue build it produces a node whose symlink target is /etc/evil.

Parsing the records locally by their leading length, instead of waiting for tar, does not work on the 0.4.45 public API: it needs Entries::raw(true), and raw mode ignores the PAX size record while EntryFields.size has no public setter — so a file above 8 GiB, where GNU tar writes size=0 in the ustar header and the real size in PAX, would read 0 bytes and desync the archive.

Upstream: PaxExtensions::new splits on b'\n' in pax.rs; path and linkpath resolve through filter_map(|f| f.ok()).find(...) in entry.rs, which is what accepts the smuggled fragment; pax_extensions_value returns None on the first error, which drops size.

@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.87500% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.59%. Comparing base (4345613) to head (e8bb403).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
builder/src/tarball.rs 96.87% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2022      +/-   ##
==========================================
+ Coverage   69.36%   73.59%   +4.23%     
==========================================
  Files         210      210              
  Lines       63850    63965     +115     
  Branches    56605    56720     +115     
==========================================
+ Hits        44290    47077    +2787     
+ Misses      17713    14774    -2939     
- Partials     1847     2114     +267     
Files with missing lines Coverage Δ
builder/src/tarball.rs 87.84% <96.87%> (+2.04%) ⬆️

... and 28 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Zephyrcf

Copy link
Copy Markdown
Member

@gliush The inline comments seem excessive for a change that only improves error messages. Updating the existing test_malformed_pax_extension_is_rejected test is sufficient; the other two tests seem unnecessary for this change.

@Zephyrcf
Zephyrcf self-requested a review August 10, 2026 06:23
@gliush gliush changed the title builder: name the entry when a PAX extended header cannot be parsed builder: explain why an unparsable PAX extended header fails the build Aug 20, 2026
@gliush
gliush force-pushed the ivan/tarball-pax-newline branch from c4de160 to 6faa56e Compare August 20, 2026 17:12
@gliush

gliush commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

I trimmed the comment and changed the title to better highlight the issue.

Some context on why I added the tests: I hit this error converting npm images. It named neither the entry nor a reason, so my first patch downgraded it to a warning and skipped the record. That patch was wrong, and the two new tests are what I wish had stopped me.

Regarding tests: all three end at the same error, so they add no code coverage. What separates them is the kind of tarball each one describes — corrupt, valid, and hostile:

  • test_malformed_pax_extension_is_rejected (existing, kept — assertion tightened) — feeds in 99 SCHILY.xattr.user.key=value, a length no writer produces. It proves we reject corrupt tarballs. It now asserts the error names the entry, instead of only that it mentions PaxExtension.

  • test_pax_record_containing_newline_is_rejected (new) — the case I actually hit: a spec-valid header that tar 0.4.45 misparses. Reading only the test above, I concluded my tarball was fine, the bug was in tar, and the error could therefore be relaxed. That test covers a different input class, so nothing contradicted me.

  • test_pax_record_smuggled_inside_a_value_cannot_reach_the_image (new) — why relaxing it is not available. Splitting the body on newlines leaves the tail of a value framed as a record, and tar returns it as one. With a warning instead of an error, that entry builds a node whose symlink target is /etc/evil while its ustar header says harmless, and docker pull of the same layer sees no linkpath at all. I confirmed this against this tree before writing the test.

The last two also survive the tar fix. Once records are walked by their leading length, each starts passing the build and fails — the newline one then asserts the full target is stored, the smuggle one that the target stays harmless. That second assertion is the one worth keeping permanently.

Happy to trim the comment further if it still reads long.

@gliush
gliush force-pushed the ivan/tarball-pax-newline branch from 6faa56e to e8bb403 Compare August 20, 2026 17:23
parse_entry() aborts the build when a record of a PAX extended header fails to
parse, and says only which crate complained:

  tarball: failed to parse PaxExtension from tar header, malformed pax extension

Nothing points at the entry, and nothing says why a build must end over a record
the builder never reads. Name the entry, and write down the reason, because it
is not the obvious one.

A PAX record is "%d %s=%s\n" and its leading length spans the whole record, so a
value may hold arbitrary bytes, newlines included. tar 0.4.45 splits the body on
newlines before honouring that length, so every record in such a header comes
back as an error. The tempting reading is that only SCHILY.xattr.* records are
consumed here, so the rest can be skipped with a warning: that would convert npm
.bin shims, stored as symlinks whose target is the shim's shell script, which
are unconvertible today while docker pull handles them.

It would also convert an attacker's symlink. Splitting the body leaves the tail
of a value framed as a record of its own, and tar hands that fragment back as
one, so a value holding "\n22 linkpath=/etc/evil\n" makes Entry::link_name()
report /etc/evil while the ustar header of the entry says harmless. A reader
walking the records by their length, Go's archive/tar among them, sees the value
verbatim and no linkpath at all. path and size are read from those same bytes.
Skipping the errors puts a symlink target, a path or a size of the tarball
author's choosing into an image which no longer agrees with the layer it was
built from, and does so silently.

Nothing at this point can tell a fragment apart from a record, so the entries
which cannot be read are the ones which must not be written. Three tests pin
that: the error names the entry, the npm shim shape is rejected, and the
smuggled linkpath never reaches a node.

Walking the records by their leading length belongs in tar. Once that lands and
the dependency is bumped, these images convert with their real targets rather
than with degraded ones.

Signed-off-by: Ivan Glushkov <ivan@reflection.ai>
@gliush
gliush force-pushed the ivan/tarball-pax-newline branch from e8bb403 to e09da0b Compare August 20, 2026 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants