builder: explain why an unparsable PAX extended header fails the build - #2022
builder: explain why an unparsable PAX extended header fails the build#2022gliush wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2022 +/- ##
==========================================
+ Coverage 69.36% 73.59% +4.23%
==========================================
Files 210 210
Lines 63850 63965 +115
Branches 56605 56720 +115
==========================================
+ Hits 44290 47077 +2787
+ Misses 17713 14774 -2939
- Partials 1847 2114 +267
🚀 New features to boost your workflow:
|
|
@gliush The inline comments seem excessive for a change that only improves error messages. Updating the existing |
c4de160 to
6faa56e
Compare
|
I trimmed the comment and changed the title to better highlight the issue. Some context on why I added the tests: I hit this error converting npm images. It named neither the entry nor a reason, so my first patch downgraded it to a warning and skipped the record. That patch was wrong, and the two new tests are what I wish had stopped me. Regarding tests: all three end at the same error, so they add no code coverage. What separates them is the kind of tarball each one describes — corrupt, valid, and hostile:
The last two also survive the tar fix. Once records are walked by their leading length, each starts passing the build and fails — the newline one then asserts the full target is stored, the smuggle one that the target stays Happy to trim the comment further if it still reads long. |
6faa56e to
e8bb403
Compare
parse_entry() aborts the build when a record of a PAX extended header fails to parse, and says only which crate complained: tarball: failed to parse PaxExtension from tar header, malformed pax extension Nothing points at the entry, and nothing says why a build must end over a record the builder never reads. Name the entry, and write down the reason, because it is not the obvious one. A PAX record is "%d %s=%s\n" and its leading length spans the whole record, so a value may hold arbitrary bytes, newlines included. tar 0.4.45 splits the body on newlines before honouring that length, so every record in such a header comes back as an error. The tempting reading is that only SCHILY.xattr.* records are consumed here, so the rest can be skipped with a warning: that would convert npm .bin shims, stored as symlinks whose target is the shim's shell script, which are unconvertible today while docker pull handles them. It would also convert an attacker's symlink. Splitting the body leaves the tail of a value framed as a record of its own, and tar hands that fragment back as one, so a value holding "\n22 linkpath=/etc/evil\n" makes Entry::link_name() report /etc/evil while the ustar header of the entry says harmless. A reader walking the records by their length, Go's archive/tar among them, sees the value verbatim and no linkpath at all. path and size are read from those same bytes. Skipping the errors puts a symlink target, a path or a size of the tarball author's choosing into an image which no longer agrees with the layer it was built from, and does so silently. Nothing at this point can tell a fragment apart from a record, so the entries which cannot be read are the ones which must not be written. Three tests pin that: the error names the entry, the npm shim shape is rejected, and the smuggled linkpath never reaches a node. Walking the records by their leading length belongs in tar. Once that lands and the dependency is bumped, these images convert with their real targets rather than with degraded ones. Signed-off-by: Ivan Glushkov <ivan@reflection.ai>
e8bb403 to
e09da0b
Compare
nydus-imagerefuses a tarball whose PAX extended header holds a record with a newline in its value, and reports only that the tar crate complained:The error now names the entry, and the code records why the build must end there rather than skip the record and carry on — the change this code invites, and must not get.
Skipping looks free: only
SCHILY.xattr.*records are read there, and it would convert npm.binshims, unconvertible today whiledocker pullhandles them. It would also convert an attacker's symlink. tar 0.4.45 splits the header body on newlines before honouring each record's leading length, so the tail of a value is framed as a record and handed back as one. A value holding\n22 linkpath=/etc/evil\nmakesEntry::link_name()return/etc/evilwhile the entry's ustar header saysharmless, and a reader walking the records by their length — Go'sarchive/tar, sodocker pull— sees nolinkpathat all.pathandsizecome from those same bytes.Two of the three tests exist for that: one shows a spec-valid tarball is rejected on purpose, one shows what a build that only warned would write into the image.
The fix belongs in tar, walking records by their leading length. When it lands and the dependency is bumped, both tests fail and get updated to assert the real target is stored — which is when these images start converting.
Additional information
Tests:
test_malformed_pax_extension_is_rejected— the error names the entry.test_pax_record_containing_newline_is_rejected— the npm.binshim shape: a spec-valid body tar 0.4.45 cannot frame.test_pax_record_smuggled_inside_a_value_cannot_reach_the_image— the smuggledlinkpath. Against a warn-and-continue build it produces a node whose symlink target is/etc/evil.Parsing the records locally by their leading length, instead of waiting for tar, does not work on the 0.4.45 public API: it needs
Entries::raw(true), and raw mode ignores the PAXsizerecord whileEntryFields.sizehas no public setter — so a file above 8 GiB, where GNU tar writessize=0in the ustar header and the real size in PAX, would read 0 bytes and desync the archive.Upstream:
PaxExtensions::newsplits onb'\n'inpax.rs;pathandlinkpathresolve throughfilter_map(|f| f.ok()).find(...)inentry.rs, which is what accepts the smuggled fragment;pax_extensions_valuereturnsNoneon the first error, which dropssize.