Repository navigation
mm: populate a subpage window's uffd-wp markers with the faulting page - #2225
jgoyvaerts wants to merge 2 commits into
Conversation
In an mm whose user page size is smaller than the native page, a write fault on unpopulated anonymous memory fills the whole native window from one fresh native page, each 4K page at its own offset. That keeps the window physically in VA order, and costs one native page for four user pages. A uffd-wp marker (UFFD_FEATURE_WP_UNPOPULATED) is unpopulated memory too, but anon_subpage_missing_mask() counted only none PTEs as missing. In a VMA write-protected before its first touch, every neighbour holds a marker, so each first write took a native page of its own for one 4K page: four times the memory, and no window was ever one native page in order. A device that maps native pages (a GPU with 16K pages taking user memory through GEM_USERPTR) can then use none of it. Count a uffd-wp marker in a uffd-wp VMA as missing, and give the neighbour it stood for the uffd-wp bit, as the faulting page already gets it from its own marker: the populated neighbours stay write protected and unwritten for PAGEMAP_SCAN. Markers in a VMA that is no longer registered are left as they were. Measured with a probe in a 4K process on an M4 Pro (16K kernel): 8 MiB of anonymous memory, write-protected through userfaultfd with WP_ASYNC | WP_UNPOPULATED before its first touch, then written once per 4K page. Before, 0 of 512 16K blocks were one native page in order; memory registered after its first touch, or never registered, gave 512 of 512. Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com> Assisted-by: LLM
Add a uffd-wp-init case that write-protects a native-aligned window with WP_ASYNC | WP_UNPOPULATED before any touch, then writes one byte to its second user page. pagemap must then show every user page of the window present in one native page in VA order, with the uffd-wp bit kept on the neighbours and cleared on the written page. markers() never reached this path: its window is not native-aligned, and its first touch is a read that maps the zero page, so its write is a copy-on-write fault. Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com> Assisted-by: LLM
|
Base control for the full Setup: arm64_user4k.config + Results:
Signature: the test kthread loops forever on a level-2 write translation fault at Why this change isn't involved:
So the stall is pre-existing and independent of #2225/#2226.
|
DjDeveloperr
left a comment
There was a problem hiding this comment.
Reviewed as core mm. The change is right and contained:
- It only reaches
do_anonymous_subpage(). With noVM_UFFD_WPthe mask and the PTEs written are identical to base: every non-target slot incurrent_maskis thenpte_none, so the new marker test is false. Native-16K processes and non-uffd users see no change. - The PTL-held recompute (mm/memory.c:6944) and the per-neighbour marker test in the set loop (6972-6973) both read under the PTL, so they agree. A racing population only shrinks
current_mask, and the dense-path abort is skipped for armed VMAs, so nothing is lost. - arm64
pte_mkuffd()write-protects, so a populated neighbour behaves like the marker it replaced: a WP fault in sync mode, a bit clear in async mode, and the marker comes back on a zap.
The stall control you posted settles #2114's question: base stalls 4/4, and the mm hunk here is byte-identical to 695bc5e. By construction this cannot reach a test that registers no userfaultfd.
Before merge, please post the negative run for the new case. Same uffd-wp-init binary, 4K worker mode, on a 16K kernel without 834e0e6 (0abc2f3 or a fleet kernel without the fix). I expect marker_window to fail at physical(a, ps) on page 0 with the earlier cases passing, then 29/29 on this head.
Two small things, neither blocking:
- Match the marker exactly (
softleaf_to_marker(...) == PTE_MARKER_UFFD_WP) rather than "has the UFFD_WP bit".handle_pte_marker()ranks GUARD and POISONED above UFFD_WP, and this path would turn such a slot into a zero page. Nothing creates the combination today, so this is hardening. - One sentence in the commit message: neighbours are now present (RSS,
PAGE_IS_PRESENT), and aUFFDIO_COPYinto one in a WP-only VMA returns-EEXISTwhere upstream would accept it. Both are already true fornoneneighbours on this path.
Optional for the test: after the first write, write a[0] and check that its bit 57 clears, pages 2 and 3 keep theirs, and physical() is unchanged.
|
Thanks, the change reads right as core mm: it only reaches One request before merge: could you add the uffd-wp case as a test that fails on a kernel without this fix (a subpage window registered for write-protect, first touch on one subpage, then check that the other subpages still report the marker)? A selftest under |
In an mm whose user page size is smaller than the native page (
CONFIG_MM_SUBPAGE: 4K processes on a 16K kernel, such as everything running under FEX), a write fault on unpopulated anonymous memory fills the whole native window from one fresh native page. Each 4K page lands at its own offset, so the window is physically in VA order.A uffd-wp marker (
UFFD_FEATURE_WP_UNPOPULATED) is unpopulated memory too, butanon_subpage_missing_mask()counted only none PTEs as missing.This change counts a uffd-wp marker in a uffd-wp VMA as missing, and gives the neighbour it stood for the uffd-wp bit, as the faulting page already gets from its own marker. The populated neighbours stay write-protected and unwritten for PAGEMAP_SCAN. Markers in a VMA that is no longer registered are left as they were.
The second commit adds a case to
tools/testing/selftests/arm64/user4k/uffd-wp-init.c. The existingmarkers()case reads before it writes, so its first touch maps the zero page and never reaches this path. The new case covers a native-aligned window, write-protected withWP_ASYNC | WP_UNPOPULATEDbefore any touch, then written once in its second 4K page:Tested on the J616s (M4 Pro, 16K kernel)
WP_ASYNC | WP_UNPOPULATEDbefore its first touch, then written once per 4K page. Before, 0 of 512 16K blocks were one native page in order; after, 512 of 512. Memory registered after its first touch, or never registered, gave 512 of 512 either way. PAGEMAP_SCAN's written/unwritten report stayed exact, native and 4K.On this branch,
mm/builds with W=1 and no warnings, and checkpatch reports 0 errors and 0 warnings on both commits.Not done
🤖 Generated with Claude Code