Skip to content

P105: integrate SwissPPNFTL reclamation into normal writes - #93

Draft
shaynelegault wants to merge 8 commits into
OrchardLinux:mainfrom
shaynelegault:p105/08-ftl-reclamation
Draft

shaynelegault wants to merge 8 commits into
OrchardLinux:mainfrom
shaynelegault:p105/08-ftl-reclamation

Conversation

@shaynelegault

@shaynelegault shaynelegault commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

This draft adds garbage collection to the normal write path of the 64 GB P105 SwissPPNFTL driver. The writer copies live data, publishes a native-format checkpoint, recovers the map and verifies the retired blocks before reuse. The full transaction runs on synthetic NAND. Physical P105 writes still append into verified blank blocks, and the installer retains its space warning.

Changes

  • Preserve the original user-sequence bits when relocating live pages. Verify ownership across the whole device map, including the 16 internal keys beyond the disk. Qualifying a clean-free destination requires zero saved mappings and a full data/OOB blank check.
  • Reserve a fresh two-block context group. After the verified base page, sample both controller statistics records. Read back each checkpoint page, recover with both generations visible, retire the old occupied context lane by lane, and cold-load again before donor erase. Reconcile wear before making a released block reusable.
  • Recompute block live counts, the global valid-page count and per-block user-sequence summaries. Preserve the native timing of cached allocation counters. Support healthy four-replica VFL bank rotation, including a source table update that can follow the latest complete checkpoint.
  • Keep physical P105 erase and SLC rotation disabled. The device capability remains reclaims = false.

Verification

  • An independent replay of the original 13G36 firmware selected donor 3 from two candidates, relocated four live keys (101–104) to block 32, updated the map and saved a checkpoint using fresh context blocks 33 and 34. The 88-byte VFL and 464-byte controller records matched the Linux accounting code byte for byte. NAND responses and the initial state were modeled in RAM.
  • Across separate comparisons, all 552 statistics bytes matched at 426 sampling points, including counter rollover. Allocation and ordered erase decisions matched across 600 block tables. Donor finalization matched across 341 captured block records.
  • The C reader recovered all 15,625,016 keys in the archived image read-only. With the missing context tails explicitly modeled as erased on a RAM overlay, it recovered 22,455 ranges through all 95 incomplete checkpoint prefixes and 15 partial old-context retirements. Healthy VFL rotation reproduced all 72 captured target pages, including OOB, and preserved all 256 source pages.
  • All 72 host sanitizer tests, 20 ARMv7 object configurations and 50 catalog tests passed. All 45 ARMv7 patches reproduced the kernel source tree from pinned Linux v6.12. Both ARM64 series retained their 651 original patches.

Kernel source · P105 bring-up evidence · GitHub Actions

Remaining before merge

Physical qualification still needs trustworthy runtime controller accounting, exclusive ownership and a measured erase footprint in normal and SLC modes. Cold-boot reuse and preservation of existing iOS data remain to be demonstrated on the P105. Scrub and bad-member replacement also remain unqualified.

@shaynelegault
shaynelegault force-pushed the p105/08-ftl-reclamation branch from e1c4d55 to 9e0fd8e Compare November 6, 2026 23:43
@shaynelegault
shaynelegault marked this pull request as ready for review November 7, 2026 05:08
henrikklee added a commit that referenced this pull request Nov 7, 2026
Add the append writes of #91 to kernel/patches/armv7/mtd: the iOS disk
of the 64 GB iPad mini 1 becomes writable. Writing is part of the driver;
the default-off MTD_APPLE_SFTL_WRITE option and the write_enable module
parameter are gone. The disk stays read-only only when the replayed
journal does not end at the allocation floor. Without reclamation (#93),
writes fail once the blank blocks are used up. The patch keeps Shayne
Legault as author and notes the changes and his original commit IDs.
The bring-up notes and the board README follow the driver as carried.

Assisted-by: LLM
henrikklee added a commit to shaynelegault/hardware that referenced this pull request Nov 7, 2026
Add the append writes of OrchardLinux#91 to kernel/patches/armv7/mtd: the iOS disk
of the 64 GB iPad mini 1 becomes writable. Writing is part of the driver;
the default-off MTD_APPLE_SFTL_WRITE option and the write_enable module
parameter are gone. The disk stays read-only only when the replayed
journal does not end at the allocation floor. Without reclamation (OrchardLinux#93),
writes fail once the blank blocks are used up. The patch keeps Shayne
Legault as author and notes the changes and his original commit IDs.
The bring-up notes and the board README follow the driver as carried.

Assisted-by: LLM
@henrikklee
henrikklee marked this pull request as draft November 17, 2026 16:47
@henrikklee

Copy link
Copy Markdown
Contributor

Thanks, Shayne. Marking this as a draft until reclamation is finished: we'll merge it once the erase and GC side that uses this planner is in place and it runs end to end. When you get there, 32-bit kernel patches now live in kernel/patches/armv7/mtd/ (listed in kernel/series/apple-armv7), without .patch.toml scope files.

Export the offline SwissPPNFTL checkpoint and reclamation work from the merged ARMv7 append writer. Carry the kernel changes in the ARMv7 MTD series and document the current append-only behavior and remaining runtime work.

Assisted-by: LLM
@shaynelegault
shaynelegault force-pushed the p105/08-ftl-reclamation branch from 9e0fd8e to 3aff97d Compare November 19, 2026 03:38
Export the verified H2FMI erase operation, live page copier and SFTL allocator from the ARMv7 kernel at ccd56d3c440c. Record the synthetic end-to-end tests and the native checkpoint and hardware qualification still required for P105.

Assisted-by: LLM
@shaynelegault shaynelegault changed the title P105: offline SwissPPNFTL checkpoint encoder and GC transaction planner P105: integrate SwissPPNFTL reclamation into normal writes Nov 19, 2026
Export the native-format save and recovery path, healthy VFL rotation and
block-accounting guards to the ARMv7 MTD series. Record the captured-media
and interruption results, and the provider accounting still needed before
enabling physical erase and reuse.

Assisted-by: LLM
Export the native provider counters, free-block and wear transitions,
fresh context allocation and checkpoint publication order. Record the
firmware replay, the integrity checks for unlisted context sites and the
remaining physical qualification requirements for erase and reuse.

Assisted-by: LLM
Serialize raw programs against the lifetime of the recovered SFTL map. Stage
a board-specific module to capture both physical views and controller status,
and compare paired observations along with adjacent and same-index controls.
Publish matching PR kernel and measurement module artifacts for device tests.

Keep physical erase disabled pending measured coverage, continuous runtime
accounting and cold-boot preservation.

Assisted-by: LLM
Use the checkout SHA provided by the pull request job when packaging the
P105 kernel and its capture module. The runner and the build container use
different owners for the worktree.

Assisted-by: LLM
Stage the ARMv7 measurement build in the kernel directory layout consumed
by the RAM project. Include the linked kernel notes digest so a running
build can be identified independently of the release string.

Let hardware-check start on kernels without the optional hotplug sysctl.
Fence the P105 SFTL disk and partitions as they register and verify their
mount and read-only state around the bounded storage probe.

Assisted-by: LLM
@henrikklee

Copy link
Copy Markdown
Contributor

@shaynelegault, a heads-up so nothing collides with your work here.

Project Cascadia now has its own P105 NAND writer (rzxvx's apple_ppn/apple_ftl, at Cascadia ad28ba4). It runs on the 16 GB iPad mini with iOS 8.4.1, and one full write, reclaim and iOS reboot cycle has completed on hardware. We compared it with your merged stack and with this PR: p105-nand-merge-proposal.md.

The proposal keeps your structure (H2FMI provider, apple-sftl, LwVM parser, then this PR) as the base. On top of it we'd port, as separate commits credited to rzxvx:

  • 16 GB geometry and iOS 8.4.1 context compatibility
  • giving up only the failed superblock on a program/erase failure instead of stopping the bus
  • torn-superblock handling at mount
  • a few controller details
  • test-only on-device fault injection

That work goes on p105-nand-fusion, branched from your head bd8916c. Your branch stays untouched; we rebase when you push, and this PR lands first as yours.

You're welcome to review it, take any of it over, or tell us to hold off on a part you're already doing.

henrikklee added a commit that referenced this pull request Nov 19, 2026
The P105's translation layer registered only on the 64 GB NAND package.
Port Cascadia's 16 GB layout into it (patch 0018, authored by rzxvx):
four banks of 1064 blocks of 256 pages, 1055 superblocks, the VFL in
blocks 1062/1063, iOS 8.4.1's context and summary rules. A 16 GB P105 now
gets a read-only applesftl disk of 16,000,000,000 bytes. 0017 first moves
#93's fixed 64 GB constants into a geometry descriptor, unchanged in
behaviour for the 64 GB package.

KUnit gains a synthetic 16 GB medium (3 cases); all 84 apple-sftl and
apple-h2fmi cases pass. Untested on hardware.

Assisted-by: LLM
Capture the native iOS counters over paired USB and reject mismatched flash geometry. Document the verified raw-write exclusion and the iOS pages read through both operating systems. Keep reclamation pending its physical qualification.

Assisted-by: LLM
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants