Skip to content

Fix PBS filesystem durability, tier enumeration and interrupted movement recovery - #209

Merged
JBailes merged 7 commits into
mainfrom
fix/pbs-storage-durability
Oct 31, 2026
Merged

JBailes merged 7 commits into
mainfrom
fix/pbs-storage-durability

Conversation

@JBailes

@JBailes JBailes commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

PBS failure testing exposed filesystem durability and movement bugs that could lose manifests/chunks or leave objects unable to move again. This change implements merged-directory fsync, flushes every backing filesystem on syncfs, prevents inode-reuse corruption, serializes repeated movement through durable completion/rollback acknowledgements, preserves enumeration across backing-tier permissions, and fixes mkdir error cleanup on supported kernels.

Startup recovery now persists intent before planning and reconciles interrupted work against live kernel placement, including a cold post-boot OID index. It syncs cleanup before acknowledging completion or rollback and leaves uncertain recovery pending.

Validation: Go tests, race checks, vet/static/script/package checks, and kernel builds on Linux 6.18 and 7.2 passed. Real XFS-backed PBS tests covered four worker-exit phases, stale/invalid rollback guards, directory sync, inode reuse, ENOSPC, tier I/O failure during GC, corruption detection, repeated movement and forced power-off. The patched PBS candidate retained all 12 snapshots, 23 indexes and 391 referenced chunks through both full power-loss repeats; native CT/VM and file restores passed.

Merge with a merge commit so e85ea3a remains an ancestor of the release tag required by SmoothNAS package gating. Publishing a release and its Debian packages remains necessary before appliance shipment. No automatic review settings are changed.

Add an explicit MOVE_COMPLETE acknowledgement after source unlink and directory fsync. Keep switched state until cleanup succeeds, reject stale or premature acknowledgements, and allow repeat moves without eviction or remount. Cover worker cleanup errors and repeated live kernel movement with a held descriptor.
Linux 6.18 leaves the lower parent locked on mkdir error; 6.19 and newer consume the dentry and unlock it. Normalize this in the compatibility helper and remove the materialization path's double unlock. Add an ENOSPC regression for both parent materialization and direct mkdir followed by lookups and write recovery. The old module hung the regression; the fixed module passes on Linux 7.2.11, with both 6.18 and 7.2 builds checked.
Run merged-directory traversal with the same privileged backing credentials as lookup, preserving upper DAC checks. Caller credentials silently hid spill-tier snapshots below private backing parents from PBS, letting verification skip them and GC delete their live chunks. Propagate contributing-tier traversal/read errors instead of returning a partial listing. Cover UID 1000, bind mounts, upper DAC denial, and lookup-error propagation.
Persist full-sync intent before the first kernel transition. Recover using live placement, durable cleanup and sequence-guarded completion or rollback; instantiate persisted names when the recovery index is cold after reboot. Keep uncertain or failed acknowledgements pending instead of reporting placed.
A stacked syncfs barrier must flush lower address spaces, not just SmoothFS metadata. Synchronize each distinct lower superblock and preserve the first reported error. Add a real XFS delayed-allocation regression that fails on the old callback and passes after the lower flushes.
@JBailes
JBailes merged commit cd2e33e into main Oct 31, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant