Support fuse with overlayfs#328
Conversation
Add a follow_redirects option to the FUSE server that controls whether the server serves file content directly from the repository (original behavior) or acts as an overlay-lower layer by synthesizing user.overlay.* xattrs. The goal here is to use this with a userxattr overlayfs mount which should improve performance for accessing file content (goes directly via kernel like a rootful composefs mount). Assisted-by: Claude Code (Opus 4.6) Signed-off-by: Alexander Larsson <alexl@redhat.com>
Add mount_fuse_overlay() which creates an overlayfs on top of a FUSE mount, using userxattr mode and data-only lower layers for file content. The FUSE server must already be running before calling this, since overlayfs probes the lower layer during setup. OverlayMountOptions controls the overlay configuration: upper/work dirs for writable mounts, read-write mode, and fs-verity enforcement. This is needed if using mount_fuse() that doesn't follow redirects. Assisted-by: Claude Code (Opus 4.6) Signed-off-by: Alexander Larsson <alexl@redhat.com>
|
Maybe some overlap with #306 ? |
|
@cgwalters I don't think there is necessary an overlap, except perhaps that the passthrough support is intended to solve the same performance issue (but is not useful as it is root only). They are however complementary, in that the readdir+ and multi-threading will increase metadata performance. |
|
That said, there might be code conflicts in the two, i'll have a look at that. |
| if let Some(rest) = name.strip_prefix(XATTR_USER_OVERLAY_PREFIX) { | ||
| [XATTR_USER_OVERLAY_ESCAPED_PREFIX, rest].concat() | ||
| } else { | ||
| name.to_vec() |
| let lookup_name = if self.follow_redirects { | ||
| name.to_os_string() | ||
| } else { | ||
| OsStr::from_bytes(&escape_xattr_name(name_bytes)).to_os_string() |
There was a problem hiding this comment.
I think this is an unnecessary copy
| /// When true, the server follows overlay redirects and serves file | ||
| /// content from the repository. When false, it synthesizes | ||
| /// `user.overlay.*` xattrs for use as an overlayfs lower layer. | ||
| /// Defaults to false. |
There was a problem hiding this comment.
But actually...I am not sure we considered this at all but - is there any reason at all to have the direct serving? I am not sure there is a strong one...I think the FUSE implementation was kind of intended mainly for unprivileged mounts, but since nowadays overlay is allowed in user namespaces, I think we should automatically set "follow_redirects" = false if running unprivileged right?
Actually to say it a different way - don't we always want to just always use overlayfs, and only have the FUSE for unprivileged EROFS? In that case, when running privileged, we should not use user.overlay right? Or I guess we still can, but there's no reason to?
There was a problem hiding this comment.
Honestly I'm not sure here. There is one way that follow_redirects mode is useful, in that it allows any non-root user to mount a composefs image in the "root" user namespace.
Like, if you're in the shell you can just mount a cfs and have some other program look into the mount. You cannot do that with an overlay mount. For that you need to be in a new user namespace (where you have cap_sysadmin) so you can do the mount. And if you create a new user+mnt namespace programs in the root namespace can't see your mount. So, non-folllow_redirect is primarily useful for container like tools, wheras follow_redirect is for traditional commandline work.
So, I do think it would make sense to support both of these modes. I wonder though if the current composefs fuse implementation is ideal for this kind of use. It starts by parsing the entire erofs into a filesystem tree, which adds quite some latency. An implementation that just reads the erofs file into memory and does metadata lookups directly from that would probably be faster at startup. And, for a metadata-only implementation (non-follow-redirects) that would probably be pretty easy to implement, as all you have to do is read inode info.
I guess that could be later work though.
There was a problem hiding this comment.
+1 to make the metadata-only serving the default.
3 participants
This adds support for a fuse version that only serves the erofs metadata, and helpers to mount an overlayfs using this with userxattrs. This allows rootless mounting of a composefs image where file content (but not metadata) is directly accessed from the regular fs by the kernel and should perform similar to the rootful overlayfs mount.