Documentation request/clarification: ZFS datasets inside jails

[sporkman]

First I have to say I am so, so happy to see this project being kept alive and being brought (somewhat?) under the FreeBSD project's umbrella. I strongly feel like FreeBSD often misses out on users who would find technology like jails or bhyve interesting because the stock OS doesn't have a user-friendly tool for creation and ongoing management of jails or VMs (and bhyve is wonderful, but I don't think any masochist out there is using it w/o a front-end).

Anyhow, I'm finding that for stuff like running PostgreSQL in a jail, or media sharing/file sharing stuff it's very helpful to have some extra zfs datasets in the jail. And iocage supports this, for sure. But I always, always set it up, come back a few years later to setup another one and cannot for the life of me remember what I did to get it working, just that I failed many times on the way. Also I think there have been some bugs squashed in that area since the project was revived.

I'd be willing to help write something for the docs (is this official or no? https://freebsd.github.io/iocage/) if I can get a little input on what the right procedure is. Here's an example that works, and I totally forget how I arrived at this:

{
    "allow_raw_sockets": 1,
    "basejail": 1,
    "boot": 1,
    "host_hostname": "db03",
    "host_hostuuid": "db03",
    "ip4_addr": "int0|1.2.3.4",
    "jail_zfs": 1,
    "jail_zfs_dataset": "db03_mount",
    "last_started": "2022-08-26 08:12:20",
    "release": "12.3-RELEASE-p5" <- don't mock me
}

my view of the zfs datasets from outside the jail:

data/iocage/jails/db03                  67.7G  1.36T   120K  /data/iocage/jails/db03
data/iocage/jails/db03/root             67.7G  1.36T  27.8G  /data/iocage/jails/db03/root
…
data/db03_mount                         90.5G  1.36T    96K  none
data/db03_mount/PG                      90.5G  1.36T    96K  /PG
data/db03_mount/PG/pgdata               90.5G  1.36T  84.7G  /PG/pgdata

And all mounts inside the jail, both via "zfs list" and "mount":

[root@db03 /PG/pgdata/pg14]# zfs list
NAME                        USED  AVAIL  REFER  MOUNTPOINT
data                        328G  1.36T    96K  /data
data/db03_mount            90.5G  1.36T    96K  none
data/db03_mount/PG         90.5G  1.36T    96K  /PG
data/db03_mount/PG/pgdata  90.5G  1.36T  84.7G  /PG/pgdata
[root@db03 /PG/pgdata/pg14]# mount
data/iocage/jails/db03/root on / (zfs, local, nfsv4acls)
/data/iocage/releases/12.3-RELEASE/root/bin on /bin (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/boot on /boot (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/lib on /lib (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/libexec on /libexec (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/rescue on /rescue (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/sbin on /sbin (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/usr/bin on /usr/bin (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/usr/include on /usr/include (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/usr/lib on /usr/lib (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/usr/libexec on /usr/libexec (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/usr/sbin on /usr/sbin (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/usr/share on /usr/share (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/usr/libdata on /usr/libdata (nullfs, local, read-only)
/data/iocage/releases/12.3-RELEASE/root/usr/lib32 on /usr/lib32 (nullfs, local, read-only)
devfs on /dev (devfs, local, multilabel)
fdescfs on /dev/fd (fdescfs)
data/db03_mount/PG on /PG (zfs, local, noatime, nfsv4acls)
data/db03_mount/PG/pgdata on /PG/pgdata (zfs, local, noatime, nfsv4acls)
[root@db03 /PG/pgdata/pg14]#

When I start googling for answers on this I'm mostly getting really old info and lots of conflicting info and some workarounds that feel a little unofficial and outdated.

I know technically this isn't a place for Q&A, but I also don't know where to go for that other than here. :)

[dgeo]

That is a really good question.
Here I never explicitely set jail_zfs_dataset, and jailed zfs is always in zpool/iocage/jails/myjail/data (this is a default value).

https://freebsd.github.io/iocage/ is the official doc, generated from this repo. Nothing automated to generate html for now.

Source files are in doc/, and docs/ is a copy of make html output, served at https://freebsd.github.io/iocage/ .

Feel free to PR !