Scanning hardcoded config directories can trigger automounts (boot hang with network-backed /usr/local)

[jstancek]

Problem

libkmod and the kmod tools scan a hardcoded list of configuration directories on
every context creation. For modprobe.d this list is (libkmod/libkmod.c,
default_config_paths):

  /etc/modprobe.d
  /run/modprobe.d
  /usr/local/lib/modprobe.d
  /usr/lib/modprobe.d (DISTCONFDIR)
  /lib/modprobe.d

depmod scans the equivalent depmod.d list, which also contains
/usr/local/lib/depmod.d.

The scan simply calls stat()/opendir() on each path. If any component of one
of these paths is an unmounted automount point (autofs or a systemd
.automount unit), the lookup triggers the mount. /usr/local is the
problematic case in practice: it is common to keep it on NFS behind an
automount, and there is no way to make kmod skip it — the config-dir list can
only be replaced via modprobe -C / depmod -C or the kmod_new()
config_paths argument, and the main early-boot consumers (systemd-udevd,
systemd-modules-load) call kmod_new(NULL, NULL) and always get the built-in
defaults.

Consequence is an early boot hang.

Expected behavior

Scanning for optional configuration directories should not force automounts.
A directory whose path crosses an unmounted automount point could be skipped
(the kernel exposes this via statx() STATX_ATTR_AUTOMOUNT +
AT_NO_AUTOMOUNT); its configuration would then simply be honored once the
filesystem is mounted by other means, matching how a nonexistent directory is
treated today.

[jstancek]

I'm leaning towards documenting this as unsupported and known limitation of network automounts on directories used in early-boot, I've opened systemd/systemd#43794 to see if doc could be updated to make this more clear.