Problems with systemd-nspawn in containers with apparmor enabled #1321
Comments
|
happens on both lts 6.0.2 and non-lts 6.6 |
|
Systemd issue: systemd/systemd#34836 |
|
This is probably an issue due to the use of In general you should avoid those containers like the plague, especially in a world where we also support just running a virtual machine for such cases. In an unprivileged container, this still confusingly fails with: This is pretty confusing as as far as I can tell, it's just the equivalent of: There is no Apparmor mount restrictions on unprivileged containers, so it's not apparmor preventing this action when run by nspawn, but the kernel itself. Actually looking at a full strace of nspawn, it looks like it's running under the equivalent of: Yet this still works when done manually, so it's something else being done as part of nspawn container initialization which is causing a situation where the kernel refuses to mount /proc. What's for sure is that I wouldn't expect the systemd folks to care at all about the privileged container use case, it's running under all kind of weird apparmor policies and tricks. The unprivileged container case is a bit cleaner, so if it's possible to track down the exact condition that's causing nspawn to fail to mount /proc, it may be possible to have it tweak that behavior slightly in such an environment. |
|
Marking as incomplete on our side given that under an unprivileged environment, we don't do any mount mediation or similar overrides, so systemd-nspawn failing in that environment likely means it's just generally failing when run from within an unprivileged container, whether that's Incus or something else having created it. Now whether this is something that upstream systemd-nspawn is interested in supporting will be up to them, if not, we conveniently also support running NixOS as a VM on Incus which should then work just fine. |
|
@stgraber The |
|
Yeah, I did try uncovering /proc, that wasn't it. And indeed when nesting is enabled we make sure that an uncovered version of /proc and /sys is present in the mount table (hidden in /dev) so that the overmounting check doesn't block us. |
Required information
Issue description
When using systemd-nspawn in a container that has security.privileged and security.nesting set to true, nspawn still fails with a mysterious failure:
Failed to mount sysfs (type sysfs) on /sys/full (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC ""): No such file or directory
This is caused by systemd-nspawn trying to mkdir a directory and failing:
https://github.com/systemd/systemd/blob/main/src/nspawn/nspawn-mount.c#L474-L478
This is caused by one of the rules here blocking it, as they match /sys/full: https://github.com/lxc/incus/blob/main/internal/server/apparmor/instance_lxc.go#L332-L426
note that at that point /sys is a tmpfs, /sys/full is a directory created there and later (i think) /sys/full is mounted into /sys
I'm not sure if this is a thing incus should fix or systemd.
Steps to reproduce
(for full reproduction use noble ubuntu desktop vm and zabbly incus, but it should work anywhere with incus + working apparmor)
systemd-nspawn --keep-unit -M dokuwiki -D /tmp --private-network --network-veth --notify-ready=yes --kill-signal=SIGRTMIN+3 --bind-ro=/nix/store --bind-ro=/nix/var/nix/db --bind-ro=/nix/var/nix/daemon-socket --link-journal=try-guestwill produce error:
this is just enough from the nspawn command to cause the error:
Failed to mount sysfs (type sysfs) on /sys/full (MS_RDONLY|MS_NOSUID|MS_NODEV|MS_NOEXEC ""): No such file or directoryInformation to attach
dmesg)incus info NAME --show-log)incus config show NAME --expanded)incus monitor --prettywhile reproducing the issue)The text was updated successfully, but these errors were encountered: