rootless `--in-vm` bootc disk image generation fails with SELinux

[jbtrystram]

Trying to build a bootc disk image rootless using the new --in-vm flag :

ibc build qcow2 \                                                                                                                                            tekton  ✭
          --in-vm \
          --bootc-build-ref $BUILDER \
          --bootc-ref $TARGET_FCOS_IMAGE \
          --output-name fedora-coreos-rawhide \
          --output-dir output \
          --with-buildlog --with-manifest \
          --blueprint /srv/shared/blueprint.toml \
          --blueprint /srv/qemu/x86_64.toml

Manifest generation step
Building manifest for bootc-based-qcow2
Image building step
starting -Pipeline source org.osbuild.containers-storage: 0079d3e26b7100b691705910fb02815586246c11b4bcff17e455be7741ce6bb4
Build
  root: <host>
Pipeline target: 1a68fb1a34ab1c055377e67097c58060f09e7df2a3483577a5eb7f2d94d992e1
Build
  root: <host>
  runner: org.osbuild.fedora38 (org.osbuild.fedora38)
org.osbuild.container-deploy: 3f49744f03469239837affabb6ae2cc3318ec0c0590801028be6ed896f22d2ba {
  "remove-signatures": true
}
075c3ef3bd4cb4de9a23cf92a141f58c0b8f8bf525c86cad2f0fa46571eee94e
umount: /run/osbuild/containers/storage/overlay: not mounted.
WARNING: umount of overlay dir failed with an error: CompletedProcess(args=['umount', '-f', '--lazy', '/run/osbuild/containers/storage/overlay'], returncode=32)

⏱  Duration: 7.71s
org.osbuild.mkdir: 8ad59a9faebf034417cd3cffc15a693bf520e2a5eef30cc46f73635676bdd709 {
  "paths": [
    {
      "path": "/usr/lib/bootc/install",
      "parents": true,
      "exist_ok": true
    },
    {
      "path": "/usr/lib/ostree",
      "parents": true,
      "exist_ok": true
    }
  ]
}

⏱  Duration: 0.23s
org.osbuild.selinux: 1a68fb1a34ab1c055377e67097c58060f09e7df2a3483577a5eb7f2d94d992e1 {
  "file_contexts": "etc/selinux/targeted/contexts/files/file_contexts",
  "exclude_paths": [
    "/sysroot"
  ],
  "labels": {
    "/usr/bin/mount": "system_u:object_r:install_exec_t:s0",
    "/usr/bin/ostree": "system_u:object_r:install_exec_t:s0",
    "/usr/bin/umount": "system_u:object_r:install_exec_t:s0"
  }
}
setfiles: Could not set context for /run/osbuild/tree/usr/bin/pesto:  Invalid argument
Traceback (most recent call last):
  File "/run/osbuild/bin/org.osbuild.selinux", line 45, in <module>
    r = main(osbuild.api.arguments())
  File "/run/osbuild/bin/org.osbuild.selinux", line 26, in main
    selinux.setfiles(file_contexts, os.path.normpath(root), target, exclude_paths=exclude_paths)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/run/osbuild/lib/osbuild/util/selinux.py", line 55, in setfiles
    subprocess.run(["setfiles", "-F",
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
                    "-r", root,
                    ^^^^^^^^^^^
    ...<2 lines>...
                    f"{root}{path}"],
                    ^^^^^^^^^^^^^^^^^
                   check=True)
                   ^^^^^^^^^^^
  File "/usr/lib64/python3.14/subprocess.py", line 578, in run
    raise CalledProcessError(retcode, process.args,
                             output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['setfiles', '-F', '-r', '/run/osbuild/tree', '-e', '/run/osbuild/tree/sysroot', '/run/osbuild/tree/etc/selinux/targeted/contexts/files/file_contexts', '/run/osbuild/tree/']' returned non-zero exit status 255.

⏱  Duration: 0.81s
manifest - failed
Failed
error: error running osbuild: exit status 1

[supakeen]

This is due to the host not knowing the policy that's trying to be applied if I recall this correctly. I'll consider it a bug.

@alexlarsson and the automotive team have been using this the most, perhaps they can share how they're using it at the moment?

[alexlarsson]

Yes, this is caused by seliux not allowing the virtiofsd process (from inside the vm) to set a selinux label on a file if the label is not know to the active selinux policy in the host kernel. This is unfortunately impossible to fix for a non-root virtiofsd process. The only way to allow this is to use a custom selinux context for the process that allows self:capability2 mac_admin, which is normally not even allowed by regular unconfined_t, or with non-enforcing mode. In osbuild normally this is handled by osbuild running as osbuild_container_t.

In automotive, we generally just run into this case with the qm_t label, and fix it by installing qm-selinux on the host. However, this is clearly not ideal. I have some vague ideas how we could work around this in virtifsd, but nothing that is really workable atm.