Add new incus module, add required rules to other modules #6

mschiff · 2024-12-06T11:27:24Z

This PR adds a new incus module I have developed. It is a first version, which is working for me so far with my current test set which creates, uses and manages (start,stop, restart, destroy) VMs and LXC containers.

The system I use to develop and test the changes is running with profile default/linux/amd64/23.0/hardened/selinux/systemd

As this test set does not cover every feature it is likely that more tweaks and changes need to be added in the future as issues may show up.

If you have comments and recommondations on further edits I am happy to update that PR.

systemd-network-generator.service unit fails without: fs_list_tmpfs(systemd_networkd_t) allow rw to /sys/fs/cgroup/system.slice/systemd-networkd.service/memory.pressure fs_rw_cgroup_files(systemd_networkd_t) Signed-off-by: Marc Schiffbauer <[email protected]>

mschiff · 2024-12-09T23:04:10Z

One note regarding MCS: LXC containers currently need to use spc_t (super privileged container) domain in this first version and MCS does not apply to spc_t.

container_t OTOH would work with MCS but was designed for docker/podman process style containers. But e.g. systemd won't boot in a container running as container_t without many more allow rules added. Therefore I think it would be best to create a new incus_lxc_t type which will be quite similar to the virt_lxc_t type from the virt module adapted to work with incusd.

0xC0ncord

Looking good so far! Just a couple questions. While we're still working on getting things working I've also left out comments on style.

0xC0ncord · 2024-12-18T16:51:34Z

policy/modules/apps/qemu.te

@@ -47,6 +47,34 @@ tunable_policy(`qemu_full_network',`
 	corenet_tcp_connect_all_ports(qemu_t)
 ')

+#tunable_policy(`qemu_managed_by_incus',`


This probably should be a tunable since we're giving some pretty powerful permissions here. qemu_enable_incus or qemu_incus_managed I think.

Ok. It works except for storage_raw_read_fixed_disk(qemu_t) and storage_raw_read_fixed_disk(qemu_t) which do not seem possible inside a tunable block:

qemu.te:58:ERROR 'syntax error' at token 'typeattribute' on line 6351: #line 58 typeattribute qemu_t fixed_disk_raw_read; /usr/bin/checkmodule: error(s) encountered while parsing configuration make: *** [Makefile:166: tmp/qemu.mod] Error 1

Ah, I forgot about this. I don't remember if there's a way to give this access conditionally under normal circumstances.

0xC0ncord · 2024-12-18T16:59:44Z

policy/modules/services/incus.te

+# no error, but this AVC:
+#  scontext="system_u:system_r:incusd_t:s0" #  tcontext="system_u:system_r:iptables_t:s0" #  class="process"
+#  perms="sigkill" #  comm="incusd" exe="" path=""
+# candidate for dontaudit instead?
+iptables_admin(incusd_t,system_r)


This should be changed to iptables_domtrans(incusd_t).

If I try I still get:

#============= incusd_t ============== # audit(1734635352.998:154972): # scontext="system_u:system_r:incusd_t:s0" tcontext="system_u:system_r:iptables_t:s0" # class="process" perms="sigkill" # comm="incusd" exe="" path="" # message="type=AVC msg=audit(1734635352.998:154972): avc: denied { sigkill } # for pid=1077723 comm="incusd" scontext=system_u:system_r:incusd_t:s0 # tcontext=system_u:system_r:iptables_t:s0 tclass=process permissive=0 " allow incusd_t iptables_t:process sigkill;

We'll probably need to add new interfaces to iptables for these missing rules then. We can't use iptables_admin() because it requires passing in system_r which the incus module doesn't own.

It works, sure, but it's unpalatable for usptream.

0xC0ncord · 2024-12-18T17:02:35Z

policy/modules/services/incus.te

+# watch /dev/hugepages
+files_watch_all_dirs(incusd_t)


Is this just for hugepages? We can create a new interface for it like fs_watch_hugetlbfs_dirs().

I removed the line noe because I cannot reproduce this denial anymore. Maybe due to an incus update, not sure. But if it reappears I will add that interface for it.

0xC0ncord · 2024-12-18T17:04:17Z

policy/modules/services/incus.te

+# incusd needs support to label VM files before launching
+# until then, VM files will unfortunately be unlabeled
+kernel_manage_unlabeled_dirs(incusd_t)
+kernel_manage_unlabeled_files(incusd_t)
+kernel_manage_unlabeled_symlinks(incusd_t)
+kernel_mounton_unlabeled_dirs(incusd_t)
+files_search_default(incusd_t)


Does this fuctionality currently exist in incus?

Incus is still missing some selinux awareness, e.g. label some objects properly before launching a qemu VM. Until it will get support for that we can remove these kernel_manage_* and (I guess) files_search_default again.

Does this answer your question?

Some more details:

# audit(1734698529.636:163140): # scontext="system_u:system_r:incusd_t:s0" tcontext="system_u:object_r:unlabeled_t:s0" # class="dir" perms="getattr" # comm="incusd" exe="" path="" # message="type=AVC msg=audit(1734698529.636:163140): avc: denied { getattr } # for pid=1166765 comm="incusd" path="/var/lib/incus/storage- # pools/default/virtual-machines/vm1" dev="zfs" ino=34 # scontext=system_u:system_r:incusd_t:s0 # tcontext=system_u:object_r:unlabeled_t:s0 tclass=dir permissive=0 " allow incusd_t unlabeled_t:dir getattr; (sysadm_r)@host ~ # mount|grep vm1 zdata/incus/virtual-machines/vm1 on /var/lib/incus/storage-pools/default/virtual-machines/vm1 type zfs (rw,relatime,seclabel,xattr,posixacl,casesensitive) zdata/incus/virtual-machines/vm1 on /var/lib/incus/devices/vm1/config.mount type zfs (ro,relatime,seclabel,xattr,posixacl,casesensitive) (sysadm_r)@host ~ # ls -lZR /var/lib/incus/virtual-machines /var/lib/incus/virtual-machines: total 9 lrwxrwxrwx. 1 root root system_u:object_r:container_var_lib_t:s0 57 20. Dez 13:44 vm1 -> /var/lib/incus/storage-pools/default/virtual-machines/vm1 (sysadm_r)@host ~/selinux/local # ls -lZRd /var/lib/incus/virtual-machines/vm1 lrwxrwxrwx. 1 root root system_u:object_r:container_var_lib_t:s0 57 20. Dez 13:44 /var/lib/incus/virtual-machines/vm1 -> /var/lib/incus/storage-pools/default/virtual-machines/vm1 (sysadm_r)@host ~ # (sysadm_r)@host ~ # ls -lZR /var/lib/incus/storage-pools/default/virtual-machines/vm1 /var/lib/incus/storage-pools/default/virtual-machines/vm1: ls: cannot read symbolic link '/var/lib/incus/storage-pools/default/virtual-machines/vm1/qemu.nvram': Permission denied total 73 -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 688 20. Dez 13:44 agent-client.crt -rw-------. 1 root root system_u:object_r:unlabeled_t:s0 288 20. Dez 13:44 agent-client.key -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 725 20. Dez 13:44 agent.crt -rw-------. 1 root root system_u:object_r:unlabeled_t:s0 288 20. Dez 13:44 agent.key -r--------. 1 root root system_u:object_r:unlabeled_t:s0 2762 20. Dez 13:44 backup.yaml dr-x------. 6 nobody root system_u:object_r:unlabeled_t:s0 14 20. Dez 13:44 config -rw-------. 1 nobody root system_u:object_r:unlabeled_t:s0 540672 20. Dez 13:44 edk2-i386-vars.fd -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 526 20. Dez 08:48 metadata.yaml lrwxrwxrwx. 1 root root system_u:object_r:unlabeled_t:s0 17 20. Dez 13:44 qemu.nvram drwxr-xr-x. 2 root root system_u:object_r:unlabeled_t:s0 4 20. Dez 08:48 templates total 13718 -r--------. 1 nobody root system_u:object_r:unlabeled_t:s0 2 20. Dez 13:44 agent-mounts.json -rw-r--r--. 1 nobody root system_u:object_r:unlabeled_t:s0 818 20. Dez 13:44 agent.conf -r--------. 1 nobody root system_u:object_r:unlabeled_t:s0 725 20. Dez 13:44 agent.crt -r--------. 1 nobody root system_u:object_r:unlabeled_t:s0 288 20. Dez 13:44 agent.key dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0 3 20. Dez 13:44 files -r-x------. 1 nobody root system_u:object_r:unlabeled_t:s0 21724049 12. Dez 17:08 incus-agent -rwx------. 1 nobody root system_u:object_r:unlabeled_t:s0 1302 20. Dez 13:44 install.sh lrwxrwxrwx. 1 root root system_u:object_r:unlabeled_t:s0 11 20. Dez 13:44 lxd-agent dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0 2 20. Dez 13:44 nics -r--------. 1 nobody root system_u:object_r:unlabeled_t:s0 688 20. Dez 13:44 server.crt dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0 4 20. Dez 13:44 systemd dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0 3 20. Dez 13:44 udev /var/lib/incus/storage-pools/default/virtual-machines/vm1/config/files: total 5 -rw-r--r--. 1 nobody root system_u:object_r:unlabeled_t:s0 526 20. Dez 13:44 metadata.yaml /var/lib/incus/storage-pools/default/virtual-machines/vm1/config/nics: total 0 /var/lib/incus/storage-pools/default/virtual-machines/vm1/config/systemd: total 9 -r-x------. 1 nobody root system_u:object_r:unlabeled_t:s0 1570 20. Dez 13:44 incus-agent-setup -r--------. 1 nobody root system_u:object_r:unlabeled_t:s0 419 20. Dez 13:44 incus-agent.service /var/lib/incus/storage-pools/default/virtual-machines/vm1/config/udev: total 5 -r--------. 1 nobody root system_u:object_r:unlabeled_t:s0 109 20. Dez 13:44 99-incus-agent.rules /var/lib/incus/storage-pools/default/virtual-machines/vm1/templates: total 9 -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 21 20. Dez 08:48 hostname.tpl -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 140 20. Dez 08:48 hosts.tpl

Ah, I see now. Thanks!

Also good to see that there is active discussion on this over in the incus repo: lxc/incus#1037

0xC0ncord · 2024-12-18T17:06:40Z

policy/modules/services/zfs.te

+files_mmap_read_usr_files(zfs_t)
+
+# auto-snapshots through systemd-timer not working without this
+allow zfs_t zfs_exec_t:file execute_no_trans;


can_exec(zfs_t, zfs_exec_t)

0xC0ncord · 2024-12-18T17:10:20Z

One note regarding MCS: LXC containers currently need to use spc_t (super privileged container) domain in this first version and MCS does not apply to spc_t.

container_t OTOH would work with MCS but was designed for docker/podman process style containers. But e.g. systemd won't boot in a container running as container_t without many more allow rules added. Therefore I think it would be best to create a new incus_lxc_t type which will be quite similar to the virt_lxc_t type from the virt module adapted to work with incusd.

I think the best way to support this would be to create a type using container_domain_template() if we can. This will set up a process type with some very barebones functionality that most containers require but without any of the "fancy" things (even networking). Take a look at container_kvm_t as an example.

for reading compatibility file /usr/share/zfs/compatibility.d/openzfs-2.2 -rw-r--r--. 1 root root system_u:object_r:usr_t:s0 584 30. Aug 01:15 /usr/share/zfs/compatibility.d/openzfs-2.2 files_read_usr_files(zfs_t) files_mmap_read_usr_files(zfs_t) auto-snapshots through systemd-timer not working without this: scontext="system_u:system_r:zfs_t:s0" tcontext="system_u:object_r:zfs_exec_t:s0" class="file" perms="execute_no_trans" comm="env" exe="" path="" message="type=AVC msg=audit(1726998333.913:106): avc: denied { execute_no_trans } for pid=1708 comm="env" path="/usr/bin/zpool" dev="zfs" ino=405615 scontext=system_u:system_r:zfs_t:s0 tcontext=system_u:object_r:zfs_exec_t:s0 tclass=file permissive=0 " allow zfs_t zfs_exec_t:file execute_no_trans; Signed-off-by: Marc Schiffbauer <[email protected]>

Signed-off-by: Marc Schiffbauer <[email protected]>

and add rule to other modules taht need to talk to incusd Signed-off-by: Marc Schiffbauer <[email protected]>

Signed-off-by: Marc Schiffbauer <[email protected]>

this turns the previous optional block into a tunable which defaults to off Signed-off-by: Marc Schiffbauer <[email protected]>

This is not reproducable anymore in current incus version Signed-off-by: Marc Schiffbauer <[email protected]>

0xC0ncord · 2024-12-23T15:50:00Z

One note regarding MCS: LXC containers currently need to use spc_t (super privileged container) domain in this first version and MCS does not apply to spc_t.

container_t OTOH would work with MCS but was designed for docker/podman process style containers. But e.g. systemd won't boot in a container running as container_t without many more allow rules added. Therefore I think it would be best to create a new incus_lxc_t type which will be quite similar to the virt_lxc_t type from the virt module adapted to work with incusd.

I did some more digging into this and these rules probably need to be added to a new container_init_t type. This type already exists in container-selinux and its purpose is to allow systemd and other init systems to boot in a container as you describe. We don't have this type in refpolicy (and Gentoo policy) yet.

Also the reason why you're seeing all containers run as spc_t is because of the way the policy is written. Container engines will need to use setexeccon() when spawning a container process, otherwise the policy does an automatic transition to spc_t by design. If a user wants a specific SELinux context for the container to run in, there's usually a way to tell the container engine to do that (e.g. podman uses --context=) which will do the same thing, but with a manually specified context.

0xC0ncord · 2025-01-08T16:35:06Z

policy/modules/apps/qemu.te

+	# this is not possible as a tunable
+	storage_raw_read_fixed_disk(qemu_t)
+	storage_raw_write_fixed_disk(qemu_t)


Okay, so we need to enable this like so:

storage_raw_read_fixed_disk_cond(qemu_t, qemu_incus_managed) storage_raw_write_fixed_disk_cond(qemu_t, qemu_incus_managed)

The storage_raw_write_fixed_disk_cond interface doesn't exist in the policy so it needs to be added.

mschiff mentioned this pull request Dec 9, 2024

Set correct SELinux context options on container & VM mounts lxc/incus#1037

Open

0xC0ncord reviewed Dec 18, 2024

View reviewed changes

mschiff added 16 commits December 20, 2024 16:42

kernel/files: add interface files_mmap_read_usr_files

3d8c8df

Signed-off-by: Marc Schiffbauer <[email protected]>

kernel/files: add interface files_create_generic_tmp_sockets

f7498bd

Signed-off-by: Marc Schiffbauer <[email protected]>

container: add interfaces required for incus

7326204

Signed-off-by: Marc Schiffbauer <[email protected]>

devices: add interfaces required for incus

5acbca9

Signed-off-by: Marc Schiffbauer <[email protected]>

files: add interfaces required for incus

e4dc0b3

Signed-off-by: Marc Schiffbauer <[email protected]>

filesystem: add interfaces required for incus

ba347c3

Signed-off-by: Marc Schiffbauer <[email protected]>

incus: add new module

027265b

Signed-off-by: Marc Schiffbauer <[email protected]>

qemu: add rules required for incus

c4ae63c

Signed-off-by: Marc Schiffbauer <[email protected]>

dnsmasq: add rules required for incus

35ccc47

Signed-off-by: Marc Schiffbauer <[email protected]>

incus: add dev_rw_vhost to allow launching qemu

b46427e

Signed-off-by: Marc Schiffbauer <[email protected]>

incus: add incusd_stream_connect interface to incus

ab25ec8

and add rule to other modules taht need to talk to incusd Signed-off-by: Marc Schiffbauer <[email protected]>

incus: fix file context for incus-user and incus-benchmark

d307a9f

Signed-off-by: Marc Schiffbauer <[email protected]>

incus: fix file context for incus-user daemon

e822989

Signed-off-by: Marc Schiffbauer <[email protected]>

qemu: add qemu_incus_managed tunable

43cd552

this turns the previous optional block into a tunable which defaults to off Signed-off-by: Marc Schiffbauer <[email protected]>

incus: remove files_watch_all_dirs(incusd_t)

4eca13b

This is not reproducable anymore in current incus version Signed-off-by: Marc Schiffbauer <[email protected]>

mschiff force-pushed the incus branch from d9150a3 to 4eca13b Compare December 20, 2024 15:43

0xC0ncord reviewed Jan 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add new incus module, add required rules to other modules #6

Add new incus module, add required rules to other modules #6

mschiff commented Dec 6, 2024

mschiff commented Dec 9, 2024

0xC0ncord left a comment

0xC0ncord Dec 18, 2024

mschiff Dec 19, 2024

0xC0ncord Dec 23, 2024

0xC0ncord Dec 18, 2024

mschiff Dec 19, 2024

0xC0ncord Dec 23, 2024

0xC0ncord Dec 18, 2024

mschiff Dec 20, 2024

0xC0ncord Dec 18, 2024

mschiff Dec 20, 2024

0xC0ncord Dec 23, 2024

0xC0ncord Dec 18, 2024

0xC0ncord commented Dec 18, 2024

0xC0ncord commented Dec 23, 2024

0xC0ncord Jan 8, 2025

Add new incus module, add required rules to other modules #6

Are you sure you want to change the base?

Add new incus module, add required rules to other modules #6

Conversation

mschiff commented Dec 6, 2024

mschiff commented Dec 9, 2024

0xC0ncord left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

0xC0ncord commented Dec 18, 2024

0xC0ncord commented Dec 23, 2024

Choose a reason for hiding this comment