Experimental fsverity.test related GA CI improvements

From: Mimi Zohar <[email protected]>

This does not make fsverity.test working on GA CI, though.

- `--device /dev/loop-control' is required for losetup(8) to work.
- --privileged' is required foo mount(8) to work, and this makes --security-opt seccomp=unconfined' redundant.
- GA container does not have /sys/kernel/security' mounted which is needed for /sys/kernel/security/integrity/ima/policy'.
- Enable set -x in CI as the logs is everything we have to analyze on failures.

Signed-off-by Vitaly Chikunov <[email protected]>

---
 .github/workflows/ci.yml |  2 +-
 build.sh                 | 12 +++++++++++-
 tests/fsverity.test      |  2 +-
 3 files changed, 13 insertions(+), 3 deletions(-)

.github/workflows/ci.yml +1 -1

index 8223b87..d2afdfe 100644
@@ -98,7 +98,7 @@ jobs:
     container:
       image: ${{ matrix.container }}
       env: ${{ matrix.env }}
-      options: --security-opt seccomp=unconfined
+      options: --privileged --device /dev/loop-control
 
     steps:
     - name: Show OS

build.sh +11 -1

index cc5a258..4e2f1bb 100755
@@ -1,6 +1,16 @@
 #!/bin/sh
 # Copyright (c) 2020 Petr Vorel <[email protected]>
 
+if [ -n "$CI" ]; then
+	# If we under CI only thing we can analyze is logs so better to enable
+	# verbosity to a maximum.
+	set -x
+	# This is to make stdout and stderr synchronous in the logs.
+	exec 2>&1
+
+	mount -t securityfs -o rw securityfs /sys/kernel/security
+fi
+
 set -e
 
 CC="${CC:-gcc}"

@@ -100,7 +110,7 @@ if [ $ret -eq 0 ]; then
 	tail -20 tests/boot_aggregate.log
 
 	if [ -f tests/fsverity.log ]; then
-		tail -4 tests/fsverity.log
+		[ -n "$CI" ] && cat tests/fsverity.log || tail tests/fsverity.log
 		grep "skipped" tests/fsverity.log  && \
 		   grep "skipped" tests/fsverity.log | wc -l
 	fi

tests/fsverity.test +1 -1

index def06f8..1bb8362 100755
@@ -78,7 +78,7 @@ mount_loopback_file() {
 		exit "$FAIL"
 	fi
 
-	mount -o loop ${TST_IMG} $TST_MNT
+	mount -v -o loop ${TST_IMG} $TST_MNT
 	ret=$?
 
 	if [ "${ret}" -eq 0 ]; then

Signature · 1 line

2.33.4

From: Mimi Zohar <[email protected]>

This does not make fsverity.test working on GA CI, though.

- `--device /dev/loop-control' is required for losetup(8) to work.
- --privileged' is required foo mount(8) to work, and this makes --security-opt seccomp=unconfined' redundant.
- GA container does not have /sys/kernel/security' mounted which is needed for /sys/kernel/security/integrity/ima/policy'.
- Enable set -x in CI as the logs is everything we have to analyze on failures.

Quoted 4 lines · Vitaly Chikunov

From: Mimi Zohar <[email protected]>

This does not make fsverity.test working on GA CI, though.

Quoted 8 lines

- `--device /dev/loop-control' is required for losetup(8) to work.
- --privileged' is required foo mount(8) to work, and this makes --security-opt seccomp=unconfined' redundant.
- GA container does not have /sys/kernel/security' mounted which is needed for /sys/kernel/security/integrity/ima/policy'.
- Enable set -x in CI as the logs is everything we have to analyze on failures.

Agreed, even with these changes the fsverity test will not be executed, but skipped.

However, the reason for them being skipped is totally different than
prior to this patch. Once the distros have enabled both fsverity
support and are running a recent enough kernel with IMA support for fsverity, the fsverity test should succeed.

So the problem isn't the GitHub actions architecture or the fsverity test itself, but the lack of IMA kernel support for it. In addition to the ima-evm-utils distro tests, there needs to be a way for testing new kernel integrity features. Roberto's proposed ima-evm-utils UML patch set downloads and uses a UML kernel for this purpose.

Unless someone can recommend a better alternative, a single UML "distro" test could be defined and would be executed if a UML kernel is
supplied. Additional UML tests could be specified.

thanks,

Mimi

Mimi,

Quoted 28 lines · Mimi Zohar

Quoted 4 lines · Vitaly Chikunov

From: Mimi Zohar <[email protected]>

This does not make fsverity.test working on GA CI, though.

Quoted 8 lines

- `--device /dev/loop-control' is required for losetup(8) to work.
- --privileged' is required foo mount(8) to work, and this makes --security-opt seccomp=unconfined' redundant.
- GA container does not have /sys/kernel/security' mounted which is needed for /sys/kernel/security/integrity/ima/policy'.
- Enable set -x in CI as the logs is everything we have to analyze on failures.

Agreed, even with these changes the fsverity test will not be executed, but skipped.

However, the reason for them being skipped is totally different than
prior to this patch. Once the distros have enabled both fsverity
support and are running a recent enough kernel with IMA support for fsverity, the fsverity test should succeed.

So the problem isn't the GitHub actions architecture or the fsverity test itself, but the lack of IMA kernel support for it. In addition to the ima-evm-utils distro tests, there needs to be a way for testing new kernel integrity features. Roberto's proposed ima-evm-utils UML patch set downloads and uses a UML kernel for this purpose.

Quoted 3 lines

Unless someone can recommend a better alternative, a single UML "distro" test could be defined and would be executed if a UML kernel is
supplied. Additional UML tests could be specified.

Just as an idea. I did some CI testing for LKRG on GA,
https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.yml
https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.sh

It's possible to boot in QEMU system created in Docker (alas without KVM as GA does not support it). But this will install distribution's kernel. So it would need to find distribution with the appropriate kernel.

Also, GA have cache functionality, so there could be dependent job to build the kernel with required options and then save it into a cache (to save time, bandwidth, and CPU resources).

And another possibility is, instead of using Docker it's possible to use cloud images that many distributions have, and then same as with docker (install or build kernel, save into cache and use in next CI runs). Never tried this method myself. AFAIK this will require to use cloud-init to set up system on first boot.

Quoted 29 lines · Mimi Zohar

Quoted 4 lines · Vitaly Chikunov

From: Mimi Zohar <[email protected]>

This does not make fsverity.test working on GA CI, though.

Quoted 8 lines

- `--device /dev/loop-control' is required for losetup(8) to work.
- --privileged' is required foo mount(8) to work, and this makes --security-opt seccomp=unconfined' redundant.
- GA container does not have /sys/kernel/security' mounted which is needed for /sys/kernel/security/integrity/ima/policy'.
- Enable set -x in CI as the logs is everything we have to analyze on failures.

Agreed, even with these changes the fsverity test will not be executed, but skipped.

However, the reason for them being skipped is totally different than
prior to this patch. Once the distros have enabled both fsverity
support and are running a recent enough kernel with IMA support for fsverity, the fsverity test should succeed.

So the problem isn't the GitHub actions architecture or the fsverity test itself, but the lack of IMA kernel support for it. In addition to the ima-evm-utils distro tests, there needs to be a way for testing new kernel integrity features. Roberto's proposed ima-evm-utils UML patch set downloads and uses a UML kernel for this purpose.

Quoted 3 lines

Unless someone can recommend a better alternative, a single UML "distro" test could be defined and would be executed if a UML kernel is
supplied. Additional UML tests could be specified.

Just as an idea. I did some CI testing for LKRG on GA,
https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.yml
https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.sh

It's possible to boot in QEMU system created in Docker (alas without KVM as GA does not support it). But this will install distribution's kernel. So it would need to find distribution with the appropriate kernel.

Also, GA have cache functionality, so there could be dependent job to build the kernel with required options and then save it into a cache (to save time, bandwidth, and CPU resources).

And another possibility is, instead of using Docker it's possible to use cloud images that many distributions have, and then same as with docker (install or build kernel, save into cache and use in next CI runs). Never tried this method myself. AFAIK this will require to use cloud-init to set up system on first boot.