selftests/bpf: convert test_dev_cgroup to test_progs
Subsystems (3)
cover selftests/bpf: convert test_dev_cgroup to test_progs
Hello,
this small series aims to integrate test_dev_cgroup in test_progs so it could be run automatically in CI. The new version brings a few differences with the current one:
- test now uses directly syscalls instead of wrapping commandline tools into system() calls
- test_progs manipulates /dev/null (eg: redirecting test logs into it), so disabling access to it in the bpf program confuses the tests. To fix this, the first commit modifies the bpf program to allow access to char devices 1:3 (/dev/null), and disable access to char devices 1:5 (/dev/zero)
- once test is converted, add a small subtest to also check for device type interpretation (char or block)
- paths used in mknod tests are now in /dev instead of /tmp: due to the CI runner organisation and mountpoints manipulations, trying to create nodes in /tmp leads to errors unrelated to the test (ie, mknod calls refused by kernel, not the bpf program). I don't understand exactly the root cause at the deepest point (all I see in CI is an -ENXIO error on mknod when trying to create the node in tmp, and I can not make sense out of it neither replicate it locally), so I would gladly take inputs from anyone more educated than me about this.
The new test_progs part has been tested in a local qemu environment as well as in upstream CI:
./test_progs -a cgroup_dev
47/1 cgroup_dev/deny-mknod:OK 47/2 cgroup_dev/allow-mknod:OK 47/3 cgroup_dev/deny-mknod-wrong-type:OK 47/4 cgroup_dev/allow-read:OK 47/5 cgroup_dev/allow-write:OK 47/6 cgroup_dev/deny-read:OK 47/7 cgroup_dev/deny-write:OK 47 cgroup_dev:OK
Summary: 1/7 PASSED, 0 SKIPPED, 0 FAILED
---
Alexis Lothoré (eBPF Foundation) (3):
selftests/bpf: do not disable /dev/null device access in cgroup dev test selftests/bpf: convert test_dev_cgroup to test_progs selftests/bpf: add wrong type test to cgroup dev
tools/testing/selftests/bpf/.gitignore | 1 - tools/testing/selftests/bpf/Makefile | 2 - .../testing/selftests/bpf/prog_tests/cgroup_dev.c | 123 +++++++++++++++++++++ tools/testing/selftests/bpf/progs/dev_cgroup.c | 4 +- tools/testing/selftests/bpf/test_dev_cgroup.c | 85 -------------- 5 files changed, 125 insertions(+), 90 deletions(-)
---
base-commit: c90e2d4a7738a24fbb3657092dbd1ca20c040ed1
change-id: 20240723-convert_dev_cgroup-6464b0d37f1a
Best regards,
Signature · 3 lines
Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/3 selftests/bpf: do not disable /dev/null device access in cgroup dev test
test_dev_cgroup currently loads a small bpf program allowing any access on urandom and zero devices, disabling access to any other device. It makes migrating this test to test_progs impossible, since this one manipulates extensively /dev/null.
Allow /dev/null manipulation in dev_cgroup program to make its usage in test_progs framework possible. Update test_dev_cgroup.c as well to match this change while it has not been removed.
--- tools/testing/selftests/bpf/progs/dev_cgroup.c | 4 ++-- tools/testing/selftests/bpf/test_dev_cgroup.c | 18 +++++++++--------- 2 files changed, 11 insertions(+), 11 deletions(-)
tools/testing/selftests/bpf/progs/dev_cgroup.c +2 -2
@@ -41,14 +41,14 @@ int bpf_prog1(struct bpf_cgroup_dev_ctx *ctx)bpf_trace_printk (fmt,sizeof (fmt), ctx->major, ctx->minor);#endif -/* Allow access to /dev/zero and /dev/random. +/* Allow access to /dev/null and /dev/urandom. * Forbid everything else. */ if (ctx->major !=1 || type !=BPF_DEVCG_DEV_CHAR )return 0 ;switch (ctx->minor) { -case 5 :/* 1:5 /dev/zero */ +case 3 :/* 1:3 /dev/null */ case 9 :/* 1:9 /dev/urandom */ return 1 ; }
tools/testing/selftests/bpf/test_dev_cgroup.c +9 -9
@@ -54,25 +54,25 @@ int main(int argc, char **argv) goto err; } -/* All operations with /dev/zero and and /dev/urandom are allowed, +/* All operations with /dev/null and /dev/urandom are allowed, * everything else is forbidden. */ -assert (system ("rm -f /tmp/test_dev_cgroup_null" ) ==0 ); -assert (system ("mknod /tmp/test_dev_cgroup_null c 1 3" )); -assert (system ("rm -f /tmp/test_dev_cgroup_null" ) ==0 ); - -/* /dev/zero is whitelisted */ assert (system ("rm -f /tmp/test_dev_cgroup_zero" ) ==0 ); -assert (system ("mknod /tmp/test_dev_cgroup_zero c 1 5" ) ==0 ); +assert (system ("mknod /tmp/test_dev_cgroup_zero c 1 5" ));assert (system ("rm -f /tmp/test_dev_cgroup_zero" ) ==0 ); -assert (system ("dd if=/dev/urandom of=/dev/zero count=64" ) ==0 ); +/* /dev/null is whitelisted */ +assert (system ("rm -f /tmp/test_dev_cgroup_null" ) ==0 ); +assert (system ("mknod /tmp/test_dev_cgroup_null c 1 3" ) ==0 ); +assert (system ("rm -f /tmp/test_dev_cgroup_null" ) ==0 ); + +assert (system ("dd if=/dev/urandom of=/dev/null count=64" ) ==0 );/* src is allowed, target is forbidden */ assert (system ("dd if=/dev/urandom of=/dev/full count=64" ));/* src is forbidden, target is allowed */ -assert (system ("dd if=/dev/random of=/dev/zero count=64" )); +assert (system ("dd if=/dev/random of=/dev/null count=64" )); error =0 ;printf ("test_dev_cgroup:PASS\n" );
Signature · 1 line
2.45.2
PATCH 2/3 selftests/bpf: convert test_dev_cgroup to test_progs
In reply to Alexis Lothoré (eBPF Foundation)test_dev_cgroup is defined as a standalone test program, and so is not executed in CI.
Convert it to test_progs framework so it is tested automatically in CI, and remove the old test. In order to be able to run it in test_progs, /dev/null must remain usable, so change the new test to test operations on devices 1:3 as valid, and operations on devices 1:5 (/dev/zero) as invalid.
--- tools/testing/selftests/bpf/.gitignore | 1 - tools/testing/selftests/bpf/Makefile | 2 - .../testing/selftests/bpf/prog_tests/cgroup_dev.c | 120 +++++++++++++++++++++ tools/testing/selftests/bpf/test_dev_cgroup.c | 85 --------------- 4 files changed, 120 insertions(+), 88 deletions(-)
tools/testing/selftests/bpf/.gitignore +0 -1
@@ -9,7 +9,6 @@ test_lpm_map test_tag FEATURE-DUMP.libbpf fixdep -test_dev_cgroup /test_progs /test_progs-no_alu32 /test_progs-bpf_gcc
tools/testing/selftests/bpf/Makefile +0 -2
@@ -69,7 +69,6 @@ endif # Order correspond to 'make run_tests' order TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test_progs \ - test_dev_cgroup \ test_sock test_sockmap get_cgroup_id_user \ test_cgroup_storage \ test_tcpnotify_user test_sysctl \
@@ -295,7 +294,6 @@ JSON_WRITER := $(OUTPUT)/json_writer.o CAP_HELPERS := $(OUTPUT)/cap_helpers.o NETWORK_HELPERS := $(OUTPUT)/network_helpers.o -$(OUTPUT)/test_dev_cgroup: $(CGROUP_HELPERS) $(TESTING_HELPERS) $(OUTPUT)/test_skb_cgroup_id_user: $(CGROUP_HELPERS) $(TESTING_HELPERS) $(OUTPUT)/test_sock: $(CGROUP_HELPERS) $(TESTING_HELPERS) $(OUTPUT)/test_sockmap: $(CGROUP_HELPERS) $(TESTING_HELPERS)
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c new file +120 -0
120 lines not on this page · raw message
tools/testing/selftests/bpf/test_dev_cgroup.c deleted +0 -85
85 lines not on this page · raw message
Signature · 1 line
2.45.2
Quoted 72 lines · Alexis Lothoré (eBPF Foundation)
test_dev_cgroupis defined as a standalone test program, and so is not executed in CI.Convert it to
test_progsframework so it is tested automatically in CI, and remove the old test. In order to be able to run it intest_progs,/dev/nullmust remain usable, so change the new test to test operations on devices 1:3 as valid, and operations on devices 1:5 (/dev/zero) as invalid.Signed-off-by Alexis Lothoré (eBPF Foundation) <[email protected]>--- tools/testing/selftests/bpf/.gitignore | 1 - tools/testing/selftests/bpf/Makefile | 2 - .../testing/selftests/bpf/prog_tests/cgroup_dev.c | 120 +++++++++++++++++++++ tools/testing/selftests/bpf/test_dev_cgroup.c | 85 --------------- 4 files changed, 120 insertions(+), 88 deletions(-)tools/testing/selftests/bpf/.gitignore +0
-1@@ -9,7 +9,6 @@ test_lpm_map test_tag FEATURE-DUMP.libbpf fixdep -test_dev_cgroup /test_progs /test_progs-no_alu32 /test_progs-bpf_gcctools/testing/selftests/bpf/Makefile +0
-2@@ -69,7 +69,6 @@ endif # Order correspond to 'make run_tests' order TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test_progs \ - test_dev_cgroup \ test_sock test_sockmap get_cgroup_id_user \ test_cgroup_storage \ test_tcpnotify_user test_sysctl \@@ -295,7 +294,6 @@ JSON_WRITER := $(OUTPUT)/json_writer.o CAP_HELPERS := $(OUTPUT)/cap_helpers.o NETWORK_HELPERS := $(OUTPUT)/network_helpers.o -$(OUTPUT)/test_dev_cgroup: $(CGROUP_HELPERS) $(TESTING_HELPERS) $(OUTPUT)/test_skb_cgroup_id_user: $(CGROUP_HELPERS) $(TESTING_HELPERS) $(OUTPUT)/test_sock: $(CGROUP_HELPERS) $(TESTING_HELPERS) $(OUTPUT)/test_sockmap: $(CGROUP_HELPERS) $(TESTING_HELPERS)tools/testing/selftests/bpf/prog_tests/cgroup_dev.c new file +18
-0@@ -0,0 +1,120 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <sys/stat.h> +#include <sys/sysmacros.h> +#include "test_progs.h" +#include "cgroup_helpers.h" +#include "dev_cgroup.skel.h" + +#define TEST_CGROUP "/test-bpf-based-device-cgroup/" +#define TEST_BUFFER_SIZE 64 + +static void test_mknod (const char *path,mode_t mode,int dev_major, +int dev_minor,int should_fail) +{ +int ret; + +unlink (path); + ret =mknod (path, mode,makedev (dev_major, dev_minor));
[..]
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c line 22
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c new file +4
-0+if (should_fail) +ASSERT_ERR (ret,"mknod" ); +else +ASSERT_OK (ret,"mknod" );
Optional: might be easier to use something like expected_ret instead of should_fail and then do:
ASSERT_EQ(ret, expected_ret)
I see this part being copy-pasted in a bunch of places below.
Quoted 13 lines
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c new file +13
-0+unlink (path); +} + +static void test_read (const char *path,int should_fail) +{ +char buf[TEST_BUFFER_SIZE ]; +int ret, fd; + + fd =open (path,O_RDONLY ); + +/* A bare open on unauthorized device should fail */ +if (should_fail) { +ASSERT_ERR (fd,"open file for read" );
[..]
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c
Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c new file +2
-0+if (fd) +close (fd);
nit: should this be 'if (fd >= 0)'? I'm assuming the intention is to avoid close(-1)?
Quoted 17 lines
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c new file +17
-0+return ; + } + +if (!ASSERT_OK_FD (fd,"open file for read" )) +return ; + + ret =read (fd, buf,TEST_BUFFER_SIZE ); +if (should_fail) +ASSERT_ERR (ret,"read" ); +else +ASSERT_EQ (ret,TEST_BUFFER_SIZE ,"read" ); + +close (fd); +} + +static void test_write (const char *path,int should_fail) +{
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c line 64
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c new file +10
-0+char buf[] ="some random test data" ; +int ret, fd; + + fd =open (path,O_WRONLY ); + +/* A bare open on unauthorized device should fail */ +if (should_fail) { +ASSERT_ERR (fd,"open file for write" ); +if (fd) +close (fd);
Same 'if (fd >= 0)'
Hello Stanislas, thanks for the review
Quoted 1 line · Stanislav Fomichev
On 07/25, Alexis Lothoré (eBPF Foundation) wrote:
[...]
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c line 22
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c new file +4
-0+if (should_fail) +ASSERT_ERR (ret,"mknod" ); +else +ASSERT_OK (ret,"mknod" );Optional: might be easier to use something like
expected_retinstead ofshould_failand then do:
ASSERT_EQ(ret, expected_ret)
Yes, you are right. I initially went with a version relying on system() to perform the mknods/dd calls, which could return different errors codes so I used this should_fail. But while debugging some issues in CI with this series, I realized that the needed commands are basic enough to be replaced with direct library calls and I forgot to update this part, which can now assert an exact return value. I will update this accordingly.
Quoted 18 lines
I see this part being copy-pasted in a bunch of places below.
Quoted 13 lines
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c new file +13
-0+unlink (path); +} + +static void test_read (const char *path,int should_fail) +{ +char buf[TEST_BUFFER_SIZE ]; +int ret, fd; + + fd =open (path,O_RDONLY ); + +/* A bare open on unauthorized device should fail */ +if (should_fail) { +ASSERT_ERR (fd,"open file for read" );[..]
Quoted 5 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (fd) + close(fd);nit: should this be 'if (fd >= 0)'? I'm assuming the intention is to avoid close(-1)?
Right as well, I'll fix it (here and below) in v2
Thanks,
Alexis
Signature · 3 lines
Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 3/3 selftests/bpf: add wrong type test to cgroup dev
In reply to Alexis Lothoré (eBPF Foundation)Current cgroup_dev test mostly tests that device operation is accepted or refused base on passed major/minor (and so, any operation performed during test involves only char device)
Add a small subtest ensuring that the device type passed to bpf program allows it to take decisions as well.
--- tools/testing/selftests/bpf/prog_tests/cgroup_dev.c | 3 +++ 1 file changed, 3 insertions(+)
tools/testing/selftests/bpf/prog_tests/cgroup_dev.c +3 -0
@@ -101,6 +101,9 @@ void test_cgroup_dev(void)if (test__start_subtest ("allow-mknod" ))test_mknod ("/dev/test_dev_cgroup_null" ,S_IFCHR ,1 ,3 ,0 ); +if (test__start_subtest ("deny-mknod-wrong-type" )) +test_mknod ("/dev/test_dev_cgroup_null_block" ,S_IFBLK ,1 ,3 ,1 ); +if (test__start_subtest ("allow-read" ))test_read ("/dev/urandom" ,0 );
Signature · 1 line
2.45.2
Quoted 47 lines · Alexis Lothoré (eBPF Foundation)
Hello,
this small series aims to integratetest_dev_cgroupintest_progsso it could be run automatically in CI. The new version brings a few differences with the current one:
- test now uses directly syscalls instead of wrapping commandline tools intosystem()calls
-test_progsmanipulates/dev/null(eg: redirecting test logs into it), so disabling access to it in the bpf program confuses the tests. To fix this, the first commit modifies the bpf program to allow access to char devices 1:3 (/dev/null), and disable access to char devices 1:5 (/dev/zero)
- once test is converted, add a small subtest to also check for device type interpretation (char or block)
- paths used in mknod tests are now in /dev instead of /tmp: due to the CI runner organisation and mountpoints manipulations, trying to create nodes in /tmp leads to errors unrelated to the test (ie, mknod calls refused by kernel, not the bpf program). I don't understand exactly the root cause at the deepest point (all I see in CI is an-ENXIOerror on mknod when trying to create the node in tmp, and I can not make sense out of it neither replicate it locally), so I would gladly take inputs from anyone more educated than me about this.The new
test_progspart has been tested in a local qemu environment as well as in upstream CI:./test_progs -a
cgroup_dev47/1 cgroup_dev/deny-mknod:OK 47/2 cgroup_dev/allow-mknod:OK 47/3 cgroup_dev/deny-mknod-wrong-type:OK 47/4 cgroup_dev/allow-read:OK 47/5 cgroup_dev/allow-write:OK 47/6 cgroup_dev/deny-read:OK 47/7 cgroup_dev/deny-write:OK 47 cgroup_dev:OKSummary: 1/7 PASSED, 0 SKIPPED, 0 FAILED
---
Alexis Lothoré (eBPF Foundation) (3):
selftests/bpf: do not disable/dev/nulldevice access in cgroup dev test selftests/bpf: converttest_dev_cgrouptotest_progsselftests/bpf: add wrong type test to cgroup devtools/testing/selftests/bpf/.gitignore | 1 - tools/testing/selftests/bpf/Makefile | 2 - .../testing/selftests/bpf/prog_tests/cgroup_dev.c | 123 +++++++++++++++++++++ tools/testing/selftests/bpf/progs/dev_cgroup.c | 4 +- tools/testing/selftests/bpf/test_dev_cgroup.c | 85 -------------- 5 files changed, 125 insertions(+), 90 deletions(-)
Quoted 10 lines
---
base-commit:c90e2d4a7738a24fbb3657092dbd1ca20c040ed1
change-id: 20240723-convert_dev_cgroup-6464b0d37f1aBest regards,
Signature · 3 lines
Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Going forward, can you pls use [PATCH bpf-next] as a subject (or bpf when targeting bpf tree)? I'm not sure whether patchworks picks up plain [PATCH] messages..
Quoted 22 lines · Stanislav Fomichev
Quoted 21 lines · Alexis Lothoré (eBPF Foundation)
Hello,
this small series aims to integratetest_dev_cgroupintest_progsso it could be run automatically in CI. The new version brings a few differences with the current one:
- test now uses directly syscalls instead of wrapping commandline tools intosystem()calls
-test_progsmanipulates/dev/null(eg: redirecting test logs into it), so disabling access to it in the bpf program confuses the tests. To fix this, the first commit modifies the bpf program to allow access to char devices 1:3 (/dev/null), and disable access to char devices 1:5 (/dev/zero)
- once test is converted, add a small subtest to also check for device type interpretation (char or block)
- paths used in mknod tests are now in /dev instead of /tmp: due to the CI runner organisation and mountpoints manipulations, trying to create nodes in /tmp leads to errors unrelated to the test (ie, mknod calls refused by kernel, not the bpf program). I don't understand exactly the root cause at the deepest point (all I see in CI is an-ENXIOerror on mknod when trying to create the node in tmp, and I can not make sense out of it neither replicate it locally), so I would gladly take inputs from anyone more educated than me about this.
[...]
Quoted 3 lines
Going forward, can you pls use [PATCH bpf-next] as a subject (or bpf when targeting bpf tree)? I'm not sure whether patchworks picks up plain [PATCH] messages..
Yes, my bad, I realized some time after sending that I may have missed some proper patch prefix. I have just checked on patchwork and see this series and the one I have sent before, so I guess there is no need to resend those, but I'll make sure to apply the relevant prefix for next series.
Thanks,
Alexis
Signature · 3 lines
Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com