Skip to contentSkip to conversation

json parsing and bpf selftests dependencies

Question Alexis Lothoré 3 participants 4 messages Feb 26 – Feb 27

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hello,
I am pursuing my quest to convert standalone bpf tests (from tools/testing/selftests/bpf) into the test_progs framework so they can be executed automatically by the CI tooling.

I would like to continue on the bpftool tests, especially test_bpftool.py ([1]). This one involves quite a lot of json parsing on bpftool output (to validate that some entries are present in the output, depending on the used command), which does not seem to be a use case currently in bpf selftests. The first tests may be handled by some manual parsing (eg strstr'ing keys in the output, that's what I've done for the recently converted bpftool_metadata test, see [2]), but that's a bit fragile, and there are more complex subtests for which this loosy strategy will get even more fragile (eg test_feature_kernel_full_vs_not_full generates json output with two different commands, and tests the diff).

I then face the need to have some proper json parsing in test_progs. I kind of understand that there is a will to keep the dependency list small for test_progs, so I'm asking here what could be the best option for this:
- is it ok to make test_progs depend on a new json parsing library (eg:
cJSON) ? and so add the library to CI images ?
- or some implicit dependency on some CLI tooling (eg: jq) ? and so, add the cli tool to CI images ?
- test_progs is reusing json_writer.c from bpftool ([2]), should we rather write a custom json_reader.c as well (even if not needed at that point by bpftool itself) ?

Any opinion on this ?

Alexis

[1] https://elixir.bootlin.com/linux/v6.19.3/source/tools/testing/selftests/bpf/test_bpftool.py
[2] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c#n49
[3] https://elixir.bootlin.com/linux/v6.19.3/source/tools/bpf/bpftool/json_writer.c

Signature · 3 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#1 lore raw
Quoted 29 lines · Alexis Lothoré

Hello,
I am pursuing my quest to convert standalone bpf tests (from tools/testing/selftests/bpf) into the test_progs framework so they can be executed automatically by the CI tooling.

I would like to continue on the bpftool tests, especially test_bpftool.py ([1]). This one involves quite a lot of json parsing on bpftool output (to validate that some entries are present in the output, depending on the used command), which does not seem to be a use case currently in bpf selftests. The first tests may be handled by some manual parsing (eg strstr'ing keys in the output, that's what I've done for the recently converted bpftool_metadata test, see [2]), but that's a bit fragile, and there are more complex subtests for which this loosy strategy will get even more fragile (eg test_feature_kernel_full_vs_not_full generates json output with two different commands, and tests the diff).

I then face the need to have some proper json parsing in test_progs. I kind of understand that there is a will to keep the dependency list small for test_progs, so I'm asking here what could be the best option for this:
- is it ok to make test_progs depend on a new json parsing library (eg:
cJSON) ? and so add the library to CI images ?
- or some implicit dependency on some CLI tooling (eg: jq) ? and so, add the cli tool to CI images ?
- test_progs is reusing json_writer.c from bpftool ([2]), should we rather write a custom json_reader.c as well (even if not needed at that point by bpftool itself) ?

Quoted 1 line

Any opinion on this ?

Can we please make that test_progs can be built without json deps and run tests that do not require them, it's useful because we build tests in a restricted container image, where adding a new library may be undesirable (I'm not talking about github CI).

Quoted 11 lines

Alexis

[1] https://elixir.bootlin.com/linux/v6.19.3/source/tools/testing/selftests/bpf/test_bpftool.py
[2] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/tools/testing/selftests/bpf/prog_tests/bpftool_metadata.c#n49
[3] https://elixir.bootlin.com/linux/v6.19.3/source/tools/bpf/bpftool/json_writer.c

Signature · 3 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#2 lore raw
In reply to Alexis Lothoré
Quoted 8 lines · Alexis Lothoré

Hello,
I am pursuing my quest to convert standalone bpf tests (from tools/testing/selftests/bpf) into the test_progs framework so they can be executed automatically by the CI tooling.

I would like to continue on the bpftool tests, especially test_bpftool.py ([1]).

I think it outlived itself long ago.
Just delete it.
Any new bpftool features can be tested through test_progs. No need to test json output specifically.
It's diminishing returns.

#3 lore raw
Quoted 8 lines · Alexei Starovoitov
Quoted 6 lines · Alexis Lothoré

Hello,
I am pursuing my quest to convert standalone bpf tests (from tools/testing/selftests/bpf) into the test_progs framework so they can be executed automatically by the CI tooling.

Quoted 8 lines
Quoted 2 lines

I would like to continue on the bpftool tests, especially test_bpftool.py ([1]).

I think it outlived itself long ago.
Just delete it.
Any new bpftool features can be tested through test_progs. No need to test json output specifically.
It's diminishing returns.

Ok. That's a doubt I had while converting the first round of tests (bpf_metadata, bpftool_map_access), whether we really want to test exact bpftool CLI output, or if test_progs->libbpf testing was fine.

I'll remote this test_bpftool.py then, thanks for the feedback.

Alexis

Signature · 3 lines
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com