selftests/bpf: Make sure libbpf doesn't enforce the signature of a func pointer.
Subsystems (3)
PATCH 1/1 selftests/bpf: Make sure libbpf doesn't enforce the signature of a func pointer.
The verifier in the kernel checks the signatures of struct_ops operators. Libbpf should not verify it in order to allow flexibility in loading different implementations of an operator with different signatures to try to comply with the kernel, even if the signature defined in the BPF programs does not match with the implementations and the kernel.
This feature enables user space applications to manage the variations between different versions of the kernel by attempting various implementations of an operator.
This is a follow-up of the commit c911fc61a7ce ("libbpf: Skip zeroed or null fields if not found in the kernel type.")
--- .../bpf/prog_tests/test_struct_ops_module.c | 24 +++++++++++++++++++ .../selftests/bpf/progs/struct_ops_module.c | 13 ++++++++++ 2 files changed, 37 insertions(+)
tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c +24 -0
@@ -138,11 +138,35 @@ static void test_struct_ops_not_zeroed(void)struct_ops_module__destroy (skel); } +/* The signature of an implementation might not match the signature of the +* function pointer prototype defined in the BPF program. This mismatch +* should be allowed as long as the behavior of the operator program +* adheres to the signature in the kernel. Libbpf should not enforce the +* signature; rather, let the kernel verifier handle the enforcement. +*/ +static void test_struct_ops_incompatible (void ) +{ +struct struct_ops_module *skel; +struct bpf_link *link; + + skel =struct_ops_module__open_and_load (); +if (!ASSERT_OK_PTR (skel,"open_and_load" )) +return ; + + link =bpf_map__attach_struct_ops (skel->maps.testmod_incompatible); +if (ASSERT_OK_PTR (link,"attach_struct_ops" )) +bpf_link__destroy (link); + +struct_ops_module__destroy (skel); +} +void serial_test_struct_ops_module (void ) {if (test__start_subtest ("test_struct_ops_load" ))test_struct_ops_load ();if (test__start_subtest ("test_struct_ops_not_zeroed" ))test_struct_ops_not_zeroed (); +if (test__start_subtest ("test_struct_ops_incompatible" )) +test_struct_ops_incompatible (); }
tools/testing/selftests/bpf/progs/struct_ops_module.c +13 -0
@@ -68,3 +68,16 @@ struct bpf_testmod_ops___zeroed testmod_zeroed = { .test_1 = (void *)test_1, .test_2 = (void *)test_2_v2, }; + +struct bpf_testmod_ops___incompatible { +int (*test_1)(void ); +void (*test_2)(int *a); +int data; +}; + +SEC(".struct_ops.link") +struct bpf_testmod_ops___incompatible testmod_incompatible = { + .test_1 = (void *)test_1, + .test_2 = (void *)test_2, + .data =3 , +};
Signature · 1 line
2.34.1
Quoted 9 lines · Kui-Feng Lee
The verifier in the kernel checks the signatures of
struct_opsoperators. Libbpf should not verify it in order to allow flexibility in loading different implementations of an operator with different signatures to try to comply with the kernel, even if the signature defined in the BPF programs does not match with the implementations and the kernel.This feature enables user space applications to manage the variations between different versions of the kernel by attempting various implementations of an operator.
What is the utility of this? I'm missing what difference it would be if libbpf rejected vs kernel rejecting it? For backwards compat the kernel will fail or libbpf might throw an error and user will have to fixup signature regardless right? Why not get the error as early as possible.
Quoted 75 lines
This is a follow-up of the commit
c911fc61a7ce("libbpf: Skip zeroed or null fields if not found in the kernel type.")Signed-off-by Kui-Feng Lee <[email protected]>--- .../bpf/prog_tests/test_struct_ops_module.c | 24 +++++++++++++++++++ .../selftests/bpf/progs/struct_ops_module.c | 13 ++++++++++ 2 files changed, 37 insertions(+)tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c +24
-0@@ -138,11 +138,35 @@ static void test_struct_ops_not_zeroed(void)struct_ops_module__destroy (skel); } +/* The signature of an implementation might not match the signature of the +* function pointer prototype defined in the BPF program. This mismatch +* should be allowed as long as the behavior of the operator program +* adheres to the signature in the kernel. Libbpf should not enforce the +* signature; rather, let the kernel verifier handle the enforcement. +*/ +static void test_struct_ops_incompatible (void ) +{ +struct struct_ops_module *skel; +struct bpf_link *link; + + skel =struct_ops_module__open_and_load (); +if (!ASSERT_OK_PTR (skel,"open_and_load" )) +return ; + + link =bpf_map__attach_struct_ops (skel->maps.testmod_incompatible); +if (ASSERT_OK_PTR (link,"attach_struct_ops" )) +bpf_link__destroy (link); + +struct_ops_module__destroy (skel); +} +void serial_test_struct_ops_module (void ) {if (test__start_subtest ("test_struct_ops_load" ))test_struct_ops_load ();if (test__start_subtest ("test_struct_ops_not_zeroed" ))test_struct_ops_not_zeroed (); +if (test__start_subtest ("test_struct_ops_incompatible" )) +test_struct_ops_incompatible (); }tools/testing/selftests/bpf/progs/struct_ops_module.c +13
-0@@ -68,3 +68,16 @@ struct bpf_testmod_ops___zeroed testmod_zeroed = { .test_1 = (void *)test_1, .test_2 = (void *)test_2_v2, }; + +struct bpf_testmod_ops___incompatible { +int (*test_1)(void ); +void (*test_2)(int *a); +int data; +}; + +SEC(".struct_ops.link") +struct bpf_testmod_ops___incompatible testmod_incompatible = { + .test_1 = (void *)test_1, + .test_2 = (void *)test_2, + .data =3 , +};Signature · 1 line
2.34.1
Quoted 11 lines · John Fastabend
Quoted 9 lines · Kui-Feng Lee
The verifier in the kernel checks the signatures of
struct_opsoperators. Libbpf should not verify it in order to allow flexibility in loading different implementations of an operator with different signatures to try to comply with the kernel, even if the signature defined in the BPF programs does not match with the implementations and the kernel.This feature enables user space applications to manage the variations between different versions of the kernel by attempting various implementations of an operator.
Quoted 5 lines
What is the utility of this? I'm missing what difference it would be if libbpf rejected vs kernel rejecting it? For backwards compat the kernel will fail or libbpf might throw an error and user will have to fixup signature regardless right? Why not get the error as early as possible.
The check described here is that libbpf compares BTF types of functions and function pointers in struct_ops types in BPF programs, which may differ from kernel definitions.
A scenario here is a struct_ops type that includes an operator op_A with different versions depending on the kernel. All other fields in the struct_ops type have the same types. The application has only one definition for this struct_ops type, but the implementation of op_A is done separately for each version.
The application can try variations by assigning implementations to the op_A field until one is accepted by the kernel if libbpf doesn’t enforce signatures. Otherwise, the application has to define this struct_ops type for each variant if libbpf enforces signatures.
Does that make sense to you?
Quoted 78 lines
Quoted 76 lines
This is a follow-up of the commit
c911fc61a7ce("libbpf: Skip zeroed or null fields if not found in the kernel type.")Signed-off-by Kui-Feng Lee <[email protected]>--- .../bpf/prog_tests/test_struct_ops_module.c | 24 +++++++++++++++++++ .../selftests/bpf/progs/struct_ops_module.c | 13 ++++++++++ 2 files changed, 37 insertions(+)tools/testing/selftests/bpf/prog_tests/test_struct_ops_module.c +24
-0@@ -138,11 +138,35 @@ static void test_struct_ops_not_zeroed(void)struct_ops_module__destroy (skel); } +/* The signature of an implementation might not match the signature of the +* function pointer prototype defined in the BPF program. This mismatch +* should be allowed as long as the behavior of the operator program +* adheres to the signature in the kernel. Libbpf should not enforce the +* signature; rather, let the kernel verifier handle the enforcement. +*/ +static void test_struct_ops_incompatible (void ) +{ +struct struct_ops_module *skel; +struct bpf_link *link; + + skel =struct_ops_module__open_and_load (); +if (!ASSERT_OK_PTR (skel,"open_and_load" )) +return ; + + link =bpf_map__attach_struct_ops (skel->maps.testmod_incompatible); +if (ASSERT_OK_PTR (link,"attach_struct_ops" )) +bpf_link__destroy (link); + +struct_ops_module__destroy (skel); +} +void serial_test_struct_ops_module (void ) {if (test__start_subtest ("test_struct_ops_load" ))test_struct_ops_load ();if (test__start_subtest ("test_struct_ops_not_zeroed" ))test_struct_ops_not_zeroed (); +if (test__start_subtest ("test_struct_ops_incompatible" )) +test_struct_ops_incompatible (); }tools/testing/selftests/bpf/progs/struct_ops_module.c +13
-0@@ -68,3 +68,16 @@ struct bpf_testmod_ops___zeroed testmod_zeroed = { .test_1 = (void *)test_1, .test_2 = (void *)test_2_v2, }; + +struct bpf_testmod_ops___incompatible { +int (*test_1)(void ); +void (*test_2)(int *a); +int data; +}; + +SEC(".struct_ops.link") +struct bpf_testmod_ops___incompatible testmod_incompatible = { + .test_1 = (void *)test_1, + .test_2 = (void *)test_2, + .data =3 , +};Signature · 1 line
2.34.1
Martin KaFai Lau
reviewerQuoted 7 lines · Kui-Feng Lee
Quoted 3 lines · John Fastabend
Quoted 2 lines · Kui-Feng Lee
The verifier in the kernel checks the signatures of
struct_opsoperators. Libbpf should not verify it in order to allow flexibility in
This description probably is not accurate. iirc, the verifier does not check the function signature either. The verifier rejects only when the struct_ops prog tries to access something invalid. e.g. reading a function argument that does not exist in the running kernel.
Quoted 18 lines
Quoted 8 lines
Quoted 7 lines
loading different implementations of an operator with different signatures to try to comply with the kernel, even if the signature defined in the BPF programs does not match with the implementations and the kernel.
This feature enables user space applications to manage the variations between different versions of the kernel by attempting various implementations of an operator.
Quoted 5 lines
What is the utility of this? I'm missing what difference it would be if libbpf rejected vs kernel rejecting it? For backwards compat the kernel will fail or libbpf might throw an error and user will have to fixup signature regardless right? Why not get the error as early as possible.
The check described here is that libbpf compares BTF types of functions and function pointers in
struct_opstypes in BPF programs, which may differ from kernel definitions.
Quoted 8 lines
A scenario here is a
struct_opstype that includes an operatorop_Awith different versions depending on the kernel. All other fields in thestruct_opstype have the same types. The application has only one definition for thisstruct_opstype, but the implementation ofop_Ais done separately for each version.The application can try variations by assigning implementations to the
op_Afield until one is accepted by the kernel if libbpf doesn’t enforce
It probably would be clearer if the test actually does the retry. e.g. Try to load a struct_ops prog which reads an extra arg that is not supported by the running kernel and gets rejected by verifier. Then assigns an older struct_ops prog to the skel->struct_ops...->fn and loads successfully by the verifier.
Quoted 39 lines · Martin KaFai Lau
Quoted 7 lines · Kui-Feng Lee
Quoted 3 lines · John Fastabend
Quoted 2 lines · Kui-Feng Lee
The verifier in the kernel checks the signatures of
struct_opsoperators. Libbpf should not verify it in order to allow flexibility inThis description probably is not accurate. iirc, the verifier does not check the function signature either. The verifier rejects only when the
struct_opsprog tries to access something invalid. e.g. reading a function argument that does not exist in the running kernel.Quoted 24 lines
Quoted 8 lines
Quoted 7 lines
loading different implementations of an operator with different signatures to try to comply with the kernel, even if the signature defined in the BPF programs does not match with the implementations and the kernel.
This feature enables user space applications to manage the variations between different versions of the kernel by attempting various implementations of an operator.
Quoted 5 lines
What is the utility of this? I'm missing what difference it would be if libbpf rejected vs kernel rejecting it? For backwards compat the kernel will fail or libbpf might throw an error and user will have to fixup signature regardless right? Why not get the error as early as possible.
The check described here is that libbpf compares BTF types of functions and function pointers in
struct_opstypes in BPF programs, which may differ from kernel definitions.A scenario here is a
struct_opstype that includes an operatorop_Awith different versions depending on the kernel. All other fields in thestruct_opstype have the same types. The application has only one definition for thisstruct_opstype, but the implementation ofop_Ais done separately for each version.
Quoted 8 lines
Quoted 2 lines
The application can try variations by assigning implementations to the
op_Afield until one is accepted by the kernel if libbpf doesn’t enforceIt probably would be clearer if the test actually does the retry. e.g. Try to load a
struct_opsprog which reads an extra arg that is not supported by the running kernel and gets rejected by verifier. Then assigns an olderstruct_opsprog to the skel->struct_ops...->fn and loads successfully by the verifier.
This is actually a discouraged practice. In practice in production user-space logic does feature detection (using BTF or whatever else necessary) and then decides on specific BPF program implementation. So I wouldn't overstress this approach (trial-and-error one) in tests, it's a bad and sloppy practice.
Quoted 49 lines · Andrii Nakryiko
Quoted 39 lines · Martin KaFai Lau
Quoted 7 lines · Kui-Feng Lee
Quoted 3 lines · John Fastabend
Quoted 2 lines · Kui-Feng Lee
The verifier in the kernel checks the signatures of
struct_opsoperators. Libbpf should not verify it in order to allow flexibility inThis description probably is not accurate. iirc, the verifier does not check the function signature either. The verifier rejects only when the
struct_opsprog tries to access something invalid. e.g. reading a function argument that does not exist in the running kernel.Quoted 24 lines
Quoted 8 lines
Quoted 7 lines
loading different implementations of an operator with different signatures to try to comply with the kernel, even if the signature defined in the BPF programs does not match with the implementations and the kernel.
This feature enables user space applications to manage the variations between different versions of the kernel by attempting various implementations of an operator.
Quoted 5 lines
What is the utility of this? I'm missing what difference it would be if libbpf rejected vs kernel rejecting it? For backwards compat the kernel will fail or libbpf might throw an error and user will have to fixup signature regardless right? Why not get the error as early as possible.
The check described here is that libbpf compares BTF types of functions and function pointers in
struct_opstypes in BPF programs, which may differ from kernel definitions.A scenario here is a
struct_opstype that includes an operatorop_Awith different versions depending on the kernel. All other fields in thestruct_opstype have the same types. The application has only one definition for thisstruct_opstype, but the implementation ofop_Ais done separately for each version.Quoted 8 lines
Quoted 2 lines
The application can try variations by assigning implementations to the
op_Afield until one is accepted by the kernel if libbpf doesn’t enforceIt probably would be clearer if the test actually does the retry. e.g. Try to load a
struct_opsprog which reads an extra arg that is not supported by the running kernel and gets rejected by verifier. Then assigns an olderstruct_opsprog to the skel->struct_ops...->fn and loads successfully by the verifier.
Quoted 5 lines
This is actually a discouraged practice. In practice in production user-space logic does feature detection (using BTF or whatever else necessary) and then decides on specific BPF program implementation. So I wouldn't overstress this approach (trial-and-error one) in tests, it's a bad and sloppy practice.
It makes sense for me. I will rephrase this paragraph by using "feature detection" to replace "Try variations...".
Quoted 10 lines · Martin KaFai Lau
Quoted 7 lines · Kui-Feng Lee
Quoted 3 lines · John Fastabend
Quoted 2 lines · Kui-Feng Lee
The verifier in the kernel checks the signatures of
struct_opsoperators. Libbpf should not verify it in order to allow flexibility inThis description probably is not accurate. iirc, the verifier does not check the function signature either. The verifier rejects only when the
struct_opsprog tries to access something invalid. e.g. reading a function argument that does not exist in the running kernel.
Yes, kernel checks the behavior of programs. I will change the description.
Quoted 30 lines
Quoted 19 lines
Quoted 9 lines
Quoted 8 lines
loading different implementations of an operator with different signatures to try to comply with the kernel, even if the signature defined in the BPF programs does not match with the implementations and the kernel.
This feature enables user space applications to manage the variations between different versions of the kernel by attempting various implementations of an operator.
Quoted 5 lines
What is the utility of this? I'm missing what difference it would be if libbpf rejected vs kernel rejecting it? For backwards compat the kernel will fail or libbpf might throw an error and user will have to fixup signature regardless right? Why not get the error as early as possible.
The check described here is that libbpf compares BTF types of functions and function pointers in
struct_opstypes in BPF programs, which may differ from kernel definitions.Quoted 8 lines
A scenario here is a
struct_opstype that includes an operatorop_Awith different versions depending on the kernel. All other fields in thestruct_opstype have the same types. The application has only one definition for thisstruct_opstype, but the implementation ofop_Ais done separately for each version.The application can try variations by assigning implementations to the
op_Afield until one is accepted by the kernel if libbpf doesn’t enforceIt probably would be clearer if the test actually does the retry. e.g. Try to load a
struct_opsprog which reads an extra arg that is not supported by the running kernel and gets rejected by verifier. Then assigns an olderstruct_opsprog to the skel->struct_ops...->fn and loads successfully by the verifier.