bpf: Inherit system settings for CPU security mitigations
Subsystems (4)
PATCH 1/1 bpf: Inherit system settings for CPU security mitigations
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Please note that there is also a 'nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.
This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
[1]. https://lore.kernel.org/bpf/[email protected]/
--- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/bpf.h +2 -2
@@ -2164,12 +2164,12 @@ static inline bool bpf_allow_uninit_stack(void)static inline bool bpf_bypass_spec_v1 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off (); }static inline bool bpf_bypass_spec_v4 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off (); }int bpf_map_new_fd (struct bpf_map *map,int flags);
Signature · 1 line
2.30.1 (Apple Git-130)
Quoted 6 lines · Yafang Shao
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Quoted 8 lines
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
Based on the discussion from [1]:
Quoted 28 lines
Signed-off-by Yafang Shao <[email protected]>Cc Luis Gerhorst <[email protected]>--- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)include/linux/bpf.h +2
-2@@ -2164,12 +2164,12 @@ static inline bool bpf_allow_uninit_stack(void)static inline bool bpf_bypass_spec_v1 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off (); }static inline bool bpf_bypass_spec_v4 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off (); }int bpf_map_new_fd (struct bpf_map *map,int flags);Signature · 1 line
2.30.1 (Apple Git-130)
Quoted 11 lines · Yafang Shao
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.
Quoted 7 lines
This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
[1]. https://lore.kernel.org/bpf/[email protected]/
Signed-off-by Yafang Shao <[email protected]>Cc Luis Gerhorst <[email protected]>
KP Singh
Acked-byQuoted 16 lines · Song Liu
Quoted 14 lines · Yafang Shao
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
Quoted 7 lines
Quoted 4 lines
[1]. https://lore.kernel.org/bpf/[email protected]/
Signed-off-by Yafang Shao <[email protected]>Cc Luis Gerhorst <[email protected]>Acked-by Song Liu <[email protected]>
Daniel Borkmann
maintainerQuoted 9 lines · KP Singh
Quoted 8 lines · Song Liu
Quoted 7 lines · Yafang Shao
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.
From reading, the cpu_mitigations_off() is a more generic toggle to turn these generally off, so going via cpu_mitigations_off() is fine in our case and does not leave some corner cases behind. I presume you mean above that in future the BPF side could respect some more fine-tuned settings, though it probably might need some more coordination wrt archs to abstract sth generic out of it.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Signed-off-by Yafang Shao <[email protected]>Cc Luis Gerhorst <[email protected]>Acked-by Song Liu <[email protected]>Acked-by KP Singh <[email protected]>
Thanks,
Daniel
This patch was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Quoted 11 lines · On Thu, 5 Oct 2023 08:41:23 +0000 you wrote:
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.[...]
Here is the summary with links:
- [bpf-next] bpf: Inherit system settings for CPU security mitigations
https://git.kernel.org/bpf/bpf-next/c/bc5bc309db45
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html
Quoted 27 lines · Yafang Shao
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
[1]. https://lore.kernel.org/bpf/[email protected]/
Signed-off-by Yafang Shao <[email protected]>Cc Luis Gerhorst <[email protected]>--- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)include/linux/bpf.h +0
-0
include/linux/bpf.h static inline bool bpf_allow_uninit_stack(void) · line 2167
Quoted 5 lines
include/linux/bpf.h +1
-1@@ -2164,12 +2164,12 @@ static inline bool bpf_allow_uninit_stack(void)static inline bool bpf_bypass_spec_v1 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off ();
Should we check cpu_mitigations_off() first before perfmon_capable() to avoid unnecessary capability checks, which generate audit messages?
Quoted 12 lines
include/linux/bpf.h +1
-2}static inline bool bpf_bypass_spec_v4 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off (); }int bpf_map_new_fd (struct bpf_map *map,int flags); --2.30.1 (Apple Git-130)
Quoted 30 lines · Andrii Nakryiko
Quoted 28 lines · Yafang Shao
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
[1]. https://lore.kernel.org/bpf/[email protected]/
Signed-off-by Yafang Shao <[email protected]>Cc Luis Gerhorst <[email protected]>--- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)include/linux/bpf.h +0
-0@@ -2164,12 +2164,12 @@ static inline bool bpf_allow_uninit_stack(void)
Quoted 7 lines
include/linux/bpf.h static inline bool bpf_allow_uninit_stack(void) · line 2167
Quoted 4 lines
include/linux/bpf.h +1
-1static inline bool bpf_bypass_spec_v1 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off ();Should we check
cpu_mitigations_off()first beforeperfmon_capable()to avoid unnecessary capability checks, which generate audit messages?
makes sense.
Should I send an additional patch, or you modify the original patch?
Signature · 2 lines
Regards Yafang
Quoted 41 lines · Yafang Shao
Quoted 30 lines · Andrii Nakryiko
Quoted 28 lines · Yafang Shao
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
[1]. https://lore.kernel.org/bpf/[email protected]/
Signed-off-by Yafang Shao <[email protected]>Cc Luis Gerhorst <[email protected]>--- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)include/linux/bpf.h +0
-0@@ -2164,12 +2164,12 @@ static inline bool bpf_allow_uninit_stack(void)Quoted 7 lines
include/linux/bpf.h static inline bool bpf_allow_uninit_stack(void) · line 2167
Quoted 4 lines
include/linux/bpf.h +1
-1static inline bool bpf_bypass_spec_v1 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off ();Should we check
cpu_mitigations_off()first beforeperfmon_capable()to avoid unnecessary capability checks, which generate audit messages?
Quoted 2 lines
makes sense.
Should I send an additional patch, or you modify the original patch?
please send a patch
Quoted 4 lines
--
Regards
Yafang
Quoted 29 lines · Yafang Shao
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
[1]. https://lore.kernel.org/bpf/[email protected]/
Signed-off-by Yafang Shao <[email protected]>Cc Luis Gerhorst <[email protected]>--- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)include/linux/bpf.h +0
-0@@ -2164,12 +2164,12 @@ static inline bool bpf_allow_uninit_stack(void)static inline bool bpf_bypass_spec_v1 (void )
include/linux/bpf.h static inline bool bpf_allow_uninit_stack(void) · line 2173
Quoted 10 lines
include/linux/bpf.h +2
-2{ -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off (); }static inline bool bpf_bypass_spec_v4 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off (); }
Yafang,
this patch breaks severaltest_progs -t verifier
tests when system is booted with mitigations=off command line.
Please follow up with a patch to fix this.
As you noticed cpu_mitigations_off() is not quite right here. The system might have booted without that command line, but spec_v1 and spec_v4 mitigations are turned off. Unfortunately there is no good way to check that atm. Have you seen this patch set ?
https://lore.kernel.org/all/[email protected]/
Please take a look at it and comment if you think it will help.
In the meantime please fix test_progs -t verifier
Thanks
Quoted 44 lines · Alexei Starovoitov
Quoted 29 lines · Yafang Shao
Currently, there exists a system-wide setting related to CPU security mitigations, denoted as 'mitigations='. When set to 'mitigations=off', it deactivates all optional CPU mitigations. Therefore, if we implement a system-wide 'mitigations=off' setting, it should inherently bypass Spectre v1 and Spectre v4 in the BPF subsystem.
Please note that there is also a '
nospectre_v1' setting on x86 and ppc architectures, though it is not currently exported. For the time being, let's disregard it.This idea emerged during our discussion about potential Spectre v1 attacks with Luis[1].
[1]. https://lore.kernel.org/bpf/[email protected]/
Signed-off-by Yafang Shao <[email protected]>Cc Luis Gerhorst <[email protected]>--- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)include/linux/bpf.h +0
-0@@ -2164,12 +2164,12 @@ static inline bool bpf_allow_uninit_stack(void)static inline bool bpf_bypass_spec_v1 (void )include/linux/bpf.h static inline bool bpf_allow_uninit_stack(void) · line 2173
Quoted 10 lines
include/linux/bpf.h +2
-2{ -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off (); }static inline bool bpf_bypass_spec_v4 (void ) { -return perfmon_capable (); +return perfmon_capable () ||cpu_mitigations_off (); }Yafang,
Quoted 2 lines
this patch breaks several
test_progs-t verifier
Sorry, I miss that.
Quoted 4 lines
tests when system is booted with mitigations=off command line.
Please follow up with a patch to fix this.
will do it.
Quoted 8 lines
As you noticed
cpu_mitigations_off()is not quite right here. The system might have booted without that command line, butspec_v1andspec_v4mitigations are turned off. Unfortunately there is no good way to check that atm. Have you seen this patch set ?
https://lore.kernel.org/all/[email protected]/
Please take a look at it and comment if you think it will help.
Thanks for your information. will take a look.
Quoted 2 lines
In the meantime please fix
test_progs-t verifier
sure
Signature · 2 lines
Regards Yafang
PATCH 1/1 selftests/bpf: Fix selftests broken by mitigations=off
In reply to Alexei StarovoitovWhen we configure the kernel command line with 'mitigations=off' and set the sysctl knob 'kernel.unprivileged_bpf_disabled' to 0, the commit bc5bc309db45 ("bpf: Inherit system settings for CPU security mitigations") causes issues in the execution of 'test_progs -t verifier.' This is because 'mitigations=off' bypasses Spectre v1 and Spectre v4 protections.
Currently, when a program requests to run in unprivileged mode (kernel.unprivileged_bpf_disabled = 0), the BPF verifier may prevent it from running due to the following conditions not being enabled:
- bypass_spec_v1
- bypass_spec_v4
- allow_ptr_leaks
- allow_uninit_stack
While 'mitigations=off' enables the first two conditions, it does not enable the latter two. As a result, some test cases in 'test_progs -t verifier' that were expected to fail to run may run successfully, while others still fail but with different error messages. This makes it challenging to address them comprehensively.
Moreover, in the future, we may introduce more fine-grained control over CPU mitigations, such as enabling only bypass_spec_v1 or bypass_spec_v4.
Given the complexity of the situation, rather than fixing each broken test case individually, it's preferable to skip them when 'mitigations=off' is in effect and introduce specific test cases for the new 'mitigations=off' scenario. For instance, we can introduce new BTF declaration tags like '__failure__nospec', '__failure_nospecv1' and '__failure_nospecv4'.
In this patch, the approach is to simply skip the broken test cases when 'mitigations=off' is enabled. The result as follows after this commit,
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- with 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED--- tools/testing/selftests/bpf/unpriv_helpers.c | 29 +++++++++++++++++++- 1 file changed, 28 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/unpriv_helpers.c +28 -1
@@ -4,9 +4,36 @@#include <stdlib.h> #include <error.h> #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> #include "unpriv_helpers.h" +static bool get_mitigations_off (void ) +{ +char cmdline[4096 ], *c; +int fd; + + fd =open ("/proc/cmdline" ,O_RDONLY ); +if (fd <0 ) { +perror ("open /proc/cmdline" ); +return false; + } + +if (read (fd, cmdline,sizeof (cmdline) -1 ) <0 ) { +perror ("read /proc/cmdline" ); +return false; + } + + cmdline[sizeof (cmdline) -1 ] ='\0' ; +for (c =strtok (cmdline," \n" ); c; c =strtok (NULL ," \n" )) { +if (!strncmp (c,"mitigtions=off" ,strlen (c))) +return true; + } +return false; +} +bool get_unpriv_disabled (void ) {bool disabled;
@@ -22,5 +49,5 @@ bool get_unpriv_disabled(void) disabled = true; } -return disabled; +return disabled ? true : !get_mitigations_off (); }
Signature · 1 line
2.39.3
PATCH v2 1/1 selftests/bpf: Fix selftests broken by mitigations=off
When we configure the kernel command line with 'mitigations=off' and set the sysctl knob 'kernel.unprivileged_bpf_disabled' to 0, the commit bc5bc309db45 ("bpf: Inherit system settings for CPU security mitigations") causes issues in the execution of 'test_progs -t verifier.' This is because 'mitigations=off' bypasses Spectre v1 and Spectre v4 protections.
Currently, when a program requests to run in unprivileged mode (kernel.unprivileged_bpf_disabled = 0), the BPF verifier may prevent it from running due to the following conditions not being enabled:
- bypass_spec_v1
- bypass_spec_v4
- allow_ptr_leaks
- allow_uninit_stack
While 'mitigations=off' enables the first two conditions, it does not enable the latter two. As a result, some test cases in 'test_progs -t verifier' that were expected to fail to run may run successfully, while others still fail but with different error messages. This makes it challenging to address them comprehensively.
Moreover, in the future, we may introduce more fine-grained control over CPU mitigations, such as enabling only bypass_spec_v1 or bypass_spec_v4.
Given the complexity of the situation, rather than fixing each broken test case individually, it's preferable to skip them when 'mitigations=off' is in effect and introduce specific test cases for the new 'mitigations=off' scenario. For instance, we can introduce new BTF declaration tags like '__failure__nospec', '__failure_nospecv1' and '__failure_nospecv4'.
In this patch, the approach is to simply skip the broken test cases when 'mitigations=off' is enabled. The result as follows after this commit,
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- with 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED--- tools/testing/selftests/bpf/unpriv_helpers.c | 34 +++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-)
---
v1 -> v2: Fix leaked fd
tools/testing/selftests/bpf/unpriv_helpers.c +33 -1
@@ -4,9 +4,41 @@#include <stdlib.h> #include <error.h> #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> #include "unpriv_helpers.h" +static bool get_mitigations_off (void ) +{ +char cmdline[4096 ], *c; +int fd, ret = false; + + fd =open ("/proc/cmdline" ,O_RDONLY ); +if (fd <0 ) { +perror ("open /proc/cmdline" ); +return false; + } + +if (read (fd, cmdline,sizeof (cmdline) -1 ) <0 ) { +perror ("read /proc/cmdline" ); + goto out; + } + + cmdline[sizeof (cmdline) -1 ] ='\0' ; +for (c =strtok (cmdline," \n" ); c; c =strtok (NULL ," \n" )) { +if (!strncmp (c,"mitigtions=off" ,strlen (c))) { + ret = true; +break ; + } + } + +out: +close (fd); +return ret; +} +bool get_unpriv_disabled (void ) {bool disabled;
@@ -22,5 +54,5 @@ bool get_unpriv_disabled(void) disabled = true; } -return disabled; +return disabled ? true : !get_mitigations_off (); }
Signature · 1 line
2.39.3
Re: [PATCH v2 bpf-next] selftests/bpf: Fix selftests broken by mitigations=off
Quoted 110 lines · Yafang Shao
When we configure the kernel command line with 'mitigations=off' and set the sysctl knob 'kernel.unprivileged_bpf_disabled' to 0, the commit
bc5bc309db45("bpf: Inherit system settings for CPU security mitigations") causes issues in the execution of 'test_progs-t verifier.' This is because 'mitigations=off' bypasses Spectre v1 and Spectre v4 protections.Currently, when a program requests to run in unprivileged mode (kernel.unprivileged_bpf_disabled = 0), the BPF verifier may prevent it from running due to the following conditions not being enabled:
-
bypass_spec_v1
-bypass_spec_v4
-allow_ptr_leaks
-allow_uninit_stackWhile 'mitigations=off' enables the first two conditions, it does not enable the latter two. As a result, some test cases in '
test_progs-t verifier' that were expected to fail to run may run successfully, while others still fail but with different error messages. This makes it challenging to address them comprehensively.Moreover, in the future, we may introduce more fine-grained control over CPU mitigations, such as enabling only
bypass_spec_v1orbypass_spec_v4.Given the complexity of the situation, rather than fixing each broken test case individually, it's preferable to skip them when 'mitigations=off' is in effect and introduce specific test cases for the new 'mitigations=off' scenario. For instance, we can introduce new BTF declaration tags like '
__failure__nospec', '__failure_nospecv1' and '__failure_nospecv4'.In this patch, the approach is to simply skip the broken test cases when 'mitigations=off' is enabled. The result as follows after this commit,
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED - kernel.unprivileged_bpf_disabled = 0 Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED - with 'mitigations=off' - kernel.unprivileged_bpf_disabled = 2 Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED - kernel.unprivileged_bpf_disabled = 0 Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILEDFixes bc5bc309db45 ("bpf: Inherit system settings for CPU security mitigations")Reported-by Alexei Starovoitov <[email protected]>Closes https://lore.kernel.org/bpf/CAADnVQKUBJqg+hHtbLeeC2jhoJAWqnmRAzXW3hmUCNSV9kx4sQ@mail.gmail.com open hereSigned-off-by Yafang Shao <[email protected]>--- tools/testing/selftests/bpf/unpriv_helpers.c | 34 +++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-)---
v1 -> v2: Fix leaked fdtools/testing/selftests/bpf/unpriv_helpers.c +33
-2@@ -4,9 +4,41 @@#include <stdlib.h> #include <error.h> #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> #include "unpriv_helpers.h" +static bool get_mitigations_off (void ) +{ +char cmdline[4096 ], *c; +int fd, ret = false; + + fd =open ("/proc/cmdline" ,O_RDONLY ); +if (fd <0 ) { +perror ("open /proc/cmdline" ); +return false; + } + +if (read (fd, cmdline,sizeof (cmdline) -1 ) <0 ) { +perror ("read /proc/cmdline" ); + goto out; + } + + cmdline[sizeof (cmdline) -1 ] ='\0' ; +for (c =strtok (cmdline," \n" ); c; c =strtok (NULL ," \n" )) { +if (!strncmp (c,"mitigtions=off" ,strlen (c))) { + ret = true; +break ; + } + } + +out: +close (fd); +return ret; +} +bool get_unpriv_disabled (void ) {bool disabled;@@ -22,5 +54,5 @@ bool get_unpriv_disabled(void) disabled = true; } -return disabled; +return disabled ? true : !get_mitigations_off (); } --
Quoted 2 lines
2.39.3
Pls. just igore this wrong patch. Sorry about the noise. I must be in a sleep state currently. I will send a new one after I get awake ...
Signature · 2 lines
Regards Yafang
kernel test robot
build botRe: [PATCH v2 bpf-next] selftests/bpf: Fix selftests broken by mitigations=off
In reply to Yafang Shaokernel test robot noticed the following build warnings:
Hi Yafang,
kernel test robot noticed the following build warnings:
[auto build test WARNING on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Yafang-Shao/selftests-bpf-Fix-selftests-broken-by-mitigations-off/20231022-175022 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/20231022094906.3003-1-laoar.shao%40gmail.com patch subject: [PATCH v2 bpf-next] selftests/bpf: Fix selftests broken by mitigations=off reproduce: (https://download.01.org/0day-ci/archive/20231022/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
# many are suggestions rather than must-fix
WARNING:QUOTED_WHITESPACE_BEFORE_NEWLINE: unnecessary whitespace before a quoted newline #89: FILE: tools/testing/selftests/bpf/unpriv_helpers.c:30:
+ for (c = strtok(cmdline, " \n"); c; c = strtok(NULL, " \n")) {
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
PATCH v3 1/1 selftests/bpf: Fix selftests broken by mitigations=off
In reply to Alexei StarovoitovWhen we configure the kernel command line with 'mitigations=off' and set the sysctl knob 'kernel.unprivileged_bpf_disabled' to 0, the commit bc5bc309db45 ("bpf: Inherit system settings for CPU security mitigations") causes issues in the execution of test_progs -t verifier. This is because 'mitigations=off' bypasses Spectre v1 and Spectre v4 protections.
Currently, when a program requests to run in unprivileged mode (kernel.unprivileged_bpf_disabled = 0), the BPF verifier may prevent it from running due to the following conditions not being enabled:
- bypass_spec_v1
- bypass_spec_v4
- allow_ptr_leaks
- allow_uninit_stack
While 'mitigations=off' enables the first two conditions, it does not enable the latter two. As a result, some test cases in 'test_progs -t verifier' that were expected to fail to run may run successfully, while others still fail but with different error messages. This makes it challenging to address them comprehensively.
Moreover, in the future, we may introduce more fine-grained control over CPU mitigations, such as enabling only bypass_spec_v1 or bypass_spec_v4.
Given the complexity of the situation, rather than fixing each broken test case individually, it's preferable to skip them when 'mitigations=off' is in effect and introduce specific test cases for the new 'mitigations=off' scenario. For instance, we can introduce new BTF declaration tags like '__failure__nospec', '__failure_nospecv1' and '__failure_nospecv4'.
In this patch, the approach is to simply skip the broken test cases when 'mitigations=off' is enabled. The result of test_progs -t verifier as follows after this commit,
Before this commit
==================
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/1336 PASSED, 0 SKIPPED, 0 FAILED <<<<
- with 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 63/1276 PASSED, 0 SKIPPED, 11 FAILED <<<< 11 FAILEDAfter this commit
=================
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/1336 PASSED, 0 SKIPPED, 0 FAILED <<<<
- with this patch, with 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED <<<< SKIPPED--- tools/testing/selftests/bpf/unpriv_helpers.c | 35 +++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/unpriv_helpers.c +34 -1
@@ -4,9 +4,42 @@#include <stdlib.h> #include <error.h> #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> #include "unpriv_helpers.h" +static bool get_mitigations_off (void ) +{ +char cmdline[4096 ], *c; +int fd, ret = false; + + fd =open ("/proc/cmdline" ,O_RDONLY ); +if (fd <0 ) { +perror ("open /proc/cmdline" ); +return false; + } + +if (read (fd, cmdline,sizeof (cmdline) -1 ) <0 ) { +perror ("read /proc/cmdline" ); + goto out; + } + + cmdline[sizeof (cmdline) -1 ] ='\0' ; +for (c =strtok (cmdline," \n" ); c; c =strtok (NULL ," \n" )) { +if (strncmp (c,"mitigations=off" ,strlen (c))) +continue ; + + ret = true; +break ; + } + +out: +close (fd); +return ret; +} +bool get_unpriv_disabled (void ) {bool disabled;
@@ -22,5 +55,5 @@ bool get_unpriv_disabled(void) disabled = true; } -return disabled; +return disabled ? true :get_mitigations_off (); }
Signature · 1 line
2.39.3
Re: [PATCH v3 bpf-next] selftests/bpf: Fix selftests broken by mitigations=off
Quoted 60 lines · Yafang Shao
When we configure the kernel command line with 'mitigations=off' and set the sysctl knob 'kernel.unprivileged_bpf_disabled' to 0, the commit
bc5bc309db45("bpf: Inherit system settings for CPU security mitigations") causes issues in the execution oftest_progs -t verifier. This is because 'mitigations=off' bypasses Spectre v1 and Spectre v4 protections.Currently, when a program requests to run in unprivileged mode (kernel.unprivileged_bpf_disabled = 0), the BPF verifier may prevent it from running due to the following conditions not being enabled:
-
bypass_spec_v1
-bypass_spec_v4
-allow_ptr_leaks
-allow_uninit_stackWhile 'mitigations=off' enables the first two conditions, it does not enable the latter two. As a result, some test cases in '
test_progs-t verifier' that were expected to fail to run may run successfully, while others still fail but with different error messages. This makes it challenging to address them comprehensively.Moreover, in the future, we may introduce more fine-grained control over CPU mitigations, such as enabling only
bypass_spec_v1orbypass_spec_v4.Given the complexity of the situation, rather than fixing each broken test case individually, it's preferable to skip them when 'mitigations=off' is in effect and introduce specific test cases for the new 'mitigations=off' scenario. For instance, we can introduce new BTF declaration tags like '
__failure__nospec', '__failure_nospecv1' and '__failure_nospecv4'.In this patch, the approach is to simply skip the broken test cases when 'mitigations=off' is enabled. The result of
test_progs -t verifieras follows after this commit,Before this commit
==================
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED - kernel.unprivileged_bpf_disabled = 0 Summary: 74/1336 PASSED, 0 SKIPPED, 0 FAILED <<<< - with 'mitigations=off' - kernel.unprivileged_bpf_disabled = 2 Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED - kernel.unprivileged_bpf_disabled = 0 Summary: 63/1276 PASSED, 0 SKIPPED, 11 FAILED <<<< 11 FAILEDAfter this commit
=================
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED - kernel.unprivileged_bpf_disabled = 0 Summary: 74/1336 PASSED, 0 SKIPPED, 0 FAILED <<<< - with this patch, with 'mitigations=off' - kernel.unprivileged_bpf_disabled = 2 Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED - kernel.unprivileged_bpf_disabled = 0 Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED <<<< SKIPPED
Quoted 4 lines
Fixes bc5bc309db45 ("bpf: Inherit system settings for CPU security mitigations")Reported-by Alexei Starovoitov <[email protected]>Closes https://lore.kernel.org/bpf/CAADnVQKUBJqg+hHtbLeeC2jhoJAWqnmRAzXW3hmUCNSV9kx4sQ@mail.gmail.com open hereSigned-off-by Yafang Shao <[email protected]>
Ack with a nit below.
Quoted 23 lines
--- tools/testing/selftests/bpf/unpriv_helpers.c | 35 +++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-)tools/testing/selftests/bpf/unpriv_helpers.c +3
-0@@ -4,9 +4,42 @@#include <stdlib.h> #include <error.h> #include <stdio.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> #include "unpriv_helpers.h" bool get_unpriv_disabled (void ) {bool disabled;
tools/testing/selftests/bpf/unpriv_helpers.c bool get_unpriv_disabled(void) · line 58
Quoted 5 lines
tools/testing/selftests/bpf/unpriv_helpers.c +1
-1@@ -22,5 +55,5 @@ bool get_unpriv_disabled(void) disabled = true; } -return disabled; +return disabled ? true :get_mitigations_off ();
Above code is correct. But you could slightly simplify it with
return disabled ? : get_mitigations_off();
I guess maintainer can decide whether simplification is needed or not.
Quoted 1 line
tools/testing/selftests/bpf/unpriv_helpers.c +0
-0}
Daniel Borkmann
maintainerRe: [PATCH v3 bpf-next] selftests/bpf: Fix selftests broken by mitigations=off
Quoted 69 lines · Yonghong Song
Quoted 60 lines · Yafang Shao
When we configure the kernel command line with 'mitigations=off' and set the sysctl knob 'kernel.unprivileged_bpf_disabled' to 0, the commit
bc5bc309db45("bpf: Inherit system settings for CPU security mitigations") causes issues in the execution oftest_progs -t verifier. This is because 'mitigations=off' bypasses Spectre v1 and Spectre v4 protections.Currently, when a program requests to run in unprivileged mode (kernel.unprivileged_bpf_disabled = 0), the BPF verifier may prevent it from running due to the following conditions not being enabled:
-
bypass_spec_v1
-bypass_spec_v4
-allow_ptr_leaks
-allow_uninit_stackWhile 'mitigations=off' enables the first two conditions, it does not enable the latter two. As a result, some test cases in '
test_progs-t verifier' that were expected to fail to run may run successfully, while others still fail but with different error messages. This makes it challenging to address them comprehensively.Moreover, in the future, we may introduce more fine-grained control over CPU mitigations, such as enabling only
bypass_spec_v1orbypass_spec_v4.Given the complexity of the situation, rather than fixing each broken test case individually, it's preferable to skip them when 'mitigations=off' is in effect and introduce specific test cases for the new 'mitigations=off' scenario. For instance, we can introduce new BTF declaration tags like '
__failure__nospec', '__failure_nospecv1' and '__failure_nospecv4'.In this patch, the approach is to simply skip the broken test cases when 'mitigations=off' is enabled. The result of
test_progs -t verifieras follows after this commit,Before this commit
==================
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/1336 PASSED, 0 SKIPPED, 0 FAILED <<<<
- with 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 63/1276 PASSED, 0 SKIPPED, 11 FAILED <<<< 11 FAILEDAfter this commit
=================
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/1336 PASSED, 0 SKIPPED, 0 FAILED <<<<
- with this patch, with 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED <<<< SKIPPEDQuoted 4 lines
Fixes bc5bc309db45 ("bpf: Inherit system settings for CPU security mitigations")Reported-by Alexei Starovoitov <[email protected]>Closes https://lore.kernel.org/bpf/CAADnVQKUBJqg+hHtbLeeC2jhoJAWqnmRAzXW3hmUCNSV9kx4sQ@mail.gmail.com open hereSigned-off-by Yafang Shao <[email protected]>Ack with a nit below.
Acked-by Yonghong Song <[email protected]>
[...]
Quoted 9 lines
Quoted 3 lines
}
- return disabled;
+ return disabled ? true :get_mitigations_off();Above code is correct. But you could slightly simplify it with
return disabled ? :get_mitigations_off();I guess maintainer can decide whether simplification is needed or not.
Turns out if you omit, then compiler will complain with a warning :)
[...]
GEN vmlinux.hunpriv_helpers.c: In function ‘get_unpriv_disabled’:
unpriv_helpers.c:56:27: error: the omitted middle operand in ‘?:’ will always be ‘true’, suggest explicit middle operand [-Werror=parentheses]
56 | return disabled ? : get_mitigations_off();
| ^
cc1: all warnings being treated as errors
make: *** [Makefile:615: /root/linux/tools/testing/selftests/bpf/unpriv_helpers.o] Error 1
So it's okay as is, applied, thanks!
Yonghong Song
reviewerRe: [PATCH v3 bpf-next] selftests/bpf: Fix selftests broken by mitigations=off
Quoted 83 lines · Daniel Borkmann
Quoted 69 lines · Yonghong Song
Quoted 60 lines · Yafang Shao
When we configure the kernel command line with 'mitigations=off' and set the sysctl knob 'kernel.unprivileged_bpf_disabled' to 0, the commit
bc5bc309db45("bpf: Inherit system settings for CPU security mitigations") causes issues in the execution oftest_progs -t verifier. This is because 'mitigations=off' bypasses Spectre v1 and Spectre v4 protections.Currently, when a program requests to run in unprivileged mode (kernel.unprivileged_bpf_disabled = 0), the BPF verifier may prevent it from running due to the following conditions not being enabled:
-
bypass_spec_v1
-bypass_spec_v4
-allow_ptr_leaks
-allow_uninit_stackWhile 'mitigations=off' enables the first two conditions, it does not enable the latter two. As a result, some test cases in '
test_progs-t verifier' that were expected to fail to run may run successfully, while others still fail but with different error messages. This makes it challenging to address them comprehensively.Moreover, in the future, we may introduce more fine-grained control over CPU mitigations, such as enabling only
bypass_spec_v1orbypass_spec_v4.Given the complexity of the situation, rather than fixing each broken test case individually, it's preferable to skip them when 'mitigations=off' is in effect and introduce specific test cases for the new 'mitigations=off' scenario. For instance, we can introduce new BTF declaration tags like '
__failure__nospec', '__failure_nospecv1' and '__failure_nospecv4'.In this patch, the approach is to simply skip the broken test cases when 'mitigations=off' is enabled. The result of
test_progs -t verifieras follows after this commit,Before this commit
==================
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/1336 PASSED, 0 SKIPPED, 0 FAILED <<<<
- with 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 63/1276 PASSED, 0 SKIPPED, 11 FAILED <<<< 11 FAILEDAfter this commit
=================
- without 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/1336 PASSED, 0 SKIPPED, 0 FAILED <<<<
- with this patch, with 'mitigations=off'
- kernel.unprivileged_bpf_disabled = 2
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED
- kernel.unprivileged_bpf_disabled = 0
Summary: 74/948 PASSED, 388 SKIPPED, 0 FAILED <<<< SKIPPEDQuoted 4 lines
Fixes bc5bc309db45 ("bpf: Inherit system settings for CPU security mitigations")Reported-by Alexei Starovoitov <[email protected]>Closes https://lore.kernel.org/bpf/CAADnVQKUBJqg+hHtbLeeC2jhoJAWqnmRAzXW3hmUCNSV9kx4sQ@mail.gmail.com open hereSigned-off-by Yafang Shao <[email protected]>Ack with a nit below.
Acked-by Yonghong Song <[email protected]>[...]
Quoted 9 lines
Quoted 3 lines
}
- return disabled;
+ return disabled ? true :get_mitigations_off();Above code is correct. But you could slightly simplify it with return disabled ? :
get_mitigations_off();I guess maintainer can decide whether simplification is needed or not.
Turns out if you omit, then compiler will complain with a warning :)
Quoted 8 lines
[...]
GENvmlinux.hunpriv_helpers.c: In function ‘get_unpriv_disabled’:
unpriv_helpers.c:56:27: error: the omitted middle operand in ‘?:’ will always be ‘true’, suggest explicit middle operand [-Werror=parentheses] 56 | return disabled ? :get_mitigations_off();
| ^
cc1: all warnings being treated as errors
make: *** [Makefile:615:/root/linux/tools/testing/selftests/bpf/unpriv_helpers.o] Error 1
clang compiler is okay with '?:' change while gcc compiler issued errors. So yes, existing code is good for both compilers. Thanks!
Quoted 3 lines
So it's okay as is, applied, thanks!
Re: [PATCH v3 bpf-next] selftests/bpf: Fix selftests broken by mitigations=off
In reply to Yafang ShaoThis patch was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Hello:
This patch was applied to bpf/bpf-next.git (master) by Daniel Borkmann <[email protected]>:
Quoted 11 lines · On Wed, 25 Oct 2023 03:11:44 +0000 you wrote:
When we configure the kernel command line with 'mitigations=off' and set the sysctl knob 'kernel.unprivileged_bpf_disabled' to 0, the commit
bc5bc309db45("bpf: Inherit system settings for CPU security mitigations") causes issues in the execution oftest_progs -t verifier. This is because 'mitigations=off' bypasses Spectre v1 and Spectre v4 protections.Currently, when a program requests to run in unprivileged mode (kernel.unprivileged_bpf_disabled = 0), the BPF verifier may prevent it from running due to the following conditions not being enabled:
[...]
Here is the summary with links:
- [v3,bpf-next] selftests/bpf: Fix selftests broken by mitigations=off
https://git.kernel.org/bpf/bpf-next/c/399f6185a1c0
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html