selftests/bpf: more precise cpu_mitigations state detection
Subsystems (3)
Eduard Zingerman
maintainerv2 cover selftests/bpf: more precise cpu_mitigations state detection
If kernel executing selftests is configured with spectre mitigations disabled, verification of unprivileged BPF won't emulate speculative branches.
A number of selftests relies on speculative branches being visited. In discussion [1] it was decided not to add additional tests classification, but to execute unprivileged tests only when mitigations are enabled.
Current mitigations status detection inspects /proc/cmdline, which is not sufficient for some configurations. This patch adds logic to also inspect /proc/config.gz and /boot/config-$(uname -r).
Changelog:
v1: https://lore.kernel.org/bpf/[email protected]/
v1 -> v2:
- added code to visit /boot/config-$(uname -r) (Yafang, Andrii)
- in case if config can't be read, print a warning and disable unprivileged tests execution.
[1] https://lore.kernel.org/bpf/[email protected]/
Eduard Zingerman (1):
selftests/bpf: more precise cpu_mitigations state detection
tools/testing/selftests/bpf/unpriv_helpers.c | 94 +++++++++++++++++++- 1 file changed, 91 insertions(+), 3 deletions(-)
Signature · 1 line
2.47.1
Eduard Zingerman
maintainerPATCH v2 1/1 selftests/bpf: more precise cpu_mitigations state detection
test_progs and test_verifier binaries execute unpriv tests under the following conditions:
- unpriv BPF is enabled;
- CPU mitigations are enabled (see [1] for details).
The detection of the "mitigations enabled" state is performed by unpriv_helpers.c:get_mitigations_off() via inspecting kernel boot command line, looking for a parameter "mitigations=off".
Such detection scheme won't work for certain configurations, e.g. when CONFIG_CPU_MITIGATIONS is disabled and boot parameter is not supplied.
Miss-detection leads to test_progs executing tests meant to be run only with mitigations enabled, e.g. verifier_and.c:known_subreg_with_unknown_reg(), and reporting false failures.
Internally, verifier sets bpf_verifier_env->bypass_spec_{v1,v4} basing on the value returned by kernel/cpu.c:cpu_mitigations_off(). This function is backed by a variable kernel/cpu.c:cpu_mitigations.
This state is not fully introspect-able via sysfs. The closest proxy is /sys/devices/system/cpu/vulnerabilities/spectre_v1, but it reports "vulnerable" state only if mitigations are disabled *and* current cpu is vulnerable, while verifier does not check cpu state.
There are only two ways the kernel/cpu.c:cpu_mitigations can be set:
- via boot parameter;
- via CONFIG_CPU_MITIGATIONS option.
This commit updates unpriv_helpers.c:get_mitigations_off() to scan /boot/config-$(uname -r) and /proc/config.gz for CONFIG_CPU_MITIGATIONS value in addition to boot command line check.
Tested using the following configurations:
- mitigations enabled (unpriv tests are enabled)
- mitigations disabled via boot cmdline (unpriv tests skipped)
- mitigations disabled via CONFIG_CPU_MITIGATIONS (unpriv tests skipped)
[1] https://lore.kernel.org/bpf/[email protected]/
--- tools/testing/selftests/bpf/unpriv_helpers.c | 94 +++++++++++++++++++- 1 file changed, 91 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/unpriv_helpers.c +91 -3
@@ -1,15 +1,76 @@// SPDX-License-Identifier: GPL-2.0-only +#include <errno.h> #include <stdbool.h> #include <stdlib.h> #include <stdio.h> #include <string.h> +#include <sys/utsname.h> #include <unistd.h> #include <fcntl.h> +#include <zlib.h> #include "unpriv_helpers.h" -static bool get_mitigations_off (void ) +static gzFile open_config (void ) +{ +struct utsname uts; +char buf[PATH_MAX ]; +gzFile config; + +if (uname (&uts)) { +perror ("uname" ); + goto config_gz; + } + +snprintf (buf,sizeof (buf),"/boot/config-%s" , uts.release); + config =gzopen (buf,"rb" ); +if (config) +return config; +fprintf (stderr,"gzopen %s: %s\n" , buf,strerror (errno)); + +config_gz: + config =gzopen ("/proc/config.gz" ,"rb" ); +if (!config) +perror ("gzopen /proc/config.gz" ); +return config; +} + +static int config_contains (const char *pat) +{ +int n, err, ret =-1 ; +const char *msg; +char buf[1024 ]; +gzFile config; + + config =open_config (); +if (!config) + goto out; + +for (;;) { +if (!gzgets (config, buf,sizeof (buf))) { + msg =gzerror (config, &err); +if (err ==Z_ERRNO ) +perror ("gzgets /proc/config.gz" ); +else if (err !=Z_OK ) +fprintf (stderr,"gzgets /proc/config.gz: %s" , msg); + goto out; + } + n =strlen (buf); +if (buf[n -1 ] =='\n' ) + buf[n -1 ] =0 ; +if (strcmp (buf, pat) ==0 ) { + ret = true; + goto out; + } + } + ret = false; +out: +gzclose (config); +return ret; +} + +static bool cmdline_contains (const char *pat) {char cmdline[4096 ], *c;int fd, ret = false;
@@ -27,7 +88,7 @@ static bool get_mitigations_off(void) cmdline[sizeof (cmdline) -1 ] ='\0' ;for (c =strtok (cmdline," \n" ); c; c =strtok (NULL ," \n" )) { -if (strncmp (c,"mitigations=off" ,strlen (c))) +if (strncmp (c, pat,strlen (c)))continue ; ret = true;break ;
@@ -37,8 +98,21 @@ static bool get_mitigations_off(void)return ret; } +static int get_mitigations_off (void ) +{ +int enabled_in_config; + +if (cmdline_contains ("mitigations=off" )) +return true; + enabled_in_config =config_contains ("CONFIG_CPU_MITIGATIONS=y" ); +if (enabled_in_config <0 ) +return -1 ; +return !enabled_in_config; +} +bool get_unpriv_disabled (void ) { +int mitigations_off;bool disabled;char buf[2 ];FILE *fd;
@@ -52,5 +126,19 @@ bool get_unpriv_disabled(void) disabled = true; } -return disabled ? true :get_mitigations_off (); +if (disabled) +return true; + +/* +* Some unpriv tests rely on spectre mitigations being on. +* If mitigations are off or status can't be determined +* assume that unpriv tests are disabled. +*/ + mitigations_off =get_mitigations_off (); +if (mitigations_off <0 ) { +fprintf (stderr, +"Can't determine if mitigations are enabled, disabling unpriv tests." ); +return true; + } +return mitigations_off; }
Signature · 1 line
2.47.1
Quoted 42 lines · Eduard Zingerman
test_progsandtest_verifierbinaries execute unpriv tests under the following conditions:
- unpriv BPF is enabled;
- CPU mitigations are enabled (see [1] for details).The detection of the "mitigations enabled" state is performed by unpriv_helpers.c:get_mitigations_off() via inspecting kernel boot command line, looking for a parameter "mitigations=off".
Such detection scheme won't work for certain configurations, e.g. when
CONFIG_CPU_MITIGATIONSis disabled and boot parameter is not supplied.Miss-detection leads to
test_progsexecuting tests meant to be run only with mitigations enabled, e.g. verifier_and.c:known_subreg_with_unknown_reg(), and reporting false failures.Internally, verifier sets bpf_verifier_env->bypass_spec_{v1,v4} basing on the value returned by kernel/cpu.c:cpu_mitigations_off(). This function is backed by a variable kernel/cpu.c:cpu_mitigations.
This state is not fully introspect-able via sysfs. The closest proxy is
/sys/devices/system/cpu/vulnerabilities/spectre_v1, but it reports "vulnerable" state only if mitigations are disabled *and* current cpu is vulnerable, while verifier does not check cpu state.There are only two ways the kernel/cpu.c:cpu_mitigations can be set:
- via boot parameter;
- viaCONFIG_CPU_MITIGATIONSoption.This commit updates unpriv_helpers.c:get_mitigations_off() to scan /boot/config-$(uname -r) and
/proc/config.gzforCONFIG_CPU_MITIGATIONSvalue in addition to boot command line check.Tested using the following configurations:
- mitigations enabled (unpriv tests are enabled)
- mitigations disabled via boot cmdline (unpriv tests skipped)
- mitigations disabled viaCONFIG_CPU_MITIGATIONS(unpriv tests skipped)
Quoted 4 lines
[1] https://lore.kernel.org/bpf/[email protected]/
Reported-by Mykyta Yatsenko <[email protected]>Signed-off-by Eduard Zingerman <[email protected]>
Quoted 141 lines
--- tools/testing/selftests/bpf/unpriv_helpers.c | 94 +++++++++++++++++++- 1 file changed, 91 insertions(+), 3 deletions(-)tools/testing/selftests/bpf/unpriv_helpers.c +91
-4@@ -1,15 +1,76 @@// SPDX-License-Identifier: GPL-2.0-only +#include <errno.h> #include <stdbool.h> #include <stdlib.h> #include <stdio.h> #include <string.h> +#include <sys/utsname.h> #include <unistd.h> #include <fcntl.h> +#include <zlib.h> #include "unpriv_helpers.h" -static bool get_mitigations_off (void ) +static gzFile open_config (void ) +{ +struct utsname uts; +char buf[PATH_MAX ]; +gzFile config; + +if (uname (&uts)) { +perror ("uname" ); + goto config_gz; + } + +snprintf (buf,sizeof (buf),"/boot/config-%s" , uts.release); + config =gzopen (buf,"rb" ); +if (config) +return config; +fprintf (stderr,"gzopen %s: %s\n" , buf,strerror (errno)); + +config_gz: + config =gzopen ("/proc/config.gz" ,"rb" ); +if (!config) +perror ("gzopen /proc/config.gz" ); +return config; +} + +static int config_contains (const char *pat) +{ +int n, err, ret =-1 ; +const char *msg; +char buf[1024 ]; +gzFile config; + + config =open_config (); +if (!config) + goto out; + +for (;;) { +if (!gzgets (config, buf,sizeof (buf))) { + msg =gzerror (config, &err); +if (err ==Z_ERRNO ) +perror ("gzgets /proc/config.gz" ); +else if (err !=Z_OK ) +fprintf (stderr,"gzgets /proc/config.gz: %s" , msg); + goto out; + } + n =strlen (buf); +if (buf[n -1 ] =='\n' ) + buf[n -1 ] =0 ; +if (strcmp (buf, pat) ==0 ) { + ret = true; + goto out; + } + } + ret = false; +out: +gzclose (config); +return ret; +} + +static bool cmdline_contains (const char *pat) {char cmdline[4096 ], *c;int fd, ret = false;@@ -27,7 +88,7 @@ static bool get_mitigations_off(void) cmdline[sizeof (cmdline) -1 ] ='\0' ;for (c =strtok (cmdline," \n" ); c; c =strtok (NULL ," \n" )) { -if (strncmp (c,"mitigations=off" ,strlen (c))) +if (strncmp (c, pat,strlen (c)))continue ; ret = true;break ;@@ -37,8 +98,21 @@ static bool get_mitigations_off(void)return ret; } +static int get_mitigations_off (void ) +{ +int enabled_in_config; + +if (cmdline_contains ("mitigations=off" )) +return true; + enabled_in_config =config_contains ("CONFIG_CPU_MITIGATIONS=y" ); +if (enabled_in_config <0 ) +return -1 ; +return !enabled_in_config; +} +bool get_unpriv_disabled (void ) { +int mitigations_off;bool disabled;char buf[2 ];FILE *fd;@@ -52,5 +126,19 @@ bool get_unpriv_disabled(void) disabled = true; } -return disabled ? true :get_mitigations_off (); +if (disabled) +return true; + +/* +* Some unpriv tests rely on spectre mitigations being on. +* If mitigations are off or status can't be determined +* assume that unpriv tests are disabled. +*/ + mitigations_off =get_mitigations_off (); +if (mitigations_off <0 ) { +fprintf (stderr, +"Can't determine if mitigations are enabled, disabling unpriv tests." ); +return true; + } +return mitigations_off; } --2.47.1
Signature · 2 lines
Regards Yafang
Quoted 85 lines · Eduard Zingerman
test_progsandtest_verifierbinaries execute unpriv tests under the following conditions:
- unpriv BPF is enabled;
- CPU mitigations are enabled (see [1] for details).The detection of the "mitigations enabled" state is performed by unpriv_helpers.c:get_mitigations_off() via inspecting kernel boot command line, looking for a parameter "mitigations=off".
Such detection scheme won't work for certain configurations, e.g. when
CONFIG_CPU_MITIGATIONSis disabled and boot parameter is not supplied.Miss-detection leads to
test_progsexecuting tests meant to be run only with mitigations enabled, e.g. verifier_and.c:known_subreg_with_unknown_reg(), and reporting false failures.Internally, verifier sets bpf_verifier_env->bypass_spec_{v1,v4} basing on the value returned by kernel/cpu.c:cpu_mitigations_off(). This function is backed by a variable kernel/cpu.c:cpu_mitigations.
This state is not fully introspect-able via sysfs. The closest proxy is
/sys/devices/system/cpu/vulnerabilities/spectre_v1, but it reports "vulnerable" state only if mitigations are disabled *and* current cpu is vulnerable, while verifier does not check cpu state.There are only two ways the kernel/cpu.c:cpu_mitigations can be set:
- via boot parameter;
- viaCONFIG_CPU_MITIGATIONSoption.This commit updates unpriv_helpers.c:get_mitigations_off() to scan /boot/config-$(uname -r) and
/proc/config.gzforCONFIG_CPU_MITIGATIONSvalue in addition to boot command line check.Tested using the following configurations:
- mitigations enabled (unpriv tests are enabled)
- mitigations disabled via boot cmdline (unpriv tests skipped)
- mitigations disabled viaCONFIG_CPU_MITIGATIONS(unpriv tests skipped)[1] https://lore.kernel.org/bpf/[email protected]/
Reported-by Mykyta Yatsenko <[email protected]>Signed-off-by Eduard Zingerman <[email protected]>--- tools/testing/selftests/bpf/unpriv_helpers.c | 94 +++++++++++++++++++- 1 file changed, 91 insertions(+), 3 deletions(-)tools/testing/selftests/bpf/unpriv_helpers.c +18
-1@@ -1,15 +1,76 @@// SPDX-License-Identifier: GPL-2.0-only +#include <errno.h> #include <stdbool.h> #include <stdlib.h> #include <stdio.h> #include <string.h> +#include <sys/utsname.h> #include <unistd.h> #include <fcntl.h> +#include <zlib.h> #include "unpriv_helpers.h" -static bool get_mitigations_off (void ) +static gzFile open_config (void ) +{ +struct utsname uts; +char buf[PATH_MAX ]; +gzFile config; + +if (uname (&uts)) { +perror ("uname" ); + goto config_gz; + } + +snprintf (buf,sizeof (buf),"/boot/config-%s" , uts.release); + config =gzopen (buf,"rb" ); +if (config) +return config;
tools/testing/selftests/bpf/unpriv_helpers.c line 39
Quoted 10 lines
tools/testing/selftests/bpf/unpriv_helpers.c +10
-0+fprintf (stderr,"gzopen %s: %s\n" , buf,strerror (errno)); + +config_gz: + config =gzopen ("/proc/config.gz" ,"rb" ); +if (!config) +perror ("gzopen /proc/config.gz" ); +return config; +} + +static int config_contains (const char *pat)
int-returning function... but we return do ret = true;, that looks accidental and sloppy. Let's add a comment that it returns <0 on error, 0 for no match, 1 for match and stick to numbers everywhere?
tools/testing/selftests/bpf/unpriv_helpers.c line 48
Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9
-0+{ +int n, err, ret =-1 ; +const char *msg; +char buf[1024 ]; +gzFile config; + + config =open_config (); +if (!config) + goto out;
nothing to gzclose if open_config() returns NULL, just return
pw-bot: cr
tools/testing/selftests/bpf/unpriv_helpers.c line 57
Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9
-0+ +for (;;) { +if (!gzgets (config, buf,sizeof (buf))) { + msg =gzerror (config, &err); +if (err ==Z_ERRNO ) +perror ("gzgets /proc/config.gz" ); +else if (err !=Z_OK ) +fprintf (stderr,"gzgets /proc/config.gz: %s" , msg); + goto out;
nit: I'd probably just do
gzclose(config);
return -EINVAL; (or whatever the error code might be)
tools/testing/selftests/bpf/unpriv_helpers.c line 64
Quoted 7 lines
tools/testing/selftests/bpf/unpriv_helpers.c +7
-0+ } + n =strlen (buf); +if (buf[n -1 ] =='\n' ) + buf[n -1 ] =0 ; +if (strcmp (buf, pat) ==0 ) { + ret = true; + goto out;
same here, gzclose() + return;
it will be easier to follow in this rather straightforward function
Quoted 25 lines
tools/testing/selftests/bpf/unpriv_helpers.c +10
-1+ } + } + ret = false; +out: +gzclose (config); +return ret; +} + +static bool cmdline_contains (const char *pat) {char cmdline[4096 ], *c;int fd, ret = false;@@ -27,7 +88,7 @@ static bool get_mitigations_off(void) cmdline[sizeof (cmdline) -1 ] ='\0' ;for (c =strtok (cmdline," \n" ); c; c =strtok (NULL ," \n" )) { -if (strncmp (c,"mitigations=off" ,strlen (c))) +if (strncmp (c, pat,strlen (c)))continue ; ret = true;break ;@@ -37,8 +98,21 @@ static bool get_mitigations_off(void)return ret; }
tools/testing/selftests/bpf/unpriv_helpers.c static bool get_mitigations_off(void) · line 110
Quoted 10 lines
tools/testing/selftests/bpf/unpriv_helpers.c +10
-0+static int get_mitigations_off (void ) +{ +int enabled_in_config; + +if (cmdline_contains ("mitigations=off" )) +return true; + enabled_in_config =config_contains ("CONFIG_CPU_MITIGATIONS=y" ); +if (enabled_in_config <0 ) +return -1 ; +return !enabled_in_config;
same mix of bool and integers
Quoted 32 lines
tools/testing/selftests/bpf/unpriv_helpers.c +18
-2+} +bool get_unpriv_disabled (void ) { +int mitigations_off;bool disabled;char buf[2 ];FILE *fd;@@ -52,5 +126,19 @@ bool get_unpriv_disabled(void) disabled = true; } -return disabled ? true :get_mitigations_off (); +if (disabled) +return true; + +/* +* Some unpriv tests rely on spectre mitigations being on. +* If mitigations are off or status can't be determined +* assume that unpriv tests are disabled. +*/ + mitigations_off =get_mitigations_off (); +if (mitigations_off <0 ) { +fprintf (stderr, +"Can't determine if mitigations are enabled, disabling unpriv tests." ); +return true; + } +return mitigations_off; } --2.47.1
Eduard Zingerman
maintainerOn Mon, 2025-06-16 at 14:13 -0700, Andrii Nakryiko wrote:
[...]
Quoted 5 lines
Quoted 1 line
+static int
config_contains(const char *pat)int-returning function... but we return do
ret = true;, that looks accidental and sloppy. Let's add a comment that it returns <0 on error, 0 for no match, 1 for match and stick to numbers everywhere?
It was actually intentional but ok, I can change that.
Quoted 24 lines
tools/testing/selftests/bpf/unpriv_helpers.c line 48
Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9
-0+{ +int n, err, ret =-1 ; +const char *msg; +char buf[1024 ]; +gzFile config; + + config =open_config (); +if (!config) + goto out;nothing to gzclose if
open_config()returns NULL, just returnpw-bot: cr
tools/testing/selftests/bpf/unpriv_helpers.c line 57
Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9
-0+ +for (;;) { +if (!gzgets (config, buf,sizeof (buf))) { + msg =gzerror (config, &err); +if (err ==Z_ERRNO ) +perror ("gzgets /proc/config.gz" ); +else if (err !=Z_OK ) +fprintf (stderr,"gzgets /proc/config.gz: %s" , msg); + goto out;
Quoted 4 lines
nit: I'd probably just do
gzclose(config);
return-EINVAL; (or whatever the error code might be)
I'll make this change just to avoid arguing but don't understand why tbh.
[...]
Quoted 5 lines · Eduard Zingerman
On Mon, 2025-06-16 at 14:13 -0700, Andrii Nakryiko wrote:
[...]
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
+static int
config_contains(const char *pat)int-returning function... but we return do
ret = true;, that looks accidental and sloppy. Let's add a comment that it returns <0 on error, 0 for no match, 1 for match and stick to numbers everywhere?It was actually intentional but ok, I can change that.
Thanks, I wouldn't mix true/false and integers, even though false -> 0, true -> 1 implicit conversion is well defined.
Quoted 25 lines
Quoted 25 lines
tools/testing/selftests/bpf/unpriv_helpers.c line 48
Quoted 10 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9
-0+{ +int n, err, ret =-1 ; +const char *msg; +char buf[1024 ]; +gzFile config; + + config =open_config (); +if (!config) + goto out;nothing to gzclose if
open_config()returns NULL, just returnpw-bot: cr
Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9
-0+ +for (;;) { +if (!gzgets (config, buf,sizeof (buf))) { + msg =gzerror (config, &err); +if (err ==Z_ERRNO ) +perror ("gzgets /proc/config.gz" ); +else if (err !=Z_OK ) +fprintf (stderr,"gzgets /proc/config.gz: %s" , msg); + goto out;
Quoted 6 lines
Quoted 4 lines
nit: I'd probably just do
gzclose(config);
return-EINVAL; (or whatever the error code might be)I'll make this change just to avoid arguing but don't understand why tbh.
I don't insist, but early return is easier for me to follow because it clearly states that nothing is going to happen after that point, so whoever is reading the code doesn't have to jump around the code to see if there is some follow up logic.
It's similar (for me) to the common pattern of
for (i = 0; i < n; i++) {
if (<some condition>) {
... here goes multi-line processing logic ...
}
}
I find it much more mentally draining than more obvious (to me)
for (i = 0; i < n; i++) {
if (!<some condition)
continue
... same logic, -1 level of nestedness, but really it's just to
know there is no other logic past <some condition> ...
}
It's all subjective, of course.
Quoted 2 lines
[...]