Skip to contentSkip to conversation

selftests/bpf: more precise cpu_mitigations state detection

PATCH Superseded Eduard Zingerman 3 participants 6 messages Jun 14, 2025 – Jun 16, 2025

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Eduard Zingerman

author maintainer
#0 lore raw

v2 cover selftests/bpf: more precise cpu_mitigations state detection

If kernel executing selftests is configured with spectre mitigations disabled, verification of unprivileged BPF won't emulate speculative branches.

A number of selftests relies on speculative branches being visited. In discussion [1] it was decided not to add additional tests classification, but to execute unprivileged tests only when mitigations are enabled.

Current mitigations status detection inspects /proc/cmdline, which is not sufficient for some configurations. This patch adds logic to also inspect /proc/config.gz and /boot/config-$(uname -r).

Changelog:
v1: https://lore.kernel.org/bpf/[email protected]/
v1 -> v2:
- added code to visit /boot/config-$(uname -r) (Yafang, Andrii)
- in case if config can't be read, print a warning and disable unprivileged tests execution.

[1] https://lore.kernel.org/bpf/[email protected]/

Eduard Zingerman (1):
selftests/bpf: more precise cpu_mitigations state detection

 tools/testing/selftests/bpf/unpriv_helpers.c | 94 +++++++++++++++++++-
 1 file changed, 91 insertions(+), 3 deletions(-)
Signature · 1 line
2.47.1
author maintainer
#1 lore raw

PATCH v2 1/1 selftests/bpf: more precise cpu_mitigations state detection

test_progs and test_verifier binaries execute unpriv tests under the following conditions:
- unpriv BPF is enabled;
- CPU mitigations are enabled (see [1] for details).

The detection of the "mitigations enabled" state is performed by unpriv_helpers.c:get_mitigations_off() via inspecting kernel boot command line, looking for a parameter "mitigations=off".

Such detection scheme won't work for certain configurations, e.g. when CONFIG_CPU_MITIGATIONS is disabled and boot parameter is not supplied.

Miss-detection leads to test_progs executing tests meant to be run only with mitigations enabled, e.g. verifier_and.c:known_subreg_with_unknown_reg(), and reporting false failures.

Internally, verifier sets bpf_verifier_env->bypass_spec_{v1,v4} basing on the value returned by kernel/cpu.c:cpu_mitigations_off(). This function is backed by a variable kernel/cpu.c:cpu_mitigations.

This state is not fully introspect-able via sysfs. The closest proxy is /sys/devices/system/cpu/vulnerabilities/spectre_v1, but it reports "vulnerable" state only if mitigations are disabled *and* current cpu is vulnerable, while verifier does not check cpu state.

There are only two ways the kernel/cpu.c:cpu_mitigations can be set:
- via boot parameter;
- via CONFIG_CPU_MITIGATIONS option.

This commit updates unpriv_helpers.c:get_mitigations_off() to scan /boot/config-$(uname -r) and /proc/config.gz for CONFIG_CPU_MITIGATIONS value in addition to boot command line check.

Tested using the following configurations:
- mitigations enabled (unpriv tests are enabled)
- mitigations disabled via boot cmdline (unpriv tests skipped)
- mitigations disabled via CONFIG_CPU_MITIGATIONS (unpriv tests skipped)

[1] https://lore.kernel.org/bpf/[email protected]/

---
 tools/testing/selftests/bpf/unpriv_helpers.c | 94 +++++++++++++++++++-
 1 file changed, 91 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/unpriv_helpers.c +91 -3
index 220f6a963813..625556a0e7f1 100644
@@ -1,15 +1,76 @@
 // SPDX-License-Identifier: GPL-2.0-only
 
+#include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <stdio.h>
 #include <string.h>
+#include <sys/utsname.h>
 #include <unistd.h>
 #include <fcntl.h>
+#include <zlib.h>
 
 #include "unpriv_helpers.h"
 
-static bool get_mitigations_off(void)
+static gzFile open_config(void)
+{
+	struct utsname uts;
+	char buf[PATH_MAX];
+	gzFile config;
+
+	if (uname(&uts)) {
+		perror("uname");
+		goto config_gz;
+	}
+
+	snprintf(buf, sizeof(buf), "/boot/config-%s", uts.release);
+	config = gzopen(buf, "rb");
+	if (config)
+		return config;
+	fprintf(stderr, "gzopen %s: %s\n", buf, strerror(errno));
+
+config_gz:
+	config = gzopen("/proc/config.gz", "rb");
+	if (!config)
+		perror("gzopen /proc/config.gz");
+	return config;
+}
+
+static int config_contains(const char *pat)
+{
+	int n, err, ret = -1;
+	const char *msg;
+	char buf[1024];
+	gzFile config;
+
+	config = open_config();
+	if (!config)
+		goto out;
+
+	for (;;) {
+		if (!gzgets(config, buf, sizeof(buf))) {
+			msg = gzerror(config, &err);
+			if (err == Z_ERRNO)
+				perror("gzgets /proc/config.gz");
+			else if (err != Z_OK)
+				fprintf(stderr, "gzgets /proc/config.gz: %s", msg);
+			goto out;
+		}
+		n = strlen(buf);
+		if (buf[n - 1] == '\n')
+			buf[n - 1] = 0;
+		if (strcmp(buf, pat) == 0) {
+			ret = true;
+			goto out;
+		}
+	}
+	ret = false;
+out:
+	gzclose(config);
+	return ret;
+}
+
+static bool cmdline_contains(const char *pat)
 {
 	char cmdline[4096], *c;
 	int fd, ret = false;
@@ -27,7 +88,7 @@ static bool get_mitigations_off(void)
 
 	cmdline[sizeof(cmdline) - 1] = '\0';
 	for (c = strtok(cmdline, " \n"); c; c = strtok(NULL, " \n")) {
-		if (strncmp(c, "mitigations=off", strlen(c)))
+		if (strncmp(c, pat, strlen(c)))
 			continue;
 		ret = true;
 		break;
@@ -37,8 +98,21 @@ static bool get_mitigations_off(void)
 	return ret;
 }
 
+static int get_mitigations_off(void)
+{
+	int enabled_in_config;
+
+	if (cmdline_contains("mitigations=off"))
+		return true;
+	enabled_in_config = config_contains("CONFIG_CPU_MITIGATIONS=y");
+	if (enabled_in_config < 0)
+		return -1;
+	return !enabled_in_config;
+}
+
 bool get_unpriv_disabled(void)
 {
+	int mitigations_off;
 	bool disabled;
 	char buf[2];
 	FILE *fd;
@@ -52,5 +126,19 @@ bool get_unpriv_disabled(void)
 		disabled = true;
 	}
 
-	return disabled ? true : get_mitigations_off();
+	if (disabled)
+		return true;
+
+	/*
+	 * Some unpriv tests rely on spectre mitigations being on.
+	 * If mitigations are off or status can't be determined
+	 * assume that unpriv tests are disabled.
+	 */
+	mitigations_off = get_mitigations_off();
+	if (mitigations_off < 0) {
+		fprintf(stderr,
+			"Can't determine if mitigations are enabled, disabling unpriv tests.");
+		return true;
+	}
+	return mitigations_off;
 }
Signature · 1 line
2.47.1
Acked-by on v2 1/1
#2 lore raw
Quoted 42 lines · Eduard Zingerman

test_progs and test_verifier binaries execute unpriv tests under the following conditions:
- unpriv BPF is enabled;
- CPU mitigations are enabled (see [1] for details).

The detection of the "mitigations enabled" state is performed by unpriv_helpers.c:get_mitigations_off() via inspecting kernel boot command line, looking for a parameter "mitigations=off".

Such detection scheme won't work for certain configurations, e.g. when CONFIG_CPU_MITIGATIONS is disabled and boot parameter is not supplied.

Miss-detection leads to test_progs executing tests meant to be run only with mitigations enabled, e.g. verifier_and.c:known_subreg_with_unknown_reg(), and reporting false failures.

Internally, verifier sets bpf_verifier_env->bypass_spec_{v1,v4} basing on the value returned by kernel/cpu.c:cpu_mitigations_off(). This function is backed by a variable kernel/cpu.c:cpu_mitigations.

This state is not fully introspect-able via sysfs. The closest proxy is /sys/devices/system/cpu/vulnerabilities/spectre_v1, but it reports "vulnerable" state only if mitigations are disabled *and* current cpu is vulnerable, while verifier does not check cpu state.

There are only two ways the kernel/cpu.c:cpu_mitigations can be set:
- via boot parameter;
- via CONFIG_CPU_MITIGATIONS option.

This commit updates unpriv_helpers.c:get_mitigations_off() to scan /boot/config-$(uname -r) and /proc/config.gz for CONFIG_CPU_MITIGATIONS value in addition to boot command line check.

Tested using the following configurations:
- mitigations enabled (unpriv tests are enabled)
- mitigations disabled via boot cmdline (unpriv tests skipped)
- mitigations disabled via CONFIG_CPU_MITIGATIONS (unpriv tests skipped)

Quoted 4 lines

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 141 lines
---
 tools/testing/selftests/bpf/unpriv_helpers.c | 94 +++++++++++++++++++-
 1 file changed, 91 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/unpriv_helpers.c +91 -4
index 220f6a963813..625556a0e7f1 100644
@@ -1,15 +1,76 @@
 // SPDX-License-Identifier: GPL-2.0-only
 
+#include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <stdio.h>
 #include <string.h>
+#include <sys/utsname.h>
 #include <unistd.h>
 #include <fcntl.h>
+#include <zlib.h>
 
 #include "unpriv_helpers.h"
 
-static bool get_mitigations_off(void)
+static gzFile open_config(void)
+{
+       struct utsname uts;
+       char buf[PATH_MAX];
+       gzFile config;
+
+       if (uname(&uts)) {
+               perror("uname");
+               goto config_gz;
+       }
+
+       snprintf(buf, sizeof(buf), "/boot/config-%s", uts.release);
+       config = gzopen(buf, "rb");
+       if (config)
+               return config;
+       fprintf(stderr, "gzopen %s: %s\n", buf, strerror(errno));
+
+config_gz:
+       config = gzopen("/proc/config.gz", "rb");
+       if (!config)
+               perror("gzopen /proc/config.gz");
+       return config;
+}
+
+static int config_contains(const char *pat)
+{
+       int n, err, ret = -1;
+       const char *msg;
+       char buf[1024];
+       gzFile config;
+
+       config = open_config();
+       if (!config)
+               goto out;
+
+       for (;;) {
+               if (!gzgets(config, buf, sizeof(buf))) {
+                       msg = gzerror(config, &err);
+                       if (err == Z_ERRNO)
+                               perror("gzgets /proc/config.gz");
+                       else if (err != Z_OK)
+                               fprintf(stderr, "gzgets /proc/config.gz: %s", msg);
+                       goto out;
+               }
+               n = strlen(buf);
+               if (buf[n - 1] == '\n')
+                       buf[n - 1] = 0;
+               if (strcmp(buf, pat) == 0) {
+                       ret = true;
+                       goto out;
+               }
+       }
+       ret = false;
+out:
+       gzclose(config);
+       return ret;
+}
+
+static bool cmdline_contains(const char *pat)
 {
        char cmdline[4096], *c;
        int fd, ret = false;
@@ -27,7 +88,7 @@ static bool get_mitigations_off(void)
 
        cmdline[sizeof(cmdline) - 1] = '\0';
        for (c = strtok(cmdline, " \n"); c; c = strtok(NULL, " \n")) {
-               if (strncmp(c, "mitigations=off", strlen(c)))
+               if (strncmp(c, pat, strlen(c)))
                        continue;
                ret = true;
                break;
@@ -37,8 +98,21 @@ static bool get_mitigations_off(void)
        return ret;
 }
 
+static int get_mitigations_off(void)
+{
+       int enabled_in_config;
+
+       if (cmdline_contains("mitigations=off"))
+               return true;
+       enabled_in_config = config_contains("CONFIG_CPU_MITIGATIONS=y");
+       if (enabled_in_config < 0)
+               return -1;
+       return !enabled_in_config;
+}
+
 bool get_unpriv_disabled(void)
 {
+       int mitigations_off;
        bool disabled;
        char buf[2];
        FILE *fd;
@@ -52,5 +126,19 @@ bool get_unpriv_disabled(void)
                disabled = true;
        }
 
-       return disabled ? true : get_mitigations_off();
+       if (disabled)
+               return true;
+
+       /*
+        * Some unpriv tests rely on spectre mitigations being on.
+        * If mitigations are off or status can't be determined
+        * assume that unpriv tests are disabled.
+        */
+       mitigations_off = get_mitigations_off();
+       if (mitigations_off < 0) {
+               fprintf(stderr,
+                       "Can't determine if mitigations are enabled, disabling unpriv tests.");
+               return true;
+       }
+       return mitigations_off;
 }
--

2.47.1

Signature · 2 lines
Regards
Yafang
#3 lore raw
In reply to Eduard Zingerman
Quoted 85 lines · Eduard Zingerman

test_progs and test_verifier binaries execute unpriv tests under the following conditions:
- unpriv BPF is enabled;
- CPU mitigations are enabled (see [1] for details).

The detection of the "mitigations enabled" state is performed by unpriv_helpers.c:get_mitigations_off() via inspecting kernel boot command line, looking for a parameter "mitigations=off".

Such detection scheme won't work for certain configurations, e.g. when CONFIG_CPU_MITIGATIONS is disabled and boot parameter is not supplied.

Miss-detection leads to test_progs executing tests meant to be run only with mitigations enabled, e.g. verifier_and.c:known_subreg_with_unknown_reg(), and reporting false failures.

Internally, verifier sets bpf_verifier_env->bypass_spec_{v1,v4} basing on the value returned by kernel/cpu.c:cpu_mitigations_off(). This function is backed by a variable kernel/cpu.c:cpu_mitigations.

This state is not fully introspect-able via sysfs. The closest proxy is /sys/devices/system/cpu/vulnerabilities/spectre_v1, but it reports "vulnerable" state only if mitigations are disabled *and* current cpu is vulnerable, while verifier does not check cpu state.

There are only two ways the kernel/cpu.c:cpu_mitigations can be set:
- via boot parameter;
- via CONFIG_CPU_MITIGATIONS option.

This commit updates unpriv_helpers.c:get_mitigations_off() to scan /boot/config-$(uname -r) and /proc/config.gz for CONFIG_CPU_MITIGATIONS value in addition to boot command line check.

Tested using the following configurations:
- mitigations enabled (unpriv tests are enabled)
- mitigations disabled via boot cmdline (unpriv tests skipped)
- mitigations disabled via CONFIG_CPU_MITIGATIONS (unpriv tests skipped)

[1] https://lore.kernel.org/bpf/[email protected]/

---
 tools/testing/selftests/bpf/unpriv_helpers.c | 94 +++++++++++++++++++-
 1 file changed, 91 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/unpriv_helpers.c +18 -1
index 220f6a963813..625556a0e7f1 100644
@@ -1,15 +1,76 @@
 // SPDX-License-Identifier: GPL-2.0-only
 
+#include <errno.h>
 #include <stdbool.h>
 #include <stdlib.h>
 #include <stdio.h>
 #include <string.h>
+#include <sys/utsname.h>
 #include <unistd.h>
 #include <fcntl.h>
+#include <zlib.h>
 
 #include "unpriv_helpers.h"
 
-static bool get_mitigations_off(void)
+static gzFile open_config(void)
+{
+       struct utsname uts;
+       char buf[PATH_MAX];
+       gzFile config;
+
+       if (uname(&uts)) {
+               perror("uname");
+               goto config_gz;
+       }
+
+       snprintf(buf, sizeof(buf), "/boot/config-%s", uts.release);
+       config = gzopen(buf, "rb");
+       if (config)
+               return config;

tools/testing/selftests/bpf/unpriv_helpers.c line 39

Quoted 10 lines
tools/testing/selftests/bpf/unpriv_helpers.c +10 -0
+       fprintf(stderr, "gzopen %s: %s\n", buf, strerror(errno));
+
+config_gz:
+       config = gzopen("/proc/config.gz", "rb");
+       if (!config)
+               perror("gzopen /proc/config.gz");
+       return config;
+}
+
+static int config_contains(const char *pat)

int-returning function... but we return do ret = true;, that looks accidental and sloppy. Let's add a comment that it returns <0 on error, 0 for no match, 1 for match and stick to numbers everywhere?

tools/testing/selftests/bpf/unpriv_helpers.c line 48

Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9 -0
+{
+       int n, err, ret = -1;
+       const char *msg;
+       char buf[1024];
+       gzFile config;
+
+       config = open_config();
+       if (!config)
+               goto out;

nothing to gzclose if open_config() returns NULL, just return

pw-bot: cr

tools/testing/selftests/bpf/unpriv_helpers.c line 57

Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9 -0
+
+       for (;;) {
+               if (!gzgets(config, buf, sizeof(buf))) {
+                       msg = gzerror(config, &err);
+                       if (err == Z_ERRNO)
+                               perror("gzgets /proc/config.gz");
+                       else if (err != Z_OK)
+                               fprintf(stderr, "gzgets /proc/config.gz: %s", msg);
+                       goto out;

nit: I'd probably just do

gzclose(config);
return -EINVAL; (or whatever the error code might be)

tools/testing/selftests/bpf/unpriv_helpers.c line 64

Quoted 7 lines
tools/testing/selftests/bpf/unpriv_helpers.c +7 -0
+               }
+               n = strlen(buf);
+               if (buf[n - 1] == '\n')
+                       buf[n - 1] = 0;
+               if (strcmp(buf, pat) == 0) {
+                       ret = true;
+                       goto out;

same here, gzclose() + return;

it will be easier to follow in this rather straightforward function

Quoted 25 lines
tools/testing/selftests/bpf/unpriv_helpers.c +10 -1
+               }
+       }
+       ret = false;
+out:
+       gzclose(config);
+       return ret;
+}
+
+static bool cmdline_contains(const char *pat)
 {
        char cmdline[4096], *c;
        int fd, ret = false;
@@ -27,7 +88,7 @@ static bool get_mitigations_off(void)
 
        cmdline[sizeof(cmdline) - 1] = '\0';
        for (c = strtok(cmdline, " \n"); c; c = strtok(NULL, " \n")) {
-               if (strncmp(c, "mitigations=off", strlen(c)))
+               if (strncmp(c, pat, strlen(c)))
                        continue;
                ret = true;
                break;
@@ -37,8 +98,21 @@ static bool get_mitigations_off(void)
        return ret;
 }
 

tools/testing/selftests/bpf/unpriv_helpers.c static bool get_mitigations_off(void) · line 110

Quoted 10 lines
tools/testing/selftests/bpf/unpriv_helpers.c +10 -0
+static int get_mitigations_off(void)
+{
+       int enabled_in_config;
+
+       if (cmdline_contains("mitigations=off"))
+               return true;
+       enabled_in_config = config_contains("CONFIG_CPU_MITIGATIONS=y");
+       if (enabled_in_config < 0)
+               return -1;
+       return !enabled_in_config;

same mix of bool and integers

Quoted 32 lines
tools/testing/selftests/bpf/unpriv_helpers.c +18 -2
+}
+
 bool get_unpriv_disabled(void)
 {
+       int mitigations_off;
        bool disabled;
        char buf[2];
        FILE *fd;
@@ -52,5 +126,19 @@ bool get_unpriv_disabled(void)
                disabled = true;
        }
 
-       return disabled ? true : get_mitigations_off();
+       if (disabled)
+               return true;
+
+       /*
+        * Some unpriv tests rely on spectre mitigations being on.
+        * If mitigations are off or status can't be determined
+        * assume that unpriv tests are disabled.
+        */
+       mitigations_off = get_mitigations_off();
+       if (mitigations_off < 0) {
+               fprintf(stderr,
+                       "Can't determine if mitigations are enabled, disabling unpriv tests.");
+               return true;
+       }
+       return mitigations_off;
 }
--

2.47.1

author maintainer
#4 lore raw

On Mon, 2025-06-16 at 14:13 -0700, Andrii Nakryiko wrote:

[...]

Quoted 5 lines
Quoted 1 line

+static int config_contains(const char *pat)

int-returning function... but we return do ret = true;, that looks accidental and sloppy. Let's add a comment that it returns <0 on error, 0 for no match, 1 for match and stick to numbers everywhere?

It was actually intentional but ok, I can change that.

Quoted 24 lines

tools/testing/selftests/bpf/unpriv_helpers.c line 48

Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9 -0
+{
+       int n, err, ret = -1;
+       const char *msg;
+       char buf[1024];
+       gzFile config;
+
+       config = open_config();
+       if (!config)
+               goto out;

nothing to gzclose if open_config() returns NULL, just return

pw-bot: cr

tools/testing/selftests/bpf/unpriv_helpers.c line 57

Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9 -0
+
+       for (;;) {
+               if (!gzgets(config, buf, sizeof(buf))) {
+                       msg = gzerror(config, &err);
+                       if (err == Z_ERRNO)
+                               perror("gzgets /proc/config.gz");
+                       else if (err != Z_OK)
+                               fprintf(stderr, "gzgets /proc/config.gz: %s", msg);
+                       goto out;
Quoted 4 lines

nit: I'd probably just do

gzclose(config);
return -EINVAL; (or whatever the error code might be)

I'll make this change just to avoid arguing but don't understand why tbh.

[...]

#5 lore raw
Quoted 5 lines · Eduard Zingerman

On Mon, 2025-06-16 at 14:13 -0700, Andrii Nakryiko wrote:

[...]

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

+static int config_contains(const char *pat)

int-returning function... but we return do ret = true;, that looks accidental and sloppy. Let's add a comment that it returns <0 on error, 0 for no match, 1 for match and stick to numbers everywhere?

It was actually intentional but ok, I can change that.

Thanks, I wouldn't mix true/false and integers, even though false -> 0, true -> 1 implicit conversion is well defined.

Quoted 25 lines
Quoted 25 lines

tools/testing/selftests/bpf/unpriv_helpers.c line 48

Quoted 10 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9 -0
+{
+       int n, err, ret = -1;
+       const char *msg;
+       char buf[1024];
+       gzFile config;
+
+       config = open_config();
+       if (!config)
+               goto out;

nothing to gzclose if open_config() returns NULL, just return

pw-bot: cr

Quoted 9 lines
tools/testing/selftests/bpf/unpriv_helpers.c +9 -0
+
+       for (;;) {
+               if (!gzgets(config, buf, sizeof(buf))) {
+                       msg = gzerror(config, &err);
+                       if (err == Z_ERRNO)
+                               perror("gzgets /proc/config.gz");
+                       else if (err != Z_OK)
+                               fprintf(stderr, "gzgets /proc/config.gz: %s", msg);
+                       goto out;
Quoted 6 lines
Quoted 4 lines

nit: I'd probably just do

gzclose(config);
return -EINVAL; (or whatever the error code might be)

I'll make this change just to avoid arguing but don't understand why tbh.

I don't insist, but early return is easier for me to follow because it clearly states that nothing is going to happen after that point, so whoever is reading the code doesn't have to jump around the code to see if there is some follow up logic.

It's similar (for me) to the common pattern of

for (i = 0; i < n; i++) {
if (<some condition>) {
... here goes multi-line processing logic ...
}
}

I find it much more mentally draining than more obvious (to me)

for (i = 0; i < n; i++) {
if (!<some condition)
continue
... same logic, -1 level of nestedness, but really it's just to
know there is no other logic past <some condition> ...
}

It's all subjective, of course.

Quoted 2 lines

[...]