Skip to contentSkip to conversation

bpf: Fix errors in verifying a union

PATCH Applied Yafang Shao 2 participants 6 messages Jul 13, 2023

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover bpf: Fix errors in verifying a union

From: root <[email protected]>

Patch #1: Fix an issue found in code review Patch #2: Selftest for #1
Patch #3: Fix an issue found in our dev server Patch #4: Selftest for #3

Changes:
- bpf: Introduce union trusted
https://lore.kernel.org/bpf/[email protected]/
- bpf: Fix errors in verifying a union
https://lore.kernel.org/bpf/[email protected]/

Yafang Shao (4):
bpf: Fix an error around PTR_UNTRUSTED selftests/bpf: Add selftests for nested_trust
bpf: Fix an error in verifying a field in a union selftests/bpf: Add selftest for PTR_UNTRUSTED

 kernel/bpf/btf.c                              | 22 ++++++------
 kernel/bpf/verifier.c                         |  5 +++
 .../selftests/bpf/prog_tests/ptr_untrusted.c  | 36 +++++++++++++++++++
 .../bpf/progs/nested_trust_failure.c          | 16 +++++++++
 .../bpf/progs/nested_trust_success.c          | 15 ++++++++
 .../selftests/bpf/progs/test_ptr_untrusted.c  | 29 +++++++++++++++
 6 files changed, 111 insertions(+), 12 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/ptr_untrusted.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_ptr_untrusted.c
Signature · 1 line
2.39.3
#1 lore raw

PATCH v2 1/4 bpf: Fix an error around PTR_UNTRUSTED

Per discussion with Alexei, the PTR_UNTRUSTED flag should not been cleared when we start to walk a new struct, because the struct in question may be a struct nested in a union. We should also check and set this flag before we walk its each member, in case itself is a union. We will clear this flag if the field is BTF_TYPE_SAFE_RCU_OR_NULL.

Fixes 6fcd486b3a0a ("bpf: Refactor RCU enforcement in the verifier.")
---
 kernel/bpf/btf.c      | 20 +++++++++-----------
 kernel/bpf/verifier.c |  5 +++++
 2 files changed, 14 insertions(+), 11 deletions(-)
kernel/bpf/btf.c +9 -11
index 3dd47451f097..fae6fc24a845 100644
@@ -6133,7 +6133,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 	const char *tname, *mname, *tag_value;
 	u32 vlen, elem_id, mid;
 
-	*flag = 0;
 again:
 	if (btf_type_is_modifier(t))
 		t = btf_type_skip_modifiers(btf, t->type, NULL);
@@ -6144,6 +6143,14 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 	}
 
 	vlen = btf_type_vlen(t);
+	if (BTF_INFO_KIND(t->info) == BTF_KIND_UNION && vlen != 1 && !(*flag & PTR_UNTRUSTED))
+		/*
+		 * walking unions yields untrusted pointers
+		 * with exception of __bpf_md_ptr and other
+		 * unions with a single member
+		 */
+		*flag |= PTR_UNTRUSTED;
+
 	if (off + size > t->size) {
 		/* If the last element is a variable size array, we may
 		 * need to relax the rule.
@@ -6304,15 +6311,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 		 * of this field or inside of this struct
 		 */
 		if (btf_type_is_struct(mtype)) {
-			if (BTF_INFO_KIND(mtype->info) == BTF_KIND_UNION &&
-			    btf_type_vlen(mtype) != 1)
-				/*
-				 * walking unions yields untrusted pointers
-				 * with exception of __bpf_md_ptr and other
-				 * unions with a single member
-				 */
-				*flag |= PTR_UNTRUSTED;
-
 			/* our field must be inside that union or struct */
 			t = mtype;
 
@@ -6478,7 +6476,7 @@ bool btf_struct_ids_match(struct bpf_verifier_log *log,
 			  bool strict)
 {
 	const struct btf_type *type;
-	enum bpf_type_flag flag;
+	enum bpf_type_flag flag = 0;
 	int err;
 
 	/* Are we already done? */
kernel/bpf/verifier.c +5 -0
index 81a93eeac7a0..584eb34dce8a 100644
@@ -6067,6 +6067,11 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env,
 				   type_is_rcu_or_null(env, reg, field_name, btf_id)) {
 				/* __rcu tagged pointers can be NULL */
 				flag |= MEM_RCU | PTR_MAYBE_NULL;
+
+				/* We always trust them */
+				if (type_is_rcu_or_null(env, reg, field_name, btf_id) &&
+				    flag & PTR_UNTRUSTED)
+					flag &= ~PTR_UNTRUSTED;
 			} else if (flag & (MEM_PERCPU | MEM_USER)) {
 				/* keep as-is */
 			} else {
Signature · 1 line
2.39.3
#2 lore raw

PATCH v2 2/4 selftests/bpf: Add selftests for nested_trust

In reply to Yafang Shao

Add selftests for nested_strust to check whehter PTR_UNTRUSTED is cleared as expected, the result as follows:

 #141/1   nested_trust/test_read_cpumask:OK
 #141/2   nested_trust/test_skb_field:OK                    <<<<
 #141/3   nested_trust/test_invalid_nested_user_cpus:OK
 #141/4   nested_trust/test_invalid_nested_offset:OK
 #141/5   nested_trust/test_invalid_skb_field:OK            <<<<
 #141     nested_trust:OK

The #141/2 and #141/5 are newly added.

---
 .../selftests/bpf/progs/nested_trust_failure.c   | 16 ++++++++++++++++
 .../selftests/bpf/progs/nested_trust_success.c   | 15 +++++++++++++++
 2 files changed, 31 insertions(+)
tools/testing/selftests/bpf/progs/nested_trust_failure.c +16 -0
index 0d1aa6bbace4..ea39497f11ed 100644
@@ -10,6 +10,13 @@
 
 char _license[] SEC("license") = "GPL";
 
+struct {
+	__uint(type, BPF_MAP_TYPE_SK_STORAGE);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__type(key, int);
+	__type(value, u64);
+} sk_storage_map SEC(".maps");
+
 /* Prototype for all of the program trace events below:
  *
  * TRACE_EVENT(task_newtask,
@@ -31,3 +38,12 @@ int BPF_PROG(test_invalid_nested_offset, struct task_struct *task, u64 clone_fla
 	bpf_cpumask_first_zero(&task->cpus_mask);
 	return 0;
 }
+
+/* Although R2 is of type sk_buff but sock_common is expected, we will hit untrusted ptr first. */
+SEC("tp_btf/tcp_probe")
+__failure __msg("R2 type=untrusted_ptr_ expected=ptr_, trusted_ptr_, rcu_ptr_")
+int BPF_PROG(test_invalid_skb_field, struct sock *sk, struct sk_buff *skb)
+{
+	bpf_sk_storage_get(&sk_storage_map, skb->next, 0, 0);
+	return 0;
+}
tools/testing/selftests/bpf/progs/nested_trust_success.c +15 -0
index 886ade4aa99d..833840bffd3b 100644
@@ -10,6 +10,13 @@
 
 char _license[] SEC("license") = "GPL";
 
+struct {
+	__uint(type, BPF_MAP_TYPE_SK_STORAGE);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__type(key, int);
+	__type(value, u64);
+} sk_storage_map SEC(".maps");
+
 SEC("tp_btf/task_newtask")
 __success
 int BPF_PROG(test_read_cpumask, struct task_struct *task, u64 clone_flags)
@@ -17,3 +24,11 @@ int BPF_PROG(test_read_cpumask, struct task_struct *task, u64 clone_flags)
 	bpf_cpumask_test_cpu(0, task->cpus_ptr);
 	return 0;
 }
+
+SEC("tp_btf/tcp_probe")
+__success
+int BPF_PROG(test_skb_field, struct sock *sk, struct sk_buff *skb)
+{
+	bpf_sk_storage_get(&sk_storage_map, skb->sk, 0, 0);
+	return 0;
+}
Signature · 1 line
2.39.3
#3 lore raw

PATCH v2 3/4 bpf: Fix an error in verifying a field in a union

In reply to Yafang Shao

We are utilizing BPF LSM to monitor BPF operations within our container environment. When we add support for raw_tracepoint, it hits below error.

; (const void *)attr->raw_tracepoint.name);
27: (79) r3 = *(u64 *)(r2 +0)
access beyond the end of member map_type (mend:4) in struct (anon) with off 0 size 8

It can be reproduced with below BPF prog.

SEC("lsm/bpf")
int BPF_PROG(bpf_audit, int cmd, union bpf_attr *attr, unsigned int size)

{
	switch (cmd) {
	case BPF_RAW_TRACEPOINT_OPEN:
		bpf_printk("raw_tracepoint is %s", attr->raw_tracepoint.name);
		break;
	default:
		break;
	}
	return 0;
}

The reason is that when accessing a field in a union, such as bpf_attr, if the field is located within a nested struct that is not the first member of the union, it can result in incorrect field verification.

  union bpf_attr {
      struct {
          __u32 map_type; <<<< Actually it will find that field.
          __u32 key_size;
          __u32 value_size;
         ...
      };
      ...
      struct {
          __u64 name;    <<<< We want to verify this field.
          __u32 prog_fd;
      } raw_tracepoint;
  };

Considering the potential deep nesting levels, finding a perfect solution to address this issue has proven challenging. Therefore, I propose a solution where we simply skip the verification process if the field in question is located within a union.

Fixes 7e3617a72df3 ("bpf: Add array support to btf_struct_access")
---
 kernel/bpf/btf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/bpf/btf.c +1 -1
index fae6fc24a845..a542760c807a 100644
@@ -6368,7 +6368,7 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 		 * that also allows using an array of int as a scratch
 		 * space. e.g. skb->cb[].
 		 */
-		if (off + size > mtrue_end) {
+		if (off + size > mtrue_end && !(*flag & PTR_UNTRUSTED)) {
 			bpf_log(log,
 				"access beyond the end of member %s (mend:%u) in struct %s with off %u size %u\n",
 				mname, mtrue_end, tname, off, size);
Signature · 1 line
2.39.3
#4 lore raw

PATCH v2 4/4 selftests/bpf: Add selftest for PTR_UNTRUSTED

In reply to Yafang Shao

Add a new selftest to check the PTR_UNTRUSTED condition. Below is the result,

#160 ptr_untrusted:OK

---
 .../selftests/bpf/prog_tests/ptr_untrusted.c  | 36 +++++++++++++++++++
 .../selftests/bpf/progs/test_ptr_untrusted.c  | 29 +++++++++++++++
 2 files changed, 65 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/ptr_untrusted.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_ptr_untrusted.c
tools/testing/selftests/bpf/prog_tests/ptr_untrusted.c new file +36 -0
new file mode 100644
index 000000000000..8d077d150c56
@@ -0,0 +1,36 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2023 Yafang Shao <[email protected]> */
+
+#include <string.h>
+#include <linux/bpf.h>
+#include <test_progs.h>
+#include "test_ptr_untrusted.skel.h"
+
+#define TP_NAME "sched_switch"
+
+void serial_test_ptr_untrusted(void)
+{
+	struct test_ptr_untrusted *skel;
+	int err;
+
+	skel = test_ptr_untrusted__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "skel_open"))
+		goto cleanup;
+
+	/* First, attach lsm prog */
+	skel->links.lsm_run = bpf_program__attach_lsm(skel->progs.lsm_run);
+	if (!ASSERT_OK_PTR(skel->links.lsm_run, "lsm_attach"))
+		goto cleanup;
+
+	/* Second, attach raw_tp prog. The lsm prog will be triggered. */
+	skel->links.raw_tp_run = bpf_program__attach_raw_tracepoint(skel->progs.raw_tp_run,
+								    TP_NAME);
+	if (!ASSERT_OK_PTR(skel->links.raw_tp_run, "raw_tp_attach"))
+		goto cleanup;
+
+	err = strncmp(skel->bss->tp_name, TP_NAME, strlen(TP_NAME));
+	ASSERT_EQ(err, 0, "cmp_tp_name");
+
+cleanup:
+	test_ptr_untrusted__destroy(skel);
+}
tools/testing/selftests/bpf/progs/test_ptr_untrusted.c new file +29 -0
new file mode 100644
index 000000000000..4bdd65b5aa2d
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2023 Yafang Shao <[email protected]> */
+
+#include "vmlinux.h"
+#include <bpf/bpf_tracing.h>
+
+char tp_name[128];
+
+SEC("lsm/bpf")
+int BPF_PROG(lsm_run, int cmd, union bpf_attr *attr, unsigned int size)
+{
+	switch (cmd) {
+	case BPF_RAW_TRACEPOINT_OPEN:
+		bpf_probe_read_user_str(tp_name, sizeof(tp_name) - 1,
+					(void *)attr->raw_tracepoint.name);
+		break;
+	default:
+		break;
+	}
+	return 0;
+}
+
+SEC("raw_tracepoint")
+int BPF_PROG(raw_tp_run)
+{
+	return 0;
+}
+
+char _license[] SEC("license") = "GPL";
Signature · 1 line
2.39.3
#5 lore raw
In reply to Yafang Shao
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 8 lines · On Thu, 13 Jul 2023 02:56:38 +0000 you wrote:

From: root <[email protected]>

Patch #1: Fix an issue found in code review Patch #2: Selftest for #1
Patch #3: Fix an issue found in our dev server Patch #4: Selftest for #3

[...]

Here is the summary with links:
- [v2,bpf-next,1/4] bpf: Fix an error around PTR_UNTRUSTED

    https://git.kernel.org/bpf/bpf-next/c/7ce4dc3e4a9d
  - [v2,bpf-next,2/4] selftests/bpf: Add selftests for nested_trust
    https://git.kernel.org/bpf/bpf-next/c/d2284d68259c
  - [v2,bpf-next,3/4] bpf: Fix an error in verifying a field in a union
    https://git.kernel.org/bpf/bpf-next/c/33937607efa0
  - [v2,bpf-next,4/4] selftests/bpf: Add selftest for PTR_UNTRUSTED
    https://git.kernel.org/bpf/bpf-next/c/1cd0e7715cad

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html