Skip to contentSkip to conversation

bpf: Fix errors in verifying a union

PATCH Superseded Yafang Shao 2 participants 6 messages Jun 28, 2023 – Jun 29, 2023 BPF

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover bpf: Fix errors in verifying a union

Patch #1: Fix an issue found in code review
Patch #2: Fix an issue found in our dev server

Yafang Shao (2):
bpf: Fix an error around PTR_UNTRUSTED
bpf: Fix an error in verifying a field in a union

 kernel/bpf/btf.c | 22 ++++++++++------------
 1 file changed, 10 insertions(+), 12 deletions(-)
Signature · 1 line
2.39.3
#1 lore raw

PATCH 1/2 bpf: Fix an error around PTR_UNTRUSTED

Per discussion with Alexei, the PTR_UNTRUSTED flag should not been cleared when we start to walk a new struct, because the struct in question may be a struct nested in a union. We should also check and set this flag before we walk its each member, in case itself is a union.

Fixes 6fcd486b3a0a ("bpf: Refactor RCU enforcement in the verifier.")
---
 kernel/bpf/btf.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)
kernel/bpf/btf.c +9 -11
index 29fe21099298..e0a493230727 100644
@@ -6133,7 +6133,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 	const char *tname, *mname, *tag_value;
 	u32 vlen, elem_id, mid;
 
-	*flag = 0;
 again:
 	tname = __btf_name_by_offset(btf, t->name_off);
 	if (!btf_type_is_struct(t)) {
@@ -6142,6 +6141,14 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 	}
 
 	vlen = btf_type_vlen(t);
+	if (BTF_INFO_KIND(t->info) == BTF_KIND_UNION && vlen != 1)
+		/*
+		 * walking unions yields untrusted pointers
+		 * with exception of __bpf_md_ptr and other
+		 * unions with a single member
+		 */
+		*flag |= PTR_UNTRUSTED;
+
 	if (off + size > t->size) {
 		/* If the last element is a variable size array, we may
 		 * need to relax the rule.
@@ -6302,15 +6309,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 		 * of this field or inside of this struct
 		 */
 		if (btf_type_is_struct(mtype)) {
-			if (BTF_INFO_KIND(mtype->info) == BTF_KIND_UNION &&
-			    btf_type_vlen(mtype) != 1)
-				/*
-				 * walking unions yields untrusted pointers
-				 * with exception of __bpf_md_ptr and other
-				 * unions with a single member
-				 */
-				*flag |= PTR_UNTRUSTED;
-
 			/* our field must be inside that union or struct */
 			t = mtype;
 
@@ -6476,7 +6474,7 @@ bool btf_struct_ids_match(struct bpf_verifier_log *log,
 			  bool strict)
 {
 	const struct btf_type *type;
-	enum bpf_type_flag flag;
+	enum bpf_type_flag flag = 0;
 	int err;
 
 	/* Are we already done? */
Signature · 1 line
2.39.3
#2 lore raw

Re: [PATCH bpf-next 1/2] bpf: Fix an error around PTR_UNTRUSTED

Quoted 65 lines · Yafang Shao

Per discussion with Alexei, the PTR_UNTRUSTED flag should not been cleared when we start to walk a new struct, because the struct in question may be a struct nested in a union. We should also check and set this flag before we walk its each member, in case itself is a union.

Fixes 6fcd486b3a0a ("bpf: Refactor RCU enforcement in the verifier.")
---
 kernel/bpf/btf.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)
kernel/bpf/btf.c +9 -12
index 29fe21099298..e0a493230727 100644
@@ -6133,7 +6133,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
        const char *tname, *mname, *tag_value;
        u32 vlen, elem_id, mid;
 
-       *flag = 0;
 again:
        tname = __btf_name_by_offset(btf, t->name_off);
        if (!btf_type_is_struct(t)) {
@@ -6142,6 +6141,14 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
        }
 
        vlen = btf_type_vlen(t);
+       if (BTF_INFO_KIND(t->info) == BTF_KIND_UNION && vlen != 1)
+               /*
+                * walking unions yields untrusted pointers
+                * with exception of __bpf_md_ptr and other
+                * unions with a single member
+                */
+               *flag |= PTR_UNTRUSTED;
+
        if (off + size > t->size) {
                /* If the last element is a variable size array, we may
                 * need to relax the rule.
@@ -6302,15 +6309,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
                 * of this field or inside of this struct
                 */
                if (btf_type_is_struct(mtype)) {
-                       if (BTF_INFO_KIND(mtype->info) == BTF_KIND_UNION &&
-                           btf_type_vlen(mtype) != 1)
-                               /*
-                                * walking unions yields untrusted pointers
-                                * with exception of __bpf_md_ptr and other
-                                * unions with a single member
-                                */
-                               *flag |= PTR_UNTRUSTED;
-
                        /* our field must be inside that union or struct */
                        t = mtype;
 
@@ -6476,7 +6474,7 @@ bool btf_struct_ids_match(struct bpf_verifier_log *log,
                          bool strict)
 {
        const struct btf_type *type;
-       enum bpf_type_flag flag;
+       enum bpf_type_flag flag = 0;
        int err;
 
        /* Are we already done? */
--
Quoted 2 lines

2.39.3

Just noticed that it breaks test_sk_storage_tracing, because skb->sk is in a union:

   struct sk_buff {
       ...
       union {
           struct sock             *sk;
           int                     ip_defrag_offset;
       };
       ...

};

I will think about it.

Signature · 2 lines
Regards
Yafang
#3 lore raw

Re: [PATCH bpf-next 1/2] bpf: Fix an error around PTR_UNTRUSTED

Quoted 81 lines · Yafang Shao
Quoted 65 lines · Yafang Shao

Per discussion with Alexei, the PTR_UNTRUSTED flag should not been cleared when we start to walk a new struct, because the struct in question may be a struct nested in a union. We should also check and set this flag before we walk its each member, in case itself is a union.

Fixes 6fcd486b3a0a ("bpf: Refactor RCU enforcement in the verifier.")
---
 kernel/bpf/btf.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)
kernel/bpf/btf.c +9 -12
index 29fe21099298..e0a493230727 100644
@@ -6133,7 +6133,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
        const char *tname, *mname, *tag_value;
        u32 vlen, elem_id, mid;
 
-       *flag = 0;
 again:
        tname = __btf_name_by_offset(btf, t->name_off);
        if (!btf_type_is_struct(t)) {
@@ -6142,6 +6141,14 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
        }
 
        vlen = btf_type_vlen(t);
+       if (BTF_INFO_KIND(t->info) == BTF_KIND_UNION && vlen != 1)
+               /*
+                * walking unions yields untrusted pointers
+                * with exception of __bpf_md_ptr and other
+                * unions with a single member
+                */
+               *flag |= PTR_UNTRUSTED;
+
        if (off + size > t->size) {
                /* If the last element is a variable size array, we may
                 * need to relax the rule.
@@ -6302,15 +6309,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
                 * of this field or inside of this struct
                 */
                if (btf_type_is_struct(mtype)) {
-                       if (BTF_INFO_KIND(mtype->info) == BTF_KIND_UNION &&
-                           btf_type_vlen(mtype) != 1)
-                               /*
-                                * walking unions yields untrusted pointers
-                                * with exception of __bpf_md_ptr and other
-                                * unions with a single member
-                                */
-                               *flag |= PTR_UNTRUSTED;
-
                        /* our field must be inside that union or struct */
                        t = mtype;
 
@@ -6476,7 +6474,7 @@ bool btf_struct_ids_match(struct bpf_verifier_log *log,
                          bool strict)
 {
        const struct btf_type *type;
-       enum bpf_type_flag flag;
+       enum bpf_type_flag flag = 0;
        int err;
 
        /* Are we already done? */
--
Quoted 2 lines

2.39.3

Just noticed that it breaks test_sk_storage_tracing, because skb->sk is in a union:

   struct sk_buff {
       ...
       union {
           struct sock             *sk;
           int                     ip_defrag_offset;
       };
       ...
   };
Quoted 1 line

I will think about it.

It can be whitelisted similar to BTF_TYPE_SAFE_*.
Please add a selftest for the new feature.

#4 lore raw

Re: [PATCH bpf-next 1/2] bpf: Fix an error around PTR_UNTRUSTED

Quoted 83 lines · Alexei Starovoitov
Quoted 81 lines · Yafang Shao
Quoted 65 lines · Yafang Shao

Per discussion with Alexei, the PTR_UNTRUSTED flag should not been cleared when we start to walk a new struct, because the struct in question may be a struct nested in a union. We should also check and set this flag before we walk its each member, in case itself is a union.

Fixes 6fcd486b3a0a ("bpf: Refactor RCU enforcement in the verifier.")
---
 kernel/bpf/btf.c | 20 +++++++++-----------
 1 file changed, 9 insertions(+), 11 deletions(-)
kernel/bpf/btf.c +9 -12
index 29fe21099298..e0a493230727 100644
@@ -6133,7 +6133,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
        const char *tname, *mname, *tag_value;
        u32 vlen, elem_id, mid;
 
-       *flag = 0;
 again:
        tname = __btf_name_by_offset(btf, t->name_off);
        if (!btf_type_is_struct(t)) {
@@ -6142,6 +6141,14 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
        }
 
        vlen = btf_type_vlen(t);
+       if (BTF_INFO_KIND(t->info) == BTF_KIND_UNION && vlen != 1)
+               /*
+                * walking unions yields untrusted pointers
+                * with exception of __bpf_md_ptr and other
+                * unions with a single member
+                */
+               *flag |= PTR_UNTRUSTED;
+
        if (off + size > t->size) {
                /* If the last element is a variable size array, we may
                 * need to relax the rule.
@@ -6302,15 +6309,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
                 * of this field or inside of this struct
                 */
                if (btf_type_is_struct(mtype)) {
-                       if (BTF_INFO_KIND(mtype->info) == BTF_KIND_UNION &&
-                           btf_type_vlen(mtype) != 1)
-                               /*
-                                * walking unions yields untrusted pointers
-                                * with exception of __bpf_md_ptr and other
-                                * unions with a single member
-                                */
-                               *flag |= PTR_UNTRUSTED;
-
                        /* our field must be inside that union or struct */
                        t = mtype;
 
@@ -6476,7 +6474,7 @@ bool btf_struct_ids_match(struct bpf_verifier_log *log,
                          bool strict)
 {
        const struct btf_type *type;
-       enum bpf_type_flag flag;
+       enum bpf_type_flag flag = 0;
        int err;
 
        /* Are we already done? */
--
Quoted 2 lines

2.39.3

Just noticed that it breaks test_sk_storage_tracing, because skb->sk is in a union:

   struct sk_buff {
       ...
       union {
           struct sock             *sk;
           int                     ip_defrag_offset;
       };
       ...
   };
Quoted 3 lines
Quoted 1 line

I will think about it.

It can be whitelisted similar to BTF_TYPE_SAFE_*.

Got it.

Quoted 1 line

Please add a selftest for the new feature.

Sure, will add it.

Signature · 2 lines
Regards
Yafang
#5 lore raw

PATCH 2/2 bpf: Fix an error in verifying a field in a union

In reply to Yafang Shao

We are utilizing BPF LSM to monitor BPF operations within our container environment. When we add support for raw_tracepoint, it hits below error.

; (const void *)attr->raw_tracepoint.name);
27: (79) r3 = *(u64 *)(r2 +0)
access beyond the end of member map_type (mend:4) in struct (anon) with off 0 size 8

It can be reproduced with below BPF prog.

SEC("lsm/bpf")
int BPF_PROG(bpf_audit, int cmd, union bpf_attr *attr, unsigned int size)

{
	switch (cmd) {
	case BPF_RAW_TRACEPOINT_OPEN:
		bpf_printk("raw_tracepoint is %s", attr->raw_tracepoint.name);
		break;
	default:
		break;
	}
	return 0;
}

The reason is that when accessing a field in a union, such as bpf_attr, if the field is located within a nested struct that is not the first member of the union, it can result in incorrect field verification.

  union bpf_attr {
      struct {
          __u32 map_type; <<<< Actually it will find that field.
          __u32 key_size;
          __u32 value_size;
         ...
      };
      ...
      struct {
          __u64 name;    <<<< We want to verify this field.
          __u32 prog_fd;
      } raw_tracepoint;
  };

Considering the potential deep nesting levels, finding a perfect solution to address this issue has proven challenging. Therefore, I propose a solution where we simply skip the verification process if the field in question is located within a union.

Fixes 7e3617a72df3 ("bpf: Add array support to btf_struct_access")
---
 kernel/bpf/btf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/bpf/btf.c +1 -1
index e0a493230727..8ad27b16bc8b 100644
@@ -6366,7 +6366,7 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 		 * that also allows using an array of int as a scratch
 		 * space. e.g. skb->cb[].
 		 */
-		if (off + size > mtrue_end) {
+		if (off + size > mtrue_end && !(*flag & PTR_UNTRUSTED)) {
 			bpf_log(log,
 				"access beyond the end of member %s (mend:%u) in struct %s with off %u size %u\n",
 				mname, mtrue_end, tname, off, size);
Signature · 1 line
2.39.3