Skip to contentSkip to conversation

bpf: add __percpu tagging in vmlinux BTF

PATCH Applied Hao Luo 5 participants 15 messages Mar 4, 2022 – Mar 9, 2022

Subsystems (5)
lore
Share & tools
lore

Hao Luo

author
#0 lore raw

cover bpf: add __percpu tagging in vmlinux BTF

This patchset is very much similar to Yonghong's patchset on adding __user tagging [1], where a "user" btf_type_tag was introduced to describe __user memory pointers. Similar approach can be applied on __percpu pointers. The __percpu attribute in kernel is used to identify pointers that point to memory allocated in percpu region. Normally, accessing __percpu memory requires using special functions like per_cpu_ptr() etc. Directly accessing __percpu pointer is meaningless.

Currently vmlinux BTF does not have a way to differentiate a __percpu pointer from a regular pointer. So BPF programs are allowed to load __percpu memory directly, which is an incorrect behavior.

With the previous work that encodes __user information in BTF, a nice framework has been set up to allow us to encode __percpu information in BTF and let the verifier to reject programs that try to directly access percpu pointer. Previously, there is a PTR_TO_PERCPU_BTF_ID reg type which is used to represent those percpu static variables in the kernel. Pahole is able to collect variables that are stored in ".data..percpu" section in the kernel image and emit BTF information for those variables. The bpf_per_cpu_ptr() and bpf_this_cpu_ptr() helper functions were added to access these variables. Now with __percpu information, we can tag those __percpu fields in a struct (such as cgroup->rstat_cpu) and allow the pair of bpf percpu helpers to access them as well.

In addition to adding __percpu tagging, this patchset also fixes a harmless bug in the previous patch that introduced __user. Patch 01/04 is for that. Patch 02/04 adds the new attribute "percpu". Patch 03/04 adds MEM_PERCPU tag for PTR_TO_BTF_ID and replaces PTR_TO_PERCPU_BTF_ID with (BTF_ID | MEM_PERCPU). Patch 04/04 refactors the btf_tag test a bit and adds tests for percpu tag.

Like [1], the minimal requirements for btf_type_tag is clang (>= clang14) and pahole (>= 1.23).

[1] https://lore.kernel.org/bpf/[email protected]/t/

Hao Luo (4):
bpf: Fix checking PTR_TO_BTF_ID in check_mem_access
compiler_types: define __percpu as __attribute__((btf_type_tag("percpu")))
bpf: Reject programs that try to load __percpu memory. selftests/bpf: Add a test for btf_type_tag "percpu"

 include/linux/bpf.h                           |  11 +-
 include/linux/compiler_types.h                |   7 +-
 kernel/bpf/btf.c                              |   8 +-
 kernel/bpf/verifier.c                         |  27 +--
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  17 ++
 .../selftests/bpf/prog_tests/btf_tag.c        | 164 ++++++++++++++----
 .../selftests/bpf/progs/btf_type_tag_percpu.c |  66 +++++++
 7 files changed, 256 insertions(+), 44 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/btf_type_tag_percpu.c
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
author
#1 lore raw

PATCH 1/4 bpf: Fix checking PTR_TO_BTF_ID in check_mem_access

With the introduction of MEM_USER in

commit c6f1bfe89ac9 ("bpf: reject program if a __user tagged memory accessed in kernel way")

PTR_TO_BTF_ID can be combined with a MEM_USER tag. Therefore, most likely, when we compare reg_type against PTR_TO_BTF_ID, we want to use the reg's base_type. Previously the check in check_mem_access() wants to say: if the reg is BTF_ID but not NULL, the execution flow falls into the 'then' branch. But now a reg of (BTF_ID | MEM_USER), which should go into the 'then' branch, goes into the 'else'.

The end results before and after this patch are the same: regs tagged with MEM_USER get rejected, but not in a way we intended. So fix the condition, the error message now is correct.

Before (log from commit 696c39011538):

  $ ./test_progs -v -n 22/3
  ...
  libbpf: prog 'test_user1': BPF program load failed: Permission denied
  libbpf: prog 'test_user1': -- BEGIN PROG LOAD LOG --
  R1 type=ctx expected=fp
  0: R1=ctx(id=0,off=0,imm=0) R10=fp0
  ; int BPF_PROG(test_user1, struct bpf_testmod_btf_type_tag_1 *arg)
  0: (79) r1 = *(u64 *)(r1 +0)
  func 'bpf_testmod_test_btf_type_tag_user_1' arg0 has btf_id 136561 type STRUCT 'bpf_testmod_btf_type_tag_1'
  1: R1_w=user_ptr_bpf_testmod_btf_type_tag_1(id=0,off=0,imm=0)
  ; g = arg->a;
  1: (61) r1 = *(u32 *)(r1 +0)
  R1 invalid mem access 'user_ptr_'

Now:

libbpf: prog 'test_user1': BPF program load failed: Permission denied
libbpf: prog 'test_user1': -- BEGIN PROG LOAD LOG --
R1 type=ctx expected=fp
0: R1=ctx(id=0,off=0,imm=0) R10=fp0
; int BPF_PROG(test_user1, struct bpf_testmod_btf_type_tag_1 *arg)
0: (79) r1 = *(u64 *)(r1 +0)
func 'bpf_testmod_test_btf_type_tag_user_1' arg0 has btf_id 104036 type STRUCT 'bpf_testmod_btf_type_tag_1' 1: R1_w=user_ptr_bpf_testmod_btf_type_tag_1(id=0,ref_obj_id=0,off=0,imm=0)
; g = arg->a;
1: (61) r1 = *(u32 *)(r1 +0)
R1 is ptr_bpf_testmod_btf_type_tag_1 access user memory: off=0

Note the error message for the reason of rejection.

Fixes c6f1bfe89ac9 ("bpf: reject program if a __user tagged memory accessed in kernel way")
Cc Yonghong Song <[email protected]>
---
 kernel/bpf/verifier.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/bpf/verifier.c +2 -1
index a57db4b2803c..d63b1f40e029 100644
@@ -4556,7 +4556,8 @@ static int check_mem_access(struct bpf_verifier_env *env, int insn_idx, u32 regn
 		err = check_tp_buffer_access(env, reg, regno, off, size);
 		if (!err && t == BPF_READ && value_regno >= 0)
 			mark_reg_unknown(env, regs, value_regno);
-	} else if (reg->type == PTR_TO_BTF_ID) {
+	} else if (base_type(reg->type) == PTR_TO_BTF_ID &&
+		   !type_may_be_null(reg->type)) {
 		err = check_ptr_to_btf_access(env, regs, regno, off, size, t,
 					      value_regno);
 	} else if (reg->type == CONST_PTR_TO_MAP) {
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
Acked-by on v1 1/4
#2 lore raw

Re: [PATCH bpf-next v1 1/4] bpf: Fix checking PTR_TO_BTF_ID in check_mem_access

Quoted 45 lines · Hao Luo

With the introduction of MEM_USER in

commit c6f1bfe89ac9 ("bpf: reject program if a __user tagged memory accessed in kernel way")

PTR_TO_BTF_ID can be combined with a MEM_USER tag. Therefore, most likely, when we compare reg_type against PTR_TO_BTF_ID, we want to use the reg's base_type. Previously the check in check_mem_access() wants to say: if the reg is BTF_ID but not NULL, the execution flow falls into the 'then' branch. But now a reg of (BTF_ID | MEM_USER), which should go into the 'then' branch, goes into the 'else'.

The end results before and after this patch are the same: regs tagged with MEM_USER get rejected, but not in a way we intended. So fix the condition, the error message now is correct.

Before (log from commit 696c39011538):

   $ ./test_progs -v -n 22/3
   ...
   libbpf: prog 'test_user1': BPF program load failed: Permission denied
   libbpf: prog 'test_user1': -- BEGIN PROG LOAD LOG --
   R1 type=ctx expected=fp
   0: R1=ctx(id=0,off=0,imm=0) R10=fp0
   ; int BPF_PROG(test_user1, struct bpf_testmod_btf_type_tag_1 *arg)
   0: (79) r1 = *(u64 *)(r1 +0)
   func 'bpf_testmod_test_btf_type_tag_user_1' arg0 has btf_id 136561 type STRUCT 'bpf_testmod_btf_type_tag_1'
   1: R1_w=user_ptr_bpf_testmod_btf_type_tag_1(id=0,off=0,imm=0)
   ; g = arg->a;
   1: (61) r1 = *(u32 *)(r1 +0)
   R1 invalid mem access 'user_ptr_'

Now:

libbpf: prog 'test_user1': BPF program load failed: Permission denied
libbpf: prog 'test_user1': -- BEGIN PROG LOAD LOG --
R1 type=ctx expected=fp
0: R1=ctx(id=0,off=0,imm=0) R10=fp0
; int BPF_PROG(test_user1, struct bpf_testmod_btf_type_tag_1 *arg)
0: (79) r1 = *(u64 *)(r1 +0)
func 'bpf_testmod_test_btf_type_tag_user_1' arg0 has btf_id 104036 type STRUCT 'bpf_testmod_btf_type_tag_1' 1: R1_w=user_ptr_bpf_testmod_btf_type_tag_1(id=0,ref_obj_id=0,off=0,imm=0)
; g = arg->a;
1: (61) r1 = *(u32 *)(r1 +0)
R1 is ptr_bpf_testmod_btf_type_tag_1 access user memory: off=0

Quoted 5 lines

Note the error message for the reason of rejection.

Fixes c6f1bfe89ac9 ("bpf: reject program if a __user tagged memory accessed in kernel way")
Cc Yonghong Song <[email protected]>

Thanks for the fix!

author
#3 lore raw

PATCH 2/4 compiler_types: define __percpu as __attribute__((btf_type_tag("percpu")))

In reply to Hao Luo

This is similar to commit 7472d5a642c9 ("compiler_types: define __user as __attribute__((btf_type_tag("user")))"), where a type tag "user" was introduced to identify the pointers that point to user memory. With that change, the newest compile toolchain can encode __user information into vmlinux BTF, which can be used by the BPF verifier to enforce safe program behaviors.

Similarly, we have __percpu attribute, which is mainly used to indicate memory is allocated in percpu region. The __percpu pointers in kernel are supposed to be used together with functions like per_cpu_ptr() and this_cpu_ptr(), which perform necessary calculation on the pointer's base address. Without the btf_type_tag introduced in this patch, __percpu pointers will be treated as regular memory pointers in vmlinux BTF and BPF programs are allowed to directly dereference them, generating incorrect behaviors. Now with "percpu" btf_type_tag, the BPF verifier is able to differentiate __percpu pointers from regular pointers and forbids unexpected behaviors like direct load.

The following is an example similar to the one given in commit 7472d5a642c9:

  [$ ~] cat test.c
  #define __percpu __attribute__((btf_type_tag("percpu")))
  int foo(int __percpu *arg) {
  	return *arg;
  }
  [$ ~] clang -O2 -g -c test.c
  [$ ~] pahole -JV test.o
  ...
  File test.o:
  [1] INT int size=4 nr_bits=32 encoding=SIGNED
  [2] TYPE_TAG percpu type_id=1
  [3] PTR (anon) type_id=2
  [4] FUNC_PROTO (anon) return=1 args=(3 arg)
  [5] FUNC foo type_id=4
  [$ ~]

for the function argument "int __percpu *arg", its type is described as

	PTR -> TYPE_TAG(percpu) -> INT

The kernel can use this information for bpf verification or other use cases.

Like commit 7472d5a642c9, this feature requires clang (>= clang14) and pahole (>= 1.23).

Cc Yonghong Song <[email protected]>
---
 include/linux/compiler_types.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
include/linux/compiler_types.h +6 -1
index 3f31ff400432..223abf43679a 100644
@@ -38,7 +38,12 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
 #  define __user
 # endif
 # define __iomem
-# define __percpu
+# if defined(CONFIG_DEBUG_INFO_BTF) && defined(CONFIG_PAHOLE_HAS_BTF_TAG) && \
+	__has_attribute(btf_type_tag)
+#  define __percpu	__attribute__((btf_type_tag("percpu")))
+# else
+#  define __percpu
+# endif
 # define __rcu
 # define __chk_user_ptr(x)	(void)0
 # define __chk_io_ptr(x)	(void)0
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
Acked-by on v1 2/4
#4 lore raw

Re: [PATCH bpf-next v1 2/4] compiler_types: define __percpu as __attribute__((btf_type_tag("percpu")))

Quoted 39 lines · Hao Luo

This is similar to commit 7472d5a642c9 ("compiler_types: define __user as __attribute__((btf_type_tag("user")))"), where a type tag "user" was introduced to identify the pointers that point to user memory. With that change, the newest compile toolchain can encode __user information into vmlinux BTF, which can be used by the BPF verifier to enforce safe program behaviors.

Similarly, we have __percpu attribute, which is mainly used to indicate memory is allocated in percpu region. The __percpu pointers in kernel are supposed to be used together with functions like per_cpu_ptr() and this_cpu_ptr(), which perform necessary calculation on the pointer's base address. Without the btf_type_tag introduced in this patch, __percpu pointers will be treated as regular memory pointers in vmlinux BTF and BPF programs are allowed to directly dereference them, generating incorrect behaviors. Now with "percpu" btf_type_tag, the BPF verifier is able to differentiate __percpu pointers from regular pointers and forbids unexpected behaviors like direct load.

The following is an example similar to the one given in commit 7472d5a642c9:

   [$ ~] cat test.c
   #define __percpu __attribute__((btf_type_tag("percpu")))
   int foo(int __percpu *arg) {
   	return *arg;
   }
   [$ ~] clang -O2 -g -c test.c
   [$ ~] pahole -JV test.o
   ...
   File test.o:
   [1] INT int size=4 nr_bits=32 encoding=SIGNED
   [2] TYPE_TAG percpu type_id=1
   [3] PTR (anon) type_id=2
   [4] FUNC_PROTO (anon) return=1 args=(3 arg)
   [5] FUNC foo type_id=4
   [$ ~]

for the function argument "int __percpu *arg", its type is described as

	PTR -> TYPE_TAG(percpu) -> INT
Quoted 8 lines

The kernel can use this information for bpf verification or other use cases.

Like commit 7472d5a642c9, this feature requires clang (>= clang14) and pahole (>= 1.23).

Cc Yonghong Song <[email protected]>
#5 lore raw

Re: [PATCH bpf-next v1 2/4] compiler_types: define __percpu as __attribute__((btf_type_tag("percpu")))

In reply to Hao Luo
Quoted 57 lines · Hao Luo

This is similar to commit 7472d5a642c9 ("compiler_types: define __user as __attribute__((btf_type_tag("user")))"), where a type tag "user" was introduced to identify the pointers that point to user memory. With that change, the newest compile toolchain can encode __user information into vmlinux BTF, which can be used by the BPF verifier to enforce safe program behaviors.

Similarly, we have __percpu attribute, which is mainly used to indicate memory is allocated in percpu region. The __percpu pointers in kernel are supposed to be used together with functions like per_cpu_ptr() and this_cpu_ptr(), which perform necessary calculation on the pointer's base address. Without the btf_type_tag introduced in this patch, __percpu pointers will be treated as regular memory pointers in vmlinux BTF and BPF programs are allowed to directly dereference them, generating incorrect behaviors. Now with "percpu" btf_type_tag, the BPF verifier is able to differentiate __percpu pointers from regular pointers and forbids unexpected behaviors like direct load.

The following is an example similar to the one given in commit 7472d5a642c9:

  [$ ~] cat test.c
  #define __percpu __attribute__((btf_type_tag("percpu")))
  int foo(int __percpu *arg) {
        return *arg;
  }
  [$ ~] clang -O2 -g -c test.c
  [$ ~] pahole -JV test.o
  ...
  File test.o:
  [1] INT int size=4 nr_bits=32 encoding=SIGNED
  [2] TYPE_TAG percpu type_id=1
  [3] PTR (anon) type_id=2
  [4] FUNC_PROTO (anon) return=1 args=(3 arg)
  [5] FUNC foo type_id=4
  [$ ~]

for the function argument "int __percpu *arg", its type is described as

        PTR -> TYPE_TAG(percpu) -> INT

The kernel can use this information for bpf verification or other use cases.

Like commit 7472d5a642c9, this feature requires clang (>= clang14) and pahole (>= 1.23).

Cc Yonghong Song <[email protected]>
---
 include/linux/compiler_types.h | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
include/linux/compiler_types.h +0 -0
index 3f31ff400432..223abf43679a 100644

include/linux/compiler_types.h static inline void __chk_io_ptr(const volatile void __iomem *ptr) { } · line 43

Quoted 7 lines
include/linux/compiler_types.h +3 -1
@@ -38,7 +38,12 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
 #  define __user
 # endif
 # define __iomem
-# define __percpu
+# if defined(CONFIG_DEBUG_INFO_BTF) && defined(CONFIG_PAHOLE_HAS_BTF_TAG) && \
+       __has_attribute(btf_type_tag)
+#  define __percpu     __attribute__((btf_type_tag("percpu")))

Maybe let's add

#if defined(CONFIG_DEBUG_INFO_BTF) && defined(CONFIG_PAHOLE_HAS_BTF_TAG) && __has_attribute(btf_type_tag) #define BTF_TYPE_TAG(value) __attribute__((btf_type_tag(#value))) #else
#define BTF_TYPE_TAG(value) /* nothing */
#endif

and use BTF_TYPE_TAG() macro unconditionally everywhere?

Quoted 9 lines
include/linux/compiler_types.h +3 -1
+# else
+#  define __percpu
+# endif
 # define __rcu
 # define __chk_user_ptr(x)     (void)0
 # define __chk_io_ptr(x)       (void)0
--

2.35.1.616.g0bdcbb4464-goog

#6 lore raw

Re: [PATCH bpf-next v1 2/4] compiler_types: define __percpu as __attribute__((btf_type_tag("percpu")))

Quoted 69 lines · Andrii Nakryiko
Quoted 57 lines · Hao Luo

This is similar to commit 7472d5a642c9 ("compiler_types: define __user as __attribute__((btf_type_tag("user")))"), where a type tag "user" was introduced to identify the pointers that point to user memory. With that change, the newest compile toolchain can encode __user information into vmlinux BTF, which can be used by the BPF verifier to enforce safe program behaviors.

Similarly, we have __percpu attribute, which is mainly used to indicate memory is allocated in percpu region. The __percpu pointers in kernel are supposed to be used together with functions like per_cpu_ptr() and this_cpu_ptr(), which perform necessary calculation on the pointer's base address. Without the btf_type_tag introduced in this patch, __percpu pointers will be treated as regular memory pointers in vmlinux BTF and BPF programs are allowed to directly dereference them, generating incorrect behaviors. Now with "percpu" btf_type_tag, the BPF verifier is able to differentiate __percpu pointers from regular pointers and forbids unexpected behaviors like direct load.

The following is an example similar to the one given in commit 7472d5a642c9:

   [$ ~] cat test.c
   #define __percpu __attribute__((btf_type_tag("percpu")))
   int foo(int __percpu *arg) {
         return *arg;
   }
   [$ ~] clang -O2 -g -c test.c
   [$ ~] pahole -JV test.o
   ...
   File test.o:
   [1] INT int size=4 nr_bits=32 encoding=SIGNED
   [2] TYPE_TAG percpu type_id=1
   [3] PTR (anon) type_id=2
   [4] FUNC_PROTO (anon) return=1 args=(3 arg)
   [5] FUNC foo type_id=4
   [$ ~]

for the function argument "int __percpu *arg", its type is described as

         PTR -> TYPE_TAG(percpu) -> INT

The kernel can use this information for bpf verification or other use cases.

Like commit 7472d5a642c9, this feature requires clang (>= clang14) and pahole (>= 1.23).

Cc Yonghong Song <[email protected]>
---
  include/linux/compiler_types.h | 7 ++++++-
  1 file changed, 6 insertions(+), 1 deletion(-)
include/linux/compiler_types.h +0 -0
index 3f31ff400432..223abf43679a 100644

include/linux/compiler_types.h static inline void __chk_io_ptr(const volatile void __iomem *ptr) { } · line 43

Quoted 7 lines
include/linux/compiler_types.h +3 -1
@@ -38,7 +38,12 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
  #  define __user
  # endif
  # define __iomem
-# define __percpu
+# if defined(CONFIG_DEBUG_INFO_BTF) && defined(CONFIG_PAHOLE_HAS_BTF_TAG) && \
+       __has_attribute(btf_type_tag)
+#  define __percpu     __attribute__((btf_type_tag("percpu")))

Maybe let's add

Quoted 8 lines

#if defined(CONFIG_DEBUG_INFO_BTF) && defined(CONFIG_PAHOLE_HAS_BTF_TAG) && __has_attribute(btf_type_tag) #define BTF_TYPE_TAG(value) __attribute__((btf_type_tag(#value))) #else
#define BTF_TYPE_TAG(value) /* nothing */
#endif

and use BTF_TYPE_TAG() macro unconditionally everywhere?

Agree that the above suggestion is a good idea, esp. we may convert others, e.g., __rcu, with btf_type_tag in the future, and a common checking will simplify things a lot.

Hao, could you send a followup patch with Andrii's suggestion?

Quoted 10 lines
Quoted 10 lines
include/linux/compiler_types.h +3 -1
+# else
+#  define __percpu
+# endif
  # define __rcu
  # define __chk_user_ptr(x)     (void)0
  # define __chk_io_ptr(x)       (void)0
--

2.35.1.616.g0bdcbb4464-goog

author
#7 lore raw

Re: [PATCH bpf-next v1 2/4] compiler_types: define __percpu as __attribute__((btf_type_tag("percpu")))

Quoted 6 lines · Yonghong Song
Quoted 2 lines · Andrii Nakryiko
Quoted 1 line · Hao Luo

[...]

Quoted 24 lines
Quoted 24 lines

include/linux/compiler_types.h static inline void __chk_io_ptr(const volatile void __iomem *ptr) { } · line 43

Quoted 13 lines
include/linux/compiler_types.h +3 -1
index 3f31ff400432..223abf43679a 100644
@@ -38,7 +38,12 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }
  #  define __user
  # endif
  # define __iomem
-# define __percpu
+# if defined(CONFIG_DEBUG_INFO_BTF) && defined(CONFIG_PAHOLE_HAS_BTF_TAG) && \
+       __has_attribute(btf_type_tag)
+#  define __percpu     __attribute__((btf_type_tag("percpu")))

Maybe let's add

#if defined(CONFIG_DEBUG_INFO_BTF) && defined(CONFIG_PAHOLE_HAS_BTF_TAG) && __has_attribute(btf_type_tag) #define BTF_TYPE_TAG(value) __attribute__((btf_type_tag(#value))) #else
#define BTF_TYPE_TAG(value) /* nothing */
#endif

Quoted 8 lines
Quoted 1 line

and use BTF_TYPE_TAG() macro unconditionally everywhere?

Agree that the above suggestion is a good idea, esp. we may convert others, e.g., __rcu, with btf_type_tag in the future, and a common checking will simplify things a lot.

Hao, could you send a followup patch with Andrii's suggestion?

No problem. Will send the patch this afternoon.

author
#8 lore raw

PATCH 3/4 bpf: Reject programs that try to load __percpu memory.

In reply to Hao Luo

With the introduction of the btf_type_tag "percpu", we can add a MEM_PERCPU to identify those pointers that point to percpu memory. The ability of differetiating percpu pointers from regular memory pointers have two benefits:

1. It forbids unexpected use of percpu pointers, such as direct loads. In kernel, there are special functions used for accessing percpu memory. Directly loading percpu memory is meaningless. We already have BPF helpers like bpf_per_cpu_ptr() and bpf_this_cpu_ptr() that wrap the kernel percpu functions. So we can now convert percpu pointers into regular pointers in a safe way.

2. Previously, bpf_per_cpu_ptr() and bpf_this_cpu_ptr() only work on PTR_TO_PERCPU_BTF_ID, a special reg_type which describes static percpu variables in kernel (we rely on pahole to encode them into vmlinux BTF). Now, since we can identify __percpu tagged pointers, we can also identify dynamically allocated percpu memory as well. It means we can use bpf_xxx_cpu_ptr() on dynamic percpu memory. This would be very convenient when accessing fields like "cgroup->rstat_cpu".

Cc Yonghong Song <[email protected]>
---
 include/linux/bpf.h   | 11 +++++++++--
 kernel/bpf/btf.c      |  8 +++++++-
 kernel/bpf/verifier.c | 24 ++++++++++++++----------
 3 files changed, 30 insertions(+), 13 deletions(-)
include/linux/bpf.h +9 -2
index f19abc59b6cd..88449fbbe063 100644
@@ -334,7 +334,15 @@ enum bpf_type_flag {
 	/* MEM is in user address space. */
 	MEM_USER		= BIT(3 + BPF_BASE_TYPE_BITS),
 
-	__BPF_TYPE_LAST_FLAG	= MEM_USER,
+	/* MEM is a percpu memory. MEM_PERCPU tags PTR_TO_BTF_ID. When tagged
+	 * with MEM_PERCPU, PTR_TO_BTF_ID _cannot_ be directly accessed. In
+	 * order to drop this tag, it must be passed into bpf_per_cpu_ptr()
+	 * or bpf_this_cpu_ptr(), which will return the pointer corresponding
+	 * to the specified cpu.
+	 */
+	MEM_PERCPU		= BIT(4 + BPF_BASE_TYPE_BITS),
+
+	__BPF_TYPE_LAST_FLAG	= MEM_PERCPU,
 };
 
 /* Max number of base types. */
@@ -516,7 +524,6 @@ enum bpf_reg_type {
 	 */
 	PTR_TO_MEM,		 /* reg points to valid memory region */
 	PTR_TO_BUF,		 /* reg points to a read/write buffer */
-	PTR_TO_PERCPU_BTF_ID,	 /* reg points to a percpu kernel variable */
 	PTR_TO_FUNC,		 /* reg points to a bpf program function */
 	__BPF_REG_TYPE_MAX,
 
kernel/bpf/btf.c +7 -1
index b472cf0c8fdb..f9f2218b27fe 100644
@@ -5057,6 +5057,8 @@ bool btf_ctx_access(int off, int size, enum bpf_access_type type,
 		tag_value = __btf_name_by_offset(btf, t->name_off);
 		if (strcmp(tag_value, "user") == 0)
 			info->reg_type |= MEM_USER;
+		if (strcmp(tag_value, "percpu") == 0)
+			info->reg_type |= MEM_PERCPU;
 	}
 
 	/* skip modifiers */
@@ -5285,12 +5287,16 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,
 				return -EACCES;
 			}
 
-			/* check __user tag */
+			/* check type tag */
 			t = btf_type_by_id(btf, mtype->type);
 			if (btf_type_is_type_tag(t)) {
 				tag_value = __btf_name_by_offset(btf, t->name_off);
+				/* check __user tag */
 				if (strcmp(tag_value, "user") == 0)
 					tmp_flag = MEM_USER;
+				/* check __percpu tag */
+				if (strcmp(tag_value, "percpu") == 0)
+					tmp_flag = MEM_PERCPU;
 			}
 
 			stype = btf_type_skip_modifiers(btf, mtype->type, &id);
kernel/bpf/verifier.c +14 -10
index d63b1f40e029..093e47360d6d 100644
@@ -554,7 +554,6 @@ static const char *reg_type_str(struct bpf_verifier_env *env,
 		[PTR_TO_TP_BUFFER]	= "tp_buffer",
 		[PTR_TO_XDP_SOCK]	= "xdp_sock",
 		[PTR_TO_BTF_ID]		= "ptr_",
-		[PTR_TO_PERCPU_BTF_ID]	= "percpu_ptr_",
 		[PTR_TO_MEM]		= "mem",
 		[PTR_TO_BUF]		= "buf",
 		[PTR_TO_FUNC]		= "func",
@@ -562,8 +561,7 @@ static const char *reg_type_str(struct bpf_verifier_env *env,
 	};
 
 	if (type & PTR_MAYBE_NULL) {
-		if (base_type(type) == PTR_TO_BTF_ID ||
-		    base_type(type) == PTR_TO_PERCPU_BTF_ID)
+		if (base_type(type) == PTR_TO_BTF_ID)
 			strncpy(postfix, "or_null_", 16);
 		else
 			strncpy(postfix, "_or_null", 16);
@@ -575,6 +573,8 @@ static const char *reg_type_str(struct bpf_verifier_env *env,
 		strncpy(prefix, "alloc_", 32);
 	if (type & MEM_USER)
 		strncpy(prefix, "user_", 32);
+	if (type & MEM_PERCPU)
+		strncpy(prefix, "percpu_", 32);
 
 	snprintf(env->type_str_buf, TYPE_STR_BUF_LEN, "%s%s%s",
 		 prefix, str[base_type(type)], postfix);
@@ -697,8 +697,7 @@ static void print_verifier_state(struct bpf_verifier_env *env,
 			const char *sep = "";
 
 			verbose(env, "%s", reg_type_str(env, t));
-			if (base_type(t) == PTR_TO_BTF_ID ||
-			    base_type(t) == PTR_TO_PERCPU_BTF_ID)
+			if (base_type(t) == PTR_TO_BTF_ID)
 				verbose(env, "%s", kernel_type_name(reg->btf, reg->btf_id));
 			verbose(env, "(");
 /*
@@ -2783,7 +2782,6 @@ static bool is_spillable_regtype(enum bpf_reg_type type)
 	case PTR_TO_XDP_SOCK:
 	case PTR_TO_BTF_ID:
 	case PTR_TO_BUF:
-	case PTR_TO_PERCPU_BTF_ID:
 	case PTR_TO_MEM:
 	case PTR_TO_FUNC:
 	case PTR_TO_MAP_KEY:
@@ -4197,6 +4195,13 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env,
 		return -EACCES;
 	}
 
+	if (reg->type & MEM_PERCPU) {
+		verbose(env,
+			"R%d is ptr_%s access percpu memory: off=%d\n",
+			regno, tname, off);
+		return -EACCES;
+	}
+
 	if (env->ops->btf_struct_access) {
 		ret = env->ops->btf_struct_access(&env->log, reg->btf, t,
 						  off, size, atype, &btf_id, &flag);
@@ -4803,7 +4808,7 @@ static int check_stack_range_initialized(
 		}
 
 		if (is_spilled_reg(&state->stack[spi]) &&
-		    state->stack[spi].spilled_ptr.type == PTR_TO_BTF_ID)
+		    base_type(state->stack[spi].spilled_ptr.type) == PTR_TO_BTF_ID)
 			goto mark;
 
 		if (is_spilled_reg(&state->stack[spi]) &&
@@ -5259,7 +5264,7 @@ static const struct bpf_reg_types alloc_mem_types = { .types = { PTR_TO_MEM | ME
 static const struct bpf_reg_types const_map_ptr_types = { .types = { CONST_PTR_TO_MAP } };
 static const struct bpf_reg_types btf_ptr_types = { .types = { PTR_TO_BTF_ID } };
 static const struct bpf_reg_types spin_lock_types = { .types = { PTR_TO_MAP_VALUE } };
-static const struct bpf_reg_types percpu_btf_ptr_types = { .types = { PTR_TO_PERCPU_BTF_ID } };
+static const struct bpf_reg_types percpu_btf_ptr_types = { .types = { PTR_TO_BTF_ID | MEM_PERCPU } };
 static const struct bpf_reg_types func_ptr_types = { .types = { PTR_TO_FUNC } };
 static const struct bpf_reg_types stack_ptr_types = { .types = { PTR_TO_STACK } };
 static const struct bpf_reg_types const_str_ptr_types = { .types = { PTR_TO_MAP_VALUE } };
@@ -9639,7 +9644,6 @@ static int check_ld_imm(struct bpf_verifier_env *env, struct bpf_insn *insn)
 			dst_reg->mem_size = aux->btf_var.mem_size;
 			break;
 		case PTR_TO_BTF_ID:
-		case PTR_TO_PERCPU_BTF_ID:
 			dst_reg->btf = aux->btf_var.btf;
 			dst_reg->btf_id = aux->btf_var.btf_id;
 			break;
@@ -11839,7 +11843,7 @@ static int check_pseudo_btf_id(struct bpf_verifier_env *env,
 	type = t->type;
 	t = btf_type_skip_modifiers(btf, type, NULL);
 	if (percpu) {
-		aux->btf_var.reg_type = PTR_TO_PERCPU_BTF_ID;
+		aux->btf_var.reg_type = PTR_TO_BTF_ID | MEM_PERCPU;
 		aux->btf_var.btf = btf;
 		aux->btf_var.btf_id = type;
 	} else if (!btf_type_is_struct(t)) {
Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
Acked-by on v1 3/4
#9 lore raw

Re: [PATCH bpf-next v1 3/4] bpf: Reject programs that try to load __percpu memory.

Quoted 21 lines · Hao Luo

With the introduction of the btf_type_tag "percpu", we can add a MEM_PERCPU to identify those pointers that point to percpu memory. The ability of differetiating percpu pointers from regular memory pointers have two benefits:

1. It forbids unexpected use of percpu pointers, such as direct loads. In kernel, there are special functions used for accessing percpu memory. Directly loading percpu memory is meaningless. We already have BPF helpers like bpf_per_cpu_ptr() and bpf_this_cpu_ptr() that wrap the kernel percpu functions. So we can now convert percpu pointers into regular pointers in a safe way.

2. Previously, bpf_per_cpu_ptr() and bpf_this_cpu_ptr() only work on PTR_TO_PERCPU_BTF_ID, a special reg_type which describes static percpu variables in kernel (we rely on pahole to encode them into vmlinux BTF). Now, since we can identify __percpu tagged pointers, we can also identify dynamically allocated percpu memory as well. It means we can use bpf_xxx_cpu_ptr() on dynamic percpu memory. This would be very convenient when accessing fields like "cgroup->rstat_cpu".

Quoted 2 lines
Cc Yonghong Song <[email protected]>
author
#10 lore raw

PATCH 4/4 selftests/bpf: Add a test for btf_type_tag "percpu"

In reply to Hao Luo

Add test for percpu btf_type_tag. Similar to the "user" tag, we test the following cases:

1. __percpu struct field.
2. __percpu as function parameter.
3. per_cpu_ptr() accepts dynamically allocated __percpu memory.

Because the test for "user" and the test for "percpu" are very similar, a little bit of refactoring has been done in btf_tag.c. Basically, both tests share the same function for loading vmlinux and module btf.

Example output from log:

Quoted 1 line

./test_progs -v -t btf_tag

libbpf: prog 'test_percpu1': BPF program load failed: Permission denied
libbpf: prog 'test_percpu1': -- BEGIN PROG LOAD LOG --
...
; g = arg->a;
1: (61) r1 = *(u32 *)(r1 +0)
R1 is ptr_bpf_testmod_btf_type_tag_1 access percpu memory: off=0
...
test_btf_type_tag_mod_percpu:PASS:btf_type_tag_percpu 0 nsec #26/6 btf_tag/btf_type_tag_percpu_mod1:OK

libbpf: prog 'test_percpu2': BPF program load failed: Permission denied
libbpf: prog 'test_percpu2': -- BEGIN PROG LOAD LOG --
...
; g = arg->p->a;
2: (61) r1 = *(u32 *)(r1 +0)
R1 is ptr_bpf_testmod_btf_type_tag_1 access percpu memory: off=0
...
test_btf_type_tag_mod_percpu:PASS:btf_type_tag_percpu 0 nsec #26/7 btf_tag/btf_type_tag_percpu_mod2:OK

libbpf: prog 'test_percpu_load': BPF program load failed: Permission denied
libbpf: prog 'test_percpu_load': -- BEGIN PROG LOAD LOG --
...
; g = (__u64)cgrp->rstat_cpu->updated_children;
2: (79) r1 = *(u64 *)(r1 +48)
R1 is ptr_cgroup_rstat_cpu access percpu memory: off=48
...
test_btf_type_tag_vmlinux_percpu:PASS:btf_type_tag_percpu_load 0 nsec #26/8 btf_tag/btf_type_tag_percpu_vmlinux_load:OK

load_btfs:PASS:could not load vmlinux BTF 0 nsec test_btf_type_tag_vmlinux_percpu:PASS:btf_type_tag_percpu 0 nsec test_btf_type_tag_vmlinux_percpu:PASS:btf_type_tag_percpu_helper 0 nsec #26/9 btf_tag/btf_type_tag_percpu_vmlinux_helper:OK

---
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  17 ++
 .../selftests/bpf/prog_tests/btf_tag.c        | 164 ++++++++++++++----
 .../selftests/bpf/progs/btf_type_tag_percpu.c |  66 +++++++
 3 files changed, 218 insertions(+), 29 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/btf_type_tag_percpu.c
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +17 -0
index 27d63be47b95..17c211f3b924 100644
@@ -33,6 +33,10 @@ struct bpf_testmod_btf_type_tag_2 {
 	struct bpf_testmod_btf_type_tag_1 __user *p;
 };
 
+struct bpf_testmod_btf_type_tag_3 {
+	struct bpf_testmod_btf_type_tag_1 __percpu *p;
+};
+
 noinline int
 bpf_testmod_test_btf_type_tag_user_1(struct bpf_testmod_btf_type_tag_1 __user *arg) {
 	BTF_TYPE_EMIT(func_proto_typedef);
@@ -46,6 +50,19 @@ bpf_testmod_test_btf_type_tag_user_2(struct bpf_testmod_btf_type_tag_2 *arg) {
 	return arg->p->a;
 }
 
+noinline int
+bpf_testmod_test_btf_type_tag_percpu_1(struct bpf_testmod_btf_type_tag_1 __percpu *arg) {
+	BTF_TYPE_EMIT(func_proto_typedef);
+	BTF_TYPE_EMIT(func_proto_typedef_nested1);
+	BTF_TYPE_EMIT(func_proto_typedef_nested2);
+	return arg->a;
+}
+
+noinline int
+bpf_testmod_test_btf_type_tag_percpu_2(struct bpf_testmod_btf_type_tag_3 *arg) {
+	return arg->p->a;
+}
+
 noinline int bpf_testmod_loop_test(int n)
 {
 	int i, sum = 0;
tools/testing/selftests/bpf/prog_tests/btf_tag.c +135 -29

230 lines not on this page · raw message

tools/testing/selftests/bpf/progs/btf_type_tag_percpu.c new file +66 -0

66 lines not on this page · raw message

Signature · 1 line
2.35.1.616.g0bdcbb4464-goog
Acked-by on v1 4/4
#11 lore raw

Re: [PATCH bpf-next v1 4/4] selftests/bpf: Add a test for btf_type_tag "percpu"

Quoted 45 lines · Hao Luo

Add test for percpu btf_type_tag. Similar to the "user" tag, we test the following cases:

1. __percpu struct field.
2. __percpu as function parameter.
3. per_cpu_ptr() accepts dynamically allocated __percpu memory.

Because the test for "user" and the test for "percpu" are very similar, a little bit of refactoring has been done in btf_tag.c. Basically, both tests share the same function for loading vmlinux and module btf.

Example output from log:

Quoted 1 line

./test_progs -v -t btf_tag

libbpf: prog 'test_percpu1': BPF program load failed: Permission denied
libbpf: prog 'test_percpu1': -- BEGIN PROG LOAD LOG --
...
; g = arg->a;
1: (61) r1 = *(u32 *)(r1 +0)
R1 is ptr_bpf_testmod_btf_type_tag_1 access percpu memory: off=0
...
test_btf_type_tag_mod_percpu:PASS:btf_type_tag_percpu 0 nsec #26/6 btf_tag/btf_type_tag_percpu_mod1:OK

libbpf: prog 'test_percpu2': BPF program load failed: Permission denied
libbpf: prog 'test_percpu2': -- BEGIN PROG LOAD LOG --
...
; g = arg->p->a;
2: (61) r1 = *(u32 *)(r1 +0)
R1 is ptr_bpf_testmod_btf_type_tag_1 access percpu memory: off=0
...
test_btf_type_tag_mod_percpu:PASS:btf_type_tag_percpu 0 nsec #26/7 btf_tag/btf_type_tag_percpu_mod2:OK

libbpf: prog 'test_percpu_load': BPF program load failed: Permission denied
libbpf: prog 'test_percpu_load': -- BEGIN PROG LOAD LOG --
...
; g = (__u64)cgrp->rstat_cpu->updated_children;
2: (79) r1 = *(u64 *)(r1 +48)
R1 is ptr_cgroup_rstat_cpu access percpu memory: off=48
...
test_btf_type_tag_vmlinux_percpu:PASS:btf_type_tag_percpu_load 0 nsec #26/8 btf_tag/btf_type_tag_percpu_vmlinux_load:OK

Quoted 6 lines

load_btfs:PASS:could not load vmlinux BTF 0 nsec test_btf_type_tag_vmlinux_percpu:PASS:btf_type_tag_percpu 0 nsec test_btf_type_tag_vmlinux_percpu:PASS:btf_type_tag_percpu_helper 0 nsec #26/9 btf_tag/btf_type_tag_percpu_vmlinux_helper:OK

With one nit below.

Quoted 23 lines
---
  .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  17 ++
  .../selftests/bpf/prog_tests/btf_tag.c        | 164 ++++++++++++++----
  .../selftests/bpf/progs/btf_type_tag_percpu.c |  66 +++++++
  3 files changed, 218 insertions(+), 29 deletions(-)

create mode 100644 tools/testing/selftests/bpf/progs/btf_type_tag_percpu.c

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +4 -0
index 27d63be47b95..17c211f3b924 100644
@@ -33,6 +33,10 @@ struct bpf_testmod_btf_type_tag_2 {
  	struct bpf_testmod_btf_type_tag_1 __user *p;
  };
 
+struct bpf_testmod_btf_type_tag_3 {
+	struct bpf_testmod_btf_type_tag_1 __percpu *p;
+};
+
  noinline int
  bpf_testmod_test_btf_type_tag_user_1(struct bpf_testmod_btf_type_tag_1 __user *arg) {
  	BTF_TYPE_EMIT(func_proto_typedef);

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c bpf_testmod_test_btf_type_tag_user_2(struct bpf_testmod_btf_type_tag_2 *arg) { · line 57

Quoted 8 lines
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +5 -0
@@ -46,6 +50,19 @@ bpf_testmod_test_btf_type_tag_user_2(struct bpf_testmod_btf_type_tag_2 *arg) {
  	return arg->p->a;
  }
 
+noinline int
+bpf_testmod_test_btf_type_tag_percpu_1(struct bpf_testmod_btf_type_tag_1 __percpu *arg) {
+	BTF_TYPE_EMIT(func_proto_typedef);
+	BTF_TYPE_EMIT(func_proto_typedef_nested1);
+	BTF_TYPE_EMIT(func_proto_typedef_nested2);

Are these necessary? They have been defined in
bpf_testmod_test_btf_type_tag_user_1().

Quoted 3 lines
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +3 -0
+	return arg->a;
+}
+

[...]

#12 lore raw

Re: [PATCH bpf-next v1 4/4] selftests/bpf: Add a test for btf_type_tag "percpu"

Quoted 82 lines · Yonghong Song
Quoted 45 lines · Hao Luo

Add test for percpu btf_type_tag. Similar to the "user" tag, we test the following cases:

1. __percpu struct field.
2. __percpu as function parameter.
3. per_cpu_ptr() accepts dynamically allocated __percpu memory.

Because the test for "user" and the test for "percpu" are very similar, a little bit of refactoring has been done in btf_tag.c. Basically, both tests share the same function for loading vmlinux and module btf.

Example output from log:

Quoted 1 line

./test_progs -v -t btf_tag

libbpf: prog 'test_percpu1': BPF program load failed: Permission denied
libbpf: prog 'test_percpu1': -- BEGIN PROG LOAD LOG --
...
; g = arg->a;
1: (61) r1 = *(u32 *)(r1 +0)
R1 is ptr_bpf_testmod_btf_type_tag_1 access percpu memory: off=0
...
test_btf_type_tag_mod_percpu:PASS:btf_type_tag_percpu 0 nsec #26/6 btf_tag/btf_type_tag_percpu_mod1:OK

libbpf: prog 'test_percpu2': BPF program load failed: Permission denied
libbpf: prog 'test_percpu2': -- BEGIN PROG LOAD LOG --
...
; g = arg->p->a;
2: (61) r1 = *(u32 *)(r1 +0)
R1 is ptr_bpf_testmod_btf_type_tag_1 access percpu memory: off=0
...
test_btf_type_tag_mod_percpu:PASS:btf_type_tag_percpu 0 nsec #26/7 btf_tag/btf_type_tag_percpu_mod2:OK

libbpf: prog 'test_percpu_load': BPF program load failed: Permission denied
libbpf: prog 'test_percpu_load': -- BEGIN PROG LOAD LOG --
...
; g = (__u64)cgrp->rstat_cpu->updated_children;
2: (79) r1 = *(u64 *)(r1 +48)
R1 is ptr_cgroup_rstat_cpu access percpu memory: off=48
...
test_btf_type_tag_vmlinux_percpu:PASS:btf_type_tag_percpu_load 0 nsec #26/8 btf_tag/btf_type_tag_percpu_vmlinux_load:OK

Quoted 6 lines

load_btfs:PASS:could not load vmlinux BTF 0 nsec test_btf_type_tag_vmlinux_percpu:PASS:btf_type_tag_percpu 0 nsec test_btf_type_tag_vmlinux_percpu:PASS:btf_type_tag_percpu_helper 0 nsec #26/9 btf_tag/btf_type_tag_percpu_vmlinux_helper:OK

With one nit below.

Quoted 24 lines
---
  .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  17 ++
  .../selftests/bpf/prog_tests/btf_tag.c        | 164 ++++++++++++++----
  .../selftests/bpf/progs/btf_type_tag_percpu.c |  66 +++++++
  3 files changed, 218 insertions(+), 29 deletions(-)

create mode 100644 tools/testing/selftests/bpf/progs/btf_type_tag_percpu.c

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +4 -0
index 27d63be47b95..17c211f3b924 100644
@@ -33,6 +33,10 @@ struct bpf_testmod_btf_type_tag_2 {
  	struct bpf_testmod_btf_type_tag_1 __user *p;
  };
+struct bpf_testmod_btf_type_tag_3 {
+	struct bpf_testmod_btf_type_tag_1 __percpu *p;
+};
+
  noinline int
  bpf_testmod_test_btf_type_tag_user_1(struct bpf_testmod_btf_type_tag_1 __user *arg) {
  	BTF_TYPE_EMIT(func_proto_typedef);
@@ -46,6 +50,19 @@ bpf_testmod_test_btf_type_tag_user_2(struct bpf_testmod_btf_type_tag_2 *arg) {
  	return arg->p->a;
  }
Quoted 8 lines

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c bpf_testmod_test_btf_type_tag_user_2(struct bpf_testmod_btf_type_tag_2 *arg) { · line 57

Quoted 5 lines
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +5 -0
+noinline int
+bpf_testmod_test_btf_type_tag_percpu_1(struct bpf_testmod_btf_type_tag_1 __percpu *arg) {
+	BTF_TYPE_EMIT(func_proto_typedef);
+	BTF_TYPE_EMIT(func_proto_typedef_nested1);
+	BTF_TYPE_EMIT(func_proto_typedef_nested2);

Are these necessary? They have been defined in
bpf_testmod_test_btf_type_tag_user_1().

Yonghong,
Thanks. Good catch. I've removed those while applying.

Hao,
Great work.
I really liked how patch 3 discovers MEM_PERCPU flag from two sources: percpu_datasec and clang tag.

author
#13 lore raw

Re: [PATCH bpf-next v1 4/4] selftests/bpf: Add a test for btf_type_tag "percpu"

Quoted 4 lines · Alexei Starovoitov
Quoted 2 lines · Yonghong Song

On 3/4/22 11:16 AM, Hao Luo wrote:

[...]

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
---
  .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  17 ++
  .../selftests/bpf/prog_tests/btf_tag.c        | 164 ++++++++++++++----
  .../selftests/bpf/progs/btf_type_tag_percpu.c |  66 +++++++
  3 files changed, 218 insertions(+), 29 deletions(-)

create mode 100644 tools/testing/selftests/bpf/progs/btf_type_tag_percpu.c

[...]

Quoted 9 lines
Quoted 8 lines

tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c bpf_testmod_test_btf_type_tag_user_2(struct bpf_testmod_btf_type_tag_2 *arg) { · line 57

Quoted 5 lines
tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +5 -0
+noinline int
+bpf_testmod_test_btf_type_tag_percpu_1(struct bpf_testmod_btf_type_tag_1 __percpu *arg) {
+   BTF_TYPE_EMIT(func_proto_typedef);
+   BTF_TYPE_EMIT(func_proto_typedef_nested1);
+   BTF_TYPE_EMIT(func_proto_typedef_nested2);

Are these necessary? They have been defined in
bpf_testmod_test_btf_type_tag_user_1().

Quoted 3 lines

Yonghong,
Thanks. Good catch. I've removed those while applying.

Thanks Yonghong and Alexei. I wasn't sure their purpose and not sure whether I should include them, so copy-pasted it.

Quoted 4 lines

Hao,
Great work.
I really liked how patch 3 discovers MEM_PERCPU flag from two sources: percpu_datasec and clang tag.

Thanks Alexei! The BTF infrastructure together with clang tag is really amazing! [thumbs up]

#14 lore raw
In reply to Hao Luo
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:

Quoted 9 lines · On Fri, 4 Mar 2022 11:16:53 -0800 you wrote:

This patchset is very much similar to Yonghong's patchset on adding __user tagging [1], where a "user" btf_type_tag was introduced to describe __user memory pointers. Similar approach can be applied on __percpu pointers. The __percpu attribute in kernel is used to identify pointers that point to memory allocated in percpu region. Normally, accessing __percpu memory requires using special functions like per_cpu_ptr() etc. Directly accessing __percpu pointer is meaningless.

[...]

Here is the summary with links:
- [bpf-next,v1,1/4] bpf: Fix checking PTR_TO_BTF_ID in check_mem_access

    https://git.kernel.org/bpf/bpf-next/c/bff61f6faedb
  - [bpf-next,v1,2/4] compiler_types: define __percpu as __attribute__((btf_type_tag("percpu")))
    https://git.kernel.org/bpf/bpf-next/c/9216c9162378
  - [bpf-next,v1,3/4] bpf: Reject programs that try to load __percpu memory.
    https://git.kernel.org/bpf/bpf-next/c/5844101a1be9
  - [bpf-next,v1,4/4] selftests/bpf: Add a test for btf_type_tag "percpu"
    https://git.kernel.org/bpf/bpf-next/c/50c6b8a9aea2

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html