bpf: Fix errors in verifying a union
Subsystems (4)
v2 cover bpf: Fix errors in verifying a union
From: root <[email protected]>
Patch #1: Fix an issue found in code review Patch #2: Selftest for #1
Patch #3: Fix an issue found in our dev server Patch #4: Selftest for #3
Changes:
- bpf: Introduce union trusted
https://lore.kernel.org/bpf/[email protected]/
- bpf: Fix errors in verifying a union
https://lore.kernel.org/bpf/[email protected]/
Yafang Shao (4):
bpf: Fix an error around PTR_UNTRUSTED selftests/bpf: Add selftests for nested_trust
bpf: Fix an error in verifying a field in a union selftests/bpf: Add selftest for PTR_UNTRUSTED
kernel/bpf/btf.c | 22 ++++++------ kernel/bpf/verifier.c | 5 +++ .../selftests/bpf/prog_tests/ptr_untrusted.c | 36 +++++++++++++++++++ .../bpf/progs/nested_trust_failure.c | 16 +++++++++ .../bpf/progs/nested_trust_success.c | 15 ++++++++ .../selftests/bpf/progs/test_ptr_untrusted.c | 29 +++++++++++++++ 6 files changed, 111 insertions(+), 12 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/ptr_untrusted.c create mode 100644 tools/testing/selftests/bpf/progs/test_ptr_untrusted.c
Signature · 1 line
2.39.3
PATCH v2 1/4 bpf: Fix an error around PTR_UNTRUSTED
Per discussion with Alexei, the PTR_UNTRUSTED flag should not been cleared when we start to walk a new struct, because the struct in question may be a struct nested in a union. We should also check and set this flag before we walk its each member, in case itself is a union. We will clear this flag if the field is BTF_TYPE_SAFE_RCU_OR_NULL.
--- kernel/bpf/btf.c | 20 +++++++++----------- kernel/bpf/verifier.c | 5 +++++ 2 files changed, 14 insertions(+), 11 deletions(-)
kernel/bpf/btf.c +9 -11
@@ -6133,7 +6133,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,const char *tname, *mname, *tag_value;u32 vlen, elem_id, mid; - *flag =0 ; again:if (btf_type_is_modifier (t)) t =btf_type_skip_modifiers (btf, t->type,NULL );
@@ -6144,6 +6143,14 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf, } vlen =btf_type_vlen (t); +if (BTF_INFO_KIND (t->info) ==BTF_KIND_UNION && vlen !=1 && !(*flag &PTR_UNTRUSTED )) +/* +* walking unions yields untrusted pointers +* with exception of __bpf_md_ptr and other +* unions with a single member +*/ + *flag |=PTR_UNTRUSTED ; +if (off + size > t->size) {/* If the last element is a variable size array, we may * need to relax the rule.
@@ -6304,15 +6311,6 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,* of this field or inside of this struct */ if (btf_type_is_struct (mtype)) { -if (BTF_INFO_KIND (mtype->info) ==BTF_KIND_UNION && -btf_type_vlen (mtype) !=1 ) -/* -* walking unions yields untrusted pointers -* with exception of __bpf_md_ptr and other -* unions with a single member -*/ - *flag |=PTR_UNTRUSTED ; -/* our field must be inside that union or struct */ t = mtype;
@@ -6478,7 +6476,7 @@ bool btf_struct_ids_match(struct bpf_verifier_log *log, bool strict) {const struct btf_type *type; -enum bpf_type_flag flag; +enum bpf_type_flag flag =0 ;int err;/* Are we already done? */
kernel/bpf/verifier.c +5 -0
@@ -6067,6 +6067,11 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env, type_is_rcu_or_null(env, reg, field_name, btf_id)) {/* __rcu tagged pointers can be NULL */ flag |=MEM_RCU |PTR_MAYBE_NULL ; + +/* We always trust them */ +if (type_is_rcu_or_null (env, reg, field_name, btf_id) && + flag &PTR_UNTRUSTED ) + flag &= ~PTR_UNTRUSTED ; } else if (flag & (MEM_PERCPU | MEM_USER)) {/* keep as-is */ } else {
Signature · 1 line
2.39.3
PATCH v2 2/4 selftests/bpf: Add selftests for nested_trust
In reply to Yafang ShaoAdd selftests for nested_strust to check whehter PTR_UNTRUSTED is cleared as expected, the result as follows:
#141/1 nested_trust/test_read_cpumask:OK #141/2 nested_trust/test_skb_field:OK <<<< #141/3 nested_trust/test_invalid_nested_user_cpus:OK #141/4 nested_trust/test_invalid_nested_offset:OK #141/5 nested_trust/test_invalid_skb_field:OK <<<< #141 nested_trust:OK
The #141/2 and #141/5 are newly added.
--- .../selftests/bpf/progs/nested_trust_failure.c | 16 ++++++++++++++++ .../selftests/bpf/progs/nested_trust_success.c | 15 +++++++++++++++ 2 files changed, 31 insertions(+)
tools/testing/selftests/bpf/progs/nested_trust_failure.c +16 -0
@@ -10,6 +10,13 @@ char _license[] SEC("license") = "GPL"; +struct { + __uint(type, BPF_MAP_TYPE_SK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, u64); +} sk_storage_map SEC(".maps"); +/* Prototype for all of the program trace events below: * * TRACE_EVENT(task_newtask,
@@ -31,3 +38,12 @@ int BPF_PROG(test_invalid_nested_offset, struct task_struct *task, u64 clone_flabpf_cpumask_first_zero (&task->cpus_mask);return 0 ; } + +/* Although R2 is of type sk_buff but sock_common is expected, we will hit untrusted ptr first. */ +SEC("tp_btf/tcp_probe") +__failure __msg("R2 type=untrusted_ptr_ expected=ptr_, trusted_ptr_, rcu_ptr_") +int BPF_PROG(test_invalid_skb_field, struct sock *sk, struct sk_buff *skb) +{ +bpf_sk_storage_get (&sk_storage_map, skb->next,0 ,0 ); +return 0 ; +}
tools/testing/selftests/bpf/progs/nested_trust_success.c +15 -0
@@ -10,6 +10,13 @@ char _license[] SEC("license") = "GPL"; +struct { + __uint(type, BPF_MAP_TYPE_SK_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, u64); +} sk_storage_map SEC(".maps"); + SEC("tp_btf/task_newtask") __successint BPF_PROG (test_read_cpumask ,struct task_struct *task,u64 clone_flags)
@@ -17,3 +24,11 @@ int BPF_PROG(test_read_cpumask, struct task_struct *task, u64 clone_flags)bpf_cpumask_test_cpu (0 , task->cpus_ptr);return 0 ; } + +SEC("tp_btf/tcp_probe") +__success +int BPF_PROG (test_skb_field ,struct sock *sk,struct sk_buff *skb) +{ +bpf_sk_storage_get (&sk_storage_map, skb->sk,0 ,0 ); +return 0 ; +}
Signature · 1 line
2.39.3
PATCH v2 3/4 bpf: Fix an error in verifying a field in a union
In reply to Yafang ShaoWe are utilizing BPF LSM to monitor BPF operations within our container environment. When we add support for raw_tracepoint, it hits below error.
; (const void *)attr->raw_tracepoint.name);
27: (79) r3 = *(u64 *)(r2 +0)
access beyond the end of member map_type (mend:4) in struct (anon) with off 0 size 8
It can be reproduced with below BPF prog.
SEC("lsm/bpf")
int BPF_PROG(bpf_audit, int cmd, union bpf_attr *attr, unsigned int size)
{
switch (cmd) {
case BPF_RAW_TRACEPOINT_OPEN:
bpf_printk("raw_tracepoint is %s", attr->raw_tracepoint.name);
break;
default:
break;
}
return 0;
}The reason is that when accessing a field in a union, such as bpf_attr, if the field is located within a nested struct that is not the first member of the union, it can result in incorrect field verification.
union bpf_attr {
struct {
__u32 map_type; <<<< Actually it will find that field.
__u32 key_size;
__u32 value_size;
...
};
...
struct {
__u64 name; <<<< We want to verify this field.
__u32 prog_fd;
} raw_tracepoint;
};Considering the potential deep nesting levels, finding a perfect solution to address this issue has proven challenging. Therefore, I propose a solution where we simply skip the verification process if the field in question is located within a union.
--- kernel/bpf/btf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/bpf/btf.c +1 -1
@@ -6368,7 +6368,7 @@ static int btf_struct_walk(struct bpf_verifier_log *log, const struct btf *btf,* that also allows using an array of int as a scratch * space. e.g. skb->cb[]. */ -if (off + size > mtrue_end) { +if (off + size > mtrue_end && !(*flag &PTR_UNTRUSTED )) {bpf_log (log,"access beyond the end of member %s (mend:%u) in struct %s with off %u size %u\n" , mname, mtrue_end, tname, off, size);
Signature · 1 line
2.39.3
PATCH v2 4/4 selftests/bpf: Add selftest for PTR_UNTRUSTED
In reply to Yafang ShaoAdd a new selftest to check the PTR_UNTRUSTED condition. Below is the result,
#160 ptr_untrusted:OK
--- .../selftests/bpf/prog_tests/ptr_untrusted.c | 36 +++++++++++++++++++ .../selftests/bpf/progs/test_ptr_untrusted.c | 29 +++++++++++++++ 2 files changed, 65 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/ptr_untrusted.c create mode 100644 tools/testing/selftests/bpf/progs/test_ptr_untrusted.c
tools/testing/selftests/bpf/prog_tests/ptr_untrusted.c new file +36 -0
@@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Yafang Shao <[email protected]> */ + +#include <string.h> +#include <linux/bpf.h> +#include <test_progs.h> +#include "test_ptr_untrusted.skel.h" + +#define TP_NAME "sched_switch" + +void serial_test_ptr_untrusted (void ) +{ +struct test_ptr_untrusted *skel; +int err; + + skel =test_ptr_untrusted__open_and_load (); +if (!ASSERT_OK_PTR (skel,"skel_open" )) + goto cleanup; + +/* First, attach lsm prog */ + skel->links.lsm_run =bpf_program__attach_lsm (skel->progs.lsm_run); +if (!ASSERT_OK_PTR (skel->links.lsm_run,"lsm_attach" )) + goto cleanup; + +/* Second, attach raw_tp prog. The lsm prog will be triggered. */ + skel->links.raw_tp_run =bpf_program__attach_raw_tracepoint (skel->progs.raw_tp_run, +TP_NAME ); +if (!ASSERT_OK_PTR (skel->links.raw_tp_run,"raw_tp_attach" )) + goto cleanup; + + err =strncmp (skel->bss->tp_name,TP_NAME ,strlen (TP_NAME )); +ASSERT_EQ (err,0 ,"cmp_tp_name" ); + +cleanup: +test_ptr_untrusted__destroy (skel); +}
tools/testing/selftests/bpf/progs/test_ptr_untrusted.c new file +29 -0
@@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (C) 2023 Yafang Shao <[email protected]> */ + +#include "vmlinux.h" +#include <bpf/bpf_tracing.h> + +char tp_name[128 ]; + +SEC("lsm/bpf") +int BPF_PROG (lsm_run ,int cmd,union bpf_attr *attr,unsigned int size) +{ +switch (cmd) { +case BPF_RAW_TRACEPOINT_OPEN : +bpf_probe_read_user_str (tp_name,sizeof (tp_name) -1 , + (void *)attr->raw_tracepoint.name); +break ; +default : +break ; + } +return 0 ; +} + +SEC("raw_tracepoint") +int BPF_PROG (raw_tp_run ) +{ +return 0 ; +} + +char _license[] SEC("license") = "GPL";
Signature · 1 line
2.39.3
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 8 lines · On Thu, 13 Jul 2023 02:56:38 +0000 you wrote:
From: root <[email protected]>
Patch #1: Fix an issue found in code review Patch #2: Selftest for #1
Patch #3: Fix an issue found in our dev server Patch #4: Selftest for #3[...]
Here is the summary with links:
- [v2,bpf-next,1/4] bpf: Fix an error around PTR_UNTRUSTED
https://git.kernel.org/bpf/bpf-next/c/7ce4dc3e4a9d - [v2,bpf-next,2/4] selftests/bpf: Add selftests for nested_trust https://git.kernel.org/bpf/bpf-next/c/d2284d68259c - [v2,bpf-next,3/4] bpf: Fix an error in verifying a field in a union https://git.kernel.org/bpf/bpf-next/c/33937607efa0 - [v2,bpf-next,4/4] selftests/bpf: Add selftest for PTR_UNTRUSTED https://git.kernel.org/bpf/bpf-next/c/1cd0e7715cad
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html