Allocated objects, BPF linked lists
Subsystems (5)
Kumar Kartikeya Dwivedi
maintainerv10 cover Allocated objects, BPF linked lists
This series introduces user defined BPF objects of a type in program BTF. This allows BPF programs to allocate their own objects, build their own object hierarchies, and use the basic building blocks provided by BPF runtime to build their own data structures flexibly.
Then, we introduce the support for single ownership BPF linked lists, which can be put inside BPF maps, or allocated objects, and hold such allocated objects as elements. It works as an instrusive collection, which is done to allow making allocated objects part of multiple data structures at the same time in the future.
The eventual goal of this and future patches is to allow one to do some limited form of kernel style programming in BPF C, and allow programmers to build their own complex data structures flexibly out of basic building blocks.
The key difference will be that such programs are verified to be safe, preserve runtime integrity of the system, and are proven to be bug free as far as the invariants of BPF specific APIs are concerned.
One immediate use case that will be using the entire infrastructure this series is introducing will be managing percpu NMI safe linked lists inside BPF programs.
The other use case this will serve in the near future will be linking kernel structures like XDP frame and sk_buff directly into user data structures (rbtree, pifomap, etc.) for packet queueing. This will follow single ownership concept included in this series.
The user has complete control of the internal locking, and hence also the batching of operations for each critical section.
The features are:
- Allocated objects.
- bpf_obj_new, bpf_obj_drop to allocate and free them.
- Single ownership BPF linked lists.
- Support for them in BPF maps.
- Support for them in allocated objects.
- Global spin locks.
- Spin locks inside allocated objects.
Some other notable things:
- Completely static verification of locking.
- Kfunc argument handling has been completely reworked.
- Argument rewriting support for kfuncs.
- A new bpf_experimental.h header as a dumping ground for these APIs.
Any functionality exposed in this series is NOT part of UAPI. It is only available through use of kfuncs, and structs that can be added to map value may also change their size or name in the future. Hence, every feature in this series must be considered experimental.
Follow-ups:
-----------
* Support for kptrs (local and kernel) in local storage and percpu maps + kptr tests
* Fixes for helper access checks rebasing on top of this series
Next steps:
-----------
* NMI safe percpu single ownership linked lists (using local_t protection).
* Lockless linked lists.
* Allow RCU protected BPF allocated objects. This then allows RCU protected list lookups, since spinlock protection for readers does not scale.
* Introduce bpf_refcount for local kptrs, shared ownership.
* Introduce shared ownership linked lists.
* Documentation.
Changelog:
----------
v9 -> v10
v9: https://lore.kernel.org/bpf/[email protected]
* Deduplicate code to find btf_record of reg (Alexei)
* Add linked_list test to DENYLIST.aarch64 (Alexei)
* Disable some linked list tests for now so that they compile with clang nightly (Alexei)
v8 -> v9
v8: https://lore.kernel.org/bpf/[email protected]
* Fix up commit log of patch 2, Simplify patch 3
* Explain the implicit requirement of bpf_list_head requiring map BTF to match in btf_record_equal in a separate patch.
v7 -> v8
v7: https://lore.kernel.org/bpf/[email protected]
* Fix early return in map_check_btf (Dan Carpenter)
* Fix two memory leak bugs in local storage maps, outer maps
* Address comments from Alexei and Dave
* More local kptr -> allocated object renaming
* Use krealloc with NULL instead kmalloc + krealloc
* Drop WARN_ON_ONCE for field_offs parsing
* Combine kfunc add + remove patches into one
* Drop STRONG suffix from KF_ARG_PTR_TO_KPTR
* Rename is_kfunc_arg_ret_buf_size to is_kfunc_arg_scalar_with_name
* Remove redundant check for reg->type and arg type in it
* Drop void * ret type check
* Remove code duplication in checks for NULL pointer with offset != 0
* Fix two bpf_list_node typos
* Improve log message for bpf_list_head operations
* Improve comments for active_lock struct
* Improve comments for Implementation details of process_spin_lock
* Add Dave's acks
v6 -> v7
v6: https://lore.kernel.org/bpf/[email protected]
* Fix uninitialized variable warning (Dan Carpenter, Kernel Test Robot)
* One more local_kptr renaming
v5 -> v6
v5: https://lore.kernel.org/bpf/[email protected]
* Replace (i && !off) check with next_off, include test (Andrii)
* Drop local kptrs naming (Andrii, Alexei)
* Drop reg->precise == EXACT patch (Andrii)
* Add comment about ptr member of struct active_lock (Andrii)
* Use btf__new_empty + btf__add_xxx APIs (Andrii)
* Address other misc nits from Andrii
v4 -> v5
v4: https://lore.kernel.org/bpf/[email protected]
* Add a lot more selftests (failure, success, runtime, BTF)
* Make sure series is bisect friendly
* Move list draining out of spin lock
* This exposed an issue where bpf_mem_free can now be called in map_free path without migrate_disable, also fixed that.
* Rename MEM_ALLOC -> MEM_RINGBUF, MEM_TYPE_LOCAL -> MEM_ALLOC (Alexei)
* Group lock identity into a struct active_lock { ptr, id } (Dave)
* Split set_release_on_unlock logic into separate patch (Alexei)
v3 -> v4
v3: https://lore.kernel.org/bpf/[email protected]
* Fix compiler error for !CONFIG_BPF_SYSCALL (Kernel Test Robot)
* Fix error due to BUILD_BUG_ON on 32-bit platforms (Kernel Test Robot)
v2 -> v3
v2: https://lore.kernel.org/bpf/[email protected]
* Add ack from Dave for patch 5
* Rename btf_type_fields -> btf_record, btf_type_fields_off -> btf_field_offs, rename functions similarly (Alexei)
* Remove 'kind' component from contains declaration tag (Alexei)
* Move bpf_list_head, bpf_list_node definitions to UAPI bpf.h (Alexei)
* Add note in commit log about modifying btf_struct_access API (Dave)
* Downgrade WARN_ON_ONCE to verbose(env, "...") and return -EFAULT (Dave)
* Add type_is_local_kptr wrapper to avoid noisy checks (Dave)
* Remove unused flags parameter from bpf_kptr_new (Alexei)
* Rename bpf_kptr_new -> bpf_obj_new, bpf_kptr_drop -> bpf_obj_drop (Alexei)
* Reword comment in ref_obj_id_set_release_on_unlock (Dave)
* Fix return type of ref_obj_id_set_release_on_unlock (Dave)
* Introduce is_bpf_list_api_kfunc to dedup checks (Dave)
* Disallow BPF_WRITE to untrusted local kptrs
* Add details about soundness of check_reg_allocation_locked logic
* List untrusted local kptrs for PROBE_MEM handling
v1 -> v2
v1: https://lore.kernel.org/bpf/[email protected]
* Rebase on bpf-next to resolve merge conflict in DENYLIST.s390x
* Fix a couple of mental lapses in bpf_list_head_free
RFC v1 -> v1
RFC v1: https://lore.kernel.org/bpf/[email protected]
* Mostly a complete rewrite of BTF parsing, refactor existing code (Kartikeya)
* Rebase kfunc rewrite for bpf-next, add support for more changes
* Cache type metadata in BTF to avoid recomputation inside verifier (Kartikeya)
* Remove __kernel tag, make things similar to map values, reserve bpf_ prefix
* bpf_kptr_new, bpf_kptr_drop
* Rename precision state enum values (Alexei)
* Drop explicit constructor/destructor support (Alexei)
* Rewrite code for constructing/destructing objects and offload to runtime
* Minimize duplication in bpf_map_value_off_desc handling (Alexei)
* Expose global memory allocator (Alexei)
* Address other nits from Alexei
* Split out local kptrs in maps, more kptrs in maps support into a follow up
Links:
------
* Dave's BPF RB-Tree RFC series
v1 (Discussion thread)
https://lore.kernel.org/bpf/[email protected] v2 (With support for static locks) https://lore.kernel.org/bpf/[email protected]
* BPF Linked Lists Discussion
https://lore.kernel.org/bpf/CAP01T74U30+yeBHEgmgzTJ-XYxZ0zj71kqCDJtTH9YQNfTK+Xw@mail.gmail.com
* BPF Memory Allocator from Alexei
https://lore.kernel.org/bpf/[email protected]
* BPF Memory Allocator UAPI Discussion
https://lore.kernel.org/bpf/[email protected]
Kumar Kartikeya Dwivedi (24):
bpf: Fix early return in map_check_btf
bpf: Do btf_record_free outside map_free callback
bpf: Free inner_map_meta when btf_record_dup fails
bpf: Populate field_offs for inner_map_meta
bpf: Introduce allocated objects support
bpf: Recognize lock and list fields in allocated objects
bpf: Verify ownership relationships for user BTF types
bpf: Allow locking bpf_spin_lock in allocated objects
bpf: Allow locking bpf_spin_lock global variables
bpf: Allow locking bpf_spin_lock in inner map values
bpf: Rewrite kfunc argument handling
bpf: Support constant scalar arguments for kfuncs
bpf: Introduce bpf_obj_new
bpf: Introduce bpf_obj_drop
bpf: Permit NULL checking pointer with non-zero fixed offset
bpf: Introduce single ownership BPF linked list API
bpf: Add 'release on unlock' logic for bpf_list_push_{front,back}
bpf: Add comments for map BTF matching requirement for bpf_list_head selftests/bpf: Add __contains macro to bpf_experimental.h selftests/bpf: Update spinlock selftest
selftests/bpf: Add failure test cases for spin lock pairing selftests/bpf: Add BPF linked list API tests selftests/bpf: Add BTF sanity tests
selftests/bpf: Temporarily disable linked list tests
Documentation/bpf/kfuncs.rst | 24 + include/linux/bpf.h | 51 +- include/linux/bpf_verifier.h | 25 +- include/linux/btf.h | 67 +- kernel/bpf/arraymap.c | 1 - kernel/bpf/btf.c | 656 ++++----- kernel/bpf/core.c | 16 + kernel/bpf/hashtab.c | 1 - kernel/bpf/helpers.c | 115 +- kernel/bpf/map_in_map.c | 48 +- kernel/bpf/syscall.c | 42 +- kernel/bpf/verifier.c | 1220 +++++++++++++++-- tools/testing/selftests/bpf/DENYLIST.aarch64 | 1 + tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../testing/selftests/bpf/bpf_experimental.h | 68 + .../bpf/prog_tests/kfunc_dynptr_param.c | 2 +- .../selftests/bpf/prog_tests/linked_list.c | 747 ++++++++++ .../selftests/bpf/prog_tests/spin_lock.c | 136 ++ .../selftests/bpf/prog_tests/spinlock.c | 45 - .../testing/selftests/bpf/progs/linked_list.c | 379 +++++ .../testing/selftests/bpf/progs/linked_list.h | 58 + .../selftests/bpf/progs/linked_list_fail.c | 581 ++++++++ .../selftests/bpf/progs/test_spin_lock.c | 4 +- .../selftests/bpf/progs/test_spin_lock_fail.c | 204 +++ tools/testing/selftests/bpf/verifier/calls.c | 2 +- .../selftests/bpf/verifier/ref_tracking.c | 4 +- 26 files changed, 3931 insertions(+), 567 deletions(-) create mode 100644 tools/testing/selftests/bpf/bpf_experimental.h create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_list.c create mode 100644 tools/testing/selftests/bpf/prog_tests/spin_lock.c delete mode 100644 tools/testing/selftests/bpf/prog_tests/spinlock.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.h create mode 100644 tools/testing/selftests/bpf/progs/linked_list_fail.c create mode 100644 tools/testing/selftests/bpf/progs/test_spin_lock_fail.c
base-commit: 98b2afc8a67f651ed01fc7d5a7e2528e63dd4e08
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 1/24 bpf: Fix early return in map_check_btf
Instead of returning directly with -EOPNOTSUPP for the timer case, we need to free the btf_record before returning to userspace.
--- kernel/bpf/syscall.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/bpf/syscall.c +1 -1
@@ -1010,7 +1010,7 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf,if (map->map_type !=BPF_MAP_TYPE_HASH && map->map_type !=BPF_MAP_TYPE_LRU_HASH && map->map_type !=BPF_MAP_TYPE_ARRAY ) { -return -EOPNOTSUPP ; + ret = -EOPNOTSUPP ; goto free_map_tab; }break ;
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 2/24 bpf: Do btf_record_free outside map_free callback
In reply to Kumar Kartikeya DwivediSince the commit being fixed, we now miss freeing btf_record for local storage maps which will have a btf_record populated in case they have bpf_spin_lock element.
This was missed because I made the choice of offloading the job to free kptr_off_tab (now btf_record) to the map_free callback when adding support for kptrs.
Revisiting the reason for this decision, there is the possibility that the btf_record gets used inside map_free callback (e.g. in case of maps embedding kptrs) to iterate over them and free them, hence doing it before the map_free callback would be leaking special field memory, and do invalid memory access. The btf_record keeps module references which is critical to ensure the dtor call made for referenced kptr is safe to do.
If doing it after map_free callback, the map area is already freed, so we cannot access bpf_map structure anymore.
To fix this and prevent such lapses in future, move bpf_map_free_record out of the map_free callback, and do it after map_free by remembering the btf_record pointer. There is no need to access bpf_map structure in that case, and we can avoid missing this case when support for new map types is added for other special fields.
Since a btf_record and its btf_field_offs are used together, for consistency delay freeing of field_offs as well. While not a problem right now, a lot of code assumes that either both record and field_offs are set or none at once.
Note that in case of map of maps (outer maps), inner_map_meta->record is only used during verification, not to free fields in map value, hence we simply keep the bpf_map_free_record call as is in bpf_map_meta_free and never touch map->inner_map_meta in bpf_map_free_deferred.
Add a comment making note of these details.
--- kernel/bpf/arraymap.c | 1 - kernel/bpf/hashtab.c | 1 - kernel/bpf/syscall.c | 18 ++++++++++++++---- 3 files changed, 14 insertions(+), 6 deletions(-)
kernel/bpf/arraymap.c +0 -1
@@ -430,7 +430,6 @@ static void array_map_free(struct bpf_map *map)for (i =0 ; i < array->map.max_entries; i++)bpf_obj_free_fields (map->record,array_map_elem_ptr (array, i)); } -bpf_map_free_record (map); }if (array->map.map_type ==BPF_MAP_TYPE_PERCPU_ARRAY )
kernel/bpf/hashtab.c +0 -1
@@ -1511,7 +1511,6 @@ static void htab_map_free(struct bpf_map *map)prealloc_destroy (htab); } -bpf_map_free_record (map);free_percpu (htab->extra_elems);bpf_map_area_free (htab->buckets);bpf_mem_alloc_destroy (&htab->pcpu_ma);
kernel/bpf/syscall.c +14 -4
@@ -659,14 +659,24 @@ void bpf_obj_free_fields(const struct btf_record *rec, void *obj)static void bpf_map_free_deferred (struct work_struct *work) { struct bpf_map *map = container_of(work, struct bpf_map, work); +struct btf_field_offs *foffs = map->field_offs; +struct btf_record *rec = map->record;security_bpf_map_free (map); -kfree (map->field_offs);bpf_map_release_memcg (map); -/* implementation dependent freeing, map_free callback also does -* bpf_map_free_record, if needed. -*/ +/* implementation dependent freeing */ map->ops->map_free (map); +/* Delay freeing of field_offs and btf_record for maps, as map_free +* callback usually needs access to them. It is better to do it here +* than require each callback to do the free itself manually. +* +* Note that the btf_record stashed in map->inner_map_meta->record was +* already freed using the map_free callback for map in map case which +* eventually calls bpf_map_free_meta, since inner_map_meta is only a +* template bpf_map struct used during verification. +*/ +kfree (foffs); +btf_record_free (rec); }static void bpf_map_put_uref (struct bpf_map *map)
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 3/24 bpf: Free inner_map_meta when btf_record_dup fails
In reply to Kumar Kartikeya DwivediWhenever btf_record_dup fails, we must free inner_map_meta that was allocated before.
This fixes a memory leak (in case of errors) during inner map creation.
--- kernel/bpf/map_in_map.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/bpf/map_in_map.c +3 -1
@@ -52,12 +52,14 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta->max_entries = inner_map->max_entries; inner_map_meta->record =btf_record_dup (inner_map->record);if (IS_ERR (inner_map_meta->record)) { +struct bpf_map *err_ptr =ERR_CAST (inner_map_meta->record);/* btf_record_dup returns NULL or valid pointer in case of * invalid/empty/valid, but ERR_PTR in case of errors. During * equality NULL or IS_ERR is equivalent. */ +kfree (inner_map_meta);fdput (f); -return ERR_CAST (inner_map_meta->record); +return err_ptr; }if (inner_map->btf) {btf_get (inner_map->btf);
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 4/24 bpf: Populate field_offs for inner_map_meta
In reply to Kumar Kartikeya DwivediFar too much code simply assumes that both btf_record and btf_field_offs are set to valid pointers together, or both are unset. They go together hand in hand as btf_record describes the special fields and btf_field_offs is compact representation for runtime copying/zeroing.
It is very difficult to make this clear in the code when the only exception to this universal invariant is inner_map_meta which is used as reg->map_ptr in the verifier. This is simply a bug waiting to happen, as in verifier context we cannot easily distinguish if PTR_TO_MAP_VALUE is coming from an inner map, and if we ever end up using field_offs for any reason in the future, we will silently ignore the special fields for inner map case (as NULL is not an error but unset field_offs).
Hence, simply copy field_offs from inner map together with btf_record.
While at it, refactor code to unwind properly on errors with gotos.
--- kernel/bpf/map_in_map.c | 44 ++++++++++++++++++++++++++++++----------- 1 file changed, 32 insertions(+), 12 deletions(-)
kernel/bpf/map_in_map.c +32 -12
@@ -12,6 +12,7 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)struct bpf_map *inner_map, *inner_map_meta;u32 inner_map_meta_size;struct fd f; +int ret; f =fdget (inner_map_ufd); inner_map =__bpf_map_get (f);
@@ -20,18 +21,18 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)/* Does not support >1 level map-in-map */ if (inner_map->inner_map_meta) { -fdput (f); -return ERR_PTR (-EINVAL ); + ret = -EINVAL ; + goto put; }if (!inner_map->ops->map_meta_equal) { -fdput (f); -return ERR_PTR (-ENOTSUPP ); + ret = -ENOTSUPP ; + goto put; }if (btf_record_has_field (inner_map->record,BPF_SPIN_LOCK )) { -fdput (f); -return ERR_PTR (-ENOTSUPP ); + ret = -ENOTSUPP ; + goto put; } inner_map_meta_size =sizeof (*inner_map_meta);
@@ -41,8 +42,8 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta =kzalloc (inner_map_meta_size,GFP_USER );if (!inner_map_meta) { -fdput (f); -return ERR_PTR (-ENOMEM ); + ret = -ENOMEM ; + goto put; } inner_map_meta->map_type = inner_map->map_type;
@@ -50,16 +51,27 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) inner_map_meta->value_size = inner_map->value_size; inner_map_meta->map_flags = inner_map->map_flags; inner_map_meta->max_entries = inner_map->max_entries; + inner_map_meta->record =btf_record_dup (inner_map->record);if (IS_ERR (inner_map_meta->record)) { -struct bpf_map *err_ptr =ERR_CAST (inner_map_meta->record);/* btf_record_dup returns NULL or valid pointer in case of * invalid/empty/valid, but ERR_PTR in case of errors. During * equality NULL or IS_ERR is equivalent. */ -kfree (inner_map_meta); -fdput (f); -return err_ptr; + ret =PTR_ERR (inner_map_meta->record); + goto free; + } +if (inner_map_meta->record) { +struct btf_field_offs *field_offs; +/* If btf_record is !IS_ERR_OR_NULL, then field_offs is always +* valid. +*/ + field_offs =kmemdup (inner_map->field_offs,sizeof (*inner_map->field_offs),GFP_KERNEL | __GFP_NOWARN); +if (!field_offs) { + ret = -ENOMEM ; + goto free_rec; + } + inner_map_meta->field_offs = field_offs; }if (inner_map->btf) {btf_get (inner_map->btf);
@@ -76,10 +88,18 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd)fdput (f);return inner_map_meta; +free_rec: +btf_record_free (inner_map_meta->record); +free: +kfree (inner_map_meta); +put: +fdput (f); +return ERR_PTR (ret); }void bpf_map_meta_free (struct bpf_map *map_meta) { +kfree (map_meta->field_offs);bpf_map_free_record (map_meta);btf_put (map_meta->btf);kfree (map_meta);
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 5/24 bpf: Introduce allocated objects support
In reply to Kumar Kartikeya DwivediIntroduce support for representing pointers to objects allocated by the BPF program, i.e. PTR_TO_BTF_ID that point to a type in program BTF. This is indicated by the presence of MEM_ALLOC type flag in reg->type to avoid having to check btf_is_kernel when trying to match argument types in helpers.
Whenever walking such types, any pointers being walked will always yield a SCALAR instead of pointer. In the future we might permit kptr inside such allocated objects (either kernel or program allocated), and it will then form a PTR_TO_BTF_ID of the respective type.
For now, such allocated objects will always be referenced in verifier context, hence ref_obj_id == 0 for them is a bug. It is allowed to write to such objects, as long fields that are special are not touched (support for which will be added in subsequent patches). Note that once such a pointer is marked PTR_UNTRUSTED, it is no longer allowed to write to it.
No PROBE_MEM handling is therefore done for loads into this type unless PTR_UNTRUSTED is part of the register type, since they can never be in an undefined state, and their lifetime will always be valid.
--- include/linux/bpf.h | 11 +++++++++++ kernel/bpf/btf.c | 5 +++++ kernel/bpf/verifier.c | 25 +++++++++++++++++++++++-- 3 files changed, 39 insertions(+), 2 deletions(-)
include/linux/bpf.h +11 -0
@@ -525,6 +525,11 @@ enum bpf_type_flag {/* Size is known at compile time. */ MEM_FIXED_SIZE =BIT (10 +BPF_BASE_TYPE_BITS ), +/* MEM is of an allocated object of type in program BTF. This is used to +* tag PTR_TO_BTF_ID allocated using bpf_obj_new. +*/ +MEM_ALLOC =BIT (11 +BPF_BASE_TYPE_BITS ), + __BPF_TYPE_FLAG_MAX, __BPF_TYPE_LAST_FLAG = __BPF_TYPE_FLAG_MAX - 1, };
@@ -2792,4 +2797,10 @@ struct bpf_key {bool has_ref; };#endif /* CONFIG_KEYS */ + +static inline bool type_is_alloc (u32 type) +{ +return type &MEM_ALLOC ; +} +#endif /* _LINUX_BPF_H */
kernel/bpf/btf.c +5 -0
@@ -6034,6 +6034,11 @@ int btf_struct_access(struct bpf_verifier_log *log,switch (err) {case WALK_PTR : +/* For local types, the destination register cannot +* become a pointer again. +*/ +if (type_is_alloc (reg->type)) +return SCALAR_VALUE ;/* If we found the pointer or scalar on t+off, * we're done. */
kernel/bpf/verifier.c +23 -2
@@ -4687,14 +4687,27 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env,return -EACCES ; } -if (env->ops->btf_struct_access) { +if (env->ops->btf_struct_access && !type_is_alloc (reg->type)) { +if (!btf_is_kernel (reg->btf)) { +verbose (env,"verifier internal error: reg->btf must be kernel btf\n" ); +return -EFAULT ; + } ret = env->ops->btf_struct_access (&env->log, reg, off, size, atype, &btf_id, &flag); }else { -if (atype !=BPF_READ ) { +/* Writes are permitted with default btf_struct_access for +* program allocated objects (which always have ref_obj_id > 0), +* but not for untrusted PTR_TO_BTF_ID | MEM_ALLOC. +*/ +if (atype !=BPF_READ && reg->type != (PTR_TO_BTF_ID |MEM_ALLOC )) {verbose (env,"only read is supported\n" );return -EACCES ; } +if (type_is_alloc (reg->type) && !reg->ref_obj_id) { +verbose (env,"verifier internal error: ref_obj_id for allocated object must be non-zero\n" ); +return -EFAULT ; + } + ret =btf_struct_access (&env->log, reg, off, size, atype, &btf_id, &flag); }
@@ -5973,6 +5986,7 @@ int check_func_arg_reg_off(struct bpf_verifier_env *env,* fixed offset. */ case PTR_TO_BTF_ID : +case PTR_TO_BTF_ID |MEM_ALLOC :/* When referenced PTR_TO_BTF_ID is passed to release function, * it's fixed offset must be 0. In the other cases, fixed offset * can be non-zero.
@@ -13690,6 +13704,13 @@ static int convert_ctx_accesses(struct bpf_verifier_env *env)break ;case PTR_TO_BTF_ID :case PTR_TO_BTF_ID |PTR_UNTRUSTED : +/* PTR_TO_BTF_ID | MEM_ALLOC always has a valid lifetime, unlike +* PTR_TO_BTF_ID, and an active ref_obj_id, but the same cannot +* be said once it is marked PTR_UNTRUSTED, hence we must handle +* any faults for loads into such types. BPF_WRITE is disallowed +* for this case. +*/ +case PTR_TO_BTF_ID |MEM_ALLOC |PTR_UNTRUSTED :if (type ==BPF_READ ) { insn->code =BPF_LDX |BPF_PROBE_MEM |BPF_SIZE ((insn)->code);
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 6/24 bpf: Recognize lock and list fields in allocated objects
In reply to Kumar Kartikeya DwivediAllow specifying bpf_spin_lock, bpf_list_head, bpf_list_node fields in a allocated object.
Also update btf_struct_access to reject direct access to these special fields.
A bpf_list_head allows implementing map-in-map style use cases, where an allocated object with bpf_list_head is linked into a list in a map value. This would require embedding a bpf_list_node, support for which is also included. The bpf_spin_lock is used to protect the bpf_list_head and other data.
While we strictly don't require to hold a bpf_spin_lock while touching the bpf_list_head in such objects, as when have access to it, we have complete ownership of the object, the locking constraint is still kept and may be conditionally lifted in the future.
Note that the specification of such types can be done just like map values, e.g.:
struct bar {
struct bpf_list_node node;
};struct foo {
struct bpf_spin_lock lock;
struct bpf_list_head head __contains(bar, node);
struct bpf_list_node node;
};struct map_value {
struct bpf_spin_lock lock;
struct bpf_list_head head __contains(foo, node);
};To recognize such types in user BTF, we build a btf_struct_metas array of metadata items corresponding to each BTF ID. This is done once during the btf_parse stage to avoid having to do it each time during the verification process's requirement to inspect the metadata.
Moreover, the computed metadata needs to be passed to some helpers in future patches which requires allocating them and storing them in the BTF that is pinned by the program itself, so that valid access can be assumed to such data during program runtime.
A key thing to note is that once a btf_struct_meta is available for a type, both the btf_record and btf_field_offs should be available. It is critical that btf_field_offs is available in case special fields are present, as we extensively rely on special fields being zeroed out in map values and allocated objects in later patches. The code ensures that by bailing out in case of errors and ensuring both are available together. If the record is not available, the special fields won't be recognized, so not having both is also fine (in terms of being a verification error and not a runtime bug).
--- include/linux/bpf.h | 7 ++ include/linux/btf.h | 35 ++++++++ kernel/bpf/btf.c | 197 +++++++++++++++++++++++++++++++++++++++---- kernel/bpf/syscall.c | 4 + 4 files changed, 225 insertions(+), 18 deletions(-)
include/linux/bpf.h +7 -0
@@ -177,6 +177,7 @@ enum btf_field_type {BPF_KPTR_REF = (1 <<3 ),BPF_KPTR =BPF_KPTR_UNREF |BPF_KPTR_REF ,BPF_LIST_HEAD = (1 <<4 ), + BPF_LIST_NODE = (1 << 5), }; struct btf_field_kptr {
@@ -277,6 +278,8 @@ static inline const char *btf_field_type_name(enum btf_field_type type)return "kptr" ;case BPF_LIST_HEAD :return "bpf_list_head" ; +case BPF_LIST_NODE : +return "bpf_list_node" ;default :WARN_ON_ONCE (1 );return "unknown" ;
@@ -295,6 +298,8 @@ static inline u32 btf_field_type_size(enum btf_field_type type)return sizeof (u64);case BPF_LIST_HEAD :return sizeof (struct bpf_list_head ); +case BPF_LIST_NODE : +return sizeof (struct bpf_list_node );default :WARN_ON_ONCE (1 );return 0 ;
@@ -313,6 +318,8 @@ static inline u32 btf_field_type_align(enum btf_field_type type)return __alignof__(u64 );case BPF_LIST_HEAD :return __alignof__(struct bpf_list_head ); +case BPF_LIST_NODE : +return __alignof__(struct bpf_list_node );default :WARN_ON_ONCE (1 );return 0 ;
include/linux/btf.h +35 -0
@@ -6,6 +6,8 @@#include <linux/types.h> #include <linux/bpfptr.h> +#include <linux/bsearch.h> +#include <linux/btf_ids.h> #include <uapi/linux/btf.h> #include <uapi/linux/bpf.h>
@@ -78,6 +80,17 @@ struct btf_id_dtor_kfunc {u32 kfunc_btf_id; }; +struct btf_struct_meta { +u32 btf_id; +struct btf_record *record; +struct btf_field_offs *field_offs; +}; + +struct btf_struct_metas { +u32 cnt; +struct btf_struct_meta types[]; +}; +typedef void (*btf_dtor_kfunc_t )(void *);extern const struct file_operations btf_fops;
@@ -408,6 +421,23 @@ static inline struct btf_param *btf_params(const struct btf_type *t)return (struct btf_param *)(t +1 ); } +static inline int btf_id_cmp_func (const void *a,const void *b) +{ +const int *pa = a, *pb = b; + +return *pa - *pb; +} + +static inline bool btf_id_set_contains (const struct btf_id_set *set,u32 id) +{ +return bsearch (&id, set->ids, set->cnt,sizeof (u32), btf_id_cmp_func) !=NULL ; +} + +static inline void *btf_id_set8_contains (const struct btf_id_set8 *set,u32 id) +{ +return bsearch (&id, set->pairs, set->cnt,sizeof (set->pairs[0 ]), btf_id_cmp_func); +} +#ifdef CONFIG_BPF_SYSCALL struct bpf_prog ;
@@ -423,6 +453,7 @@ int register_btf_kfunc_id_set(enum bpf_prog_type prog_type,s32 btf_find_dtor_kfunc (struct btf *btf,u32 btf_id);int register_btf_id_dtor_kfuncs (const struct btf_id_dtor_kfunc *dtors,u32 add_cnt,struct module *owner); +struct btf_struct_meta *btf_find_struct_meta (const struct btf *btf,u32 btf_id);#else static inline const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id)
@@ -454,6 +485,10 @@ static inline int register_btf_id_dtor_kfuncs(const struct btf_id_dtor_kfunc *dt {return 0 ; } +static inline struct btf_struct_meta *btf_find_struct_meta (const struct btf *btf,u32 btf_id) +{ +return NULL ; +}#endif static inline bool btf_type_is_struct_ptr (struct btf *btf,const struct btf_type *t)
kernel/bpf/btf.c +179 -18
273 lines not on this page · raw message
kernel/bpf/syscall.c +4 -0
@@ -537,6 +537,7 @@ void btf_record_free(struct btf_record *rec)btf_put (rec->fields[i].kptr.btf);break ;case BPF_LIST_HEAD : +case BPF_LIST_NODE :/* Nothing to release for bpf_list_head */ break ;default :
@@ -582,6 +583,7 @@ struct btf_record *btf_record_dup(const struct btf_record *rec) }break ;case BPF_LIST_HEAD : +case BPF_LIST_NODE :/* Nothing to acquire for bpf_list_head */ break ;default :
@@ -648,6 +650,8 @@ void bpf_obj_free_fields(const struct btf_record *rec, void *obj)continue ;bpf_list_head_free (field, field_ptr, obj + rec->spin_lock_off);break ; +case BPF_LIST_NODE : +break ;default :WARN_ON_ONCE (1 );continue ;
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 7/24 bpf: Verify ownership relationships for user BTF types
In reply to Kumar Kartikeya DwivediEnsure that there can be no ownership cycles among different types by way of having owning objects that can hold some other type as their element. For instance, a map value can only hold allocated objects, but these are allowed to have another bpf_list_head. To prevent unbounded recursion while freeing resources, elements of bpf_list_head in local kptrs can never have a bpf_list_head which are part of list in a map value. Later patches will verify this by having dedicated BTF selftests.
Also, to make runtime destruction easier, once btf_struct_metas is fully populated, we can stash the metadata of the value type directly in the metadata of the list_head fields, as that allows easier access to the value type's layout to destruct it at runtime from the btf_field entry of the list head itself.
--- include/linux/bpf.h | 1 + include/linux/btf.h | 1 + kernel/bpf/btf.c | 71 ++++++++++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 4 +++ 4 files changed, 77 insertions(+)
include/linux/bpf.h +1 -0
@@ -191,6 +191,7 @@ struct btf_field_list_head {struct btf *btf;u32 value_btf_id;u32 node_offset; +struct btf_record *value_rec; };struct btf_field {
include/linux/btf.h +1 -0
@@ -178,6 +178,7 @@ int btf_find_spin_lock(const struct btf *btf, const struct btf_type *t);int btf_find_timer (const struct btf *btf,const struct btf_type *t);struct btf_record *btf_parse_fields (const struct btf *btf,const struct btf_type *t,u32 field_mask,u32 value_size); +int btf_check_and_fixup_fields (const struct btf *btf,struct btf_record *rec);struct btf_field_offs *btf_parse_field_offs (struct btf_record *rec);bool btf_type_is_void (const struct btf_type *t);s32 btf_find_by_name_kind (const struct btf *btf,const char *name,u8 kind);
kernel/bpf/btf.c +71 -0
83 lines not on this page · raw message
kernel/bpf/syscall.c +4 -0
@@ -1054,6 +1054,10 @@ static int map_check_btf(struct bpf_map *map, const struct btf *btf, } } + ret =btf_check_and_fixup_fields (btf, map->record); +if (ret <0 ) + goto free_map_tab; +if (map->ops->map_check_btf) { ret = map->ops->map_check_btf (map, btf, key_type, value_type);if (ret <0 )
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 8/24 bpf: Allow locking bpf_spin_lock in allocated objects
In reply to Kumar Kartikeya DwivediAllow locking a bpf_spin_lock in an allocated object, in addition to already supported map value pointers. The handling is similar to that of map values, by just preserving the reg->id of PTR_TO_BTF_ID | MEM_ALLOC as well, and adjusting process_spin_lock to work with them and remember the id in verifier state.
Refactor the existing process_spin_lock to work with PTR_TO_BTF_ID | MEM_ALLOC in addition to PTR_TO_MAP_VALUE. We need to update the reg_may_point_to_spin_lock which is used in mark_ptr_or_null_reg to preserve reg->id, that will be used in env->cur_state->active_spin_lock to remember the currently held spin lock.
Also update the comment describing bpf_spin_lock implementation details to also talk about PTR_TO_BTF_ID | MEM_ALLOC type.
--- kernel/bpf/helpers.c | 2 + kernel/bpf/verifier.c | 90 +++++++++++++++++++++++++++++++------------ 2 files changed, 67 insertions(+), 25 deletions(-)
kernel/bpf/helpers.c +2 -0
@@ -336,6 +336,7 @@ const struct bpf_func_proto bpf_spin_lock_proto = { .gpl_only = false, .ret_type = RET_VOID, .arg1_type = ARG_PTR_TO_SPIN_LOCK, + .arg1_btf_id = BPF_PTR_POISON, };static inline void __bpf_spin_unlock_irqrestore (struct bpf_spin_lock *lock)
@@ -358,6 +359,7 @@ const struct bpf_func_proto bpf_spin_unlock_proto = { .gpl_only = false, .ret_type = RET_VOID, .arg1_type = ARG_PTR_TO_SPIN_LOCK, + .arg1_btf_id = BPF_PTR_POISON, }; void copy_map_value_locked(struct bpf_map *map, void *dst, void *src,
kernel/bpf/verifier.c +65 -25
160 lines not on this page · raw message
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 9/24 bpf: Allow locking bpf_spin_lock global variables
In reply to Kumar Kartikeya DwivediGlobal variables reside in maps accessible using direct_value_addr callbacks, so giving each load instruction's rewrite a unique reg->id disallows us from holding locks which are global.
The reason for preserving reg->id as a unique value for registers that may point to spin lock is that two separate lookups are treated as two separate memory regions, and any possible aliasing is ignored for the purposes of spin lock correctness.
This is not great especially for the global variable case, which are served from maps that have max_entries == 1, i.e. they always lead to map values pointing into the same map value.
So refactor the active_spin_lock into a 'active_lock' structure which represents the lock identity, and instead of the reg->id, remember two fields, a pointer and the reg->id. The pointer will store reg->map_ptr or reg->btf. It's only necessary to distinguish for the id == 0 case of global variables, but always setting the pointer to a non-NULL value and using the pointer to check whether the lock is held simplifies code in the verifier.
This is generic enough to allow it for global variables, map lookups, and allocated objects at the same time.
Note that while whether a lock is held can be answered by just comparing active_lock.ptr to NULL, to determine whether the register is pointing to the same held lock requires comparing _both_ ptr and id.
Finally, as a result of this refactoring, pseudo load instructions are not given a unique reg->id, as they are doing lookup for the same map value (max_entries is never greater than 1).
Essentially, we consider that the tuple of (ptr, id) will always be unique for any kind of argument to bpf_spin_{lock,unlock}.
Note that this can be extended in the future to also remember offset used for locking, so that we can introduce multiple bpf_spin_lock fields in the same allocation.
--- include/linux/bpf_verifier.h | 16 +++++++++++++- kernel/bpf/verifier.c | 41 ++++++++++++++++++++++++------------ 2 files changed, 43 insertions(+), 14 deletions(-)
include/linux/bpf_verifier.h +15 -1
@@ -323,7 +323,21 @@ struct bpf_verifier_state {u32 branches;u32 insn_idx;u32 curframe; -u32 active_spin_lock; +/* For every reg representing a map value or allocated object pointer, +* we consider the tuple of (ptr, id) for them to be unique in verifier +* context and conside them to not alias each other for the purposes of +* tracking lock state. +*/ +struct { +/* This can either be reg->map_ptr or reg->btf. If ptr is NULL, +* there's no active lock held, and other fields have no +* meaning. If non-NULL, it indicates that a lock is held and +* id member has the reg->id of the register which can be >= 0. +*/ +void *ptr; +/* This will be reg->id */ +u32 id; + } active_lock;bool speculative;/* first and last insn idx of this verifier state */
kernel/bpf/verifier.c +28 -13
100 lines not on this page · raw message
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 10/24 bpf: Allow locking bpf_spin_lock in inner map values
In reply to Kumar Kartikeya DwivediThere is no need to restrict users from locking bpf_spin_lock in map values of inner maps. Each inner map lookup gets a unique reg->id assigned to the returned PTR_TO_MAP_VALUE which will be preserved after the NULL check. Distinct lookups into different inner map get unique IDs, and distinct lookups into same inner map also get unique IDs.
Hence, lift the restriction by removing the check return -ENOTSUPP in map_in_map.c. Later commits will add comprehensive test cases to ensure that invalid cases are rejected.
--- kernel/bpf/map_in_map.c | 5 ----- 1 file changed, 5 deletions(-)
kernel/bpf/map_in_map.c +0 -5
@@ -30,11 +30,6 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) goto put; } -if (btf_record_has_field (inner_map->record,BPF_SPIN_LOCK )) { - ret = -ENOTSUPP ; - goto put; - } - inner_map_meta_size =sizeof (*inner_map_meta);/* In some cases verifier needs to access beyond just base map. */ if (inner_map->ops == &array_map_ops)
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 11/24 bpf: Rewrite kfunc argument handling
In reply to Kumar Kartikeya DwivediAs we continue to add more features, argument types, kfunc flags, and different extensions to kfuncs, the code to verify the correctness of the kfunc prototype wrt the passed in registers has become ad-hoc and ugly to read. To make life easier, and make a very clear split between different stages of argument processing, move all the code into verifier.c and refactor into easier to read helpers and functions.
This also makes sharing code within the verifier easier with kfunc argument processing. This will be more and more useful in later patches as we are now moving to implement very core BPF helpers as kfuncs, to keep them experimental before baking into UAPI.
Remove all kfunc related bits now from btf_check_func_arg_match, as users have been converted away to refactored kfunc argument handling.
--- include/linux/bpf.h | 11 - include/linux/bpf_verifier.h | 2 - include/linux/btf.h | 31 +- kernel/bpf/btf.c | 380 +----------- kernel/bpf/verifier.c | 545 +++++++++++++++++- .../bpf/prog_tests/kfunc_dynptr_param.c | 2 +- tools/testing/selftests/bpf/verifier/calls.c | 2 +- .../selftests/bpf/verifier/ref_tracking.c | 4 +- 8 files changed, 573 insertions(+), 404 deletions(-)
include/linux/bpf.h +0 -11
@@ -2109,22 +2109,11 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,const char *func_name, struct btf_func_model *m); -struct bpf_kfunc_arg_meta { -u64 r0_size; -bool r0_rdonly; -int ref_obj_id; -u32 flags; -}; -struct bpf_reg_state ;int btf_check_subprog_arg_match (struct bpf_verifier_env *env,int subprog,struct bpf_reg_state *regs);int btf_check_subprog_call (struct bpf_verifier_env *env,int subprog,struct bpf_reg_state *regs); -int btf_check_kfunc_arg_match (struct bpf_verifier_env *env, -const struct btf *btf,u32 func_id, -struct bpf_reg_state *regs, -struct bpf_kfunc_arg_meta *meta);int btf_prepare_func_args (struct bpf_verifier_env *env,int subprog,struct bpf_reg_state *reg); int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *prog,
include/linux/bpf_verifier.h +0 -2
@@ -603,8 +603,6 @@ int check_ptr_off_reg(struct bpf_verifier_env *env,int check_func_arg_reg_off (struct bpf_verifier_env *env,const struct bpf_reg_state *reg,int regno,enum bpf_arg_type arg_type); -int check_kfunc_mem_size_reg (struct bpf_verifier_env *env,struct bpf_reg_state *reg, -u32 regno);int check_mem_reg (struct bpf_verifier_env *env,struct bpf_reg_state *reg,u32 regno,u32 mem_size); bool is_dynptr_reg_valid_init(struct bpf_verifier_env *env,
include/linux/btf.h +30 -1
@@ -338,6 +338,16 @@ static inline bool btf_type_is_struct(const struct btf_type *t)return kind ==BTF_KIND_STRUCT || kind ==BTF_KIND_UNION ; } +static inline bool __btf_type_is_struct (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_STRUCT ; +} + +static inline bool btf_type_is_array (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_ARRAY ; +} +static inline u16 btf_type_vlen (const struct btf_type *t) {return BTF_INFO_VLEN (t->info);
@@ -439,9 +449,10 @@ static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)return bsearch (&id, set->pairs, set->cnt,sizeof (set->pairs[0 ]), btf_id_cmp_func); } -#ifdef CONFIG_BPF_SYSCALL struct bpf_prog ; +struct bpf_verifier_log ; +#ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id);const char *btf_name_by_offset (const struct btf *btf,u32 offset);struct btf *btf_parse_vmlinux (void );
@@ -455,6 +466,12 @@ s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id);int register_btf_id_dtor_kfuncs (const struct btf_id_dtor_kfunc *dtors,u32 add_cnt,struct module *owner);struct btf_struct_meta *btf_find_struct_meta (const struct btf *btf,u32 btf_id); +const struct btf_member * +btf_get_prog_ctx_type (struct bpf_verifier_log *log,const struct btf *btf, +const struct btf_type *t,enum bpf_prog_type prog_type, +int arg); +bool btf_types_are_same (const struct btf *btf1,u32 id1, +const struct btf *btf2,u32 id2);#else static inline const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id)
@@ -490,6 +507,18 @@ static inline struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf {return NULL ; } +static inline const struct btf_member * +btf_get_prog_ctx_type (struct bpf_verifier_log *log,const struct btf *btf, +const struct btf_type *t,enum bpf_prog_type prog_type, +int arg) +{ +return NULL ; +} +static inline bool btf_types_are_same (const struct btf *btf1,u32 id1, +const struct btf *btf2,u32 id2) +{ +return false; +}#endif static inline bool btf_type_is_struct_ptr (struct btf *btf,const struct btf_type *t)
kernel/bpf/btf.c +11 -369
481 lines not on this page · raw message
kernel/bpf/verifier.c +528 -17
606 lines not on this page · raw message
tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c +1 -1
@@ -22,7 +22,7 @@ static struct { "arg#0 pointer type STRUCT bpf_dynptr_kern points to unsupported dynamic pointer type", 0}, {"not_valid_dynptr" , "arg#0 pointer type STRUCT bpf_dynptr_kern must be valid and initialized", 0}, - {"not_ptr_to_stack", "arg#0 pointer type STRUCT bpf_dynptr_kern not to stack", 0}, + {"not_ptr_to_stack", "arg#0 expected pointer to stack", 0}, {"dynptr_data_null", NULL, -EBADMSG}, };
tools/testing/selftests/bpf/verifier/calls.c +1 -1
@@ -109,7 +109,7 @@ }, .prog_type = BPF_PROG_TYPE_SCHED_CLS, .result = REJECT, - .errstr = "arg#0 pointer type STRUCT prog_test_ref_kfunc must point", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_kfunc_call_test_acquire", 3 }, { "bpf_kfunc_call_test_release", 5 },
tools/testing/selftests/bpf/verifier/ref_tracking.c +2 -2
@@ -142,7 +142,7 @@ .kfunc = "bpf", .expected_attach_type = BPF_LSM_MAC, .flags = BPF_F_SLEEPABLE, - .errstr = "arg#0 pointer type STRUCT bpf_key must point to scalar, or struct with scalar", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_lookup_user_key", 2 }, { "bpf_key_put", 4 },
@@ -163,7 +163,7 @@ .kfunc = "bpf", .expected_attach_type = BPF_LSM_MAC, .flags = BPF_F_SLEEPABLE, - .errstr = "arg#0 pointer type STRUCT bpf_key must point to scalar, or struct with scalar", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_lookup_system_key", 1 }, { "bpf_key_put", 3 },
Signature · 1 line
2.38.1
Re: [PATCH bpf-next v10 11/24] bpf: Rewrite kfunc argument handling
kernel/bpf/btf.c int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *pr · line 6602
Quoted 6 lines · Kumar Kartikeya Dwivedi
static int btf_check_func_arg_match(struct
bpf_verifier_env*env,kernel/bpf/btf.c +0
-1const struct btf *btf, u32 func_id, struct bpf_reg_state *regs, bool ptr_to_mem_ok, - struct bpf_kfunc_arg_meta *kfunc_meta, bool processing_call)
Something odd here.
Benjamin added the processing_call flag in
commit 95f2f26f3cac ("bpf: split btf_check_subprog_arg_match in two") and we discussed to remove it.
Quoted 1 line
} else if (
ptr_to_mem_ok&&processing_call) {
since kfunc bit is gone from here the processing_call can be removed. ptr_to_mem_ok and processing_call are two bool flags for the same thing, right?
Quoted 1 line
+static int process_kf_arg_ptr_to_kptr_strong(struct
bpf_verifier_env*env,
I fixed this bit while applying.
Quoted 1 line
+static int
check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta)
This function looks much better now.
The split of kfunc vs helper was long overdue.
Thank you for doing this.
I'm not convinced that KF_ARG_* is necessary, but it's much better than before. So it's a step forward.
Pls watch for CI errors and follow up when necessary.
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v10 11/24] bpf: Rewrite kfunc argument handling
Quoted 13 lines · Alexei Starovoitov
kernel/bpf/btf.c int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *pr · line 6602
Quoted 6 lines · Kumar Kartikeya Dwivedi
static int btf_check_func_arg_match(struct
bpf_verifier_env*env,kernel/bpf/btf.c +0
-1const struct btf *btf, u32 func_id, struct bpf_reg_state *regs, bool ptr_to_mem_ok, - struct bpf_kfunc_arg_meta *kfunc_meta, bool processing_call)Something odd here.
Benjamin added theprocessing_callflag in
commit95f2f26f3cac("bpf: splitbtf_check_subprog_arg_matchin two") and we discussed to remove it.
Quoted 5 lines
Quoted 1 line
} else if (
ptr_to_mem_ok&&processing_call) {since kfunc bit is gone from here the
processing_callcan be removed.ptr_to_mem_okandprocessing_callare two bool flags for the same thing, right?
I think so, I'll check it out and send a follow up patch.
Quoted 4 lines
Quoted 1 line
+static int process_kf_arg_ptr_to_kptr_strong(struct
bpf_verifier_env*env,I fixed this bit while applying.
Thanks.
Quoted 9 lines
Quoted 1 line
+static int
check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta)This function looks much better now.
The split of kfunc vs helper was long overdue.
Thank you for doing this.I'm not convinced that KF_ARG_* is necessary, but it's much better than before. So it's a step forward.
Yes. Eventually we should be merging checks for both helpers and kfuncs, but that needs more work and would have been out of scope for this set. We can probably synthesize a bpf_func_proto for the kfunc from BTF and then offload to check_helper_call.
Quoted 1 line
Pls watch for CI errors and follow up when necessary.
Will do.
Thanks!
Re: [PATCH bpf-next v10 11/24] bpf: Rewrite kfunc argument handling
Quoted 36 lines · Kumar Kartikeya Dwivedi
Quoted 13 lines · Alexei Starovoitov
Quoted 6 lines · Kumar Kartikeya Dwivedi
static int btf_check_func_arg_match(struct
bpf_verifier_env*env,
const struct btf *btf, u32func_id, structbpf_reg_state*regs, boolptr_to_mem_ok,
- structbpf_kfunc_arg_meta*kfunc_meta,
boolprocessing_call)Something odd here.
Benjamin added theprocessing_callflag in
commit95f2f26f3cac("bpf: splitbtf_check_subprog_arg_matchin two") and we discussed to remove it.Quoted 5 lines
Quoted 1 line
} else if (ptr_to_mem_ok && processing_call) {since kfunc bit is gone from here the
processing_callcan be removed.ptr_to_mem_okandprocessing_callare two bool flags for the same thing, right?I think so, I'll check it out and send a follow up patch.
Quoted 4 lines
Quoted 1 line
+static int process_kf_arg_ptr_to_kptr_strong(struct
bpf_verifier_env*env,I fixed this bit while applying.
Thanks.
Quoted 6 lines
Quoted 1 line
+static int
check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta)This function looks much better now.
The split of kfunc vs helper was long overdue.
Thank you for doing this.
Quoted 8 lines
Quoted 3 lines
I'm not convinced that KF_ARG_* is necessary, but it's much better than before. So it's a step forward.
Yes. Eventually we should be merging checks for both helpers and kfuncs, but that needs more work and would have been out of scope for this set. We can probably synthesize a
bpf_func_protofor the kfunc from BTF and then offload tocheck_helper_call.
Yeah. If kfunc BTFs plus KF_ flags can be synthesized to bpf_func_proto that would be the best. If such conversion is possible then it should be possible to do it in resolve_btfid in user space.
One more thing that I forgot to mention earlier.
Could you follow up with a patch to get rid of bpf_global_ma_set check in the run-time and variable itself?
If bpf_mem_alloc_init fails the boot fails too.
If we're paranoid we can add:
special_kfunc_list[KF_bpf_obj_new_impl] = 0;
to bpf_mem_alloc_init() to prevent bpf_obj_new to ever be called.
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v10 11/24] bpf: Rewrite kfunc argument handling
Quoted 42 lines · Alexei Starovoitov
Quoted 40 lines · Kumar Kartikeya Dwivedi
Quoted 13 lines · Alexei Starovoitov
Quoted 6 lines · Kumar Kartikeya Dwivedi
static int btf_check_func_arg_match(struct
bpf_verifier_env*env,
const struct btf *btf, u32func_id, structbpf_reg_state*regs, boolptr_to_mem_ok,
- structbpf_kfunc_arg_meta*kfunc_meta,
boolprocessing_call)Something odd here.
Benjamin added theprocessing_callflag in
commit95f2f26f3cac("bpf: splitbtf_check_subprog_arg_matchin two") and we discussed to remove it.Quoted 5 lines
Quoted 1 line
} else if (ptr_to_mem_ok && processing_call) {since kfunc bit is gone from here the
processing_callcan be removed.ptr_to_mem_okandprocessing_callare two bool flags for the same thing, right?I think so, I'll check it out and send a follow up patch.
Quoted 4 lines
Quoted 1 line
+static int process_kf_arg_ptr_to_kptr_strong(struct
bpf_verifier_env*env,I fixed this bit while applying.
Thanks.
Quoted 9 lines
Quoted 1 line
+static int
check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta)This function looks much better now.
The split of kfunc vs helper was long overdue.
Thank you for doing this.I'm not convinced that KF_ARG_* is necessary, but it's much better than before. So it's a step forward.
Quoted 9 lines
Quoted 4 lines
Yes. Eventually we should be merging checks for both helpers and kfuncs, but that needs more work and would have been out of scope for this set. We can probably synthesize a
bpf_func_protofor the kfunc from BTF and then offload tocheck_helper_call.Yeah. If kfunc BTFs plus
KF_flags can be synthesized tobpf_func_protothat would be the best. If such conversion is possible then it should be possible to do it inresolve_btfidin user space.
Yep. I'll poke at it some more later.
Quoted 7 lines
One more thing that I forgot to mention earlier.
Could you follow up with a patch to get rid ofbpf_global_ma_setcheck in the run-time and variable itself?
Ifbpf_mem_alloc_initfails the boot fails too.
If we're paranoid we can add:
special_kfunc_list[KF_bpf_obj_new_impl] = 0;
tobpf_mem_alloc_init()to preventbpf_obj_newto ever be called.
I did it a bit differently, but it does the same thing, and sent it out with the s390x fix. PTAL.
Re: [PATCH bpf-next v10 11/24] bpf: Rewrite kfunc argument handling
In reply to Kumar Kartikeya DwivediQuoted 4 lines · Kumar Kartikeya Dwivedi
Quoted 2 lines
Pls watch for CI errors and follow up when necessary.
Will do.
test_progs is failing on s390 with:
test_spin_lock_fail_prog:PASS:test_spin_lock_fail__load must fail 0 nsec test_spin_lock_fail_prog:FAIL:expected error message unexpected error: -524
I bet it's your change.
Please take a look.
Re: [PATCH bpf-next v10 11/24] bpf: Rewrite kfunc argument handling
In reply to Kumar Kartikeya DwivediQuoted 12 lines · Kumar Kartikeya Dwivedi
As we continue to add more features, argument types, kfunc flags, and different extensions to kfuncs, the code to verify the correctness of the kfunc prototype wrt the passed in registers has become ad-hoc and ugly to read. To make life easier, and make a very clear split between different stages of argument processing, move all the code into
verifier.cand refactor into easier to read helpers and functions.This also makes sharing code within the verifier easier with kfunc argument processing. This will be more and more useful in later patches as we are now moving to implement very core BPF helpers as kfuncs, to keep them experimental before baking into UAPI.
Quoted 4 lines
Remove all kfunc related bits now from
btf_check_func_arg_match, as users have been converted away to refactored kfunc argument handling.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
Thanks for working on this. I'm relieved to see this work being done. I have a few comments but overall this is great. I'll take a closer look later.
Quoted 693 lines
--- include/linux/bpf.h | 11 - include/linux/bpf_verifier.h | 2 - include/linux/btf.h | 31 +- kernel/bpf/btf.c | 380 +----------- kernel/bpf/verifier.c | 545 +++++++++++++++++- .../bpf/prog_tests/kfunc_dynptr_param.c | 2 +- tools/testing/selftests/bpf/verifier/calls.c | 2 +- .../selftests/bpf/verifier/ref_tracking.c | 4 +- 8 files changed, 573 insertions(+), 404 deletions(-)include/linux/bpf.h +0
-11@@ -2109,22 +2109,11 @@ int btf_distill_func_proto(struct bpf_verifier_log *log,const char *func_name, struct btf_func_model *m); -struct bpf_kfunc_arg_meta { -u64 r0_size; -bool r0_rdonly; -int ref_obj_id; -u32 flags; -}; -struct bpf_reg_state ;int btf_check_subprog_arg_match (struct bpf_verifier_env *env,int subprog,struct bpf_reg_state *regs);int btf_check_subprog_call (struct bpf_verifier_env *env,int subprog,struct bpf_reg_state *regs); -int btf_check_kfunc_arg_match (struct bpf_verifier_env *env, -const struct btf *btf,u32 func_id, -struct bpf_reg_state *regs, -struct bpf_kfunc_arg_meta *meta);int btf_prepare_func_args (struct bpf_verifier_env *env,int subprog,struct bpf_reg_state *reg); int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *prog,include/linux/bpf_verifier.h +0
-2@@ -603,8 +603,6 @@ int check_ptr_off_reg(struct bpf_verifier_env *env,int check_func_arg_reg_off (struct bpf_verifier_env *env,const struct bpf_reg_state *reg,int regno,enum bpf_arg_type arg_type); -int check_kfunc_mem_size_reg (struct bpf_verifier_env *env,struct bpf_reg_state *reg, -u32 regno);int check_mem_reg (struct bpf_verifier_env *env,struct bpf_reg_state *reg,u32 regno,u32 mem_size); bool is_dynptr_reg_valid_init(struct bpf_verifier_env *env,include/linux/btf.h +30
-1@@ -338,6 +338,16 @@ static inline bool btf_type_is_struct(const struct btf_type *t)return kind ==BTF_KIND_STRUCT || kind ==BTF_KIND_UNION ; } +static inline bool __btf_type_is_struct (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_STRUCT ; +} + +static inline bool btf_type_is_array (const struct btf_type *t) +{ +return BTF_INFO_KIND (t->info) ==BTF_KIND_ARRAY ; +} +static inline u16 btf_type_vlen (const struct btf_type *t) {return BTF_INFO_VLEN (t->info);@@ -439,9 +449,10 @@ static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)return bsearch (&id, set->pairs, set->cnt,sizeof (set->pairs[0 ]), btf_id_cmp_func); } -#ifdef CONFIG_BPF_SYSCALL struct bpf_prog ; +struct bpf_verifier_log ; +#ifdef CONFIG_BPF_SYSCALL const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id);const char *btf_name_by_offset (const struct btf *btf,u32 offset);struct btf *btf_parse_vmlinux (void );@@ -455,6 +466,12 @@ s32 btf_find_dtor_kfunc(struct btf *btf, u32 btf_id);int register_btf_id_dtor_kfuncs (const struct btf_id_dtor_kfunc *dtors,u32 add_cnt,struct module *owner);struct btf_struct_meta *btf_find_struct_meta (const struct btf *btf,u32 btf_id); +const struct btf_member * +btf_get_prog_ctx_type (struct bpf_verifier_log *log,const struct btf *btf, +const struct btf_type *t,enum bpf_prog_type prog_type, +int arg); +bool btf_types_are_same (const struct btf *btf1,u32 id1, +const struct btf *btf2,u32 id2);#else static inline const struct btf_type *btf_type_by_id (const struct btf *btf,u32 type_id)@@ -490,6 +507,18 @@ static inline struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf {return NULL ; } +static inline const struct btf_member * +btf_get_prog_ctx_type (struct bpf_verifier_log *log,const struct btf *btf, +const struct btf_type *t,enum bpf_prog_type prog_type, +int arg) +{ +return NULL ; +} +static inline bool btf_types_are_same (const struct btf *btf1,u32 id1, +const struct btf *btf2,u32 id2) +{ +return false; +}#endif static inline bool btf_type_is_struct_ptr (struct btf *btf,const struct btf_type *t)kernel/bpf/btf.c +11
-369@@ -478,16 +478,6 @@ static bool btf_type_nosize_or_null(const struct btf_type *t)return !t ||btf_type_nosize (t); } -static bool __btf_type_is_struct (const struct btf_type *t) -{ -return BTF_INFO_KIND (t->info) ==BTF_KIND_STRUCT ; -} - -static bool btf_type_is_array (const struct btf_type *t) -{ -return BTF_INFO_KIND (t->info) ==BTF_KIND_ARRAY ; -} -static bool btf_type_is_datasec (const struct btf_type *t) {return BTF_INFO_KIND (t->info) ==BTF_KIND_DATASEC ;@@ -5536,7 +5526,7 @@ static u8 bpf_ctx_convert_map[] = {#undef BPF_MAP_TYPE#undef BPF_LINK_TYPE -static const struct btf_member * +const struct btf_member *btf_get_prog_ctx_type (struct bpf_verifier_log *log,const struct btf *btf,const struct btf_type *t,enum bpf_prog_type prog_type,int arg)@@ -6322,8 +6312,8 @@ int btf_struct_access(struct bpf_verifier_log *log,* end up with two different module BTFs, but IDs point to the common type in * vmlinux BTF. */ -static bool btf_types_are_same (const struct btf *btf1,u32 id1, -const struct btf *btf2,u32 id2) +bool btf_types_are_same (const struct btf *btf1,u32 id1, +const struct btf *btf2,u32 id2) {if (id1 != id2)return false;@@ -6605,122 +6595,19 @@ int btf_check_type_match(struct bpf_verifier_log *log, const struct bpf_prog *prreturn btf_check_func_type_match (log, btf1, t1, btf2, t2); } -static u32 *reg2btf_ids[__BPF_REG_TYPE_MAX] = { -#ifdef CONFIG_NET - [PTR_TO_SOCKET ] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK ], - [PTR_TO_SOCK_COMMON ] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK_COMMON ], - [PTR_TO_TCP_SOCK ] = &btf_sock_ids[BTF_SOCK_TYPE_TCP ], -#endif -}; - -/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ -static bool __btf_type_is_scalar_struct (struct bpf_verifier_log *log, -const struct btf *btf, -const struct btf_type *t,int rec) -{ -const struct btf_type *member_type; -const struct btf_member *member; -u32 i; - -if (!btf_type_is_struct (t)) -return false; - -for_each_member (i, t, member) { -const struct btf_array *array; - - member_type =btf_type_skip_modifiers (btf, member->type,NULL ); -if (btf_type_is_struct (member_type)) { -if (rec >=3 ) { -bpf_log (log,"max struct nesting depth exceeded\n" ); -return false; - } -if (!__btf_type_is_scalar_struct (log, btf, member_type, rec +1 )) -return false; -continue ; - } -if (btf_type_is_array (member_type)) { - array =btf_type_array (member_type); -if (!array->nelems) -return false; - member_type =btf_type_skip_modifiers (btf, array->type,NULL ); -if (!btf_type_is_scalar (member_type)) -return false; -continue ; - } -if (!btf_type_is_scalar (member_type)) -return false; - } -return true; -} - -static bool is_kfunc_arg_mem_size (const struct btf *btf, -const struct btf_param *arg, -const struct bpf_reg_state *reg) -{ -int len, sfx_len =sizeof ("__sz" ) -1 ; -const struct btf_type *t; -const char *param_name; - - t =btf_type_skip_modifiers (btf, arg->type,NULL ); -if (!btf_type_is_scalar (t) || reg->type !=SCALAR_VALUE ) -return false; - -/* In the future, this can be ported to use BTF tagging */ - param_name =btf_name_by_offset (btf, arg->name_off); -if (str_is_empty (param_name)) -return false; - len =strlen (param_name); -if (len < sfx_len) -return false; - param_name += len - sfx_len; -if (strncmp (param_name,"__sz" , sfx_len)) -return false; - -return true; -} - -static bool btf_is_kfunc_arg_mem_size (const struct btf *btf, -const struct btf_param *arg, -const struct bpf_reg_state *reg, -const char *name) -{ -int len, target_len =strlen (name); -const struct btf_type *t; -const char *param_name; - - t =btf_type_skip_modifiers (btf, arg->type,NULL ); -if (!btf_type_is_scalar (t) || reg->type !=SCALAR_VALUE ) -return false; - - param_name =btf_name_by_offset (btf, arg->name_off); -if (str_is_empty (param_name)) -return false; - len =strlen (param_name); -if (len != target_len) -return false; -if (strcmp (param_name, name)) -return false; - -return true; -} -static int btf_check_func_arg_match (struct bpf_verifier_env *env,const struct btf *btf,u32 func_id,struct bpf_reg_state *regs,bool ptr_to_mem_ok, -struct bpf_kfunc_arg_meta *kfunc_meta,bool processing_call) {enum bpf_prog_type prog_type =resolve_prog_type (env->prog); -bool rel = false, kptr_get = false, trusted_args = false; -bool sleepable = false;struct bpf_verifier_log *log = &env->log; -u32 i, nargs, ref_id, ref_obj_id =0 ; -bool is_kfunc =btf_is_kernel (btf);const char *func_name, *ref_tname;const struct btf_type *t, *ref_t;const struct btf_param *args; -int ref_regno =0 , ret; +u32 i, nargs, ref_id; +int ret; t =btf_type_by_id (btf, func_id);if (!t || !btf_type_is_func (t)) {@@ -6746,14 +6633,6 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,return -EINVAL ; } -if (is_kfunc && kfunc_meta) { -/* Only kfunc can be release func */ - rel = kfunc_meta->flags &KF_RELEASE ; - kptr_get = kfunc_meta->flags &KF_KPTR_GET ; - trusted_args = kfunc_meta->flags &KF_TRUSTED_ARGS ; - sleepable = kfunc_meta->flags &KF_SLEEPABLE ; - } -/* check that BTF function arguments match actual types that the * verifier sees. */ @@ -6761,42 +6640,9 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,enum bpf_arg_type arg_type =ARG_DONTCARE ;u32 regno = i +1 ;struct bpf_reg_state *reg = ®s[regno]; -bool obj_ptr = false; t =btf_type_skip_modifiers (btf, args[i].type,NULL );if (btf_type_is_scalar (t)) { -if (is_kfunc && kfunc_meta) { -bool is_buf_size = false; - -/* check for any const scalar parameter of name "rdonly_buf_size" -* or "rdwr_buf_size" -*/ -if (btf_is_kfunc_arg_mem_size (btf, &args[i], reg, -"rdonly_buf_size" )) { - kfunc_meta->r0_rdonly = true; - is_buf_size = true; - }else if (btf_is_kfunc_arg_mem_size (btf, &args[i], reg, -"rdwr_buf_size" )) - is_buf_size = true; - -if (is_buf_size) { -if (kfunc_meta->r0_size) { -bpf_log (log,"2 or more rdonly/rdwr_buf_size parameters for kfunc" ); -return -EINVAL ; - } - -if (!tnum_is_const (reg->var_off)) { -bpf_log (log,"R%d is not a const\n" , regno); -return -EINVAL ; - } - - kfunc_meta->r0_size = reg->var_off.value; - ret =mark_chain_precision (env, regno); -if (ret) -return ret; - } - } -if (reg->type ==SCALAR_VALUE )continue ;bpf_log (log,"R%d is not a scalar\n" , regno);@@ -6809,88 +6655,14 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,return -EINVAL ; } -/* These register types have special constraints wrt ref_obj_id -* and offset checks. The rest of trusted args don't. -*/ - obj_ptr = reg->type ==PTR_TO_CTX || reg->type ==PTR_TO_BTF_ID || - reg2btf_ids[base_type (reg->type)]; - -/* Check if argument must be a referenced pointer, args + i has -* been verified to be a pointer (after skipping modifiers). -* PTR_TO_CTX is ok without having non-zero ref_obj_id. -*/ -if (is_kfunc && trusted_args && (obj_ptr && reg->type !=PTR_TO_CTX ) && !reg->ref_obj_id) { -bpf_log (log,"R%d must be referenced\n" , regno); -return -EINVAL ; - } - ref_t =btf_type_skip_modifiers (btf, t->type, &ref_id); ref_tname =btf_name_by_offset (btf, ref_t->name_off); -/* Trusted args have the same offset checks as release arguments */ -if ((trusted_args && obj_ptr) || (rel && reg->ref_obj_id)) - arg_type |=OBJ_RELEASE ; ret =check_func_arg_reg_off (env, reg, regno, arg_type);if (ret <0 )return ret; -if (is_kfunc && reg->ref_obj_id) { -/* Ensure only one argument is referenced PTR_TO_BTF_ID */ -if (ref_obj_id) { -bpf_log (log,"verifier internal error: more than one arg with ref_obj_id R%d %u %u\n" , - regno, reg->ref_obj_id, ref_obj_id); -return -EFAULT ; - } - ref_regno = regno; - ref_obj_id = reg->ref_obj_id; - } - -/* kptr_get is only true for kfunc */ -if (i ==0 && kptr_get) { -struct btf_field *kptr_field; - -if (reg->type !=PTR_TO_MAP_VALUE ) { -bpf_log (log,"arg#0 expected pointer to map value\n" ); -return -EINVAL ; - } - -/* check_func_arg_reg_off allows var_off for -* PTR_TO_MAP_VALUE, but we need fixed offset to find -* off_desc. -*/ -if (!tnum_is_const (reg->var_off)) { -bpf_log (log,"arg#0 must have constant offset\n" ); -return -EINVAL ; - } - - kptr_field =btf_record_find (reg->map_ptr->record, reg->off + reg->var_off.value,BPF_KPTR ); -if (!kptr_field || kptr_field->type !=BPF_KPTR_REF ) { -bpf_log (log,"arg#0 no referenced kptr at map value offset=%llu\n" , - reg->off + reg->var_off.value); -return -EINVAL ; - } - -if (!btf_type_is_ptr (ref_t)) { -bpf_log (log,"arg#0 BTF type must be a double pointer\n" ); -return -EINVAL ; - } - - ref_t =btf_type_skip_modifiers (btf, ref_t->type, &ref_id); - ref_tname =btf_name_by_offset (btf, ref_t->name_off); - -if (!btf_type_is_struct (ref_t)) { -bpf_log (log,"kernel function %s args#%d pointer type %s %s is not supported\n" , - func_name, i,btf_type_str (ref_t), ref_tname); -return -EINVAL ; - } -if (!btf_struct_ids_match (log, btf, ref_id,0 , kptr_field->kptr.btf, - kptr_field->kptr.btf_id, true)) { -bpf_log (log,"kernel function %s args#%d expected pointer to %s %s\n" , - func_name, i,btf_type_str (ref_t), ref_tname); -return -EINVAL ; - } -/* rest of the arguments can be anything, like normal kfunc */ - }else if (btf_get_prog_ctx_type (log, btf, t, prog_type, i)) { +if (btf_get_prog_ctx_type (log, btf, t, prog_type, i)) {/* If function expects ctx type in BTF check that caller * is passing PTR_TO_CTX. */ @@ -6900,109 +6672,10 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env, i, btf_type_str(t));return -EINVAL ; } - }else if (is_kfunc && (reg->type ==PTR_TO_BTF_ID || - (reg2btf_ids[base_type (reg->type)] && !type_flag (reg->type)))) { -const struct btf_type *reg_ref_t; -const struct btf *reg_btf; -const char *reg_ref_tname; -u32 reg_ref_id; - -if (!btf_type_is_struct (ref_t)) { -bpf_log (log,"kernel function %s args#%d pointer type %s %s is not supported\n" , - func_name, i,btf_type_str (ref_t), - ref_tname); -return -EINVAL ; - } - -if (reg->type ==PTR_TO_BTF_ID ) { - reg_btf = reg->btf; - reg_ref_id = reg->btf_id; - }else { - reg_btf = btf_vmlinux; - reg_ref_id = *reg2btf_ids[base_type (reg->type)]; - } - - reg_ref_t =btf_type_skip_modifiers (reg_btf, reg_ref_id, - ®_ref_id); - reg_ref_tname =btf_name_by_offset (reg_btf, - reg_ref_t->name_off); -if (!btf_struct_ids_match (log, reg_btf, reg_ref_id, - reg->off, btf, ref_id, - trusted_args || (rel && reg->ref_obj_id))) { -bpf_log (log,"kernel function %s args#%d expected pointer to %s %s but R%d has a pointer to %s %s\n" , - func_name, i, -btf_type_str (ref_t), ref_tname, - regno,btf_type_str (reg_ref_t), - reg_ref_tname); -return -EINVAL ; - } }else if (ptr_to_mem_ok && processing_call) {const struct btf_type *resolve_ret;u32 type_size; -if (is_kfunc) { -bool arg_mem_size = i +1 < nargs &&is_kfunc_arg_mem_size (btf, &args[i +1 ], ®s[regno +1 ]); -bool arg_dynptr =btf_type_is_struct (ref_t) && - !strcmp (ref_tname, -stringify_struct (bpf_dynptr_kern)); - -/* Permit pointer to mem, but only when argument -* type is pointer to scalar, or struct composed -* (recursively) of scalars. -* When arg_mem_size is true, the pointer can be -* void *. -* Also permit initialized local dynamic pointers. -*/ -if (!btf_type_is_scalar (ref_t) && - !__btf_type_is_scalar_struct (log, btf, ref_t,0 ) && - !arg_dynptr && - (arg_mem_size ? !btf_type_is_void (ref_t) :1 )) { -bpf_log (log, -"arg#%d pointer type %s %s must point to %sscalar, or struct with scalar\n" , - i,btf_type_str (ref_t), ref_tname, arg_mem_size ?"void, " :"" ); -return -EINVAL ; - } - -if (arg_dynptr) { -if (reg->type !=PTR_TO_STACK ) { -bpf_log (log,"arg#%d pointer type %s %s not to stack\n" , - i,btf_type_str (ref_t), - ref_tname); -return -EINVAL ; - } - -if (!is_dynptr_reg_valid_init (env, reg)) { -bpf_log (log, -"arg#%d pointer type %s %s must be valid and initialized\n" , - i,btf_type_str (ref_t), - ref_tname); -return -EINVAL ; - } - -if (!is_dynptr_type_expected (env, reg, -ARG_PTR_TO_DYNPTR |DYNPTR_TYPE_LOCAL )) { -bpf_log (log, -"arg#%d pointer type %s %s points to unsupported dynamic pointer type\n" , - i,btf_type_str (ref_t), - ref_tname); -return -EINVAL ; - } - -continue ; - } - -/* Check for mem, len pair */ -if (arg_mem_size) { -if (check_kfunc_mem_size_reg (env, ®s[regno +1 ], regno +1 )) { -bpf_log (log,"arg#%d arg#%d memory, len pair leads to invalid memory access\n" , - i, i +1 ); -return -EINVAL ; - } - i++; -continue ; - } - } - resolve_ret =btf_resolve_size (btf, ref_t, &type_size);if (IS_ERR (resolve_ret)) { bpf_log(log,@@ -7015,36 +6688,13 @@ static int btf_check_func_arg_match(struct bpf_verifier_env *env,if (check_mem_reg (env, reg, regno, type_size))return -EINVAL ; }else { -bpf_log (log,"reg type unsupported for arg#%d %sfunction %s#%d\n" , i, - is_kfunc ?"kernel " :"" , func_name, func_id); +bpf_log (log,"reg type unsupported for arg#%d function %s#%d\n" , i, + func_name, func_id);return -EINVAL ; } } -/* Either both are set, or neither */ -WARN_ON_ONCE ((ref_obj_id && !ref_regno) || (!ref_obj_id && ref_regno)); -/* We already made sure ref_obj_id is set only for one argument. We do -* allow (!rel && ref_obj_id), so that passing such referenced -* PTR_TO_BTF_ID to other kfuncs works. Note that rel is only true when -* is_kfunc is true. -*/ -if (rel && !ref_obj_id) { -bpf_log (log,"release kernel function %s expects refcounted PTR_TO_BTF_ID\n" , - func_name); -return -EINVAL ; - } - -if (sleepable && !env->prog->aux->sleepable) { -bpf_log (log,"kernel function %s is sleepable but the program is not\n" , - func_name); -return -EINVAL ; - } - -if (kfunc_meta && ref_obj_id) - kfunc_meta->ref_obj_id = ref_obj_id; - -/* returns argument register number > 0 in case of reference release kfunc */ -return rel ? ref_regno :0 ; +return 0 ; }/* Compare BTF of a function declaration with given bpf_reg_state. @@ -7074,7 +6724,7 @@ int btf_check_subprog_arg_match(struct bpf_verifier_env *env, int subprog,return -EINVAL ; is_global = prog->aux->func_info_aux[subprog].linkage ==BTF_FUNC_GLOBAL ; - err =btf_check_func_arg_match (env, btf, btf_id, regs, is_global,NULL , false); + err =btf_check_func_arg_match (env, btf, btf_id, regs, is_global, false);/* Compiler optimizations can remove arguments from static functions * or mismatched type can be passed into a global function. @@ -7117,7 +6767,7 @@ int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog,return -EINVAL ; is_global = prog->aux->func_info_aux[subprog].linkage ==BTF_FUNC_GLOBAL ; - err =btf_check_func_arg_match (env, btf, btf_id, regs, is_global,NULL , true); + err =btf_check_func_arg_match (env, btf, btf_id, regs, is_global, true);/* Compiler optimizations can remove arguments from static functions * or mismatched type can be passed into a global function. @@ -7128,14 +6778,6 @@ int btf_check_subprog_call(struct bpf_verifier_env *env, int subprog,return err; } -int btf_check_kfunc_arg_match (struct bpf_verifier_env *env, -const struct btf *btf,u32 func_id, -struct bpf_reg_state *regs, -struct bpf_kfunc_arg_meta *meta) -{ -return btf_check_func_arg_match (env, btf, func_id, regs, true, meta, true); -} -/* Convert BTF of a function into bpf_reg_state if possible * Returns: * EFAULT - there is a verifier bug. Abort verification. kernel/bpf/verifier.c +63
-2@@ -5550,8 +5550,8 @@ int check_mem_reg(struct bpf_verifier_env *env, struct bpf_reg_state *reg,return err; } -int check_kfunc_mem_size_reg (struct bpf_verifier_env *env,struct bpf_reg_state *reg, -u32 regno) +static int check_kfunc_mem_size_reg (struct bpf_verifier_env *env,struct bpf_reg_state *reg, +u32 regno) {struct bpf_reg_state *mem_reg = &cur_regs (env)[regno -1 ];bool may_be_null =type_may_be_null (mem_reg->type);@@ -7863,19 +7863,517 @@ static void mark_btf_func_reg_size(struct bpf_verifier_env *env, u32 regno, } } +struct bpf_kfunc_call_arg_meta { +/* In parameters */ +struct btf *btf; +u32 func_id; +u32 kfunc_flags; +const struct btf_type *func_proto; +const char *func_name; +/* Out parameters */ +u32 ref_obj_id; +u8 release_regno; +bool r0_rdonly; +u64 r0_size; +}; + +static bool is_kfunc_acquire (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_ACQUIRE ; +} + +static bool is_kfunc_ret_null (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_RET_NULL ; +} + +static bool is_kfunc_release (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_RELEASE ; +} + +static bool is_kfunc_trusted_args (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_TRUSTED_ARGS ; +} + +static bool is_kfunc_sleepable (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_SLEEPABLE ; +} + +static bool is_kfunc_destructive (struct bpf_kfunc_call_arg_meta *meta) +{ +return meta->kfunc_flags &KF_DESTRUCTIVE ; +} + +static bool is_kfunc_arg_kptr_get (struct bpf_kfunc_call_arg_meta *meta,int arg) +{ +return arg ==0 && (meta->kfunc_flags &KF_KPTR_GET ); +} + +static bool is_kfunc_arg_mem_size (const struct btf *btf, +const struct btf_param *arg, +const struct bpf_reg_state *reg) +{ +int len, sfx_len =sizeof ("__sz" ) -1 ; +const struct btf_type *t; +const char *param_name; + + t =btf_type_skip_modifiers (btf, arg->type,NULL ); +if (!btf_type_is_scalar (t) || reg->type !=SCALAR_VALUE ) +return false; +
kernel/bpf/verifier.c static void mark_btf_func_reg_size(struct bpf_verifier_env *env, u32 regno, · line 7936
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+/* In the future, this can be ported to use BTF tagging */ + param_name =btf_name_by_offset (btf, arg->name_off); +if (str_is_empty (param_name)) +return false; + len =strlen (param_name); +if (len < sfx_len) +return false; + param_name += len - sfx_len; +if (strncmp (param_name,"__sz" , sfx_len)) +return false;
Oh, I thought we weren't doing this arg-type-by-name-suffix thing? I see that you're just moving it around so it's fine to move it here, but it seems to diverge from the discussions I've seen in the past. Don't want to distract from the patch but is there a plan to get rid of this at some point?
Quoted 12 lines
kernel/bpf/verifier.c +12
-0+ +return true; +} + +static bool is_kfunc_arg_scalar_with_name (const struct btf *btf, +const struct btf_param *arg, +const char *name) +{ +int len, target_len =strlen (name); +const char *param_name; + + param_name =btf_name_by_offset (btf, arg->name_off);
kernel/bpf/verifier.c static void mark_btf_func_reg_size(struct bpf_verifier_env *env, u32 regno, · line 7958
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+if (str_is_empty (param_name)) +return false; + len =strlen (param_name); +if (len != target_len) +return false; +if (strcmp (param_name, name)) +return false; + +return true; +}
It doesn't look like this function has anything to do with the arg being a scalar, does it? Should we just rename it something like, "kfunc_arg_has_name()"?
Quoted 215 lines
kernel/bpf/verifier.c +215
-0+ +enum { +KF_ARG_DYNPTR_ID , +}; + +BTF_ID_LIST (kf_arg_btf_ids) +BTF_ID(struct, bpf_dynptr_kern) + +static bool is_kfunc_arg_dynptr(const struct btf *btf, + const struct btf_param *arg) +{ +const struct btf_type *t; +u32 res_id; + + t =btf_type_skip_modifiers (btf, arg->type,NULL ); +if (!t) +return false; +if (!btf_type_is_ptr (t)) +return false; + t =btf_type_skip_modifiers (btf, t->type, &res_id); +if (!t) +return false; +return btf_types_are_same (btf, res_id, btf_vmlinux, kf_arg_btf_ids[KF_ARG_DYNPTR_ID ]); +} + +/* Returns true if struct is composed of scalars, 4 levels of nesting allowed */ +static bool __btf_type_is_scalar_struct (struct bpf_verifier_env *env, +const struct btf *btf, +const struct btf_type *t,int rec) +{ +const struct btf_type *member_type; +const struct btf_member *member; +u32 i; + +if (!btf_type_is_struct (t)) +return false; + +for_each_member (i, t, member) { +const struct btf_array *array; + + member_type =btf_type_skip_modifiers (btf, member->type,NULL ); +if (btf_type_is_struct (member_type)) { +if (rec >=3 ) { +verbose (env,"max struct nesting depth exceeded\n" ); +return false; + } +if (!__btf_type_is_scalar_struct (env, btf, member_type, rec +1 )) +return false; +continue ; + } +if (btf_type_is_array (member_type)) { + array =btf_array (member_type); +if (!array->nelems) +return false; + member_type =btf_type_skip_modifiers (btf, array->type,NULL ); +if (!btf_type_is_scalar (member_type)) +return false; +continue ; + } +if (!btf_type_is_scalar (member_type)) +return false; + } +return true; +} + + +static u32 *reg2btf_ids[__BPF_REG_TYPE_MAX] = { +#ifdef CONFIG_NET + [PTR_TO_SOCKET ] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK ], + [PTR_TO_SOCK_COMMON ] = &btf_sock_ids[BTF_SOCK_TYPE_SOCK_COMMON ], + [PTR_TO_TCP_SOCK ] = &btf_sock_ids[BTF_SOCK_TYPE_TCP ], +#endif +}; + +enum kfunc_ptr_arg_type { +KF_ARG_PTR_TO_CTX , +KF_ARG_PTR_TO_KPTR ,/* PTR_TO_KPTR but type specific */ +KF_ARG_PTR_TO_DYNPTR , +KF_ARG_PTR_TO_BTF_ID ,/* Also covers reg2btf_ids conversions */ +KF_ARG_PTR_TO_MEM , +KF_ARG_PTR_TO_MEM_SIZE ,/* Size derived from next argument, skip it */ +}; + +static enum kfunc_ptr_arg_type +get_kfunc_ptr_arg_type (struct bpf_verifier_env *env, +struct bpf_kfunc_call_arg_meta *meta, +const struct btf_type *t,const struct btf_type *ref_t, +const char *ref_tname,const struct btf_param *args, +int argno,int nargs) +{ +u32 regno = argno +1 ; +struct bpf_reg_state *regs =cur_regs (env); +struct bpf_reg_state *reg = ®s[regno]; +bool arg_mem_size = false; + +/* In this function, we verify the kfunc's BTF as per the argument type, +* leaving the rest of the verification with respect to the register +* type to our caller. When a set of conditions hold in the BTF type of +* arguments, we resolve it to a known kfunc_ptr_arg_type. +*/ +if (btf_get_prog_ctx_type (&env->log, meta->btf, t,resolve_prog_type (env->prog), argno)) +return KF_ARG_PTR_TO_CTX ; + +if (is_kfunc_arg_kptr_get (meta, argno)) { +if (!btf_type_is_ptr (ref_t)) { +verbose (env,"arg#0 BTF type must be a double pointer for kptr_get kfunc\n" ); +return -EINVAL ; + } + ref_t =btf_type_by_id (meta->btf, ref_t->type); + ref_tname =btf_name_by_offset (meta->btf, ref_t->name_off); +if (!btf_type_is_struct (ref_t)) { +verbose (env,"kernel function %s args#0 pointer type %s %s is not supported\n" , + meta->func_name,btf_type_str (ref_t), ref_tname); +return -EINVAL ; + } +return KF_ARG_PTR_TO_KPTR ; + } + +if (is_kfunc_arg_dynptr (meta->btf, &args[argno])) +return KF_ARG_PTR_TO_DYNPTR ; + +if ((base_type (reg->type) ==PTR_TO_BTF_ID || reg2btf_ids[base_type (reg->type)])) { +if (!btf_type_is_struct (ref_t)) { +verbose (env,"kernel function %s args#%d pointer type %s %s is not supported\n" , + meta->func_name, argno,btf_type_str (ref_t), ref_tname); +return -EINVAL ; + } +return KF_ARG_PTR_TO_BTF_ID ; + } + +if (argno +1 < nargs &&is_kfunc_arg_mem_size (meta->btf, &args[argno +1 ], ®s[regno +1 ])) + arg_mem_size = true; + +/* This is the catch all argument type of register types supported by +* check_helper_mem_access. However, we only allow when argument type is +* pointer to scalar, or struct composed (recursively) of scalars. When +* arg_mem_size is true, the pointer can be void *. +*/ +if (!btf_type_is_scalar (ref_t) && !__btf_type_is_scalar_struct (env, meta->btf, ref_t,0 ) && + (arg_mem_size ? !btf_type_is_void (ref_t) :1 )) { +verbose (env,"arg#%d pointer type %s %s must point to %sscalar, or struct with scalar\n" , + argno,btf_type_str (ref_t), ref_tname, arg_mem_size ?"void, " :"" ); +return -EINVAL ; + } +return arg_mem_size ?KF_ARG_PTR_TO_MEM_SIZE :KF_ARG_PTR_TO_MEM ; +} + +static int process_kf_arg_ptr_to_btf_id (struct bpf_verifier_env *env, +struct bpf_reg_state *reg, +const struct btf_type *ref_t, +const char *ref_tname,u32 ref_id, +struct bpf_kfunc_call_arg_meta *meta, +int argno) +{ +const struct btf_type *reg_ref_t; +bool strict_type_match = false; +const struct btf *reg_btf; +const char *reg_ref_tname; +u32 reg_ref_id; + +if (reg->type ==PTR_TO_BTF_ID ) { + reg_btf = reg->btf; + reg_ref_id = reg->btf_id; + }else { + reg_btf = btf_vmlinux; + reg_ref_id = *reg2btf_ids[base_type (reg->type)]; + } + +if (is_kfunc_trusted_args (meta) || (is_kfunc_release (meta) && reg->ref_obj_id)) + strict_type_match = true; + + reg_ref_t =btf_type_skip_modifiers (reg_btf, reg_ref_id, ®_ref_id); + reg_ref_tname =btf_name_by_offset (reg_btf, reg_ref_t->name_off); +if (!btf_struct_ids_match (&env->log, reg_btf, reg_ref_id, reg->off, meta->btf, ref_id, strict_type_match)) { +verbose (env,"kernel function %s args#%d expected pointer to %s %s but R%d has a pointer to %s %s\n" , + meta->func_name, argno,btf_type_str (ref_t), ref_tname, argno +1 , +btf_type_str (reg_ref_t), reg_ref_tname); +return -EINVAL ; + } +return 0 ; +} + +static int process_kf_arg_ptr_to_kptr_strong (struct bpf_verifier_env *env, +struct bpf_reg_state *reg, +const struct btf_type *ref_t, +const char *ref_tname, +struct bpf_kfunc_call_arg_meta *meta, +int argno) +{ +struct btf_field *kptr_field; + +/* check_func_arg_reg_off allows var_off for +* PTR_TO_MAP_VALUE, but we need fixed offset to find +* off_desc. +*/ +if (!tnum_is_const (reg->var_off)) { +verbose (env,"arg#0 must have constant offset\n" ); +return -EINVAL ; + } + + kptr_field =btf_record_find (reg->map_ptr->record, reg->off + reg->var_off.value,BPF_KPTR ); +if (!kptr_field || kptr_field->type !=BPF_KPTR_REF ) { +verbose (env,"arg#0 no referenced kptr at map value offset=%llu\n" , + reg->off + reg->var_off.value); +return -EINVAL ; + } + +if (!btf_struct_ids_match (&env->log, meta->btf, ref_t->type,0 , kptr_field->kptr.btf, + kptr_field->kptr.btf_id, true)) { +verbose (env,"kernel function %s args#%d expected pointer to %s %s\n" , + meta->func_name, argno,btf_type_str (ref_t), ref_tname); +return -EINVAL ; + } +return 0 ; +}
kernel/bpf/verifier.c static void mark_btf_func_reg_size(struct bpf_verifier_env *env, u32 regno, · line 8183
Quoted 10 lines
kernel/bpf/verifier.c +10
-0+ +static int check_kfunc_args (struct bpf_verifier_env *env,struct bpf_kfunc_call_arg_meta *meta) +{ +const char *func_name = meta->func_name, *ref_tname; +const struct btf *btf = meta->btf; +const struct btf_param *args; +u32 i, nargs; +int ret; + + args = (const struct btf_param *)(meta->func_proto +1 );
Not your change and it's fine to not block this cleanup on fixing an issue that's been in the verifier for a while, but it's unfortunate that we never built proper encapsulations for fetching params from the func proto. This is pretty brittle. A cleanup for another day...
Quoted 243 lines
kernel/bpf/verifier.c +201
-11+ nargs =btf_type_vlen (meta->func_proto); +if (nargs >MAX_BPF_FUNC_REG_ARGS ) { +verbose (env,"Function %s has %d > %d args\n" , func_name, nargs, +MAX_BPF_FUNC_REG_ARGS ); +return -EINVAL ; + } + +/* Check that BTF function arguments match actual types that the +* verifier sees. +*/ +for (i =0 ; i < nargs; i++) { +struct bpf_reg_state *regs =cur_regs (env), *reg = ®s[i +1 ]; +const struct btf_type *t, *ref_t, *resolve_ret; +enum bpf_arg_type arg_type =ARG_DONTCARE ; +u32 regno = i +1 , ref_id, type_size; +bool is_ret_buf_sz = false; +int kf_arg_type; + + t =btf_type_skip_modifiers (btf, args[i].type,NULL ); +if (btf_type_is_scalar (t)) { +if (reg->type !=SCALAR_VALUE ) { +verbose (env,"R%d is not a scalar\n" , regno); +return -EINVAL ; + } +if (is_kfunc_arg_scalar_with_name (btf, &args[i],"rdonly_buf_size" )) { + meta->r0_rdonly = true; + is_ret_buf_sz = true; + }else if (is_kfunc_arg_scalar_with_name (btf, &args[i],"rdwr_buf_size" )) { + is_ret_buf_sz = true; + } + +if (is_ret_buf_sz) { +if (meta->r0_size) { +verbose (env,"2 or more rdonly/rdwr_buf_size parameters for kfunc" ); +return -EINVAL ; + } + +if (!tnum_is_const (reg->var_off)) { +verbose (env,"R%d is not a const\n" , regno); +return -EINVAL ; + } + + meta->r0_size = reg->var_off.value; + ret =mark_chain_precision (env, regno); +if (ret) +return ret; + } +continue ; + } + +if (!btf_type_is_ptr (t)) { +verbose (env,"Unrecognized arg#%d type %s\n" , i,btf_type_str (t)); +return -EINVAL ; + } + +if (reg->ref_obj_id) { +if (is_kfunc_release (meta) && meta->ref_obj_id) { +verbose (env,"verifier internal error: more than one arg with ref_obj_id R%d %u %u\n" , + regno, reg->ref_obj_id, + meta->ref_obj_id); +return -EFAULT ; + } + meta->ref_obj_id = reg->ref_obj_id; +if (is_kfunc_release (meta)) + meta->release_regno = regno; + } + + ref_t =btf_type_skip_modifiers (btf, t->type, &ref_id); + ref_tname =btf_name_by_offset (btf, ref_t->name_off); + + kf_arg_type =get_kfunc_ptr_arg_type (env, meta, t, ref_t, ref_tname, args, i, nargs); +if (kf_arg_type <0 ) +return kf_arg_type; + +switch (kf_arg_type) { +case KF_ARG_PTR_TO_BTF_ID : +if (!is_kfunc_trusted_args (meta)) +break ; +if (!reg->ref_obj_id) { +verbose (env,"R%d must be referenced\n" , regno); +return -EINVAL ; + } + fallthrough; +case KF_ARG_PTR_TO_CTX : +/* Trusted arguments have the same offset checks as release arguments */ + arg_type |=OBJ_RELEASE ; +break ; +case KF_ARG_PTR_TO_KPTR : +case KF_ARG_PTR_TO_DYNPTR : +case KF_ARG_PTR_TO_MEM : +case KF_ARG_PTR_TO_MEM_SIZE : +/* Trusted by default */ +break ; +default : +WARN_ON_ONCE (1 ); +return -EFAULT ; + } + +if (is_kfunc_release (meta) && reg->ref_obj_id) + arg_type |=OBJ_RELEASE ; + ret =check_func_arg_reg_off (env, reg, regno, arg_type); +if (ret <0 ) +return ret; + +switch (kf_arg_type) { +case KF_ARG_PTR_TO_CTX : +if (reg->type !=PTR_TO_CTX ) { +verbose (env,"arg#%d expected pointer to ctx, but got %s\n" , i,btf_type_str (t)); +return -EINVAL ; + } +break ; +case KF_ARG_PTR_TO_KPTR : +if (reg->type !=PTR_TO_MAP_VALUE ) { +verbose (env,"arg#0 expected pointer to map value\n" ); +return -EINVAL ; + } + ret =process_kf_arg_ptr_to_kptr_strong (env, reg, ref_t, ref_tname, meta, i); +if (ret <0 ) +return ret; +break ; +case KF_ARG_PTR_TO_DYNPTR : +if (reg->type !=PTR_TO_STACK ) { +verbose (env,"arg#%d expected pointer to stack\n" , i); +return -EINVAL ; + } + +if (!is_dynptr_reg_valid_init (env, reg)) { +verbose (env,"arg#%d pointer type %s %s must be valid and initialized\n" , + i,btf_type_str (ref_t), ref_tname); +return -EINVAL ; + } + +if (!is_dynptr_type_expected (env, reg,ARG_PTR_TO_DYNPTR |DYNPTR_TYPE_LOCAL )) { +verbose (env,"arg#%d pointer type %s %s points to unsupported dynamic pointer type\n" , + i,btf_type_str (ref_t), ref_tname); +return -EINVAL ; + } +break ; +case KF_ARG_PTR_TO_BTF_ID : +/* Only base_type is checked, further checks are done here */ +if (reg->type !=PTR_TO_BTF_ID && + (!reg2btf_ids[base_type (reg->type)] ||type_flag (reg->type))) { +verbose (env,"arg#%d expected pointer to btf or socket\n" , i); +return -EINVAL ; + } + ret =process_kf_arg_ptr_to_btf_id (env, reg, ref_t, ref_tname, ref_id, meta, i); +if (ret <0 ) +return ret; +break ; +case KF_ARG_PTR_TO_MEM : + resolve_ret =btf_resolve_size (btf, ref_t, &type_size); +if (IS_ERR (resolve_ret)) { +verbose (env,"arg#%d reference type('%s %s') size cannot be determined: %ld\n" , + i,btf_type_str (ref_t), ref_tname,PTR_ERR (resolve_ret)); +return -EINVAL ; + } + ret =check_mem_reg (env, reg, regno, type_size); +if (ret <0 ) +return ret; +break ; +case KF_ARG_PTR_TO_MEM_SIZE : + ret =check_kfunc_mem_size_reg (env, ®s[regno +1 ], regno +1 ); +if (ret <0 ) { +verbose (env,"arg#%d arg#%d memory, len pair leads to invalid memory access\n" , i, i +1 ); +return ret; + } +/* Skip next '__sz' argument */ + i++; +break ; + } + } + +if (is_kfunc_release (meta) && !meta->release_regno) { +verbose (env,"release kernel function %s expects refcounted PTR_TO_BTF_ID\n" , + func_name); +return -EINVAL ; + } + +return 0 ; +} +static int check_kfunc_call (struct bpf_verifier_env *env,struct bpf_insn *insn,int *insn_idx_p) {const struct btf_type *t, *func, *func_proto, *ptr_type;struct bpf_reg_state *regs =cur_regs (env); -struct bpf_kfunc_arg_meta meta = {0 };const char *func_name, *ptr_type_name; +struct bpf_kfunc_call_arg_meta meta;u32 i, nargs, func_id, ptr_type_id;int err, insn_idx = *insn_idx_p;const struct btf_param *args;struct btf *desc_btf;u32 *kfunc_flags; -bool acq;/* skip for now, but return error when we find this in fixup_kfunc_call */ if (!insn->imm)@@ -7896,24 +8394,34 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, func_name);return -EACCES ; } -if (*kfunc_flags &KF_DESTRUCTIVE && !capable (CAP_SYS_BOOT )) { -verbose (env,"destructive kfunc calls require CAP_SYS_BOOT capabilities\n" ); + +/* Prepare kfunc call metadata */ +memset (&meta,0 ,sizeof (meta)); + meta.btf = desc_btf; + meta.func_id = func_id; + meta.kfunc_flags = *kfunc_flags; + meta.func_proto = func_proto; + meta.func_name = func_name; + +if (is_kfunc_destructive (&meta) && !capable (CAP_SYS_BOOT )) { +verbose (env,"destructive kfunc calls require CAP_SYS_BOOT capability\n" );return -EACCES ; } - acq = *kfunc_flags &KF_ACQUIRE ; - - meta.flags = *kfunc_flags; +if (is_kfunc_sleepable (&meta) && !env->prog->aux->sleepable) { +verbose (env,"program must be sleepable to call sleepable kfunc %s\n" , func_name); +return -EACCES ; + }/* Check the arguments */ - err =btf_check_kfunc_arg_match (env, desc_btf, func_id, regs, &meta); + err =check_kfunc_args (env, &meta);if (err <0 )return err;/* In case of release function, we get register number of refcounted -* PTR_TO_BTF_ID back from btf_check_kfunc_arg_match, do the release now +* PTR_TO_BTF_ID in bpf_kfunc_arg_meta, do the release now. */ -if (err) { - err =release_reference (env, regs[err].ref_obj_id); +if (meta.release_regno) { + err =release_reference (env, regs[meta.release_regno].ref_obj_id);if (err) {verbose (env,"kfunc %s#%d reference has not been acquired before\n" , func_name, func_id);
kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 8441
Quoted 8 lines
kernel/bpf/verifier.c +1
-1@@ -7927,7 +8435,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,/* Check return type */ t =btf_type_skip_modifiers (desc_btf, func_proto->type,NULL ); -if (acq && !btf_type_is_struct_ptr (desc_btf, t)) { +if (is_kfunc_acquire (&meta) && !btf_type_is_struct_ptr (meta.btf, t)) {verbose (env,"acquire kernel function does not return PTR_TO_BTF_ID\n" );return -EINVAL ; }
Can you move this up to where we check is_kfunc_desctructive(), is_kfunc_sleepable(), etc to group logically similar code together?
Quoted 78 lines
kernel/bpf/verifier.c +6
-3@@ -7966,20 +8474,23 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0 ].type =PTR_TO_BTF_ID ; regs[BPF_REG_0 ].btf_id = ptr_type_id; } -if (*kfunc_flags &KF_RET_NULL ) { +if (is_kfunc_ret_null (&meta)) { regs[BPF_REG_0 ].type |=PTR_MAYBE_NULL ;/* For mark_ptr_or_null_reg, see 93c230e3f5bd6 */ regs[BPF_REG_0 ].id = ++env->id_gen; }mark_btf_func_reg_size (env,BPF_REG_0 ,sizeof (void *)); -if (acq) { +if (is_kfunc_acquire (&meta)) {int id =acquire_reference_state (env, insn_idx);if (id <0 )return id; - regs[BPF_REG_0 ].id = id; +if (is_kfunc_ret_null (&meta)) + regs[BPF_REG_0 ].id = id; regs[BPF_REG_0 ].ref_obj_id = id; } +if (reg_may_point_to_spin_lock (®s[BPF_REG_0 ]) && !regs[BPF_REG_0 ].id) + regs[BPF_REG_0 ].id = ++env->id_gen; }/* else { add_kfunc_call() ensures it is btf_type_is_void(t) } */ nargs =btf_type_vlen (func_proto);tools/testing/selftests/bpf/prog_tests/kfunc_dynptr_param.c +1
-1@@ -22,7 +22,7 @@ static struct { "arg#0 pointer type STRUCT bpf_dynptr_kern points to unsupported dynamic pointer type", 0}, {"not_valid_dynptr" , "arg#0 pointer type STRUCT bpf_dynptr_kern must be valid and initialized", 0}, - {"not_ptr_to_stack", "arg#0 pointer type STRUCT bpf_dynptr_kern not to stack", 0}, + {"not_ptr_to_stack", "arg#0 expected pointer to stack", 0}, {"dynptr_data_null", NULL, -EBADMSG}, };tools/testing/selftests/bpf/verifier/calls.c +1
-1@@ -109,7 +109,7 @@ }, .prog_type = BPF_PROG_TYPE_SCHED_CLS, .result = REJECT, - .errstr = "arg#0 pointer type STRUCT prog_test_ref_kfunc must point", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_kfunc_call_test_acquire", 3 }, { "bpf_kfunc_call_test_release", 5 },tools/testing/selftests/bpf/verifier/ref_tracking.c +2
-2@@ -142,7 +142,7 @@ .kfunc = "bpf", .expected_attach_type = BPF_LSM_MAC, .flags = BPF_F_SLEEPABLE, - .errstr = "arg#0 pointer type STRUCT bpf_key must point to scalar, or struct with scalar", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_lookup_user_key", 2 }, { "bpf_key_put", 4 },@@ -163,7 +163,7 @@ .kfunc = "bpf", .expected_attach_type = BPF_LSM_MAC, .flags = BPF_F_SLEEPABLE, - .errstr = "arg#0 pointer type STRUCT bpf_key must point to scalar, or struct with scalar", + .errstr = "arg#0 expected pointer to btf or socket", .fixup_kfunc_btf_id = { { "bpf_lookup_system_key", 1 }, { "bpf_key_put", 3 },Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v10 11/24] bpf: Rewrite kfunc argument handling
Quoted 44 lines · David Vernet
Quoted 12 lines · Kumar Kartikeya Dwivedi
As we continue to add more features, argument types, kfunc flags, and different extensions to kfuncs, the code to verify the correctness of the kfunc prototype wrt the passed in registers has become ad-hoc and ugly to read. To make life easier, and make a very clear split between different stages of argument processing, move all the code into
verifier.cand refactor into easier to read helpers and functions.This also makes sharing code within the verifier easier with kfunc argument processing. This will be more and more useful in later patches as we are now moving to implement very core BPF helpers as kfuncs, to keep them experimental before baking into UAPI.
Quoted 4 lines
Remove all kfunc related bits now from
btf_check_func_arg_match, as users have been converted away to refactored kfunc argument handling.Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>Thanks for working on this. I'm relieved to see this work being done. I have a few comments but overall this is great. I'll take a closer look later.
Quoted 22 lines
---
[...]kernel/bpf/verifier.c +20
-0+static bool is_kfunc_arg_mem_size (const struct btf *btf, +const struct btf_param *arg, +const struct bpf_reg_state *reg) +{ +int len, sfx_len =sizeof ("__sz" ) -1 ; +const struct btf_type *t; +const char *param_name; + + t =btf_type_skip_modifiers (btf, arg->type,NULL ); +if (!btf_type_is_scalar (t) || reg->type !=SCALAR_VALUE ) +return false; + +/* In the future, this can be ported to use BTF tagging */ + param_name =btf_name_by_offset (btf, arg->name_off); +if (str_is_empty (param_name)) +return false; + len =strlen (param_name); +if (len < sfx_len) +return false; + param_name += len - sfx_len;
Quoted 9 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ if (strncmp(param_name, "__sz", sfx_len)) + return false;Oh, I thought we weren't doing this arg-type-by-name-suffix thing? I see that you're just moving it around so it's fine to move it here, but it seems to diverge from the discussions I've seen in the past. Don't want to distract from the patch but is there a plan to get rid of this at some point?
I think unless we have BTF tags in GCC, it's not possible to drop this suffix based tagging. Also not sure I remember/was part of the discussion you're referring to.
Quoted 18 lines
Quoted 18 lines
kernel/bpf/verifier.c +18
-0+ +return true; +} + +static bool is_kfunc_arg_scalar_with_name (const struct btf *btf, +const struct btf_param *arg, +const char *name) +{ +int len, target_len =strlen (name); +const char *param_name; + + param_name =btf_name_by_offset (btf, arg->name_off); +if (str_is_empty (param_name)) +return false; + len =strlen (param_name); +if (len != target_len) +return false; +if (strcmp (param_name, name))
Quoted 9 lines
kernel/bpf/verifier.c static void mark_btf_func_reg_size(struct bpf_verifier_env *env, u32 regno, · line 7939
Quoted 4 lines
kernel/bpf/verifier.c +4
-0+return false; + +return true; +}It doesn't look like this function has anything to do with the arg being a scalar, does it? Should we just rename it something like, "
kfunc_arg_has_name()"?
The scalar_with_name was a suggestion from Alexei, but I think it's fine. Since this already got applied not sure it's worth it now.
Quoted 7 lines
Quoted 7 lines
[...]
kernel/bpf/verifier.c +6
-0+static int check_kfunc_args (struct bpf_verifier_env *env,struct bpf_kfunc_call_arg_meta *meta) +{ +const char *func_name = meta->func_name, *ref_tname; +const struct btf *btf = meta->btf; +const struct btf_param *args; +u32 i, nargs;
Quoted 9 lines
kernel/bpf/verifier.c static void mark_btf_func_reg_size(struct bpf_verifier_env *env, u32 regno, · line 8183
Quoted 3 lines
kernel/bpf/verifier.c +3
-0+int ret; + + args = (const struct btf_param *)(meta->func_proto +1 );Not your change and it's fine to not block this cleanup on fixing an issue that's been in the verifier for a while, but it's unfortunate that we never built proper encapsulations for fetching params from the func proto. This is pretty brittle. A cleanup for another day...
This was just kept same as the older code, but we do have an accessor for this
case: btf_params (in include/linux/btf.h). I will roll it in with a few other clean ups (or you can beat me to it).
Quoted 10 lines
[...]
kernel/bpf/verifier.c static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, · line 8441
Quoted 5 lines
kernel/bpf/verifier.c +1
-1-if (acq && !btf_type_is_struct_ptr (desc_btf, t)) { +if (is_kfunc_acquire (&meta) && !btf_type_is_struct_ptr (meta.btf, t)) {verbose (env,"acquire kernel function does not return PTR_TO_BTF_ID\n" );return -EINVAL ; }Can you move this up to where we check
is_kfunc_desctructive(),is_kfunc_sleepable(), etc to group logically similar code together?
I think I prefer it here, unlike those checks which apply to the kfunc, this is located along with other checks for the return type, post argument processing. But let me know if you think otherwise.
Kumar Kartikeya Dwivedi
maintainerPATCH v10 12/24 bpf: Support constant scalar arguments for kfuncs
In reply to Kumar Kartikeya DwivediAllow passing known constant scalars as arguments to kfuncs that do not represent a size parameter. We use mark_chain_precision for the constant scalar argument to mark it precise. This makes the search pruning optimization of verifier more conservative for such kfunc calls, and each non-distinct argument is considered unequivalent.
We will use this support to then expose a bpf_obj_new function where it takes the local type ID of a type in program BTF, and returns a PTR_TO_BTF_ID | MEM_ALLOC to the local type, and allows programs to allocate their own objects.
Each type ID resolves to a distinct type with a possibly distinct size, hence the type ID constant matters in terms of program safety and its precision needs to be checked between old and cur states inside regsafe. The use of mark_chain_precision enables this.
--- Documentation/bpf/kfuncs.rst | 24 +++++++++++++++ kernel/bpf/verifier.c | 57 +++++++++++++++++++++++++++--------- 2 files changed, 67 insertions(+), 14 deletions(-)
Documentation/bpf/kfuncs.rst +24 -0
@@ -72,6 +72,30 @@ argument as its size. By default, without __sz annotation, the size of the type of the pointer is used. Without __sz annotation, a kfunc cannot accept a void pointer. +2.2.2 __k Annotation +-------------------- + +This annotation is only understood for scalar arguments, where it indicates that +the verifier must check the scalar argument to be a known constant, which does +not indicate a size parameter, and the value of the constant is relevant to the +safety of the program. + +An example is given below:: + + void *bpf_obj_new(u32 local_type_id__k, ...) + { + ... + } + +Here, bpf_obj_new uses local_type_id argument to find out the size of that type +ID in program's BTF and return a sized pointer to it. Each type ID will have a +distinct size, hence it is crucial to treat each such call as distinct when +values don't match during verifier state pruning checks. + +Hence, whenever a constant scalar argument is accepted by a kfunc which is not a +size parameter, and the value of the constant matters for program safety, __k +suffix should be used. + .. _BPF_kfunc_nodef: 2.3 Using an existing kernel function
kernel/bpf/verifier.c +43 -14
86 lines not on this page · raw message
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 13/24 bpf: Introduce bpf_obj_new
In reply to Kumar Kartikeya DwivediIntroduce type safe memory allocator bpf_obj_new for BPF programs. The kernel side kfunc is named bpf_obj_new_impl, as passing hidden arguments to kfuncs still requires having them in prototype, unlike BPF helpers which always take 5 arguments and have them checked using bpf_func_proto in verifier, ignoring unset argument types.
Introduce __ign suffix to ignore a specific kfunc argument during type checks, then use this to introduce support for passing type metadata to the bpf_obj_new_impl kfunc.
The user passes BTF ID of the type it wants to allocates in program BTF, the verifier then rewrites the first argument as the size of this type, after performing some sanity checks (to ensure it exists and it is a struct type).
The second argument is also fixed up and passed by the verifier. This is the btf_struct_meta for the type being allocated. It would be needed mostly for the offset array which is required for zero initializing special fields while leaving the rest of storage in unitialized state.
It would also be needed in the next patch to perform proper destruction of the object's special fields.
Under the hood, bpf_obj_new will call bpf_mem_alloc and bpf_mem_free, using the any context BPF memory allocator introduced recently. To this end, a global instance of the BPF memory allocator is initialized on boot to be used for this purpose. This 'bpf_global_ma' serves all allocations for bpf_obj_new. In the future, bpf_obj_new variants will allow specifying a custom allocator.
Note that now that bpf_obj_new can be used to allocate objects that can be linked to BPF linked list (when future linked list helpers are available), we need to also free the elements using bpf_mem_free. However, since the draining of elements is done outside the bpf_spin_lock, we need to do migrate_disable around the call since bpf_list_head_free can be called from map free path where migration is enabled. Otherwise, when called from BPF programs migration is already disabled.
A convenience macro is included in the bpf_experimental.h header to hide over the ugly details of the implementation, leading to user code looking similar to a language level extension which allocates and constructs fields of a user type.
struct bar {
struct bpf_list_node node;
};struct foo {
struct bpf_spin_lock lock;
struct bpf_list_head head __contains(bar, node);
};void prog(void) {
struct foo *f;
f = bpf_obj_new(typeof(*f));
if (!f)
return;
...
}A key piece of this story is still missing, i.e. the free function, which will come in the next patch.
--- include/linux/bpf.h | 21 ++-- include/linux/bpf_verifier.h | 2 + kernel/bpf/core.c | 16 +++ kernel/bpf/helpers.c | 47 ++++++-- kernel/bpf/verifier.c | 102 ++++++++++++++++-- .../testing/selftests/bpf/bpf_experimental.h | 25 +++++ 6 files changed, 190 insertions(+), 23 deletions(-) create mode 100644 tools/testing/selftests/bpf/bpf_experimental.h
include/linux/bpf.h +13 -8
@@ -54,6 +54,8 @@ struct cgroup;extern struct idr btf_idr;extern spinlock_t btf_idr_lock;extern struct kobject *btf_kobj; +extern struct bpf_mem_alloc bpf_global_ma; +extern bool bpf_global_ma_set;typedef u64 (*bpf_callback_t )(u64 ,u64 ,u64 ,u64 ,u64 ); typedef int (*bpf_iter_init_seq_priv_t)(void *private_data,
@@ -334,16 +336,19 @@ static inline bool btf_record_has_field(const struct btf_record *rec, enum btf_freturn rec->field_mask & type; } -static inline void check_and_init_map_value (struct bpf_map *map,void *dst) +static inline void bpf_obj_init (const struct btf_field_offs *foffs,void *obj) { -if (!IS_ERR_OR_NULL (map->record)) { -struct btf_field *fields = map->record->fields; -u32 cnt = map->record->cnt; -int i; +int i; -for (i =0 ; i < cnt; i++) -memset (dst + fields[i].offset,0 ,btf_field_type_size (fields[i].type)); - } +if (!foffs) +return ; +for (i =0 ; i < foffs->cnt; i++) +memset (obj + foffs->field_off[i],0 , foffs->field_sz[i]); +} + +static inline void check_and_init_map_value (struct bpf_map *map,void *dst) +{ +bpf_obj_init (map->field_offs, dst); }/* memcpy that is used with 8-byte aligned pointers, power-of-8 size and
include/linux/bpf_verifier.h +2 -0
@@ -433,6 +433,8 @@ struct bpf_insn_aux_data {*/ struct bpf_loop_inline_state loop_inline_state; }; +u64 obj_new_size;/* remember the size of type passed to bpf_obj_new to rewrite R1 */ +struct btf_struct_meta *kptr_struct_meta;u64 map_key_state;/* constant (32 bit) key tracking for maps */ int ctx_field_size;/* the ctx field size for load insn, maybe 0 */ u32 seen;/* this insn was processed by the verifier at env->pass_cnt */
kernel/bpf/core.c +16 -0
@@ -34,6 +34,7 @@#include <linux/log2.h> #include <linux/bpf_verifier.h> #include <linux/nodemask.h> +#include <linux/bpf_mem_alloc.h> #include <asm/barrier.h> #include <asm/unaligned.h>
@@ -60,6 +61,9 @@#define CTX regs[BPF_REG_CTX]#define IMM insn->imm +struct bpf_mem_alloc bpf_global_ma; +bool bpf_global_ma_set; +/* No hurry in this branch * * Exported for the bpf jit load helper.
@@ -2746,6 +2750,18 @@ int __weak bpf_arch_text_invalidate(void *dst, size_t len)return -ENOTSUPP ; } +#ifdef CONFIG_BPF_SYSCALL +static int __init bpf_global_ma_init (void ) +{ +int ret; + + ret =bpf_mem_alloc_init (&bpf_global_ma,0 , false); + bpf_global_ma_set = !ret; +return ret; +} +late_initcall (bpf_global_ma_init); +#endif +DEFINE_STATIC_KEY_FALSE (bpf_stats_enabled_key);EXPORT_SYMBOL (bpf_stats_enabled_key);
kernel/bpf/helpers.c +40 -7
71 lines not on this page · raw message
kernel/bpf/verifier.c +94 -8
160 lines not on this page · raw message
tools/testing/selftests/bpf/bpf_experimental.h new file +25 -0
@@ -0,0 +1,25 @@ +#ifndef __BPF_EXPERIMENTAL__ +#define __BPF_EXPERIMENTAL__ + +#include <vmlinux.h> +#include <bpf/bpf_tracing.h> +#include <bpf/bpf_helpers.h> +#include <bpf/bpf_core_read.h> + +/* Description +* Allocates an object of the type represented by 'local_type_id' in +* program BTF. User may use the bpf_core_type_id_local macro to pass the +* type ID of a struct in program BTF. +* +* The 'local_type_id' parameter must be a known constant. +* The 'meta' parameter is a hidden argument that is ignored. +* Returns +* A pointer to an object of the type corresponding to the passed in +* 'local_type_id', or NULL on failure. +*/ +extern void *bpf_obj_new_impl (__u64 local_type_id,void *meta) __ksym; + +/* Convenience macro to wrap over bpf_obj_new_impl */ +#define bpf_obj_new (type) ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) + +#endif
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 14/24 bpf: Introduce bpf_obj_drop
In reply to Kumar Kartikeya DwivediIntroduce bpf_obj_drop, which is the kfunc used to free allocated objects (allocated using bpf_obj_new). Pairing with bpf_obj_new, it implicitly destructs the fields part of object automatically without user intervention.
Just like the previous patch, btf_struct_meta that is needed to free up the special fields is passed as a hidden argument to the kfunc.
For the user, a convenience macro hides over the kernel side kfunc which is named bpf_obj_drop_impl.
Continuing the previous example:
void prog(void) {
struct foo *f;
f = bpf_obj_new(typeof(*f));
if (!f)
return;
bpf_obj_drop(f);
}--- kernel/bpf/helpers.c | 11 ++++ kernel/bpf/verifier.c | 66 +++++++++++++++---- .../testing/selftests/bpf/bpf_experimental.h | 13 ++++ 3 files changed, 79 insertions(+), 11 deletions(-)
kernel/bpf/helpers.c +11 -0
@@ -1770,6 +1770,16 @@ void *bpf_obj_new_impl(u64 local_type_id__k, void *meta__ign)return p; } +void bpf_obj_drop_impl (void *p__alloc,void *meta__ign) +{ +struct btf_struct_meta *meta = meta__ign; +void *p = p__alloc; + +if (meta) +bpf_obj_free_fields (meta->record, p); +bpf_mem_free (&bpf_global_ma, p); +} +__diag_pop ();BTF_SET8_START (generic_btf_ids)
@@ -1777,6 +1787,7 @@ BTF_SET8_START(generic_btf_ids) BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE)#endif BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_obj_drop_impl, KF_RELEASE)BTF_SET8_END (generic_btf_ids) static const struct btf_kfunc_id_set generic_kfunc_set = {
kernel/bpf/verifier.c +55 -11
126 lines not on this page · raw message
tools/testing/selftests/bpf/bpf_experimental.h +13 -0
@@ -22,4 +22,17 @@ extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym;/* Convenience macro to wrap over bpf_obj_new_impl */ #define bpf_obj_new (type) ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) +/* Description +* Free an allocated object. All fields of the object that require +* destruction will be destructed before the storage is freed. +* +* The 'meta' parameter is a hidden argument that is ignored. +* Returns +* Void. +*/ +extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym; + +/* Convenience macro to wrap over bpf_obj_drop_impl */ +#define bpf_obj_drop (kptr) bpf_obj_drop_impl(kptr, NULL) +#endif
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 15/24 bpf: Permit NULL checking pointer with non-zero fixed offset
In reply to Kumar Kartikeya DwivediPointer increment on seeing PTR_MAYBE_NULL is already protected against, hence make an exception for PTR_TO_BTF_ID | MEM_ALLOC while still keeping the warning for other unintended cases that might creep in.
bpf_list_pop_{front,_back} helpers planned to be introduced in next commit will return a MEM_ALLOC register with incremented offset pointing to bpf_list_node field. The user is supposed to then obtain the pointer to the entry using container_of after NULL checking it. The current restrictions trigger a warning when doing the NULL checking. Revisiting the reason, it is meant as an assertion which seems to actually work and catch the bad case.
Hence, under no other circumstances can reg->off be non-zero for a register that has the PTR_MAYBE_NULL type flag set.
--- kernel/bpf/verifier.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-)
kernel/bpf/verifier.c +12 -9
@@ -10791,16 +10791,19 @@ static void mark_ptr_or_null_reg(struct bpf_func_state *state, {if (type_may_be_null (reg->type) && reg->id == id && !WARN_ON_ONCE (!reg->id)) { -if (WARN_ON_ONCE (reg->smin_value || reg->smax_value || - !tnum_equals_const (reg->var_off,0 ) || - reg->off)) { -/* Old offset (both fixed and variable parts) should -* have been known-zero, because we don't allow pointer -* arithmetic on pointers that might be NULL. If we -* see this happening, don't convert the register. -*/ +/* Old offset (both fixed and variable parts) should have been +* known-zero, because we don't allow pointer arithmetic on +* pointers that might be NULL. If we see this happening, don't +* convert the register. +* +* But in some cases, some helpers that return local kptrs +* advance offset for the returned pointer. In those cases, it +* is fine to expect to see reg->off. +*/ +if (WARN_ON_ONCE (reg->smin_value || reg->smax_value || !tnum_equals_const (reg->var_off,0 ))) +return ; +if (reg->type != (PTR_TO_BTF_ID |MEM_ALLOC |PTR_MAYBE_NULL ) &&WARN_ON_ONCE (reg->off))return ; - }if (is_null) { reg->type =SCALAR_VALUE ;/* We don't need id and ref_obj_id from this point
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 16/24 bpf: Introduce single ownership BPF linked list API
In reply to Kumar Kartikeya DwivediAdd a linked list API for use in BPF programs, where it expects protection from the bpf_spin_lock in the same allocation as the bpf_list_head. For now, only one bpf_spin_lock can be present hence that is assumed to be the one protecting the bpf_list_head.
The following functions are added to kick things off:
// Add node to beginning of list
void bpf_list_push_front(struct bpf_list_head *head, struct bpf_list_node *node);
// Add node to end of list
void bpf_list_push_back(struct bpf_list_head *head, struct bpf_list_node *node);
// Remove node at beginning of list and return it
struct bpf_list_node *bpf_list_pop_front(struct bpf_list_head *head);
// Remove node at end of list and return it
struct bpf_list_node *bpf_list_pop_back(struct bpf_list_head *head);
The lock protecting the bpf_list_head needs to be taken for all operations. The verifier ensures that the lock that needs to be taken is always held, and only the correct lock is taken for these operations. These checks are made statically by relying on the reg->id preserved for registers pointing into regions having both bpf_spin_lock and the objects protected by it. The comment over check_reg_allocation_locked in this change describes the logic in detail.
Note that bpf_list_push_front and bpf_list_push_back are meant to consume the object containing the node in the 1st argument, however that specific mechanism is intended to not release the ref_obj_id directly until the bpf_spin_unlock is called. In this commit, nothing is done, but the next commit will be introducing logic to handle this case, so it has been left as is for now.
bpf_list_pop_front and bpf_list_pop_back delete the first or last item of the list respectively, and return pointer to the element at the list_node offset. The user can then use container_of style macro to get the actual entry type. The verifier however statically knows the actual type, so the safety properties are still preserved.
With these additions, programs can now manage their own linked lists and store their objects in them.
--- kernel/bpf/helpers.c | 55 +++- kernel/bpf/verifier.c | 275 +++++++++++++++++- .../testing/selftests/bpf/bpf_experimental.h | 28 ++ 3 files changed, 349 insertions(+), 9 deletions(-)
kernel/bpf/helpers.c +54 -1
@@ -1780,6 +1780,50 @@ void bpf_obj_drop_impl(void *p__alloc, void *meta__ign)bpf_mem_free (&bpf_global_ma, p); } +static void __bpf_list_add (struct bpf_list_node *node,struct bpf_list_head *head,bool tail) +{ +struct list_head *n = (void *)node, *h = (void *)head; + +if (unlikely (!h->next)) +INIT_LIST_HEAD (h); +if (unlikely (!n->next)) +INIT_LIST_HEAD (n); + tail ?list_add_tail (n, h) :list_add (n, h); +} + +void bpf_list_push_front (struct bpf_list_head *head,struct bpf_list_node *node) +{ +return __bpf_list_add (node, head, false); +} + +void bpf_list_push_back (struct bpf_list_head *head,struct bpf_list_node *node) +{ +return __bpf_list_add (node, head, true); +} + +static struct bpf_list_node *__bpf_list_del (struct bpf_list_head *head,bool tail) +{ +struct list_head *n, *h = (void *)head; + +if (unlikely (!h->next)) +INIT_LIST_HEAD (h); +if (list_empty (h)) +return NULL ; + n = tail ? h->prev : h->next; +list_del_init (n); +return (struct bpf_list_node *)n; +} + +struct bpf_list_node *bpf_list_pop_front (struct bpf_list_head *head) +{ +return __bpf_list_del (head, false); +} + +struct bpf_list_node *bpf_list_pop_back (struct bpf_list_head *head) +{ +return __bpf_list_del (head, true); +} +__diag_pop ();BTF_SET8_START (generic_btf_ids)
@@ -1788,6 +1832,10 @@ BTF_ID_FLAGS(func, crash_kexec, KF_DESTRUCTIVE)#endif BTF_ID_FLAGS(func, bpf_obj_new_impl, KF_ACQUIRE | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_obj_drop_impl, KF_RELEASE) +BTF_ID_FLAGS(func, bpf_list_push_front) +BTF_ID_FLAGS(func, bpf_list_push_back) +BTF_ID_FLAGS(func, bpf_list_pop_front, KF_ACQUIRE | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_list_pop_back, KF_ACQUIRE | KF_RET_NULL)BTF_SET8_END (generic_btf_ids) static const struct btf_kfunc_id_set generic_kfunc_set = {
@@ -1797,7 +1845,12 @@ static const struct btf_kfunc_id_set generic_kfunc_set = {static int __init kfunc_init (void ) { -return register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &generic_kfunc_set); +int ret; + + ret =register_btf_kfunc_id_set (BPF_PROG_TYPE_TRACING , &generic_kfunc_set); +if (ret) +return ret; +return register_btf_kfunc_id_set (BPF_PROG_TYPE_SCHED_CLS , &generic_kfunc_set); }late_initcall (kfunc_init);
kernel/bpf/verifier.c +267 -8
356 lines not on this page · raw message
tools/testing/selftests/bpf/bpf_experimental.h +28 -0
@@ -35,4 +35,32 @@ extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym;/* Convenience macro to wrap over bpf_obj_drop_impl */ #define bpf_obj_drop (kptr) bpf_obj_drop_impl(kptr, NULL) +/* Description +* Add a new entry to the beginning of the BPF linked list. +* Returns +* Void. +*/ +extern void bpf_list_push_front(struct bpf_list_head *head, struct bpf_list_node *node) __ksym; + +/* Description +* Add a new entry to the end of the BPF linked list. +* Returns +* Void. +*/ +extern void bpf_list_push_back(struct bpf_list_head *head, struct bpf_list_node *node) __ksym; + +/* Description +* Remove the entry at the beginning of the BPF linked list. +* Returns +* Pointer to bpf_list_node of deleted entry, or NULL if list is empty. +*/ +extern struct bpf_list_node *bpf_list_pop_front (struct bpf_list_head *head) __ksym; + +/* Description +* Remove the entry at the end of the BPF linked list. +* Returns +* Pointer to bpf_list_node of deleted entry, or NULL if list is empty. +*/ +extern struct bpf_list_node *bpf_list_pop_back (struct bpf_list_head *head) __ksym; +#endif
Signature · 1 line
2.38.1
Re: [PATCH bpf-next v10 16/24] bpf: Introduce single ownership BPF linked list API
Hi Kumar,
Quoted 40 lines · Kumar Kartikeya Dwivedi
Add a linked list API for use in BPF programs, where it expects protection from the
bpf_spin_lockin the same allocation as thebpf_list_head. For now, only onebpf_spin_lockcan be present hence that is assumed to be the one protecting thebpf_list_head.The following functions are added to kick things off:
// Add node to beginning of list
voidbpf_list_push_front(struct bpf_list_head *head, struct bpf_list_node *node);// Add node to end of list
voidbpf_list_push_back(struct bpf_list_head *head, struct bpf_list_node *node);// Remove node at beginning of list and return it
structbpf_list_node*bpf_list_pop_front(structbpf_list_head*head);// Remove node at end of list and return it
structbpf_list_node*bpf_list_pop_back(structbpf_list_head*head);The lock protecting the
bpf_list_headneeds to be taken for all operations. The verifier ensures that the lock that needs to be taken is always held, and only the correct lock is taken for these operations. These checks are made statically by relying on thereg->idpreserved for registers pointing into regions having bothbpf_spin_lockand the objects protected by it. The comment overcheck_reg_allocation_lockedin this change describes the logic in detail.Note that
bpf_list_push_frontandbpf_list_push_backare meant to consume the object containing the node in the 1st argument, however that specific mechanism is intended to not release theref_obj_iddirectly until thebpf_spin_unlockis called. In this commit, nothing is done, but the next commit will be introducing logic to handle this case, so it has been left as is for now.
bpf_list_pop_frontandbpf_list_pop_backdelete the first or last item of the list respectively, and return pointer to the element at thelist_nodeoffset. The user can then usecontainer_ofstyle macro to get the actual entry type. The verifier however statically knows the actual type, so the safety properties are still preserved.
Quoted 4 lines
With these additions, programs can now manage their own linked lists and store their objects in them.
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
Apologies if this has already been reported or discussed somewhere else; I did a search of the mailing list and did find a sparse report that seems to be complaining about this same issue but it does not look like you were cc'd on that [1].
This commit is now in -next as commit 8cab76ec6349 ("bpf: Introduce single ownership BPF linked list API") and I just bisected a new series of warnings I see with clang as starting with this change; Yonghong's recent "bpf: Implement two type cast kfuncs" [2] added a couple more but they start here. When CONFIG_DEBUG_INFO_BTF is disabled (as is the case with allmodconfig), I see the following clang warnings at this change:
../kernel/bpf/verifier.c:8340:19: error: array index 5 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
btf_id == special_kfunc_list[KF_bpf_list_pop_back];
^ ~~~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8113:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
../kernel/bpf/verifier.c:8819:24: error: array index 5 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
meta.func_id == special_kfunc_list[KF_bpf_list_pop_back]) {
^ ~~~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8113:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
2 errors generated.At ToT -next (next-20221121), I see:
../kernel/bpf/verifier.c:8196:23: error: array index 6 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
if (meta->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx])
^ ~~~~~~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8174:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
../kernel/bpf/verifier.c:8443:19: error: array index 5 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
btf_id == special_kfunc_list[KF_bpf_list_pop_back];
^ ~~~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8174:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
../kernel/bpf/verifier.c:8686:25: error: array index 6 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
if (meta->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx]) {
^ ~~~~~~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8174:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
../kernel/bpf/verifier.c:8938:24: error: array index 5 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
meta.func_id == special_kfunc_list[KF_bpf_list_pop_back]) {
^ ~~~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8174:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
../kernel/bpf/verifier.c:8946:31: error: array index 6 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
} else if (meta.func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx]) {
^ ~~~~~~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8174:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
../kernel/bpf/verifier.c:8951:31: error: array index 7 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
} else if (meta.func_id == special_kfunc_list[KF_bpf_rdonly_cast]) {
^ ~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8174:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
../kernel/bpf/verifier.c:15216:30: error: array index 6 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
} else if (desc->func_id == special_kfunc_list[KF_bpf_cast_to_kern_ctx] ||
^ ~~~~~~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8174:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
../kernel/bpf/verifier.c:15217:23: error: array index 7 is past the end of the array (that has type 'u32[5]' (aka 'unsigned int[5]')) [-Werror,-Warray-bounds]
desc->func_id == special_kfunc_list[KF_bpf_rdonly_cast]) {
^ ~~~~~~~~~~~~~~~~~~
../kernel/bpf/verifier.c:8174:1: note: array 'special_kfunc_list' declared here
BTF_ID_LIST(special_kfunc_list)
^
../include/linux/btf_ids.h:207:27: note: expanded from macro 'BTF_ID_LIST'
#define BTF_ID_LIST(name) static u32 __maybe_unused name[5];
^
8 errors generated.Somewhat surprisingly, I do not see these with GCC 11 (-Warray-bounds is explicitly disabled for GCC 12) but I do not see how they could not be legitimate, unless all this code gets optimized away when CONFIG_DEBUG_INFO_BTF is disabled, which I could be missing. Any reason that the fix is not something like:
include/linux/btf_ids.h +1 -1
@@ -204,7 +204,7 @@ extern struct btf_id_set8 name;#else -#define BTF_ID_LIST (name) static u32 __maybe_unused name[5]; +#define BTF_ID_LIST (name) static u32 __maybe_unused name[8];#define BTF_ID (prefix, name)#define BTF_ID_FLAGS (prefix, name, ...)#define BTF_ID_UNUSED
I suppose that there should probably be some sort of assertion to catch this discrepancy in the future.
Cheers,
Nathan
[1]: https://lore.kernel.org/[email protected]/
[2]: https://lore.kernel.org/[email protected]/
# bad: [e4cd8d3ff7f9efeb97330e5e9b99eeb2a68f5cf9] Add linux-next specific files for 20221121
# good: [894909f95aa1473f49f767dcd5750ba152b85e13] Merge tag 'x86_urgent_for_v6.1_rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect start 'e4cd8d3ff7f9efeb97330e5e9b99eeb2a68f5cf9' '894909f95aa1473f49f767dcd5750ba152b85e13'
# bad: [42c3120ba7547d18e7788707e67ff01dc220f09b] Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
git bisect bad 42c3120ba7547d18e7788707e67ff01dc220f09b
# good: [ceda86b061181705d751fce5971da7da32aa9afc] Merge branch 'master' of https://github.com/Paragon-Software-Group/linux-ntfs3.git
git bisect good ceda86b061181705d751fce5971da7da32aa9afc
# good: [c609d739947894d7370eae4cf04eb2c49e910bcf] Merge tag 'wireless-next-2022-11-18' of git://git.kernel.org/pub/scm/linux/kernel/git/wireless/wireless-next
git bisect good c609d739947894d7370eae4cf04eb2c49e910bcf
# good: [4b39aeeaec47f5cea7f6ab76fa2da428e4bf5c7e] Merge branch 'master' of git://linuxtv.org/media_tree.git
git bisect good 4b39aeeaec47f5cea7f6ab76fa2da428e4bf5c7e
# good: [4c071062e639f74d95a229b5efc22cb1c9ed3d49] Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git
git bisect good 4c071062e639f74d95a229b5efc22cb1c9ed3d49
# bad: [8d72bd74ae946456865574a0390979af5aa74bcb] Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/bluetooth/bluetooth-next.git
git bisect bad 8d72bd74ae946456865574a0390979af5aa74bcb
# bad: [e181d3f143f7957a73c8365829249d8084602606] bpf: Disallow bpf_obj_new_impl call when bpf_mem_alloc_init fails
git bisect bad e181d3f143f7957a73c8365829249d8084602606
# good: [f73e601aafb2ad9f2b2012b969f86f4a41141a7d] bpf: Populate field_offs for inner_map_meta
git bisect good f73e601aafb2ad9f2b2012b969f86f4a41141a7d
# bad: [64069c72b4b8e44f6876249cc8f2e2ee4d209a93] selftests/bpf: Add __contains macro to bpf_experimental.h
git bisect bad 64069c72b4b8e44f6876249cc8f2e2ee4d209a93
# good: [00b85860feb809852af9a88cb4ca8766d7dff6a3] bpf: Rewrite kfunc argument handling
git bisect good 00b85860feb809852af9a88cb4ca8766d7dff6a3
# good: [df57f38a0d081f05ec48ea5aa7ca0564918ed915] bpf: Permit NULL checking pointer with non-zero fixed offset
git bisect good df57f38a0d081f05ec48ea5aa7ca0564918ed915
# bad: [534e86bc6c66e1e0c798a1c0a6a680bb231c08db] bpf: Add 'release on unlock' logic for bpf_list_push_{front,back}
git bisect bad 534e86bc6c66e1e0c798a1c0a6a680bb231c08db
# bad: [8cab76ec634995e59a8b6346bf8b835ab7fad3a3] bpf: Introduce single ownership BPF linked list API
git bisect bad 8cab76ec634995e59a8b6346bf8b835ab7fad3a3
# first bad commit: [8cab76ec634995e59a8b6346bf8b835ab7fad3a3] bpf: Introduce single ownership BPF linked list API
Kumar Kartikeya Dwivedi
maintainerPATCH v10 17/24 bpf: Add 'release on unlock' logic for bpf_list_push_{front,back}
In reply to Kumar Kartikeya DwivediThis commit implements the delayed release logic for bpf_list_push_front and bpf_list_push_back.
Once a node has been added to the list, it's pointer changes to PTR_UNTRUSTED. However, it is only released once the lock protecting the list is unlocked. For such PTR_TO_BTF_ID | MEM_ALLOC with PTR_UNTRUSTED set but an active ref_obj_id, it is still permitted to read them as long as the lock is held. Writing to them is not allowed.
This allows having read access to push items we no longer own until we release the lock guarding the list, allowing a little more flexibility when working with these APIs.
Note that enabling write support has fairly tricky interactions with what happens inside the critical section. Just as an example, currently, bpf_obj_drop is not permitted, but if it were, being able to write to the PTR_UNTRUSTED pointer while the object gets released back to the memory allocator would violate safety properties we wish to guarantee (i.e. not crashing the kernel). The memory could be reused for a different type in the BPF program or even in the kernel as it gets eventually kfree'd.
Not enabling bpf_obj_drop inside the critical section would appear to prevent all of the above, but that is more of an artifical limitation right now. Since the write support is tangled with how we handle potential aliasing of nodes inside the critical section that may or may not be part of the list anymore, it has been deferred to a future patch.
--- include/linux/bpf_verifier.h | 5 ++++ kernel/bpf/verifier.c | 58 +++++++++++++++++++++++++++++++++++- 2 files changed, 62 insertions(+), 1 deletion(-)
include/linux/bpf_verifier.h +5 -0
@@ -223,6 +223,11 @@ struct bpf_reference_state {* exiting a callback function. */ int callback_ref; +/* Mark the reference state to release the registers sharing the same id +* on bpf_spin_unlock (for nodes that we will lose ownership to but are +* safe to access inside the critical section). +*/ +bool release_on_unlock; };/* state of the program:
kernel/bpf/verifier.c +57 -1
83 lines not on this page · raw message
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 18/24 bpf: Add comments for map BTF matching requirement for bpf_list_head
In reply to Kumar Kartikeya DwivediThe old behavior of bpf_map_meta_equal was that it compared timer_off to be equal (but not spin_lock_off, because that was not allowed), and did memcmp of kptr_off_tab.
Now, we memcmp the btf_record of two bpf_map structs, which has all fields.
We preserve backwards compat as we kzalloc the array, so if only spin lock and timer exist in map, we only compare offset while the rest of unused members in the btf_field struct are zeroed out.
In case of kptr, btf and everything else is of vmlinux or module, so as long type is same it will match, since kernel btf, module, dtor pointer will be same across maps.
Now with list_head in the mix, things are a bit complicated. We implicitly add a requirement that both BTFs are same, because struct btf_field_list_head has btf and value_rec members.
We obviously shouldn't force BTFs to be equal by default, as that breaks backwards compatibility.
Currently it is only implicitly required due to list_head matching struct btf and value_rec member. value_rec points back into a btf_record stashed in the map BTF (btf member of btf_field_list_head). So that pointer and btf member has to match exactly.
Document all these subtle details so that things don't break in the future when touching this code.
--- kernel/bpf/btf.c | 3 +++ kernel/bpf/map_in_map.c | 5 +++++ kernel/bpf/syscall.c | 14 ++++++++++++++ 3 files changed, 22 insertions(+)
kernel/bpf/btf.c +3 -0
@@ -3648,6 +3648,9 @@ struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_typereturn NULL ; cnt = ret; +/* This needs to be kzalloc to zero out padding and unused fields, see +* comment in btf_record_equal. +*/ rec = kzalloc(offsetof(struct btf_record, fields[cnt]), GFP_KERNEL | __GFP_NOWARN);if (!rec)return ERR_PTR (-ENOMEM );
kernel/bpf/map_in_map.c +5 -0
@@ -68,6 +68,11 @@ struct bpf_map *bpf_map_meta_alloc(int inner_map_ufd) } inner_map_meta->field_offs = field_offs; } +/* Note: We must use the same BTF, as we also used btf_record_dup above +* which relies on BTF being same for both maps, as some members like +* record->fields.list_head have pointers like value_rec pointing into +* inner_map->btf. +*/ if (inner_map->btf) {btf_get (inner_map->btf); inner_map_meta->btf = inner_map->btf;
kernel/bpf/syscall.c +14 -0
@@ -611,6 +611,20 @@ bool btf_record_equal(const struct btf_record *rec_a, const struct btf_record *rif (rec_a->cnt != rec_b->cnt)return false; size = offsetof(struct btf_record, fields[rec_a->cnt]); +/* btf_parse_fields uses kzalloc to allocate a btf_record, so unused +* members are zeroed out. So memcmp is safe to do without worrying +* about padding/unused fields. +* +* While spin_lock, timer, and kptr have no relation to map BTF, +* list_head metadata is specific to map BTF, the btf and value_rec +* members in particular. btf is the map BTF, while value_rec points to +* btf_record in that map BTF. +* +* So while by default, we don't rely on the map BTF (which the records +* were parsed from) matching for both records, which is not backwards +* compatible, in case list_head is part of it, we implicitly rely on +* that by way of depending on memcmp succeeding for it. +*/ return !memcmp (rec_a, rec_b, size); }
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 19/24 selftests/bpf: Add __contains macro to bpf_experimental.h
In reply to Kumar Kartikeya DwivediAdd user facing __contains macro which provides a convenient wrapper over the verbose kernel specific BTF declaration tag required to annotate BPF list head structs in user types.
--- tools/testing/selftests/bpf/bpf_experimental.h | 2 ++ 1 file changed, 2 insertions(+)
tools/testing/selftests/bpf/bpf_experimental.h +2 -0
@@ -6,6 +6,8 @@#include <bpf/bpf_helpers.h> #include <bpf/bpf_core_read.h> +#define __contains (name, node) __attribute__((btf_decl_tag("contains:" #name ":" #node))) +/* Description * Allocates an object of the type represented by 'local_type_id' in * program BTF. User may use the bpf_core_type_id_local macro to pass the
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 20/24 selftests/bpf: Update spinlock selftest
In reply to Kumar Kartikeya DwivediMake updates in preparation for adding more test cases to this selftest:
- Convert from CHECK_ to ASSERT macros.
- Use BPF skeleton
- Fix typo sping -> spin
- Rename spinlock.c -> spin_lock.c
--- .../selftests/bpf/prog_tests/spin_lock.c | 49 +++++++++++++++++++ .../selftests/bpf/prog_tests/spinlock.c | 45 ----------------- .../selftests/bpf/progs/test_spin_lock.c | 4 +- 3 files changed, 51 insertions(+), 47 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/spin_lock.c delete mode 100644 tools/testing/selftests/bpf/prog_tests/spinlock.c
tools/testing/selftests/bpf/prog_tests/spin_lock.c new file +49 -0
@@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> +#include <network_helpers.h> + +#include "test_spin_lock.skel.h" + +static void *spin_lock_thread (void *arg) +{ +int err, prog_fd = *(u32 *) arg; +LIBBPF_OPTS (bpf_test_run_opts, topts, + .data_in = &pkt_v4, + .data_size_in = sizeof(pkt_v4), + .repeat = 10000, + ); + + err =bpf_prog_test_run_opts (prog_fd, &topts); +ASSERT_OK (err,"test_run" ); +ASSERT_OK (topts.retval,"test_run retval" ); +pthread_exit (arg); +} + +void test_spinlock (void ) +{ +struct test_spin_lock *skel; +pthread_t thread_id[4 ]; +int prog_fd, i; +void *ret; + + skel =test_spin_lock__open_and_load (); +if (!ASSERT_OK_PTR (skel,"test_spin_lock__open_and_load" )) +return ; + prog_fd =bpf_program__fd (skel->progs.bpf_spin_lock_test); +for (i =0 ; i <4 ; i++) { +int err; + + err =pthread_create (&thread_id[i],NULL , &spin_lock_thread, &prog_fd); +if (!ASSERT_OK (err,"pthread_create" )) + goto end; + } + +for (i =0 ; i <4 ; i++) { +if (!ASSERT_OK (pthread_join (thread_id[i], &ret),"pthread_join" )) + goto end; +if (!ASSERT_EQ (ret, &prog_fd,"ret == prog_fd" )) + goto end; + } +end: +test_spin_lock__destroy (skel); +}
tools/testing/selftests/bpf/prog_tests/spinlock.c deleted +0 -45
45 lines not on this page · raw message
tools/testing/selftests/bpf/progs/test_spin_lock.c +2 -2
@@ -45,8 +45,8 @@ struct {#define CREDIT_PER_NS (delta, rate) (((delta) * rate) >> 20) -SEC("tc") -int bpf_sping_lock_test (struct __sk_buff *skb) +SEC("cgroup_skb/ingress") +int bpf_spin_lock_test (struct __sk_buff *skb) {volatile int credit =0 , max_credit =100 , pkt_len =64 ;struct hmap_elem zero = {}, *val;
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 21/24 selftests/bpf: Add failure test cases for spin lock pairing
In reply to Kumar Kartikeya DwivediFirst, ensure that whenever a bpf_spin_lock is present in an allocation, the reg->id is preserved. This won't be true for global variables however, since they have a single map value per map, hence the verifier harcodes it to 0 (so that multiple pseudo ldimm64 insns can yield the same lock object per map at a given offset).
Next, add test cases for all possible combinations (kptr, global, map value, inner map value). Since we lifted restriction on locking in inner maps, also add test cases for them. Currently, each lookup into an inner map gets a fresh reg->id, so even if the reg->map_ptr is same, they will be treated as separate allocations and the incorrect unlock pairing will be rejected.
--- .../selftests/bpf/prog_tests/spin_lock.c | 89 +++++++- .../selftests/bpf/progs/test_spin_lock_fail.c | 204 ++++++++++++++++++ 2 files changed, 292 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/progs/test_spin_lock_fail.c
tools/testing/selftests/bpf/prog_tests/spin_lock.c +88 -1
@@ -3,6 +3,79 @@#include <network_helpers.h> #include "test_spin_lock.skel.h" +#include "test_spin_lock_fail.skel.h" + +static char log_buf[1024 *1024 ]; + +static struct { +const char *prog_name; +const char *err_msg; +} spin_lock_fail_tests[] = { + {"lock_id_kptr_preserve" , +"5: (bf) r1 = r0 ; R0_w=ptr_foo(id=2,ref_obj_id=2,off=0,imm=0) " +"R1_w=ptr_foo(id=2,ref_obj_id=2,off=0,imm=0) refs=2\n6: (85) call bpf_this_cpu_ptr#154\n" +"R1 type=ptr_ expected=percpu_ptr_" }, + {"lock_id_global_zero" , +"; R1_w=map_value(off=0,ks=4,vs=4,imm=0)\n2: (85) call bpf_this_cpu_ptr#154\n" +"R1 type=map_value expected=percpu_ptr_" }, + {"lock_id_mapval_preserve" , +"8: (bf) r1 = r0 ; R0_w=map_value(id=1,off=0,ks=4,vs=8,imm=0) " +"R1_w=map_value(id=1,off=0,ks=4,vs=8,imm=0)\n9: (85) call bpf_this_cpu_ptr#154\n" +"R1 type=map_value expected=percpu_ptr_" }, + {"lock_id_innermapval_preserve" , +"13: (bf) r1 = r0 ; R0=map_value(id=2,off=0,ks=4,vs=8,imm=0) " +"R1_w=map_value(id=2,off=0,ks=4,vs=8,imm=0)\n14: (85) call bpf_this_cpu_ptr#154\n" +"R1 type=map_value expected=percpu_ptr_" }, + {"lock_id_mismatch_kptr_kptr" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_kptr_global" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_kptr_mapval" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_kptr_innermapval" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_global_global" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_global_kptr" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_global_mapval" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_global_innermapval" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_mapval_mapval" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_mapval_kptr" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_mapval_global" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_mapval_innermapval" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_innermapval_innermapval1" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_innermapval_innermapval2" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_innermapval_kptr" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_innermapval_global" ,"bpf_spin_unlock of different lock" }, + {"lock_id_mismatch_innermapval_mapval" ,"bpf_spin_unlock of different lock" }, +}; + +static void test_spin_lock_fail_prog (const char *prog_name,const char *err_msg) +{ + LIBBPF_OPTS(bpf_object_open_opts, opts, .kernel_log_buf = log_buf, + .kernel_log_size = sizeof(log_buf), + .kernel_log_level = 1); +struct test_spin_lock_fail *skel; +struct bpf_program *prog; +int ret; + + skel =test_spin_lock_fail__open_opts (&opts); +if (!ASSERT_OK_PTR (skel,"test_spin_lock_fail__open_opts" )) +return ; + + prog =bpf_object__find_program_by_name (skel->obj, prog_name); +if (!ASSERT_OK_PTR (prog,"bpf_object__find_program_by_name" )) + goto end; + +bpf_program__set_autoload (prog, true); + + ret =test_spin_lock_fail__load (skel); +if (!ASSERT_ERR (ret,"test_spin_lock_fail__load must fail" )) + goto end; + +if (!ASSERT_OK_PTR (strstr (log_buf, err_msg),"expected error message" )) { +fprintf (stderr,"Expected: %s\n" , err_msg); +fprintf (stderr,"Verifier: %s\n" , log_buf); + } + +end: +test_spin_lock_fail__destroy (skel); +}static void *spin_lock_thread (void *arg) {
@@ -19,7 +92,7 @@ static void *spin_lock_thread(void *arg)pthread_exit (arg); } -void test_spinlock (void ) +void test_spin_lock_success (void ) {struct test_spin_lock *skel;pthread_t thread_id[4 ];
@@ -47,3 +120,17 @@ void test_spinlock(void) end:test_spin_lock__destroy (skel); } + +void test_spin_lock (void ) +{ +int i; + +test_spin_lock_success (); + +for (i =0 ; i <ARRAY_SIZE (spin_lock_fail_tests); i++) { +if (!test__start_subtest (spin_lock_fail_tests[i].prog_name)) +continue ; +test_spin_lock_fail_prog (spin_lock_fail_tests[i].prog_name, + spin_lock_fail_tests[i].err_msg); + } +}
tools/testing/selftests/bpf/progs/test_spin_lock_fail.c new file +204 -0
204 lines not on this page · raw message
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 22/24 selftests/bpf: Add BPF linked list API tests
In reply to Kumar Kartikeya DwivediInclude various tests covering the success and failure cases. Also, run the success cases at runtime to verify correctness of linked list manipulation routines, in addition to ensuring successful verification.
--- tools/testing/selftests/bpf/DENYLIST.aarch64 | 1 + tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/linked_list.c | 255 ++++++++ .../testing/selftests/bpf/progs/linked_list.c | 370 +++++++++++ .../testing/selftests/bpf/progs/linked_list.h | 56 ++ .../selftests/bpf/progs/linked_list_fail.c | 581 ++++++++++++++++++ 6 files changed, 1264 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.h create mode 100644 tools/testing/selftests/bpf/progs/linked_list_fail.c
tools/testing/selftests/bpf/DENYLIST.aarch64 +1 -0
@@ -38,6 +38,7 @@ kprobe_multi_test/skel_api # kprobe_multi__attach unexpect ksyms_module/libbpf # 'bpf_testmod_ksym_percpu': not found in kernel BTF ksyms_module/lskel # test_ksyms_module_lskel__open_and_load unexpected error: -2 libbpf_get_fd_by_id_opts # test_libbpf_get_fd_by_id_opts__attach unexpected error: -524 (errno 524) +linked_list lookup_key # test_lookup_key__attach unexpected error: -524 (errno 524) lru_bug # lru_bug__attach unexpected error: -524 (errno 524) modify_return # modify_return__attach failed unexpected error: -524 (errno 524)
tools/testing/selftests/bpf/DENYLIST.s390x +1 -0
@@ -33,6 +33,7 @@ ksyms_module # test_ksyms_module__open_and_load unex ksyms_module_libbpf # JIT does not support calling kernel function (kfunc) ksyms_module_lskel # test_ksyms_module_lskel__open_and_load unexpected error: -9 (?) libbpf_get_fd_by_id_opts # failed to attach: ERROR: strerror_r(-524)=22 (trampoline) +linked_list # JIT does not support calling kernel function (kfunc) lookup_key # JIT does not support calling kernel function (kfunc) lru_bug # prog 'printk': failed to auto-attach: -524 map_kptr # failed to open_and_load program: -524 (trampoline)
tools/testing/selftests/bpf/prog_tests/linked_list.c new file +255 -0
255 lines not on this page · raw message
tools/testing/selftests/bpf/progs/linked_list.c new file +370 -0
370 lines not on this page · raw message
tools/testing/selftests/bpf/progs/linked_list.h new file +56 -0
56 lines not on this page · raw message
tools/testing/selftests/bpf/progs/linked_list_fail.c new file +581 -0
581 lines not on this page · raw message
Signature · 1 line
2.38.1
Re: [PATCH bpf-next v10 22/24] selftests/bpf: Add BPF linked list API tests
Quoted 144 lines · Kumar Kartikeya Dwivedi
Include various tests covering the success and failure cases. Also, run the success cases at runtime to verify correctness of linked list manipulation routines, in addition to ensuring successful verification.
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- tools/testing/selftests/bpf/DENYLIST.aarch64 | 1 + tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/linked_list.c | 255 ++++++++ .../testing/selftests/bpf/progs/linked_list.c | 370 +++++++++++ .../testing/selftests/bpf/progs/linked_list.h | 56 ++ .../selftests/bpf/progs/linked_list_fail.c | 581 ++++++++++++++++++ 6 files changed, 1264 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.h create mode 100644 tools/testing/selftests/bpf/progs/linked_list_fail.ctools/testing/selftests/bpf/DENYLIST.aarch64 +1
-0@@ -38,6 +38,7 @@ kprobe_multi_test/skel_api # kprobe_multi__attach unexpect ksyms_module/libbpf # 'bpf_testmod_ksym_percpu': not found in kernel BTF ksyms_module/lskel # test_ksyms_module_lskel__open_and_load unexpected error: -2 libbpf_get_fd_by_id_opts # test_libbpf_get_fd_by_id_opts__attach unexpected error: -524 (errno 524) +linked_list lookup_key # test_lookup_key__attach unexpected error: -524 (errno 524) lru_bug # lru_bug__attach unexpected error: -524 (errno 524) modify_return # modify_return__attach failed unexpected error: -524 (errno 524)tools/testing/selftests/bpf/DENYLIST.s390x +1
-0@@ -33,6 +33,7 @@ ksyms_module # test_ksyms_module__open_and_load unex ksyms_module_libbpf # JIT does not support calling kernel function (kfunc) ksyms_module_lskel # test_ksyms_module_lskel__open_and_load unexpected error: -9 (?) libbpf_get_fd_by_id_opts # failed to attach: ERROR: strerror_r(-524)=22 (trampoline) +linked_list # JIT does not support calling kernel function (kfunc) lookup_key # JIT does not support calling kernel function (kfunc) lru_bug # prog 'printk': failed to auto-attach: -524 map_kptr # failed to open_and_load program: -524 (trampoline)tools/testing/selftests/bpf/prog_tests/linked_list.c new file +95
-0@@ -0,0 +1,255 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> +#include <network_helpers.h> + +#include "linked_list.skel.h" +#include "linked_list_fail.skel.h" + +static char log_buf[1024 *1024 ]; + +static struct { + const char *prog_name; + const char *err_msg; +} linked_list_fail_tests[] = { +#define TEST(test, off) \ + { #test "_missing_lock_push_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_push_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, + TEST(kptr, 32) + TEST(global, 16) + TEST(map, 0) + TEST(inner_map, 0) +#undef TEST +#define TEST (test, op) \ + { #test "_kptr_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=32 must be held for bpf_list_head" }, \ + { #test "_global_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=16 must be held for bpf_list_head" }, \ + { #test "_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, \ + { #test "_inner_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, + TEST(kptr, push_front) + TEST(kptr, push_back) + TEST(kptr, pop_front) + TEST(kptr, pop_back) + TEST(global, push_front) + TEST(global, push_back) + TEST(global, pop_front) + TEST(global, pop_back) + TEST(map, push_front) + TEST(map, push_back) + TEST(map, pop_front) + TEST(map, pop_back) + TEST(inner_map, push_front) + TEST(inner_map, push_back) + TEST(inner_map, pop_front) + TEST(inner_map, pop_back) +#undef TEST + { "map_compat_kprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_kretprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_perf", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp_w", "tracing progs cannot use bpf_list_head yet" }, + { "obj_type_id_oor", "local type ID argument must be in range [0, U32_MAX]" }, + { "obj_new_no_composite", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_new_no_struct", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_drop_non_zero_off", "R1 must have zero offset when passed to release func" }, + { "new_null_ret", "R0 invalid mem access 'ptr_or_null_'" }, + { "obj_new_acq", "Unreleased reference id=" }, + { "use_after_drop", "invalid mem access 'scalar'" }, + { "ptr_walk_scalar", "type=scalar expected=percpu_ptr_" }, + { "direct_read_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_write_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_read_head", "direct access to bpf_list_head is disallowed" }, + { "direct_write_head", "direct access to bpf_list_head is disallowed" }, + { "direct_read_node", "direct access to bpf_list_node is disallowed" }, + { "direct_write_node", "direct access to bpf_list_node is disallowed" }, + { "write_after_push_front", "only read is supported" }, + { "write_after_push_back", "only read is supported" }, + { "use_after_unlock_push_front", "invalid mem access 'scalar'" }, + { "use_after_unlock_push_back", "invalid mem access 'scalar'" }, + { "double_push_front", "arg#1 expected pointer to allocated object" }, + { "double_push_back", "arg#1 expected pointer to allocated object" }, + { "no_node_value_type", "bpf_list_node not found at offset=0" }, + { "incorrect_value_type", +"operation on bpf_list_head expects arg#1 bpf_list_node at offset=0 in struct foo, " + "but arg is at offset=0 in struct bar" }, + { "incorrect_node_var_off", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_node_off1", "bpf_list_node not found at offset=1" }, + { "incorrect_node_off2", "arg#1 offset=40, but expected bpf_list_node at offset=0 in struct foo" }, + { "no_head_type", "bpf_list_head not found at offset=0" }, + { "incorrect_head_var_off1", "R1 doesn't have constant offset" }, + { "incorrect_head_var_off2", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_head_off1", "bpf_list_head not found at offset=17" }, + { "incorrect_head_off2", "bpf_list_head not found at offset=1" },
tools/testing/selftests/bpf/prog_tests/linked_list.c line 105
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/linked_list.c new file +10
-0+ {"pop_front_off" , +"15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " +"R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, + { "pop_back_off", +"15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " +"R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, +}; +
Hey Kumar,
pop_front_off/pop_back_off validation seems to rely on exact register usage (r6 in this case) generated by the compiler, while the test itself is written in C, so really nothing is guaranteed. And that's exactly what seems to happen to me locally, as in my case compiler chose to use r7 in this particular spot (see logs below).
Can you please take a look and try to make it more robust? Ideally we should probably rewrite BPF program to use inline assembly if we are to check the exact instruction index and registers.
Here are error logs:
test_linked_list_fail_prog:PASS:linked_list_fail__open_opts 0 nsec test_linked_list_fail_prog:PASS:bpf_object__find_program_by_name 0 nsec
libbpf: prog 'pop_front_off': BPF program load failed: Permission denied
libbpf: prog 'pop_front_off': failed to load: -13
libbpf: failed to load object 'linked_list_fail'
libbpf: failed to load BPF skeleton 'linked_list_fail': -13 test_linked_list_fail_prog:PASS:linked_list_fail__load must fail 0 nsec test_linked_list_fail_prog:FAIL:expected error message unexpected error: -13
Expected: 15: (bf) r1 = r6 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4 16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_
Verifier: reg type unsupported for arg#0 function pop_front_off#173 0: R1=ctx(off=0,imm=0) R10=fp0
; p = bpf_obj_new(typeof(*p));
0: (18) r1 = 0xae ; R1_w=174 2: (b7) r2 = 0 ; R2_w=0 3: (85) call bpf_obj_new_impl#25364 ;
R0_w=ptr_or_null_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
; if (!p)
4: (15) if r0 == 0x0 goto pc+12 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
; bpf_spin_lock(&p->lock);
5: (bf) r6 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
6: (07) r6 += 16 ;
R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
; bpf_spin_lock(&p->lock);
7: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
8: (bf) r7 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
9: (85) call bpf_spin_lock#93 ; refs=2
; n = op(&p->head);
10: (bf) r1 = r7 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
11: (85) call bpf_list_pop_front#25345 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
12: (bf) r7 = r0 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
; bpf_spin_unlock(&p->lock);
13: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2,4
14: (85) call bpf_spin_unlock#94 ; refs=2,4
; bpf_this_cpu_ptr(n);
15: (bf) r1 = r7 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_, percpu_rcu_ptr_, percpu_trusted_ptr_ processed 16 insns (limit 1000000) max_states_per_insn 0 total_states 1 peak_states 1 mark_read 1
#126/115 linked_list/pop_front_off:FAIL test_linked_list_fail_prog:PASS:linked_list_fail__open_opts 0 nsec test_linked_list_fail_prog:PASS:bpf_object__find_program_by_name 0 nsec
libbpf: prog 'pop_back_off': BPF program load failed: Permission denied
libbpf: prog 'pop_back_off': failed to load: -13
libbpf: failed to load object 'linked_list_fail'
libbpf: failed to load BPF skeleton 'linked_list_fail': -13 test_linked_list_fail_prog:PASS:linked_list_fail__load must fail 0 nsec test_linked_list_fail_prog:FAIL:expected error message unexpected error: -13
Expected: 15: (bf) r1 = r6 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4 16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_
Verifier: reg type unsupported for arg#0 function pop_back_off#176 0: R1=ctx(off=0,imm=0) R10=fp0
; p = bpf_obj_new(typeof(*p));
0: (18) r1 = 0xae ; R1_w=174 2: (b7) r2 = 0 ; R2_w=0 3: (85) call bpf_obj_new_impl#25364 ;
R0_w=ptr_or_null_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
; if (!p)
4: (15) if r0 == 0x0 goto pc+12 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
; bpf_spin_lock(&p->lock);
5: (bf) r6 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
6: (07) r6 += 16 ;
R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
; bpf_spin_lock(&p->lock);
7: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
8: (bf) r7 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
9: (85) call bpf_spin_lock#93 ; refs=2
; n = op(&p->head);
10: (bf) r1 = r7 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
11: (85) call bpf_list_pop_back#25344 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
12: (bf) r7 = r0 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
; bpf_spin_unlock(&p->lock);
13: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2,4
14: (85) call bpf_spin_unlock#94 ; refs=2,4
; bpf_this_cpu_ptr(n);
15: (bf) r1 = r7 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_, percpu_rcu_ptr_, percpu_trusted_ptr_ processed 16 insns (limit 1000000) max_states_per_insn 0 total_states 1 peak_states 1 mark_read 1
#126/116 linked_list/pop_back_off:FAIL
Thanks!
[...]
Quoted 37 lines
tools/testing/selftests/bpf/prog_tests/linked_list.c new file +34
-1+ +static __always_inline +int pop_ptr_off(void *(*op)(void *head)) +{ +struct { + struct bpf_list_head head __contains(foo, node2); +struct bpf_spin_lock lock; + } *p; +struct bpf_list_node *n; + + p =bpf_obj_new (typeof (*p)); +if (!p) +return 0 ; +bpf_spin_lock (&p->lock); + n =op (&p->head); +bpf_spin_unlock (&p->lock); + +bpf_this_cpu_ptr (n); +return 0 ; +} + +SEC("?tc") +int pop_front_off (void *ctx) +{ +return pop_ptr_off ((void *)bpf_list_pop_front); +} + +SEC("?tc") +int pop_back_off (void *ctx) +{ +return pop_ptr_off ((void *)bpf_list_pop_back); +} + +char _license[] SEC("license") = "GPL"; --2.38.1
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v10 22/24] selftests/bpf: Add BPF linked list API tests
Quoted 166 lines · Andrii Nakryiko
Quoted 144 lines · Kumar Kartikeya Dwivedi
Include various tests covering the success and failure cases. Also, run the success cases at runtime to verify correctness of linked list manipulation routines, in addition to ensuring successful verification.
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- tools/testing/selftests/bpf/DENYLIST.aarch64 | 1 + tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/linked_list.c | 255 ++++++++ .../testing/selftests/bpf/progs/linked_list.c | 370 +++++++++++ .../testing/selftests/bpf/progs/linked_list.h | 56 ++ .../selftests/bpf/progs/linked_list_fail.c | 581 ++++++++++++++++++ 6 files changed, 1264 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.h create mode 100644 tools/testing/selftests/bpf/progs/linked_list_fail.ctools/testing/selftests/bpf/DENYLIST.aarch64 +1
-0@@ -38,6 +38,7 @@ kprobe_multi_test/skel_api # kprobe_multi__attach unexpect ksyms_module/libbpf # 'bpf_testmod_ksym_percpu': not found in kernel BTF ksyms_module/lskel # test_ksyms_module_lskel__open_and_load unexpected error: -2 libbpf_get_fd_by_id_opts # test_libbpf_get_fd_by_id_opts__attach unexpected error: -524 (errno 524) +linked_list lookup_key # test_lookup_key__attach unexpected error: -524 (errno 524) lru_bug # lru_bug__attach unexpected error: -524 (errno 524) modify_return # modify_return__attach failed unexpected error: -524 (errno 524)tools/testing/selftests/bpf/DENYLIST.s390x +1
-0@@ -33,6 +33,7 @@ ksyms_module # test_ksyms_module__open_and_load unex ksyms_module_libbpf # JIT does not support calling kernel function (kfunc) ksyms_module_lskel # test_ksyms_module_lskel__open_and_load unexpected error: -9 (?) libbpf_get_fd_by_id_opts # failed to attach: ERROR: strerror_r(-524)=22 (trampoline) +linked_list # JIT does not support calling kernel function (kfunc) lookup_key # JIT does not support calling kernel function (kfunc) lru_bug # prog 'printk': failed to auto-attach: -524 map_kptr # failed to open_and_load program: -524 (trampoline)tools/testing/selftests/bpf/prog_tests/linked_list.c new file +95
-0@@ -0,0 +1,255 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> +#include <network_helpers.h> + +#include "linked_list.skel.h" +#include "linked_list_fail.skel.h" + +static char log_buf[1024 *1024 ]; + +static struct { + const char *prog_name; + const char *err_msg; +} linked_list_fail_tests[] = { +#define TEST(test, off) \ + { #test "_missing_lock_push_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_push_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, + TEST(kptr, 32) + TEST(global, 16) + TEST(map, 0) + TEST(inner_map, 0) +#undef TEST +#define TEST (test, op) \ + { #test "_kptr_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=32 must be held for bpf_list_head" }, \ + { #test "_global_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=16 must be held for bpf_list_head" }, \ + { #test "_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, \ + { #test "_inner_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, + TEST(kptr, push_front) + TEST(kptr, push_back) + TEST(kptr, pop_front) + TEST(kptr, pop_back) + TEST(global, push_front) + TEST(global, push_back) + TEST(global, pop_front) + TEST(global, pop_back) + TEST(map, push_front) + TEST(map, push_back) + TEST(map, pop_front) + TEST(map, pop_back) + TEST(inner_map, push_front) + TEST(inner_map, push_back) + TEST(inner_map, pop_front) + TEST(inner_map, pop_back) +#undef TEST + { "map_compat_kprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_kretprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_perf", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp_w", "tracing progs cannot use bpf_list_head yet" }, + { "obj_type_id_oor", "local type ID argument must be in range [0, U32_MAX]" }, + { "obj_new_no_composite", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_new_no_struct", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_drop_non_zero_off", "R1 must have zero offset when passed to release func" }, + { "new_null_ret", "R0 invalid mem access 'ptr_or_null_'" }, + { "obj_new_acq", "Unreleased reference id=" }, + { "use_after_drop", "invalid mem access 'scalar'" }, + { "ptr_walk_scalar", "type=scalar expected=percpu_ptr_" }, + { "direct_read_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_write_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_read_head", "direct access to bpf_list_head is disallowed" }, + { "direct_write_head", "direct access to bpf_list_head is disallowed" }, + { "direct_read_node", "direct access to bpf_list_node is disallowed" }, + { "direct_write_node", "direct access to bpf_list_node is disallowed" }, + { "write_after_push_front", "only read is supported" }, + { "write_after_push_back", "only read is supported" }, + { "use_after_unlock_push_front", "invalid mem access 'scalar'" }, + { "use_after_unlock_push_back", "invalid mem access 'scalar'" }, + { "double_push_front", "arg#1 expected pointer to allocated object" }, + { "double_push_back", "arg#1 expected pointer to allocated object" }, + { "no_node_value_type", "bpf_list_node not found at offset=0" }, + { "incorrect_value_type", +"operation on bpf_list_head expects arg#1 bpf_list_node at offset=0 in struct foo, " + "but arg is at offset=0 in struct bar" }, + { "incorrect_node_var_off", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_node_off1", "bpf_list_node not found at offset=1" }, + { "incorrect_node_off2", "arg#1 offset=40, but expected bpf_list_node at offset=0 in struct foo" }, + { "no_head_type", "bpf_list_head not found at offset=0" }, + { "incorrect_head_var_off1", "R1 doesn't have constant offset" }, + { "incorrect_head_var_off2", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_head_off1", "bpf_list_head not found at offset=17" }, + { "incorrect_head_off2", "bpf_list_head not found at offset=1" },tools/testing/selftests/bpf/prog_tests/linked_list.c line 105
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/linked_list.c new file +10
-0+ {"pop_front_off" , +"15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " +"R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, + { "pop_back_off", +"15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " +"R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, +}; +Hey Kumar,
pop_front_off/pop_back_off validation seems to rely on exact register usage (r6 in this case) generated by the compiler, while the test itself is written in C, so really nothing is guaranteed. And that's exactly what seems to happen to me locally, as in my case compiler chose to use r7 in this particular spot (see logs below).
Quoted 3 lines
Can you please take a look and try to make it more robust? Ideally we should probably rewrite BPF program to use inline assembly if we are to check the exact instruction index and registers.
Thanks for the report Andrii.
I'll take a look and send a patch to address this.
Quoted 169 lines
Here are error logs:
test_linked_list_fail_prog:PASS:linked_list_fail__open_opts 0 nsec test_linked_list_fail_prog:PASS:bpf_object__find_program_by_name 0 nsec
libbpf: prog 'pop_front_off': BPF program load failed: Permission denied
libbpf: prog 'pop_front_off': failed to load: -13
libbpf: failed to load object 'linked_list_fail'
libbpf: failed to load BPF skeleton 'linked_list_fail': -13 test_linked_list_fail_prog:PASS:linked_list_fail__load must fail 0 nsec test_linked_list_fail_prog:FAIL:expected error message unexpected error: -13
Expected: 15: (bf) r1 = r6 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4 16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_
Verifier: reg type unsupported for arg#0 function pop_front_off#173 0: R1=ctx(off=0,imm=0) R10=fp0
; p = bpf_obj_new(typeof(*p));0: (18) r1 = 0xae ; R1_w=174 2: (b7) r2 = 0 ; R2_w=0 3: (85) call bpf_obj_new_impl#25364 ;R0_w=ptr_or_null_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
; if (!p)
4: (15) if r0 ==0x0goto pc+12 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
;bpf_spin_lock(&p->lock);
5: (bf) r6 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
6: (07) r6 += 16 ;
R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
;bpf_spin_lock(&p->lock);
7: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
8: (bf) r7 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
9: (85) call bpf_spin_lock#93 ; refs=2
; n =op(&p->head);
10: (bf) r1 = r7 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
11: (85) call bpf_list_pop_front#25345 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
12: (bf) r7 = r0 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
;bpf_spin_unlock(&p->lock);
13: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2,4
14: (85) call bpf_spin_unlock#94 ; refs=2,4
;bpf_this_cpu_ptr(n);
15: (bf) r1 = r7 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_,percpu_rcu_ptr_,percpu_trusted_ptr_processed 16 insns (limit 1000000)max_states_per_insn0total_states1peak_states1mark_read1#126/115 linked_list/pop_front_off:FAIL test_linked_list_fail_prog:PASS:linked_list_fail__open_opts 0 nsec test_linked_list_fail_prog:PASS:bpf_object__find_program_by_name 0 nsec
libbpf: prog 'pop_back_off': BPF program load failed: Permission denied
libbpf: prog 'pop_back_off': failed to load: -13
libbpf: failed to load object 'linked_list_fail'
libbpf: failed to load BPF skeleton 'linked_list_fail': -13 test_linked_list_fail_prog:PASS:linked_list_fail__load must fail 0 nsec test_linked_list_fail_prog:FAIL:expected error message unexpected error: -13
Expected: 15: (bf) r1 = r6 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4 16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_
Verifier: reg type unsupported for arg#0 function pop_back_off#176 0: R1=ctx(off=0,imm=0) R10=fp0
; p = bpf_obj_new(typeof(*p));0: (18) r1 = 0xae ; R1_w=174 2: (b7) r2 = 0 ; R2_w=0 3: (85) call bpf_obj_new_impl#25364 ;R0_w=ptr_or_null_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
; if (!p)
4: (15) if r0 ==0x0goto pc+12 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
;bpf_spin_lock(&p->lock);
5: (bf) r6 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
6: (07) r6 += 16 ;
R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
;bpf_spin_lock(&p->lock);
7: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
8: (bf) r7 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
9: (85) call bpf_spin_lock#93 ; refs=2
; n =op(&p->head);
10: (bf) r1 = r7 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
11: (85) call bpf_list_pop_back#25344 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
12: (bf) r7 = r0 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
;bpf_spin_unlock(&p->lock);
13: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2,4
14: (85) call bpf_spin_unlock#94 ; refs=2,4
;bpf_this_cpu_ptr(n);
15: (bf) r1 = r7 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_,percpu_rcu_ptr_,percpu_trusted_ptr_processed 16 insns (limit 1000000)max_states_per_insn0total_states1peak_states1mark_read1#126/116 linked_list/pop_back_off:FAIL
Thanks!
[...]
Quoted 37 lines
tools/testing/selftests/bpf/prog_tests/linked_list.c new file +34
-1+ +static __always_inline +int pop_ptr_off(void *(*op)(void *head)) +{ +struct { + struct bpf_list_head head __contains(foo, node2); +struct bpf_spin_lock lock; + } *p; +struct bpf_list_node *n; + + p =bpf_obj_new (typeof (*p)); +if (!p) +return 0 ; +bpf_spin_lock (&p->lock); + n =op (&p->head); +bpf_spin_unlock (&p->lock); + +bpf_this_cpu_ptr (n); +return 0 ; +} + +SEC("?tc") +int pop_front_off (void *ctx) +{ +return pop_ptr_off ((void *)bpf_list_pop_front); +} + +SEC("?tc") +int pop_back_off (void *ctx) +{ +return pop_ptr_off ((void *)bpf_list_pop_back); +} + +char _license[] SEC("license") = "GPL"; --2.38.1
Re: [PATCH bpf-next v10 22/24] selftests/bpf: Add BPF linked list API tests
Quoted 168 lines · Kumar Kartikeya Dwivedi
Quoted 166 lines · Andrii Nakryiko
Quoted 144 lines · Kumar Kartikeya Dwivedi
Include various tests covering the success and failure cases. Also, run the success cases at runtime to verify correctness of linked list manipulation routines, in addition to ensuring successful verification.
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- tools/testing/selftests/bpf/DENYLIST.aarch64 | 1 + tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/linked_list.c | 255 ++++++++ .../testing/selftests/bpf/progs/linked_list.c | 370 +++++++++++ .../testing/selftests/bpf/progs/linked_list.h | 56 ++ .../selftests/bpf/progs/linked_list_fail.c | 581 ++++++++++++++++++ 6 files changed, 1264 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.h create mode 100644 tools/testing/selftests/bpf/progs/linked_list_fail.ctools/testing/selftests/bpf/DENYLIST.aarch64 +1
-0@@ -38,6 +38,7 @@ kprobe_multi_test/skel_api # kprobe_multi__attach unexpect ksyms_module/libbpf # 'bpf_testmod_ksym_percpu': not found in kernel BTF ksyms_module/lskel # test_ksyms_module_lskel__open_and_load unexpected error: -2 libbpf_get_fd_by_id_opts # test_libbpf_get_fd_by_id_opts__attach unexpected error: -524 (errno 524) +linked_list lookup_key # test_lookup_key__attach unexpected error: -524 (errno 524) lru_bug # lru_bug__attach unexpected error: -524 (errno 524) modify_return # modify_return__attach failed unexpected error: -524 (errno 524)tools/testing/selftests/bpf/DENYLIST.s390x +1
-0@@ -33,6 +33,7 @@ ksyms_module # test_ksyms_module__open_and_load unex ksyms_module_libbpf # JIT does not support calling kernel function (kfunc) ksyms_module_lskel # test_ksyms_module_lskel__open_and_load unexpected error: -9 (?) libbpf_get_fd_by_id_opts # failed to attach: ERROR: strerror_r(-524)=22 (trampoline) +linked_list # JIT does not support calling kernel function (kfunc) lookup_key # JIT does not support calling kernel function (kfunc) lru_bug # prog 'printk': failed to auto-attach: -524 map_kptr # failed to open_and_load program: -524 (trampoline)tools/testing/selftests/bpf/prog_tests/linked_list.c new file +95
-0@@ -0,0 +1,255 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> +#include <network_helpers.h> + +#include "linked_list.skel.h" +#include "linked_list_fail.skel.h" + +static char log_buf[1024 *1024 ]; + +static struct { + const char *prog_name; + const char *err_msg; +} linked_list_fail_tests[] = { +#define TEST(test, off) \ + { #test "_missing_lock_push_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_push_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, + TEST(kptr, 32) + TEST(global, 16) + TEST(map, 0) + TEST(inner_map, 0) +#undef TEST +#define TEST (test, op) \ + { #test "_kptr_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=32 must be held for bpf_list_head" }, \ + { #test "_global_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=16 must be held for bpf_list_head" }, \ + { #test "_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, \ + { #test "_inner_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, + TEST(kptr, push_front) + TEST(kptr, push_back) + TEST(kptr, pop_front) + TEST(kptr, pop_back) + TEST(global, push_front) + TEST(global, push_back) + TEST(global, pop_front) + TEST(global, pop_back) + TEST(map, push_front) + TEST(map, push_back) + TEST(map, pop_front) + TEST(map, pop_back) + TEST(inner_map, push_front) + TEST(inner_map, push_back) + TEST(inner_map, pop_front) + TEST(inner_map, pop_back) +#undef TEST + { "map_compat_kprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_kretprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_perf", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp_w", "tracing progs cannot use bpf_list_head yet" }, + { "obj_type_id_oor", "local type ID argument must be in range [0, U32_MAX]" }, + { "obj_new_no_composite", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_new_no_struct", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_drop_non_zero_off", "R1 must have zero offset when passed to release func" }, + { "new_null_ret", "R0 invalid mem access 'ptr_or_null_'" }, + { "obj_new_acq", "Unreleased reference id=" }, + { "use_after_drop", "invalid mem access 'scalar'" }, + { "ptr_walk_scalar", "type=scalar expected=percpu_ptr_" }, + { "direct_read_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_write_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_read_head", "direct access to bpf_list_head is disallowed" }, + { "direct_write_head", "direct access to bpf_list_head is disallowed" }, + { "direct_read_node", "direct access to bpf_list_node is disallowed" }, + { "direct_write_node", "direct access to bpf_list_node is disallowed" }, + { "write_after_push_front", "only read is supported" }, + { "write_after_push_back", "only read is supported" }, + { "use_after_unlock_push_front", "invalid mem access 'scalar'" }, + { "use_after_unlock_push_back", "invalid mem access 'scalar'" }, + { "double_push_front", "arg#1 expected pointer to allocated object" }, + { "double_push_back", "arg#1 expected pointer to allocated object" }, + { "no_node_value_type", "bpf_list_node not found at offset=0" }, + { "incorrect_value_type", +"operation on bpf_list_head expects arg#1 bpf_list_node at offset=0 in struct foo, " + "but arg is at offset=0 in struct bar" }, + { "incorrect_node_var_off", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_node_off1", "bpf_list_node not found at offset=1" }, + { "incorrect_node_off2", "arg#1 offset=40, but expected bpf_list_node at offset=0 in struct foo" }, + { "no_head_type", "bpf_list_head not found at offset=0" }, + { "incorrect_head_var_off1", "R1 doesn't have constant offset" }, + { "incorrect_head_var_off2", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_head_off1", "bpf_list_head not found at offset=17" }, + { "incorrect_head_off2", "bpf_list_head not found at offset=1" },tools/testing/selftests/bpf/prog_tests/linked_list.c line 105
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/linked_list.c new file +10
-0+ {"pop_front_off" , +"15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " +"R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, + { "pop_back_off", +"15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " +"R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, +}; +Hey Kumar,
pop_front_off/pop_back_off validation seems to rely on exact register usage (r6 in this case) generated by the compiler, while the test itself is written in C, so really nothing is guaranteed. And that's exactly what seems to happen to me locally, as in my case compiler chose to use r7 in this particular spot (see logs below).
Quoted 6 lines
Quoted 3 lines
Can you please take a look and try to make it more robust? Ideally we should probably rewrite BPF program to use inline assembly if we are to check the exact instruction index and registers.
Thanks for the report Andrii.
I'll take a look and send a patch to address this.
Friendly ping! Did you get a chance to look at this?
Quoted 170 lines
Quoted 170 lines
Here are error logs:
test_linked_list_fail_prog:PASS:linked_list_fail__open_opts 0 nsec test_linked_list_fail_prog:PASS:bpf_object__find_program_by_name 0 nsec
libbpf: prog 'pop_front_off': BPF program load failed: Permission denied
libbpf: prog 'pop_front_off': failed to load: -13
libbpf: failed to load object 'linked_list_fail'
libbpf: failed to load BPF skeleton 'linked_list_fail': -13 test_linked_list_fail_prog:PASS:linked_list_fail__load must fail 0 nsec test_linked_list_fail_prog:FAIL:expected error message unexpected error: -13
Expected: 15: (bf) r1 = r6 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4 16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_
Verifier: reg type unsupported for arg#0 function pop_front_off#173 0: R1=ctx(off=0,imm=0) R10=fp0
; p = bpf_obj_new(typeof(*p));0: (18) r1 = 0xae ; R1_w=174 2: (b7) r2 = 0 ; R2_w=0 3: (85) call bpf_obj_new_impl#25364 ;R0_w=ptr_or_null_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
; if (!p)
4: (15) if r0 ==0x0goto pc+12 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
;bpf_spin_lock(&p->lock);
5: (bf) r6 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
6: (07) r6 += 16 ;
R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
;bpf_spin_lock(&p->lock);
7: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
8: (bf) r7 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
9: (85) call bpf_spin_lock#93 ; refs=2
; n =op(&p->head);
10: (bf) r1 = r7 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
11: (85) call bpf_list_pop_front#25345 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
12: (bf) r7 = r0 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
;bpf_spin_unlock(&p->lock);
13: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2,4
14: (85) call bpf_spin_unlock#94 ; refs=2,4
;bpf_this_cpu_ptr(n);
15: (bf) r1 = r7 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_,percpu_rcu_ptr_,percpu_trusted_ptr_processed 16 insns (limit 1000000)max_states_per_insn0total_states1peak_states1mark_read1#126/115 linked_list/pop_front_off:FAIL test_linked_list_fail_prog:PASS:linked_list_fail__open_opts 0 nsec test_linked_list_fail_prog:PASS:bpf_object__find_program_by_name 0 nsec
libbpf: prog 'pop_back_off': BPF program load failed: Permission denied
libbpf: prog 'pop_back_off': failed to load: -13
libbpf: failed to load object 'linked_list_fail'
libbpf: failed to load BPF skeleton 'linked_list_fail': -13 test_linked_list_fail_prog:PASS:linked_list_fail__load must fail 0 nsec test_linked_list_fail_prog:FAIL:expected error message unexpected error: -13
Expected: 15: (bf) r1 = r6 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4 16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_
Verifier: reg type unsupported for arg#0 function pop_back_off#176 0: R1=ctx(off=0,imm=0) R10=fp0
; p = bpf_obj_new(typeof(*p));0: (18) r1 = 0xae ; R1_w=174 2: (b7) r2 = 0 ; R2_w=0 3: (85) call bpf_obj_new_impl#25364 ;R0_w=ptr_or_null_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
; if (!p)
4: (15) if r0 ==0x0goto pc+12 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
;bpf_spin_lock(&p->lock);
5: (bf) r6 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
6: (07) r6 += 16 ;
R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
;bpf_spin_lock(&p->lock);
7: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2
8: (bf) r7 = r0 ;
R0_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
9: (85) call bpf_spin_lock#93 ; refs=2
; n =op(&p->head);
10: (bf) r1 = r7 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=0,imm=0) R7=ptr_(id=2,ref_obj_id=2,off=0,imm=0) refs=2
11: (85) call bpf_list_pop_back#25344 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
12: (bf) r7 = r0 ;
R0_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
;bpf_spin_unlock(&p->lock);
13: (bf) r1 = r6 ;
R1_w=ptr_(id=2,ref_obj_id=2,off=16,imm=0) R6=ptr_(id=2,ref_obj_id=2,off=16,imm=0) refs=2,4
14: (85) call bpf_spin_unlock#94 ; refs=2,4
;bpf_this_cpu_ptr(n);
15: (bf) r1 = r7 ;
R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) R7_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=48,imm=0) refs=2,4
16: (85) call bpf_this_cpu_ptr#154
R1 type=ptr_or_null_ expected=percpu_ptr_,percpu_rcu_ptr_,percpu_trusted_ptr_processed 16 insns (limit 1000000)max_states_per_insn0total_states1peak_states1mark_read1#126/116 linked_list/pop_back_off:FAIL
Thanks!
[...]
Quoted 37 lines
tools/testing/selftests/bpf/prog_tests/linked_list.c new file +34
-1+ +static __always_inline +int pop_ptr_off(void *(*op)(void *head)) +{ +struct { + struct bpf_list_head head __contains(foo, node2); +struct bpf_spin_lock lock; + } *p; +struct bpf_list_node *n; + + p =bpf_obj_new (typeof (*p)); +if (!p) +return 0 ; +bpf_spin_lock (&p->lock); + n =op (&p->head); +bpf_spin_unlock (&p->lock); + +bpf_this_cpu_ptr (n); +return 0 ; +} + +SEC("?tc") +int pop_front_off (void *ctx) +{ +return pop_ptr_off ((void *)bpf_list_pop_front); +} + +SEC("?tc") +int pop_back_off (void *ctx) +{ +return pop_ptr_off ((void *)bpf_list_pop_back); +} + +char _license[] SEC("license") = "GPL"; --2.38.1
Kumar Kartikeya Dwivedi
maintainerRe: [PATCH bpf-next v10 22/24] selftests/bpf: Add BPF linked list API tests
Quoted 171 lines · Andrii Nakryiko
Quoted 168 lines · Kumar Kartikeya Dwivedi
Quoted 166 lines · Andrii Nakryiko
Quoted 144 lines · Kumar Kartikeya Dwivedi
Include various tests covering the success and failure cases. Also, run the success cases at runtime to verify correctness of linked list manipulation routines, in addition to ensuring successful verification.
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>--- tools/testing/selftests/bpf/DENYLIST.aarch64 | 1 + tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../selftests/bpf/prog_tests/linked_list.c | 255 ++++++++ .../testing/selftests/bpf/progs/linked_list.c | 370 +++++++++++ .../testing/selftests/bpf/progs/linked_list.h | 56 ++ .../selftests/bpf/progs/linked_list_fail.c | 581 ++++++++++++++++++ 6 files changed, 1264 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.c create mode 100644 tools/testing/selftests/bpf/progs/linked_list.h create mode 100644 tools/testing/selftests/bpf/progs/linked_list_fail.ctools/testing/selftests/bpf/DENYLIST.aarch64 +1
-0@@ -38,6 +38,7 @@ kprobe_multi_test/skel_api # kprobe_multi__attach unexpect ksyms_module/libbpf # 'bpf_testmod_ksym_percpu': not found in kernel BTF ksyms_module/lskel # test_ksyms_module_lskel__open_and_load unexpected error: -2 libbpf_get_fd_by_id_opts # test_libbpf_get_fd_by_id_opts__attach unexpected error: -524 (errno 524) +linked_list lookup_key # test_lookup_key__attach unexpected error: -524 (errno 524) lru_bug # lru_bug__attach unexpected error: -524 (errno 524) modify_return # modify_return__attach failed unexpected error: -524 (errno 524)tools/testing/selftests/bpf/DENYLIST.s390x +1
-0@@ -33,6 +33,7 @@ ksyms_module # test_ksyms_module__open_and_load unex ksyms_module_libbpf # JIT does not support calling kernel function (kfunc) ksyms_module_lskel # test_ksyms_module_lskel__open_and_load unexpected error: -9 (?) libbpf_get_fd_by_id_opts # failed to attach: ERROR: strerror_r(-524)=22 (trampoline) +linked_list # JIT does not support calling kernel function (kfunc) lookup_key # JIT does not support calling kernel function (kfunc) lru_bug # prog 'printk': failed to auto-attach: -524 map_kptr # failed to open_and_load program: -524 (trampoline)tools/testing/selftests/bpf/prog_tests/linked_list.c new file +95
-0@@ -0,0 +1,255 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <test_progs.h> +#include <network_helpers.h> + +#include "linked_list.skel.h" +#include "linked_list_fail.skel.h" + +static char log_buf[1024 *1024 ]; + +static struct { + const char *prog_name; + const char *err_msg; +} linked_list_fail_tests[] = { +#define TEST(test, off) \ + { #test "_missing_lock_push_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_push_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_front", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, \ + { #test "_missing_lock_pop_back", \ + "bpf_spin_lock at off=" #off " must be held for bpf_list_head" }, + TEST(kptr, 32) + TEST(global, 16) + TEST(map, 0) + TEST(inner_map, 0) +#undef TEST +#define TEST (test, op) \ + { #test "_kptr_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=32 must be held for bpf_list_head" }, \ + { #test "_global_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=16 must be held for bpf_list_head" }, \ + { #test "_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, \ + { #test "_inner_map_incorrect_lock_" #op, \ + "held lock and object are not in the same allocation\n" \ + "bpf_spin_lock at off=0 must be held for bpf_list_head" }, + TEST(kptr, push_front) + TEST(kptr, push_back) + TEST(kptr, pop_front) + TEST(kptr, pop_back) + TEST(global, push_front) + TEST(global, push_back) + TEST(global, pop_front) + TEST(global, pop_back) + TEST(map, push_front) + TEST(map, push_back) + TEST(map, pop_front) + TEST(map, pop_back) + TEST(inner_map, push_front) + TEST(inner_map, push_back) + TEST(inner_map, pop_front) + TEST(inner_map, pop_back) +#undef TEST + { "map_compat_kprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_kretprobe", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_perf", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp", "tracing progs cannot use bpf_list_head yet" }, + { "map_compat_raw_tp_w", "tracing progs cannot use bpf_list_head yet" }, + { "obj_type_id_oor", "local type ID argument must be in range [0, U32_MAX]" }, + { "obj_new_no_composite", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_new_no_struct", "bpf_obj_new type ID argument must be of a struct" }, + { "obj_drop_non_zero_off", "R1 must have zero offset when passed to release func" }, + { "new_null_ret", "R0 invalid mem access 'ptr_or_null_'" }, + { "obj_new_acq", "Unreleased reference id=" }, + { "use_after_drop", "invalid mem access 'scalar'" }, + { "ptr_walk_scalar", "type=scalar expected=percpu_ptr_" }, + { "direct_read_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_write_lock", "direct access to bpf_spin_lock is disallowed" }, + { "direct_read_head", "direct access to bpf_list_head is disallowed" }, + { "direct_write_head", "direct access to bpf_list_head is disallowed" }, + { "direct_read_node", "direct access to bpf_list_node is disallowed" }, + { "direct_write_node", "direct access to bpf_list_node is disallowed" }, + { "write_after_push_front", "only read is supported" }, + { "write_after_push_back", "only read is supported" }, + { "use_after_unlock_push_front", "invalid mem access 'scalar'" }, + { "use_after_unlock_push_back", "invalid mem access 'scalar'" }, + { "double_push_front", "arg#1 expected pointer to allocated object" }, + { "double_push_back", "arg#1 expected pointer to allocated object" }, + { "no_node_value_type", "bpf_list_node not found at offset=0" }, + { "incorrect_value_type", +"operation on bpf_list_head expects arg#1 bpf_list_node at offset=0 in struct foo, " + "but arg is at offset=0 in struct bar" }, + { "incorrect_node_var_off", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_node_off1", "bpf_list_node not found at offset=1" }, + { "incorrect_node_off2", "arg#1 offset=40, but expected bpf_list_node at offset=0 in struct foo" }, + { "no_head_type", "bpf_list_head not found at offset=0" }, + { "incorrect_head_var_off1", "R1 doesn't have constant offset" }, + { "incorrect_head_var_off2", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, + { "incorrect_head_off1", "bpf_list_head not found at offset=17" }, + { "incorrect_head_off2", "bpf_list_head not found at offset=1" },tools/testing/selftests/bpf/prog_tests/linked_list.c line 105
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/linked_list.c new file +10
-0+ {"pop_front_off" , +"15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " +"R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, + { "pop_back_off", +"15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " +"R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n" + "16: (85) call bpf_this_cpu_ptr#154\nR1 type=ptr_or_null_ expected=percpu_ptr_" }, +}; +Hey Kumar,
pop_front_off/pop_back_off validation seems to rely on exact register usage (r6 in this case) generated by the compiler, while the test itself is written in C, so really nothing is guaranteed. And that's exactly what seems to happen to me locally, as in my case compiler chose to use r7 in this particular spot (see logs below).
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
Can you please take a look and try to make it more robust? Ideally we should probably rewrite BPF program to use inline assembly if we are to check the exact instruction index and registers.
Thanks for the report Andrii.
I'll take a look and send a patch to address this.Friendly ping! Did you get a chance to look at this?
Hi Andrii, sorry for the delay. The fix should be relatively simple, we just need to check whether the offset on the returned pointer is 48.
I have posted a fix here, PTAL. Thanks.
Kumar Kartikeya Dwivedi
maintainerPATCH v10 23/24 selftests/bpf: Add BTF sanity tests
In reply to Kumar Kartikeya DwivediPreparing the metadata for bpf_list_head involves a complicated parsing step and type resolution for the contained value. Ensure that corner cases are tested against and invalid specifications in source are duly rejected. Also include tests for incorrect ownership relationships in the BTF.
--- .../selftests/bpf/prog_tests/linked_list.c | 485 ++++++++++++++++++ 1 file changed, 485 insertions(+)
tools/testing/selftests/bpf/prog_tests/linked_list.c +485 -0
@@ -1,4 +1,7 @@// SPDX-License-Identifier: GPL-2.0 +#include <bpf/btf.h> +#include <test_btf.h> +#include <linux/btf.h> #include <test_progs.h> #include <network_helpers.h>
@@ -235,6 +238,487 @@ static void test_linked_list_success(int mode, bool leave_in_map)linked_list__destroy (skel); } +#define SPIN_LOCK 2 +#define LIST_HEAD 3 +#define LIST_NODE 4 + +static struct btf *init_btf (void ) +{ +int id, lid, hid, nid; +struct btf *btf; + + btf =btf__new_empty (); +if (!ASSERT_OK_PTR (btf,"btf__new_empty" )) +return NULL ; + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +if (!ASSERT_EQ (id,1 ,"btf__add_int" )) + goto end; + lid =btf__add_struct (btf,"bpf_spin_lock" ,4 ); +if (!ASSERT_EQ (lid,SPIN_LOCK ,"btf__add_struct bpf_spin_lock" )) + goto end; + hid =btf__add_struct (btf,"bpf_list_head" ,16 ); +if (!ASSERT_EQ (hid,LIST_HEAD ,"btf__add_struct bpf_list_head" )) + goto end; + nid =btf__add_struct (btf,"bpf_list_node" ,16 ); +if (!ASSERT_EQ (nid,LIST_NODE ,"btf__add_struct bpf_list_node" )) + goto end; +return btf; +end: +btf__free (btf); +return NULL ; +} + +static void test_btf (void ) +{ +struct btf *btf =NULL ; +int id, err; + +while (test__start_subtest ("btf: too many locks" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,24 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,SPIN_LOCK ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_struct foo::a" )) +break ; + err =btf__add_field (btf,"b" ,SPIN_LOCK ,32 ,0 ); +if (!ASSERT_OK (err,"btf__add_struct foo::a" )) +break ; + err =btf__add_field (btf,"c" ,LIST_HEAD ,64 ,0 ); +if (!ASSERT_OK (err,"btf__add_struct foo::a" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -E2BIG ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: missing lock" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,16 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_struct foo::a" )) +break ; + id =btf__add_decl_tag (btf,"contains:baz:a" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:baz:a" )) +break ; + id =btf__add_struct (btf,"baz" ,16 ); +if (!ASSERT_EQ (id,7 ,"btf__add_struct baz" )) +break ; + err =btf__add_field (btf,"a" ,LIST_NODE ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field baz::a" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -EINVAL ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: bad offset" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,36 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + err =btf__add_field (btf,"c" ,SPIN_LOCK ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::c" )) +break ; + id =btf__add_decl_tag (btf,"contains:foo:b" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:foo:b" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -EEXIST ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: missing contains:" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,24 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,SPIN_LOCK ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_HEAD ,64 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -EINVAL ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: missing struct" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,24 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,SPIN_LOCK ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_HEAD ,64 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + id =btf__add_decl_tag (btf,"contains:bar:bar" ,5 ,1 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:bar:bar" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -ENOENT ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: missing node" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,24 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,SPIN_LOCK ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_HEAD ,64 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + id =btf__add_decl_tag (btf,"contains:foo:c" ,5 ,1 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:foo:c" )) +break ; + + err =btf__load_into_kernel (btf); +btf__free (btf); +ASSERT_EQ (err, -ENOENT ,"check btf" ); +break ; + } + +while (test__start_subtest ("btf: node incorrect type" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,20 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,SPIN_LOCK ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + id =btf__add_decl_tag (btf,"contains:bar:a" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:bar:a" )) +break ; + id =btf__add_struct (btf,"bar" ,4 ); +if (!ASSERT_EQ (id,7 ,"btf__add_struct bar" )) +break ; + err =btf__add_field (btf,"a" ,SPIN_LOCK ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::a" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -EINVAL ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: multiple bpf_list_node with name b" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,52 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,256 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::c" )) +break ; + err =btf__add_field (btf,"d" ,SPIN_LOCK ,384 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::d" )) +break ; + id =btf__add_decl_tag (btf,"contains:foo:b" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:foo:b" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -EINVAL ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: owning | owned AA cycle" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,36 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + err =btf__add_field (btf,"c" ,SPIN_LOCK ,256 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::c" )) +break ; + id =btf__add_decl_tag (btf,"contains:foo:b" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:foo:b" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -ELOOP ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: owning | owned ABA cycle" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,36 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + err =btf__add_field (btf,"c" ,SPIN_LOCK ,256 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::c" )) +break ; + id =btf__add_decl_tag (btf,"contains:bar:b" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:bar:b" )) +break ; + id =btf__add_struct (btf,"bar" ,36 ); +if (!ASSERT_EQ (id,7 ,"btf__add_struct bar" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::b" )) +break ; + err =btf__add_field (btf,"c" ,SPIN_LOCK ,256 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::c" )) +break ; + id =btf__add_decl_tag (btf,"contains:foo:b" ,7 ,0 ); +if (!ASSERT_EQ (id,8 ,"btf__add_decl_tag contains:foo:b" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -ELOOP ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: owning -> owned" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,20 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,SPIN_LOCK ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + id =btf__add_decl_tag (btf,"contains:bar:a" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:bar:a" )) +break ; + id =btf__add_struct (btf,"bar" ,16 ); +if (!ASSERT_EQ (id,7 ,"btf__add_struct bar" )) +break ; + err =btf__add_field (btf,"a" ,LIST_NODE ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::a" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err,0 ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: owning -> owning | owned -> owned" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,20 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,SPIN_LOCK ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + id =btf__add_decl_tag (btf,"contains:bar:b" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:bar:b" )) +break ; + id =btf__add_struct (btf,"bar" ,36 ); +if (!ASSERT_EQ (id,7 ,"btf__add_struct bar" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::b" )) +break ; + err =btf__add_field (btf,"c" ,SPIN_LOCK ,256 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::c" )) +break ; + id =btf__add_decl_tag (btf,"contains:baz:a" ,7 ,0 ); +if (!ASSERT_EQ (id,8 ,"btf__add_decl_tag contains:baz:a" )) +break ; + id =btf__add_struct (btf,"baz" ,16 ); +if (!ASSERT_EQ (id,9 ,"btf__add_struct baz" )) +break ; + err =btf__add_field (btf,"a" ,LIST_NODE ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field baz:a" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err,0 ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: owning | owned -> owning | owned -> owned" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,36 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + err =btf__add_field (btf,"c" ,SPIN_LOCK ,256 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::c" )) +break ; + id =btf__add_decl_tag (btf,"contains:bar:b" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:bar:b" )) +break ; + id =btf__add_struct (btf,"bar" ,36 ); +if (!ASSERT_EQ (id,7 ,"btf__add_struct bar" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar:a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar:b" )) +break ; + err =btf__add_field (btf,"c" ,SPIN_LOCK ,256 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar:c" )) +break ; + id =btf__add_decl_tag (btf,"contains:baz:a" ,7 ,0 ); +if (!ASSERT_EQ (id,8 ,"btf__add_decl_tag contains:baz:a" )) +break ; + id =btf__add_struct (btf,"baz" ,16 ); +if (!ASSERT_EQ (id,9 ,"btf__add_struct baz" )) +break ; + err =btf__add_field (btf,"a" ,LIST_NODE ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field baz:a" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -ELOOP ,"check btf" ); +btf__free (btf); +break ; + } + +while (test__start_subtest ("btf: owning -> owning | owned -> owning | owned -> owned" )) { + btf =init_btf (); +if (!ASSERT_OK_PTR (btf,"init_btf" )) +break ; + id =btf__add_struct (btf,"foo" ,20 ); +if (!ASSERT_EQ (id,5 ,"btf__add_struct foo" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::a" )) +break ; + err =btf__add_field (btf,"b" ,SPIN_LOCK ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field foo::b" )) +break ; + id =btf__add_decl_tag (btf,"contains:bar:b" ,5 ,0 ); +if (!ASSERT_EQ (id,6 ,"btf__add_decl_tag contains:bar:b" )) +break ; + id =btf__add_struct (btf,"bar" ,36 ); +if (!ASSERT_EQ (id,7 ,"btf__add_struct bar" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::b" )) +break ; + err =btf__add_field (btf,"c" ,SPIN_LOCK ,256 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::c" )) +break ; + id =btf__add_decl_tag (btf,"contains:baz:b" ,7 ,0 ); +if (!ASSERT_EQ (id,8 ,"btf__add_decl_tag" )) +break ; + id =btf__add_struct (btf,"baz" ,36 ); +if (!ASSERT_EQ (id,9 ,"btf__add_struct baz" )) +break ; + err =btf__add_field (btf,"a" ,LIST_HEAD ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::a" )) +break ; + err =btf__add_field (btf,"b" ,LIST_NODE ,128 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::b" )) +break ; + err =btf__add_field (btf,"c" ,SPIN_LOCK ,256 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bar::c" )) +break ; + id =btf__add_decl_tag (btf,"contains:bam:a" ,9 ,0 ); +if (!ASSERT_EQ (id,10 ,"btf__add_decl_tag contains:bam:a" )) +break ; + id =btf__add_struct (btf,"bam" ,16 ); +if (!ASSERT_EQ (id,11 ,"btf__add_struct bam" )) +break ; + err =btf__add_field (btf,"a" ,LIST_NODE ,0 ,0 ); +if (!ASSERT_OK (err,"btf__add_field bam::a" )) +break ; + + err =btf__load_into_kernel (btf); +ASSERT_EQ (err, -ELOOP ,"check btf" ); +btf__free (btf); +break ; + } +} +void test_linked_list (void ) {int i;
@@ -245,6 +729,7 @@ void test_linked_list(void)test_linked_list_fail_prog (linked_list_fail_tests[i].prog_name, linked_list_fail_tests[i].err_msg); } +test_btf ();test_linked_list_success (PUSH_POP , false);test_linked_list_success (PUSH_POP , true);test_linked_list_success (PUSH_POP_MULT , false);
Signature · 1 line
2.38.1
Kumar Kartikeya Dwivedi
maintainerPATCH v10 24/24 selftests/bpf: Temporarily disable linked list tests
In reply to Kumar Kartikeya DwivediThe latest clang nightly as of writing crashes with the given test case for BPF linked lists wherever global glock, ghead, glock2 are used, hence comment out the parts that cause the crash, and prepare this commit so that it can be reverted when the fix has been made. More context in [0].
[0]: https://lore.kernel.org/bpf/[email protected]
--- .../selftests/bpf/prog_tests/linked_list.c | 21 ++++++++++++------- .../testing/selftests/bpf/progs/linked_list.c | 11 +++++++++- .../testing/selftests/bpf/progs/linked_list.h | 2 ++ .../selftests/bpf/progs/linked_list_fail.c | 16 +++++++------- 4 files changed, 34 insertions(+), 16 deletions(-)
tools/testing/selftests/bpf/prog_tests/linked_list.c +14 -7
@@ -24,7 +24,9 @@ static struct { {#test "_missing_lock_pop_back", \ "bpf_spin_lock at off=" #off " must be held for bpf_list_head" },TEST (kptr,32 ) +/* FIXME TEST(global, 16) +*/ TEST(map, 0) TEST(inner_map, 0)#undef TEST
@@ -32,9 +34,6 @@ static struct { {#test "_kptr_incorrect_lock_" #op, \ "held lock and object are not in the same allocation\n" \ "bpf_spin_lock at off=32 must be held for bpf_list_head" }, \ - { #test "_global_incorrect_lock_" #op, \ - "held lock and object are not in the same allocation\n" \ - "bpf_spin_lock at off=16 must be held for bpf_list_head" }, \ { #test "_map_incorrect_lock_" #op, \ "held lock and object are not in the same allocation\n" \ "bpf_spin_lock at off=0 must be held for bpf_list_head" }, \
@@ -45,10 +44,6 @@ static struct { TEST(kptr, push_back) TEST(kptr, pop_front) TEST(kptr, pop_back) - TEST(global, push_front) - TEST(global, push_back) - TEST(global, pop_front) - TEST(global, pop_back) TEST(map, push_front) TEST(map, push_back)TEST (map, pop_front)
@@ -58,12 +53,14 @@ static struct { TEST(inner_map, pop_front) TEST(inner_map, pop_back)#undef TEST +/* FIXME { "map_compat_kprobe", "tracing progs cannot use bpf_list_head yet" }, { "map_compat_kretprobe", "tracing progs cannot use bpf_list_head yet" }, { "map_compat_tp", "tracing progs cannot use bpf_list_head yet" }, { "map_compat_perf", "tracing progs cannot use bpf_list_head yet" }, { "map_compat_raw_tp", "tracing progs cannot use bpf_list_head yet" }, { "map_compat_raw_tp_w", "tracing progs cannot use bpf_list_head yet" }, +*/ { "obj_type_id_oor", "local type ID argument must be in range [0, U32_MAX]" }, { "obj_new_no_composite", "bpf_obj_new type ID argument must be of a struct" }, { "obj_new_no_struct", "bpf_obj_new type ID argument must be of a struct" },
@@ -78,6 +75,7 @@ static struct { { "direct_write_head", "direct access to bpf_list_head is disallowed" }, { "direct_read_node", "direct access to bpf_list_node is disallowed" }, { "direct_write_node", "direct access to bpf_list_node is disallowed" }, +/* FIXME { "write_after_push_front", "only read is supported" }, { "write_after_push_back", "only read is supported" }, { "use_after_unlock_push_front", "invalid mem access 'scalar'" },
@@ -94,8 +92,11 @@ static struct {{ "no_head_type", "bpf_list_head not found at offset=0" }, { "incorrect_head_var_off1", "R1 doesn't have constant offset" }, { "incorrect_head_var_off2", "variable ptr_ access var_off=(0x0; 0xffffffff) disallowed" }, +*/ { "incorrect_head_off1", "bpf_list_head not found at offset=17" }, +/* FIXME { "incorrect_head_off2", "bpf_list_head not found at offset=1" }, +*/ { "pop_front_off","15: (bf) r1 = r6 ; R1_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) " "R6_w=ptr_or_null_foo(id=4,ref_obj_id=4,off=40,imm=0) refs=2,4\n"
@@ -188,8 +189,10 @@ static void test_linked_list_success(int mode, bool leave_in_map) ret =bpf_prog_test_run_opts (bpf_program__fd (skel->progs.global_list_push_pop), &opts);ASSERT_OK (ret,"global_list_push_pop" );ASSERT_OK (opts.retval,"global_list_push_pop retval" ); +/* FIXME: if (!leave_in_map) clear_fields(skel->maps.data_A); +*/ if (mode ==PUSH_POP ) goto end;
@@ -210,8 +213,10 @@ static void test_linked_list_success(int mode, bool leave_in_map) ret =bpf_prog_test_run_opts (bpf_program__fd (skel->progs.global_list_push_pop_multiple), &opts);ASSERT_OK (ret,"global_list_push_pop_multiple" );ASSERT_OK (opts.retval,"global_list_push_pop_multiple retval" ); +/* FIXME: if (!leave_in_map) clear_fields(skel->maps.data_A); +*/ if (mode ==PUSH_POP_MULT ) goto end;
@@ -232,8 +237,10 @@ static void test_linked_list_success(int mode, bool leave_in_map) ret =bpf_prog_test_run_opts (bpf_program__fd (skel->progs.global_list_in_list), &opts);ASSERT_OK (ret,"global_list_in_list" );ASSERT_OK (opts.retval,"global_list_in_list retval" ); +/* FIXME: if (!leave_in_map) clear_fields(skel->maps.data_A); +*/ end:linked_list__destroy (skel); }
tools/testing/selftests/bpf/progs/linked_list.c +10 -1
@@ -291,7 +291,10 @@ int inner_map_list_push_pop(void *ctx) SEC("tc")int global_list_push_pop (void *ctx) { -return test_list_push_pop (&glock, &ghead); +/* FIXME: +* return test_list_push_pop(&glock, &ghead); +*/ +return 0 ; }SEC ("tc" )
@@ -327,10 +330,13 @@ int global_list_push_pop_multiple(void *ctx) {int ret; +/* FIXME: ret = list_push_pop_multiple(&glock, &ghead, false); if (ret) return ret; return list_push_pop_multiple(&glock, &ghead, true); +*/ +return 0 ; }SEC ("tc" )
@@ -364,7 +370,10 @@ int inner_map_list_in_list(void *ctx) SEC("tc")int global_list_in_list (void *ctx) { +/* FIXME return test_list_in_list(&glock, &ghead); +*/ +return 0 ; } char _license[] SEC("license") = "GPL";
tools/testing/selftests/bpf/progs/linked_list.h +2 -0
@@ -47,10 +47,12 @@ struct { }, }; +/* FIXME #define private(name) SEC(".data." #name) __hidden __attribute__((aligned(8))) private(A) struct bpf_spin_lock glock; private(A) struct bpf_list_head ghead __contains(foo, node); private(B) struct bpf_spin_lock glock2; +*/ #endif
tools/testing/selftests/bpf/progs/linked_list_fail.c +8 -8
@@ -59,10 +59,12 @@ CHECK(kptr, push_back, &f->head);CHECK (kptr, pop_front, &f->head);CHECK (kptr, pop_back, &f->head); +/* FIXME CHECK(global, push_front, &ghead); CHECK(global, push_back, &ghead); CHECK(global, pop_front, &ghead); CHECK(global, pop_back, &ghead); +*/ CHECK (map, push_front, &v->head);CHECK (map, push_back, &v->head);
@@ -89,23 +91,15 @@ CHECK(inner_map, pop_back, &iv->head);#define CHECK_OP (op) \ CHECK(kptr_kptr, op, &f1->lock, &f2->head); \ - CHECK(kptr_global, op, &f1->lock, &ghead); \ CHECK(kptr_map, op, &f1->lock, &v->head); \ CHECK(kptr_inner_map, op, &f1->lock, &iv->head); \ \ - CHECK(global_global, op, &glock2, &ghead); \ - CHECK(global_kptr, op, &glock, &f1->head); \ - CHECK(global_map, op, &glock, &v->head); \ - CHECK(global_inner_map, op, &glock, &iv->head); \ - \ CHECK(map_map, op, &v->lock, &v2->head); \ CHECK(map_kptr, op, &v->lock, &f2->head); \ - CHECK(map_global, op, &v->lock, &ghead); \ CHECK(map_inner_map, op, &v->lock, &iv->head); \ \ CHECK(inner_map_inner_map, op, &iv->lock, &iv2->head); \ CHECK(inner_map_kptr, op, &iv->lock, &f2->head); \ - CHECK(inner_map_global, op, &iv->lock, &ghead); \ CHECK(inner_map_map, op, &iv->lock, &v->head);CHECK_OP (push_front);
@@ -117,6 +111,7 @@ CHECK_OP(pop_back);#undef CHECK_OP#undef INIT +/* FIXME SEC("?kprobe/xyz") int map_compat_kprobe(void *ctx) {
@@ -158,6 +153,7 @@ int map_compat_raw_tp_w(void *ctx)bpf_list_push_front(&ghead, NULL); return 0; } +*/ SEC("?tc")int obj_type_id_oor (void *ctx)
@@ -303,6 +299,7 @@ int direct_write_node(void *ctx)return 0 ; } +/* FIXME static __always_inline int write_after_op(void (*push_op)(void *head, void *node)) {
@@ -506,6 +503,7 @@ int incorrect_head_var_off2(struct __sk_buff *ctx)return 0; } +*/ SEC("?tc")int incorrect_head_off1 (void *ctx)
@@ -529,6 +527,7 @@ int incorrect_head_off1(void *ctx)return 0 ; } +/* FIXME SEC("?tc") int incorrect_head_off2(void *ctx) {
@@ -545,6 +544,7 @@ int incorrect_head_off2(void *ctx)return 0; } +*/ static __always_inline int pop_ptr_off(void *(*op)(void *head))
Signature · 1 line
2.38.1
Re: [PATCH bpf-next v10 24/24] selftests/bpf: Temporarily disable linked list tests
Quoted 9 lines · Kumar Kartikeya Dwivedi
The latest clang nightly as of writing crashes with the given test case for BPF linked lists wherever global glock, ghead, glock2 are used, hence comment out the parts that cause the crash, and prepare this commit so that it can be reverted when the fix has been made. More context in [0].
[0]: https://lore.kernel.org/bpf/[email protected]
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
Hi All,
While looking at Dave's llvm fix I realized that there is a trivial workaround.
So I've reverted this commit and pushed a workaround:
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=59b9bdd725bc39a1e1a408a6aaffce8fdfd44366
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Hello:
This series was applied to bpf/bpf-next.git (master) by Alexei Starovoitov <[email protected]>:
Quoted 5 lines · On Fri, 18 Nov 2022 07:25:50 +0530 you wrote:
This series introduces user defined BPF objects of a type in program BTF. This allows BPF programs to allocate their own objects, build their own object hierarchies, and use the basic building blocks provided by BPF runtime to build their own data structures flexibly.
Quoted 7 lines
Then, we introduce the support for single ownership BPF linked lists, which can be put inside BPF maps, or allocated objects, and hold such allocated objects as elements. It works as an instrusive collection, which is done to allow making allocated objects part of multiple data structures at the same time in the future.
[...]
Here is the summary with links:
- [bpf-next,v10,01/24] bpf: Fix early return in map_check_btf
https://git.kernel.org/bpf/bpf-next/c/c237bfa5283a - [bpf-next,v10,02/24] bpf: Do btf_record_free outside map_free callback https://git.kernel.org/bpf/bpf-next/c/d7f5ef653c3d - [bpf-next,v10,03/24] bpf: Free inner_map_meta when btf_record_dup fails https://git.kernel.org/bpf/bpf-next/c/d48995723c9a - [bpf-next,v10,04/24] bpf: Populate field_offs for inner_map_meta https://git.kernel.org/bpf/bpf-next/c/f73e601aafb2 - [bpf-next,v10,05/24] bpf: Introduce allocated objects support https://git.kernel.org/bpf/bpf-next/c/282de143ead9 - [bpf-next,v10,06/24] bpf: Recognize lock and list fields in allocated objects https://git.kernel.org/bpf/bpf-next/c/8ffa5cc14213 - [bpf-next,v10,07/24] bpf: Verify ownership relationships for user BTF types https://git.kernel.org/bpf/bpf-next/c/865ce09a49d7 - [bpf-next,v10,08/24] bpf: Allow locking bpf_spin_lock in allocated objects https://git.kernel.org/bpf/bpf-next/c/4e814da0d599 - [bpf-next,v10,09/24] bpf: Allow locking bpf_spin_lock global variables https://git.kernel.org/bpf/bpf-next/c/d0d78c1df9b1 - [bpf-next,v10,10/24] bpf: Allow locking bpf_spin_lock in inner map values https://git.kernel.org/bpf/bpf-next/c/b7ff97925b55 - [bpf-next,v10,11/24] bpf: Rewrite kfunc argument handling https://git.kernel.org/bpf/bpf-next/c/00b85860feb8 - [bpf-next,v10,12/24] bpf: Support constant scalar arguments for kfuncs https://git.kernel.org/bpf/bpf-next/c/a50388dbb328 - [bpf-next,v10,13/24] bpf: Introduce bpf_obj_new https://git.kernel.org/bpf/bpf-next/c/958cf2e273f0 - [bpf-next,v10,14/24] bpf: Introduce bpf_obj_drop https://git.kernel.org/bpf/bpf-next/c/ac9f06050a35 - [bpf-next,v10,15/24] bpf: Permit NULL checking pointer with non-zero fixed offset https://git.kernel.org/bpf/bpf-next/c/df57f38a0d08 - [bpf-next,v10,16/24] bpf: Introduce single ownership BPF linked list API https://git.kernel.org/bpf/bpf-next/c/8cab76ec6349 - [bpf-next,v10,17/24] bpf: Add 'release on unlock' logic for bpf_list_push_{front,back} https://git.kernel.org/bpf/bpf-next/c/534e86bc6c66 - [bpf-next,v10,18/24] bpf: Add comments for map BTF matching requirement for bpf_list_head https://git.kernel.org/bpf/bpf-next/c/c22dfdd21592 - [bpf-next,v10,19/24] selftests/bpf: Add __contains macro to bpf_experimental.h https://git.kernel.org/bpf/bpf-next/c/64069c72b4b8 - [bpf-next,v10,20/24] selftests/bpf: Update spinlock selftest https://git.kernel.org/bpf/bpf-next/c/d85aedac4dc4 - [bpf-next,v10,21/24] selftests/bpf: Add failure test cases for spin lock pairing https://git.kernel.org/bpf/bpf-next/c/c48748aea4f8 - [bpf-next,v10,22/24] selftests/bpf: Add BPF linked list API tests https://git.kernel.org/bpf/bpf-next/c/300f19dcdb99 - [bpf-next,v10,23/24] selftests/bpf: Add BTF sanity tests https://git.kernel.org/bpf/bpf-next/c/dc2df7bf4c8a - [bpf-next,v10,24/24] selftests/bpf: Temporarily disable linked list tests https://git.kernel.org/bpf/bpf-next/c/0a2f85a1be43
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html