bpf: Add slab iterator and kfunc (v1)
Subsystems (6)
cover bpf: Add slab iterator and kfunc (v1)
Hello,
I'm proposing a new iterator and a kfunc for the slab memory allocator to get information of each kmem_cache like in /proc/slabinfo or /sys/kernel/slab. Maybe I need to call it kmem_cache iter but slab was short and easier to call. :)
My use case is perf lock contention tool which shows contended locks but many of them are not global locks and don't have symbols. If it can tranlate the address of the lock in a slab object to the name of the slab, it'd be much more useful.
I'm not aware of type information in slab yet, but I was told there's a work to associate BTF ID with it. It'd be definitely helpful to my use case. Probably we need another kfunc to get the start address of the object or the offset in the object from an address if the type info is available. But I want to start with a simple thing first.
The slab_iter iterates kmem_cache objects under slab_mutex and will be useful for userspace to prepare some work for specific slabs like setting up filters in advance. And the bpf_get_slab_cache() kfunc will return a pointer to a slab from the address of a lock. And the test code is to read from the iterator and make sure it finds a slab cache of the task_struct for the current task.
The code is available at 'bpf/slab-iter-v1' branch in
https://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
Thanks,
Namhyung
Namhyung Kim (3):
bpf: Add slab iterator
mm/bpf: Add bpf_get_slab_cache() kfunc
selftests/bpf: Add a test for slab_iter
include/linux/btf_ids.h | 1 + kernel/bpf/Makefile | 1 + kernel/bpf/helpers.c | 1 + kernel/bpf/slab_iter.c | 131 ++++++++++++++++++ mm/slab_common.c | 14 ++ .../selftests/bpf/prog_tests/slab_iter.c | 64 +++++++++ tools/testing/selftests/bpf/progs/bpf_iter.h | 7 + tools/testing/selftests/bpf/progs/slab_iter.c | 62 +++++++++ 8 files changed, 281 insertions(+) create mode 100644 kernel/bpf/slab_iter.c create mode 100644 tools/testing/selftests/bpf/prog_tests/slab_iter.c create mode 100644 tools/testing/selftests/bpf/progs/slab_iter.c
Signature · 1 line
2.46.0.792.g87dc391469-goog
PATCH 1/3 bpf: Add slab iterator
The new "slab" iterator will traverse the list of slab caches (kmem_cache) and call attached BPF programs for each entry. It should check the argument (ctx.s) if it's NULL before using it.
The iteration will be done with slab_mutex held but it'd break and return to user if the BPF program emits data to seq buffer more than the buffer size given by the user. IOW the whole iteration would be protected by the slab_mutex as long as it won't emit anything.
It includes the internal "mm/slab.h" header to access kmem_cache, slab_caches and slab_mutex. Hope it's ok to mm folks.
--- include/linux/btf_ids.h | 1 + kernel/bpf/Makefile | 1 + kernel/bpf/slab_iter.c | 131 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 133 insertions(+) create mode 100644 kernel/bpf/slab_iter.c
include/linux/btf_ids.h +1 -0
@@ -283,5 +283,6 @@ extern u32 btf_tracing_ids[];extern u32 bpf_cgroup_btf_id[];extern u32 bpf_local_storage_map_btf_id[];extern u32 btf_bpf_map_id[]; +extern u32 bpf_slab_btf_id[];#endif
kernel/bpf/Makefile +1 -0
@@ -52,3 +52,4 @@ obj-$(CONFIG_BPF_PRELOAD) += preload/ obj-$(CONFIG_BPF_SYSCALL) += relo_core.o obj-$(CONFIG_BPF_SYSCALL) += btf_iter.o obj-$(CONFIG_BPF_SYSCALL) += btf_relocate.o +obj-$(CONFIG_BPF_SYSCALL) += slab_iter.o
kernel/bpf/slab_iter.c new file +131 -0
131 lines not on this page · raw message
Signature · 1 line
2.46.0.792.g87dc391469-goog
PATCH 2/3 mm/bpf: Add bpf_get_slab_cache() kfunc
In reply to Namhyung KimThe bpf_get_slab_cache() is to get a slab cache information from a virtual address like virt_to_cache(). If the address is a pointer to a slab object, it'd return a valid kmem_cache pointer, otherwise NULL is returned.
It doesn't grab a reference count of the kmem_cache so the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.
--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 14 ++++++++++++++ 2 files changed, 15 insertions(+)
kernel/bpf/helpers.c +1 -0
@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_slab_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {
mm/slab_common.c +14 -0
@@ -1322,6 +1322,20 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_slab_cache(u64 addr) +{ +struct slab *slab; + + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +} + +__bpf_kfunc_end_defs (); +#endif /* CONFIG_BPF_SYSCALL */ +/* Tracepoints definitions. */ EXPORT_TRACEPOINT_SYMBOL (kmalloc);EXPORT_TRACEPOINT_SYMBOL (kmem_cache_alloc);
Signature · 1 line
2.46.0.792.g87dc391469-goog
PATCH 3/3 selftests/bpf: Add a test for slab_iter
In reply to Namhyung KimThe test traverses all slab caches using the slab_iter and check if current task's pointer is from "task_struct" slab cache.
--- .../selftests/bpf/prog_tests/slab_iter.c | 64 +++++++++++++++++++ tools/testing/selftests/bpf/progs/bpf_iter.h | 7 ++ tools/testing/selftests/bpf/progs/slab_iter.c | 62 ++++++++++++++++++ 3 files changed, 133 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/slab_iter.c create mode 100644 tools/testing/selftests/bpf/progs/slab_iter.c
tools/testing/selftests/bpf/prog_tests/slab_iter.c new file +64 -0
@@ -0,0 +1,64 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Google */ + +#include <test_progs.h> +#include <bpf/libbpf.h> +#include <bpf/btf.h> +#include "slab_iter.skel.h" + +static void test_slab_iter_check_task (struct slab_iter *skel) +{ +LIBBPF_OPTS (bpf_test_run_opts, opts, + .flags = BPF_F_TEST_RUN_ON_CPU, + ); +int prog_fd =bpf_program__fd (skel->progs.check_task_struct); + +/* get task_struct and check it if's from a slab cache */ +bpf_prog_test_run_opts (prog_fd, &opts); + +/* the BPF program should set 'found' variable */ +ASSERT_EQ (skel->bss->found,1 ,"found task_struct" ); +} + +void test_slab_iter (void ) +{ +DECLARE_LIBBPF_OPTS (bpf_iter_attach_opts, opts); +struct slab_iter *skel =NULL ; +union bpf_iter_link_info linfo = {}; +struct bpf_link *link; +char buf[1024 ]; +int iter_fd; + + skel =slab_iter__open_and_load (); +if (!ASSERT_OK_PTR (skel,"slab_iter__open_and_load" )) +return ; + + opts.link_info = &linfo; + opts.link_info_len =sizeof (linfo); + + link =bpf_program__attach_iter (skel->progs.slab_info_collector, &opts); +if (!ASSERT_OK_PTR (link,"attach_iter" )) + goto destroy; + + iter_fd =bpf_iter_create (bpf_link__fd (link)); +if (!ASSERT_GE (iter_fd,0 ,"iter_create" )) + goto free_link; + +memset (buf,0 ,sizeof (buf)); +while (read (iter_fd, buf,sizeof (buf) >0 )) { +/* read out all contents */ +printf ("%s" , buf); + } + +/* next reads should return 0 */ +ASSERT_EQ (read (iter_fd, buf,sizeof (buf)),0 ,"read" ); + +test_slab_iter_check_task (skel); + +close (iter_fd); + +free_link: +bpf_link__destroy (link); +destroy: +slab_iter__destroy (skel); +}
tools/testing/selftests/bpf/progs/bpf_iter.h +7 -0
@@ -24,6 +24,7 @@#define BTF_F_PTR_RAW BTF_F_PTR_RAW___not_used#define BTF_F_ZERO BTF_F_ZERO___not_used#define bpf_iter__ksym bpf_iter__ksym___not_used +#define bpf_iter__slab bpf_iter__slab___not_used#include "vmlinux.h" #undef bpf_iter_meta#undef bpf_iter__bpf_map
@@ -48,6 +49,7 @@#undef BTF_F_PTR_RAW#undef BTF_F_ZERO#undef bpf_iter__ksym +#undef bpf_iter__slabstruct bpf_iter_meta {struct seq_file *seq;
@@ -165,3 +167,8 @@ struct bpf_iter__ksym {struct bpf_iter_meta *meta;struct kallsym_iter *ksym; }; + +struct bpf_iter__slab { +struct bpf_iter_meta *meta; +struct kmem_cache *s; +} __attribute__((preserve_access_index));
tools/testing/selftests/bpf/progs/slab_iter.c new file +62 -0
62 lines not on this page · raw message
Signature · 1 line
2.46.0.792.g87dc391469-goog
Quoted 5 lines · Namhyung Kim
Hello,
I'm proposing a new iterator and a kfunc for the slab memory allocator to get information of each
kmem_cachelike in/proc/slabinfoor/sys/kernel/slab.
Hello, Namhyung!
I personally like the idea very much. With a growing number of kmem_caches /proc/slabinfo getting close to it's limit, so having a more flexible interface makes a lot of sense.
Quoted 2 lines
Maybe I need to call it
kmem_cacheiter but slab
was short and easier to call. :)
I'd personally prefer kmem_cache or slab_cache, just in case somebody later would propose an iterator over individual slab objects within a kmem_cache.
Thanks!
Quoted 8 lines · Roman Gushchin
Quoted 5 lines · Namhyung Kim
Hello,
I'm proposing a new iterator and a kfunc for the slab memory allocator to get information of each
kmem_cachelike in/proc/slabinfoor/sys/kernel/slab.Hello, Namhyung!
Hello Roman!
Quoted 10 lines
I personally like the idea very much. With a growing number of
kmem_caches/proc/slabinfogetting close to it's limit, so having a more flexible interface makes a lot of sense.Quoted 2 lines
Maybe I need to call it
kmem_cacheiter but slab
was short and easier to call. :)I'd personally prefer
kmem_cacheorslab_cache, just in case somebody later would propose an iterator over individual slab objects within akmem_cache.
I think we can add a parameter to limit or extend the functionality like task iter and cgroup iter. But I'm not sure we need to use a different name for that. Anyway I'm ok to rename it kmem_cache iter.
Quoted 2 lines
Thanks for your review!
Namhyung
Vlastimil Babka
Acked-byQuoted 13 lines · Roman Gushchin
Quoted 5 lines · Namhyung Kim
Hello,
I'm proposing a new iterator and a kfunc for the slab memory allocator to get information of each
kmem_cachelike in/proc/slabinfoor/sys/kernel/slab.Hello, Namhyung!
I personally like the idea very much. With a growing number of
kmem_caches/proc/slabinfogetting close to it's limit, so having a more flexible interface makes a lot of sense.
Quoted 5 lines
Quoted 2 lines
Maybe I need to call it
kmem_cacheiter but slab
was short and easier to call. :)I'd personally prefer
kmem_cacheorslab_cache, just in case somebody later would propose an iterator over individual slab objects within akmem_cache.
Yeah it would be better. Even though I doubt an iterator of slabs or objects would be really feasible.
Quoted 1 line
Quoted 2 lines
Thanks!