bpf: Add kmem_cache iterator and kfunc
Subsystems (7)
v4 cover bpf: Add kmem_cache iterator and kfunc
Hello,
I'm proposing a new iterator and a kfunc for the slab memory allocator to get information of each kmem_cache like in /proc/slabinfo or /sys/kernel/slab in more flexible way.
v4 changes)
* skip kmem_cache_destroy() in kmem_cache_iter_seq_stop() if possible (Vlastimil)
* fix a bug in the kmem_cache_iter_seq_start() for the last entry
v3: https://lore.kernel.org/lkml/[email protected]/
* rework kmem_cache_iter not to hold slab_mutex when running BPF (Alexei)
* add virt_addr_valid() check (Alexei)
* fix random test failure by running test with the current task (Hyeonggon)
v2: https://lore.kernel.org/lkml/[email protected]/
* rename it to "kmem_cache_iter"
* fix a build issue
* add Acked-by's from Roman and Vlastimil (Thanks!)
* add error codes in the test for debugging
v1: https://lore.kernel.org/lkml/[email protected]/
My use case is perf lock contention tool which shows contended locks but many of them are not global locks and don't have symbols. If it can tranlate the address of the lock in a slab object to the name of the slab, it'd be much more useful.
I'm not aware of type information in slab yet, but I was told there's a work to associate BTF ID with it. It'd be definitely helpful to my use case. Probably we need another kfunc to get the start address of the object or the offset in the object from an address if the type info is available. But I want to start with a simple thing first.
The kmem_cache_iter iterates kmem_cache objects under slab_mutex and will be useful for userspace to prepare some work for specific slabs like setting up filters in advance. And the bpf_get_kmem_cache() kfunc will return a pointer to a slab from the address of a lock. And the test code is to read from the iterator and make sure it finds a slab cache of the task_struct for the current task.
The code is available at 'bpf/slab-iter-v4' branch in
https://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
Thanks,
Namhyung
Namhyung Kim (3):
bpf: Add kmem_cache iterator
mm/bpf: Add bpf_get_kmem_cache() kfunc
selftests/bpf: Add a test for kmem_cache_iter
include/linux/btf_ids.h | 1 + kernel/bpf/Makefile | 1 + kernel/bpf/helpers.c | 1 + kernel/bpf/kmem_cache_iter.c | 174 ++++++++++++++++++ mm/slab_common.c | 19 ++ .../bpf/prog_tests/kmem_cache_iter.c | 64 +++++++ tools/testing/selftests/bpf/progs/bpf_iter.h | 7 + .../selftests/bpf/progs/kmem_cache_iter.c | 66 +++++++ 8 files changed, 333 insertions(+) create mode 100644 kernel/bpf/kmem_cache_iter.c create mode 100644 tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c create mode 100644 tools/testing/selftests/bpf/progs/kmem_cache_iter.c
base-commit: 9502a7de5a61bec3bda841a830560c5d6d40ecac
Signature · 1 line
2.46.1.824.gd892dcdcdd-goog
PATCH v4 1/3 bpf: Add kmem_cache iterator
The new "kmem_cache" iterator will traverse the list of slab caches and call attached BPF programs for each entry. It should check the argument (ctx.s) if it's NULL before using it.
Now the iteration grabs the slab_mutex only if it traverse the list and releases the mutex when it runs the BPF program. The kmem_cache entry is protected by a refcount during the execution.
It includes the internal "mm/slab.h" header to access kmem_cache, slab_caches and slab_mutex. Hope it's ok to mm folks.
---
I've removed the Acked-by's from Roman and Vlastimil since it's changed not to hold the slab_mutex and to manage the refcount. Please review this change again!
include/linux/btf_ids.h | 1 + kernel/bpf/Makefile | 1 + kernel/bpf/kmem_cache_iter.c | 174 +++++++++++++++++++++++++++++++++++ 3 files changed, 176 insertions(+) create mode 100644 kernel/bpf/kmem_cache_iter.c
include/linux/btf_ids.h +1 -0
@@ -283,5 +283,6 @@ extern u32 btf_tracing_ids[];extern u32 bpf_cgroup_btf_id[];extern u32 bpf_local_storage_map_btf_id[];extern u32 btf_bpf_map_id[]; +extern u32 bpf_kmem_cache_btf_id[];#endif
kernel/bpf/Makefile +1 -0
@@ -52,3 +52,4 @@ obj-$(CONFIG_BPF_PRELOAD) += preload/ obj-$(CONFIG_BPF_SYSCALL) += relo_core.o obj-$(CONFIG_BPF_SYSCALL) += btf_iter.o obj-$(CONFIG_BPF_SYSCALL) += btf_relocate.o +obj-$(CONFIG_BPF_SYSCALL) += kmem_cache_iter.o
kernel/bpf/kmem_cache_iter.c new file +174 -0
174 lines not on this page · raw message
Signature · 1 line
2.46.1.824.gd892dcdcdd-goog
Re: [PATCH v4 bpf-next 1/3] bpf: Add kmem_cache iterator
Quoted 4 lines · Namhyung Kim
The new "
kmem_cache" iterator will traverse the list of slab caches and call attached BPF programs for each entry. It should check the argument (ctx.s) if it's NULL before using it.
Quoted 8 lines
Now the iteration grabs the
slab_mutexonly if it traverse the list and releases the mutex when it runs the BPF program. Thekmem_cacheentry is protected by a refcount during the execution.It includes the internal "
mm/slab.h" header to accesskmem_cache,slab_cachesandslab_mutex. Hope it's ok to mm folks.Signed-off-by Namhyung Kim <[email protected]>
Song Liu
reviewerRe: [PATCH v4 bpf-next 1/3] bpf: Add kmem_cache iterator
In reply to Namhyung KimQuoted 1 line · Namhyung Kim
[...]
kernel/bpf/kmem_cache_iter.c line 25
Quoted 5 lines
kernel/bpf/kmem_cache_iter.c new file +5
-0+ +mutex_lock (&slab_mutex); + +/* +* Find an entry at the given position in the slab_caches list instead
Nit: style of multi-line comment: "/* Find ...".
Quoted 64 lines
kernel/bpf/kmem_cache_iter.c new file +64
-0+* of keeping a reference (of the last visited entry, if any) out of +* slab_mutex. It might miss something if one is deleted in the middle +* while it releases the lock. But it should be rare and there's not +* much we can do about it. +*/ +list_for_each_entry (s, &slab_caches, list) { +if (cnt == *pos) { +/* +* Make sure this entry remains in the list by getting +* a new reference count. Note that boot_cache entries +* have a negative refcount, so don't touch them. +*/ +if (s->refcount >0 ) + s->refcount++; + found = true; +break ; + } + cnt++; + } +mutex_unlock (&slab_mutex); + +if (!found) +return NULL ; + + ++*pos; +return s; +} + +static void kmem_cache_iter_seq_stop (struct seq_file *seq,void *v) +{ +struct bpf_iter_meta meta; +struct bpf_iter__kmem_cache ctx = { + .meta = &meta, + .s = v, + }; +struct bpf_prog *prog; +bool destroy = false; + + meta.seq = seq; + prog =bpf_iter_get_info (&meta, true); +if (prog) +bpf_iter_run_prog (prog, &ctx); + +if (ctx.s ==NULL ) +return ; + +mutex_lock (&slab_mutex); + +/* Skip kmem_cache_destroy() for active entries */ +if (ctx.s->refcount >1 ) + ctx.s->refcount--; +else if (ctx.s->refcount ==1 ) + destroy = true; + +mutex_unlock (&slab_mutex); + +if (destroy) +kmem_cache_destroy (ctx.s); +} + +static void *kmem_cache_iter_seq_next (struct seq_file *seq,void *v,loff_t *pos) +{ +struct kmem_cache *s = v; +struct kmem_cache *next =NULL ;
kernel/bpf/kmem_cache_iter.c line 99
Quoted 10 lines
kernel/bpf/kmem_cache_iter.c new file +10
-0+bool destroy = false; + + ++*pos; + +mutex_lock (&slab_mutex); + + if (list_last_entry(&slab_caches, struct kmem_cache, list) != s) { + next =list_next_entry (s, list); +if (next->refcount >0 ) + next->refcount++;
What if next->refcount <=0? Shall we find next of next?
Quoted 15 lines
kernel/bpf/kmem_cache_iter.c new file +15
-0+ } + +/* Skip kmem_cache_destroy() for active entries */ +if (s->refcount >1 ) + s->refcount--; +else if (s->refcount ==1 ) + destroy = true; + +mutex_unlock (&slab_mutex); + +if (destroy) +kmem_cache_destroy (s); + +return next; +}
[...]
Re: [PATCH v4 bpf-next 1/3] bpf: Add kmem_cache iterator
Hi Song,
Quoted 10 lines · Song Liu
Quoted 1 line · Namhyung Kim
[...]
kernel/bpf/kmem_cache_iter.c line 25
Quoted 5 lines
kernel/bpf/kmem_cache_iter.c new file +5
-0+ +mutex_lock (&slab_mutex); + +/* +* Find an entry at the given position in the slab_caches list instead Nit: style of multi-line comment: "/* Find ...".
Ok, will update.
Quoted 69 lines
Quoted 69 lines
kernel/bpf/kmem_cache_iter.c new file +68
-0+* of keeping a reference (of the last visited entry, if any) out of +* slab_mutex. It might miss something if one is deleted in the middle +* while it releases the lock. But it should be rare and there's not +* much we can do about it. +*/ +list_for_each_entry (s, &slab_caches, list) { +if (cnt == *pos) { +/* +* Make sure this entry remains in the list by getting +* a new reference count. Note that boot_cache entries +* have a negative refcount, so don't touch them. +*/ +if (s->refcount >0 ) + s->refcount++; + found = true; +break ; + } + cnt++; + } +mutex_unlock (&slab_mutex); + +if (!found) +return NULL ; + + ++*pos; +return s; +} + +static void kmem_cache_iter_seq_stop (struct seq_file *seq,void *v) +{ +struct bpf_iter_meta meta; +struct bpf_iter__kmem_cache ctx = { + .meta = &meta, + .s = v, + }; +struct bpf_prog *prog; +bool destroy = false; + + meta.seq = seq; + prog =bpf_iter_get_info (&meta, true); +if (prog) +bpf_iter_run_prog (prog, &ctx); + +if (ctx.s ==NULL ) +return ; + +mutex_lock (&slab_mutex); + +/* Skip kmem_cache_destroy() for active entries */ +if (ctx.s->refcount >1 ) + ctx.s->refcount--; +else if (ctx.s->refcount ==1 ) + destroy = true; + +mutex_unlock (&slab_mutex); + +if (destroy) +kmem_cache_destroy (ctx.s); +} + +static void *kmem_cache_iter_seq_next (struct seq_file *seq,void *v,loff_t *pos) +{ +struct kmem_cache *s = v; +struct kmem_cache *next =NULL ; +bool destroy = false; + + ++*pos; +
Quoted 8 lines
kernel/bpf/kmem_cache_iter.c line 99
Quoted 6 lines
kernel/bpf/kmem_cache_iter.c new file +6
-0+mutex_lock (&slab_mutex); + + if (list_last_entry(&slab_caches, struct kmem_cache, list) != s) { + next =list_next_entry (s, list); +if (next->refcount >0 ) + next->refcount++;What if
next->refcount<=0? Shall we find next of next?
The slab_mutex should protect refcount == 0 case so it won't see that. The negative refcount means it's a boot_cache and we shouldn't touch the refcount.
Thanks,
Namhyung
Quoted 17 lines
Quoted 16 lines
kernel/bpf/kmem_cache_iter.c new file +15
-0+ } + +/* Skip kmem_cache_destroy() for active entries */ +if (s->refcount >1 ) + s->refcount--; +else if (s->refcount ==1 ) + destroy = true; + +mutex_unlock (&slab_mutex); + +if (destroy) +kmem_cache_destroy (s); + +return next; +}[...]
Song Liu
reviewerRe: [PATCH v4 bpf-next 1/3] bpf: Add kmem_cache iterator
Quoted 4 lines · Namhyung Kim
Hi Song,
On Fri, Oct 04, 2024 at 01:33:19PM -0700, Song Liu wrote:
[...]
Quoted 18 lines
Quoted 9 lines
Quoted 9 lines
kernel/bpf/kmem_cache_iter.c new file +9
-0+ +static void *kmem_cache_iter_seq_next (struct seq_file *seq,void *v,loff_t *pos) +{ +struct kmem_cache *s = v; +struct kmem_cache *next =NULL ; +bool destroy = false; + + ++*pos; +Quoted 8 lines
kernel/bpf/kmem_cache_iter.c line 99
Quoted 6 lines
kernel/bpf/kmem_cache_iter.c new file +6
-0+mutex_lock (&slab_mutex); + + if (list_last_entry(&slab_caches, struct kmem_cache, list) != s) { + next =list_next_entry (s, list); +if (next->refcount >0 ) + next->refcount++;What if
next->refcount<=0? Shall we find next of next?
Quoted 3 lines
The
slab_mutexshould protect refcount == 0 case so it won't see that. The negative refcount means it's aboot_cacheand we shouldn't touch the refcount.
I see. Thanks for the explanation!
Please add a comment here, and maybe also add
WARN_ON_ONCE(next ->refcount == 0).
Song
Re: [PATCH v4 bpf-next 1/3] bpf: Add kmem_cache iterator
Quoted 28 lines · Song Liu
Quoted 4 lines · Namhyung Kim
Hi Song,
On Fri, Oct 04, 2024 at 01:33:19PM -0700, Song Liu wrote:
[...]
Quoted 18 lines
Quoted 9 lines
Quoted 9 lines
kernel/bpf/kmem_cache_iter.c new file +9
-0+ +static void *kmem_cache_iter_seq_next (struct seq_file *seq,void *v,loff_t *pos) +{ +struct kmem_cache *s = v; +struct kmem_cache *next =NULL ; +bool destroy = false; + + ++*pos; +Quoted 8 lines
kernel/bpf/kmem_cache_iter.c line 99
Quoted 6 lines
kernel/bpf/kmem_cache_iter.c new file +6
-0+mutex_lock (&slab_mutex); + + if (list_last_entry(&slab_caches, struct kmem_cache, list) != s) { + next =list_next_entry (s, list); +if (next->refcount >0 ) + next->refcount++;What if
next->refcount<=0? Shall we find next of next?Quoted 3 lines
The
slab_mutexshould protect refcount == 0 case so it won't see that. The negative refcount means it's aboot_cacheand we shouldn't touch the refcount.
Quoted 5 lines
I see. Thanks for the explanation!
Please add a comment here, and maybe also add
WARN_ON_ONCE(next ->refcount == 0).
Sure, thanks for your review!
Namhyung
Song Liu
reviewerRe: [PATCH v4 bpf-next 1/3] bpf: Add kmem_cache iterator
In reply to Namhyung KimQuoted 1 line · Namhyung Kim
[...]
Quoted 26 lines
kernel/bpf/kmem_cache_iter.c new file +26
-0+ +static void *kmem_cache_iter_seq_start (struct seq_file *seq,loff_t *pos) +{ +loff_t cnt =0 ; +bool found = false; +struct kmem_cache *s; + +mutex_lock (&slab_mutex); + +/* +* Find an entry at the given position in the slab_caches list instead +* of keeping a reference (of the last visited entry, if any) out of +* slab_mutex. It might miss something if one is deleted in the middle +* while it releases the lock. But it should be rare and there's not +* much we can do about it. +*/ +list_for_each_entry (s, &slab_caches, list) { +if (cnt == *pos) { +/* +* Make sure this entry remains in the list by getting +* a new reference count. Note that boot_cache entries +* have a negative refcount, so don't touch them. +*/ +if (s->refcount >0 ) + s->refcount++; + found = true;
kernel/bpf/kmem_cache_iter.c line 50
Quoted 10 lines
kernel/bpf/kmem_cache_iter.c new file +10
-0+break ; + } + cnt++; + } +mutex_unlock (&slab_mutex); + +if (!found) +return NULL ; + + ++*pos;
This should be
if (*pos == 0)
++*pos;
Quoted 4 lines
kernel/bpf/kmem_cache_iter.c new file +4
-0+return s; +} + +static void kmem_cache_iter_seq_stop (struct seq_file *seq,void *v)
[...]
Re: [PATCH v4 bpf-next 1/3] bpf: Add kmem_cache iterator
Quoted 42 lines · Song Liu
Quoted 1 line · Namhyung Kim
[...]
Quoted 26 lines
kernel/bpf/kmem_cache_iter.c new file +26
-0+ +static void *kmem_cache_iter_seq_start (struct seq_file *seq,loff_t *pos) +{ +loff_t cnt =0 ; +bool found = false; +struct kmem_cache *s; + +mutex_lock (&slab_mutex); + +/* +* Find an entry at the given position in the slab_caches list instead +* of keeping a reference (of the last visited entry, if any) out of +* slab_mutex. It might miss something if one is deleted in the middle +* while it releases the lock. But it should be rare and there's not +* much we can do about it. +*/ +list_for_each_entry (s, &slab_caches, list) { +if (cnt == *pos) { +/* +* Make sure this entry remains in the list by getting +* a new reference count. Note that boot_cache entries +* have a negative refcount, so don't touch them. +*/ +if (s->refcount >0 ) + s->refcount++; + found = true;kernel/bpf/kmem_cache_iter.c line 50
Quoted 10 lines
kernel/bpf/kmem_cache_iter.c new file +10
-0+break ; + } + cnt++; + } +mutex_unlock (&slab_mutex); + +if (!found) +return NULL ; + + ++*pos;This should be
Quoted 2 lines
if (*pos == 0)
++*pos;
Oh, I thought there's check for seq->count after the seq->op->show() for the ->start(). I need to check this logic again, thanks for pointing this out.
Thanks,
Namhyung
Quoted 6 lines
Quoted 5 lines
kernel/bpf/kmem_cache_iter.c new file +4
-0+return s; +} + +static void kmem_cache_iter_seq_stop (struct seq_file *seq,void *v)[...]
PATCH v4 2/3 mm/bpf: Add bpf_get_kmem_cache() kfunc
In reply to Namhyung KimThe bpf_get_kmem_cache() is to get a slab cache information from a virtual address like virt_to_cache(). If the address is a pointer to a slab object, it'd return a valid kmem_cache pointer, otherwise NULL is returned.
It doesn't grab a reference count of the kmem_cache so the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.
--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)
kernel/bpf/helpers.c +1 -0
@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {
mm/slab_common.c +19 -0
@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ; + + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +} + +__bpf_kfunc_end_defs (); +#endif /* CONFIG_BPF_SYSCALL */ +/* Tracepoints definitions. */ EXPORT_TRACEPOINT_SYMBOL (kmalloc);EXPORT_TRACEPOINT_SYMBOL (kmem_cache_alloc);
Signature · 1 line
2.46.1.824.gd892dcdcdd-goog
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 38 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +0
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize);
mm/slab_common.c size_t ksize(const void *objp) · line 1334
Quoted 10 lines
mm/slab_common.c +10
-0+#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr))
Hmm.. 32-bit systems don't like this. Is it ok to change the type of the parameter (addr) to 'unsigned long'? Or do you want to keep it as u64 and add a cast here?
Thanks,
Namhyung
Quoted 15 lines
mm/slab_common.c +9
-0+return NULL ; + + slab =virt_to_slab ((void *)(long )addr); + return slab ? slab->slab_cache : NULL; +} + +__bpf_kfunc_end_defs (); +#endif /* CONFIG_BPF_SYSCALL */ +/* Tracepoints definitions. */ EXPORT_TRACEPOINT_SYMBOL (kmalloc);EXPORT_TRACEPOINT_SYMBOL (kmem_cache_alloc);Signature · 1 line
2.46.1.824.gd892dcdcdd-goog
kernel test robot
build botRe: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
In reply to Namhyung Kimkernel test robot noticed the following build warnings:
Hi Namhyung,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 9502a7de5a61bec3bda841a830560c5d6d40ecac]
url: https://github.com/intel-lab-lkp/linux/commits/Namhyung-Kim/bpf-Add-kmem_cache-iterator/20241003-021144 base: 9502a7de5a61bec3bda841a830560c5d6d40ecac patch link: https://lore.kernel.org/r/20241002180956.1781008-3-namhyung%40kernel.org patch subject: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc config: mips-randconfig-r064-20241004 (https://download.01.org/0day-ci/archive/20241004/[email protected]/config) compiler: mipsel-linux-gcc (GCC) 13.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241004/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
In file included from include/linux/mm_types_task.h:13,
from include/linux/mm_types.h:5, from include/linux/mmzone.h:22, from include/linux/gfp.h:7, from include/linux/slab.h:16, from mm/slab_common.c:7:
mm/slab_common.c: In function 'bpf_get_kmem_cache':
>> arch/mips/include/asm/page.h:218:27: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
218 | __virt_addr_valid((const volatile void *) (kaddr))
| ^
mm/slab_common.c:1334:14: note: in expansion of macro 'virt_addr_valid'
1334 | if (!virt_addr_valid(addr))
| ^~~~~~~~~~~~~~~vim +218 arch/mips/include/asm/page.h
d3ce88431892b7 arch/mips/include/asm/page.h Ralf Baechle 2012-12-28 215 31875a5432248e arch/mips/include/asm/page.h Paul Burton 2019-05-28 216 extern bool __virt_addr_valid(const volatile void *kaddr); d3ce88431892b7 arch/mips/include/asm/page.h Ralf Baechle 2012-12-28 217 #define virt_addr_valid(kaddr) \ d3ce88431892b7 arch/mips/include/asm/page.h Ralf Baechle 2012-12-28 @218 __virt_addr_valid((const volatile void *) (kaddr))
^1da177e4c3f41 include/asm-mips/page.h Linus Torvalds 2005-04-16 219
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
In reply to Namhyung Kimkernel test robot noticed the following build warnings:
Hi Namhyung,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 9502a7de5a61bec3bda841a830560c5d6d40ecac]
url: https://github.com/intel-lab-lkp/linux/commits/Namhyung-Kim/bpf-Add-kmem_cache-iterator/20241003-021144 base: 9502a7de5a61bec3bda841a830560c5d6d40ecac patch link: https://lore.kernel.org/r/20241002180956.1781008-3-namhyung%40kernel.org patch subject: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc config: arm64-defconfig (https://download.01.org/0day-ci/archive/20241005/[email protected]/config) compiler: aarch64-linux-gcc (GCC) 13.3.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241005/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
In file included from arch/arm64/include/asm/thread_info.h:17,
from include/linux/thread_info.h:60, from arch/arm64/include/asm/preempt.h:6, from include/linux/preempt.h:79, from include/linux/spinlock.h:56, from include/linux/mmzone.h:8,
from include/linux/gfp.h:7,
from include/linux/slab.h:16,
from mm/slab_common.c:7:
mm/slab_common.c: In function 'bpf_get_kmem_cache':
>> arch/arm64/include/asm/memory.h:427:66: warning: passing argument 1 of 'virt_to_pfn' makes pointer from integer without a cast [-Wint-conversion]
427 | __is_lm_address(__addr) && pfn_is_map_memory(virt_to_pfn(__addr)); \
| ^~~~~~
| |
| u64 {aka long long unsigned int}
mm/slab_common.c:1334:14: note: in expansion of macro 'virt_addr_valid'
1334 | if (!virt_addr_valid(addr))
| ^~~~~~~~~~~~~~~ arch/arm64/include/asm/memory.h:382:53: note: expected 'const void *' but argument is of type 'u64' {aka 'long long unsigned int'}
382 | static inline unsigned long virt_to_pfn(const void *kaddr)
| ~~~~~~~~~~~~^~~~~
vim +/virt_to_pfn +427 arch/arm64/include/asm/memory.h
4f04d8f0054511 Catalin Marinas 2012-03-05 424
68dd8ef321626f Will Deacon 2019-08-13 425 #define virt_addr_valid(addr) ({ \
91cb2c8b072e00 Catalin Marinas 2021-02-01 426 __typeof__(addr) __addr = __tag_reset(addr); \
873ba463914cf4 Mike Rapoport 2021-06-30 @427 __is_lm_address(__addr) && pfn_is_map_memory(virt_to_pfn(__addr)); \
68dd8ef321626f Will Deacon 2019-08-13 428 })4f04d8f0054511 Catalin Marinas 2012-03-05 429
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Song Liu
reviewerRe: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
In reply to Namhyung KimQuoted 44 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +5
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); +
mm/slab_common.c size_t ksize(const void *objp) · line 1339
Quoted 10 lines
mm/slab_common.c +10
-0+__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ; + + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +}
Do we need to hold a refcount to the slab_cache? Given we make this kfunc available everywhere, including sleepable contexts, I think it is necessary.
Thanks
Song
Quoted 10 lines
mm/slab_common.c +4
-1+ +__bpf_kfunc_end_defs (); +#endif /* CONFIG_BPF_SYSCALL */ +/* Tracepoints definitions. */ EXPORT_TRACEPOINT_SYMBOL (kmalloc);EXPORT_TRACEPOINT_SYMBOL (kmem_cache_alloc); --2.46.1.824.gd892dcdcdd-goog
Roman Gushchin
reviewerRe: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 51 lines · Song Liu
Quoted 50 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +11
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ;
Quoted 8 lines
mm/slab_common.c size_t ksize(const void *objp) · line 1339
Quoted 4 lines
mm/slab_common.c +4
-0+ + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +}Do we need to hold a refcount to the
slab_cache? Given we make this kfunc available everywhere, including sleepable contexts, I think it is necessary.
It's a really good question.
If the callee somehow owns the slab object, as in the example provided in the series (current task), it's not necessarily.
If a user can pass a random address, you're right, we need to grab the slab_cache's refcnt. But then we also can't guarantee that the object still belongs to the same slab_cache, the function becomes racy by the definition.
Song Liu
reviewerRe: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 64 lines · Roman Gushchin
Quoted 51 lines · Song Liu
Quoted 50 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +11
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ;Quoted 8 lines
mm/slab_common.c size_t ksize(const void *objp) · line 1339
Quoted 4 lines
mm/slab_common.c +4
-0+ + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +}Do we need to hold a refcount to the
slab_cache? Given we make this kfunc available everywhere, including sleepable contexts, I think it is necessary.It's a really good question.
Quoted 7 lines
If the callee somehow owns the slab object, as in the example provided in the series (current task), it's not necessarily.
If a user can pass a random address, you're right, we need to grab the slab_cache's refcnt. But then we also can't guarantee that the object still belongs to the same
slab_cache, the function becomes racy by the definition.
To be safe, we can limit the kfunc to sleepable context only. Then we can lock slab_mutex for virt_to_slab, and hold a refcount to slab_cache. We will need a KF_RELEASE kfunc to release the refcount later.
IIUC, this limitation (sleepable context only) shouldn't be a problem for perf use case?
Thanks,
Song
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 73 lines · Song Liu
Quoted 64 lines · Roman Gushchin
Quoted 51 lines · Song Liu
Quoted 50 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +11
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ;Quoted 8 lines
mm/slab_common.c size_t ksize(const void *objp) · line 1339
Quoted 4 lines
mm/slab_common.c +4
-0+ + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +}Do we need to hold a refcount to the
slab_cache? Given we make this kfunc available everywhere, including sleepable contexts, I think it is necessary.It's a really good question.
Quoted 7 lines
If the callee somehow owns the slab object, as in the example provided in the series (current task), it's not necessarily.
If a user can pass a random address, you're right, we need to grab the slab_cache's refcnt. But then we also can't guarantee that the object still belongs to the same
slab_cache, the function becomes racy by the definition.
Quoted 4 lines
To be safe, we can limit the kfunc to sleepable context only. Then we can lock
slab_mutexforvirt_to_slab, and hold a refcount toslab_cache. We will need aKF_RELEASEkfunc to release the refcount later.
Then it needs to call kmem_cache_destroy() for release which contains rcu_barrier. :(
Quoted 3 lines
IIUC, this limitation (sleepable context only) shouldn't be a problem for perf use case?
No, it would be called from the lock contention path including spinlocks. :(
Can we limit it to non-sleepable ctx and not to pass arbtrary address somehow (or not to save the result pointer)?
Thanks,
Namhyung
Song Liu
reviewerRe: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 84 lines · Namhyung Kim
Quoted 73 lines · Song Liu
Quoted 64 lines · Roman Gushchin
Quoted 51 lines · Song Liu
Quoted 50 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +11
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ;Quoted 8 lines
mm/slab_common.c size_t ksize(const void *objp) · line 1339
Quoted 4 lines
mm/slab_common.c +4
-0+ + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +}Do we need to hold a refcount to the
slab_cache? Given we make this kfunc available everywhere, including sleepable contexts, I think it is necessary.It's a really good question.
Quoted 7 lines
If the callee somehow owns the slab object, as in the example provided in the series (current task), it's not necessarily.
If a user can pass a random address, you're right, we need to grab the slab_cache's refcnt. But then we also can't guarantee that the object still belongs to the same
slab_cache, the function becomes racy by the definition.Quoted 4 lines
To be safe, we can limit the kfunc to sleepable context only. Then we can lock
slab_mutexforvirt_to_slab, and hold a refcount toslab_cache. We will need aKF_RELEASEkfunc to release the refcount later.Then it needs to call
kmem_cache_destroy()for release which containsrcu_barrier. :(Quoted 1 line
Quoted 8 lines
Quoted 2 lines
IIUC, this limitation (sleepable context only) shouldn't be a problem for perf use case?
No, it would be called from the lock contention path including spinlocks. :(
Can we limit it to non-sleepable ctx and not to pass arbtrary address somehow (or not to save the result pointer)?
I hacked something like the following. It is not ideal, because we are taking spinlock_t pointer instead of void pointer. To use this with void 'pointer, we will need some verifier changes.
Thanks,
Song
kernel/bpf/helpers.c +1 -1
@@ -3090,7 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) -BTF_ID_FLAGS (func, bpf_get_kmem_cache,KF_RET_NULL ) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL | KF_TRUSTED_ARGS
| KF_RCU_PROTECTED)
BTF_KFUNCS_END(common_btf_ids)
static const struct btf_kfunc_id_set common_kfunc_set = {
mm/slab_common.c +4 -4
@@ -1327,14 +1327,15 @@ EXPORT_SYMBOL(ksize);__bpf_kfunc_start_defs (); -__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(spinlock_t *addr) {struct slab *slab; +unsigned long a = (unsigned long )addr; -if (!virt_addr_valid (addr)) +if (!virt_addr_valid (a))return NULL ; - slab =virt_to_slab ((void *)(long )addr); + slab =virt_to_slab (addr);return slab ? slab->slab_cache :NULL ; }
@@ -1346,4 +1347,3 @@ EXPORT_TRACEPOINT_SYMBOL(kmalloc);EXPORT_TRACEPOINT_SYMBOL (kmem_cache_alloc);EXPORT_TRACEPOINT_SYMBOL (kfree);EXPORT_TRACEPOINT_SYMBOL (kmem_cache_free); -
(quoted diff) +0 -0
w/tools/testing/selftests/bpf/progs/kmem_cache_iter.c index 3f6ec15a1bf6..8238155a5055 100644
tools/testing/selftests/bpf/progs/kmem_cache_iter.c +5 -3
@@ -16,7 +16,7 @@ struct {__uint (max_entries,1024 ); } slab_hash SEC(".maps"); -extern struct kmem_cache *bpf_get_kmem_cache (__u64 addr) __ksym; +extern struct kmem_cache *bpf_get_kmem_cache (spinlock_t *addr) __ksym;/* result, will be checked by userspace */ int found;
@@ -46,21 +46,23 @@ int slab_info_collector(struct bpf_iter__kmem_cache *ctx) SEC("raw_tp/bpf_test_finish")int BPF_PROG (check_task_struct ) { -__u64 curr =bpf_get_current_task (); +struct task_struct *curr =bpf_get_current_task_btf ();struct kmem_cache *s;char *name; - s =bpf_get_kmem_cache (curr); + s =bpf_get_kmem_cache (&curr->alloc_lock);if (s ==NULL ) { found =-1 ;return 0 ; } +bpf_rcu_read_lock (); name =bpf_map_lookup_elem (&slab_hash, &s);if (name && !bpf_strncmp (name,11 ,"task_struct" )) found =1 ;else found =-2 ; +bpf_rcu_read_unlock ();return 0 ; }
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 91 lines · Song Liu
Quoted 90 lines · Namhyung Kim
Quoted 73 lines · Song Liu
Quoted 64 lines · Roman Gushchin
Quoted 51 lines · Song Liu
Quoted 50 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +11
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ;Quoted 8 lines
mm/slab_common.c size_t ksize(const void *objp) · line 1339
Quoted 4 lines
mm/slab_common.c +4
-0+ + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +}Do we need to hold a refcount to the
slab_cache? Given we make this kfunc available everywhere, including sleepable contexts, I think it is necessary.It's a really good question.
Quoted 7 lines
If the callee somehow owns the slab object, as in the example provided in the series (current task), it's not necessarily.
If a user can pass a random address, you're right, we need to grab the slab_cache's refcnt. But then we also can't guarantee that the object still belongs to the same
slab_cache, the function becomes racy by the definition.Quoted 4 lines
To be safe, we can limit the kfunc to sleepable context only. Then we can lock
slab_mutexforvirt_to_slab, and hold a refcount toslab_cache. We will need aKF_RELEASEkfunc to release the refcount later.Then it needs to call
kmem_cache_destroy()for release which containsrcu_barrier. :(Quoted 3 lines
IIUC, this limitation (sleepable context only) shouldn't be a problem for perf use case?
No, it would be called from the lock contention path including spinlocks. :(
Quoted 6 lines
Quoted 2 lines
Can we limit it to non-sleepable ctx and not to pass arbtrary address somehow (or not to save the result pointer)?
I hacked something like the following. It is not ideal, because we are taking
spinlock_tpointer instead of void pointer. To use this with void 'pointer, we will need some verifier changes.
Thanks a lot for doing this!! I'll take a look at the verifier what needs to be done.
Namhyung
Quoted 84 lines
kernel/bpf/helpers.c +1
-1@@ -3090,7 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) -BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL) +BTF_ID_FLAGS (func, bpf_get_kmem_cache,KF_RET_NULL |KF_TRUSTED_ARGS | KF_RCU_PROTECTED)BTF_KFUNCS_END (common_btf_ids)static const struct
btf_kfunc_id_setcommon_kfunc_set= {mm/slab_common.c +4
-4@@ -1327,14 +1327,15 @@ EXPORT_SYMBOL(ksize);__bpf_kfunc_start_defs (); -__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(spinlock_t *addr) {struct slab *slab; +unsigned long a = (unsigned long )addr; -if (!virt_addr_valid (addr)) +if (!virt_addr_valid (a))return NULL ; - slab =virt_to_slab ((void *)(long )addr); + slab =virt_to_slab (addr);return slab ? slab->slab_cache :NULL ; }@@ -1346,4 +1347,3 @@ EXPORT_TRACEPOINT_SYMBOL(kmalloc);EXPORT_TRACEPOINT_SYMBOL (kmem_cache_alloc);EXPORT_TRACEPOINT_SYMBOL (kfree);EXPORT_TRACEPOINT_SYMBOL (kmem_cache_free); -(quoted diff) +0
-0
w/tools/testing/selftests/bpf/progs/kmem_cache_iter.cindex3f6ec15a1bf6..8238155a5055100644tools/testing/selftests/bpf/progs/kmem_cache_iter.c +5
-3@@ -16,7 +16,7 @@ struct {__uint (max_entries,1024 ); } slab_hash SEC(".maps"); -extern struct kmem_cache *bpf_get_kmem_cache (__u64 addr) __ksym; +extern struct kmem_cache *bpf_get_kmem_cache (spinlock_t *addr) __ksym;/* result, will be checked by userspace */ int found;@@ -46,21 +46,23 @@ int slab_info_collector(struct bpf_iter__kmem_cache *ctx) SEC("raw_tp/bpf_test_finish")int BPF_PROG (check_task_struct ) { -__u64 curr =bpf_get_current_task (); +struct task_struct *curr =bpf_get_current_task_btf ();struct kmem_cache *s;char *name; - s =bpf_get_kmem_cache (curr); + s =bpf_get_kmem_cache (&curr->alloc_lock);if (s ==NULL ) { found =-1 ;return 0 ; } +bpf_rcu_read_lock (); name =bpf_map_lookup_elem (&slab_hash, &s);if (name && !bpf_strncmp (name,11 ,"task_struct" )) found =1 ;else found =-2 ; +bpf_rcu_read_unlock ();return 0 ; }
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
In reply to Song LiuQuoted 108 lines · Song Liu
Quoted 84 lines · Namhyung Kim
Quoted 73 lines · Song Liu
Quoted 64 lines · Roman Gushchin
Quoted 51 lines · Song Liu
Quoted 50 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +11
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ;Quoted 8 lines
mm/slab_common.c size_t ksize(const void *objp) · line 1339
Quoted 4 lines
mm/slab_common.c +4
-0+ + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +}Do we need to hold a refcount to the
slab_cache? Given we make this kfunc available everywhere, including sleepable contexts, I think it is necessary.It's a really good question.
Quoted 7 lines
If the callee somehow owns the slab object, as in the example provided in the series (current task), it's not necessarily.
If a user can pass a random address, you're right, we need to grab the slab_cache's refcnt. But then we also can't guarantee that the object still belongs to the same
slab_cache, the function becomes racy by the definition.Quoted 4 lines
To be safe, we can limit the kfunc to sleepable context only. Then we can lock
slab_mutexforvirt_to_slab, and hold a refcount toslab_cache. We will need aKF_RELEASEkfunc to release the refcount later.Then it needs to call
kmem_cache_destroy()for release which containsrcu_barrier. :(Quoted 1 line
Quoted 8 lines
Quoted 2 lines
IIUC, this limitation (sleepable context only) shouldn't be a problem for perf use case?
No, it would be called from the lock contention path including spinlocks. :(
Can we limit it to non-sleepable ctx and not to pass arbtrary address somehow (or not to save the result pointer)?
I hacked something like the following. It is not ideal, because we are taking
spinlock_tpointer instead of void pointer. To use this with void 'pointer, we will need some verifier changes.Thanks,
Songkernel/bpf/helpers.c +0
-0
kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) · line 3094
Quoted 6 lines
kernel/bpf/helpers.c +1
-1@@ -3090,7 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) -BTF_ID_FLAGS (func, bpf_get_kmem_cache,KF_RET_NULL ) +BTF_ID_FLAGS (func, bpf_get_kmem_cache,KF_RET_NULL |KF_TRUSTED_ARGS |KF_RCU_PROTECTED )
I don't think KF_TRUSTED_ARGS approach would fit here. Namhyung's use case is tracing. The 'addr' will be some potentially arbitrary address from somewhere. The chance to see a trusted pointer is probably very low in such a tracing use case.
The verifier change can mainly be the following:
kernel/bpf/verifier.c +0 -0
@@ -12834,6 +12834,9 @@ static int check_kfunc_call(struct
bpf_verifier_env *env, struct bpf_insn *insn,
regs[BPF_REG_0].type = PTR_TO_BTF_ID;
regs[BPF_REG_0].btf_id = ptr_type_id;+ if (meta.func_id ==
special_kfunc_list[KF_get_kmem_cache])
+ regs[BPF_REG_0].type |= PTR_UNTRUSTED;
+
if (is_iter_next_kfunc(&meta)) {
struct bpf_reg_state *cur_iter;The returned 'struct kmem_cache *' won't be refcnt-ed (acquired). It will be readonly via ptr_to_btf_id logic.s->flags;s->size;s->offset;
access will be allowed but the verifier will sanitize them with an inlined version of probe_read_kernel.
Even KF_RET_NULL can be dropped.
Song Liu
reviewerRe: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
On Fri, Oct 4, 2024 at 4:44 PM Alexei Starovoitov
<[email protected]> wrote:
[...]
Quoted 8 lines
Quoted 8 lines
kernel/bpf/helpers.c +0
-0@@ -3090,7 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY)BTF_ID_FLAGS (func, bpf_copy_from_user_str,KF_SLEEPABLE )
Quoted 8 lines
kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) · line 3094
Quoted 3 lines
kernel/bpf/helpers.c +1
-1-BTF_ID_FLAGS (func, bpf_get_kmem_cache,KF_RET_NULL ) +BTF_ID_FLAGS (func, bpf_get_kmem_cache,KF_RET_NULL |KF_TRUSTED_ARGS |KF_RCU_PROTECTED )I don't think
KF_TRUSTED_ARGSapproach would fit here. Namhyung's use case is tracing. The 'addr' will be some potentially arbitrary address from somewhere. The chance to see a trusted pointer is probably very low in such a tracing use case.
I thought the primary use case was to trace lock contention, for example, queued_spin_lock_slowpath(). Of course, a more general solution is better.
Quoted 17 lines
The verifier change can mainly be the following:
kernel/bpf/verifier.c +3
-0@@ -12834,6 +12834,9 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0 ].btf_id = ptr_type_id; +if (meta.func_id == special_kfunc_list[KF_get_kmem_cache]) + regs[BPF_REG_0 ].type |=PTR_UNTRUSTED ; +if (is_iter_next_kfunc (&meta)) {
Quoted 1 line
struct bpf_reg_state *cur_iter;
This is easier than I thought.
Thanks,
Song
Quoted 8 lines
The returned 'struct
kmem_cache*' won't be refcnt-ed (acquired). It will be readonly viaptr_to_btf_idlogic.s->flags;s->size;s->offset;
access will be allowed but the verifier will sanitize them with an inlined version ofprobe_read_kernel.
EvenKF_RET_NULLcan be dropped.
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Hello,
Quoted 20 lines · Song Liu
On Fri, Oct 4, 2024 at 4:44 PM Alexei Starovoitov
<[email protected]> wrote:
[...]Quoted 8 lines
Quoted 8 lines
kernel/bpf/helpers.c +0
-0@@ -3090,7 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY)BTF_ID_FLAGS (func, bpf_copy_from_user_str,KF_SLEEPABLE )Quoted 8 lines
kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) · line 3094
Quoted 3 lines
kernel/bpf/helpers.c +1
-1-BTF_ID_FLAGS (func, bpf_get_kmem_cache,KF_RET_NULL ) +BTF_ID_FLAGS (func, bpf_get_kmem_cache,KF_RET_NULL |KF_TRUSTED_ARGS |KF_RCU_PROTECTED )I don't think
KF_TRUSTED_ARGSapproach would fit here. Namhyung's use case is tracing. The 'addr' will be some potentially arbitrary address from somewhere. The chance to see a trusted pointer is probably very low in such a tracing use case.
Quoted 3 lines
I thought the primary use case was to trace lock contention, for example,
queued_spin_lock_slowpath(). Of course, a more general solution is better.
Right, my intended use case is the lock contention profiling so probably it's ok to limit it for trusted pointers if it helps. But as Song said, a general solution should be better. :)
Quoted 13 lines
Quoted 13 lines
The verifier change can mainly be the following:
kernel/bpf/verifier.c +0
-0@@ -12834,6 +12834,9 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0 ].btf_id = ptr_type_id;
Quoted 7 lines
kernel/bpf/verifier.c static int check_kfunc_call(struct · line 12842
Quoted 5 lines
kernel/bpf/verifier.c +3
-0+if (meta.func_id == special_kfunc_list[KF_get_kmem_cache]) + regs[BPF_REG_0 ].type |=PTR_UNTRUSTED ; +if (is_iter_next_kfunc (&meta)) {struct bpf_reg_state *cur_iter;This is easier than I thought.
Indeed! Thanks for providing the code.
Quoted 9 lines
Quoted 9 lines
The returned 'struct
kmem_cache*' won't be refcnt-ed (acquired). It will be readonly viaptr_to_btf_idlogic.s->flags;s->size;s->offset;
access will be allowed but the verifier will sanitize them with an inlined version ofprobe_read_kernel.
EvenKF_RET_NULLcan be dropped.
Ok, I'll check this out. By having PTR_UNTRUSTED, are the callers still required to check NULL or is it handled by probe_read_kernel()?
Thanks,
Namhyung
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
In reply to Roman GushchinQuoted 8 lines · Roman Gushchin
Quoted 7 lines · Song Liu
Quoted 6 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>
So IIRC from our discussions with Namhyung and Arnaldo at LSF/MM I thought the perf use case was:
- at the beginning it iterates the kmem caches and stores anything of possible interest in bpf maps or somewhere - hence we have the iterator
- during profiling, from object it gets to a cache, but doesn't need to access the cache - just store the kmem_cache address in the perf record
- after profiling itself, use the information in the maps from the first step together with cache pointers from the second step to calculate whatever is necessary
So at no point it should be necessary to take refcount to a kmem_cache?
But maybe "bpf_get_kmem_cache()" is implemented here as too generic given the above use case and it should be implemented in a way that the pointer it returns cannot be used to access anything (which could be unsafe), but only as a bpf map key - so it should return e.g. an unsigned long instead?
Quoted 54 lines
Quoted 36 lines
Quoted 36 lines
--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +11
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ;Quoted 8 lines
mm/slab_common.c size_t ksize(const void *objp) · line 1339
Quoted 4 lines
mm/slab_common.c +4
-0+ + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +}Do we need to hold a refcount to the
slab_cache? Given we make this kfunc available everywhere, including sleepable contexts, I think it is necessary.It's a really good question.
If the callee somehow owns the slab object, as in the example provided in the series (current task), it's not necessarily.
If a user can pass a random address, you're right, we need to grab the slab_cache's refcnt. But then we also can't guarantee that the object still belongs to the same
slab_cache, the function becomes racy by the definition.
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 22 lines · Vlastimil Babka
Quoted 8 lines · Roman Gushchin
Quoted 7 lines · Song Liu
Quoted 6 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>So IIRC from our discussions with Namhyung and Arnaldo at LSF/MM I thought the perf use case was:
Quoted 7 lines
- at the beginning it iterates the kmem caches and stores anything of possible interest in bpf maps or somewhere - hence we have the iterator
- during profiling, from object it gets to a cache, but doesn't need to access the cache - just store thekmem_cacheaddress in the perf record
- after profiling itself, use the information in the maps from the first step together with cache pointers from the second step to calculate whatever is necessary
Correct.
Quoted 8 lines
So at no point it should be necessary to take refcount to a
kmem_cache?But maybe "
bpf_get_kmem_cache()" is implemented here as too generic given the above use case and it should be implemented in a way that the pointer it returns cannot be used to access anything (which could be unsafe), but only as a bpf map key - so it should return e.g. an unsigned long instead?
Yep, this should work for my use case. Maybe we don't need the iterator when bpf_get_kmem_cache() kfunc returns the valid pointer as we can get the necessary info at the moment. But I think it'd be less efficient as more work need to be done at the event (lock contention). It'd better setting up necessary info in a map before monitoring (using the iterator), and just looking up the map with the kfunc while monitoring the lock contention.
Thanks,
Namhyung
Quoted 55 lines
Quoted 55 lines
Quoted 37 lines
Quoted 37 lines
--- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+)kernel/bpf/helpers.c +1
-0@@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL)BTF_KFUNCS_END (common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = {mm/slab_common.c +11
-0@@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) }EXPORT_SYMBOL (ksize); +#ifdef CONFIG_BPF_SYSCALL +#include <linux/btf.h> + +__bpf_kfunc_start_defs (); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ +struct slab *slab; + +if (!virt_addr_valid (addr)) +return NULL ;Quoted 8 lines
mm/slab_common.c size_t ksize(const void *objp) · line 1339
Quoted 4 lines
mm/slab_common.c +4
-0+ + slab =virt_to_slab ((void *)(long )addr); +return slab ? slab->slab_cache :NULL ; +}Do we need to hold a refcount to the
slab_cache? Given we make this kfunc available everywhere, including sleepable contexts, I think it is necessary.It's a really good question.
If the callee somehow owns the slab object, as in the example provided in the series (current task), it's not necessarily.
If a user can pass a random address, you're right, we need to grab the slab_cache's refcnt. But then we also can't guarantee that the object still belongs to the same
slab_cache, the function becomes racy by the definition.
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 42 lines · Namhyung Kim
Quoted 22 lines · Vlastimil Babka
Quoted 8 lines · Roman Gushchin
Quoted 7 lines · Song Liu
Quoted 6 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>So IIRC from our discussions with Namhyung and Arnaldo at LSF/MM I thought the perf use case was:
Quoted 7 lines
- at the beginning it iterates the kmem caches and stores anything of possible interest in bpf maps or somewhere - hence we have the iterator
- during profiling, from object it gets to a cache, but doesn't need to access the cache - just store thekmem_cacheaddress in the perf record
- after profiling itself, use the information in the maps from the first step together with cache pointers from the second step to calculate whatever is necessaryCorrect.
Quoted 8 lines
So at no point it should be necessary to take refcount to a
kmem_cache?But maybe "
bpf_get_kmem_cache()" is implemented here as too generic given the above use case and it should be implemented in a way that the pointer it returns cannot be used to access anything (which could be unsafe), but only as a bpf map key - so it should return e.g. an unsigned long instead?
Quoted 7 lines
Yep, this should work for my use case. Maybe we don't need the iterator when
bpf_get_kmem_cache()kfunc returns the valid pointer as we can get the necessary info at the moment. But I think it'd be less efficient as more work need to be done at the event (lock contention). It'd better setting up necessary info in a map before monitoring (using the iterator), and just looking up the map with the kfunc while monitoring the lock contention.
Maybe it's still better to return a non-refcounted pointer for future use. I'll leave it for v5.
Thanks,
Namhyung
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 52 lines · Namhyung Kim
Quoted 42 lines · Namhyung Kim
Quoted 22 lines · Vlastimil Babka
Quoted 8 lines · Roman Gushchin
Quoted 7 lines · Song Liu
Quoted 6 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>So IIRC from our discussions with Namhyung and Arnaldo at LSF/MM I thought the perf use case was:
Quoted 7 lines
- at the beginning it iterates the kmem caches and stores anything of possible interest in bpf maps or somewhere - hence we have the iterator
- during profiling, from object it gets to a cache, but doesn't need to access the cache - just store thekmem_cacheaddress in the perf record
- after profiling itself, use the information in the maps from the first step together with cache pointers from the second step to calculate whatever is necessaryCorrect.
Quoted 8 lines
So at no point it should be necessary to take refcount to a
kmem_cache?But maybe "
bpf_get_kmem_cache()" is implemented here as too generic given the above use case and it should be implemented in a way that the pointer it returns cannot be used to access anything (which could be unsafe), but only as a bpf map key - so it should return e.g. an unsigned long instead?Quoted 7 lines
Yep, this should work for my use case. Maybe we don't need the iterator when
bpf_get_kmem_cache()kfunc returns the valid pointer as we can get the necessary info at the moment. But I think it'd be less efficient as more work need to be done at the event (lock contention). It'd better setting up necessary info in a map before monitoring (using the iterator), and just looking up the map with the kfunc while monitoring the lock contention.
Quoted 2 lines
Maybe it's still better to return a non-refcounted pointer for future use. I'll leave it for v5.
Pls keep it as:__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr)
just make sure it's PTR_UNTRUSTED.
No need to make it return long or void *.
The users can do:
bpf_core_cast(any_value, struct kmem_cache);
anyway, but it would be an unnecessary step.
Re: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc
Quoted 53 lines · Alexei Starovoitov
Quoted 52 lines · Namhyung Kim
Quoted 49 lines · Namhyung Kim
Quoted 22 lines · Vlastimil Babka
Quoted 8 lines · Roman Gushchin
Quoted 7 lines · Song Liu
Quoted 6 lines · Namhyung Kim
The
bpf_get_kmem_cache()is to get a slab cache information from a virtual address likevirt_to_cache(). If the address is a pointer to a slab object, it'd return a validkmem_cachepointer, otherwise NULL is returned.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
It doesn't grab a reference count of the
kmem_cacheso the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints.Suggested-by Vlastimil Babka <[email protected]>Signed-off-by Namhyung Kim <[email protected]>So IIRC from our discussions with Namhyung and Arnaldo at LSF/MM I thought the perf use case was:
Quoted 7 lines
- at the beginning it iterates the kmem caches and stores anything of possible interest in bpf maps or somewhere - hence we have the iterator
- during profiling, from object it gets to a cache, but doesn't need to access the cache - just store thekmem_cacheaddress in the perf record
- after profiling itself, use the information in the maps from the first step together with cache pointers from the second step to calculate whatever is necessaryCorrect.
Quoted 8 lines
So at no point it should be necessary to take refcount to a
kmem_cache?But maybe "
bpf_get_kmem_cache()" is implemented here as too generic given the above use case and it should be implemented in a way that the pointer it returns cannot be used to access anything (which could be unsafe), but only as a bpf map key - so it should return e.g. an unsigned long instead?Yep, this should work for my use case. Maybe we don't need the iterator when
bpf_get_kmem_cache()kfunc returns the valid pointer as we can get the necessary info at the moment. But I think it'd be less efficient as more work need to be done at the event (lock contention). It'd better setting up necessary info in a map before monitoring (using the iterator), and just looking up the map with the kfunc while monitoring the lock contention.
Quoted 7 lines
Quoted 2 lines
Maybe it's still better to return a non-refcounted pointer for future use. I'll leave it for v5.
Pls keep it as:
__bpf_kfuncstructkmem_cache*bpf_get_kmem_cache(u64 addr)just make sure it's
PTR_UNTRUSTED.
Sure, will do.
Quoted 4 lines
No need to make it return long or void *.
The users can do:
bpf_core_cast(any_value, struct kmem_cache);
anyway, but it would be an unnecessary step.
Yeah I thought there would be a way to do that.
Thanks,
Namhyung
PATCH v4 3/3 selftests/bpf: Add a test for kmem_cache_iter
In reply to Namhyung KimThe test traverses all slab caches using the kmem_cache_iter and check if current task's pointer is from "task_struct" slab cache.
--- .../bpf/prog_tests/kmem_cache_iter.c | 64 ++++++++++++++++++ tools/testing/selftests/bpf/progs/bpf_iter.h | 7 ++ .../selftests/bpf/progs/kmem_cache_iter.c | 66 +++++++++++++++++++ 3 files changed, 137 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c create mode 100644 tools/testing/selftests/bpf/progs/kmem_cache_iter.c
tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c new file +64 -0
@@ -0,0 +1,64 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Google */ + +#include <test_progs.h> +#include <bpf/libbpf.h> +#include <bpf/btf.h> +#include "kmem_cache_iter.skel.h" + +static void test_kmem_cache_iter_check_task (struct kmem_cache_iter *skel) +{ +LIBBPF_OPTS (bpf_test_run_opts, opts, + .flags = 0, /* run it with the current task */ + ); +int prog_fd =bpf_program__fd (skel->progs.check_task_struct); + +/* get task_struct and check it if's from a slab cache */ +bpf_prog_test_run_opts (prog_fd, &opts); + +/* the BPF program should set 'found' variable */ +ASSERT_EQ (skel->bss->found,1 ,"found task_struct" ); +} + +void test_kmem_cache_iter (void ) +{ +DECLARE_LIBBPF_OPTS (bpf_iter_attach_opts, opts); +struct kmem_cache_iter *skel =NULL ; +union bpf_iter_link_info linfo = {}; +struct bpf_link *link; +char buf[1024 ]; +int iter_fd; + + skel =kmem_cache_iter__open_and_load (); +if (!ASSERT_OK_PTR (skel,"kmem_cache_iter__open_and_load" )) +return ; + + opts.link_info = &linfo; + opts.link_info_len =sizeof (linfo); + + link =bpf_program__attach_iter (skel->progs.slab_info_collector, &opts); +if (!ASSERT_OK_PTR (link,"attach_iter" )) + goto destroy; + + iter_fd =bpf_iter_create (bpf_link__fd (link)); +if (!ASSERT_GE (iter_fd,0 ,"iter_create" )) + goto free_link; + +memset (buf,0 ,sizeof (buf)); +while (read (iter_fd, buf,sizeof (buf) >0 )) { +/* read out all contents */ +printf ("%s" , buf); + } + +/* next reads should return 0 */ +ASSERT_EQ (read (iter_fd, buf,sizeof (buf)),0 ,"read" ); + +test_kmem_cache_iter_check_task (skel); + +close (iter_fd); + +free_link: +bpf_link__destroy (link); +destroy: +kmem_cache_iter__destroy (skel); +}
tools/testing/selftests/bpf/progs/bpf_iter.h +7 -0
@@ -24,6 +24,7 @@#define BTF_F_PTR_RAW BTF_F_PTR_RAW___not_used#define BTF_F_ZERO BTF_F_ZERO___not_used#define bpf_iter__ksym bpf_iter__ksym___not_used +#define bpf_iter__kmem_cache bpf_iter__kmem_cache___not_used#include "vmlinux.h" #undef bpf_iter_meta#undef bpf_iter__bpf_map
@@ -48,6 +49,7 @@#undef BTF_F_PTR_RAW#undef BTF_F_ZERO#undef bpf_iter__ksym +#undef bpf_iter__kmem_cachestruct bpf_iter_meta {struct seq_file *seq;
@@ -165,3 +167,8 @@ struct bpf_iter__ksym {struct bpf_iter_meta *meta;struct kallsym_iter *ksym; }; + +struct bpf_iter__kmem_cache { +struct bpf_iter_meta *meta; +struct kmem_cache *s; +} __attribute__((preserve_access_index));
tools/testing/selftests/bpf/progs/kmem_cache_iter.c new file +66 -0
66 lines not on this page · raw message
Signature · 1 line
2.46.1.824.gd892dcdcdd-goog
A message that is not in the archive (sent off-list or not yet received): 94bdb7a4cb0f83adf655d98a5c5f5df5299b960d2af54c87eba08de9646d0e42@mail.kernel.org
A message that is not in the archive (sent off-list or not yet received): CAM9d7cjGh5+5Cgw-5Nc5oO88HgJz33BUuMGYREExEgWXND3B_A@mail.gmail.com
Hi Namhyung,
Quoted 12 lines · Namhyung Kim
Hello,
Quoted 9 lines · On Wed, Oct 2, 2024 at 12:06 PM <[email protected]> wrote:
Dear patch submitter,
CI has tested the following submission:
Status: FAILURE Name: [v4,bpf-next,0/3] bpf: Add kmem_cache iterator and kfunc Patchwork: https://patchwork.kernel.org/project/netdevbpf/list/?series=894947&state=* Matrix: https://github.com/kernel-patches/bpf/actions/runs/11149350866
Quoted 5 lines
Quoted 2 lines
Failed jobs:
test_progs-aarch64-gcc: https://github.com/kernel-patches/bpf/actions/runs/11149350866/job/30988341564I'm not sure if it's because of my change. It seems to have failed on unrelated tests. Can you please double check?
I ran some queries on the BPF CI dataset (unfortunately not public) and I found at least one other instance of this failure [0].
I've also manually triggered a re-run and now it passes. So it's probably not related to your change.
I'll try to find the right person to debug it further.
Thanks,
Daniel
[0]:
https://github.com/kernel-patches/bpf/actions/runs/11010557218/job/30573607777
Hi Daniel,
Quoted 22 lines · Daniel Xu
Hi Namhyung,
Quoted 12 lines · Namhyung Kim
Hello,
Quoted 9 lines · On Wed, Oct 2, 2024 at 12:06 PM <[email protected]> wrote:
Dear patch submitter,
CI has tested the following submission:
Status: FAILURE Name: [v4,bpf-next,0/3] bpf: Add kmem_cache iterator and kfunc Patchwork: https://patchwork.kernel.org/project/netdevbpf/list/?series=894947&state=* Matrix: https://github.com/kernel-patches/bpf/actions/runs/11149350866Quoted 5 lines
Quoted 2 lines
Failed jobs:
test_progs-aarch64-gcc: https://github.com/kernel-patches/bpf/actions/runs/11149350866/job/30988341564I'm not sure if it's because of my change. It seems to have failed on unrelated tests. Can you please double check?
Quoted 7 lines
I ran some queries on the BPF CI dataset (unfortunately not public) and I found at least one other instance of this failure [0].
I've also manually triggered a re-run and now it passes. So it's probably not related to your change.
I'll try to find the right person to debug it further.
Thanks a lot for taking care of this!
Namhyung