objpool: fix choosing allocation for percpu slots
Subsystems (4)
PATCH v2 1/1 objpool: fix choosing allocation for percpu slots
objpool intends to use vmalloc for default (non-atomic) allocations of percpu slots and objects. However, the condition checking if GFP flags are equal to GFP_ATOMIC is wrong b/c GFP_ATOMIC is a combination of bits (__GFP_HIGH|__GFP_KSWAPD_RECLAIM) and so pool->gfp & GFP_ATOMIC will be true if either bit is set. Since GFP_ATOMIC and GFP_KERNEL share the ___GFP_KSWAPD_RECLAIM bit, kmalloc will be used in cases when GFP_KERNEL is specified, i.e. in all current usages of objpool.
This may lead to unexpected OOM errors since kmalloc cannot allocate large amounts of memory.
For instance, objpool is used by fprobe rethook which in turn is used by BPF kretprobe.multi and kprobe.session probe types. Trying to attach these to all kernel functions with libbpf using
SEC("kprobe.session/*")
int kprobe(struct pt_regs *ctx)
{
[...]
}fails on objpool slot allocation with ENOMEM.
Fix the condition to truly use vmalloc by default.
--- lib/objpool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/objpool.c +1 -1
@@ -76,7 +76,7 @@ objpool_init_percpu_slots(struct objpool_head *pool, int nr_objs,* mimimal size of vmalloc is one page since vmalloc would * always align the requested size to page size */ -if (pool->gfp &GFP_ATOMIC ) +if ((pool->gfp &GFP_ATOMIC ) ==GFP_ATOMIC ) slot =kmalloc_node (size, pool->gfp,cpu_to_node (i));else slot = __vmalloc_node(size, sizeof(void *), pool->gfp,
Signature · 1 line
2.46.0
Quoted 21 lines · On Mon, 26 Aug 2024 08:07:18 +0200 Viktor Malik <[email protected]> wrote:
objpool intends to use vmalloc for default (non-atomic) allocations of percpu slots and objects. However, the condition checking if GFP flags are equal to
GFP_ATOMICis wrong b/cGFP_ATOMICis a combination of bits (__GFP_HIGH|__GFP_KSWAPD_RECLAIM) and sopool->gfp & GFP_ATOMICwill be true if either bit is set. SinceGFP_ATOMICandGFP_KERNELshare the___GFP_KSWAPD_RECLAIMbit, kmalloc will be used in cases whenGFP_KERNELis specified, i.e. in all current usages of objpool.This may lead to unexpected OOM errors since kmalloc cannot allocate large amounts of memory.
For instance, objpool is used by fprobe rethook which in turn is used by BPF kretprobe.multi and kprobe.session probe types. Trying to attach these to all kernel functions with libbpf using
SEC("kprobe.session/*") int kprobe(struct pt_regs *ctx) { [...] }
Quoted 8 lines
fails on objpool slot allocation with ENOMEM.
Fix the condition to truly use vmalloc by default.
Fixes b4edb8d2d464 ("lib: objpool added: ring-array based lockless MPMC")Signed-off-by Viktor Malik <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Reviewed-by Matt Wu <[email protected]>
-- Steve
Quoted 17 lines
--- lib/objpool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/objpool.c +1
-1@@ -76,7 +76,7 @@ objpool_init_percpu_slots(struct objpool_head *pool, int nr_objs,* mimimal size of vmalloc is one page since vmalloc would * always align the requested size to page size */ -if (pool->gfp &GFP_ATOMIC ) +if ((pool->gfp &GFP_ATOMIC ) ==GFP_ATOMIC ) slot =kmalloc_node (size, pool->gfp,cpu_to_node (i));else slot = __vmalloc_node(size, sizeof(void *), pool->gfp,
Masami Hiramatsu (Google)
maintainerQuoted 3 lines · On Mon, 26 Aug 2024 08:07:18 +0200 Viktor Malik <[email protected]> wrote:
objpool intends to use vmalloc for default (non-atomic) allocations of percpu slots and objects. However, the condition checking if GFP flags are equal to
GFP_ATOMICis wrong b/cGFP_ATOMICis a combination of bits
You meant "whether GFP flags sets any bit of GFP_ATOMIC is wrong"?
Quoted 18 lines
(__GFP_HIGH|__GFP_KSWAPD_RECLAIM) and so
pool->gfp & GFP_ATOMICwill be true if either bit is set. SinceGFP_ATOMICandGFP_KERNELshare the___GFP_KSWAPD_RECLAIMbit, kmalloc will be used in cases whenGFP_KERNELis specified, i.e. in all current usages of objpool.This may lead to unexpected OOM errors since kmalloc cannot allocate large amounts of memory.
For instance, objpool is used by fprobe rethook which in turn is used by BPF kretprobe.multi and kprobe.session probe types. Trying to attach these to all kernel functions with libbpf using
SEC("kprobe.session/*") int kprobe(struct pt_regs *ctx) { [...] }
Quoted 4 lines
fails on objpool slot allocation with ENOMEM.
Fix the condition to truly use vmalloc by default.
Anyway, this looks good to me.
Thank you,
Quoted 24 lines
Fixes b4edb8d2d464 ("lib: objpool added: ring-array based lockless MPMC")Signed-off-by Viktor Malik <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Reviewed-by Matt Wu <[email protected]>--- lib/objpool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/objpool.c +1
-1@@ -76,7 +76,7 @@ objpool_init_percpu_slots(struct objpool_head *pool, int nr_objs,* mimimal size of vmalloc is one page since vmalloc would * always align the requested size to page size */ -if (pool->gfp &GFP_ATOMIC ) +if ((pool->gfp &GFP_ATOMIC ) ==GFP_ATOMIC ) slot =kmalloc_node (size, pool->gfp,cpu_to_node (i));else slot = __vmalloc_node(size, sizeof(void *), pool->gfp,Signature · 1 line
2.46.0
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>
Quoted 8 lines · Masami Hiramatsu (Google)
Quoted 3 lines · On Mon, 26 Aug 2024 08:07:18 +0200 Viktor Malik <[email protected]> wrote:
objpool intends to use vmalloc for default (non-atomic) allocations of percpu slots and objects. However, the condition checking if GFP flags are equal to
GFP_ATOMICis wrong b/cGFP_ATOMICis a combination of bitsYou meant "whether GFP flags sets any bit of
GFP_ATOMICis wrong"?
Well, I meant that the condition is wrong w.r.t. what is supposedly its original purpose. But feel free to rephrase as you seem fit or I can send v3 if you prefer.
Thanks.
Viktor
Quoted 54 lines
Quoted 19 lines
(__GFP_HIGH|__GFP_KSWAPD_RECLAIM) and so
pool->gfp & GFP_ATOMICwill be true if either bit is set. SinceGFP_ATOMICandGFP_KERNELshare the___GFP_KSWAPD_RECLAIMbit, kmalloc will be used in cases whenGFP_KERNELis specified, i.e. in all current usages of objpool.This may lead to unexpected OOM errors since kmalloc cannot allocate large amounts of memory.
For instance, objpool is used by fprobe rethook which in turn is used by BPF kretprobe.multi and kprobe.session probe types. Trying to attach these to all kernel functions with libbpf using
SEC("kprobe.session/*") int kprobe(struct pt_regs *ctx) { [...] }Quoted 4 lines
fails on objpool slot allocation with ENOMEM.
Fix the condition to truly use vmalloc by default.
Anyway, this looks good to me.
Thank you,
Quoted 24 lines
Fixes b4edb8d2d464 ("lib: objpool added: ring-array based lockless MPMC")Signed-off-by Viktor Malik <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Reviewed-by Matt Wu <[email protected]>--- lib/objpool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/objpool.c +1
-1@@ -76,7 +76,7 @@ objpool_init_percpu_slots(struct objpool_head *pool, int nr_objs,* mimimal size of vmalloc is one page since vmalloc would * always align the requested size to page size */ -if (pool->gfp &GFP_ATOMIC ) +if ((pool->gfp &GFP_ATOMIC ) ==GFP_ATOMIC ) slot =kmalloc_node (size, pool->gfp,cpu_to_node (i));else slot = __vmalloc_node(size, sizeof(void *), pool->gfp,Signature · 1 line
2.46.0
Masami Hiramatsu (Google)
maintainerQuoted 8 lines · On Tue, 22 Oct 2024 13:45:08 +0200 Viktor Malik <[email protected]> wrote:
Quoted 7 lines · Masami Hiramatsu (Google)
Quoted 3 lines · On Mon, 26 Aug 2024 08:07:18 +0200 Viktor Malik <[email protected]> wrote:
objpool intends to use vmalloc for default (non-atomic) allocations of percpu slots and objects. However, the condition checking if GFP flags are equal to
GFP_ATOMICis wrong b/cGFP_ATOMICis a combination of bits
Quoted 5 lines
Quoted 1 line
You meant "whether GFP flags sets any bit of
GFP_ATOMICis wrong"?Well, I meant that the condition is wrong w.r.t. what is supposedly its original purpose. But feel free to rephrase as you seem fit or I can send v3 if you prefer.
No problem :) let me rephrase that part.
Thank you!
Quoted 59 lines
Thanks.
ViktorQuoted 54 lines
Quoted 19 lines
(__GFP_HIGH|__GFP_KSWAPD_RECLAIM) and so
pool->gfp & GFP_ATOMICwill be true if either bit is set. SinceGFP_ATOMICandGFP_KERNELshare the___GFP_KSWAPD_RECLAIMbit, kmalloc will be used in cases whenGFP_KERNELis specified, i.e. in all current usages of objpool.This may lead to unexpected OOM errors since kmalloc cannot allocate large amounts of memory.
For instance, objpool is used by fprobe rethook which in turn is used by BPF kretprobe.multi and kprobe.session probe types. Trying to attach these to all kernel functions with libbpf using
SEC("kprobe.session/*") int kprobe(struct pt_regs *ctx) { [...] }Quoted 4 lines
fails on objpool slot allocation with ENOMEM.
Fix the condition to truly use vmalloc by default.
Anyway, this looks good to me.
Thank you,
Quoted 24 lines
Fixes b4edb8d2d464 ("lib: objpool added: ring-array based lockless MPMC")Signed-off-by Viktor Malik <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Reviewed-by Matt Wu <[email protected]>--- lib/objpool.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/objpool.c +1
-1@@ -76,7 +76,7 @@ objpool_init_percpu_slots(struct objpool_head *pool, int nr_objs,* mimimal size of vmalloc is one page since vmalloc would * always align the requested size to page size */ -if (pool->gfp &GFP_ATOMIC ) +if ((pool->gfp &GFP_ATOMIC ) ==GFP_ATOMIC ) slot =kmalloc_node (size, pool->gfp,cpu_to_node (i));else slot = __vmalloc_node(size, sizeof(void *), pool->gfp,Signature · 1 line
2.46.0
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>