bpf: add bpf_dynptr_memset() kfunc
Subsystems (3)
PATCH 1/2 bpf: add bpf_dynptr_memset() kfunc
Currently there is no straightforward way to fill dynptr memory with a value (most commonly zero). One can do it with bpf_dynptr_write(), but a temporary buffer is necessary for that.
Implement bpf_dynptr_memset() - an analogue of memset() from libc.
--- kernel/bpf/helpers.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
kernel/bpf/helpers.c +28 -0
@@ -2906,6 +2906,33 @@ __bpf_kfunc int bpf_dynptr_copy(struct bpf_dynptr *dst_ptr, u32 dst_off,return 0 ; } +/** +* bpf_dynptr_memset() - Fill dynptr memory with a constant byte. +* @ptr: Destination dynptr - where data will be filled +* @val: Constant byte to fill the memory with +* @n: Number of bytes to fill +* +* Fills the first n bytes of the memory area pointed to by ptr +* with the constant byte val. +* Returns 0 on success; negative error, otherwise. +*/ + __bpf_kfunc int bpf_dynptr_memset(struct bpf_dynptr *ptr, u8 val, u32 n) + { +struct bpf_dynptr_kern *p = (struct bpf_dynptr_kern *)ptr; +int err; + +if (__bpf_dynptr_is_rdonly (p)) +return -EINVAL ; + + err =bpf_dynptr_check_off_len (p,0 , n); +if (err) +return err; + +memset (p->data + p->offset, val, n); + +return 0 ; +} + __bpf_kfunc void *bpf_cast_to_kern_ctx(void *obj) {return obj;
@@ -3364,6 +3391,7 @@ BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) BTF_ID_FLAGS(func, bpf_dynptr_size) BTF_ID_FLAGS(func, bpf_dynptr_clone) BTF_ID_FLAGS(func, bpf_dynptr_copy) +BTF_ID_FLAGS(func, bpf_dynptr_memset)#ifdef CONFIG_NET BTF_ID_FLAGS(func, bpf_modify_return_test_tp)#endif
Signature · 1 line
2.47.1
PATCH 2/2 selftests/bpf: add test cases for bpf_dynptr_memset()
Add tests to verify the behavior of bpf_dynptr_memset():
* normal memset 0
* memset in dynptr that was adjusted
* error: size overflow
* error: readonly dynptr
--- .../testing/selftests/bpf/prog_tests/dynptr.c | 4 ++ .../selftests/bpf/progs/dynptr_success.c | 66 +++++++++++++++++++ 2 files changed, 70 insertions(+)
tools/testing/selftests/bpf/prog_tests/dynptr.c +4 -0
@@ -21,6 +21,10 @@ static struct { {"test_dynptr_data", SETUP_SYSCALL_SLEEP}, {"test_dynptr_copy", SETUP_SYSCALL_SLEEP}, {"test_dynptr_copy_xdp", SETUP_XDP_PROG}, + {"test_dynptr_memset_zero", SETUP_SYSCALL_SLEEP}, + {"test_dynptr_memset_zero_offset", SETUP_SYSCALL_SLEEP}, + {"test_dynptr_memset_overflow", SETUP_SYSCALL_SLEEP}, + {"test_dynptr_memset_readonly", SETUP_SKB_PROG}, {"test_ringbuf", SETUP_SYSCALL_SLEEP}, {"test_skb_readonly", SETUP_SKB_PROG}, {"test_dynptr_skb_data", SETUP_SKB_PROG},
tools/testing/selftests/bpf/progs/dynptr_success.c +66 -0
72 lines not on this page · raw message
Signature · 1 line
2.47.1
Quoted 32 lines · Ihor Solodrai
Currently there is no straightforward way to fill dynptr memory with a value (most commonly zero). One can do it with
bpf_dynptr_write(), but a temporary buffer is necessary for that.Implement
bpf_dynptr_memset()- an analogue ofmemset()from libc.Signed-off-by Ihor Solodrai <[email protected]>--- kernel/bpf/helpers.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)kernel/bpf/helpers.c +13
-0@@ -2906,6 +2906,33 @@ __bpf_kfunc int bpf_dynptr_copy(struct bpf_dynptr *dst_ptr, u32 dst_off,return 0 ; } +/** +* bpf_dynptr_memset() - Fill dynptr memory with a constant byte. +* @ptr: Destination dynptr - where data will be filled +* @val: Constant byte to fill the memory with +* @n: Number of bytes to fill +* +* Fills the first n bytes of the memory area pointed to by ptr +* with the constant byte val. +* Returns 0 on success; negative error, otherwise. +*/ + __bpf_kfunc int bpf_dynptr_memset(struct bpf_dynptr *ptr, u8 val, u32 n) + { +struct bpf_dynptr_kern *p = (struct bpf_dynptr_kern *)ptr;
kernel/bpf/helpers.c __bpf_kfunc int bpf_dynptr_copy(struct bpf_dynptr *dst_ptr, u32 dst_off, · line 2931
Quoted 10 lines
kernel/bpf/helpers.c +10
-0+int err; + +if (__bpf_dynptr_is_rdonly (p)) +return -EINVAL ; + + err =bpf_dynptr_check_off_len (p,0 , n); +if (err) +return err; + +memset (p->data + p->offset, val, n);
Do we need to handle non-contiguous buffers, similarly to bpf_dynptr_write (BPF_DYNPTR_TYPE_XDP case)?
Quoted 15 lines
kernel/bpf/helpers.c +5
-0+ +return 0 ; +} + __bpf_kfunc void *bpf_cast_to_kern_ctx(void *obj) {return obj;@@ -3364,6 +3391,7 @@ BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) BTF_ID_FLAGS(func, bpf_dynptr_size) BTF_ID_FLAGS(func, bpf_dynptr_clone) BTF_ID_FLAGS(func, bpf_dynptr_copy) +BTF_ID_FLAGS(func, bpf_dynptr_memset)#ifdef CONFIG_NET BTF_ID_FLAGS(func, bpf_modify_return_test_tp)#endif
Ihor Solodrai
reviewerQuoted 37 lines · Mykyta Yatsenko
Quoted 36 lines · Ihor Solodrai
Currently there is no straightforward way to fill dynptr memory with a value (most commonly zero). One can do it with
bpf_dynptr_write(), but a temporary buffer is necessary for that.Implement
bpf_dynptr_memset()- an analogue ofmemset()from libc.Signed-off-by Ihor Solodrai <[email protected]>---
kernel/bpf/helpers.c| 28 ++++++++++++++++++++++++++++1 file changed, 28 insertions(+)kernel/bpf/helpers.c +16
-0@@ -2906,6 +2906,33 @@ __bpf_kfunc int bpf_dynptr_copy(struct bpf_dynptr *dst_ptr, u32 dst_off, return 0; } +/** +* bpf_dynptr_memset() - Fill dynptr memory with a constant byte. +* @ptr: Destination dynptr - where data will be filled +* @val: Constant byte to fill the memory with +* @n: Number of bytes to fill +* +* Fills the first n bytes of the memory area pointed to by ptr +* with the constant byte val. +* Returns 0 on success; negative error, otherwise. +*/ + __bpf_kfunc int bpf_dynptr_memset(struct bpf_dynptr *ptr, u8 val,u32 n) + { + struct bpf_dynptr_kern *p = (struct bpf_dynptr_kern *)ptr; + int err; + + if (__bpf_dynptr_is_rdonly(p))
Quoted 8 lines
kernel/bpf/helpers.c __bpf_kfunc int bpf_dynptr_copy(struct bpf_dynptr *dst_ptr, u32 dst_off, · line 2931
Quoted 7 lines
kernel/bpf/helpers.c +7
-0+ return -EINVAL; + + err = bpf_dynptr_check_off_len(p, 0, n); + if (err) + return err; + + memset(p->data + p->offset, val, n);Do we need to handle non-contiguous buffers, similarly to
bpf_dynptr_write(BPF_DYNPTR_TYPE_XDPcase)?
It appears so, yes. I am a little lost on how exactly to do it though.
Looking at bpf_xdp_store_bytes [1]:
BPF_CALL_4(bpf_xdp_store_bytes, struct xdp_buff *, xdp, u32, offset,
void *, buf, u32, len)
{
void *ptr;
ptr = bpf_xdp_pointer(xdp, offset, len); if (IS_ERR(ptr)) return PTR_ERR(ptr); if (!ptr) bpf_xdp_copy_buf(xdp, offset, buf, len, true); else memcpy(ptr, buf, len); return 0; }
...
int __bpf_xdp_store_bytes(struct xdp_buff *xdp, u32 offset, void
*buf, u32 len)
{
return ____bpf_xdp_store_bytes(xdp, offset, buf, len);
}
It looks like I'd need to implement similar logic for memset.
But then, does memset even make sense for xdp/skb buffers? Maybe -ENOTSUPP is more appropriate?
I'd appreciate any hints.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/tree/net/core/filter.c#n4084
Quoted 16 lines
Quoted 15 lines
kernel/bpf/helpers.c +4
-0+ + return 0; +} + __bpf_kfunc void *bpf_cast_to_kern_ctx(void *obj) {return obj;
kernel/bpf/helpers.c +1
-0@@ -3364,6 +3391,7 @@ BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) BTF_ID_FLAGS(func, bpf_dynptr_size) BTF_ID_FLAGS(func, bpf_dynptr_clone) BTF_ID_FLAGS(func, bpf_dynptr_copy) +BTF_ID_FLAGS(func, bpf_dynptr_memset) #ifdef CONFIG_NET BTF_ID_FLAGS(func, bpf_modify_return_test_tp) #endif
Eduard Zingerman
maintainerOn Thu, 2025-06-19 at 10:09 -0700, Ihor Solodrai wrote:
[...]
Quoted 1 line
But then, does memset even make sense for xdp/skb buffers?
Why not?
Quoted 3 lines
Maybe
-ENOTSUPPis more appropriate?I'd appreciate any hints.
I think Mykyta has kernel/trace/bpf_trace.c:__bpf_dynptr_copy_str() in mind.
Ihor Solodrai
reviewerQuoted 7 lines · Eduard Zingerman
On Thu, 2025-06-19 at 10:09 -0700, Ihor Solodrai wrote:
[...]
Quoted 1 line
But then, does memset even make sense for xdp/skb buffers?
Why not?
I was thinking that in a BPF program you'd usually be reading xdp/skb and not writing to it, especially not via memset. But never mind, I am likely wrong.
Quoted 6 lines
Quoted 4 lines
Maybe
-ENOTSUPPis more appropriate?I'd appreciate any hints.
I think Mykyta has kernel/trace/bpf_trace.c:__bpf_dynptr_copy_str() in mind.
Thanks for the pointer. So it looks like memset can use bpf_dynptr_slice_rdwr() to handle xdp/skb cases. Something like:
void *ptr = bpf_dynptr_slice_rdwr(&dynptr, 0, buffer, sizeof(buffer));
if (!ptr)
return -EINVAL;
memset(ptr, val, n);
if (ptr == buffer)
bpf_dynptr_write(&dynptr, 0, buffer, sizeof(buffer), 0);Quoted 1 line
Eduard Zingerman
maintainerOn Thu, 2025-06-19 at 10:55 -0700, Ihor Solodrai wrote:
[...]
Quoted 5 lines
Quoted 1 line
I think Mykyta has kernel/trace/bpf_trace.c:__bpf_dynptr_copy_str() in mind.
Thanks for the pointer. So it looks like memset can use
bpf_dynptr_slice_rdwr()to handle xdp/skb cases. Something like:
Quoted 10 lines
void *ptr = bpf_dynptr_slice_rdwr(&dynptr, 0, buffer, sizeof(buffer));
if (!ptr)
return -EINVAL;
memset(ptr, val, n);
if (ptr == buffer)
bpf_dynptr_write(&dynptr, 0, buffer, sizeof(buffer), 0);I think so. In a loop.
Ihor Solodrai
reviewerQuoted 13 lines · Eduard Zingerman
On Thu, 2025-06-19 at 10:55 -0700, Ihor Solodrai wrote:
[...]
Quoted 9 lines
Quoted 1 line
I think Mykyta has kernel/trace/bpf_trace.c:__bpf_dynptr_copy_str() in mind.
Thanks for the pointer. So it looks like memset can use
bpf_dynptr_slice_rdwr()to handle xdp/skb cases. Something like:void *ptr = bpf_dynptr_slice_rdwr(&dynptr, 0, buffer, sizeof(buffer)); if (!ptr) return -EINVAL;
Quoted 8 lines
Quoted 6 lines
memset(ptr, val, n); if (ptr == buffer) bpf_dynptr_write(&dynptr, 0, buffer, sizeof(buffer), 0);I think so. In a loop.
But allocating and copying buffers for memset...
It should be possible to walk through fragments like net/core/filter.c:bpf_xdp_copy_buf does.
Any reasons it's a bad idea?
Quoted 1 line
Eduard Zingerman
maintainerOn Thu, 2025-06-19 at 11:04 -0700, Ihor Solodrai wrote:
[...]
Quoted 5 lines
It should be possible to walk through fragments like net/core/filter.c:bpf_xdp_copy_buf does.
Any reasons it's a bad idea?
You can do that as well.
Also, what's the plan if you'd like to memset only a fragment of the memory pointed-to by dynptr?
Eduard Zingerman
maintainerOn Thu, 2025-06-19 at 11:13 -0700, Eduard Zingerman wrote:
[...]
Quoted 2 lines
Also, what's the plan if you'd like to memset only a fragment of the memory pointed-to by dynptr?
Oh, I see, there is bpf_dynptr_adjust(), sorry for the noise.
Quoted 10 lines · Eduard Zingerman
On Thu, 2025-06-19 at 11:13 -0700, Eduard Zingerman wrote:
[...]
Quoted 2 lines
Also, what's the plan if you'd like to memset only a fragment of the memory pointed-to by dynptr?
Oh, I see, there is
bpf_dynptr_adjust(), sorry for the noise.
Even though we do have bpf_dynptr_adjust() for maximum generality and flexibility, for most dynptr-based APIs we try to pass also additional offset into dynptr to avoid unnecessary overhead. So it's not a bad idea to add this to bpf_memset(), IMO.
bpf_memset(struct bpf_dynptr *dptr, u32 off, u8 val, u32 n) ?
a bit unfortunate that we have 3 integers that you need to be careful to not swap accidentally, but even with just val and n you'd have to be careful. For other APIs we normally have offset to follow dynptr pointer, so hopefully this arrangement won't that surprising.
Thoughts?
Eduard Zingerman
maintainerQuoted 17 lines · Andrii Nakryiko
Quoted 10 lines · Eduard Zingerman
On Thu, 2025-06-19 at 11:13 -0700, Eduard Zingerman wrote:
[...]
Quoted 2 lines
Also, what's the plan if you'd like to memset only a fragment of the memory pointed-to by dynptr?
Oh, I see, there is
bpf_dynptr_adjust(), sorry for the noise.Even though we do have
bpf_dynptr_adjust()for maximum generality and flexibility, for most dynptr-based APIs we try to pass also additional offset into dynptr to avoid unnecessary overhead. So it's not a bad idea to add this tobpf_memset(), IMO.
Quoted 8 lines
bpf_memset(struct bpf_dynptr *dptr, u32 off, u8 val, u32 n)?a bit unfortunate that we have 3 integers that you need to be careful to not swap accidentally, but even with just val and n you'd have to be careful. For other APIs we normally have offset to follow dynptr pointer, so hopefully this arrangement won't that surprising.
Thoughts?
Unfortunate indeed.
The off and n being separated looks weird, tbh.
For dynptr funcs we actually have "dptr, off, size" everywhere, maybe do the same here?
Quoted 26 lines · Eduard Zingerman
Quoted 24 lines · Andrii Nakryiko
Quoted 10 lines · Eduard Zingerman
On Thu, 2025-06-19 at 11:13 -0700, Eduard Zingerman wrote:
[...]
Quoted 2 lines
Also, what's the plan if you'd like to memset only a fragment of the memory pointed-to by dynptr?
Oh, I see, there is
bpf_dynptr_adjust(), sorry for the noise.Even though we do have
bpf_dynptr_adjust()for maximum generality and flexibility, for most dynptr-based APIs we try to pass also additional offset into dynptr to avoid unnecessary overhead. So it's not a bad idea to add this tobpf_memset(), IMO.
bpf_memset(struct bpf_dynptr *dptr, u32 off, u8 val, u32 n)?a bit unfortunate that we have 3 integers that you need to be careful to not swap accidentally, but even with just val and n you'd have to be careful. For other APIs we normally have offset to follow dynptr pointer, so hopefully this arrangement won't that surprising.
Quoted 6 lines
Quoted 1 line
Thoughts?
Unfortunate indeed.
The off and n being separated looks weird, tbh.
For dynptr funcs we actually have "dptr, off, size" everywhere, maybe do the same here?
yep, fine by me. I never liked libc's memset() order anyways :)
Quoted 4 lines · Eduard Zingerman
On Thu, 2025-06-19 at 10:09 -0700, Ihor Solodrai wrote:
[...]
Quoted 8 lines
Quoted 1 line
But then, does memset even make sense for xdp/skb buffers?
Why not?
Quoted 3 lines
Maybe
-ENOTSUPPis more appropriate?I'd appreciate any hints.
I think Mykyta has kernel/trace/bpf_trace.c:__bpf_dynptr_copy_str() in mind.
Right, similarly to __bpf_dynptr_copy from bpf_trace.c: fast path if destination has enough contiguous memory, just call memset.
Slow path - use temp buffer, that you can memset once and then __bpf_dynptr_write it in a loop. __bpf_dynptr_write handles the trickiness of the non-contiguous buffers.