bpf: warn against BPF_RB_NO_WAKEUP in bpf_ringbuf_discard()
PATCH 1/1 bpf: warn against BPF_RB_NO_WAKEUP in bpf_ringbuf_discard()
Document that BPF_RB_NO_WAKEUP is not recommended for bpf_ringbuf_discard().
A discard done with BPF_RB_NO_WAKEUP can suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.
Scenario:
epoll_wait(rb_fd); // blocks
rec = bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);
rec = bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_submit(rec, 0); // valid record, but no wakeup
This behavior is surprising in the context of bpf_ringbuf_discard() as it seems natural not to want to wake userspace.
--- include/uapi/linux/bpf.h | 3 ++- tools/include/uapi/linux/bpf.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)
include/uapi/linux/bpf.h +2 -1
@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent. +* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit. * If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification
tools/include/uapi/linux/bpf.h +2 -1
@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent. +* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit. * If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification
Signature · 1 line
2.43.0
Quoted 11 lines · Eyal Birger
Document that
BPF_RB_NO_WAKEUPis not recommended forbpf_ringbuf_discard().A discard done with
BPF_RB_NO_WAKEUPcan suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.Scenario:
epoll_wait(rb_fd); // blocks
Quoted 8 lines
rec =
bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);rec =
bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_submit(rec, 0); // valid record, but no wakeupThis behavior is surprising in the context of
bpf_ringbuf_discard()as it seems natural not to want to wake userspace.
It appears that once you do a submit or discard with NO_WAKEUP, you can never go back to adaptive mode. You will need to do an explicit BPF_RB_FORCE_WAKEUP to do a wakeup.
This looks like expected behaviour (by design) but for discard the programmer might think: "I'm discarding and there's no data, why would I wake anyone? Let me pass BPF_RB_NO_WAKEUP to be a good citizen."
I am not sure if we just want to change the description of the helper or also fix the code to ignore the BPF_RB_NO_WAKEUP for discard?
Or a better approach would be to add a wakeup_needed flag to struct bpf_ringbuf. When NO_WAKEUP suppresses a wakeup that would have fired (cons_pos == rec_pos), it sets the flag. Any subsequent adaptive commit sees the flag and sends the wakeup, clearing it. FORCE_WAKEUP also clears it. But this could change how the ringbuf behaves for existing programs.
I will let others comment on this.
Quoted 35 lines
Reported-by Shmulik Ladkani <[email protected]>Signed-off-by Eyal Birger <[email protected]>--- include/uapi/linux/bpf.h | 3 ++- tools/include/uapi/linux/bpf.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)include/uapi/linux/bpf.h +2
-1@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent. +* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit. * If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification tools/include/uapi/linux/bpf.h +2
-1@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent. +* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit. * If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification
Hi,
Quoted 23 lines · Puranjay Mohan
Quoted 11 lines · Eyal Birger
Document that
BPF_RB_NO_WAKEUPis not recommended forbpf_ringbuf_discard().A discard done with
BPF_RB_NO_WAKEUPcan suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.Scenario:
epoll_wait(rb_fd); // blocksQuoted 8 lines
rec =
bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);rec =
bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_submit(rec, 0); // valid record, but no wakeupThis behavior is surprising in the context of
bpf_ringbuf_discard()as it seems natural not to want to wake userspace.
Quoted 7 lines
It appears that once you do a submit or discard with
NO_WAKEUP, you can never go back to adaptive mode. You will need to do an explicitBPF_RB_FORCE_WAKEUPto do a wakeup.This looks like expected behaviour (by design) but for discard the programmer might think: "I'm discarding and there's no data, why would I wake anyone? Let me pass
BPF_RB_NO_WAKEUPto be a good citizen."
This is exactly what I've seen and it created real bugs.
Quoted 3 lines
I am not sure if we just want to change the description of the helper or also fix the code to ignore the
BPF_RB_NO_WAKEUPfor discard?
I wanted to avoid breaking something that might be in use by advanced use-cases.
Quoted 7 lines
Or a better approach would be to add a
wakeup_neededflag to structbpf_ringbuf. WhenNO_WAKEUPsuppresses a wakeup that would have fired (cons_pos==rec_pos), it sets the flag. Any subsequent adaptive commit sees the flag and sends the wakeup, clearing it.FORCE_WAKEUPalso clears it. But this could change how the ringbuf behaves for existing programs.
Considered this, but wanted to avoid a runtime penalty for this edge case.
Quoted 2 lines
I will let others comment on this.
Thanks for your input!
Eyal.
Quoted 36 lines
Quoted 36 lines
Reported-by Shmulik Ladkani <[email protected]>Signed-off-by Eyal Birger <[email protected]>--- include/uapi/linux/bpf.h | 3 ++- tools/include/uapi/linux/bpf.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)include/uapi/linux/bpf.h +2
-1@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent. +* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit. * If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification tools/include/uapi/linux/bpf.h +2
-1@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent. +* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit. * If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification
Andrii Nakryiko
NAKQuoted 12 lines · Eyal Birger
Document that
BPF_RB_NO_WAKEUPis not recommended forbpf_ringbuf_discard().A discard done with
BPF_RB_NO_WAKEUPcan suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.Scenario:
epoll_wait(rb_fd); // blocks
Quoted 8 lines
rec =
bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);rec =
bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_submit(rec, 0); // valid record, but no wakeupThis behavior is surprising in the context of
bpf_ringbuf_discard()as it seems natural not to want to wake userspace.
why? discarded record still takes space in ringbuf and needs to be consumed by user-space. So all the same concerns apply here whether it's discarded or submitted ringbuf record.
Quoted 13 lines
Reported-by Shmulik Ladkani <[email protected]>Signed-off-by Eyal Birger <[email protected]>--- include/uapi/linux/bpf.h | 3 ++- tools/include/uapi/linux/bpf.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)include/uapi/linux/bpf.h +0
-0
include/uapi/linux/bpf.h union bpf_attr { · line 4649
Quoted 6 lines
include/uapi/linux/bpf.h +2
-1@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent. +* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit.
NAK, it's not "not recommended". If you are using NO_WAKEUP you normally would use it with FORCE_WAKEUP as well. It's not a rule, but that's the most practical combination. But if you are fancy and know what you are doing, you are free to combine NO_WAKEUP, FORCE_WAKEUP and "adaptive wakeup" in any form or shape you want.
If anything, instead, it might maybe help to emphasize that bpf_ringbuf_discard() does live data in the ringbuf that needs to be consumed by user-space code (normally transparently in libbpf or other BPF libraries), and the only distinction between submitted and discarded record is whether application's user code sees that record or it's silently skipped.
Quoted 20 lines
include/uapi/linux/bpf.h +0
-0* If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification tools/include/uapi/linux/bpf.h +2
-2@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent. +* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit. * If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification -- 2.43.0
Hi,
Quoted 20 lines · Andrii Nakryiko
Quoted 18 lines · Eyal Birger
Document that
BPF_RB_NO_WAKEUPis not recommended forbpf_ringbuf_discard().A discard done with
BPF_RB_NO_WAKEUPcan suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.Scenario:
epoll_wait(rb_fd); // blocksrec =
bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);rec =
bpf_ringbuf_reserve(&rb, ...);bpf_ringbuf_submit(rec, 0); // valid record, but no wakeup
Quoted 6 lines
Quoted 2 lines
This behavior is surprising in the context of
bpf_ringbuf_discard()as it seems natural not to want to wake userspace.why? discarded record still takes space in ringbuf and needs to be consumed by user-space. So all the same concerns apply here whether it's discarded or submitted ringbuf record.
Right, but sadly this isn't obvious to someone who hasn't carefully inspected the implementation.
Quoted 18 lines
Quoted 18 lines
Reported-by Shmulik Ladkani <[email protected]>Signed-off-by Eyal Birger <[email protected]>--- include/uapi/linux/bpf.h | 3 ++- tools/include/uapi/linux/bpf.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)include/uapi/linux/bpf.h +0
-1@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent.
Quoted 8 lines
include/uapi/linux/bpf.h union bpf_attr { · line 4649
Quoted 2 lines
include/uapi/linux/bpf.h +2
-0+* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit. NAK, it's not "not recommended". If you are using
NO_WAKEUPyou normally would use it withFORCE_WAKEUPas well. It's not a rule, but that's the most practical combination. But if you are fancy and know what you are doing, you are free to combineNO_WAKEUP,FORCE_WAKEUPand "adaptive wakeup" in any form or shape you want.
There are still possible corner cases - for example, multiple silent discards might fill the ringbuf preventing a later placement of a valid record - even if it's meant to use "force".
Point is, the user should be extra careful, that fact is documented in other places, and imho should be noted here too.
Quoted 8 lines
If anything, instead, it might maybe help to emphasize that
bpf_ringbuf_discard()does live data in the ringbuf that needs to be consumed by user-space code (normally transparently in libbpf or other BPF libraries), and the only distinction between submitted and discarded record is whether application's user code sees that record or it's silently skipped.
Ack. Will send a v2 in that spirit.
Eyal.
Quoted 21 lines
Quoted 21 lines
include/uapi/linux/bpf.h +0
-0* If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification tools/include/uapi/linux/bpf.h +2
-2@@ -4645,7 +4645,8 @@ union bpf_attr {* Description * Discard reserved ring buffer sample, pointed to by *data*. * If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification -* of new data availability is sent. +* of new data availability is sent, which is not recommended as +* it can suppress a later adaptive wakeup from a subsequent submit. * If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification * of new data availability is sent unconditionally. * If **0** is specified in *flags*, an adaptive notification -- 2.43.0