Skip to contentSkip to conversation

bpf: warn against BPF_RB_NO_WAKEUP in bpf_ringbuf_discard()

PATCH NAK Eyal Birger 3 participants 5 messages Mar 30 – Mar 31 BPF

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bpf: warn against BPF_RB_NO_WAKEUP in bpf_ringbuf_discard()

Document that BPF_RB_NO_WAKEUP is not recommended for bpf_ringbuf_discard().

A discard done with BPF_RB_NO_WAKEUP can suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.

Scenario:

epoll_wait(rb_fd); // blocks

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_submit(rec, 0); // valid record, but no wakeup

This behavior is surprising in the context of bpf_ringbuf_discard() as it seems natural not to want to wake userspace.

---
 include/uapi/linux/bpf.h       | 3 ++-
 tools/include/uapi/linux/bpf.h | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)
include/uapi/linux/bpf.h +2 -1
index c8d400b7680a..c46b06d45904 100644
@@ -4645,7 +4645,8 @@ union bpf_attr {
  * 	Description
  * 		Discard reserved ring buffer sample, pointed to by *data*.
  * 		If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- * 		of new data availability is sent.
+ * 		of new data availability is sent, which is not recommended as
+ * 		it can suppress a later adaptive wakeup from a subsequent submit.
  * 		If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  * 		of new data availability is sent unconditionally.
  * 		If **0** is specified in *flags*, an adaptive notification
tools/include/uapi/linux/bpf.h +2 -1
index 5e38b4887de6..96de37c3b896 100644
@@ -4645,7 +4645,8 @@ union bpf_attr {
  * 	Description
  * 		Discard reserved ring buffer sample, pointed to by *data*.
  * 		If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- * 		of new data availability is sent.
+ * 		of new data availability is sent, which is not recommended as
+ * 		it can suppress a later adaptive wakeup from a subsequent submit.
  * 		If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  * 		of new data availability is sent unconditionally.
  * 		If **0** is specified in *flags*, an adaptive notification
Signature · 1 line
2.43.0
#1 lore raw
Quoted 11 lines · Eyal Birger

Document that BPF_RB_NO_WAKEUP is not recommended for bpf_ringbuf_discard().

A discard done with BPF_RB_NO_WAKEUP can suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.

Scenario:

epoll_wait(rb_fd); // blocks

Quoted 8 lines

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_submit(rec, 0); // valid record, but no wakeup

This behavior is surprising in the context of bpf_ringbuf_discard() as it seems natural not to want to wake userspace.

It appears that once you do a submit or discard with NO_WAKEUP, you can never go back to adaptive mode. You will need to do an explicit BPF_RB_FORCE_WAKEUP to do a wakeup.

This looks like expected behaviour (by design) but for discard the programmer might think: "I'm discarding and there's no data, why would I wake anyone? Let me pass BPF_RB_NO_WAKEUP to be a good citizen."

I am not sure if we just want to change the description of the helper or also fix the code to ignore the BPF_RB_NO_WAKEUP for discard?

Or a better approach would be to add a wakeup_needed flag to struct bpf_ringbuf. When NO_WAKEUP suppresses a wakeup that would have fired (cons_pos == rec_pos), it sets the flag. Any subsequent adaptive commit sees the flag and sends the wakeup, clearing it. FORCE_WAKEUP also clears it. But this could change how the ringbuf behaves for existing programs.

I will let others comment on this.

Quoted 35 lines
---
 include/uapi/linux/bpf.h       | 3 ++-
 tools/include/uapi/linux/bpf.h | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)
include/uapi/linux/bpf.h +2 -1
index c8d400b7680a..c46b06d45904 100644
@@ -4645,7 +4645,8 @@ union bpf_attr {
  * 	Description
  * 		Discard reserved ring buffer sample, pointed to by *data*.
  * 		If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- * 		of new data availability is sent.
+ * 		of new data availability is sent, which is not recommended as
+ * 		it can suppress a later adaptive wakeup from a subsequent submit.
  * 		If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  * 		of new data availability is sent unconditionally.
  * 		If **0** is specified in *flags*, an adaptive notification
tools/include/uapi/linux/bpf.h +2 -1
index 5e38b4887de6..96de37c3b896 100644
@@ -4645,7 +4645,8 @@ union bpf_attr {
  * 	Description
  * 		Discard reserved ring buffer sample, pointed to by *data*.
  * 		If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- * 		of new data availability is sent.
+ * 		of new data availability is sent, which is not recommended as
+ * 		it can suppress a later adaptive wakeup from a subsequent submit.
  * 		If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  * 		of new data availability is sent unconditionally.
  * 		If **0** is specified in *flags*, an adaptive notification
#2 lore raw

Hi,

Quoted 23 lines · Puranjay Mohan
Quoted 11 lines · Eyal Birger

Document that BPF_RB_NO_WAKEUP is not recommended for bpf_ringbuf_discard().

A discard done with BPF_RB_NO_WAKEUP can suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.

Scenario:

epoll_wait(rb_fd); // blocks

Quoted 8 lines

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_submit(rec, 0); // valid record, but no wakeup

This behavior is surprising in the context of bpf_ringbuf_discard() as it seems natural not to want to wake userspace.

Quoted 7 lines

It appears that once you do a submit or discard with NO_WAKEUP, you can never go back to adaptive mode. You will need to do an explicit BPF_RB_FORCE_WAKEUP to do a wakeup.

This looks like expected behaviour (by design) but for discard the programmer might think: "I'm discarding and there's no data, why would I wake anyone? Let me pass BPF_RB_NO_WAKEUP to be a good citizen."

This is exactly what I've seen and it created real bugs.

Quoted 3 lines

I am not sure if we just want to change the description of the helper or also fix the code to ignore the BPF_RB_NO_WAKEUP for discard?

I wanted to avoid breaking something that might be in use by advanced use-cases.

Quoted 7 lines

Or a better approach would be to add a wakeup_needed flag to struct bpf_ringbuf. When NO_WAKEUP suppresses a wakeup that would have fired (cons_pos == rec_pos), it sets the flag. Any subsequent adaptive commit sees the flag and sends the wakeup, clearing it. FORCE_WAKEUP also clears it. But this could change how the ringbuf behaves for existing programs.

Considered this, but wanted to avoid a runtime penalty for this edge case.

Quoted 2 lines

I will let others comment on this.

Thanks for your input!

Eyal.

Quoted 36 lines
Quoted 36 lines
---
 include/uapi/linux/bpf.h       | 3 ++-
 tools/include/uapi/linux/bpf.h | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)
include/uapi/linux/bpf.h +2 -1
index c8d400b7680a..c46b06d45904 100644
@@ -4645,7 +4645,8 @@ union bpf_attr {
  *   Description
  *           Discard reserved ring buffer sample, pointed to by *data*.
  *           If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- *           of new data availability is sent.
+ *           of new data availability is sent, which is not recommended as
+ *           it can suppress a later adaptive wakeup from a subsequent submit.
  *           If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  *           of new data availability is sent unconditionally.
  *           If **0** is specified in *flags*, an adaptive notification
tools/include/uapi/linux/bpf.h +2 -1
index 5e38b4887de6..96de37c3b896 100644
@@ -4645,7 +4645,8 @@ union bpf_attr {
  *   Description
  *           Discard reserved ring buffer sample, pointed to by *data*.
  *           If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- *           of new data availability is sent.
+ *           of new data availability is sent, which is not recommended as
+ *           it can suppress a later adaptive wakeup from a subsequent submit.
  *           If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  *           of new data availability is sent unconditionally.
  *           If **0** is specified in *flags*, an adaptive notification
#3 lore raw
In reply to Eyal Birger
Quoted 12 lines · Eyal Birger

Document that BPF_RB_NO_WAKEUP is not recommended for bpf_ringbuf_discard().

A discard done with BPF_RB_NO_WAKEUP can suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.

Scenario:

epoll_wait(rb_fd); // blocks

Quoted 8 lines

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_submit(rec, 0); // valid record, but no wakeup

This behavior is surprising in the context of bpf_ringbuf_discard() as it seems natural not to want to wake userspace.

why? discarded record still takes space in ringbuf and needs to be consumed by user-space. So all the same concerns apply here whether it's discarded or submitted ringbuf record.

Quoted 13 lines
---
 include/uapi/linux/bpf.h       | 3 ++-
 tools/include/uapi/linux/bpf.h | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)
include/uapi/linux/bpf.h +0 -0
index c8d400b7680a..c46b06d45904 100644

include/uapi/linux/bpf.h union bpf_attr { · line 4649

Quoted 6 lines
include/uapi/linux/bpf.h +2 -1
@@ -4645,7 +4645,8 @@ union bpf_attr {
  *     Description
  *             Discard reserved ring buffer sample, pointed to by *data*.
  *             If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- *             of new data availability is sent.
+ *             of new data availability is sent, which is not recommended as
+ *             it can suppress a later adaptive wakeup from a subsequent submit.

NAK, it's not "not recommended". If you are using NO_WAKEUP you normally would use it with FORCE_WAKEUP as well. It's not a rule, but that's the most practical combination. But if you are fancy and know what you are doing, you are free to combine NO_WAKEUP, FORCE_WAKEUP and "adaptive wakeup" in any form or shape you want.

If anything, instead, it might maybe help to emphasize that bpf_ringbuf_discard() does live data in the ringbuf that needs to be consumed by user-space code (normally transparently in libbpf or other BPF libraries), and the only distinction between submitted and discarded record is whether application's user code sees that record or it's silently skipped.

Quoted 20 lines
include/uapi/linux/bpf.h +0 -0
  *             If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  *             of new data availability is sent unconditionally.
  *             If **0** is specified in *flags*, an adaptive notification
tools/include/uapi/linux/bpf.h +2 -2
index 5e38b4887de6..96de37c3b896 100644
@@ -4645,7 +4645,8 @@ union bpf_attr {
  *     Description
  *             Discard reserved ring buffer sample, pointed to by *data*.
  *             If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- *             of new data availability is sent.
+ *             of new data availability is sent, which is not recommended as
+ *             it can suppress a later adaptive wakeup from a subsequent submit.
  *             If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  *             of new data availability is sent unconditionally.
  *             If **0** is specified in *flags*, an adaptive notification
--

2.43.0

#4 lore raw

Hi,

Quoted 20 lines · Andrii Nakryiko
Quoted 18 lines · Eyal Birger

Document that BPF_RB_NO_WAKEUP is not recommended for bpf_ringbuf_discard().

A discard done with BPF_RB_NO_WAKEUP can suppress a later adaptive wakeup from a valid record, leaving an epoll-based userspace consumer asleep even though data is available in the ring buffer.

Scenario:

epoll_wait(rb_fd); // blocks

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_discard(rec, BPF_RB_NO_WAKEUP);

rec = bpf_ringbuf_reserve(&rb, ...);
bpf_ringbuf_submit(rec, 0); // valid record, but no wakeup

Quoted 6 lines
Quoted 2 lines

This behavior is surprising in the context of bpf_ringbuf_discard() as it seems natural not to want to wake userspace.

why? discarded record still takes space in ringbuf and needs to be consumed by user-space. So all the same concerns apply here whether it's discarded or submitted ringbuf record.

Right, but sadly this isn't obvious to someone who hasn't carefully inspected the implementation.

Quoted 18 lines
Quoted 18 lines
---
 include/uapi/linux/bpf.h       | 3 ++-
 tools/include/uapi/linux/bpf.h | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)
include/uapi/linux/bpf.h +0 -1
index c8d400b7680a..c46b06d45904 100644
@@ -4645,7 +4645,8 @@ union bpf_attr {
  *     Description
  *             Discard reserved ring buffer sample, pointed to by *data*.
  *             If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- *             of new data availability is sent.
Quoted 8 lines

include/uapi/linux/bpf.h union bpf_attr { · line 4649

Quoted 2 lines
include/uapi/linux/bpf.h +2 -0
+ *             of new data availability is sent, which is not recommended as
+ *             it can suppress a later adaptive wakeup from a subsequent submit.

NAK, it's not "not recommended". If you are using NO_WAKEUP you normally would use it with FORCE_WAKEUP as well. It's not a rule, but that's the most practical combination. But if you are fancy and know what you are doing, you are free to combine NO_WAKEUP, FORCE_WAKEUP and "adaptive wakeup" in any form or shape you want.

There are still possible corner cases - for example, multiple silent discards might fill the ringbuf preventing a later placement of a valid record - even if it's meant to use "force".
Point is, the user should be extra careful, that fact is documented in other places, and imho should be noted here too.

Quoted 8 lines

If anything, instead, it might maybe help to emphasize that bpf_ringbuf_discard() does live data in the ringbuf that needs to be consumed by user-space code (normally transparently in libbpf or other BPF libraries), and the only distinction between submitted and discarded record is whether application's user code sees that record or it's silently skipped.

Ack. Will send a v2 in that spirit.

Eyal.

Quoted 21 lines
Quoted 21 lines
include/uapi/linux/bpf.h +0 -0
  *             If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  *             of new data availability is sent unconditionally.
  *             If **0** is specified in *flags*, an adaptive notification
tools/include/uapi/linux/bpf.h +2 -2
index 5e38b4887de6..96de37c3b896 100644
@@ -4645,7 +4645,8 @@ union bpf_attr {
  *     Description
  *             Discard reserved ring buffer sample, pointed to by *data*.
  *             If **BPF_RB_NO_WAKEUP** is specified in *flags*, no notification
- *             of new data availability is sent.
+ *             of new data availability is sent, which is not recommended as
+ *             it can suppress a later adaptive wakeup from a subsequent submit.
  *             If **BPF_RB_FORCE_WAKEUP** is specified in *flags*, notification
  *             of new data availability is sent unconditionally.
  *             If **0** is specified in *flags*, an adaptive notification
--

2.43.0