Skip to contentSkip to conversation

Fix ring buffer callback consumption

PATCH Applied Kumar Kartikeya Dwivedi 2 participants 8 messages Nov 9

Subsystems (4)
Kselftest BPF BPF selftests libbpf
lore
Share & tools
lore

Kumar Kartikeya Dwivedi

author maintainer
#0 lore raw

cover Fix ring buffer callback consumption

Commandeering some patches from Tamir for libbpf ringbuf callback consumption fixes. See commit logs for details.

Tamir Duberstein (6):
libbpf: Honor zero consume bounds
libbpf: Prevent NULL callback crash
libbpf: Reject overwrite callback use
selftests/bpf: Test ring buffer consumption with a zero bound selftests/bpf: Test the ring buffer consumer with a NULL sample
callback
selftests/bpf: Test that callbacks reject an overwritable ring buffer

 tools/lib/bpf/libbpf.h                        |  12 +-
 tools/lib/bpf/ringbuf.c                       |  50 +++++-
 .../selftests/bpf/prog_tests/ringbuf.c        | 145 ++++++++++++++++++
 3 files changed, 196 insertions(+), 11 deletions(-)

base-commit: bd75869299f1d4ff6e9f45117cce28f8badee1cd

Signature · 1 line
2.53.0
author maintainer
#1 lore raw

PATCH 1/6 libbpf: Honor zero consume bounds

From: Tamir Duberstein <[email protected]>

ringbuf_process_ring() checks the record bound only after advancing the consumer position and invoking the callback. A zero bound therefore consumes the first available record.

Return before reading the ring positions when the bound is zero so ring_buffer__consume_n() and ring__consume_n() leave all records queued.

Fixes 4d22ea94ea33 ("libbpf: Add ring__consume_n / ring_buffer__consume_n")
Reviewed-by Emil Tsalapatis <[email protected]>
Acked-by Eduard Zingerman <[email protected]>
Signed-off-by Tamir Duberstein <[email protected]>
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
---
 tools/lib/bpf/ringbuf.c | 3 +++
 1 file changed, 3 insertions(+)
tools/lib/bpf/ringbuf.c +3 -0
index c8fe57401a8b..6e88ac209298 100644
@@ -240,6 +240,9 @@ static int64_t ringbuf_process_ring(struct ring *r, size_t n)
 	bool got_new_data;
 	void *sample;
 
+	if (n == 0)
+		return 0;
+
 	cons_pos = smp_load_acquire(r->consumer_pos);
 	do {
 		got_new_data = false;
Signature · 1 line
2.53.0
author maintainer
#2 lore raw

PATCH 2/6 libbpf: Prevent NULL callback crash

In reply to Kumar Kartikeya Dwivedi

From: Tamir Duberstein <[email protected]>

ring_buffer__new() and ring_buffer__add() allow a NULL sample callback. When callback-based consumption reaches such a ring, it calls through the NULL function pointer and crashes.

Check every ring before manager polling or consumption so a missing callback returns -EINVAL before the manager waits or consumes another ring. Check the selected ring before direct per-ring consumption. Perform both checks before honoring a zero record bound so invalid callback use always reports the error.

Fixes bf99c936f947 ("libbpf: Add BPF ring buffer support")
Signed-off-by Tamir Duberstein <[email protected]>
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
---
 tools/lib/bpf/libbpf.h  | 11 +++++------
 tools/lib/bpf/ringbuf.c | 43 ++++++++++++++++++++++++++++++++++++-----
 2 files changed, 43 insertions(+), 11 deletions(-)
tools/lib/bpf/libbpf.h +5 -6
index a8a23c135c17..1c45648fa578 100644
@@ -1628,18 +1628,17 @@ LIBBPF_API int ring__map_fd(const struct ring *r);
  *
  * @param r A ringbuffer object.
  * @return The number of records consumed (or INT_MAX, whichever is less), or
- * a negative number if any of the callbacks return an error.
+ * a negative error code on failure.
  */
 LIBBPF_API int ring__consume(struct ring *r);
 
 /**
- * @brief **ring__consume_n()** consumes up to a requested amount of items from
- * a ringbuffer without event polling.
+ * @brief **ring__consume_n()** consumes up to a requested number of records
+ * from a ringbuffer without event polling.
  *
  * @param r A ringbuffer object.
- * @param n Maximum amount of items to consume.
- * @return The number of items consumed, or a negative number if any of the
- * callbacks return an error.
+ * @param n Maximum number of records to consume.
+ * @return The number of records consumed, or a negative error code on failure.
  */
 LIBBPF_API int ring__consume_n(struct ring *r, size_t n);
tools/lib/bpf/ringbuf.c +38 -5

91 lines not on this page · raw message

Signature · 1 line
2.53.0
author maintainer
#3 lore raw

PATCH 3/6 libbpf: Reject overwrite callback use

In reply to Kumar Kartikeya Dwivedi

From: Tamir Duberstein <[email protected]>

BPF_F_RB_OVERWRITE can advance overwrite_pos past consumer_pos. Callback traversal does not read overwrite_pos, so after the producer laps the consumer it can treat overwritten data as a record header.

An earlier proposal [0] copied the readable window before invoking callbacks. Review concluded that callbacks are a poor fit because copying penalizes zero-copy users and the API cannot report skipped records.

Record the map flag and reject callback consumption with -EOPNOTSUPP.

Link https://lore.kernel.org/bpf/CAEf4Bzaq5drHWChXoRBnrmkb6reAsSVj8r=uByFSup31FMA7hw@mail.gmail.com open here
Fixes feeaf1346f80 ("bpf: Add overwrite mode for BPF ring buffer")
Signed-off-by Tamir Duberstein <[email protected]>
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
---
 tools/lib/bpf/libbpf.h  | 1 +
 tools/lib/bpf/ringbuf.c | 4 ++++
 2 files changed, 5 insertions(+)
tools/lib/bpf/libbpf.h +1 -0
index 1c45648fa578..49f1120f4021 100644
@@ -1541,6 +1541,7 @@ struct ring_buffer;
 struct ring;
 struct user_ring_buffer;
 
+/* Callback-based consumption is unsupported for BPF_F_RB_OVERWRITE maps. */
 typedef int (*ring_buffer_sample_fn)(void *ctx, void *data, size_t size);
 
 struct ring_buffer_opts {
tools/lib/bpf/ringbuf.c +4 -0
index b7fde7d0a0fd..78fcb3df243f 100644
@@ -30,6 +30,7 @@ struct ring {
 	unsigned long *producer_pos;
 	unsigned long mask;
 	int map_fd;
+	bool overwrite;
 };
 
 struct ring_buffer {

@@ -118,6 +119,7 @@ int ring_buffer__add(struct ring_buffer *rb, int map_fd,
 	r->sample_cb = sample_cb;
 	r->ctx = ctx;
 	r->mask = info.max_entries - 1;
+	r->overwrite = info.map_flags & BPF_F_RB_OVERWRITE;
 
 	/* Map writable consumer page */
 	tmp = mmap(NULL, rb->page_size, PROT_READ | PROT_WRITE, MAP_SHARED, map_fd, 0);

@@ -233,6 +235,8 @@ static inline int roundup_len(__u32 len)
 
 static int ringbuf_validate(const struct ring *r)
 {
+	if (unlikely(r->overwrite))
+		return -EOPNOTSUPP;
 	if (unlikely(!r->sample_cb))
 		return -EINVAL;
 	return 0;
Signature · 1 line
2.53.0
author maintainer
#4 lore raw

PATCH 4/6 selftests/bpf: Test ring buffer consumption with a zero bound

In reply to Kumar Kartikeya Dwivedi

From: Tamir Duberstein <[email protected]>

Extend the ringbuf_n subtest. With all samples queued, ring_buffer__consume_n() and ring__consume_n() with a zero bound must return 0, and the batched consumption that follows must still find N_SAMPLES records in every batch, so neither call may consume a record.

[kkd: split out of "libbpf: Honor zero consume bounds" (Eduard)]
Reviewed-by Emil Tsalapatis <[email protected]>
Acked-by Eduard Zingerman <[email protected]>
Signed-off-by Tamir Duberstein <[email protected]>
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
---
 tools/testing/selftests/bpf/prog_tests/ringbuf.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
tools/testing/selftests/bpf/prog_tests/ringbuf.c +13 -0
index 64520684d2cb..4f0558f14847 100644
@@ -404,6 +404,7 @@ static int process_n_sample(void *ctx, void *data, size_t len)
 static void ringbuf_n_subtest(void)
 {
 	struct test_ringbuf_n_lskel *skel_n;
+	struct ring *ring;
 	int err, i;
 
 	skel_n = test_ringbuf_n_lskel__open();

@@ -431,6 +432,18 @@ static void ringbuf_n_subtest(void)
 	for (i = 0; i < N_TOT_SAMPLES; i++)
 		syscall(__NR_getpgid);
 
+	ring = ring_buffer__ring(ringbuf, 0);
+	if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
+		goto cleanup_ringbuf;
+
+	err = ring_buffer__consume_n(ringbuf, 0);
+	if (!ASSERT_EQ(err, 0, "ringbuf_consume_zero"))
+		goto cleanup_ringbuf;
+
+	err = ring__consume_n(ring, 0);
+	if (!ASSERT_EQ(err, 0, "ring_consume_zero"))
+		goto cleanup_ringbuf;
+
 	/* Consume all samples from the ring buffer in batches of N_SAMPLES */
 	for (i = 0; i < N_TOT_SAMPLES; i += err) {
 		err = ring_buffer__consume_n(ringbuf, N_SAMPLES);
Signature · 1 line
2.53.0
author maintainer
#5 lore raw

PATCH 5/6 selftests/bpf: Test the ring buffer consumer with a NULL sample callback

In reply to Kumar Kartikeya Dwivedi

From: Tamir Duberstein <[email protected]>

Add the ringbuf_null_cb subtest. A manager holding a ring with a sample callback and one added without must return -EINVAL from ring_buffer__consume_n(), ring_buffer__consume() and ring_buffer__poll() and leave the consumer position of the valid ring alone, since the checks run before any ring is consumed. A manager created with a NULL callback must return -EINVAL from the same entry points and from ring__consume_n() and ring__consume() on its ring, instead of calling through the NULL pointer.

[kkd: split out of "libbpf: Prevent NULL callback crash" (Eduard)]
Signed-off-by Tamir Duberstein <[email protected]>
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
---
 .../selftests/bpf/prog_tests/ringbuf.c        | 93 +++++++++++++++++++
 1 file changed, 93 insertions(+)
tools/testing/selftests/bpf/prog_tests/ringbuf.c +93 -0
index 4f0558f14847..9ce996bcea8c 100644
@@ -401,6 +401,97 @@ static int process_n_sample(void *ctx, void *data, size_t len)
 	return 0;
 }
 
+static int process_noop_sample(void *ctx, void *data, size_t len)
+{
+	return 0;
+}
+
+static void ringbuf_null_cb_subtest(void)
+{
+	struct test_ringbuf_n_lskel *skel_n;
+	struct ring_buffer *ringbuf = NULL;
+	struct ring *ring;
+	unsigned long consumer_pos;
+	int no_cb_map_fd = -1;
+	int err;
+
+	skel_n = test_ringbuf_n_lskel__open();
+	if (!ASSERT_OK_PTR(skel_n, "test_ringbuf_n_lskel__open"))
+		return;
+
+	skel_n->maps.ringbuf.max_entries = getpagesize();
+	skel_n->bss->pid = getpid();
+	skel_n->bss->value = SAMPLE_VALUE;
+
+	err = test_ringbuf_n_lskel__load(skel_n);
+	if (!ASSERT_OK(err, "test_ringbuf_n_lskel__load"))
+		goto cleanup;
+
+	err = test_ringbuf_n_lskel__attach(skel_n);
+	if (!ASSERT_OK(err, "test_ringbuf_n_lskel__attach"))
+		goto cleanup;
+
+	syscall(__NR_getpgid);
+
+	no_cb_map_fd = bpf_map_create(BPF_MAP_TYPE_RINGBUF, NULL, 0, 0,
+				      getpagesize(), NULL);
+	if (!ASSERT_OK_FD(no_cb_map_fd, "bpf_map_create"))
+		goto cleanup;
+
+	/* Manager APIs must validate all rings before consuming any of them. */
+	ringbuf = ring_buffer__new(skel_n->maps.ringbuf.map_fd,
+				   process_noop_sample, NULL, NULL);
+	if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
+		goto cleanup_fd;
+
+	ring = ring_buffer__ring(ringbuf, 0);
+	if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
+		goto cleanup_ringbuf;
+
+	err = ring_buffer__add(ringbuf, no_cb_map_fd, NULL, NULL);
+	if (!ASSERT_OK(err, "ring_buffer__add_no_cb"))
+		goto cleanup_ringbuf;
+
+	consumer_pos = ring__consumer_pos(ring);
+	ASSERT_GT(ring__producer_pos(ring), consumer_pos,
+		  "producer_pos_mixed_cb");
+
+	err = ring_buffer__consume_n(ringbuf, 0);
+	ASSERT_EQ(err, -EINVAL, "ringbuf_consume_zero_mixed_cb");
+	err = ring_buffer__consume(ringbuf);
+	ASSERT_EQ(err, -EINVAL, "ringbuf_consume_mixed_cb");
+	err = ring_buffer__poll(ringbuf, 0);
+	ASSERT_EQ(err, -EINVAL, "ringbuf_poll_mixed_cb");
+	ASSERT_EQ(ring__consumer_pos(ring), consumer_pos,
+		  "consumer_pos_mixed_cb");
+
+	ring_buffer__free(ringbuf);
+	ringbuf =
+		ring_buffer__new(skel_n->maps.ringbuf.map_fd, NULL, NULL, NULL);
+	if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new_no_cb"))
+		goto cleanup_fd;
+
+	ring = ring_buffer__ring(ringbuf, 0);
+	if (!ASSERT_OK_PTR(ring, "ring_buffer__ring_no_cb"))
+		goto cleanup_ringbuf;
+	consumer_pos = ring__consumer_pos(ring);
+
+	err = ring_buffer__consume_n(ringbuf, 0);
+	ASSERT_EQ(err, -EINVAL, "ringbuf_consume_zero_no_cb");
+	err = ring__consume_n(ring, 0);
+	ASSERT_EQ(err, -EINVAL, "ring_consume_zero_no_cb");
+	err = ring__consume(ring);
+	ASSERT_EQ(err, -EINVAL, "ring_consume_no_cb");
+	ASSERT_EQ(ring__consumer_pos(ring), consumer_pos, "consumer_pos_no_cb");
+
+cleanup_ringbuf:
+	ring_buffer__free(ringbuf);
+cleanup_fd:
+	close(no_cb_map_fd);
+cleanup:
+	test_ringbuf_n_lskel__destroy(skel_n);
+}
+
 static void ringbuf_n_subtest(void)
 {
 	struct test_ringbuf_n_lskel *skel_n;

@@ -579,6 +670,8 @@ void test_ringbuf(void)
 		ringbuf_subtest();
 	if (test__start_subtest("ringbuf_n"))
 		ringbuf_n_subtest();
+	if (test__start_subtest("ringbuf_null_cb"))
+		ringbuf_null_cb_subtest();
 	if (test__start_subtest("ringbuf_map_key"))
 		ringbuf_map_key_subtest();
 	if (test__start_subtest("ringbuf_write"))
Signature · 1 line
2.53.0
author maintainer
#6 lore raw

PATCH 6/6 selftests/bpf: Test that callbacks reject an overwritable ring buffer

In reply to Kumar Kartikeya Dwivedi

From: Tamir Duberstein <[email protected]>

Add the ringbuf_overwrite_callback subtest: on a BPF_F_RB_OVERWRITE map, creating the manager succeeds, and ring_buffer__consume_n(), ring_buffer__consume(), ring_buffer__poll(), ring__consume_n() and ring__consume() all return -EOPNOTSUPP.

[kkd: split out of "libbpf: Reject overwrite callback use" (Eduard)]
Signed-off-by Tamir Duberstein <[email protected]>
Signed-off-by Kumar Kartikeya Dwivedi <[email protected]>
---
 .../selftests/bpf/prog_tests/ringbuf.c        | 39 +++++++++++++++++++
 1 file changed, 39 insertions(+)
tools/testing/selftests/bpf/prog_tests/ringbuf.c +39 -0
index 9ce996bcea8c..29be2476c478 100644
@@ -602,6 +602,43 @@ static void ringbuf_map_key_subtest(void)
 	test_ringbuf_map_key_lskel__destroy(skel_map_key);
 }
 
+static void ringbuf_overwrite_callback_subtest(void)
+{
+	LIBBPF_OPTS(bpf_map_create_opts, opts, .map_flags = BPF_F_RB_OVERWRITE);
+	struct ring_buffer *ringbuf;
+	struct ring *ring;
+	int map_fd, err;
+
+	map_fd = bpf_map_create(BPF_MAP_TYPE_RINGBUF, NULL, 0, 0, getpagesize(),
+				&opts);
+	if (!ASSERT_OK_FD(map_fd, "bpf_map_create"))
+		return;
+
+	ringbuf = ring_buffer__new(map_fd, process_noop_sample, NULL, NULL);
+	if (!ASSERT_OK_PTR(ringbuf, "ring_buffer__new"))
+		goto cleanup_fd;
+
+	ring = ring_buffer__ring(ringbuf, 0);
+	if (!ASSERT_OK_PTR(ring, "ring_buffer__ring"))
+		goto cleanup_ringbuf;
+
+	err = ring_buffer__consume_n(ringbuf, 0);
+	ASSERT_EQ(err, -EOPNOTSUPP, "ringbuf_consume_zero");
+	err = ring_buffer__consume(ringbuf);
+	ASSERT_EQ(err, -EOPNOTSUPP, "ringbuf_consume");
+	err = ring_buffer__poll(ringbuf, 0);
+	ASSERT_EQ(err, -EOPNOTSUPP, "ringbuf_poll");
+	err = ring__consume_n(ring, 0);
+	ASSERT_EQ(err, -EOPNOTSUPP, "ring_consume_zero");
+	err = ring__consume(ring);
+	ASSERT_EQ(err, -EOPNOTSUPP, "ring_consume");
+
+cleanup_ringbuf:
+	ring_buffer__free(ringbuf);
+cleanup_fd:
+	close(map_fd);
+}
+
 static void ringbuf_overwrite_mode_subtest(void)
 {
 	unsigned long size, len1, len2, len3, len4, len5;

@@ -676,6 +713,8 @@ void test_ringbuf(void)
 		ringbuf_map_key_subtest();
 	if (test__start_subtest("ringbuf_write"))
 		ringbuf_write_subtest();
+	if (test__start_subtest("ringbuf_overwrite_callback"))
+		ringbuf_overwrite_callback_subtest();
 	if (test__start_subtest("ringbuf_overwrite_mode"))
 		ringbuf_overwrite_mode_subtest();
 }
Signature · 1 line
2.53.0
CI Applied
#7 lore raw
In reply to Kumar Kartikeya Dwivedi
This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Hello:

This series was applied to bpf/bpf-next.git (master) by Kumar Kartikeya Dwivedi <[email protected]>:

Quoted 12 lines · On Mon, 9 Nov 2026 20:47:26 +0100 you wrote:

Commandeering some patches from Tamir for libbpf ringbuf callback consumption fixes. See commit logs for details.

Tamir Duberstein (6):
libbpf: Honor zero consume bounds
libbpf: Prevent NULL callback crash
libbpf: Reject overwrite callback use
selftests/bpf: Test ring buffer consumption with a zero bound selftests/bpf: Test the ring buffer consumer with a NULL sample
callback
selftests/bpf: Test that callbacks reject an overwritable ring buffer

Quoted 1 line

[...]

Here is the summary with links:
- [bpf-next,v1,1/6] libbpf: Honor zero consume bounds

    https://git.kernel.org/bpf/bpf-next/c/40edacd47993
  - [bpf-next,v1,2/6] libbpf: Prevent NULL callback crash
    https://git.kernel.org/bpf/bpf-next/c/c0628a0659b6
  - [bpf-next,v1,3/6] libbpf: Reject overwrite callback use
    https://git.kernel.org/bpf/bpf-next/c/f756eedb24d4
  - [bpf-next,v1,4/6] selftests/bpf: Test ring buffer consumption with a zero bound
    https://git.kernel.org/bpf/bpf-next/c/c82153312a0a
  - [bpf-next,v1,5/6] selftests/bpf: Test the ring buffer consumer with a NULL sample callback
    https://git.kernel.org/bpf/bpf-next/c/21f96738feb7
  - [bpf-next,v1,6/6] selftests/bpf: Test that callbacks reject an overwritable ring buffer
    https://git.kernel.org/bpf/bpf-next/c/249c9bb9f3ac

You are awesome, thank you!

Signature · 2 lines
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html