Skip to contentSkip to conversation

ring-buffer: Updates for 7.4

PATCH New Steven Rostedt 1 participant 6 messages Oct 19

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover ring-buffer: Updates for 7.4

git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
ring-buffer/for-next

Head SHA1: 453d0f47b174cac387af48d303c503b84f82212e

David Carlier (1):
ring-buffer: Take cpus_read_lock() when changing the subbuf order

Steven Rostedt (1):
ring-buffer: Add a MAINTAINERS category specific for the tracing ring-buffer

Vincent Donnefort (3):
ring-buffer: Unify ring buffer minimum page calculations
ring-buffer: Clean up resize_disabled checks
ring-buffer: Remove unnecessary cpumask_test_cpu

----

 MAINTAINERS                       |  16 ++++++
 include/linux/ring_buffer.h       |   9 +++-
 kernel/trace/ring_buffer.c        | 102 ++++++++++++++++----------------------
 kernel/trace/simple_ring_buffer.c |   4 +-
 4 files changed, 68 insertions(+), 63 deletions(-)
#1 lore raw

PATCH 1/5 ring-buffer: Add a MAINTAINERS category specific for the tracing ring-buffer

From: Steven Rostedt <[email protected]>

The ring buffer code in the tracing directory has always been decoupled from the tracing code. It had a clean abstracted API to allow it to be used by systems outside of tracing. The API even allowed the ring buffer code to be drastically changed without affecting the tracing sub-system.

Recently, Vincent Donnefort has been contributing a lot of work to enhance the ring buffer. He implemented the user space mappings and recently created a version to allow pKVM hypervisors to use the code to trace the hypervisor and expose the events to the user.

Create a separate category for maintaining the ring buffer code so that Vincent can become an official maintainer of it. His work has little to do with the tracing code itself. In the future, he may even be the main maintainer of the ring buffer.

---
 MAINTAINERS | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
MAINTAINERS +16 -0
index f53047aa8d3c..c29e2a4f1f69 100644
@@ -27794,6 +27794,22 @@ F:	kernel/trace/trace_irqsoff.c
 F:	kernel/trace/trace_osnoise.c
 F:	kernel/trace/trace_sched_wakeup.c
 
+TRACING RING BUFFER
+M:	Steven Rostedt <[email protected]>
+M:	Vincent Donnefort <[email protected]>
+M:	Masami Hiramatsu <[email protected]>
+L:	[email protected]
+L:	[email protected]
+S:	Maintained
+Q:	https://patchwork.kernel.org/project/linux-trace-kernel/list/
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
+F:	Documentation/trace/ring-buffer*
+F:	Documentation/trace/remotes.rst
+F:	include/linux/ring-buffer*.h
+F:	kernel/trace/*ring_buffer*.c
+F:	kernel/trace/remote*.c
+F:	tools/testing/selftests/ring-buffer/
+
 TRADITIONAL CHINESE DOCUMENTATION
 M:	Chen-Yu Yeh <[email protected]>
 M:	Weijie Yuan <[email protected]>
Signature · 1 line
2.53.0
#2 lore raw

PATCH 2/5 ring-buffer: Unify ring buffer minimum page calculations

In reply to Steven Rostedt

From: Vincent Donnefort <[email protected]>

The codebase scatters hardcoded minimum page limits across multiple places. Introduce RING_BUFFER_MIN_NR_PAGES and rb_calculate_nr_pages() to formalize the requirement of two pages in a ring buffer.

---
 include/linux/ring_buffer.h       |  9 ++++++++-
 kernel/trace/ring_buffer.c        | 32 +++++++++++++------------------
 kernel/trace/simple_ring_buffer.c |  4 ++--
 3 files changed, 23 insertions(+), 22 deletions(-)
include/linux/ring_buffer.h +8 -1
index eac3e9080c3c..e4370064273e 100644
@@ -98,6 +98,13 @@ struct trace_buffer *__ring_buffer_alloc_range(unsigned long size, unsigned flag
 
 void *ring_buffer_meta_scratch(struct trace_buffer *buffer, unsigned int *size);
 
+/*
+ * A minimum of 2 pages ensures we can always swap a page out of the ring
+ * buffer. This does not account for the reader page or any other page
+ * containing meta data used by static ring buffers.
+ */
+#define RING_BUFFER_MIN_NR_PAGES 2UL
+
 /*
  * Because the ring buffer is generic, if other users of the ring buffer get
  * traced by ftrace, it can produce lockdep warnings. We need to keep each
@@ -287,7 +294,7 @@ static inline struct ring_buffer_desc *__first_ring_buffer_desc(struct trace_buf
 static inline unsigned long __calc_nr_pages_ring_buffer_desc(size_t size)
 {
 	/* Takes into account the reader page */
-	return max(DIV_ROUND_UP(size, PAGE_SIZE - BUF_PAGE_HDR_SIZE), 2UL) + 1;
+	return max(DIV_ROUND_UP(size, PAGE_SIZE - BUF_PAGE_HDR_SIZE), RING_BUFFER_MIN_NR_PAGES) + 1;
 }
 
 static inline size_t trace_buffer_desc_size(size_t buffer_size, unsigned int nr_cpus)
kernel/trace/ring_buffer.c +13 -19
index 04bb94c29f58..1b148cd891c8 100644
@@ -642,6 +642,11 @@ static __always_inline unsigned int rb_subbuf_max_data_size(struct trace_buffer
 	return rb_subbuf_capacity(buffer) - RB_EVNT_HDR_SIZE - sizeof(event->array[0]);
 }
 
+static unsigned long rb_calculate_nr_pages(struct trace_buffer *buffer, unsigned long size)
+{
+	return max(DIV_ROUND_UP(size, rb_subbuf_capacity(buffer)), RING_BUFFER_MIN_NR_PAGES);
+}
+
 /**
  * rb_subbuf_start - Get the start address of a subbuffer
  * @buffer:	A trace buffer
@@ -2864,7 +2869,7 @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
 			goto fail_free_buffers;
 
 		/* Need at least two pages plus the reader page */
-		if (nr_pages < 3)
+		if (nr_pages < RING_BUFFER_MIN_NR_PAGES + 1)
 			goto fail_free_buffers;
 
  again:
@@ -2876,7 +2881,7 @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
 			ptr += subbuf_size * nr_pages;
 		}
 		if (ptr > end) {
-			if (nr_pages <= 3)
+			if (nr_pages <= RING_BUFFER_MIN_NR_PAGES + 1)
 				goto fail_free_buffers;
 			nr_pages--;
 			goto again;
@@ -2899,13 +2904,10 @@ static struct trace_buffer *alloc_buffer(unsigned long size, unsigned flags,
 		if (nr_pages > rb_static_max_pages())
 			goto fail_free_buffers;
 
-		if (nr_pages < 2)
+		if (nr_pages < RING_BUFFER_MIN_NR_PAGES)
 			goto fail_free_buffers;
 	} else {
-		/* need at least two pages */
-		nr_pages = DIV_ROUND_UP(size, rb_subbuf_capacity(buffer));
-		if (nr_pages < 2)
-			nr_pages = 2;
+		nr_pages = rb_calculate_nr_pages(buffer, size);
 	}
 
 	cpumask_set_cpu(cpu, buffer->cpumask);
@@ -3318,11 +3320,7 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size,
 	mutex_lock(&buffer->mutex);
 	atomic_inc(&buffer->resizing);
 
-	nr_pages = DIV_ROUND_UP(size, rb_subbuf_capacity(buffer));
-
-	/* we need a minimum of two pages */
-	if (nr_pages < 2)
-		nr_pages = 2;
+	nr_pages = rb_calculate_nr_pages(buffer, size);
 
 	if (cpu_id == RING_BUFFER_ALL_CPUS) {
 		/*
@@ -7502,12 +7500,8 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)
 		}
 
 		/* Update the number of pages to match the new size */
-		nr_pages = old_capacity * buffer->buffers[cpu]->nr_pages;
-		nr_pages = DIV_ROUND_UP(nr_pages, rb_subbuf_capacity(buffer));
-
-		/* we need a minimum of two pages */
-		if (nr_pages < 2)
-			nr_pages = 2;
+		nr_pages = rb_calculate_nr_pages(buffer,
+						 old_capacity * buffer->buffers[cpu]->nr_pages);
 
 		cpu_buffer->nr_pages_to_update = nr_pages;
 
@@ -8102,7 +8096,7 @@ int trace_rb_cpu_prepare(unsigned int cpu, struct hlist_node *node)
 	}
 	/* allocate minimum pages, user can later expand it */
 	if (!nr_pages_same)
-		nr_pages = 2;
+		nr_pages = RING_BUFFER_MIN_NR_PAGES;
 	buffer->buffers[cpu] =
 		rb_allocate_cpu_buffer(buffer, nr_pages, cpu);
 	if (!buffer->buffers[cpu]) {
kernel/trace/simple_ring_buffer.c +2 -2
index 49913bb0057a..abfaf9bee12e 100644
@@ -383,8 +383,8 @@ int simple_ring_buffer_init_mm(struct simple_rb_per_cpu *cpu_buffer,
 	void *page;
 	int i;
 
-	/* At least 1 reader page and two pages in the ring-buffer */
-	if (desc->nr_page_va < 3)
+	/* At least RING_BUFFER_MIN_NR_PAGES and 1 reader page */
+	if (desc->nr_page_va < RING_BUFFER_MIN_NR_PAGES + 1)
 		return -EINVAL;
 
 	memset(cpu_buffer, 0, sizeof(*cpu_buffer));
Signature · 1 line
2.53.0
#3 lore raw

PATCH 3/5 ring-buffer: Clean up resize_disabled checks

In reply to Steven Rostedt

From: Vincent Donnefort <[email protected]>

ring_buffer_subbuf_order_set() checks for resize_disabled twice under the same buffer->mutex hold. Moreover, this check duplicates the logic in ring_buffer_resize(). Create a common helper rb_resize_disabled() to factor out this code.

---
 kernel/trace/ring_buffer.c | 63 +++++++++++++++++---------------------
 1 file changed, 28 insertions(+), 35 deletions(-)
kernel/trace/ring_buffer.c +28 -35
index 1b148cd891c8..15e3b9938e0f 100644
@@ -3282,6 +3282,21 @@ static void update_pages_handler(struct work_struct *work)
 	complete(&cpu_buffer->update_done);
 }
 
+static bool rb_resize_disabled(struct trace_buffer *buffer, int cpu)
+{
+	lockdep_assert_held(&buffer->mutex);
+
+	if (cpu != RING_BUFFER_ALL_CPUS)
+		return atomic_read(&buffer->buffers[cpu]->resize_disabled);
+
+	for_each_buffer_cpu(buffer, cpu) {
+		if (atomic_read(&buffer->buffers[cpu]->resize_disabled))
+			return true;
+	}
+
+	return false;
+}
+
 /**
  * ring_buffer_resize - resize the ring buffer
  * @buffer: the buffer to resize.
@@ -3322,20 +3337,17 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size,
 
 	nr_pages = rb_calculate_nr_pages(buffer, size);
 
-	if (cpu_id == RING_BUFFER_ALL_CPUS) {
-		/*
-		 * Don't succeed if resizing is disabled, as a reader might be
-		 * manipulating the ring buffer and is expecting a sane state while
-		 * this is true.
-		 */
-		for_each_buffer_cpu(buffer, cpu) {
-			cpu_buffer = buffer->buffers[cpu];
-			if (atomic_read(&cpu_buffer->resize_disabled)) {
-				err = -EBUSY;
-				goto out_err_unlock;
-			}
-		}
+	/*
+	 * Don't succeed if resizing is disabled, as a reader might be
+	 * manipulating the ring buffer and is expecting a sane state while
+	 * this is true.
+	 */
+	if (rb_resize_disabled(buffer, cpu_id)) {
+		err = -EBUSY;
+		goto out_err_unlock;
+	}
 
+	if (cpu_id == RING_BUFFER_ALL_CPUS) {
 		/* calculate the pages to update */
 		for_each_buffer_cpu(buffer, cpu) {
 			cpu_buffer = buffer->buffers[cpu];
@@ -3407,16 +3419,6 @@ int ring_buffer_resize(struct trace_buffer *buffer, unsigned long size,
 		if (nr_pages == cpu_buffer->nr_pages)
 			goto out;
 
-		/*
-		 * Don't succeed if resizing is disabled, as a reader might be
-		 * manipulating the ring buffer and is expecting a sane state while
-		 * this is true.
-		 */
-		if (atomic_read(&cpu_buffer->resize_disabled)) {
-			err = -EBUSY;
-			goto out_err_unlock;
-		}
-
 		cpu_buffer->nr_pages_to_update = nr_pages -
 						cpu_buffer->nr_pages;
 
@@ -7471,13 +7473,9 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)
 
 	old_capacity = rb_subbuf_capacity(buffer);
 
-	/* The mmap fast path reads subbuf_order without buffer->mutex. */
-	for_each_buffer_cpu(buffer, cpu) {
-		if (!cpumask_test_cpu(cpu, buffer->cpumask))
-			continue;
-		if (atomic_read(&buffer->buffers[cpu]->resize_disabled))
-			return -EBUSY;
-	}
+	/* Check it is resizable before we touch subbuf_order */
+	if (rb_resize_disabled(buffer, RING_BUFFER_ALL_CPUS))
+		return -EBUSY;
 
 	atomic_inc(&buffer->record_disabled);
 
@@ -7494,11 +7492,6 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)
 
 		cpu_buffer = buffer->buffers[cpu];
 
-		if (atomic_read(&cpu_buffer->resize_disabled)) {
-			err = -EBUSY;
-			goto error;
-		}
-
 		/* Update the number of pages to match the new size */
 		nr_pages = rb_calculate_nr_pages(buffer,
 						 old_capacity * buffer->buffers[cpu]->nr_pages);
Signature · 1 line
2.53.0
#4 lore raw

PATCH 4/5 ring-buffer: Remove unnecessary cpumask_test_cpu

In reply to Steven Rostedt

From: Vincent Donnefort <[email protected]>

for_each_buffer_cpu() already iterates over buffer->cpumask. Remove the unnecessary cpumask_test_cpu() found in ring_buffer_subbuf_order_set().

---
 kernel/trace/ring_buffer.c | 7 -------
 1 file changed, 7 deletions(-)
kernel/trace/ring_buffer.c +0 -7
index 15e3b9938e0f..0068bd185945 100644
@@ -7486,10 +7486,6 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)
 
 	/* Make sure all new buffers are allocated, before deleting the old ones */
 	for_each_buffer_cpu(buffer, cpu) {
-
-		if (!cpumask_test_cpu(cpu, buffer->cpumask))
-			continue;
-
 		cpu_buffer = buffer->buffers[cpu];
 
 		/* Update the number of pages to match the new size */
@@ -7516,9 +7512,6 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)
 		struct list_head old_pages;
 		unsigned long flags;
 
-		if (!cpumask_test_cpu(cpu, buffer->cpumask))
-			continue;
-
 		cpu_buffer = buffer->buffers[cpu];
 
 		raw_spin_lock_irqsave(&cpu_buffer->reader_lock, flags);
Signature · 1 line
2.53.0
#5 lore raw

PATCH 5/5 ring-buffer: Take cpus_read_lock() when changing the subbuf order

In reply to Steven Rostedt

From: David Carlier <[email protected]>

A CPU coming online for the first time gets its per CPU buffer allocated by trace_rb_cpu_prepare(), which then adds it to buffer->cpumask. If that happens while ring_buffer_subbuf_order_set() is running, the CPU shows up after the allocation loop and the install loop walks it with an empty new_pages list, corrupting the buffer.

Take cpus_read_lock() as ring_buffer_resize() does.

Fixes f9b94daa542a ("ring-buffer: Set new size of the ring buffer sub page")
---
 kernel/trace/ring_buffer.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/trace/ring_buffer.c +2 -0
index 0068bd185945..443271779e94 100644
@@ -7464,6 +7464,8 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)
 	if (psize > RB_WRITE_MASK + 1)
 		return -EINVAL;
 
+	guard(cpus_read_lock)();
+
 	/* prevent another thread from changing buffer sizes */
 	guard(mutex)(&buffer->mutex);
 
Signature · 1 line
2.53.0