ring-buffer: Fix sub-buffer order updates and free_page leaks
Subsystems (2)
Masami Hiramatsu (Google)
maintainercover ring-buffer: Fix sub-buffer order updates and free_page leaks
Hi Vincent,
Here is my patch to fix the race and free_page problem. Let's see what Sashiko says for this.
Thank you,
---
base-commit: 3d6d817622b0a9721e3cc404df3469171582be13
Masami Hiramatsu (Google) (1):
ring-buffer: Fix race between ring_buffer_subbuf_order_set() and readers
kernel/trace/ring_buffer.c | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-)
--
Masami Hiramatsu (Google) <[email protected]>
Masami Hiramatsu (Google)
maintainerPATCH 1/1 ring-buffer: Fix race between ring_buffer_subbuf_order_set() and readers
From: Masami Hiramatsu (Google) <[email protected]>
When ring_buffer_subbuf_order_set() updates buffer->subbuf_order, it previously modified buffer->subbuf_order before clearing the cached per-CPU free_page entries. Furthermore, clearing cpu_buffer->free_page was done under cpu_buffer->reader_lock, whereas ring_buffer_alloc_read_page() protects cpu_buffer->free_page using arch_spin_lock(&cpu_buffer->lock).
Because ring_buffer_alloc_read_page(), ring_buffer_free_read_page(), and ring_buffer_read_page() checked buffer->subbuf_order locklessly before accessing reader resources, a TOCTOU race allowed a concurrent reader to obtain, cache, or swap a page allocated under an outdated order while tagging bpage->order with the new order. This allowed undersized pages to be swapped into the ring buffer, leading to heap buffer overflows, or caused free_pages() to be called with an invalid order.
Fix this by:
1. Flushing and freeing all per-CPU cached free_page entries under arch_spin_lock(&cpu_buffer->lock) using old_order before modifying buffer->subbuf_order.
2. Protecting bpage->order assignment under arch_spin_lock(&cpu_buffer->lock) in ring_buffer_alloc_read_page().
3. Moving the buffer->subbuf_order validation inside arch_spin_lock(&cpu_buffer->lock) in ring_buffer_free_read_page().
4. Re-validating buffer->subbuf_order inside reader_lock in ring_buffer_read_page().
5. Protecting cpu_buffer->free_page extraction with arch_spin_lock(&cpu_buffer->lock) in ring_buffer_subbuf_order_set().
--- kernel/trace/ring_buffer.c | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-)
kernel/trace/ring_buffer.c +29 -6
@@ -6957,11 +6957,11 @@ ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu)if (!bpage)return ERR_PTR (-ENOMEM ); - bpage->order = buffer->subbuf_order; cpu_buffer = buffer->buffers[cpu];local_irq_save (flags);arch_spin_lock (&cpu_buffer->lock); + bpage->order = buffer->subbuf_order;if (cpu_buffer->free_page) { bpage->data = cpu_buffer->free_page; cpu_buffer->free_page =NULL ;
@@ -7010,13 +7010,13 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu,* is different from the subbuffer order of the buffer - * we can't reuse it */ -if (page_ref_count (page) >1 || data_page->order != buffer->subbuf_order) +if (page_ref_count (page) >1 ) goto out;local_irq_save (flags);arch_spin_lock (&cpu_buffer->lock); -if (!cpu_buffer->free_page) { +if (data_page->order == buffer->subbuf_order && !cpu_buffer->free_page) { cpu_buffer->free_page = dpage; dpage =NULL ; }
@@ -7094,15 +7094,15 @@ int ring_buffer_read_page(struct trace_buffer *buffer,if (!data_page || !data_page->data)return -1 ; -if (data_page->order != buffer->subbuf_order) -return -1 ; - dpage = data_page->data;if (!dpage)return -1 ;guard (raw_spinlock_irqsave)(&cpu_buffer->reader_lock); +if (data_page->order != buffer->subbuf_order) +return -1 ; + reader =rb_get_reader_page (cpu_buffer);if (!reader)return -1 ;
@@ -7350,6 +7350,27 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)/* Make sure all commits have finished */ synchronize_rcu (); +/* Flush any cached free_page allocated with old_order */ +for_each_buffer_cpu (buffer, cpu) { +struct buffer_data_page *old_free; +unsigned long flags; + +if (!cpumask_test_cpu (cpu, buffer->cpumask)) +continue ; + + cpu_buffer = buffer->buffers[cpu]; + +local_irq_save (flags); +arch_spin_lock (&cpu_buffer->lock); + old_free = cpu_buffer->free_page; + cpu_buffer->free_page =NULL ; +arch_spin_unlock (&cpu_buffer->lock); +local_irq_restore (flags); + +if (old_free) +free_pages ((unsigned long )old_free, old_order); + } + buffer->subbuf_order = order; buffer->subbuf_size = psize -BUF_PAGE_HDR_SIZE ;
@@ -7431,8 +7452,10 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) cpu_buffer->nr_pages = cpu_buffer->nr_pages_to_update; cpu_buffer->nr_pages_to_update =0 ; +arch_spin_lock (&cpu_buffer->lock); old_free_data_page = cpu_buffer->free_page; cpu_buffer->free_page =NULL ; +arch_spin_unlock (&cpu_buffer->lock);rb_head_page_activate (cpu_buffer);
[email protected]
AI reviewRe: [PATCH] ring-buffer: Fix race between ring_buffer_subbuf_order_set() and readers
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
Pre-existing issues:
- [Critical] Structural TOCTOU race: global order and size are updated before replacing per-CPU buffer pages, leading to Heap Buffer Overflow.
- [High] Incomplete TOCTOU fix: alloc_cpu_data() evaluates buffer->subbuf_order locklessly, causing bpage->order mismatch.
- [High] Race window between pre-flush loop and buffer->subbuf_order update allows caching undersized pages.
- [High] Post-update flush loop unconditionally frees newly cached pages using old_order.
- [High] Error path fails to flush cached pages when rolling back to old_order.
--
--- Patch [1]: [PATCH] ring-buffer: Fix race between ring_buffer_subbuf_order_set() and readers --- commit d1c3f71c852aab912bda60e5894b681e43da7c7b
Author: Masami Hiramatsu (Google) <[email protected]>
ring-buffer: Fix race between ring_buffer_subbuf_order_set() and readers
This commit attempts to fix a TOCTOU race condition in the ring buffer code by ensuring that the per-CPU free_page cache is flushed and protected by the cpu_buffer lock during order changes. It also updates reader functions to validate the page order under the same lock to prevent using mismatched page sizes.
Quoted 17 lines
kernel/trace/ring_buffer.c +1
-1@@ -6957,11 +6957,11 @@ ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu)if (!bpage)return ERR_PTR (-ENOMEM ); - bpage->order = buffer->subbuf_order; cpu_buffer = buffer->buffers[cpu];local_irq_save (flags);arch_spin_lock (&cpu_buffer->lock); + bpage->order = buffer->subbuf_order;if (cpu_buffer->free_page) { bpage->data = cpu_buffer->free_page; cpu_buffer->free_page =NULL ;
Quoted 1 line
}
[Severity: High]
This isn't a bug introduced by this patch, but does this locklessly evaluate buffer->subbuf_order after dropping the lock?
Looking at the code just below this hunk in ring_buffer_alloc_read_page():
arch_spin_unlock(&cpu_buffer->lock);
local_irq_restore(flags);
if (bpage->data) {
rb_init_data_page(bpage->data);
} else {
bpage->data = alloc_cpu_data(cpu, cpu_buffer->buffer->subbuf_order);
...If a concurrent writer changes the global order after the lock is dropped but before alloc_cpu_data() evaluates the argument, the allocated page size will mismatch the stored bpage->order.
When ring_buffer_free_read_page() later frees this page using bpage->order, it could free a mismatched page size and corrupt the buddy allocator.
[ ... ]
Quoted 17 lines
kernel/trace/ring_buffer.c +13
-0@@ -7350,6 +7350,27 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)/* Make sure all commits have finished */ synchronize_rcu (); +/* Flush any cached free_page allocated with old_order */ +for_each_buffer_cpu (buffer, cpu) { +struct buffer_data_page *old_free; +unsigned long flags; + +if (!cpumask_test_cpu (cpu, buffer->cpumask)) +continue ; + + cpu_buffer = buffer->buffers[cpu]; + +local_irq_save (flags); +arch_spin_lock (&cpu_buffer->lock); + old_free = cpu_buffer->free_page;
kernel/trace/ring_buffer.c int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) · line 7375
Quoted 10 lines
kernel/trace/ring_buffer.c +8
-0+ cpu_buffer->free_page =NULL ; +arch_spin_unlock (&cpu_buffer->lock); +local_irq_restore (flags); + +if (old_free) +free_pages ((unsigned long )old_free, old_order); + } + buffer->subbuf_order = order; buffer->subbuf_size = psize -BUF_PAGE_HDR_SIZE ;
[Severity: High]
This isn't a bug introduced by this patch, but does updating the global order after the flush loop leave a race window?
During the unprotected window between the flush loop completing and updating buffer->subbuf_order, the global order is still old_order.
A concurrent reader could call ring_buffer_free_read_page(), see that data_page->order matches buffer->subbuf_order (both old), and cache an old_order page back into cpu_buffer->free_page.
Once the order is updated below, subsequent readers might extract this cached old_order page, tag it with the new order, and eventually free it with the new order, causing buddy allocator corruption.
[Severity: Critical]
This isn't a bug introduced by this patch, but does updating the global order and size here, before replacing per-CPU buffer pages, create a structural race?
If a concurrent reader calls ring_buffer_read_page(), it validates its new-order data_page against the new global order and swaps it into the ring buffer. It extracts the old, unreplaced reader->page into dpage.
If the buffer is expanding, dpage is an undersized old-order page.
However, the reader executes memset() using the prematurely updated global buffer->subbuf_size:
ring_buffer_read_page():
if (size < buffer->subbuf_size) memset(&dpage->data[size], 0, buffer->subbuf_size - size);
This would blindly write zeroes far beyond the bounds of the extracted old-order dpage, resulting in a heap buffer overflow reachable via unprivileged tracefs reads.
[ ... ]
kernel/trace/ring_buffer.c int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) · line 7460
Quoted 10 lines
kernel/trace/ring_buffer.c +2
-0@@ -7431,8 +7452,10 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) cpu_buffer->nr_pages = cpu_buffer->nr_pages_to_update; cpu_buffer->nr_pages_to_update =0 ; +arch_spin_lock (&cpu_buffer->lock); old_free_data_page = cpu_buffer->free_page; cpu_buffer->free_page =NULL ; +arch_spin_unlock (&cpu_buffer->lock);rb_head_page_activate (cpu_buffer);
[Severity: High]
This isn't a bug introduced by this patch, but does this second flush loop unconditionally free potentially new-order pages with old_order?
After the global order is set to new_order, this function performs a sleepable allocation loop (__rb_allocate_pages). During this time, concurrent readers can allocate and free new_order pages, successfully caching them into cpu_buffer->free_page.
When this code extracts cpu_buffer->free_page and subsequently frees it down below:
free_pages((unsigned long)old_free_data_page, old_order);
It blindly forces what could be a new_order page to be freed with old_order, corrupting the buddy allocator.
[Severity: High]
This isn't a bug introduced by this patch, but does the error path fail to flush cached pages when rolling back?
If __rb_allocate_pages() fails, the code jumps to the error label and restores buffer->subbuf_order = old_order:
ring_buffer_subbuf_order_set():
error:
buffer->subbuf_order = old_order; buffer->subbuf_size = old_size;
During the window where it was new_order, concurrent readers may have cached new_order pages into cpu_buffer->free_page. The error path returns without flushing cpu_buffer->free_page.
Subsequent readers would extract these new_order pages, tag them as old_order, and free them with old_order, causing a memory leak (if expanding) or allocator corruption (if shrinking).
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/178663777320.475864.4716637934003507750.stgit@devnote2?part=1
Vincent Donnefort
maintainerQuoted 14 lines · Masami Hiramatsu (Google)
Hi Vincent,
Here is my patch to fix the race and
free_pageproblem. Let's see what Sashiko says for this.Thank you,
---
base-commit:3d6d817622b0a9721e3cc404df3469171582be13Masami Hiramatsu (Google) (1):
ring-buffer: Fix race betweenring_buffer_subbuf_order_set()and readers
Quoted 5 lines
kernel/trace/ring_buffer.c | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-)--
Masami Hiramatsu (Google) <[email protected]>
Sorry I am not sure to understand what is wrong with that series I made here?
https://lore.kernel.org/all/[email protected]/
Signature · 1 line
Vincent
Steven Rostedt
maintainerRe: [PATCH] ring-buffer: Fix race between ring_buffer_subbuf_order_set() and readers
In reply to Masami Hiramatsu (Google)Quoted 30 lines · On Fri, 14 Aug 2026 01:16:13 +0900 "Masami Hiramatsu (Google)"…
From: Masami Hiramatsu (Google) <[email protected]>
When
ring_buffer_subbuf_order_set()updatesbuffer->subbuf_order, it previously modifiedbuffer->subbuf_orderbefore clearing the cached per-CPUfree_pageentries. Furthermore, clearingcpu_buffer->free_pagewas done undercpu_buffer->reader_lock, whereasring_buffer_alloc_read_page()protectscpu_buffer->free_pageusingarch_spin_lock(&cpu_buffer->lock).Because
ring_buffer_alloc_read_page(),ring_buffer_free_read_page(), andring_buffer_read_page()checkedbuffer->subbuf_orderlocklessly before accessing reader resources, a TOCTOU race allowed a concurrent reader to obtain, cache, or swap a page allocated under an outdated order while taggingbpage->orderwith the new order. This allowed undersized pages to be swapped into the ring buffer, leading to heap buffer overflows, or causedfree_pages()to be called with an invalid order.Fix this by:
1. Flushing and freeing all per-CPU cachedfree_pageentries underarch_spin_lock(&cpu_buffer->lock)usingold_orderbefore modifyingbuffer->subbuf_order.
2. Protectingbpage->orderassignment underarch_spin_lock(&cpu_buffer->lock)inring_buffer_alloc_read_page().
3. Moving thebuffer->subbuf_ordervalidation insidearch_spin_lock(&cpu_buffer->lock)inring_buffer_free_read_page().
4. Re-validatingbuffer->subbuf_orderinsidereader_lockinring_buffer_read_page().
5. Protectingcpu_buffer->free_pageextraction witharch_spin_lock(&cpu_buffer->lock)inring_buffer_subbuf_order_set().
Quoted 4 lines
Fixes 2808e31ec12e ("ring-buffer: Add interface for configuring trace sub buffer size")Assisted-by Antigravity:gemini-3.6-flashSigned-off-by Masami Hiramatsu (Google) <[email protected]>---
Please rebase on top of ring-buffer/for-next, as I added Vincent's patches to that.
Quoted 8 lines
kernel/trace/ring_buffer.c | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-)kernel/trace/ring_buffer.c +0
-0
kernel/trace/ring_buffer.c ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) · line 6964
Quoted 9 lines
kernel/trace/ring_buffer.c +1
-1@@ -6957,11 +6957,11 @@ ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu)if (!bpage)return ERR_PTR (-ENOMEM ); - bpage->order = buffer->subbuf_order; cpu_buffer = buffer->buffers[cpu];local_irq_save (flags);arch_spin_lock (&cpu_buffer->lock); + bpage->order = buffer->subbuf_order;
This is still needed.
Quoted 6 lines
kernel/trace/ring_buffer.c +0
-0if (cpu_buffer->free_page) { bpage->data = cpu_buffer->free_page; cpu_buffer->free_page =NULL ;@@ -7010,13 +7010,13 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu,* is different from the subbuffer order of the buffer - * we can't reuse it
kernel/trace/ring_buffer.c void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, · line 7019
Quoted 10 lines
kernel/trace/ring_buffer.c +2
-2*/ -if (page_ref_count (page) >1 || data_page->order != buffer->subbuf_order) +if (page_ref_count (page) >1 ) goto out;local_irq_save (flags);arch_spin_lock (&cpu_buffer->lock); -if (!cpu_buffer->free_page) { +if (data_page->order == buffer->subbuf_order && !cpu_buffer->free_page) {
Swap the order please. It has to check both to continue and if one fails it will not continue. Checking for cpu_buffer->free_page to be NULL first is the quicker check. And also the more likely one to fail.
Quoted 36 lines
kernel/trace/ring_buffer.c +13
-3cpu_buffer->free_page = dpage; dpage =NULL ; }@@ -7094,15 +7094,15 @@ int ring_buffer_read_page(struct trace_buffer *buffer,if (!data_page || !data_page->data)return -1 ; -if (data_page->order != buffer->subbuf_order) -return -1 ; - dpage = data_page->data;if (!dpage)return -1 ;guard (raw_spinlock_irqsave)(&cpu_buffer->reader_lock); +if (data_page->order != buffer->subbuf_order) +return -1 ; + reader =rb_get_reader_page (cpu_buffer);if (!reader)return -1 ;@@ -7350,6 +7350,27 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)/* Make sure all commits have finished */ synchronize_rcu (); +/* Flush any cached free_page allocated with old_order */ +for_each_buffer_cpu (buffer, cpu) { +struct buffer_data_page *old_free; +unsigned long flags; + +if (!cpumask_test_cpu (cpu, buffer->cpumask)) +continue ; + + cpu_buffer = buffer->buffers[cpu]; +
kernel/trace/ring_buffer.c int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) · line 7372
Quoted 10 lines
kernel/trace/ring_buffer.c +10
-0+local_irq_save (flags); +arch_spin_lock (&cpu_buffer->lock); + old_free = cpu_buffer->free_page; + cpu_buffer->free_page =NULL ; +arch_spin_unlock (&cpu_buffer->lock); +local_irq_restore (flags); + +if (old_free) +free_pages ((unsigned long )old_free, old_order); + }
Honestly, this should be a separate patch. The first part of this patch is data races with adding and freeing, but this is about changes to the size.
kernel/trace/ring_buffer.c int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) · line 7458
Quoted 12 lines
kernel/trace/ring_buffer.c +3
-0+ buffer->subbuf_order = order; buffer->subbuf_size = psize -BUF_PAGE_HDR_SIZE ;@@ -7431,8 +7452,10 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) cpu_buffer->nr_pages = cpu_buffer->nr_pages_to_update; cpu_buffer->nr_pages_to_update =0 ; +arch_spin_lock (&cpu_buffer->lock); old_free_data_page = cpu_buffer->free_page; cpu_buffer->free_page =NULL ; +arch_spin_unlock (&cpu_buffer->lock);
This is already fixed by Vincent (and I would have asked this to be a separate patch too if it hadn't).
-- Steve
Quoted 3 lines
kernel/trace/ring_buffer.c +0
-0rb_head_page_activate (cpu_buffer);
Vincent Donnefort
maintainerRe: [PATCH] ring-buffer: Fix race between ring_buffer_subbuf_order_set() and readers
Quoted 52 lines · Steven Rostedt
Quoted 30 lines · On Fri, 14 Aug 2026 01:16:13 +0900 "Masami Hiramatsu (Google)"…
From: Masami Hiramatsu (Google) <[email protected]>
When
ring_buffer_subbuf_order_set()updatesbuffer->subbuf_order, it previously modifiedbuffer->subbuf_orderbefore clearing the cached per-CPUfree_pageentries. Furthermore, clearingcpu_buffer->free_pagewas done undercpu_buffer->reader_lock, whereasring_buffer_alloc_read_page()protectscpu_buffer->free_pageusingarch_spin_lock(&cpu_buffer->lock).Because
ring_buffer_alloc_read_page(),ring_buffer_free_read_page(), andring_buffer_read_page()checkedbuffer->subbuf_orderlocklessly before accessing reader resources, a TOCTOU race allowed a concurrent reader to obtain, cache, or swap a page allocated under an outdated order while taggingbpage->orderwith the new order. This allowed undersized pages to be swapped into the ring buffer, leading to heap buffer overflows, or causedfree_pages()to be called with an invalid order.Fix this by:
1. Flushing and freeing all per-CPU cachedfree_pageentries underarch_spin_lock(&cpu_buffer->lock)usingold_orderbefore modifyingbuffer->subbuf_order.
2. Protectingbpage->orderassignment underarch_spin_lock(&cpu_buffer->lock)inring_buffer_alloc_read_page().
3. Moving thebuffer->subbuf_ordervalidation insidearch_spin_lock(&cpu_buffer->lock)inring_buffer_free_read_page().
4. Re-validatingbuffer->subbuf_orderinsidereader_lockinring_buffer_read_page().
5. Protectingcpu_buffer->free_pageextraction witharch_spin_lock(&cpu_buffer->lock)inring_buffer_subbuf_order_set().Quoted 4 lines
Fixes 2808e31ec12e ("ring-buffer: Add interface for configuring trace sub buffer size")Assisted-by Antigravity:gemini-3.6-flashSigned-off-by Masami Hiramatsu (Google) <[email protected]>---
Please rebase on top of ring-buffer/for-next, as I added Vincent's patches to that.
Quoted 11 lines
kernel/trace/ring_buffer.c | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-)kernel/trace/ring_buffer.c +0
-0@@ -6957,11 +6957,11 @@ ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu)if (!bpage)return ERR_PTR (-ENOMEM );
Quoted 8 lines
kernel/trace/ring_buffer.c ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) · line 6964
Quoted 6 lines
kernel/trace/ring_buffer.c +1
-1- bpage->order = buffer->subbuf_order; cpu_buffer = buffer->buffers[cpu];local_irq_save (flags);arch_spin_lock (&cpu_buffer->lock); + bpage->order = buffer->subbuf_order;This is still needed.
I am not sure, the lock is per-cpu_buffer but subbuf_order. is global to trace_buffer?
Quoted 36 lines
Quoted 7 lines
kernel/trace/ring_buffer.c +0
-0if (cpu_buffer->free_page) { bpage->data = cpu_buffer->free_page; cpu_buffer->free_page =NULL ;@@ -7010,13 +7010,13 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu,* is different from the subbuffer order of the buffer - * we can't reuse it kernel/trace/ring_buffer.c void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, · line 7019
Quoted 10 lines
kernel/trace/ring_buffer.c +2
-2*/ -if (page_ref_count (page) >1 || data_page->order != buffer->subbuf_order) +if (page_ref_count (page) >1 ) goto out;local_irq_save (flags);arch_spin_lock (&cpu_buffer->lock); -if (!cpu_buffer->free_page) { +if (data_page->order == buffer->subbuf_order && !cpu_buffer->free_page) {Swap the order please. It has to check both to continue and if one fails it will not continue. Checking for
cpu_buffer->free_pageto be NULL first is the quicker check. And also the more likely one to fail.Quoted 14 lines
kernel/trace/ring_buffer.c +0
-3cpu_buffer->free_page = dpage; dpage =NULL ; }@@ -7094,15 +7094,15 @@ int ring_buffer_read_page(struct trace_buffer *buffer,if (!data_page || !data_page->data)return -1 ; -if (data_page->order != buffer->subbuf_order) -return -1 ; - dpage = data_page->data;if (!dpage)return -1 ;
Quoted 5 lines
kernel/trace/ring_buffer.c int ring_buffer_read_page(struct trace_buffer *buffer, · line 7105
Quoted 5 lines
kernel/trace/ring_buffer.c +3
-0guard (raw_spinlock_irqsave)(&cpu_buffer->reader_lock); +if (data_page->order != buffer->subbuf_order) +return -1 ; +
I have modified this as part of
tracing: Fix subbuf resize races with trace_pipe_raw readers
Quoted 53 lines
Quoted 17 lines
kernel/trace/ring_buffer.c +10
-0reader =rb_get_reader_page (cpu_buffer);if (!reader)return -1 ;@@ -7350,6 +7350,27 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)/* Make sure all commits have finished */ synchronize_rcu (); +/* Flush any cached free_page allocated with old_order */ +for_each_buffer_cpu (buffer, cpu) { +struct buffer_data_page *old_free; +unsigned long flags; + +if (!cpumask_test_cpu (cpu, buffer->cpumask)) +continue ; + + cpu_buffer = buffer->buffers[cpu]; +kernel/trace/ring_buffer.c int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) · line 7372
Quoted 10 lines
kernel/trace/ring_buffer.c +10
-0+local_irq_save (flags); +arch_spin_lock (&cpu_buffer->lock); + old_free = cpu_buffer->free_page; + cpu_buffer->free_page =NULL ; +arch_spin_unlock (&cpu_buffer->lock); +local_irq_restore (flags); + +if (old_free) +free_pages ((unsigned long )old_free, old_order); + }Honestly, this should be a separate patch. The first part of this patch is data races with adding and freeing, but this is about changes to the size.
kernel/trace/ring_buffer.c int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) · line 7458
Quoted 12 lines
kernel/trace/ring_buffer.c +3
-0+ buffer->subbuf_order = order; buffer->subbuf_size = psize -BUF_PAGE_HDR_SIZE ;@@ -7431,8 +7452,10 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) cpu_buffer->nr_pages = cpu_buffer->nr_pages_to_update; cpu_buffer->nr_pages_to_update =0 ; +arch_spin_lock (&cpu_buffer->lock); old_free_data_page = cpu_buffer->free_page; cpu_buffer->free_page =NULL ; +arch_spin_unlock (&cpu_buffer->lock);This is already fixed by Vincent (and I would have asked this to be a separate patch too if it hadn't).
-- Steve
Quoted 3 lines
kernel/trace/ring_buffer.c +0
-0rb_head_page_activate (cpu_buffer);
Signature · 1 line
Vincent
Masami Hiramatsu (Google)
maintainerRe: [PATCH] ring-buffer: Fix race between ring_buffer_subbuf_order_set() and readers
Quoted 58 lines · On Fri, 14 Aug 2026 15:53:43 +0100 Vincent Donnefort <[email protected]>…
Quoted 57 lines · Steven Rostedt
Quoted 30 lines · On Fri, 14 Aug 2026 01:16:13 +0900 "Masami Hiramatsu (Google)"…
From: Masami Hiramatsu (Google) <[email protected]>
When
ring_buffer_subbuf_order_set()updatesbuffer->subbuf_order, it previously modifiedbuffer->subbuf_orderbefore clearing the cached per-CPUfree_pageentries. Furthermore, clearingcpu_buffer->free_pagewas done undercpu_buffer->reader_lock, whereasring_buffer_alloc_read_page()protectscpu_buffer->free_pageusingarch_spin_lock(&cpu_buffer->lock).Because
ring_buffer_alloc_read_page(),ring_buffer_free_read_page(), andring_buffer_read_page()checkedbuffer->subbuf_orderlocklessly before accessing reader resources, a TOCTOU race allowed a concurrent reader to obtain, cache, or swap a page allocated under an outdated order while taggingbpage->orderwith the new order. This allowed undersized pages to be swapped into the ring buffer, leading to heap buffer overflows, or causedfree_pages()to be called with an invalid order.Fix this by:
1. Flushing and freeing all per-CPU cachedfree_pageentries underarch_spin_lock(&cpu_buffer->lock)usingold_orderbefore modifyingbuffer->subbuf_order.
2. Protectingbpage->orderassignment underarch_spin_lock(&cpu_buffer->lock)inring_buffer_alloc_read_page().
3. Moving thebuffer->subbuf_ordervalidation insidearch_spin_lock(&cpu_buffer->lock)inring_buffer_free_read_page().
4. Re-validatingbuffer->subbuf_orderinsidereader_lockinring_buffer_read_page().
5. Protectingcpu_buffer->free_pageextraction witharch_spin_lock(&cpu_buffer->lock)inring_buffer_subbuf_order_set().Quoted 4 lines
Fixes 2808e31ec12e ("ring-buffer: Add interface for configuring trace sub buffer size")Assisted-by Antigravity:gemini-3.6-flashSigned-off-by Masami Hiramatsu (Google) <[email protected]>---
Please rebase on top of ring-buffer/for-next, as I added Vincent's patches to that.
Quoted 16 lines
kernel/trace/ring_buffer.c | 35 +++++++++++++++++++++++++++++------ 1 file changed, 29 insertions(+), 6 deletions(-)kernel/trace/ring_buffer.c +0
-1@@ -6957,11 +6957,11 @@ ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu)if (!bpage)return ERR_PTR (-ENOMEM ); - bpage->order = buffer->subbuf_order; cpu_buffer = buffer->buffers[cpu];local_irq_save (flags);arch_spin_lock (&cpu_buffer->lock);
Quoted 6 lines
Quoted 3 lines
kernel/trace/ring_buffer.c ring_buffer_alloc_read_page(struct trace_buffer *buffer, int cpu) · line 6964
Quoted 1 line
kernel/trace/ring_buffer.c +1
-0+ bpage->order = buffer->subbuf_order;This is still needed.
I am not sure, the lock is per-cpu_buffer but
subbuf_order. is global totrace_buffer?
Yeah, we don't need this.
I got new ring_buffer_read_page() ensures the data_page->order == cpu_buffer->reader_page->order. This is a bit tricky but it allows us to asynchronously allocate reader_page in different order outside of locking. (and if free_page exists, it is copied with its order, so it is safe.)
Quoted 39 lines
Quoted 39 lines
Quoted 8 lines
kernel/trace/ring_buffer.c +0
-0if (cpu_buffer->free_page) { bpage->data = cpu_buffer->free_page; cpu_buffer->free_page =NULL ;@@ -7010,13 +7010,13 @@ void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu,* is different from the subbuffer order of the buffer - * we can't reuse it kernel/trace/ring_buffer.c void ring_buffer_free_read_page(struct trace_buffer *buffer, int cpu, · line 7019
Quoted 10 lines
kernel/trace/ring_buffer.c +2
-2*/ -if (page_ref_count (page) >1 || data_page->order != buffer->subbuf_order) +if (page_ref_count (page) >1 ) goto out;local_irq_save (flags);arch_spin_lock (&cpu_buffer->lock); -if (!cpu_buffer->free_page) { +if (data_page->order == buffer->subbuf_order && !cpu_buffer->free_page) {Swap the order please. It has to check both to continue and if one fails it will not continue. Checking for
cpu_buffer->free_pageto be NULL first is the quicker check. And also the more likely one to fail.Quoted 16 lines
kernel/trace/ring_buffer.c +0
-3cpu_buffer->free_page = dpage; dpage =NULL ; }@@ -7094,15 +7094,15 @@ int ring_buffer_read_page(struct trace_buffer *buffer,if (!data_page || !data_page->data)return -1 ; -if (data_page->order != buffer->subbuf_order) -return -1 ; - dpage = data_page->data;if (!dpage)return -1 ;guard (raw_spinlock_irqsave)(&cpu_buffer->reader_lock);
Quoted 7 lines
Quoted 3 lines
kernel/trace/ring_buffer.c int ring_buffer_read_page(struct trace_buffer *buffer, · line 7105
Quoted 3 lines
kernel/trace/ring_buffer.c +3
-0+if (data_page->order != buffer->subbuf_order) +return -1 ; +I have modified this as part of
tracing: Fix subbuf resize races with
trace_pipe_rawreaders
Got it. Yours changes the data structure. Mine is for older kernels as a minimal change.
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
kernel/trace/ring_buffer.c +15
-0reader =rb_get_reader_page (cpu_buffer);if (!reader)return -1 ;@@ -7350,6 +7350,27 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order)/* Make sure all commits have finished */ synchronize_rcu (); +/* Flush any cached free_page allocated with old_order */ +for_each_buffer_cpu (buffer, cpu) { +struct buffer_data_page *old_free; +unsigned long flags; + +if (!cpumask_test_cpu (cpu, buffer->cpumask)) +continue ; + + cpu_buffer = buffer->buffers[cpu]; + +local_irq_save (flags); +arch_spin_lock (&cpu_buffer->lock); + old_free = cpu_buffer->free_page; + cpu_buffer->free_page =NULL ; +arch_spin_unlock (&cpu_buffer->lock);
Quoted 8 lines
Quoted 8 lines
kernel/trace/ring_buffer.c int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) · line 7372
Quoted 5 lines
kernel/trace/ring_buffer.c +5
-0+local_irq_restore (flags); + +if (old_free) +free_pages ((unsigned long )old_free, old_order); + }Honestly, this should be a separate patch. The first part of this patch is data races with adding and freeing, but this is about changes to the size.
Since Vincent's patch[1] handles data page pointer with its order, we don't this part anymore.
[1] ring-buffer: Make cpu_buffer::free_page a buffer_data_read_page
So I confirmed Vincent's patches with Fixed tag are fixing the racing problem in the different way.
Thank you,
Signature · 1 line
Masami Hiramatsu (Google) <[email protected]>