Skip to contentSkip to conversation

tracing: zero the pipe cpumask on alloc to avoid spurious -EBUSY

PATCH In review Brian Foster 3 participants 4 messages Aug 31, 2023 – Sep 1, 2023 Tracing

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tracing: zero the pipe cpumask on alloc to avoid spurious -EBUSY

The pipe cpumask used to serialize opens between the main and percpu trace pipes is not zeroed or initialized. This can result in spurious -EBUSY returns if underlying memory is not fully zeroed. This has been observed by immediate failure to read the main trace_pipe file on an otherwise newly booted and idle system:

 # cat /sys/kernel/debug/tracing/trace_pipe
 cat: /sys/kernel/debug/tracing/trace_pipe: Device or resource busy

Zero the allocation of pipe_cpumask to avoid the problem.

Fixes c2489bb7e6be ("tracing: Introduce pipe_cpumask to avoid race on trace_pipes")

---

Hi,

I ran into this problem just recently on one of my test VMs immediately after updating to a v6.5 base. A revert of the aforementioned commit addressed the problem. I'm not terribly familiar with the tracing code, but on further inspection I noticed the cpumask doesn't appear to be initialized anywhere. I suppose this could alternatively do a cpumask_clear() or whatever after allocation, but either way this addresses the problem for me.

Please CC on replies as I'm not subscribed to the list. Thanks.

Brian

 kernel/trace/trace.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/trace/trace.c +2 -2
index 8e64aaad5361..2656ca3b9b39 100644
@@ -9486,7 +9486,7 @@ static struct trace_array *trace_array_create(const char *name)
 	if (!alloc_cpumask_var(&tr->tracing_cpumask, GFP_KERNEL))
 		goto out_free_tr;
 
-	if (!alloc_cpumask_var(&tr->pipe_cpumask, GFP_KERNEL))
+	if (!zalloc_cpumask_var(&tr->pipe_cpumask, GFP_KERNEL))
 		goto out_free_tr;
 
 	tr->trace_flags = global_trace.trace_flags & ~ZEROED_TRACE_FLAGS;
@@ -10431,7 +10431,7 @@ __init static int tracer_alloc_buffers(void)
 	if (trace_create_savedcmd() < 0)
 		goto out_free_temp_buffer;
 
-	if (!alloc_cpumask_var(&global_trace.pipe_cpumask, GFP_KERNEL))
+	if (!zalloc_cpumask_var(&global_trace.pipe_cpumask, GFP_KERNEL))
 		goto out_free_savedcmd;
 
 	/* TODO: make the number of buffers hot pluggable with CPUS */
Signature · 1 line
2.41.0
#1 lore raw
Quoted 17 lines · Brian Foster

The pipe cpumask used to serialize opens between the main and percpu trace pipes is not zeroed or initialized. This can result in spurious -EBUSY returns if underlying memory is not fully zeroed. This has been observed by immediate failure to read the main trace_pipe file on an otherwise newly booted and idle system:

  # cat /sys/kernel/debug/tracing/trace_pipe
  cat: /sys/kernel/debug/tracing/trace_pipe: Device or resource busy

Zero the allocation of pipe_cpumask to avoid the problem.

Fixes c2489bb7e6be ("tracing: Introduce pipe_cpumask to avoid race on trace_pipes")

---

Hi,

Quoted 7 lines

I ran into this problem just recently on one of my test VMs immediately after updating to a v6.5 base. A revert of the aforementioned commit addressed the problem. I'm not terribly familiar with the tracing code, but on further inspection I noticed the cpumask doesn't appear to be initialized anywhere. I suppose this could alternatively do a cpumask_clear() or whatever after allocation, but either way this addresses the problem for me.

Yes, pipe_cpumask must be initialized.

--

Thanks,
Zheng Yejian

Quoted 30 lines

Please CC on replies as I'm not subscribed to the list. Thanks.

Brian

  kernel/trace/trace.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
kernel/trace/trace.c +2 -2
index 8e64aaad5361..2656ca3b9b39 100644
@@ -9486,7 +9486,7 @@ static struct trace_array *trace_array_create(const char *name)
  	if (!alloc_cpumask_var(&tr->tracing_cpumask, GFP_KERNEL))
  		goto out_free_tr;
 
-	if (!alloc_cpumask_var(&tr->pipe_cpumask, GFP_KERNEL))
+	if (!zalloc_cpumask_var(&tr->pipe_cpumask, GFP_KERNEL))
  		goto out_free_tr;
 
  	tr->trace_flags = global_trace.trace_flags & ~ZEROED_TRACE_FLAGS;
@@ -10431,7 +10431,7 @@ __init static int tracer_alloc_buffers(void)
  	if (trace_create_savedcmd() < 0)
  		goto out_free_temp_buffer;
 
-	if (!alloc_cpumask_var(&global_trace.pipe_cpumask, GFP_KERNEL))
+	if (!zalloc_cpumask_var(&global_trace.pipe_cpumask, GFP_KERNEL))
  		goto out_free_savedcmd;
 
  	/* TODO: make the number of buffers hot pluggable with CPUS */
maintainer
#2 lore raw
Quoted 11 lines · On Thu, 31 Aug 2023 21:51:18 +0800 Zheng Yejian <[email protected]> wrote:
Quoted 9 lines

Hi,

I ran into this problem just recently on one of my test VMs immediately after updating to a v6.5 base. A revert of the aforementioned commit addressed the problem. I'm not terribly familiar with the tracing code, but on further inspection I noticed the cpumask doesn't appear to be initialized anywhere. I suppose this could alternatively do a cpumask_clear() or whatever after allocation, but either way this addresses the problem for me.

Yes, pipe_cpumask must be initialized.

Can I add a Reviewed-by tag from you?

Quoted 3 lines
Quoted 3 lines

Please CC on replies as I'm not subscribed to the list. Thanks.

That's the default with Linux kernel lists.

-- Steve

Reviewed-by
#3 lore raw
Quoted 13 lines · Steven Rostedt
Quoted 10 lines · On Thu, 31 Aug 2023 21:51:18 +0800 Zheng Yejian <[email protected]> wrote:
Quoted 9 lines

Hi,

I ran into this problem just recently on one of my test VMs immediately after updating to a v6.5 base. A revert of the aforementioned commit addressed the problem. I'm not terribly familiar with the tracing code, but on further inspection I noticed the cpumask doesn't appear to be initialized anywhere. I suppose this could alternatively do a cpumask_clear() or whatever after allocation, but either way this addresses the problem for me.

Quoted 3 lines
Quoted 1 line

Yes, pipe_cpumask must be initialized.

Can I add a Reviewed-by tag from you?

Of course :)

--

Thanks,
Zheng Yejian

Quoted 9 lines
Quoted 4 lines
Quoted 4 lines

Please CC on replies as I'm not subscribed to the list. Thanks.

That's the default with Linux kernel lists.

-- Steve