Skip to contentSkip to conversation

refscale: Fix use of uninitalized wait_queue_head_t

PATCH Applied Waiman Long 5 participants 9 messages Jul 7, 2023

Subsystems (2)

v1 of 2 Applied Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 refscale: Fix use of uninitalized wait_queue_head_t

It was found that running the refscale test might sometimes crash the kernel with the following error:

[ 8569.952896] BUG: unable to handle page fault for address: ffffffffffffffe8
[ 8569.952900] #PF: supervisor read access in kernel mode
[ 8569.952902] #PF: error_code(0x0000) - not-present page
[ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0
[ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI
[ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.2.4 05/28/2021
[ 8569.952917] RIP: 0010:prepare_to_wait_event+0x101/0x190
  :
[ 8569.952940] Call Trace:
[ 8569.952941]  <TASK>
[ 8569.952944]  ref_scale_reader+0x380/0x4a0 [refscale]
[ 8569.952959]  kthread+0x10e/0x130
[ 8569.952966]  ret_from_fork+0x1f/0x30
[ 8569.952973]  </TASK>

This is likely caused by the fact that init_waitqueue_head() is called after the ref_scale_reader kthread is created. So the kthread may try to use the waitqueue head before it is properly initialized. Fix this by initializing the waitqueue head first before kthread creation.

Fixes 653ed64b01dc ("refperf: Add a test to measure performance of read-side synchronization")
---
 kernel/rcu/refscale.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
kernel/rcu/refscale.c +1 -2
index 1970ce5f22d4..e365d6f8c139 100644
@@ -1107,12 +1107,11 @@ ref_scale_init(void)
 	VERBOSE_SCALEOUT("Starting %d reader threads", nreaders);
 
 	for (i = 0; i < nreaders; i++) {
+		init_waitqueue_head(&(reader_tasks[i].wq));
 		firsterr = torture_create_kthread(ref_scale_reader, (void *)i,
 						  reader_tasks[i].task);
 		if (torture_init_error(firsterr))
 			goto unwind;
-
-		init_waitqueue_head(&(reader_tasks[i].wq));
 	}
 
 	// Main Task
Signature · 1 line
2.31.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 38 lines

From: Waiman Long <[email protected]>
...
Subject: [PATCH] refscale: Fix use of uninitalized wait_queue_head_t

It was found that running the refscale test might sometimes crash the kernel with the following error:

[ 8569.952896] BUG: unable to handle page fault for address: ffffffffffffffe8
[ 8569.952900] #PF: supervisor read access in kernel mode [ 8569.952902]

#PF: error_code(0x0000) - not-present page [ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0 [ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI [ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.2.4 05/28/2021 [ 8569.952917] RIP:
0010:prepare_to_wait_event+0x101/0x190
:

[ 8569.952940] Call Trace:
[ 8569.952941]  <TASK>
[ 8569.952944]  ref_scale_reader+0x380/0x4a0 [refscale] [ 8569.952959]
kthread+0x10e/0x130 [ 8569.952966]  ret_from_fork+0x1f/0x30
[ 8569.952973]  </TASK>

This is likely caused by the fact that init_waitqueue_head() is called after the ref_scale_reader kthread is created. So the kthread may try to use the waitqueue head before it is properly initialized. Fix this by initializing the waitqueue head first before kthread creation.

Fixes 653ed64b01dc ("refperf: Add a test to measure performance of read-

side synchronization")

---
 kernel/rcu/refscale.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
kernel/rcu/refscale.c index renamed from kernel/rcu/refscale.c +0 -0

1970ce5f22d4..e365d6f8c139 100644

kernel/rcu/refscale.c ref_scale_init(void) · line 1110

Quoted 4 lines
kernel/rcu/refscale.c +1 -0
@@ -1107,12 +1107,11 @@ ref_scale_init(void)
 	VERBOSE_SCALEOUT("Starting %d reader threads", nreaders);
 
 	for (i = 0; i < nreaders; i++) {
+		init_waitqueue_head(&(reader_tasks[i].wq));

Running checkpatch.pl tool with the " --strict" option, it complained that "CHECK: Unnecessary parentheses around reader_tasks[i].wq".
I know that you just moved the code position. The tool should have complained the original code. 😊

Other than that, this patch LGTM.

Thanks!
-Qiuxu

Quoted 11 lines
kernel/rcu/refscale.c +0 -3
 		firsterr = torture_create_kthread(ref_scale_reader, (void *)i,
 						  reader_tasks[i].task);
 		if (torture_init_error(firsterr))
 			goto unwind;
-
-		init_waitqueue_head(&(reader_tasks[i].wq));
 	}
 
 	// Main Task
--

2.31.1

#2 lore raw
Quoted 47 lines · Qiuxu
Quoted 38 lines

From: Waiman Long <[email protected]>
...
Subject: [PATCH] refscale: Fix use of uninitalized wait_queue_head_t

It was found that running the refscale test might sometimes crash the kernel with the following error:

[ 8569.952896] BUG: unable to handle page fault for address: ffffffffffffffe8
[ 8569.952900] #PF: supervisor read access in kernel mode [ 8569.952902]

#PF: error_code(0x0000) - not-present page [ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0 [ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI [ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.2.4 05/28/2021 [ 8569.952917] RIP:
0010:prepare_to_wait_event+0x101/0x190
:

[ 8569.952940] Call Trace:
[ 8569.952941]  <TASK>
[ 8569.952944]  ref_scale_reader+0x380/0x4a0 [refscale] [ 8569.952959]
kthread+0x10e/0x130 [ 8569.952966]  ret_from_fork+0x1f/0x30
[ 8569.952973]  </TASK>

This is likely caused by the fact that init_waitqueue_head() is called after the ref_scale_reader kthread is created. So the kthread may try to use the waitqueue head before it is properly initialized. Fix this by initializing the waitqueue head first before kthread creation.

Fixes 653ed64b01dc ("refperf: Add a test to measure performance of read-

side synchronization")

---
  kernel/rcu/refscale.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)
kernel/rcu/refscale.c index renamed from kernel/rcu/refscale.c +0 -0

1970ce5f22d4..e365d6f8c139 100644

kernel/rcu/refscale.c ref_scale_init(void) · line 1110

Quoted 4 lines
kernel/rcu/refscale.c +1 -0
@@ -1107,12 +1107,11 @@ ref_scale_init(void)
  	VERBOSE_SCALEOUT("Starting %d reader threads", nreaders);
 
  	for (i = 0; i < nreaders; i++) {
+		init_waitqueue_head(&(reader_tasks[i].wq));

Running checkpatch.pl tool with the " --strict" option, it complained that "CHECK: Unnecessary parentheses around reader_tasks[i].wq".
I know that you just moved the code position. The tool should have complained the original code. 😊

Quoted 6 lines

Other than that, this patch LGTM.

Thanks!
-Qiuxu

Thanks for the review. I will do --strict test next time.

Cheers,
Longman

Quoted 12 lines
Quoted 12 lines
kernel/rcu/refscale.c +0 -3
  		firsterr = torture_create_kthread(ref_scale_reader, (void *)i,
  						  reader_tasks[i].task);
  		if (torture_init_error(firsterr))
  			goto unwind;
-
-		init_waitqueue_head(&(reader_tasks[i].wq));
  	}
 
  	// Main Task
--

2.31.1

maintainer Reviewed-by on v1 1/1
#3 lore raw
In reply to Waiman Long
Quoted 18 lines · Waiman Long

It was found that running the refscale test might sometimes crash the kernel with the following error:

[ 8569.952896] BUG: unable to handle page fault for address: ffffffffffffffe8
[ 8569.952900] #PF: supervisor read access in kernel mode
[ 8569.952902] #PF: error_code(0x0000) - not-present page
[ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0
[ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI
[ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.2.4 05/28/2021
[ 8569.952917] RIP: 0010:prepare_to_wait_event+0x101/0x190
 :
[ 8569.952940] Call Trace:
[ 8569.952941]  <TASK>
[ 8569.952944]  ref_scale_reader+0x380/0x4a0 [refscale]
[ 8569.952959]  kthread+0x10e/0x130
[ 8569.952966]  ret_from_fork+0x1f/0x30
[ 8569.952973]  </TASK>
Quoted 7 lines

This is likely caused by the fact that init_waitqueue_head() is called after the ref_scale_reader kthread is created. So the kthread may try to use the waitqueue head before it is properly initialized. Fix this by initializing the waitqueue head first before kthread creation.

Fixes 653ed64b01dc ("refperf: Add a test to measure performance of read-side synchronization")

Strange this wasn't reported sooner.

#4 lore raw
Quoted 25 lines · Davidlohr Bueso
Quoted 23 lines · Waiman Long

It was found that running the refscale test might sometimes crash the kernel with the following error:

[ 8569.952896] BUG: unable to handle page fault for address: ffffffffffffffe8
[ 8569.952900] #PF: supervisor read access in kernel mode
[ 8569.952902] #PF: error_code(0x0000) - not-present page
[ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0
[ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI
[ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.2.4 05/28/2021
[ 8569.952917] RIP: 0010:prepare_to_wait_event+0x101/0x190
 :
[ 8569.952940] Call Trace:
[ 8569.952941]  <TASK>
[ 8569.952944]  ref_scale_reader+0x380/0x4a0 [refscale]
[ 8569.952959]  kthread+0x10e/0x130
[ 8569.952966]  ret_from_fork+0x1f/0x30
[ 8569.952973]  </TASK>

This is likely caused by the fact that init_waitqueue_head() is called after the ref_scale_reader kthread is created. So the kthread may try to use the waitqueue head before it is properly initialized. Fix this by initializing the waitqueue head first before kthread creation.

Quoted 4 lines
Quoted 2 lines
Fixes 653ed64b01dc ("refperf: Add a test to measure performance of read-side synchronization")

Strange this wasn't reported sooner.

Red Hat does have a pretty large QE organization that run all sort of tests include this one pretty frequently. The race window is pretty small, but they did hit this once in a while.

Cheers,
Longman

maintainer
#5 lore raw
Quoted 32 lines · Waiman Long
Quoted 31 lines · Davidlohr Bueso
Quoted 28 lines · Waiman Long

It was found that running the refscale test might sometimes crash the kernel with the following error:

[ 8569.952896] BUG: unable to handle page fault for address:
ffffffffffffffe8
[ 8569.952900] #PF: supervisor read access in kernel mode
[ 8569.952902] #PF: error_code(0x0000) - not-present page
[ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0
[ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI
[ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS
1.2.4 05/28/2021
[ 8569.952917] RIP: 0010:prepare_to_wait_event+0x101/0x190
 :
[ 8569.952940] Call Trace:
[ 8569.952941]  <TASK>
[ 8569.952944]  ref_scale_reader+0x380/0x4a0 [refscale]
[ 8569.952959]  kthread+0x10e/0x130
[ 8569.952966]  ret_from_fork+0x1f/0x30
[ 8569.952973]  </TASK>

This is likely caused by the fact that init_waitqueue_head() is called after the ref_scale_reader kthread is created. So the kthread may try to use the waitqueue head before it is properly initialized. Fix this by initializing the waitqueue head first before kthread creation.

Fixes 653ed64b01dc ("refperf: Add a test to measure performance of

read-side synchronization")

Quoted 5 lines
Quoted 1 line

Strange this wasn't reported sooner.

Red Hat does have a pretty large QE organization that run all sort of tests include this one pretty frequently. The race window is pretty small, but they did hit this once in a while.

I do run this fairly frequently, but haven't managed to hit it.

Good show on making it happen, and looking forward to the updated patch!

Thanx, Paul

#6 lore raw
Quoted 32 lines · Paul E. McKenney
Quoted 31 lines · Waiman Long
Quoted 30 lines · Davidlohr Bueso
Quoted 28 lines · Waiman Long

It was found that running the refscale test might sometimes crash the kernel with the following error:

[ 8569.952896] BUG: unable to handle page fault for address:
ffffffffffffffe8
[ 8569.952900] #PF: supervisor read access in kernel mode
[ 8569.952902] #PF: error_code(0x0000) - not-present page
[ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0
[ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI
[ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS
1.2.4 05/28/2021
[ 8569.952917] RIP: 0010:prepare_to_wait_event+0x101/0x190
  :
[ 8569.952940] Call Trace:
[ 8569.952941]  <TASK>
[ 8569.952944]  ref_scale_reader+0x380/0x4a0 [refscale]
[ 8569.952959]  kthread+0x10e/0x130
[ 8569.952966]  ret_from_fork+0x1f/0x30
[ 8569.952973]  </TASK>

This is likely caused by the fact that init_waitqueue_head() is called after the ref_scale_reader kthread is created. So the kthread may try to use the waitqueue head before it is properly initialized. Fix this by initializing the waitqueue head first before kthread creation.

Fixes 653ed64b01dc ("refperf: Add a test to measure performance of

read-side synchronization")

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Strange this wasn't reported sooner.

Red Hat does have a pretty large QE organization that run all sort of tests include this one pretty frequently. The race window is pretty small, but they did hit this once in a while.

I do run this fairly frequently, but haven't managed to hit it.

Good show on making it happen, and looking forward to the updated patch!

v2 sent with a bit more explanation text in the commit log.

Cheers,
Longman

#7 lore raw
In reply to Waiman Long
Quoted 30 lines · Waiman Long
Quoted 25 lines · Davidlohr Bueso
Quoted 25 lines · Waiman Long

It was found that running the refscale test might sometimes crash the kernel with the following error:

[ 8569.952896] BUG: unable to handle page fault for address: ffffffffffffffe8
[ 8569.952900] #PF: supervisor read access in kernel mode
[ 8569.952902] #PF: error_code(0x0000) - not-present page
[ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0
[ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI
[ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.2.4 05/28/2021
[ 8569.952917] RIP: 0010:prepare_to_wait_event+0x101/0x190
 :
[ 8569.952940] Call Trace:
[ 8569.952941]  <TASK>
[ 8569.952944]  ref_scale_reader+0x380/0x4a0 [refscale]
[ 8569.952959]  kthread+0x10e/0x130
[ 8569.952966]  ret_from_fork+0x1f/0x30
[ 8569.952973]  </TASK>

This is likely caused by the fact that init_waitqueue_head() is called after the ref_scale_reader kthread is created. So the kthread may try to use the waitqueue head before it is properly initialized. Fix this by initializing the waitqueue head first before kthread creation.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
Fixes 653ed64b01dc ("refperf: Add a test to measure performance of read-side synchronization")

Strange this wasn't reported sooner.

Red Hat does have a pretty large QE organization that run all sort of tests include this one pretty frequently. The race window is pretty small, but they did hit this once in a while.

Having worked on this test initially, I am happy to hear that Redhat runs this test!

Thanks for fixing this issue.

-Joel

Quoted 4 lines

Cheers,
Longman

maintainer Applied
#8 lore raw
Quoted 33 lines · Joel Fernandes
Quoted 33 lines · Waiman Long
Quoted 28 lines · Davidlohr Bueso
Quoted 27 lines · Waiman Long

It was found that running the refscale test might sometimes crash the kernel with the following error:

[ 8569.952896] BUG: unable to handle page fault for address: ffffffffffffffe8
[ 8569.952900] #PF: supervisor read access in kernel mode
[ 8569.952902] #PF: error_code(0x0000) - not-present page
[ 8569.952904] PGD c4b048067 P4D c4b049067 PUD c4b04b067 PMD 0
[ 8569.952910] Oops: 0000 [#1] PREEMPT_RT SMP NOPTI
[ 8569.952916] Hardware name: Dell Inc. PowerEdge R750/0WMWCR, BIOS 1.2.4 05/28/2021
[ 8569.952917] RIP: 0010:prepare_to_wait_event+0x101/0x190
 :
[ 8569.952940] Call Trace:
[ 8569.952941]  <TASK>
[ 8569.952944]  ref_scale_reader+0x380/0x4a0 [refscale]
[ 8569.952959]  kthread+0x10e/0x130
[ 8569.952966]  ret_from_fork+0x1f/0x30
[ 8569.952973]  </TASK>

This is likely caused by the fact that init_waitqueue_head() is called after the ref_scale_reader kthread is created. So the kthread may try to use the waitqueue head before it is properly initialized. Fix this by initializing the waitqueue head first before kthread creation.

Fixes 653ed64b01dc ("refperf: Add a test to measure performance of read-side synchronization")
Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Strange this wasn't reported sooner.

Red Hat does have a pretty large QE organization that run all sort of tests include this one pretty frequently. The race window is pretty small, but they did hit this once in a while.

Having worked on this test initially, I am happy to hear that Redhat runs this test!

Thanks for fixing this issue.

Applied, thank you!

Thanx, Paul