Skip to contentSkip to conversation

rcu/kvfree: Use READ_ONCE() when access to krcp->head

PATCH Applied Uladzislau Rezki (Sony) 2 participants 2 messages Dec 2, 2022 RCU

lore
Share & tools
lore

Uladzislau Rezki (Sony)

author maintainer
#0 lore raw

PATCH 1/1 rcu/kvfree: Use READ_ONCE() when access to krcp->head

A need_offload_krc() function is now lock-free. A compiler can optimize readers in way that they see an old value even though writers already updated the krcp->head from another path.

---
 kernel/rcu/tree.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/rcu/tree.c +2 -2
index efc79f942b30..d155f2594317 100644
@@ -3029,7 +3029,7 @@ need_offload_krc(struct kfree_rcu_cpu *krcp)
 		if (!list_empty(&krcp->bulk_head[i]))
 			return true;
 
-	return !!krcp->head;
+	return !!READ_ONCE(krcp->head);
 }
 
 static void
@@ -3070,7 +3070,7 @@ static void kfree_rcu_monitor(struct work_struct *work)
 		// in that case the monitor work is rearmed.
 		if ((!list_empty(&krcp->bulk_head[0]) && list_empty(&krwp->bulk_head_free[0])) ||
 			(!list_empty(&krcp->bulk_head[1]) && list_empty(&krwp->bulk_head_free[1])) ||
-				(krcp->head && !krwp->head_free)) {
+				(READ_ONCE(krcp->head) && !krwp->head_free)) {
 
 			// Channel 1 corresponds to the SLAB-pointer bulk path.
 			// Channel 2 corresponds to vmalloc-pointer bulk path.
Signature · 1 line
2.30.2
maintainer Applied
#1 lore raw
Quoted 6 lines · Uladzislau Rezki (Sony)

A need_offload_krc() function is now lock-free. A compiler can optimize readers in way that they see an old value even though writers already updated the krcp->head from another path.

Queued, thank you!!!

With the usual wordsmithing, so please let me know if I messed something up.

Thanx, Paul

------------------------------------------------------------------------

commit 9615ca28bc2c94cf2e092f06baace941fbf3b7e8

Author: Uladzislau Rezki (Sony) <[email protected]>
Date:   Fri Dec 2 14:18:37 2022 +0100

rcu/kvfree: Use READ_ONCE() when access to krcp->head

The need_offload_krc() function is now lock-free, which gives the compiler freedom to load old values from plain C-language loads from the kfree_rcu_cpu struture's ->head pointer. This commit therefore applied READ_ONCE() to these loads.

kernel/rcu/tree.c +2 -2
index 2d16a0eee792d..ee8a6a711719a 100644
@@ -3029,7 +3029,7 @@ need_offload_krc(struct kfree_rcu_cpu *krcp)
 		if (!list_empty(&krcp->bulk_head[i]))
 			return true;
 
-	return !!krcp->head;
+	return !!READ_ONCE(krcp->head);
 }
 
 static void
@@ -3070,7 +3070,7 @@ static void kfree_rcu_monitor(struct work_struct *work)
 		// in that case the monitor work is rearmed.
 		if ((!list_empty(&krcp->bulk_head[0]) && list_empty(&krwp->bulk_head_free[0])) ||
 			(!list_empty(&krcp->bulk_head[1]) && list_empty(&krwp->bulk_head_free[1])) ||
-				(krcp->head && !krwp->head_free)) {
+				(READ_ONCE(krcp->head) && !krwp->head_free)) {
 
 			// Channel 1 corresponds to the SLAB-pointer bulk path.
 			// Channel 2 corresponds to vmalloc-pointer bulk path.