rcu/kvfree: Use READ_ONCE() when access to krcp->head
Uladzislau Rezki (Sony)
maintainerPATCH 1/1 rcu/kvfree: Use READ_ONCE() when access to krcp->head
A need_offload_krc() function is now lock-free. A compiler can optimize readers in way that they see an old value even though writers already updated the krcp->head from another path.
--- kernel/rcu/tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/rcu/tree.c +2 -2
@@ -3029,7 +3029,7 @@ need_offload_krc(struct kfree_rcu_cpu *krcp)if (!list_empty (&krcp->bulk_head[i]))return true; -return !!krcp->head; +return !!READ_ONCE (krcp->head); } static void
@@ -3070,7 +3070,7 @@ static void kfree_rcu_monitor(struct work_struct *work)// in that case the monitor work is rearmed. if ((!list_empty (&krcp->bulk_head[0 ]) &&list_empty (&krwp->bulk_head_free[0 ])) || (!list_empty (&krcp->bulk_head[1 ]) &&list_empty (&krwp->bulk_head_free[1 ])) || - (krcp->head && !krwp->head_free)) { + (READ_ONCE (krcp->head) && !krwp->head_free)) {// Channel 1 corresponds to the SLAB-pointer bulk path. // Channel 2 corresponds to vmalloc-pointer bulk path.
Signature · 1 line
2.30.2
Quoted 6 lines · Uladzislau Rezki (Sony)
A
need_offload_krc()function is now lock-free. A compiler can optimize readers in way that they see an old value even though writers already updated thekrcp->headfrom another path.Signed-off-by Uladzislau Rezki (Sony) <[email protected]>
Queued, thank you!!!
With the usual wordsmithing, so please let me know if I messed something up.
Thanx, Paul
------------------------------------------------------------------------
commit 9615ca28bc2c94cf2e092f06baace941fbf3b7e8
Author: Uladzislau Rezki (Sony) <[email protected]> Date: Fri Dec 2 14:18:37 2022 +0100
rcu/kvfree: Use READ_ONCE() when access to krcp->head
The need_offload_krc() function is now lock-free, which gives the compiler freedom to load old values from plain C-language loads from the kfree_rcu_cpu struture's ->head pointer. This commit therefore applied READ_ONCE() to these loads.
kernel/rcu/tree.c +2 -2
@@ -3029,7 +3029,7 @@ need_offload_krc(struct kfree_rcu_cpu *krcp)if (!list_empty (&krcp->bulk_head[i]))return true; -return !!krcp->head; +return !!READ_ONCE (krcp->head); } static void
@@ -3070,7 +3070,7 @@ static void kfree_rcu_monitor(struct work_struct *work)// in that case the monitor work is rearmed. if ((!list_empty (&krcp->bulk_head[0 ]) &&list_empty (&krwp->bulk_head_free[0 ])) || (!list_empty (&krcp->bulk_head[1 ]) &&list_empty (&krwp->bulk_head_free[1 ])) || - (krcp->head && !krwp->head_free)) { + (READ_ONCE (krcp->head) && !krwp->head_free)) {// Channel 1 corresponds to the SLAB-pointer bulk path. // Channel 2 corresponds to vmalloc-pointer bulk path.