Toyrcu: replace plain accesses with READ_ONCE() and
v2 cover Toyrcu: replace plain accesses with READ_ONCE() and
Hi Paul and Akira,
This patch set replaces plain accesses and ACCESS_ONCE() with READ_ONCE() and WRITE_ONCE() in Toy RCU examples. I will send another patch updating tex file accordingly on top of this patch set.
--
Junchang Wang (3):
rcu: Use READ_ONCE() and WRITE_ONCE() for shared variable rcu_gp_ctr
rcu_nest: Use READ_ONCE() and WRITE_ONCE() for shared variable
rcu_gp_ctr
rcu_qs: Use READ_ONCE() AND WRITE_ONCE() for shared variable
rcu_gp_ctr
CodeSamples/defer/rcu.c | 2 +- CodeSamples/defer/rcu.h | 4 ++-- CodeSamples/defer/rcu_nest.c | 2 +- CodeSamples/defer/rcu_nest.h | 2 +- CodeSamples/defer/rcu_qs.c | 2 +- CodeSamples/defer/rcu_qs.h | 4 ++-- 6 files changed, 8 insertions(+), 8 deletions(-)
Signature · 1 line
2.7.4
PATCH v2 1/3 rcu: Use READ_ONCE() and WRITE_ONCE() for shared variable rcu_gp_ctr
--- CodeSamples/defer/rcu.c | 2 +- CodeSamples/defer/rcu.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
CodeSamples/defer/rcu.c +1 -1
@@ -35,7 +35,7 @@ void synchronize_rcu(void)/* Advance to a new grace-period number, enforce ordering. */ - rcu_gp_ctr +=2 ; +WRITE_ONCE (rcu_gp_ctr, rcu_gp_ctr +2 );smp_mb ();/*
CodeSamples/defer/rcu.h +2 -2
@@ -41,7 +41,7 @@ static inline void rcu_read_lock(void)* periodic per-thread processing.) */ -__get_thread_var (rcu_reader_gp) = rcu_gp_ctr +1 ; +__get_thread_var (rcu_reader_gp) =READ_ONCE (rcu_gp_ctr) +1 ;smp_mb (); }
@@ -55,7 +55,7 @@ static inline void rcu_read_unlock(void)*/ smp_mb (); -__get_thread_var (rcu_reader_gp) = rcu_gp_ctr; +__get_thread_var (rcu_reader_gp) =READ_ONCE (rcu_gp_ctr); }extern void synchronize_rcu (void );
Signature · 1 line
2.7.4
PATCH v2 2/3 rcu_nest: Use READ_ONCE() and WRITE_ONCE() for shared variable rcu_gp_ctr
In reply to Junchang Wang--- CodeSamples/defer/rcu_nest.c | 2 +- CodeSamples/defer/rcu_nest.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
CodeSamples/defer/rcu_nest.c +1 -1
@@ -35,7 +35,7 @@ void synchronize_rcu(void)/* Advance to a new grace-period number, enforce ordering. */ - rcu_gp_ctr +=RCU_GP_CTR_BOTTOM_BIT ; +WRITE_ONCE (rcu_gp_ctr, rcu_gp_ctr +RCU_GP_CTR_BOTTOM_BIT );smp_mb ();/*
CodeSamples/defer/rcu_nest.h +1 -1
@@ -52,7 +52,7 @@ static void rcu_read_lock(void) retry: tmp = *rrgp;if ((tmp &RCU_GP_CTR_NEST_MASK ) ==0 ) - tmp = rcu_gp_ctr; + tmp =READ_ONCE (rcu_gp_ctr); tmp++; *rrgp = tmp;smp_mb ();
Signature · 1 line
2.7.4
PATCH v2 3/3 rcu_qs: Use READ_ONCE() AND WRITE_ONCE() for shared variable rcu_gp_ctr
In reply to Junchang Wang--- CodeSamples/defer/rcu_qs.c | 2 +- CodeSamples/defer/rcu_qs.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
CodeSamples/defer/rcu_qs.c +1 -1
@@ -41,7 +41,7 @@ void synchronize_rcu(void)/* Advance to a new grace-period number, enforce ordering. */ - rcu_gp_ctr +=2 ; +WRITE_ONCE (rcu_gp_ctr, rcu_gp_ctr +2 );smp_mb ();/*
CodeSamples/defer/rcu_qs.h +2 -2
@@ -59,14 +59,14 @@ static void rcu_read_unlock(void)static void rcu_quiescent_state (void ) {smp_mb (); -__get_thread_var (rcu_reader_qs_gp) =ACCESS_ONCE (rcu_gp_ctr) +1 ; +__get_thread_var (rcu_reader_qs_gp) =READ_ONCE (rcu_gp_ctr) +1 ;smp_mb (); }static void rcu_thread_offline (void ) {smp_mb (); -__get_thread_var (rcu_reader_qs_gp) =ACCESS_ONCE (rcu_gp_ctr); +__get_thread_var (rcu_reader_qs_gp) =READ_ONCE (rcu_gp_ctr);smp_mb (); }
Signature · 1 line
2.7.4
Hi Junchang,
Quoted 5 lines · Junchang Wang
Hi Paul and Akira,
This patch set replaces plain accesses and
ACCESS_ONCE()withREAD_ONCE()andWRITE_ONCE()in Toy RCU examples. I will send another patch updating tex file accordingly on top of this patch set.
As far as accesses to rcu_gp_ctr, this patch set looks good to me.
In updating toyrcu.tex file, I suggested to extract the snippets from code under CodeSamples/defer, but some snippets in toyrcu.tex consist of code from both .h and .c. Current scheme does not support such merging of snippets from multiple source files.
Your option is to manually convert verbbox environment to Verbatim environment. Sorry I suggested infeasible extractions the other day. rcu_qs.c and rcu_qs.h are in separate snippets, and they can be extracted by fcvextract.pl. In this case, commit 4a41491166cd ("defer/hazptr: Extract snippet from hazptr.c") should be a good example, especially the "gobbleblank=yes" option.
If you have any question, feel free to ask me. I'm looking forward to seeing your update.
Paul, in the sources Junchang has touched, per-thread variable "rcu_reader_gp" is updated by its owner, but is read accessed from for_each_thread loops in updater side. So I think accesses to each of them need to be annotated by WRITE_ONCE (in owner) and READ_ONCE (in updater). I looked at only the diffs in the 1st patch set and didn't notice the raciness of them.
Macros for accessing per-thread variables are expanded to array references, and there are "barrier()"s in these loops.
So I think there is little chance those accesses would be optimized by currently available compilers.
Do you see the need to annotate the accesses to per-thread variable?
Thanks, Akira
Quoted 17 lines
--
Junchang Wang (3):
rcu: UseREAD_ONCE()andWRITE_ONCE()for shared variablercu_gp_ctr
rcu_nest: UseREAD_ONCE()andWRITE_ONCE()for shared variable
rcu_gp_ctr
rcu_qs: UseREAD_ONCE()ANDWRITE_ONCE()for shared variable
rcu_gp_ctrCodeSamples/defer/rcu.c | 2 +- CodeSamples/defer/rcu.h | 4 ++-- CodeSamples/defer/rcu_nest.c | 2 +- CodeSamples/defer/rcu_nest.h | 2 +- CodeSamples/defer/rcu_qs.c | 2 +- CodeSamples/defer/rcu_qs.h | 4 ++-- 6 files changed, 8 insertions(+), 8 deletions(-)
Quoted 24 lines · Akira Yokosawa
Hi Junchang,
Quoted 5 lines · Junchang Wang
Hi Paul and Akira,
This patch set replaces plain accesses and
ACCESS_ONCE()withREAD_ONCE()andWRITE_ONCE()in Toy RCU examples. I will send another patch updating tex file accordingly on top of this patch set.As far as accesses to
rcu_gp_ctr, this patch set looks good to me.In updating toyrcu.tex file, I suggested to extract the snippets from code under CodeSamples/defer, but some snippets in toyrcu.tex consist of code from both .h and .c. Current scheme does not support such merging of snippets from multiple source files.
Your option is to manually convert verbbox environment to Verbatim environment. Sorry I suggested infeasible extractions the other day.
rcu_qs.candrcu_qs.hare in separate snippets, and they can be extracted by fcvextract.pl. In this case, commit4a41491166cd("defer/hazptr: Extract snippet fromhazptr.c") should be a good example, especially the "gobbleblank=yes" option.
Quoted 2 lines
If you have any question, feel free to ask me. I'm looking forward to seeing your update.
Hi Akira,
Thanks for the note. Would be happy to submit a new patch for toyrcu.tex according to your suggestions soon.
Quoted 8 lines
Paul, in the sources Junchang has touched, per-thread variable "
rcu_reader_gp" is updated by its owner, but is read accessed fromfor_each_threadloops in updater side. So I think accesses to each of them need to be annotated byWRITE_ONCE(in owner) andREAD_ONCE(in updater). I looked at only the diffs in the 1st patch set and didn't notice the raciness of them.
Quoted 5 lines
Macros for accessing per-thread variables are expanded to array references, and there are "barrier()"s in these loops.
So I think there is little chance those accesses would be optimized by currently available compilers.
Great catch! My understanding is that one of the nice benefits of READ_ONCE/WRITE_ONCE is that they explicitly notify programmers the spots where there might be racy accesses. The more we eliminate them, the harder a programmer verifies the code. Of course, that's my current point of view. I would be happy to see your choices.
Thanks,
--Junchang
Quoted 22 lines
Do you see the need to annotate the accesses to per-thread variable?
Thanks, Akira
Quoted 17 lines
--
Junchang Wang (3):
rcu: UseREAD_ONCE()andWRITE_ONCE()for shared variablercu_gp_ctr
rcu_nest: UseREAD_ONCE()andWRITE_ONCE()for shared variable
rcu_gp_ctr
rcu_qs: UseREAD_ONCE()ANDWRITE_ONCE()for shared variable
rcu_gp_ctrCodeSamples/defer/rcu.c | 2 +- CodeSamples/defer/rcu.h | 4 ++-- CodeSamples/defer/rcu_nest.c | 2 +- CodeSamples/defer/rcu_nest.h | 2 +- CodeSamples/defer/rcu_qs.c | 2 +- CodeSamples/defer/rcu_qs.h | 4 ++-- 6 files changed, 8 insertions(+), 8 deletions(-)
Quoted 16 lines · Akira Yokosawa
Hi Junchang,
Quoted 5 lines · Junchang Wang
Hi Paul and Akira,
This patch set replaces plain accesses and
ACCESS_ONCE()withREAD_ONCE()andWRITE_ONCE()in Toy RCU examples. I will send another patch updating tex file accordingly on top of this patch set.As far as accesses to
rcu_gp_ctr, this patch set looks good to me.In updating toyrcu.tex file, I suggested to extract the snippets from code under CodeSamples/defer, but some snippets in toyrcu.tex consist of code from both .h and .c. Current scheme does not support such merging of snippets from multiple source files.
Quoted 6 lines
Your option is to manually convert verbbox environment to Verbatim environment. Sorry I suggested infeasible extractions the other day.
rcu_qs.candrcu_qs.hare in separate snippets, and they can be extracted by fcvextract.pl. In this case, commit4a41491166cd("defer/hazptr: Extract snippet fromhazptr.c") should be a good example, especially the "gobbleblank=yes" option.
Given that these are toy implementations, pleaee feel free to reorganize the code to make snippet extraction work better. (Please check to see if anything else uses these -- I do not believe so, but memory can be a tricky thing sometimes.)
Quoted 9 lines
If you have any question, feel free to ask me. I'm looking forward to seeing your update.
Paul, in the sources Junchang has touched, per-thread variable "
rcu_reader_gp" is updated by its owner, but is read accessed fromfor_each_threadloops in updater side. So I think accesses to each of them need to be annotated byWRITE_ONCE(in owner) andREAD_ONCE(in updater). I looked at only the diffs in the 1st patch set and didn't notice the raciness of them.
Agreed, WRITE_ONCE() by owner and READ_ONCE() by updater.
Quoted 6 lines
Macros for accessing per-thread variables are expanded to array references, and there are "barrier()"s in these loops.
So I think there is little chance those accesses would be optimized by currently available compilers.Do you see the need to annotate the accesses to per-thread variable?
If a per-thread variable is accessed by multiple threads, it would be good to annotate the accesses as needed. That said, I have no idea how well annotation interacts with the per-thread accesses, so some experimentation might be required.
Thnax, Paul
Quoted 20 lines
Thanks, Akira
Quoted 17 lines
--
Junchang Wang (3):
rcu: UseREAD_ONCE()andWRITE_ONCE()for shared variablercu_gp_ctr
rcu_nest: UseREAD_ONCE()andWRITE_ONCE()for shared variable
rcu_gp_ctr
rcu_qs: UseREAD_ONCE()ANDWRITE_ONCE()for shared variable
rcu_gp_ctrCodeSamples/defer/rcu.c | 2 +- CodeSamples/defer/rcu.h | 4 ++-- CodeSamples/defer/rcu_nest.c | 2 +- CodeSamples/defer/rcu_nest.h | 2 +- CodeSamples/defer/rcu_qs.c | 2 +- CodeSamples/defer/rcu_qs.h | 4 ++-- 6 files changed, 8 insertions(+), 8 deletions(-)
Hi Junchang, Paul,
Quoted 10 lines · Paul E. McKenney
Quoted 9 lines · Akira Yokosawa
Hi Junchang,
Quoted 5 lines · Junchang Wang
Hi Paul and Akira,
This patch set replaces plain accesses and
ACCESS_ONCE()withREAD_ONCE()andWRITE_ONCE()in Toy RCU examples. I will send another patch updating tex file accordingly on top of this patch set.
Quoted 6 lines
Quoted 6 lines
As far as accesses to
rcu_gp_ctr, this patch set looks good to me.In updating toyrcu.tex file, I suggested to extract the snippets from code under CodeSamples/defer, but some snippets in toyrcu.tex consist of code from both .h and .c. Current scheme does not support such merging of snippets from multiple source files.
As a matter of fact, it turns out this is actually possible.
Please see the patch below, which converts Listing B.1.
Junchang, if it is not too late, you can use the same approach to snippets from rcu.[ch] and rcu_nest.[ch].
Thanks, Akira
-----8<-----------------------
From 3b3c6d99bcd580385175bf2a062dcb038f0cfc9e Mon Sep 17 00:00:00 2001
From: Akira Yokosawa <[email protected]>
Date: Sat, 27 Apr 2019 22:34:40 +0900
Subject: [PATCH] toyrcu: Convert Listing B.1 to new scheme
This commit is an example where code extracted from two source files under CodeSamples/ is merged into a code snippet. Extra vertical space needs to be canceled by the \vspace*{} command between the two \input{} commands. \fvset{firstnumber=last} is to continue line numbering. \fvset{firstnumber=auto} restores the default setting.
Also tweak positioning option of listing environment.
--- CodeSamples/defer/rcu_lock.c | 2 ++ CodeSamples/defer/rcu_lock.h | 2 ++ appendix/toyrcu/toyrcu.tex | 26 ++++---------------------- 3 files changed, 8 insertions(+), 22 deletions(-)
CodeSamples/defer/rcu_lock.c +2 -0
@@ -20,12 +20,14 @@#include "../api.h" #include "rcu_lock.h" +//\begin{snippet}[labelbase=ln:defer:rcu_lock:synchronize,commandchars=\\\[\]] void synchronize_rcu (void ) {spin_lock (&rcu_gp_lock);spin_unlock (&rcu_gp_lock); } +//\end{snippet} #ifdef TEST #include "rcutorture.h"
CodeSamples/defer/rcu_lock.h +2 -0
@@ -26,6 +26,7 @@ static void rcu_init(void) { } +//\begin{snippet}[labelbase=ln:defer:rcu_lock:lock_unlock,commandchars=\\\[\]] static void rcu_read_lock (void ) {spin_lock (&rcu_gp_lock);
@@ -35,5 +36,6 @@ static void rcu_read_unlock(void) {spin_unlock (&rcu_gp_lock); } +//\end{snippet} extern void synchronize_rcu (void );
appendix/toyrcu/toyrcu.tex +4 -22
@@ -30,29 +30,11 @@ provides a summary and a list of desirable RCU properties. \section{Lock-Based RCU} \label{sec:app:toyrcu:Lock-Based RCU} +\NoIndentAfterThis -\begin{listing}[bp] -{ \scriptsize -\begin{verbbox} - 1 static void rcu_read_lock(void) - 2 { - 3 spin_lock(&rcu_gp_lock); - 4 } - 5 - 6 static void rcu_read_unlock(void) - 7 { - 8 spin_unlock(&rcu_gp_lock); - 9 } - 10 - 11 void synchronize_rcu(void) - 12 { - 13 spin_lock(&rcu_gp_lock); - 14 spin_unlock(&rcu_gp_lock); - 15 } -\end{verbbox} -} -\centering -\theverbbox +\begin{listing}[htbp] +\input{CodeSamples/defer/rcu_lock@lock_unlock.fcv}\vspace*{-11pt}\fvset{firstnumber=last} +\input{CodeSamples/defer/[email protected]}\fvset{firstnumber=auto} \caption{Lock-Based RCU Implementation} \label{lst:app:toyrcu:Lock-Based RCU Implementation} \end{listing}
Signature · 1 line
2.17.1
Quoted 21 lines · Akira Yokosawa
Hi Junchang, Paul,
Quoted 10 lines · Paul E. McKenney
Quoted 9 lines · Akira Yokosawa
Hi Junchang,
Quoted 5 lines · Junchang Wang
Hi Paul and Akira,
This patch set replaces plain accesses and
ACCESS_ONCE()withREAD_ONCE()andWRITE_ONCE()in Toy RCU examples. I will send another patch updating tex file accordingly on top of this patch set.Quoted 6 lines
Quoted 6 lines
As far as accesses to
rcu_gp_ctr, this patch set looks good to me.In updating toyrcu.tex file, I suggested to extract the snippets from code under CodeSamples/defer, but some snippets in toyrcu.tex consist of code from both .h and .c. Current scheme does not support such merging of snippets from multiple source files.
Quoted 7 lines
As a matter of fact, it turns out this is actually possible.
Please see the patch below, which converts Listing B.1.
Junchang, if it is not too late, you can use the same approach to snippets from rcu.[ch] and rcu_nest.[ch].
Hi Akira,
Thanks for the patch, which works well for rcu.[ch]. I can apply this scheme to other examples and submit the patch set tomorrow.
Thanks,
--Junchang
Quoted 103 lines
Thanks, Akira
-----8<-----------------------
From3b3c6d99bcd580385175bf2a062dcb038f0cfc9eMon Sep 17 00:00:00 2001
From: Akira Yokosawa <[email protected]>
Date: Sat, 27 Apr 2019 22:34:40 +0900
Subject: [PATCH] toyrcu: Convert Listing B.1 to new schemeThis commit is an example where code extracted from two source files under CodeSamples/ is merged into a code snippet. Extra vertical space needs to be canceled by the \vspace*{} command between the two \input{} commands. \fvset{firstnumber=last} is to continue line numbering. \fvset{firstnumber=auto} restores the default setting.
Also tweak positioning option of listing environment.
Signed-off-by Akira Yokosawa <[email protected]>--- CodeSamples/defer/rcu_lock.c | 2 ++ CodeSamples/defer/rcu_lock.h | 2 ++ appendix/toyrcu/toyrcu.tex | 26 ++++---------------------- 3 files changed, 8 insertions(+), 22 deletions(-)CodeSamples/defer/rcu_lock.c +2
-0@@ -20,12 +20,14 @@#include "../api.h" #include "rcu_lock.h" +//\begin{snippet}[labelbase=ln:defer:rcu_lock:synchronize,commandchars=\\\[\]] void synchronize_rcu (void ) {spin_lock (&rcu_gp_lock);spin_unlock (&rcu_gp_lock); } +//\end{snippet} #ifdef TEST #include "rcutorture.h" CodeSamples/defer/rcu_lock.h +2
-0@@ -26,6 +26,7 @@ static void rcu_init(void) { } +//\begin{snippet}[labelbase=ln:defer:rcu_lock:lock_unlock,commandchars=\\\[\]] static void rcu_read_lock (void ) {spin_lock (&rcu_gp_lock);@@ -35,5 +36,6 @@ static void rcu_read_unlock(void) {spin_unlock (&rcu_gp_lock); } +//\end{snippet} extern void synchronize_rcu (void );appendix/toyrcu/toyrcu.tex +4
-23@@ -30,29 +30,11 @@ provides a summary and a list of desirable RCU properties. \section{Lock-Based RCU} \label{sec:app:toyrcu:Lock-Based RCU} +\NoIndentAfterThis -\begin{listing}[bp] -{ \scriptsize -\begin{verbbox} - 1 static void rcu_read_lock(void) - 2 { - 3 spin_lock(&rcu_gp_lock); - 4 } - 5 - 6 static void rcu_read_unlock(void) - 7 { - 8 spin_unlock(&rcu_gp_lock); - 9 } - 10 - 11 void synchronize_rcu(void) - 12 { - 13 spin_lock(&rcu_gp_lock); - 14 spin_unlock(&rcu_gp_lock); - 15 } -\end{verbbox} -} -\centering -\theverbbox +\begin{listing}[htbp] +\input{CodeSamples/defer/rcu_lock@lock_unlock.fcv}\vspace*{-11pt}\fvset{firstnumber=last} +\input{CodeSamples/defer/[email protected]}\fvset{firstnumber=auto} \caption{Lock-Based RCU Implementation} \label{lst:app:toyrcu:Lock-Based RCU Implementation} \end{listing} --2.17.1