rcu: small rcu_dereference doc update
Subsystems (2)
PATCH 1/1 rcu: small rcu_dereference doc update
Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.
--- Documentation/RCU/whatisRCU.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Documentation/RCU/whatisRCU.txt +3 -1
@@ -256,7 +256,9 @@ rcu_dereference() If you are going to be fetching multiple fields from the RCU-protected structure, using the local variable is of course preferred. Repeated rcu_dereference() calls look - ugly and incur unnecessary overhead on Alpha CPUs. + ugly, do not guarantee that same pointer will be returned + if update happened while in critical section and incur + unnecessary overhead on Alpha CPUs. Note that the value returned by rcu_dereference() is valid only within the enclosing RCU read-side critical section.
Signature · 1 line
2.1.0
Quoted 5 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>
Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.
Queued for 4.2.
Thanx, Paul
Quoted 22 lines
--- Documentation/RCU/whatisRCU.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)Documentation/RCU/whatisRCU.txt +3
-1@@ -256,7 +256,9 @@ rcu_dereference() If you are going to be fetching multiple fields from the RCU-protected structure, using the local variable is of course preferred. Repeated rcu_dereference() calls look - ugly and incur unnecessary overhead on Alpha CPUs. + ugly, do not guarantee that same pointer will be returned + if update happened while in critical section and incur + unnecessary overhead on Alpha CPUs. Note that the value returned by rcu_dereference() is valid only within the enclosing RCU read-side critical section.Signature · 1 line
2.1.0
Steven Rostedt
reviewerQuoted 4 lines · On Fri, 17 Apr 2015 07:06:38 -0700 "Paul E. McKenney"…
Quoted 3 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.
Quoted 8 lines
Quoted 2 lines
Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.
Queued for 4.2.
Before you queue it, there's a few articles that are screaming to be present...
Quoted 7 lines
Thanx, Paul
Quoted 4 lines
--- Documentation/RCU/whatisRCU.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Quoted 12 lines
Documentation/RCU/whatisRCU.txt rcu_dereference() · line 261
Quoted 12 lines
Documentation/RCU/whatisRCU.txt +3
-1@@ -256,7 +256,9 @@ rcu_dereference() If you are going to be fetching multiple fields from the RCU-protected structure, using the local variable is of course preferred. Repeated rcu_dereference() calls look - ugly and incur unnecessary overhead on Alpha CPUs. + ugly, do not guarantee that same pointer will be returned + if update happened while in critical section and incur + unnecessary overhead on Alpha CPUs.
ugly, do not guarantee that the same pointer will be returned if an update happened while in the critical section, and incur unnecessary overhead on Alpha CPUs.
-- Steve
Quoted 6 lines
Quoted 6 lines
Documentation/RCU/whatisRCU.txt +0
-0Note that the value returned by rcu_dereference() is valid only within the enclosing RCU read-side critical section.Signature · 1 line
2.1.0
Quoted 10 lines · Steven Rostedt
Quoted 7 lines · On Fri, 17 Apr 2015 07:06:38 -0700 "Paul E. McKenney"…
Quoted 5 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>
Quoted 8 lines
Quoted 5 lines
Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.
Queued for 4.2.
Before you queue it, there's a few articles that are screaming to be present...
OK, I removed it. Milos, when you send me a version that Steve is happy with, I will requeue it.
Thanx, Paul
Quoted 30 lines
Quoted 4 lines
Quoted 4 lines
--- Documentation/RCU/whatisRCU.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)Quoted 12 lines
Documentation/RCU/whatisRCU.txt rcu_dereference() · line 261
Quoted 12 lines
Documentation/RCU/whatisRCU.txt +3
-1@@ -256,7 +256,9 @@ rcu_dereference() If you are going to be fetching multiple fields from the RCU-protected structure, using the local variable is of course preferred. Repeated rcu_dereference() calls look - ugly and incur unnecessary overhead on Alpha CPUs. + ugly, do not guarantee that same pointer will be returned + if update happened while in critical section and incur + unnecessary overhead on Alpha CPUs.ugly, do not guarantee that the same pointer will be returned if an update happened while in the critical section, and incur unnecessary overhead on Alpha CPUs.
-- Steve
Quoted 6 lines
Quoted 6 lines
Documentation/RCU/whatisRCU.txt +0
-0Note that the value returned by rcu_dereference() is valid only within the enclosing RCU read-side critical section.Signature · 1 line
2.1.0
Quoted 26 lines · Steven Rostedt
Quoted 4 lines · On Fri, 17 Apr 2015 07:06:38 -0700 "Paul E. McKenney"…
Quoted 3 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.Quoted 8 lines
Quoted 2 lines
Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.
Queued for 4.2.
Before you queue it, there's a few articles that are screaming to be present...
Quoted 7 lines
Thanx, Paul
Quoted 4 lines
--- Documentation/RCU/whatisRCU.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Quoted 17 lines
Quoted 4 lines
Quoted 4 lines
Documentation/RCU/whatisRCU.txt +0
-0Quoted 8 lines
Documentation/RCU/whatisRCU.txt rcu_dereference() · line 261
Quoted 8 lines
Documentation/RCU/whatisRCU.txt +3
-1@@ -256,7 +256,9 @@ rcu_dereference() If you are going to be fetching multiple fields from the RCU-protected structure, using the local variable is of course preferred. Repeated rcu_dereference() calls look - ugly and incur unnecessary overhead on Alpha CPUs. + ugly, do not guarantee that same pointer will be returned + if update happened while in critical section and incur + unnecessary overhead on Alpha CPUs.ugly, do not guarantee that the same pointer will be returned if an update happened while in the critical section, and incur unnecessary overhead on Alpha CPUs.
Thanks. Corrected. Sending v2 in a bit.
Milos
Quoted 9 lines
-- Steve
Quoted 6 lines
Quoted 6 lines
Documentation/RCU/whatisRCU.txt +0
-0Note that the value returned by rcu_dereference() is valid only within the enclosing RCU read-side critical section.Signature · 1 line
2.1.0
Quoted 16 lines
-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 7:07 AM
To: Milos Vyletel
Cc: Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION; Jeff Haran
Subject: Re: [PATCH] rcu: smallrcu_dereferencedoc updateQuoted 5 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>
Quoted 7 lines
Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.
Queued for 4.2.
Thanx, Paul
It's just that the original text suggests repeated rcu_dereference() calls are discouraged because they are ugly and not efficient on some architectures. When I read that I concluded that those were the only reasons not to do it, that despite the possible inefficiency it would always return the same pointer. Depending on how one's code is structured, being able to do this could be advantageous. Then I started looking at the code that implements it and I couldn't see how it could possibly be the case. I even wrote a little kernel module to prove to myself that doing this could return different pointer values. If I misinterpreted the original text I figured others might also. Milos even found some code in the kernel where it's author had done this, so it might be a widely held misunderstanding. It's easy for people who have worked with rwlock_ts to think an RCU read lock works the same.
Thanks,
Jeff Haran
Quoted 22 lines
Quoted 22 lines
--- Documentation/RCU/whatisRCU.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/Documentation/RCU/whatisRCU.txtindex 88dfce1..82b1b2c 100644Documentation/RCU/whatisRCU.txt +3
-2@@ -256,7 +256,9 @@ rcu_dereference() If you are going to be fetching multiple fields from the RCU-protected structure, using the local variable is of course preferred. Repeated rcu_dereference() calls look - ugly and incur unnecessary overhead on Alpha CPUs. + ugly, do not guarantee that same pointer will be returned + if update happened while in critical section and incur + unnecessary overhead on Alpha CPUs. Note that the value returned by rcu_dereference() is valid only within the enclosing RCU read-side critical section. --2.1.0
Quoted 20 lines · Jeff Haran
Quoted 20 lines
-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 7:07 AM
To: Milos Vyletel
Cc: Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION; Jeff Haran
Subject: Re: [PATCH] rcu: smallrcu_dereferencedoc updateQuoted 5 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.
Quoted 5 lines
Quoted 3 lines
Queued for 4.2.
Thanx, Paul
It's just that the original text suggests repeated
rcu_dereference()calls are discouraged because they are ugly and not efficient on some architectures. When I read that I concluded that those were the only reasons not to do it, that despite the possible inefficiency it would always return the same pointer. Depending on how one's code is structured, being able to do this could be advantageous. Then I started looking at the code that implements it and I couldn't see how it could possibly be the case. I even wrote a little kernel module to prove to myself that doing this could return different pointer values. If I misinterpreted the original text I figured others might also. Milos even found some code in the kernel where it's author had done this, so it might be a widely held misunderstanding. It's easy for people who have worked withrwlock_tsto think an RCU read lock works the same.
Fair point, and thank you the rationale! Are there any other parts of the RCU documentation that are similarly blind to your initial point of view? If so, it would be good for them to be fixed.
Thanx, Paul
Quoted 27 lines
Thanks,
Jeff Haran
Quoted 22 lines
Quoted 22 lines
--- Documentation/RCU/whatisRCU.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)(quoted diff) +0
-0
b/Documentation/RCU/whatisRCU.txtindex 88dfce1..82b1b2c 100644Documentation/RCU/whatisRCU.txt +3
-2@@ -256,7 +256,9 @@ rcu_dereference() If you are going to be fetching multiple fields from the RCU-protected structure, using the local variable is of course preferred. Repeated rcu_dereference() calls look - ugly and incur unnecessary overhead on Alpha CPUs. + ugly, do not guarantee that same pointer will be returned + if update happened while in critical section and incur + unnecessary overhead on Alpha CPUs. Note that the value returned by rcu_dereference() is valid only within the enclosing RCU read-side critical section. --2.1.0
Quoted 47 lines
-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 11:41 AM
To: Jeff Haran
Cc: Milos Vyletel; Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION
Subject: Re: [PATCH] rcu: smallrcu_dereferencedoc updateQuoted 20 lines · Jeff Haran
Quoted 20 lines
-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 7:07 AM
To: Milos Vyletel
Cc: Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION; Jeff Haran
Subject: Re: [PATCH] rcu: smallrcu_dereferencedoc updateQuoted 5 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.
Quoted 5 lines
Quoted 3 lines
Queued for 4.2.
Thanx, Paul
It's just that the original text suggests repeated
rcu_dereference()calls arediscouraged because they are ugly and not efficient on some architectures. When I read that I concluded that those were the only reasons not to do it, that despite the possible inefficiency it would always return the same pointer. Depending on how one's code is structured, being able to do this could be advantageous. Then I started looking at the code that implements it and I couldn't see how it could possibly be the case. I even wrote a little kernel module to prove to myself that doing this could return different pointer values. If I misinterpreted the original text I figured others might also. Milos even found some code in the kernel where it's author had done this, so it might be a widely held misunderstanding. It's easy for people who have worked with
rwlock_tsto think an RCU read lock works the same.
Quoted 5 lines
Fair point, and thank you the rationale! Are there any other parts of the RCU documentation that are similarly blind to your initial point of view? If so, it would be good for them to be fixed.
Thanx, Paul
I can't think of much off the top of my head, but I'm hoping I might get some time to review it again and perhaps provide some more concrete suggestions.
One thing that does come to mind is the article you wrote in LWN, http://lwn.net/Articles/263130/, where you discussed RCU as a reader-write lock replacement. whatisRCU.txt seems to incorporated some of that. Something along the lines of the original section in the LWN article where there was some discussion of the differences between a rwlock_t read lock critical section and a RCU read lock critical section might be beneficial, a key thing being that with RCU there really is no locking, the value of the pointer can change in a RCU critical section because writers aren't blocked from updating it. Another thing might be some discussion that the cases where you'd call read_lock_bh() are way different than when you'd call rcu_read_lock_bh() and as a corollary why there is no rcu_read_lock_irq().
To me it seems that the names of some of these functions are perhaps misleading. rcu_read_lock() sort of implies there is some locking going on when there isn't. It might have been easier to understand if rcu_read_lock() was called rcu_get() and rcu_read_unlock() was called rcu_put() to reflect that they are really as much about memory management as synchronization. Too late the change any of that obviously.
Thanks,
Jeff Haran
Quoted 51 lines · Jeff Haran
Quoted 51 lines
-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 11:41 AM
To: Jeff Haran
Cc: Milos Vyletel; Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION
Subject: Re: [PATCH] rcu: smallrcu_dereferencedoc updateQuoted 20 lines · Jeff Haran
Quoted 20 lines
-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 7:07 AM
To: Milos Vyletel
Cc: Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION; Jeff Haran
Subject: Re: [PATCH] rcu: smallrcu_dereferencedoc updateQuoted 5 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.
Quoted 5 lines
Quoted 3 lines
Queued for 4.2.
Thanx, Paul
It's just that the original text suggests repeated
rcu_dereference()calls arediscouraged because they are ugly and not efficient on some architectures. When I read that I concluded that those were the only reasons not to do it, that despite the possible inefficiency it would always return the same pointer. Depending on how one's code is structured, being able to do this could be advantageous. Then I started looking at the code that implements it and I couldn't see how it could possibly be the case. I even wrote a little kernel module to prove to myself that doing this could return different pointer values. If I misinterpreted the original text I figured others might also. Milos even found some code in the kernel where it's author had done this, so it might be a widely held misunderstanding. It's easy for people who have worked with
rwlock_tsto think an RCU read lock works the same.Fair point, and thank you the rationale! Are there any other parts of the RCU documentation that are similarly blind to your initial point of view? If so, it would be good for them to be fixed.
Quoted 7 lines
Quoted 1 line
Thanx, Paul
I can't think of much off the top of my head, but I'm hoping I might get some time to review it again and perhaps provide some more concrete suggestions.
One thing that does come to mind is the article you wrote in LWN, http://lwn.net/Articles/263130/, where you discussed RCU as a reader-write lock replacement. whatisRCU.txt seems to incorporated some of that. Something along the lines of the original section in the LWN article where there was some discussion of the differences between a
rwlock_tread lock critical section and a RCU read lock critical section might be beneficial, a key thing being that with RCU there really is no locking, the value of the pointer can change in a RCU critical section because writers aren't blocked from updating it. Another thing might be some discussion that the cases where you'd callread_lock_bh()are way different than when you'd callrcu_read_lock_bh()and as a corollary why there is norcu_read_lock_irq().To me it seems that the names of some of these functions are perhaps misleading.
rcu_read_lock()sort of implies there is some locking going on when there isn't. It might have been easier to understand ifrcu_read_lock()was calledrcu_get()andrcu_read_unlock()was calledrcu_put()to reflect that they are really as much about memory management as synchronization. Too late the change any of that obviously.
Indeed, proposing names for things that have not yet been used much is always a bit hazardous. But we nevertheless did have to pick a name for it back in the day. That said, RCU is used in different ways, so it is not clear that any given change the the current set of names would be an overall improvement.
Thanx, Paul
Quoted 2 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.
Might as well make it more explicit with an example then. See below:
Quoted 12 lines
Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>--- Documentation/RCU/whatisRCU.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)Documentation/RCU/whatisRCU.txt +0
-0
Documentation/RCU/whatisRCU.txt rcu_dereference() · line 262
Quoted 8 lines
Documentation/RCU/whatisRCU.txt +3
-1@@ -256,7 +256,9 @@ rcu_dereference() If you are going to be fetching multiple fields from the RCU-protected structure, using the local variable is of course preferred. Repeated rcu_dereference() calls look - ugly and incur unnecessary overhead on Alpha CPUs. + ugly, do not guarantee that same pointer will be returned + if update happened while in critical section and incur + unnecessary overhead on Alpha CPUs.
An example like follows:
struct some_ds {
int data;
bool ready;
};struct some_ds *p = ...;
rcu_read_lock();
if (rcu_dereference(p->ready))
data = rcu_dereference(p->data); // bug
rcu_read_unlock();
or some such.
Signature · 1 line
Pranith
Quoted 34 lines · Pranith Kumar
Quoted 2 lines · Milos Vyletel
Make a note stating that repeated calls of
rcu_dereference()may not return the same pointer if update happens while in critical section.Might as well make it more explicit with an example then. See below:
Quoted 12 lines
Reported-by Jeff Haran <[email protected]>Signed-off-by Milos Vyletel <[email protected]>--- Documentation/RCU/whatisRCU.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)Documentation/RCU/whatisRCU.txt +0
-0Documentation/RCU/whatisRCU.txt rcu_dereference() · line 262
Quoted 8 lines
Documentation/RCU/whatisRCU.txt +3
-1@@ -256,7 +256,9 @@ rcu_dereference() If you are going to be fetching multiple fields from the RCU-protected structure, using the local variable is of course preferred. Repeated rcu_dereference() calls look - ugly and incur unnecessary overhead on Alpha CPUs. + ugly, do not guarantee that same pointer will be returned + if update happened while in critical section and incur + unnecessary overhead on Alpha CPUs.An example like follows:
struct some_ds { int data; bool ready; };
Quoted 8 lines
struct
some_ds*p = ...;
rcu_read_lock();
if (rcu_dereference(p->ready))data = rcu_dereference(p->data); // bug
rcu_read_unlock();or some such.
Hello, Pranith!
Sounds like a good thought for a separate patch. Please take a look through the rest of the documentation -- this might well be the right place for such an example, but there might well be a better place. Is this issue mentioned in the checklist? If not, another item might be good.
Thanx, Paul
Quoted 6 lines · Paul E. McKenney
Sounds like a good thought for a separate patch. Please take a look through the rest of the documentation -- this might well be the right place for such an example, but there might well be a better place. Is this issue mentioned in the checklist? If not, another item might be good.
Yup, I will take a look and send a patch for this.
Signature · 1 line
Pranith
Quoted 9 lines · Pranith Kumar
Quoted 5 lines · Paul E. McKenney
Sounds like a good thought for a separate patch. Please take a look through the rest of the documentation -- this might well be the right place for such an example, but there might well be a better place. Is this issue mentioned in the checklist? If not, another item might be good.
Yup, I will take a look and send a patch for this.
Sounds good!
Thanx, Paul