Skip to contentSkip to conversation

rcu: small rcu_dereference doc update

PATCH Superseded Milos Vyletel 5 participants 13 messages Apr 17, 2015 – Apr 21, 2015

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 rcu: small rcu_dereference doc update

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Documentation/RCU/whatisRCU.txt +3 -1
index 88dfce1..82b1b2c 100644
@@ -256,7 +256,9 @@ rcu_dereference()
 	If you are going to be fetching multiple fields from the
 	RCU-protected structure, using the local variable is of
 	course preferred.  Repeated rcu_dereference() calls look
-	ugly and incur unnecessary overhead on Alpha CPUs.
+	ugly, do not guarantee that same pointer will be returned
+	if update happened while in critical section and incur
+	unnecessary overhead on Alpha CPUs.
 
 	Note that the value returned by rcu_dereference() is valid
 	only within the enclosing RCU read-side critical section.
Signature · 1 line
2.1.0
#1 lore raw
Quoted 5 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.

Queued for 4.2.

Thanx, Paul

Quoted 22 lines
---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Documentation/RCU/whatisRCU.txt +3 -1
index 88dfce1..82b1b2c 100644
@@ -256,7 +256,9 @@ rcu_dereference()
 	If you are going to be fetching multiple fields from the
 	RCU-protected structure, using the local variable is of
 	course preferred.  Repeated rcu_dereference() calls look
-	ugly and incur unnecessary overhead on Alpha CPUs.
+	ugly, do not guarantee that same pointer will be returned
+	if update happened while in critical section and incur
+	unnecessary overhead on Alpha CPUs.
 
 	Note that the value returned by rcu_dereference() is valid
 	only within the enclosing RCU read-side critical section.
Signature · 1 line
2.1.0
#2 lore raw
Quoted 4 lines · On Fri, 17 Apr 2015 07:06:38 -0700 "Paul E. McKenney"…
Quoted 3 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Quoted 8 lines
Quoted 2 lines

Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.

Queued for 4.2.

Before you queue it, there's a few articles that are screaming to be present...

Quoted 7 lines

Thanx, Paul

Quoted 4 lines
---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Quoted 12 lines

Documentation/RCU/whatisRCU.txt rcu_dereference() · line 261

Quoted 12 lines
Documentation/RCU/whatisRCU.txt +3 -1
index 88dfce1..82b1b2c 100644
@@ -256,7 +256,9 @@ rcu_dereference()
 	If you are going to be fetching multiple fields from the
 	RCU-protected structure, using the local variable is of
 	course preferred.  Repeated rcu_dereference() calls look
-	ugly and incur unnecessary overhead on Alpha CPUs.
+	ugly, do not guarantee that same pointer will be returned
+	if update happened while in critical section and incur
+	unnecessary overhead on Alpha CPUs.

ugly, do not guarantee that the same pointer will be returned if an update happened while in the critical section, and incur unnecessary overhead on Alpha CPUs.

-- Steve

Quoted 6 lines
Quoted 6 lines
Documentation/RCU/whatisRCU.txt +0 -0
 	Note that the value returned by rcu_dereference() is valid
 	only within the enclosing RCU read-side critical section.
Signature · 1 line
2.1.0
#3 lore raw
Quoted 10 lines · Steven Rostedt
Quoted 7 lines · On Fri, 17 Apr 2015 07:06:38 -0700 "Paul E. McKenney"…
Quoted 5 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Quoted 8 lines
Quoted 5 lines

Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.

Queued for 4.2.

Before you queue it, there's a few articles that are screaming to be present...

OK, I removed it. Milos, when you send me a version that Steve is happy with, I will requeue it.

Thanx, Paul

Quoted 30 lines
Quoted 4 lines
Quoted 4 lines
---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Quoted 12 lines

Documentation/RCU/whatisRCU.txt rcu_dereference() · line 261

Quoted 12 lines
Documentation/RCU/whatisRCU.txt +3 -1
index 88dfce1..82b1b2c 100644
@@ -256,7 +256,9 @@ rcu_dereference()
 	If you are going to be fetching multiple fields from the
 	RCU-protected structure, using the local variable is of
 	course preferred.  Repeated rcu_dereference() calls look
-	ugly and incur unnecessary overhead on Alpha CPUs.
+	ugly, do not guarantee that same pointer will be returned
+	if update happened while in critical section and incur
+	unnecessary overhead on Alpha CPUs.

ugly, do not guarantee that the same pointer will be returned if an update happened while in the critical section, and incur unnecessary overhead on Alpha CPUs.

-- Steve

Quoted 6 lines
Quoted 6 lines
Documentation/RCU/whatisRCU.txt +0 -0
 	Note that the value returned by rcu_dereference() is valid
 	only within the enclosing RCU read-side critical section.
Signature · 1 line
2.1.0
#4 lore raw
In reply to Steven Rostedt
Quoted 26 lines · Steven Rostedt
Quoted 4 lines · On Fri, 17 Apr 2015 07:06:38 -0700 "Paul E. McKenney"…
Quoted 3 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Quoted 8 lines
Quoted 2 lines

Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.

Queued for 4.2.

Before you queue it, there's a few articles that are screaming to be present...

Quoted 7 lines

Thanx, Paul

Quoted 4 lines
---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Quoted 17 lines
Quoted 4 lines
Quoted 4 lines
Documentation/RCU/whatisRCU.txt +0 -0
index 88dfce1..82b1b2c 100644
Quoted 8 lines

Documentation/RCU/whatisRCU.txt rcu_dereference() · line 261

Quoted 8 lines
Documentation/RCU/whatisRCU.txt +3 -1
@@ -256,7 +256,9 @@ rcu_dereference()
 	If you are going to be fetching multiple fields from the
 	RCU-protected structure, using the local variable is of
 	course preferred.  Repeated rcu_dereference() calls look
-	ugly and incur unnecessary overhead on Alpha CPUs.
+	ugly, do not guarantee that same pointer will be returned
+	if update happened while in critical section and incur
+	unnecessary overhead on Alpha CPUs.

ugly, do not guarantee that the same pointer will be returned if an update happened while in the critical section, and incur unnecessary overhead on Alpha CPUs.

Thanks. Corrected. Sending v2 in a bit.

Milos

Quoted 9 lines

-- Steve

Quoted 6 lines
Quoted 6 lines
Documentation/RCU/whatisRCU.txt +0 -0
 	Note that the value returned by rcu_dereference() is valid
 	only within the enclosing RCU read-side critical section.
Signature · 1 line
2.1.0
#5 lore raw
In reply to Paul E. McKenney
Quoted 16 lines

-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 7:07 AM
To: Milos Vyletel
Cc: Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION; Jeff Haran
Subject: Re: [PATCH] rcu: small rcu_dereference doc update

Quoted 5 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Quoted 7 lines

Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.

Queued for 4.2.

Thanx, Paul

It's just that the original text suggests repeated rcu_dereference() calls are discouraged because they are ugly and not efficient on some architectures. When I read that I concluded that those were the only reasons not to do it, that despite the possible inefficiency it would always return the same pointer. Depending on how one's code is structured, being able to do this could be advantageous. Then I started looking at the code that implements it and I couldn't see how it could possibly be the case. I even wrote a little kernel module to prove to myself that doing this could return different pointer values. If I misinterpreted the original text I figured others might also. Milos even found some code in the kernel where it's author had done this, so it might be a widely held misunderstanding. It's easy for people who have worked with rwlock_ts to think an RCU read lock works the same.

Thanks,

Jeff Haran

Quoted 22 lines
Quoted 22 lines
---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/Documentation/RCU/whatisRCU.txt index 88dfce1..82b1b2c 100644

Documentation/RCU/whatisRCU.txt +3 -2
@@ -256,7 +256,9 @@ rcu_dereference()
 	If you are going to be fetching multiple fields from the
 	RCU-protected structure, using the local variable is of
 	course preferred.  Repeated rcu_dereference() calls look
-	ugly and incur unnecessary overhead on Alpha CPUs.
+	ugly, do not guarantee that same pointer will be returned
+	if update happened while in critical section and incur
+	unnecessary overhead on Alpha CPUs.
 
 	Note that the value returned by rcu_dereference() is valid
 	only within the enclosing RCU read-side critical section.
--

2.1.0

#6 lore raw
Quoted 20 lines · Jeff Haran
Quoted 20 lines

-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 7:07 AM
To: Milos Vyletel
Cc: Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION; Jeff Haran
Subject: Re: [PATCH] rcu: small rcu_dereference doc update

Quoted 5 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.

Quoted 5 lines
Quoted 3 lines

Queued for 4.2.

Thanx, Paul

It's just that the original text suggests repeated rcu_dereference() calls are discouraged because they are ugly and not efficient on some architectures. When I read that I concluded that those were the only reasons not to do it, that despite the possible inefficiency it would always return the same pointer. Depending on how one's code is structured, being able to do this could be advantageous. Then I started looking at the code that implements it and I couldn't see how it could possibly be the case. I even wrote a little kernel module to prove to myself that doing this could return different pointer values. If I misinterpreted the original text I figured others might also. Milos even found some code in the kernel where it's author had done this, so it might be a widely held misunderstanding. It's easy for people who have worked with rwlock_ts to think an RCU read lock works the same.

Fair point, and thank you the rationale! Are there any other parts of the RCU documentation that are similarly blind to your initial point of view? If so, it would be good for them to be fixed.

Thanx, Paul

Quoted 27 lines

Thanks,

Jeff Haran

Quoted 22 lines
Quoted 22 lines
---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/Documentation/RCU/whatisRCU.txt index 88dfce1..82b1b2c 100644

Documentation/RCU/whatisRCU.txt +3 -2
@@ -256,7 +256,9 @@ rcu_dereference()
 	If you are going to be fetching multiple fields from the
 	RCU-protected structure, using the local variable is of
 	course preferred.  Repeated rcu_dereference() calls look
-	ugly and incur unnecessary overhead on Alpha CPUs.
+	ugly, do not guarantee that same pointer will be returned
+	if update happened while in critical section and incur
+	unnecessary overhead on Alpha CPUs.
 
 	Note that the value returned by rcu_dereference() is valid
 	only within the enclosing RCU read-side critical section.
--

2.1.0

#7 lore raw
Quoted 47 lines

-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 11:41 AM
To: Jeff Haran
Cc: Milos Vyletel; Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION
Subject: Re: [PATCH] rcu: small rcu_dereference doc update

Quoted 20 lines · Jeff Haran
Quoted 20 lines

-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 7:07 AM
To: Milos Vyletel
Cc: Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION; Jeff Haran
Subject: Re: [PATCH] rcu: small rcu_dereference doc update

Quoted 5 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.

Quoted 5 lines
Quoted 3 lines

Queued for 4.2.

Thanx, Paul

It's just that the original text suggests repeated rcu_dereference() calls are

discouraged because they are ugly and not efficient on some architectures. When I read that I concluded that those were the only reasons not to do it, that despite the possible inefficiency it would always return the same pointer. Depending on how one's code is structured, being able to do this could be advantageous. Then I started looking at the code that implements it and I couldn't see how it could possibly be the case. I even wrote a little kernel module to prove to myself that doing this could return different pointer values. If I misinterpreted the original text I figured others might also. Milos even found some code in the kernel where it's author had done this, so it might be a widely held misunderstanding. It's easy for people who have worked with rwlock_ts to think an RCU read lock works the same.

Quoted 5 lines

Fair point, and thank you the rationale! Are there any other parts of the RCU documentation that are similarly blind to your initial point of view? If so, it would be good for them to be fixed.

Thanx, Paul

I can't think of much off the top of my head, but I'm hoping I might get some time to review it again and perhaps provide some more concrete suggestions.

One thing that does come to mind is the article you wrote in LWN, http://lwn.net/Articles/263130/, where you discussed RCU as a reader-write lock replacement. whatisRCU.txt seems to incorporated some of that. Something along the lines of the original section in the LWN article where there was some discussion of the differences between a rwlock_t read lock critical section and a RCU read lock critical section might be beneficial, a key thing being that with RCU there really is no locking, the value of the pointer can change in a RCU critical section because writers aren't blocked from updating it. Another thing might be some discussion that the cases where you'd call read_lock_bh() are way different than when you'd call rcu_read_lock_bh() and as a corollary why there is no rcu_read_lock_irq().

To me it seems that the names of some of these functions are perhaps misleading. rcu_read_lock() sort of implies there is some locking going on when there isn't. It might have been easier to understand if rcu_read_lock() was called rcu_get() and rcu_read_unlock() was called rcu_put() to reflect that they are really as much about memory management as synchronization. Too late the change any of that obviously.

Thanks,

Jeff Haran

#8 lore raw
Quoted 51 lines · Jeff Haran
Quoted 51 lines

-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 11:41 AM
To: Jeff Haran
Cc: Milos Vyletel; Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION
Subject: Re: [PATCH] rcu: small rcu_dereference doc update

Quoted 20 lines · Jeff Haran
Quoted 20 lines

-----Original Message-----
From: Paul E. McKenney [mailto:[email protected]]
Sent: Friday, April 17, 2015 7:07 AM
To: Milos Vyletel
Cc: Josh Triplett; Steven Rostedt; Mathieu Desnoyers; Lai Jiangshan; Jonathan Corbet; open list:READ-COPY UPDATE...; open list:DOCUMENTATION; Jeff Haran
Subject: Re: [PATCH] rcu: small rcu_dereference doc update

Quoted 5 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Hmmm... Seems like that should be obvious, but on the other hand, I have been using RCU for more than twenty years, so my obviousness sensors might need recalibration.

Quoted 5 lines
Quoted 3 lines

Queued for 4.2.

Thanx, Paul

It's just that the original text suggests repeated rcu_dereference() calls are

discouraged because they are ugly and not efficient on some architectures. When I read that I concluded that those were the only reasons not to do it, that despite the possible inefficiency it would always return the same pointer. Depending on how one's code is structured, being able to do this could be advantageous. Then I started looking at the code that implements it and I couldn't see how it could possibly be the case. I even wrote a little kernel module to prove to myself that doing this could return different pointer values. If I misinterpreted the original text I figured others might also. Milos even found some code in the kernel where it's author had done this, so it might be a widely held misunderstanding. It's easy for people who have worked with rwlock_ts to think an RCU read lock works the same.

Fair point, and thank you the rationale! Are there any other parts of the RCU documentation that are similarly blind to your initial point of view? If so, it would be good for them to be fixed.

Quoted 7 lines
Quoted 1 line

Thanx, Paul

I can't think of much off the top of my head, but I'm hoping I might get some time to review it again and perhaps provide some more concrete suggestions.

One thing that does come to mind is the article you wrote in LWN, http://lwn.net/Articles/263130/, where you discussed RCU as a reader-write lock replacement. whatisRCU.txt seems to incorporated some of that. Something along the lines of the original section in the LWN article where there was some discussion of the differences between a rwlock_t read lock critical section and a RCU read lock critical section might be beneficial, a key thing being that with RCU there really is no locking, the value of the pointer can change in a RCU critical section because writers aren't blocked from updating it. Another thing might be some discussion that the cases where you'd call read_lock_bh() are way different than when you'd call rcu_read_lock_bh() and as a corollary why there is no rcu_read_lock_irq().

To me it seems that the names of some of these functions are perhaps misleading. rcu_read_lock() sort of implies there is some locking going on when there isn't. It might have been easier to understand if rcu_read_lock() was called rcu_get() and rcu_read_unlock() was called rcu_put() to reflect that they are really as much about memory management as synchronization. Too late the change any of that obviously.

Indeed, proposing names for things that have not yet been used much is always a bit hazardous. But we nevertheless did have to pick a name for it back in the day. That said, RCU is used in different ways, so it is not clear that any given change the the current set of names would be an overall improvement.

Thanx, Paul

#9 lore raw
In reply to Milos Vyletel
Quoted 2 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Might as well make it more explicit with an example then. See below:

Quoted 12 lines
---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Documentation/RCU/whatisRCU.txt +0 -0
index 88dfce1..82b1b2c 100644

Documentation/RCU/whatisRCU.txt rcu_dereference() · line 262

Quoted 8 lines
Documentation/RCU/whatisRCU.txt +3 -1
@@ -256,7 +256,9 @@ rcu_dereference()
        If you are going to be fetching multiple fields from the
        RCU-protected structure, using the local variable is of
        course preferred.  Repeated rcu_dereference() calls look
-       ugly and incur unnecessary overhead on Alpha CPUs.
+       ugly, do not guarantee that same pointer will be returned
+       if update happened while in critical section and incur
+       unnecessary overhead on Alpha CPUs.
 

An example like follows:

struct some_ds {
        int data;
        bool ready;
};

struct some_ds *p = ...;

rcu_read_lock();
if (rcu_dereference(p->ready))

    data = rcu_dereference(p->data); // bug

rcu_read_unlock();

or some such.

Signature · 1 line
Pranith
#10 lore raw
Quoted 34 lines · Pranith Kumar
Quoted 2 lines · Milos Vyletel

Make a note stating that repeated calls of rcu_dereference() may not return the same pointer if update happens while in critical section.

Might as well make it more explicit with an example then. See below:

Quoted 12 lines
---
 Documentation/RCU/whatisRCU.txt | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Documentation/RCU/whatisRCU.txt +0 -0
index 88dfce1..82b1b2c 100644

Documentation/RCU/whatisRCU.txt rcu_dereference() · line 262

Quoted 8 lines
Documentation/RCU/whatisRCU.txt +3 -1
@@ -256,7 +256,9 @@ rcu_dereference()
        If you are going to be fetching multiple fields from the
        RCU-protected structure, using the local variable is of
        course preferred.  Repeated rcu_dereference() calls look
-       ugly and incur unnecessary overhead on Alpha CPUs.
+       ugly, do not guarantee that same pointer will be returned
+       if update happened while in critical section and incur
+       unnecessary overhead on Alpha CPUs.
 

An example like follows:

struct some_ds {
        int data;
        bool ready;
};
Quoted 8 lines

struct some_ds *p = ...;

rcu_read_lock();
if (rcu_dereference(p->ready))

    data = rcu_dereference(p->data); // bug

rcu_read_unlock();

or some such.

Hello, Pranith!

Sounds like a good thought for a separate patch. Please take a look through the rest of the documentation -- this might well be the right place for such an example, but there might well be a better place. Is this issue mentioned in the checklist? If not, another item might be good.

Thanx, Paul

#11 lore raw
Quoted 6 lines · Paul E. McKenney

Sounds like a good thought for a separate patch. Please take a look through the rest of the documentation -- this might well be the right place for such an example, but there might well be a better place. Is this issue mentioned in the checklist? If not, another item might be good.

Yup, I will take a look and send a patch for this.

Signature · 1 line
Pranith
#12 lore raw
Quoted 9 lines · Pranith Kumar
Quoted 5 lines · Paul E. McKenney

Sounds like a good thought for a separate patch. Please take a look through the rest of the documentation -- this might well be the right place for such an example, but there might well be a better place. Is this issue mentioned in the checklist? If not, another item might be good.

Yup, I will take a look and send a patch for this.

Sounds good!

Thanx, Paul