"volatile considered harmful", take 3
Jonathan Corbet
maintainerPATCH 1/1 "volatile considered harmful", take 3
Here's another version of the volatile document. Once again, I've tried to address all of the comments. There haven't really been any recent comments addressing the correctness of the document; people have been more concerned with how it's expressed. I'm glad to see files in Documentation/ held to a high standard of writing, but, unless somebody has a factual issue this time around I would like to declare Mission Accomplished and move on.
Thanks,
jon
---
Encourage developers to avoid the volatile type class in kernel code.
Documentation/volatile-considered-harmful.txt new file +119 -0
@@ -0,0 +1,119 @@ +Why the "volatile" type class should not be used +------------------------------------------------ + +C programmers have often taken volatile to mean that the variable could be +changed outside of the current thread of execution; as a result, they are +sometimes tempted to use it in kernel code when shared data structures are +being used. In other words, they have been known to treat volatile types +as a sort of easy atomic variable, which they are not. The use of volatile in +kernel code is almost never correct; this document describes why. + +The key point to understand with regard to volatile is that its purpose is +to suppress optimization, which is almost never what one really wants to +do. In the kernel, one must protect shared data structures against +unwanted concurrent access, which is very much a different task. The +process of protecting against unwanted concurrency will also avoid almost +all optimization-related problems in a more efficient way. + +Like volatile, the kernel primitives which make concurrent access to data +safe (spinlocks, mutexes, memory barriers, etc.) are designed to prevent +unwanted optimization. If they are being used properly, there will be no +need to use volatile as well. If volatile is still necessary, there is +almost certainly a bug in the code somewhere. In properly-written kernel +code, volatile can only serve to slow things down. + +Consider a typical block of kernel code: + + spin_lock(&the_lock); + do_something_on(&shared_data); + do_something_else_with(&shared_data); + spin_unlock(&the_lock); + +If all the code follows the locking rules, the value of shared_data cannot +change unexpectedly while the_lock is held. Any other code which might +want to play with that data will be waiting on the lock. The spinlock +primitives act as memory barriers - they are explicitly written to do so - +meaning that data accesses will not be optimized across them. So the +compiler might think it knows what will be in shared_data, but the +spin_lock() call, since it acts as a memory barrier, will force it to +forget anything it knows. There will be no optimization problems with +accesses to that data. + +If shared_data were declared volatile, the locking would still be +necessary. But the compiler would also be prevented from optimizing access +to shared_data _within_ the critical section, when we know that nobody else +can be working with it. While the lock is held, shared_data is not +volatile. When dealing with shared data, proper locking makes volatile +unnecessary - and potentially harmful. + +The volatile storage class was originally meant for memory-mapped I/O +registers. Within the kernel, register accesses, too, should be protected +by locks, but one also does not want the compiler "optimizing" register +accesses within a critical section. But, within the kernel, I/O memory +accesses are always done through accessor functions; accessing I/O memory +directly through pointers is frowned upon and does not work on all +architectures. Those accessors are written to prevent unwanted +optimization, so, once again, volatile is unnecessary. + +Another situation where one might be tempted to use volatile is +when the processor is busy-waiting on the value of a variable. The right +way to perform a busy wait is: + + while (my_variable != what_i_want) + cpu_relax(); + +The cpu_relax() call can lower CPU power consumption or yield to a +hyperthreaded twin processor; it also happens to serve as a memory barrier, +so, once again, volatile is unnecessary. Of course, busy-waiting is +generally an anti-social act to begin with. + +There are still a few rare situations where volatile makes sense in the +kernel: + + - The above-mentioned accessor functions might use volatile on + architectures where direct I/O memory access does work. Essentially, + each accessor call becomes a little critical section on its own and + ensures that the access happens as expected by the programmer. + + - Inline assembly code which changes memory, but which has no other + visible side effects, risks being deleted by GCC. Adding the volatile + keyword to asm statements will prevent this removal. + + - The jiffies variable is special in that it can have a different value + every time it is referenced, but it can be read without any special + locking. So jiffies can be volatile, but the addition of other + variables of this type is strongly frowned upon. Jiffies is considered + to be a "stupid legacy" issue (Linus's words) in this regard; fixing it + would be more trouble than it is worth. + + - Pointers to data structures in coherent memory which might be modified + by I/O devices can, sometimes, legitimately be volatile. A ring buffer + used by a network adapter, where that adapter changes pointers to + indicate which descriptors have been processed, is an example of this + type of situation. + +For most code, none of the above justifications for volatile apply. As a +result, the use of volatile is likely to be seen as a bug and will bring +additional scrutiny to the code. Developers who are tempted to use +volatile should take a step back and think about what they are truly trying +to accomplish. + +Patches to remove volatile variables are generally welcome - as long as +they come with a justification which shows that the concurrency issues have +been properly thought through. + + +NOTES +----- + +[1] http://lwn.net/Articles/233481/ +[2] http://lwn.net/Articles/233482/ + +CREDITS +------- + +Original impetus and research by Randy Dunlap +Written by Jonathan Corbet +Improvements via coments from Satyam Sharma, Johannes Stezenbach, Jesper + Juhl, Heikki Orsila, H. Peter Anvin, Philipp Hahn, and Stefan + Richter.
Quoted 10 lines · Jonathan Corbet
Here's another version of the volatile document. Once again, I've tried to address all of the comments. There haven't really been any recent comments addressing the correctness of the document; people have been more concerned with how it's expressed. I'm glad to see files in
Documentation/held to a high standard of writing, but, unless somebody has a factual issue this time around I would like to declare Mission Accomplished and move on.Thanks,
Quoted 8 lines
jon
---
Encourage developers to avoid the volatile type class in kernel code.
Signed-off-by Jonathan Corbet <[email protected]>
Looks good to me.
Signature · 3 lines
Jesper Juhl <[email protected]> Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html Plain text mails only, please http://www.expita.com/nomime.html
Quoted 7 lines · Jonathan Corbet
Here's another version of the volatile document. Once again, I've tried to address all of the comments. There haven't really been any recent comments addressing the correctness of the document; people have been more concerned with how it's expressed. I'm glad to see files in
Documentation/held to a high standard of writing, but, unless somebody has a factual issue this time around I would like to declare Mission Accomplished and move on.
The document looks good, but whether:
Documentation/volatile-considered-harmful.txt line 93
Quoted 5 lines
Documentation/volatile-considered-harmful.txt new file +5
-0+ - Pointers to data structures in coherent memory which might be modified + by I/O devices can, sometimes, legitimately be volatile. A ring buffer + used by a network adapter, where that adapter changes pointers to + indicate which descriptors have been processed, is an example of this + type of situation.
is a legitimate use case for volatile is still not clear to me (I agree with Alan's
comment in a previous thread that this seems to be a case where a memory barrier would be applicable^Wbetter, actually). I could be wrong here, so would be nice if Peter explains why volatile is legitimate here.
Otherwise, it's fine with me.
Thanks,
Satyam
Quoted 11 lines · Satyam Sharma
Quoted 8 lines · Jonathan Corbet
+ - Pointers to data structures in coherent memory which might be modified
+ by I/O devices can, sometimes, legitimately be volatile. A ring
buffer(quoted diff) +2
-0+ used by a network adapter, where that adapter changes pointers to + indicate which descriptors have been processed, is an example ofthis
+ type of situation.is a legitimate use case for volatile is still not clear to me (I
IMO it is not. We do /not/ want to encourage volatile use in those cases, and indeed, it's not necessary even if you can rationalize the use of the English word "volatile" to describe the situation.
Drivers work quite well without volatile in such situations.
Jeff
Quoted 10 lines · Satyam Sharma
Quoted 9 lines
+ - Pointers to data structures in coherent memory which might be modified
+ by I/O devices can, sometimes, legitimately be volatile. A ring
buffer(quoted diff) +2
-0+ used by a network adapter, where that adapter changes pointers to + indicate which descriptors have been processed, is an example ofthis
+ type of situation.
Quoted 8 lines
is a legitimate use case for volatile is still not clear to me (I agree with Alan's
comment in a previous thread that this seems to be a case where a memory barrier would be applicable^Wbetter, actually). I could be wrong here, so would be nice if Peter explains why volatile is legitimate here.Otherwise, it's fine with me.
I don't see why Alan's way is necessarily better; it should work but is more heavy-handed as it's disabling *all* optimization such as loop invariants across the barrier.
-hpa
Quoted 5 lines · H. Peter Anvin
I don't see why Alan's way is necessarily better; it should work but is more heavy-handed as it's disabling *all* optimization such as loop invariants across the barrier.
To expand on this further: the way this probably *should* be handled, Linux-style, is with internally-volatile versions of le32_to_cpup() and friends. That obeys the concept that the volatility should be associated with an operation, not a data structure, and, being related to an I/O device, should have its endianness explicitly declared.
Right now those macros don't exist, however.
-hpa
Quoted 17 lines · H. Peter Anvin
Quoted 16 lines · Satyam Sharma
Quoted 9 lines
+ - Pointers to data structures in coherent memory which might be modified
+ by I/O devices can, sometimes, legitimately be volatile. A ring
buffer(quoted diff) +2
-0+ used by a network adapter, where that adapter changes pointers to + indicate which descriptors have been processed, is an example ofthis
+ type of situation.is a legitimate use case for volatile is still not clear to me (I agree with Alan's
comment in a previous thread that this seems to be a case where a memory barrier would be applicable^Wbetter, actually). I could be wrong here, so would be nice if Peter explains why volatile is legitimate here.
Quoted 4 lines
Quoted 2 lines
Otherwise, it's fine with me.
I don't see why Alan's way is necessarily better;
Because volatile is ill-defined? Or actually, *undefined* (well, implementation-defined is as good as that)? It's *so* _vague_, one doesn't _feel_ like using it at all!
We already have a complete API containing optimization barriers, load/store/full memory barriers. With well-defined and well-understood semantics. Just ... _why_ use volatile?
Quoted 1 line
it should work but is
It will _always_ work. In fact you can't really say the same for volatile. We already assume the compiler _actually_ took some pains to stuff meaning into C's (lack of) definition of volatile and implement it -- but in what sense, nobody knows (the C standard doesn't, so what are we).
Quoted 2 lines
more heavy-handed as it's disabling *all* optimization such as loop invariants across the barrier.
This is a legitimate criticism, I agree.
Thanks,
Satyam
Quoted 5 lines · Satyam Sharma
Because volatile is ill-defined? Or actually, *undefined* (well, implementation-defined is as good as that)? It's *so* _vague_, one doesn't _feel_ like using it at all!
Sorry, that's just utter crap. Linux isn't written in some mythical C which only exists in standard document, it is written in a particular subset of GNU C. "volatile" is well enough defined in that context, it is just frequently misused.
Quoted 3 lines
We already have a complete API containing optimization barriers, load/store/full memory barriers. With well-defined and well-understood semantics. Just ... _why_ use volatile?
See below.
Quoted 5 lines
It will _always_ work. In fact you can't really say the same for volatile. We already assume the compiler _actually_ took some pains to stuff meaning into C's (lack of) definition of volatile and implement it -- but in what sense, nobody knows (the C standard doesn't, so what are we).
It will always work within the context of GNU C.
Quoted 4 lines
Quoted 2 lines
more heavy-handed as it's disabling *all* optimization such as loop invariants across the barrier.
This is a legitimate criticism, I agree.
There you have it.
-hpa
Quoted 11 lines · H. Peter Anvin
Quoted 5 lines · Satyam Sharma
Because volatile is ill-defined? Or actually, *undefined* (well, implementation-defined is as good as that)? It's *so* _vague_, one doesn't _feel_ like using it at all!
Sorry, that's just utter crap. Linux isn't written in some mythical C which only exists in standard document, it is written in a particular subset of GNU C. "volatile" is well enough defined in that context, it is just frequently misused.
Of course, volatile _is_ defined (well, _anything_ that is implemented _is_ defined, after all) in the context of GNU C, and if you're saying that the kernel (and all its subsystems) is and should _continue_ to be (the purpose of this document) written within that context, then that's your opinion and I would not disagree with you. If you do prefer to continue using that dialect, then I wouldn't stop you either.
Personally, I'd prefer writing in a slightly more portable / larger context (using well-defined and understood APIs), thank you, and hope you'd allow me to do so myself.
Coming back to the document, we do need to document / find consensus on the "preferred" way to do similar business in the kernel, and my opinion as far as that is concerned is to shun volatile wherever possible (which includes the case originally discussed above).
Satyam
Quoted 6 lines · Satyam Sharma
Quoted 5 lines
Sorry, that's just utter crap. Linux isn't written in some mythical C which only exists in standard document, it is written in a particular subset of GNU C. "volatile" is well enough defined in that context, it is just frequently misused.
Quoted 7 lines
Of course, volatile _is_ defined (well, _anything_ that is implemented _is_ defined, after all) in the context of GNU C, and if you're saying that the kernel (and all its subsystems) is and should _continue_ to be (the purpose of this document) written within that context, then that's your opinion and I would not disagree with you. If you do prefer to continue using that dialect, then I wouldn't stop you either.
This isn't just an opinion, this is the language the Linux kernel is written in today, and has been for the duration of its 16-year existence. It contains *many* constructs that are not defined in, for example, C99, and it would in fact be impossible to write the Linux kernel using only C99-compliant constructs.
Quoted 3 lines
Personally, I'd prefer writing in a slightly more portable / larger context (using well-defined and understood APIs), thank you, and hope you'd allow me to do so myself.
There is no such "slightly more portable/larger context/well-defined and understood" context in existence. If you think so, you're deluding yourself.
-hpa
Quoted 15 lines · H. Peter Anvin
Quoted 6 lines · Satyam Sharma
Quoted 5 lines
Sorry, that's just utter crap. Linux isn't written in some mythical C which only exists in standard document, it is written in a particular subset of GNU C. "volatile" is well enough defined in that context, it is just frequently misused.
Quoted 7 lines
Of course, volatile _is_ defined (well, _anything_ that is implemented _is_ defined, after all) in the context of GNU C, and if you're saying that the kernel (and all its subsystems) is and should _continue_ to be (the purpose of this document) written within that context, then that's your opinion and I would not disagree with you. If you do prefer to continue using that dialect, then I wouldn't stop you either.
Quoted 2 lines
This isn't just an opinion, this is the language the Linux kernel is written in today, and has been for the duration of its 16-year
I said "and should _continue_ to be (the purpose of this document) written within that context, then that's your opinion". That "is" before the "and should ..." was not meant to be combined with "your opinion".
Quoted 3 lines
existence. It contains *many* constructs that are not defined in, for example, C99, and it would in fact be impossible to write the Linux kernel using only C99-compliant constructs.
And that is not what I attempt to do (or even suggest) either. Which is why I do *not* disagree with the jiffies clause -- it exists in that document merely to _inform_ readers about _current_ behaviour / dialect of the kernel and not to influence code written in the future. The problem with the last clause is that it explicitly influences *future* code (by coming in as a "rare situation where volatile makes sense in the kernel").
Quoted 7 lines
Quoted 3 lines
Personally, I'd prefer writing in a slightly more portable / larger context (using well-defined and understood APIs), thank you, and hope you'd allow me to do so myself.
There is no such "slightly more portable/larger context/well-defined and understood" context in existence. If you think so, you're deluding yourself.
If you remember my original mail was in the "_why_ use volatile" tone, which means to _dissuade_ future spurious additions of volatile to the kernel. Now this _is_ a matter of opinion: I personally prefer as much (if not all) future code to be dialect-independent, you might not.
Satyam
H. Peter Anvin wrote:
[slightly off topic: GCCisms in Linux kernel]
Quoted 3 lines
It contains *many* constructs that are not defined in, for example, C99, and it would in fact be impossible to write the Linux kernel using only C99-compliant constructs.
True. On the other hand, it is possible to keep large parts of the kernel independent of compiler implementation details. And it is not only possible but also beneficial, e.g. because the compiler's implementation changes over time.
Signature · 3 lines
Stefan Richter -=====-=-=== -=-= -==-- http://arcgraph.de/sr/
Quoted 10 lines · Stefan Richter
H. Peter Anvin wrote:
[slightly off topic: GCCisms in Linux kernel]Quoted 3 lines
It contains *many* constructs that are not defined in, for example, C99, and it would in fact be impossible to write the Linux kernel using only C99-compliant constructs.
True. On the other hand, it is possible to keep large parts of the kernel independent of compiler implementation details. And it is not only possible but also beneficial, e.g. because the compiler's implementation changes over time.
I think the most important reason for portable code is that new readers are more familiar with effects of the code.
Signature · 3 lines
Heikki Orsila Barbie's law: [email protected] "Math is hard, let's go shopping!" http://www.iki.fi/shd
Quoted 10 lines · Stefan Richter
H. Peter Anvin wrote:
[slightly off topic: GCCisms in Linux kernel]Quoted 3 lines
It contains *many* constructs that are not defined in, for example, C99, and it would in fact be impossible to write the Linux kernel using only C99-compliant constructs.
True. On the other hand, it is possible to keep large parts of the kernel independent of compiler implementation details. And it is not only possible but also beneficial, e.g. because the compiler's implementation changes over time.
It is, but this is not likely to be one of those things.
Either way, I fully agree with the following (from Jeff):
Quoted 7 lines
Quoted 4 lines · jimmy bahuleyan
i believe, the doc here is pretty unambiguous regarding the fact that volatile should be avoided. And as Stefan pointed out, anyone who feels the need to use, must surely _know_ what he is doing & hence is in a position t make that decision
Honestly, the above quoted paragraph states the situation better than any long, complicated document.
-hpa
Quoted 5 lines · Satyam Sharma
Coming back to the document, we do need to document / find consensus on the "preferred" way to do similar business in the kernel, and my opinion as far as that is concerned is to shun volatile wherever possible (which includes the case originally discussed above).
I too recommend that volatile-considered-harmful.txt is not watered down by an ever growing "but if" list. If anybody knows what he does, he still can program in a deviating way --- provided that he leaves a brief comment in the code, telling why it is possible and beneficial to use the volatile qualifier in this special case.
Signature · 3 lines
Stefan Richter -=====-=-=== -=-= -==-- http://arcgraph.de/sr/
Quoted 7 lines · Stefan Richter
Quoted 5 lines · Satyam Sharma
Coming back to the document, we do need to document / find consensus on the "preferred" way to do similar business in the kernel, and my opinion as far as that is concerned is to shun volatile wherever possible (which includes the case originally discussed above).
Quoted 5 lines
I too recommend that volatile-considered-harmful.txt is not watered down by an ever growing "but if" list. If anybody knows what he does, he still can program in a deviating way --- provided that he leaves a brief comment in the code, telling why it is possible and beneficial to use the volatile qualifier in this special case.
yes, this seems the better option. generally, the more complex rules you have, the more people tend to break it (either through not being able t comprehend it or cos it's too difficult to follow).
i believe, the doc here is pretty unambiguous regarding the fact that volatile should be avoided. And as Stefan pointed out, anyone who feels the need to use, must surely _know_ what he is doing & hence is in a position t make that decision (followed ofcourse with a doc of why it was done).
it would be better we didn't grow the list of exceptions - IMHO.
-jb
Signature · 1 line
Tact is the art of making a point without making an enemy.
Quoted 4 lines · jimmy bahuleyan
i believe, the doc here is pretty unambiguous regarding the fact that volatile should be avoided. And as Stefan pointed out, anyone who feels the need to use, must surely _know_ what he is doing & hence is in a position t make that decision
Honestly, the above quoted paragraph states the situation better than any long, complicated document.
Jeff
Quoted 11 lines · * H. Peter Anvin ([email protected]) wrote:
Quoted 5 lines · Satyam Sharma
Because volatile is ill-defined? Or actually, *undefined* (well, implementation-defined is as good as that)? It's *so* _vague_, one doesn't _feel_ like using it at all!
Sorry, that's just utter crap. Linux isn't written in some mythical C which only exists in standard document, it is written in a particular subset of GNU C. "volatile" is well enough defined in that context, it is just frequently misused.
Where? I don't ever recall seeing something that defines Gcc's behaviour with volatile on different architectures.
I know on some architectures gcc generates different instructions for volatile accesses (e.g. load acquire/store release on IA64); I'd be pleasently surprised if gcc's behaviour was consistent accross architectures.
Dave
Signature · 4 lines
-----Open up your eyes, open up your mind, open up your code ------- / Dr. David Alan Gilbert | Running GNU/Linux on Alpha,68K| Happy \ \ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex / \ _________________________|_____ http://www.treblig.org |_______/