RCU changes for v3.4
Subsystems (12)
Linus,
Please pull the latest core-rcu-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-rcu-for-linus
HEAD: bdd4431c8d071491a68a65d9457996f222b5ecd3 Merge branch 'rcu/next' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu
Thanks,
Ingo
------------------>
Frederic Weisbecker (1):
rcu: Improve synchronize_rcu() diagnostics
Heiko Carstens (1):
rcu: Convert WARN_ON_ONCE() in rcu_lock_acquire() to lockdep
Hugh Dickins (1):
rcu: Stop spurious warnings from synchronize_sched_expedited
Julia Lawall (1):
PTR_ERR should be called before its argument is cleared.
Paul E. McKenney (44):
rcu: Bring RTFP.txt up to date.
rcu: Add lockdep-RCU checks for simple self-deadlock
rcu: Add diagnostic for misaligned rcu_head structures
rcu: Avoid waking up CPUs having only kfree_rcu() callbacks
rcu: Move RCU_TRACE to lib/Kconfig.debug
s390: Convert call_rcu() to kfree_rcu(), drop ext_int_hash_update()
tcm_fc: Convert call_rcu() to kfree_rcu(), drop ft_tport_rcu_free()
ipv4: Convert call_rcu() to kfree_rcu(), drop opt_kfree_rcu()
ipv4: Convert call_rcu() to kfree_rcu(), drop opt_kfree_rcu
mac80211: Convert call_rcu() to kfree_rcu(), drop mesh_gate_node_reclaim()
rcu: Simplify offline processing
rcu: Make rcutorture flag online/offline failures
rcu: Limit lazy-callback duration
rcu: Check for callback invocation from offline CPUs
rcu: Don't make callbacks go through second full grace period
rcu: Remove single-rcu_node optimization in rcu_start_gp()
rcu: Protect __rcu_read_unlock() against scheduler-using irq handlers
rcu: Streamline code produced by __rcu_read_unlock()
rcu: Prevent RCU callbacks from executing before scheduler initialized
rcu: Inform RCU of irq_exit() activity
rcu: Simplify unboosting checks
rcu: Clean up straggling rcu_preempt_needs_cpu() name
rcu: Check for idle-loop entry while in RCU read-side critical section
rcu: Make rcu_sleep_check() also check rcu_lock_map
rcu: Note that rcu_access_pointer() can be used for teardown
rcu: Remove #ifdef CONFIG_SMP from TREE_RCU
rcu: Set RCU CPU stall times via sysfs
rcu: Print scheduling-clock information on RCU CPU stall-warning messages
rcutorture: Permit holding off CPU-hotplug operations during boot
rcu: Make documentation give more realistic rcutorture duration
rcu: Add CPU-stall capability to rcutorture
rcu: Update stall-warning documentation
rcu: Check for illegal use of RCU from offlined CPUs
rcu: Move synchronize_sched_expedited() to rcutree.c
rcu: No interrupt disabling for rcu_prepare_for_idle()
lockdep: Add CPU-idle/offline warning to lockdep-RCU splat
rcu: Rework detection of use of RCU by offline CPUs
rcu: Call out dangers of expedited RCU primitives
rcu: Trace only after NULL-pointer check
rcu: Remove redundant check for rcu_head misalignment
rcu: Allow nesting of rcu_idle_enter() and rcu_idle_exit()
rcu: Add RCU_NONIDLE() for idle-loop RCU read-side critical sections
rcu: Eliminate softirq-mediated RCU_FAST_NO_HZ idle-entry loop
rcu: Hold off RCU_FAST_NO_HZ after timer posted
Documentation/RCU/RTFP.txt | 1902 ++++++++++++++++++++++++++++++++++---- Documentation/RCU/checklist.txt | 14 + Documentation/RCU/stallwarn.txt | 87 ++- Documentation/RCU/torture.txt | 33 +- Documentation/RCU/trace.txt | 36 +- arch/s390/kernel/irq.c | 9 +- drivers/target/tcm_fc/tfc_sess.c | 12 +- include/linux/rcupdate.h | 83 ++- include/linux/rcutiny.h | 10 +- include/linux/rcutree.h | 19 + include/linux/sched.h | 3 +- include/linux/srcu.h | 15 +- include/trace/events/rcu.h | 63 +- init/Kconfig | 9 - kernel/lockdep.c | 8 +- kernel/rcu.h | 26 +- kernel/rcupdate.c | 5 + kernel/rcutiny.c | 26 +- kernel/rcutiny_plugin.h | 77 ++- kernel/rcutorture.c | 91 ++- kernel/rcutree.c | 507 ++++++++--- kernel/rcutree.h | 27 +- kernel/rcutree_plugin.h | 450 ++++++---- kernel/rcutree_trace.c | 12 +- kernel/srcu.c | 33 +- lib/Kconfig.debug | 24 + net/ipv4/cipso_ipv4.c | 11 +- net/ipv4/ip_sockglue.c | 7 +- net/mac80211/mesh_pathtbl.c | 8 +- 29 files changed, 2965 insertions(+), 642 deletions(-)
Documentation/RCU/RTFP.txt +1745 -157
@@ -1,9 +1,10 @@ -Read the F-ing Papers! +Read the Fscking Papers! This document describes RCU-related publications, and is followed by the corresponding bibtex entries. A number of the publications may -be found at http://www.rdrop.com/users/paulmck/RCU/. +be found at http://www.rdrop.com/users/paulmck/RCU/. For others, browsers +and search engines will usually find what you are looking for. The first thing resembling RCU was published in 1980, when Kung and Lehman [Kung80] recommended use of a garbage collector to defer destruction
@@ -160,7 +161,26 @@ which Mathieu Desnoyers is now maintaining [MathieuDesnoyers2009URCU] [MathieuDesnoyersPhD]. TINY_RCU [PaulEMcKenney2009BloatWatchRCU] made its appearance, as did expedited RCU [PaulEMcKenney2009expeditedRCU]. The problem of resizeable RCU-protected hash tables may now be on a path -to a solution [JoshTriplett2009RPHash]. +to a solution [JoshTriplett2009RPHash]. A few academic researchers are now +using RCU to solve their parallel problems [HariKannan2009DynamicAnalysisRCU]. + +2010 produced a simpler preemptible-RCU implementation +based on TREE_RCU [PaulEMcKenney2010SimpleOptRCU], lockdep-RCU +[PaulEMcKenney2010LockdepRCU], another resizeable RCU-protected hash +table [HerbertXu2010RCUResizeHash] (this one consuming more memory, +but allowing arbitrary changes in hash function, as required for DoS +avoidance in the networking code), realization of the 2009 RCU-protected +hash table with atomic node move [JoshTriplett2010RPHash], an update on +the RCU API [PaulEMcKenney2010RCUAPI]. + +2011 marked the inclusion of Nick Piggin's fully lockless dentry search +[LinusTorvalds2011Linux2:6:38:rc1:NPigginVFS], an RCU-protected red-black +tree using software transactional memory to protect concurrent updates +(strange, but true!) [PhilHoward2011RCUTMRBTree], yet another variant of +RCU-protected resizeable hash tables [Triplett:2011:RPHash], the 3.0 RCU +trainwreck [PaulEMcKenney2011RCU3.0trainwreck], and Neil Brown's "Meet the +Lockers" LWN article [NeilBrown2011MeetTheLockers]. + Bibtex Entries
@@ -173,6 +193,14 @@ Bibtex Entries ,volume="5" ,number="3" ,pages="354-382" +,note="Available: +\url{http://portal.acm.org/citation.cfm?id=320619&dl=GUIDE,} +[Viewed December 3, 2007]" +,annotation={ + Use garbage collector to clean up data after everyone is done with it. + . + Oldest use of something vaguely resembling RCU that I have found. +} } @techreport{Manber82
@@ -184,6 +212,31 @@ Bibtex Entries ,number="82-01-01" ,month="January" ,pages="28" +,annotation={ + . + Superseded by Manber84. + . + Describes concurrent AVL tree implementation. Uses a + garbage-collection mechanism to handle concurrent use and deletion + of nodes in the tree, but lacks the summary-of-execution-history + concept of read-copy locking. + . + Keeps full list of processes that were active when a given + node was to be deleted, and waits until all such processes have + -terminated- before allowing this node to be reused. This is + not described in great detail -- one could imagine using process + IDs for this if the ID space was large enough that overlapping + never occurred. + . + This restriction makes this algorithm unsuitable for use in + systems comprised of long-lived processes. It also produces + completely unacceptable overhead in systems with large numbers + of processes. Finally, it is specific to AVL trees. + . + Cites Kung80, so not an independent invention, but the first + RCU-like usage that does not rely on an automatic garbage + collector. +} } @article{Manber84
@@ -195,6 +248,74 @@ Bibtex Entries ,volume="9" ,number="3" ,pages="439-455" +,annotation={ + Describes concurrent AVL tree implementation. Uses a + garbage-collection mechanism to handle concurrent use and deletion + of nodes in the tree, but lacks the summary-of-execution-history + concept of read-copy locking. + . + Keeps full list of processes that were active when a given + node was to be deleted, and waits until all such processes have + -terminated- before allowing this node to be reused. This is + not described in great detail -- one could imagine using process + IDs for this if the ID space was large enough that overlapping + never occurred. + . + This restriction makes this algorithm unsuitable for use in + systems comprised of long-lived processes. It also produces + completely unacceptable overhead in systems with large numbers + of processes. Finally, it is specific to AVL trees. +} +} + +@Conference{RichardRashid87a +,Author="Richard Rashid and Avadis Tevanian and Michael Young and +David Golub and Robert Baron and David Black and William Bolosky and +Jonathan Chew" +,Title="Machine-Independent Virtual Memory Management for Paged +Uniprocessor and Multiprocessor Architectures" +,Booktitle="{2\textsuperscript{nd} Symposium on Architectural Support +for Programming Languages and Operating Systems}" +,Publisher="Association for Computing Machinery" +,Month="October" +,Year="1987" +,pages="31-39" +,Address="Palo Alto, CA" +,note="Available: +\url{http://www.cse.ucsc.edu/~randal/221/rashid-machvm.pdf} +[Viewed February 17, 2005]" +,annotation={ + Describes lazy TLB flush, where one waits for each CPU to pass + through a scheduling-clock interrupt before reusing a given range + of virtual address. Does not describe how one determines that + all CPUs have in fact taken such an interrupt, though there are + no shortage of straightforward methods for accomplishing this. + . + Note that it does not make sense to just wait a fixed amount of + time, since a given CPU might have interrupts disabled for an + extended amount of time. +} +} + +@article{BarbaraLiskov1988ArgusCACM +,author = {Barbara Liskov} +,title = {Distributed programming in {Argus}} +,journal = {Commun. ACM} +,volume = {31} +,number = {3} +,year = {1988} +,issn = {0001-0782} +,pages = {300--312} +,doi = {http://doi.acm.org/10.1145/42392.42399} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation= { + At the top of page 307: "Conflicts with deposits and withdrawals + are necessary if the reported total is to be up to date. They + could be avoided by having total return a sum that is slightly + out of date." Relies on semantics -- approximate numerical + values sometimes OK. +} } @techreport{Hennessy89
@@ -216,6 +337,13 @@ Bibtex Entries ,year="1990" ,number="CS-TR-2222.1" ,month="June" +,annotation={ + Concurrent access to skip lists. Has both weak and strong search. + Uses concept of ``garbage queue'', but has no real way of cleaning + the garbage efficiently. + . + Appears to be an independent invention of an RCU-like mechanism. +} } @Book{Adams91
@@ -223,20 +351,15 @@ Bibtex Entries ,title="Concurrent Programming, Principles, and Practices" ,Publisher="Benjamin Cummins" ,Year="1991" +,annotation={ + Has a few paragraphs describing ``chaotic relaxation'', a + numerical analysis technique that allows multiprocessors to + avoid synchronization overhead by using possibly-stale data. + . + Seems like this is descended from yet another independent + invention of RCU-like function -- but this is restricted + in that reclamation is not necessary. } - -@phdthesis{HMassalinPhD -,author="H. Massalin" -,title="Synthesis: An Efficient Implementation of Fundamental Operating -System Services" -,school="Columbia University" -,address="New York, NY" -,year="1992" -,annotation=" - Mondo optimizing compiler. - Wait-free stuff. - Good advice: defer work to avoid synchronization. -" } @unpublished{Jacobson93
@@ -244,7 +367,13 @@ System Services" ,title="Avoid Read-Side Locking Via Delayed Free" ,year="1993" ,month="September" -,note="Verbal discussion" +,note="private communication" +,annotation={ + Use fixed time delay to approximate grace period. Very simple, + but subject to random memory corruption under heavy load. + . + Independent invention of RCU-like mechanism. +} } @Conference{AjuJohn95
@@ -256,6 +385,17 @@ System Services" ,Year="1995" ,pages="11-23" ,Address="New Orleans, LA" +,note="Available: +\url{https://www.usenix.org/publications/library/proceedings/neworl/full_papers/john.a} +[Viewed October 1, 2010]" +,annotation={ + Age vnodes out of the cache, and have a fixed time set by a kernel + parameter. Not clear that all races were in fact correctly handled. + Used a 20-minute time by default, which would most definitely not + be suitable during DoS attacks or virus scans. + . + Apparently independent invention of RCU-like mechanism. +} } @conference{Pu95a,
@@ -301,31 +441,47 @@ Utilizing Execution History and Thread Monitoring" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="1995" -,number="US Patent 5,442,758 (contributed under GPL)" +,number="US Patent 5,442,758" ,month="August" +,annotation={ + Describes the parallel RCU infrastructure. Includes NUMA aspect + (structure of bitmap can reflect bus structure of computer system). + . + Another independent invention of an RCU-like mechanism, but the + "real" RCU this time! +} } @techreport{Slingwine97 ,author="John D. Slingwine and Paul E. McKenney" -,title="Method for maintaining data coherency using thread -activity summaries in a multicomputer system" +,title="Method for Maintaining Data Coherency Using Thread Activity +Summaries in a Multicomputer System" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="1997" -,number="US Patent 5,608,893 (contributed under GPL)" +,number="US Patent 5,608,893" ,month="March" +,pages="19" +,annotation={ + Describes use of RCU to synchronize data between a pair of + SMP/NUMA computer systems. +} } @techreport{Slingwine98 ,author="John D. Slingwine and Paul E. McKenney" -,title="Apparatus and method for achieving reduced overhead -mutual exclusion and maintaining coherency in a multiprocessor -system utilizing execution history and thread monitoring" +,title="Apparatus and Method for Achieving Reduced Overhead Mutual +Exclusion and Maintaining Coherency in a Multiprocessor System +Utilizing Execution History and Thread Monitoring" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="1998" -,number="US Patent 5,727,209 (contributed under GPL)" +,number="US Patent 5,727,209" ,month="March" +,annotation={ + Describes doing an atomic update by copying the data item and + then substituting it into the data structure. +} } @Conference{McKenney98
@@ -337,6 +493,15 @@ Problems" ,Year="1998" ,pages="509-518" ,Address="Las Vegas, NV" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/rclockpdcsproof.pdf} +[Viewed December 3, 2007]" +,annotation={ + Describes and analyzes RCU mechanism in DYNIX/ptx. Describes + application to linked list update and log-buffer flushing. + Defines 'quiescent state'. Includes both measured and analytic + evaluation. +} } @Conference{Gamsa99
@@ -349,18 +514,76 @@ Operating System Design and Implementation}" ,Year="1999" ,pages="87-100" ,Address="New Orleans, LA" +,note="Available: +\url{http://www.usenix.org/events/osdi99/full_papers/gamsa/gamsa.pdf} +[Viewed August 30, 2006]" +,annotation={ + Use of RCU-like facility in K42/Tornado. Another independent + invention of RCU. + See especially pages 7-9 (Section 5). +} +} + +@unpublished{RustyRussell2000a +,Author="Rusty Russell" +,Title="Re: modular net drivers" +,month="June" +,year="2000" +,day="23" +,note="Available: +\url{http://oss.sgi.com/projects/netdev/archive/2000-06/msg00250.html} +[Viewed April 10, 2006]" +,annotation={ + Proto-RCU proposal from Phil Rumpf and Rusty Russell. + Yet another independent invention of RCU. + Outline of algorithm to unload modules... + . + Appeared on net-dev mailing list. +} +} + +@unpublished{RustyRussell2000b +,Author="Rusty Russell" +,Title="Re: modular net drivers" +,month="June" +,year="2000" +,day="24" +,note="Available: +\url{http://oss.sgi.com/projects/netdev/archive/2000-06/msg00254.html} +[Viewed April 10, 2006]" +,annotation={ + Proto-RCU proposal from Phil Rumpf and Rusty Russell. + . + Appeared on net-dev mailing list. +} +} + +@unpublished{McKenney01b +,Author="Paul E. McKenney and Dipankar Sarma" +,Title="Read-Copy Update Mutual Exclusion in {Linux}" +,month="February" +,year="2001" +,note="Available: +\url{http://lse.sourceforge.net/locking/rcu/rcupdate_doc.html} +[Viewed October 18, 2004]" +,annotation={ + Prototypical Linux documentation for RCU. +} } @techreport{Slingwine01 ,author="John D. Slingwine and Paul E. McKenney" -,title="Apparatus and method for achieving reduced overhead -mutual exclusion and maintaining coherency in a multiprocessor -system utilizing execution history and thread monitoring" +,title="Apparatus and Method for Achieving Reduced Overhead Mutual +Exclusion and Maintaining Coherency in a Multiprocessor System +Utilizing Execution History and Thread Monitoring" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="2001" -,number="US Patent 5,219,690 (contributed under GPL)" +,number="US Patent 6,219,690" ,month="April" +,annotation={ + 'Change in mode' aspect of RCU. Can be thought of as a lazy barrier. +} } @Conference{McKenney01a
@@ -372,14 +595,61 @@ Orran Krieger and Rusty Russell and Dipankar Sarma and Maneesh Soni" ,Year="2001" ,note="Available: \url{http://www.linuxsymposium.org/2001/abstracts/readcopy.php} -\url{http://www.rdrop.com/users/paulmck/rclock/rclock_OLS.2001.05.01c.pdf} +\url{http://www.rdrop.com/users/paulmck/RCU/rclock_OLS.2001.05.01c.pdf} [Viewed June 23, 2004]" -annotation=" -Described RCU, and presented some patches implementing and using it in -the Linux kernel. +,annotation={ + Described RCU, and presented some patches implementing and using + it in the Linux kernel. +} +} + +@unpublished{McKenney01f +,Author="Paul E. McKenney" +,Title="{RFC:} patch to allow lock-free traversal of lists with insertion" +,month="October" +,year="2001" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=100259266316456&w=2} +[Viewed June 23, 2004]" +,annotation=" + Memory-barrier and Alpha thread. 100 messages, not too bad... +" +} + +@unpublished{Spraul01 +,Author="Manfred Spraul" +,Title="Re: {RFC:} patch to allow lock-free traversal of lists with insertion" +,month="October" +,year="2001" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=100264675012867&w=2} +[Viewed June 23, 2004]" +,annotation=" + Suggested burying memory barriers in Linux's list-manipulation + primitives. " } +@unpublished{LinusTorvalds2001a +,Author="Linus Torvalds" +,Title="{Re:} {[Lse-tech]} {Re:} {RFC:} patch to allow lock-free traversal of lists with insertion" +,month="October" +,year="2001" +,note="Available: +\url{http://lkml.org/lkml/2001/10/13/105} +[Viewed August 21, 2004]" +} + +@unpublished{Blanchard02a +,Author="Anton Blanchard" +,Title="some RCU dcache and ratcache results" +,month="March" +,year="2002" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=101637107412972&w=2} +[Viewed October 18, 2004]" +} + @Conference{Linder02a ,Author="Hanna Linder and Dipankar Sarma and Maneesh Soni" ,Title="Scalability of the Directory Entry Cache"
@@ -387,6 +657,10 @@ the Linux kernel. ,Month="June" ,Year="2002" ,pages="289-300" +,annotation=" + Measured scalability of Linux 2.4 kernel's directory-entry cache + (dcache), and measured some scalability enhancements. +" } @Conference{McKenney02a
@@ -400,49 +674,76 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell" ,note="Available: \url{http://www.linux.org.uk/~ajh/ols2002_proceedings.pdf.gz} [Viewed June 23, 2004]" +,annotation=" + Presented and compared a number of RCU implementations for the + Linux kernel. +" } -@conference{Michael02a -,author="Maged M. Michael" -,title="Safe Memory Reclamation for Dynamic Lock-Free Objects Using Atomic -Reads and Writes" -,Year="2002" -,Month="August" -,booktitle="{Proceedings of the 21\textsuperscript{st} Annual ACM -Symposium on Principles of Distributed Computing}" -,pages="21-30" +@unpublished{Sarma02a +,Author="Dipankar Sarma" +,Title="specweb99: dcache scalability results" +,month="July" +,year="2002" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=102645767914212&w=2} +[Viewed June 23, 2004]" ,annotation=" - Each thread keeps an array of pointers to items that it is - currently referencing. Sort of an inside-out garbage collection - mechanism, but one that requires the accessing code to explicitly - state its needs. Also requires read-side memory barriers on - most architectures. + Compare fastwalk and RCU for dcache. RCU won. " } -@conference{Michael02b -,author="Maged M. Michael" -,title="High Performance Dynamic Lock-Free Hash Tables and List-Based Sets" -,Year="2002" -,Month="August" -,booktitle="{Proceedings of the 14\textsuperscript{th} Annual ACM -Symposium on Parallel -Algorithms and Architecture}" -,pages="73-82" +@unpublished{Barbieri02 +,Author="Luca Barbieri" +,Title="Re: {[PATCH]} Initial support for struct {vfs\_cred}" +,month="August" +,year="2002" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=103082050621241&w=2} +[Viewed: June 23, 2004]" ,annotation=" - Like the title says... + Suggested RCU for vfs\_shared\_cred. " } -@InProceedings{HerlihyLM02 -,author={Maurice Herlihy and Victor Luchangco and Mark Moir} -,title="The Repeat Offender Problem: A Mechanism for Supporting Dynamic-Sized, -Lock-Free Data Structures" -,booktitle={Proceedings of 16\textsuperscript{th} International -Symposium on Distributed Computing} -,year=2002 +@unpublished{Dickins02a +,author="Hugh Dickins" +,title="Use RCU for System-V IPC" +,year="2002" +,month="October" +,note="private communication" +} + +@unpublished{Sarma02b +,Author="Dipankar Sarma" +,Title="Some dcache\_rcu benchmark numbers" ,month="October" -,pages="339-353" +,year="2002" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=103462075416638&w=2} +[Viewed June 23, 2004]" +,annotation=" + Performance of dcache RCU on kernbench for 16x NUMA-Q and 1x, + 2x, and 4x systems. RCU does no harm, and helps on 16x. +" +} + +@unpublished{LinusTorvalds2003a +,Author="Linus Torvalds" +,Title="Re: {[PATCH]} small fixes in brlock.h" +,month="March" +,year="2003" +,note="Available: +\url{http://lkml.org/lkml/2003/3/9/205} +[Viewed March 13, 2006]" +,annotation=" + Linus suggests replacing brlock with RCU and/or seqlocks: + . + 'It's entirely possible that the current user could be replaced + by RCU and/or seqlocks, and we could get rid of brlocks entirely.' + . + Steve Hemminger responds by replacing them with RCU. +" } @article{Appavoo03a
@@ -457,6 +758,20 @@ B. Rosenburg and M. Stumm and J. Xenidis" ,volume="42" ,number="1" ,pages="60-76" +,annotation=" + Use of RCU to enable hot-swapping for autonomic behavior in K42. +" +} + +@unpublished{Seigh03 +,author="Joseph W. {Seigh II}" +,title="Read Copy Update" +,Year="2003" +,Month="March" +,note="email correspondence" +,annotation=" + Described the relationship of the VM/XA passive serialization to RCU. +" } @Conference{Arcangeli03
@@ -470,6 +785,27 @@ Dipankar Sarma" ,year="2003" ,month="June" ,pages="297-310" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/rcu.FREENIX.2003.06.14.pdf} +[Viewed November 21, 2007]" +,annotation=" + Compared updated RCU implementations for the Linux kernel, and + described System V IPC use of RCU, including order-of-magnitude + performance improvements. +" +} + +@Conference{Soules03a +,Author="Craig A. N. Soules and Jonathan Appavoo and Kevin Hui and +Dilma {Da Silva} and Gregory R. Ganger and Orran Krieger and +Michael Stumm and Robert W. Wisniewski and Marc Auslander and +Michal Ostrowski and Bryan Rosenburg and Jimi Xenidis" +,Title="System Support for Online Reconfiguration" +,Booktitle="Proceedings of the 2003 USENIX Annual Technical Conference" +,Publisher="USENIX Association" +,year="2003" +,month="June" +,pages="141-154" } @article{McKenney03a
@@ -481,6 +817,22 @@ Dipankar Sarma" ,volume="1" ,number="114" ,pages="18-26" +,note="Available: +\url{http://www.linuxjournal.com/article/6993} +[Viewed November 14, 2007]" +,annotation=" + Reader-friendly intro to RCU, with the infamous old-man-and-brat + cartoon. +" +} + +@unpublished{Sarma03a +,Author="Dipankar Sarma" +,Title="RCU low latency patches" +,month="December" +,year="2003" +,note="Message ID: [email protected]" +,annotation="dipankar/ct.2004.03.27/RCUll.2003.12.22.patch" } @techreport{Friedberg03a
@@ -489,9 +841,14 @@ Dipankar Sarma" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="2003" -,number="US Patent 6,662,184 (contributed under GPL)" +,number="US Patent 6,662,184" ,month="December" ,pages="112" +,annotation=" + Applies RCU to a wildcard-search Patricia tree in order to permit + synchronization-free lookup. RCU is used to retain removed nodes + for a grace period before freeing them. +" } @article{McKenney04a
@@ -503,6 +860,12 @@ Dipankar Sarma" ,volume="1" ,number="118" ,pages="38-46" +,note="Available: +\url{http://www.linuxjournal.com/node/7124} +[Viewed December 26, 2010]" +,annotation=" + Reader friendly intro to dcache and RCU. +" } @Conference{McKenney04b
@@ -514,152 +877,824 @@ Dipankar Sarma" ,Address="Adelaide, Australia" ,note="Available: \url{http://www.linux.org.au/conf/2004/abstracts.html#90} -\url{http://www.rdrop.com/users/paulmck/rclock/lockperf.2004.01.17a.pdf} +\url{http://www.rdrop.com/users/paulmck/RCU/lockperf.2004.01.17a.pdf} [Viewed June 23, 2004]" +,annotation=" + Compares performance of RCU to that of other locking primitives + over a number of CPUs (x86, Opteron, Itanium, and PPC). +" } -@phdthesis{PaulEdwardMcKenneyPhD -,author="Paul E. McKenney" -,title="Exploiting Deferred Destruction: -An Analysis of Read-Copy-Update Techniques -in Operating System Kernels" -,school="OGI School of Science and Engineering at -Oregon Health and Sciences University" +@unpublished{Sarma04a +,Author="Dipankar Sarma" +,Title="{[PATCH]} {RCU} for low latency (experimental)" +,month="March" +,year="2004" +,note="\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=108003746402892&w=2}" +,annotation="Head of thread: dipankar/2004.03.23/rcu-low-lat.1.patch" +} + +@unpublished{Sarma04b +,Author="Dipankar Sarma" +,Title="Re: {[PATCH]} {RCU} for low latency (experimental)" +,month="March" +,year="2004" +,note="\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=108016474829546&w=2}" +,annotation="dipankar/rcuth.2004.03.24/rcu-throttle.patch" +} + +@unpublished{Spraul04a +,Author="Manfred Spraul" +,Title="[RFC] 0/5 rcu lock update" +,month="May" ,year="2004" ,note="Available: -\url{http://www.rdrop.com/users/paulmck/RCU/RCUdissertation.2004.07.14e1.pdf} -[Viewed October 15, 2004]" +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=108546407726602&w=2} +[Viewed June 23, 2004]" +,annotation=" + Hierarchical-bitmap patch for RCU infrastructure. +" +} + +@unpublished{Steiner04a +,Author="Jack Steiner" +,Title="Re: [Lse-tech] [RFC, PATCH] 1/5 rcu lock update: +Add per-cpu batch counter" +,month="May" +,year="2004" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=108551764515332&w=2} +[Viewed June 23, 2004]" +,annotation={ + RCU runs reasonably on a 512-CPU SGI using Manfred Spraul's patches, + which may be found at: + https://lkml.org/lkml/2004/5/20/49 (split vars into cachelines) + https://lkml.org/lkml/2004/5/22/114 (cpu_quiet() patch) + https://lkml.org/lkml/2004/5/25/24 (0/5) + https://lkml.org/lkml/2004/5/25/23 (1/5) + https://lkml.org/lkml/2004/5/25/265 (works for Jack) + https://lkml.org/lkml/2004/5/25/20 (2/5) + https://lkml.org/lkml/2004/5/25/22 (3/5) + https://lkml.org/lkml/2004/5/25/19 (4/5) + https://lkml.org/lkml/2004/5/25/21 (5/5) +} } @Conference{Sarma04c ,Author="Dipankar Sarma and Paul E. McKenney" -,Title="Making RCU Safe for Deep Sub-Millisecond Response Realtime Applications" +,Title="Making {RCU} Safe for Deep Sub-Millisecond Response +Realtime Applications" ,Booktitle="Proceedings of the 2004 USENIX Annual Technical Conference (FREENIX Track)" ,Publisher="USENIX Association" ,year="2004" ,month="June" ,pages="182-191" +,annotation=" + Describes and compares a number of modifications to the Linux RCU + implementation that make it friendly to realtime applications. +" } -@unpublished{JamesMorris04b -,Author="James Morris" -,Title="Recent Developments in {SELinux} Kernel Performance" -,month="December" +@phdthesis{PaulEdwardMcKenneyPhD +,author="Paul E. McKenney" +,title="Exploiting Deferred Destruction: +An Analysis of Read-Copy-Update Techniques +in Operating System Kernels" +,school="OGI School of Science and Engineering at +Oregon Health and Sciences University" ,year="2004" ,note="Available: -\url{http://www.livejournal.com/users/james_morris/2153.html} -[Viewed December 10, 2004]" +\url{http://www.rdrop.com/users/paulmck/RCU/RCUdissertation.2004.07.14e1.pdf} +[Viewed October 15, 2004]" +,annotation=" + Describes RCU implementations and presents design patterns + corresponding to common uses of RCU in several operating-system + kernels. +" } -@unpublished{PaulMcKenney05a -,Author="Paul E. McKenney" -,Title="{[RFC]} {RCU} and {CONFIG\_PREEMPT\_RT} progress" -,month="May" -,year="2005" +@unpublished{PaulEMcKenney2004rcu:dereference +,Author="Dipankar Sarma" +,Title="{Re: RCU : Abstracted RCU dereferencing [5/5]}" +,month="August" +,year="2004" ,note="Available: -\url{http://lkml.org/lkml/2005/5/9/185} -[Viewed May 13, 2005]" +\url{http://lkml.org/lkml/2004/8/6/237} +[Viewed June 8, 2010]" ,annotation=" - First publication of working lock-based deferred free patches - for the CONFIG_PREEMPT_RT environment. + Introduce rcu_dereference(). " } -@conference{PaulMcKenney05b -,Author="Paul E. McKenney and Dipankar Sarma" -,Title="Towards Hard Realtime Response from the Linux Kernel on SMP Hardware" -,Booktitle="linux.conf.au 2005" -,month="April" -,year="2005" -,address="Canberra, Australia" +@unpublished{JimHouston04a +,Author="Jim Houston" +,Title="{[RFC\&PATCH] Alternative {RCU} implementation}" +,month="August" +,year="2004" ,note="Available: -\url{http://www.rdrop.com/users/paulmck/RCU/realtimeRCU.2005.04.23a.pdf} -[Viewed May 13, 2005]" +\url{http://lkml.org/lkml/2004/8/30/87} +[Viewed February 17, 2005]" ,annotation=" - Realtime turns into making RCU yet more realtime friendly. + Uses active code in rcu_read_lock() and rcu_read_unlock() to + make RCU happen, allowing RCU to function on CPUs that do not + receive a scheduling-clock interrupt. " } -@conference{ThomasEHart2006a -,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown" -,Title="Making Lockless Synchronization Fast: Performance Implications -of Memory Reclamation" -,Booktitle="20\textsuperscript{th} {IEEE} International Parallel and -Distributed Processing Symposium" -,month="April" -,year="2006" -,day="25-29" -,address="Rhodes, Greece" +@unpublished{TomHart04a +,Author="Thomas E. Hart" +,Title="Master's Thesis: Applying Lock-free Techniques to the {Linux} Kernel" +,month="October" +,year="2004" +,note="Available: +\url{http://www.cs.toronto.edu/~tomhart/masters_thesis.html} +[Viewed October 15, 2004]" ,annotation=" - Compares QSBR (AKA "classic RCU"), HPBR, EBR, and lock-free - reference counting. + Proposes comparing RCU to lock-free methods for the Linux kernel. " } -@Conference{PaulEMcKenney2006b -,Author="Paul E. McKenney and Dipankar Sarma and Ingo Molnar and -Suparna Bhattacharya" -,Title="Extending RCU for Realtime and Embedded Workloads" -,Booktitle="{Ottawa Linux Symposium}" -,Month="July" -,Year="2006" -,pages="v2 123-138" +@unpublished{Vaddagiri04a +,Author="Srivatsa Vaddagiri" +,Title="Subject: [RFC] Use RCU for tcp\_ehash lookup" +,month="October" +,year="2004" ,note="Available: -\url{http://www.linuxsymposium.org/2006/index_2006.php} -\url{http://www.rdrop.com/users/paulmck/RCU/OLSrtRCU.2006.08.11a.pdf} -[Viewed January 1, 2007]" +\url{http://marc.theaimsgroup.com/?t=109395731700004&r=1&w=2} +[Viewed October 18, 2004]" ,annotation=" - Described how to improve the -rt implementation of realtime RCU. + Srivatsa's RCU patch for tcp_ehash lookup. " } -@unpublished{PaulEMcKenney2006c -,Author="Paul E. McKenney" -,Title="Sleepable {RCU}" +@unpublished{Thirumalai04a +,Author="Ravikiran Thirumalai" +,Title="Subject: [patchset] Lockfree fd lookup 0 of 5" ,month="October" -,day="9" -,year="2006" +,year="2004" ,note="Available: -\url{http://lwn.net/Articles/202847/} -Revised: -\url{http://www.rdrop.com/users/paulmck/RCU/srcu.2007.01.14a.pdf} -[Viewed August 21, 2006]" +\url{http://marc.theaimsgroup.com/?t=109144217400003&r=1&w=2} +[Viewed October 18, 2004]" ,annotation=" - LWN article introducing SRCU. + Ravikiran's lockfree FD patch. +" +} + +@unpublished{Thirumalai04b +,Author="Ravikiran Thirumalai" +,Title="Subject: Re: [patchset] Lockfree fd lookup 0 of 5" +,month="October" +,year="2004" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=109152521410459&w=2} +[Viewed October 18, 2004]" +,annotation=" + Ravikiran's lockfree FD patch. +" +} + +@unpublished{PaulEMcKenney2004rcu:assign:pointer +,Author="Paul E. McKenney" +,Title="{[PATCH 1/3] RCU: \url{rcu_assign_pointer()} removal of memory barriers}" +,month="October" +,year="2004" +,note="Available: +\url{http://lkml.org/lkml/2004/10/23/241} +[Viewed June 8, 2010]" +,annotation=" + Introduce rcu_assign_pointer(). +" +} + +@unpublished{JamesMorris04a +,Author="James Morris" +,Title="{[PATCH 2/3] SELinux} scalability - convert {AVC} to {RCU}" +,day="15" +,month="November" +,year="2004" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=110054979416004&w=2} +[Viewed December 10, 2004]" +,annotation=" + James Morris posts Kaigai Kohei's patch to LKML. +" +} + +@unpublished{JamesMorris04b +,Author="James Morris" +,Title="Recent Developments in {SELinux} Kernel Performance" +,month="December" +,year="2004" +,note="Available: +\url{http://www.livejournal.com/users/james_morris/2153.html} +[Viewed December 10, 2004]" +,annotation=" + RCU helps SELinux performance. ;-) Made LWN. +" +} + +@unpublished{PaulMcKenney2005RCUSemantics +,Author="Paul E. McKenney and Jonathan Walpole" +,Title="{RCU} Semantics: A First Attempt" +,month="January" +,year="2005" +,day="30" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/rcu-semantics.2005.01.30a.pdf} +[Viewed December 6, 2009]" +,annotation=" + Early derivation of RCU semantics. +" +} + +@unpublished{PaulMcKenney2005e +,Author="Paul E. McKenney" +,Title="Real-Time Preemption and {RCU}" +,month="March" +,year="2005" +,day="17" +,note="Available: +\url{http://lkml.org/lkml/2005/3/17/199} +[Viewed September 5, 2005]" +,annotation=" + First posting showing how RCU can be safely adapted for + preemptable RCU read side critical sections. +" +} + +@unpublished{EsbenNeilsen2005a +,Author="Esben Neilsen" +,Title="Re: Real-Time Preemption and {RCU}" +,month="March" +,year="2005" +,day="18" +,note="Available: +\url{http://lkml.org/lkml/2005/3/18/122} +[Viewed March 30, 2006]" +,annotation=" + Esben Neilsen suggests read-side suppression of grace-period + processing for crude-but-workable realtime RCU. The downside + is indefinite grace periods...But this is OK for experimentation + and testing. +" +} + +@unpublished{TomHart05a +,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown" +,Title="Efficient Memory Reclamation is Necessary for Fast Lock-Free +Data Structures" +,month="March" +,year="2005" +,note="Available: +\url{ftp://ftp.cs.toronto.edu/csrg-technical-reports/515/} +[Viewed March 4, 2005]" +,annotation=" + Comparison of RCU, QBSR, and EBSR. RCU wins for read-mostly + workloads. ;-) +" +} + +@unpublished{JonCorbet2005DeprecateSyncKernel +,Author="Jonathan Corbet" +,Title="API change: synchronize_kernel() deprecated" +,month="May" +,day="3" +,year="2005" +,note="Available: +\url{http://lwn.net/Articles/134484/} +[Viewed May 3, 2005]" +,annotation=" + Jon Corbet describes deprecation of synchronize_kernel() + in favor of synchronize_rcu() and synchronize_sched(). +" +} + +@unpublished{PaulMcKenney05a +,Author="Paul E. McKenney" +,Title="{[RFC]} {RCU} and {CONFIG\_PREEMPT\_RT} progress" +,month="May" +,year="2005" +,note="Available: +\url{http://lkml.org/lkml/2005/5/9/185} +[Viewed May 13, 2005]" +,annotation=" + First publication of working lock-based deferred free patches + for the CONFIG_PREEMPT_RT environment. +" +} + +@conference{PaulMcKenney05b +,Author="Paul E. McKenney and Dipankar Sarma" +,Title="Towards Hard Realtime Response from the {Linux} Kernel on {SMP} Hardware" +,Booktitle="linux.conf.au 2005" +,month="April" +,year="2005" +,address="Canberra, Australia" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/realtimeRCU.2005.04.23a.pdf} +[Viewed May 13, 2005]" +,annotation=" + Realtime turns into making RCU yet more realtime friendly. + http://lca2005.linux.org.au/Papers/Paul%20McKenney/Towards%20Hard%20Realtime%20Response%20from%20the%20Linux%20Kernel/LKS.2005.04.22a.pdf +" +} + +@unpublished{PaulEMcKenneyHomePage +,Author="Paul E. McKenney" +,Title="{Paul} {E.} {McKenney}" +,month="May" +,year="2005" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/} +[Viewed May 25, 2005]" +,annotation=" + Paul McKenney's home page. +" +} + +@unpublished{PaulEMcKenneyRCUPage +,Author="Paul E. McKenney" +,Title="Read-Copy Update {(RCU)}" +,month="May" +,year="2005" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU} +[Viewed May 25, 2005]" +,annotation=" + Paul McKenney's RCU page. +" +} + +@unpublished{JosephSeigh2005a +,Author="Joseph Seigh" +,Title="{RCU}+{SMR} (hazard pointers)" +,month="July" +,year="2005" +,note="Personal communication" +,annotation=" + Joe Seigh announcing his atomic-ptr-plus project. + http://sourceforge.net/projects/atomic-ptr-plus/ +" +} + +@unpublished{JosephSeigh2005b +,Author="Joseph Seigh" +,Title="Lock-free synchronization primitives" +,month="July" +,day="6" +,year="2005" +,note="Available: +\url{http://sourceforge.net/projects/atomic-ptr-plus/} +[Viewed August 8, 2005]" +,annotation=" + Joe Seigh's atomic-ptr-plus project. +" +} + +@unpublished{PaulMcKenney2005c +,Author="Paul E.McKenney" +,Title="{[RFC,PATCH] RCU} and {CONFIG\_PREEMPT\_RT} sane patch" +,month="August" +,day="1" +,year="2005" +,note="Available: +\url{http://lkml.org/lkml/2005/8/1/155} +[Viewed March 14, 2006]" +,annotation=" + First operating counter-based realtime RCU patch posted to LKML. +" +} + +@unpublished{PaulMcKenney2005d +,Author="Paul E. McKenney" +,Title="Re: [Fwd: Re: [patch] Real-Time Preemption, -RT-2.6.13-rc4-V0.7.52-01]" +,month="August" +,day="8" +,year="2005" +,note="Available: +\url{http://lkml.org/lkml/2005/8/8/108} +[Viewed March 14, 2006]" +,annotation=" + First operating counter-based realtime RCU patch posted to LKML, + but fixed so that various unusual combinations of configuration + parameters all function properly. +" +} + +@unpublished{PaulMcKenney2005rcutorture +,Author="Paul E. McKenney" +,Title="{[PATCH]} {RCU} torture testing" +,month="October" +,day="1" +,year="2005" +,note="Available: +\url{http://lkml.org/lkml/2005/10/1/70} +[Viewed March 14, 2006]" +,annotation=" + First rcutorture patch. +" +} + +@conference{ThomasEHart2006a +,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown" +,Title="Making Lockless Synchronization Fast: Performance Implications +of Memory Reclamation" +,Booktitle="20\textsuperscript{th} {IEEE} International Parallel and +Distributed Processing Symposium" +,month="April" +,year="2006" +,day="25-29" +,address="Rhodes, Greece" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/hart_ipdps06.pdf} +[Viewed April 28, 2008]" +,annotation=" + Compares QSBR, HPBR, EBR, and lock-free reference counting. + http://www.cs.toronto.edu/~tomhart/perflab/ipdps06.tgz +" +} + +@unpublished{NickPiggin2006radixtree +,Author="Nick Piggin" +,Title="[patch 3/3] radix-tree: {RCU} lockless readside" +,month="June" +,day="20" +,year="2006" +,note="Available: +\url{http://lkml.org/lkml/2006/6/20/238} +[Viewed March 25, 2008]" +,annotation=" + RCU-protected radix tree. +" +} + +@Conference{PaulEMcKenney2006b +,Author="Paul E. McKenney and Dipankar Sarma and Ingo Molnar and +Suparna Bhattacharya" +,Title="Extending {RCU} for Realtime and Embedded Workloads" +,Booktitle="{Ottawa Linux Symposium}" +,Month="July" +,Year="2006" +,pages="v2 123-138" +,note="Available: +\url{http://www.linuxsymposium.org/2006/view_abstract.php?content_key=184} +\url{http://www.rdrop.com/users/paulmck/RCU/OLSrtRCU.2006.08.11a.pdf} +[Viewed January 1, 2007]" +,annotation=" + Described how to improve the -rt implementation of realtime RCU. +" +} + +@unpublished{WikipediaRCU +,Author="Paul E. McKenney and Chris Purcell and Algae and Ben Schumin and +Gaius Cornelius and Qwertyus and Neil Conway and Sbw and Blainster and +Canis Rufus and Zoicon5 and Anome and Hal Eisen" +,Title="Read-Copy Update" +,month="July" +,day="8" +,year="2006" +,note="Available: +\url{http://en.wikipedia.org/wiki/Read-copy-update} +[Viewed August 21, 2006]" +,annotation=" + Wikipedia RCU page as of July 8 2006. +" +} + +@Conference{NickPiggin2006LocklessPageCache +,Author="Nick Piggin" +,Title="A Lockless Pagecache in Linux---Introduction, Progress, Performance" +,Booktitle="{Ottawa Linux Symposium}" +,Month="July" +,Year="2006" +,pages="v2 249-254" +,note="Available: +\url{http://www.linuxsymposium.org/2006/view_abstract.php?content_key=184} +[Viewed January 11, 2009]" +,annotation=" + Uses RCU-protected radix tree for a lockless page cache. +" +} + +@unpublished{PaulEMcKenney2006c +,Author="Paul E. McKenney" +,Title="Sleepable {RCU}" +,month="October" +,day="9" +,year="2006" +,note="Available: +\url{http://lwn.net/Articles/202847/} +Revised: +\url{http://www.rdrop.com/users/paulmck/RCU/srcu.2007.01.14a.pdf} +[Viewed August 21, 2006]" +,annotation=" + LWN article introducing SRCU. " } @unpublished{RobertOlsson2006a ,Author="Robert Olsson and Stefan Nilsson" ,Title="{TRASH}: A dynamic {LC}-trie and hash data structure" -,month="August" -,day="18" -,year="2006" +,month="August" +,day="18" +,year="2006" +,note="Available: +\url{http://www.nada.kth.se/~snilsson/publications/TRASH/trash.pdf} +[Viewed March 4, 2011]" +,annotation=" + RCU-protected dynamic trie-hash combination. +" +} + +@unpublished{ChristophHellwig2006RCU2SRCU +,Author="Christoph Hellwig" +,Title="Re: {[-mm PATCH 1/4]} {RCU}: split classic rcu" +,month="September" +,day="28" +,year="2006" +,note="Available: +\url{http://lkml.org/lkml/2006/9/28/160} +[Viewed March 27, 2008]" +} + +@unpublished{PaulEMcKenneyRCUusagePage +,Author="Paul E. McKenney" +,Title="{RCU} {Linux} Usage" +,month="October" +,year="2006" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/linuxusage.html} +[Viewed January 14, 2007]" +,annotation=" + Paul McKenney's RCU page showing graphs plotting Linux-kernel + usage of RCU. +" +} + +@unpublished{PaulEMcKenneyRCUusageRawDataPage +,Author="Paul E. McKenney" +,Title="Read-Copy Update {(RCU)} Usage in {Linux} Kernel" +,month="October" +,year="2006" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html} +[Viewed January 14, 2007]" +,annotation=" + Paul McKenney's RCU page showing Linux usage of RCU in tabular + form, with links to corresponding cscope databases. +" +} + +@unpublished{GauthamShenoy2006RCUrwlock +,Author="Gautham R. Shenoy" +,Title="[PATCH 4/5] lock\_cpu\_hotplug: Redesign - Lightweight implementation of lock\_cpu\_hotplug" +,month="October" +,year="2006" +,day=26 +,note="Available: +\url{http://lkml.org/lkml/2006/10/26/73} +[Viewed January 26, 2009]" +,annotation=" + RCU-based reader-writer lock that allows readers to proceed with + no memory barriers or atomic instruction in absence of writers. + If writer do show up, readers must of course wait as required by + the semantics of reader-writer locking. This is a recursive + lock. +" +} + +@unpublished{JensAxboe2006SlowSRCU +,Author="Jens Axboe" +,Title="Re: [patch] cpufreq: mark \url{cpufreq_tsc()} as +\url{core_initcall_sync}" +,month="November" +,year="2006" +,day=17 +,note="Available: +\url{http://lkml.org/lkml/2006/11/17/56} +[Viewed May 28, 2007]" +,annotation=" + SRCU's grace periods are too slow for Jens, even after a + factor-of-three speedup. + Sped-up version of SRCU at http://lkml.org/lkml/2006/11/17/359. +" +} + +@unpublished{OlegNesterov2006QRCU +,Author="Oleg Nesterov" +,Title="Re: [patch] cpufreq: mark {\tt cpufreq\_tsc()} as +{\tt core\_initcall\_sync}" +,month="November" +,year="2006" +,day=19 +,note="Available: +\url{http://lkml.org/lkml/2006/11/19/69} +[Viewed May 28, 2007]" +,annotation=" + First cut of QRCU. Expanded/corrected versions followed. + Used to be OlegNesterov2007QRCU, now time-corrected. +" +} + +@unpublished{OlegNesterov2006aQRCU +,Author="Oleg Nesterov" +,Title="Re: [RFC, PATCH 1/2] qrcu: {"quick"} srcu implementation" +,month="November" +,year="2006" +,day=30 +,note="Available: +\url{http://lkml.org/lkml/2006/11/29/330} +[Viewed November 26, 2008]" +,annotation=" + Expanded/corrected version of QRCU. + Used to be OlegNesterov2007aQRCU, now time-corrected. +" +} + +@unpublished{EvgeniyPolyakov2006RCUslowdown +,Author="Evgeniy Polyakov" +,Title="Badness in postponing work" +,month="December" +,year="2006" +,day=05 +,note="Available: +\url{http://www.ioremap.net/node/41} +[Viewed October 28, 2008]" +,annotation=" + Using RCU as a pure delay leads to a 2.5x slowdown in skbs in + the Linux kernel. +" +} + +@inproceedings{ChrisMatthews2006ClusteredObjectsRCU +,author = {Matthews, Chris and Coady, Yvonne and Appavoo, Jonathan} +,title = {Portability events: a programming model for scalable system infrastructures} +,booktitle = {PLOS '06: Proceedings of the 3rd workshop on Programming languages and operating systems} +,year = {2006} +,isbn = {1-59593-577-0} +,pages = {11} +,location = {San Jose, California} +,doi = {http://doi.acm.org/10.1145/1215995.1216006} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation={ + Uses K42's RCU-like functionality to manage clustered-object + lifetimes. +}} + +@article{DilmaDaSilva2006K42 +,author = {Silva, Dilma Da and Krieger, Orran and Wisniewski, Robert W. and Waterland, Amos and Tam, David and Baumann, Andrew} +,title = {K42: an infrastructure for operating system research} +,journal = {SIGOPS Oper. Syst. Rev.} +,volume = {40} +,number = {2} +,year = {2006} +,issn = {0163-5980} +,pages = {34--42} +,doi = {http://doi.acm.org/10.1145/1131322.1131333} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation={ + Describes relationship of K42 generations to RCU. +}} + +# CoreyMinyard2007list_splice_rcu +@unpublished{CoreyMinyard2007list:splice:rcu +,Author="Corey Minyard and Paul E. McKenney" +,Title="{[PATCH]} add an {RCU} version of list splicing" +,month="January" +,year="2007" +,day=3 +,note="Available: +\url{http://lkml.org/lkml/2007/1/3/112} +[Viewed May 28, 2007]" +,annotation=" + Patch for list_splice_rcu(). +" +} + +@unpublished{PaulEMcKenney2007rcubarrier +,Author="Paul E. McKenney" +,Title="{RCU} and Unloadable Modules" +,month="January" +,day="14" +,year="2007" +,note="Available: +\url{http://lwn.net/Articles/217484/} +[Viewed November 22, 2007]" +,annotation=" + LWN article introducing the rcu_barrier() primitive. +" +} + +@unpublished{PeterZijlstra2007SyncBarrier +,Author="Peter Zijlstra and Ingo Molnar" +,Title="{[PATCH 3/7]} barrier: a scalable synchonisation barrier" +,month="January" +,year="2007" +,day=28 +,note="Available: +\url{http://lkml.org/lkml/2007/1/28/34} +[Viewed March 27, 2008]" +,annotation=" + RCU-like implementation for frequent updaters and rare readers(!). + Subsumed into QRCU. Maybe... +" +} + +@unpublished{PaulEMcKenney2007BoostRCU +,Author="Paul E. McKenney" +,Title="Priority-Boosting {RCU} Read-Side Critical Sections" +,month="February" +,day="5" +,year="2007" +,note="Available: +\url{http://lwn.net/Articles/220677/} +Revised: +\url{http://www.rdrop.com/users/paulmck/RCU/RCUbooststate.2007.04.16a.pdf} +[Viewed September 7, 2007]" +,annotation=" + LWN article introducing RCU priority boosting. +" +} + +@unpublished{PaulMcKenney2007QRCUpatch +,Author="Paul E. McKenney" +,Title="{[PATCH]} {QRCU} with lockless fastpath" +,month="February" +,year="2007" +,day=24 +,note="Available: +\url{http://lkml.org/lkml/2007/2/25/18} +[Viewed March 27, 2008]" +,annotation=" + Patch for QRCU supplying lock-free fast path. +" +} + +@article{JonathanAppavoo2007K42RCU +,author = {Appavoo, Jonathan and Silva, Dilma Da and Krieger, Orran and Auslander, Marc and Ostrowski, Michal and Rosenburg, Bryan and Waterland, Amos and Wisniewski, Robert W. and Xenidis, Jimi and Stumm, Michael and Soares, Livio} +,title = {Experience distributing objects in an SMMP OS} +,journal = {ACM Trans. Comput. Syst.} +,volume = {25} +,number = {3} +,year = {2007} +,issn = {0734-2071} +,pages = {6/1--6/52} +,doi = {http://doi.acm.org/10.1145/1275517.1275518} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation={ + Role of RCU in K42. +}} + +@conference{RobertOlsson2007Trash +,Author="Robert Olsson and Stefan Nilsson" +,Title="{TRASH}: A dynamic {LC}-trie and hash data structure" +,booktitle="Workshop on High Performance Switching and Routing (HPSR'07)" +,month="May" +,year="2007" ,note="Available: -\url{http://www.nada.kth.se/~snilsson/public/papers/trash/trash.pdf} -[Viewed February 24, 2007]" +\url{http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=4281239} +[Viewed October 1, 2010]" ,annotation=" RCU-protected dynamic trie-hash combination. " } -@unpublished{ThomasEHart2007a -,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown and Jonathan Walpole" -,Title="Performance of memory reclamation for lockless synchronization" -,journal="J. Parallel Distrib. Comput." +@conference{PeterZijlstra2007ConcurrentPagecacheRCU +,Author="Peter Zijlstra" +,Title="Concurrent Pagecache" +,Booktitle="Linux Symposium" +,month="June" +,year="2007" +,address="Ottawa, Canada" +,note="Available: +\url{http://ols.108.redhat.com/2007/Reprints/zijlstra-Reprint.pdf} +[Viewed April 14, 2008]" +,annotation=" + Page-cache modifications permitting RCU readers and concurrent + updates. +" +} + +@unpublished{PaulEMcKenney2007whatisRCU +,Author="Paul E. McKenney" +,Title="What is {RCU}?" ,year="2007" -,note="To appear in J. Parallel Distrib. Comput. - \url{doi=10.1016/j.jpdc.2007.04.010}" +,month="07" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/whatisRCU.html} +[Viewed July 6, 2007]" ,annotation={ - Compares QSBR (AKA "classic RCU"), HPBR, EBR, and lock-free - reference counting. Journal version of ThomasEHart2006a. + Describes RCU in Linux kernel. } } @unpublished{PaulEMcKenney2007QRCUspin ,Author="Paul E. McKenney" -,Title="Using Promela and Spin to verify parallel algorithms" +,Title="Using {Promela} and {Spin} to verify parallel algorithms" ,month="August" ,day="1" ,year="2007"
@@ -669,6 +1704,50 @@ Revised: ,annotation=" LWN article describing Promela and spin, and also using Oleg Nesterov's QRCU as an example (with Paul McKenney's fastpath). + Merged patch at: http://lkml.org/lkml/2007/2/25/18 +" +} + +@unpublished{PaulEMcKenney2007WG21DDOatomics +,Author="Paul E. McKenney and Hans-J. Boehm and Lawrence Crowl" +,Title="C++ Data-Dependency Ordering: Atomics and Memory Model" +,month="August" +,day="3" +,year="2007" +,note="Preprint: +\url{http://open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2664.htm} +[Viewed December 7, 2009]" +,annotation=" + RCU for C++, parts 1 and 2. +" +} + +@unpublished{PaulEMcKenney2007WG21DDOannotation +,Author="Paul E. McKenney and Lawrence Crowl" +,Title="C++ Data-Dependency Ordering: Function Annotation" +,month="September" +,day="18" +,year="2008" +,note="Preprint: +\url{http://open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2782.htm} +[Viewed December 7, 2009]" +,annotation=" + RCU for C++, part 2, updated many times. +" +} + +@unpublished{PaulEMcKenney2007PreemptibleRCUPatch +,Author="Paul E. McKenney" +,Title="[PATCH RFC 0/9] {RCU}: Preemptible {RCU}" +,month="September" +,day="10" +,year="2007" +,note="Available: +\url{http://lkml.org/lkml/2007/9/10/213} +[Viewed October 25, 2007]" +,annotation=" + Final patch for preemptable RCU to -rt. (Later patches were + to mainline, eventually incorporated.) " }
@@ -686,10 +1765,46 @@ Revised: " } +@article{ThomasEHart2007a +,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown and Jonathan Walpole" +,Title="Performance of memory reclamation for lockless synchronization" +,journal="J. Parallel Distrib. Comput." +,volume={67} +,number="12" +,year="2007" +,issn="0743-7315" +,pages="1270--1285" +,doi="http://dx.doi.org/10.1016/j.jpdc.2007.04.010" +,publisher="Academic Press, Inc." +,address="Orlando, FL, USA" +,annotation={ + Compares QSBR, HPBR, EBR, and lock-free reference counting. + Journal version of ThomasEHart2006a. +} +} + +@unpublished{MathieuDesnoyers2007call:rcu:schedNeeded +,Author="Mathieu Desnoyers" +,Title="Re: [patch 1/2] {Linux} Kernel Markers - Support Multiple Probes" +,month="December" +,day="20" +,year="2007" +,note="Available: +\url{http://lkml.org/lkml/2007/12/20/244} +[Viewed March 27, 2008]" +,annotation=" + Request for call_rcu_sched() and rcu_barrier_sched(). +" +} + + ######################################################################## # # "What is RCU?" LWN series. # +# http://lwn.net/Articles/262464/ (What is RCU, Fundamentally?) +# http://lwn.net/Articles/263130/ (What is RCU's Usage?) +# http://lwn.net/Articles/264090/ (What is RCU's API?) @unpublished{PaulEMcKenney2007WhatIsRCUFundamentally ,Author="Paul E. McKenney and Jonathan Walpole"
@@ -723,7 +1838,7 @@ Revised: 3. RCU is a Bulk Reference-Counting Mechanism 4. RCU is a Poor Man's Garbage Collector 5. RCU is a Way of Providing Existence Guarantees - 6. RCU is a Way of Waiting for Things to Finish + 6. RCU is a Way of Waiting for Things to Finish " }
@@ -747,20 +1862,96 @@ Revised: # ######################################################################## + +@unpublished{SteveRostedt2008dyntickRCUpatch +,Author="Steven Rostedt and Paul E. McKenney" +,Title="{[PATCH]} add support for dynamic ticks and preempt rcu" +,month="January" +,day="29" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/1/29/208} +[Viewed March 27, 2008]" +,annotation=" + Patch that prevents preemptible RCU from unnecessarily waking + up dynticks-idle CPUs. +" +} + +@unpublished{PaulEMcKenney2008LKMLDependencyOrdering +,Author="Paul E. McKenney" +,Title="Re: [PATCH 02/22 -v7] Add basic support for gcc profiler instrumentation" +,month="February" +,day="1" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/2/2/255} +[Viewed October 18, 2008]" +,annotation=" + Explanation of compilers violating dependency ordering. +" +} + +@Conference{PaulEMcKenney2008Beijing +,Author="Paul E. McKenney" +,Title="Introducing Technology Into {Linux} Or: +Introducing your technology Into {Linux} will require introducing a +lot of {Linux} into your technology!!!" +,Booktitle="2008 Linux Developer Symposium - China" +,Publisher="OSS China" +,Month="February" +,Year="2008" +,Address="Beijing, China" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/TechIntroLinux.2008.02.19a.pdf} +[Viewed August 12, 2008]" +} + +@unpublished{PaulEMcKenney2008dynticksRCU +,Author="Paul E. McKenney and Steven Rostedt" +,Title="Integrating and Validating dynticks and Preemptable RCU" +,month="April" +,day="24" +,year="2008" +,note="Available: +\url{http://lwn.net/Articles/279077/} +[Viewed April 24, 2008]" +,annotation=" + Describes use of Promela and Spin to validate (and fix!) the + dynticks/RCU interface. +" +} + @article{DinakarGuniguntala2008IBMSysJ ,author="D. Guniguntala and P. E. McKenney and J. Triplett and J. Walpole" ,title="The read-copy-update mechanism for supporting real-time applications on shared-memory multiprocessor systems with {Linux}" ,Year="2008" -,Month="April" +,Month="April-June" ,journal="IBM Systems Journal" ,volume="47" ,number="2" -,pages="@@-@@" +,pages="221-236" ,annotation=" RCU, realtime RCU, sleepable RCU, performance. " } +@unpublished{LaiJiangshan2008NewClassicAlgorithm +,Author="Lai Jiangshan" +,Title="[{RFC}][{PATCH}] rcu classic: new algorithm for callbacks-processing" +,month="June" +,day="3" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/6/2/539} +[Viewed December 10, 2008]" +,annotation=" + Updated RCU classic algorithm. Introduced multi-tailed list + for RCU callbacks and also pulling common code into + __call_rcu(). +" +} + @article{PaulEMcKenney2008RCUOSR ,author="Paul E. McKenney and Jonathan Walpole" ,title="Introducing technology into the {Linux} kernel: a case study"
@@ -778,6 +1969,52 @@ Revised: } } +@unpublished{ManfredSpraul2008StateMachineRCU +,Author="Manfred Spraul" +,Title="[{RFC}, {PATCH}] state machine based rcu" +,month="August" +,day="21" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/8/21/336} +[Viewed December 8, 2008]" +,annotation=" + State-based RCU. One key thing that this patch does is to + separate the dynticks handling of NMIs and IRQs. +" +} + +@unpublished{ManfredSpraul2008dyntickIRQNMI +,Author="Manfred Spraul" +,Title="Re: [{RFC}, {PATCH}] v4 scalable classic {RCU} implementation" +,month="September" +,day="6" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/9/6/86} +[Viewed December 8, 2008]" +,annotation=" + Manfred notes a fix required to my attempt to separate irq + and NMI processing for hierarchical RCU's dynticks interface. +" +} + +@techreport{PaulEMcKenney2008cyclicRCU +,author="Paul E. McKenney" +,title="Efficient Support of Consistent Cyclic Search With Read-Copy Update" +,institution="US Patent and Trademark Office" +,address="Washington, DC" +,year="2008" +,number="US Patent 7,426,511" +,month="September" +,pages="23" +,annotation=" + Maintains an additional level of indirection to allow + readers to confine themselves to the desired snapshot of the + data structure. Only permits one update at a time. +" +} + @unpublished{PaulEMcKenney2008HierarchicalRCU ,Author="Paul E. McKenney" ,Title="Hierarchical {RCU}"
@@ -793,6 +2030,21 @@ Revised: " } +@unpublished{PaulEMcKenney2009BloatwatchRCU +,Author="Paul E. McKenney" +,Title="Re: [PATCH fyi] RCU: the bloatwatch edition" +,month="January" +,day="14" +,year="2009" +,note="Available: +\url{http://lkml.org/lkml/2009/1/14/449} +[Viewed January 15, 2009]" +,annotation=" + Small-footprint implementation of RCU for uniprocessor + embedded applications -- and also for exposition purposes. +" +} + @conference{PaulEMcKenney2009MaliciousURCU ,Author="Paul E. McKenney" ,Title="Using a Malicious User-Level {RCU} to Torture {RCU}-Based Algorithms"
@@ -816,15 +2068,17 @@ Revised: ,year="2009" ,note="Available: \url{http://lkml.org/lkml/2009/2/5/572} -\url{git://lttng.org/userspace-rcu.git} +\url{http://lttng.org/urcu} [Viewed February 20, 2009]" ,annotation=" Mathieu Desnoyers's user-space RCU implementation. git://lttng.org/userspace-rcu.git + http://lttng.org/cgi-bin/gitweb.cgi?p=userspace-rcu.git + http://lttng.org/urcu " } -@unpublished{PaulEMcKenney2009BloatWatchRCU +@unpublished{PaulEMcKenney2009LWNBloatWatchRCU ,Author="Paul E. McKenney" ,Title="{RCU}: The {Bloatwatch} Edition" ,month="March"
@@ -852,14 +2106,29 @@ Revised: " } -@unpublished{JoshTriplett2009RPHash +@unpublished{PaulEMcKenney2009fastRTRCU +,Author="Paul E. McKenney" +,Title="[{PATCH} {RFC} -tip 0/4] {RCU} cleanups and simplified preemptable {RCU}" +,month="July" +,day="23" +,year="2009" +,note="Available: +\url{http://lkml.org/lkml/2009/7/23/294} +[Viewed August 15, 2009]" +,annotation=" + First posting of simple and fast preemptable RCU. +" +} + +@InProceedings{JoshTriplett2009RPHash ,Author="Josh Triplett" ,Title="Scalable concurrent hash tables via relativistic programming" ,month="September" ,year="2009" -,note="Linux Plumbers Conference presentation" +,booktitle="Linux Plumbers Conference 2009" ,annotation=" RP fun with hash tables. + See also JoshTriplett2010RPHash " }
@@ -872,4 +2141,323 @@ Revised: ,note="Available: \url{http://www.lttng.org/pub/thesis/desnoyers-dissertation-2009-12.pdf} [Viewed December 9, 2009]" +,annotation={ + Chapter 6 (page 97) covers user-level RCU. +} +} + +@unpublished{RelativisticProgrammingWiki +,Author="Josh Triplett and Paul E. McKenney and Jonathan Walpole" +,Title="Relativistic Programming" +,month="September" +,year="2009" +,note="Available: +\url{http://wiki.cs.pdx.edu/rp/} +[Viewed December 9, 2009]" +,annotation=" + Main Relativistic Programming Wiki. +" +} + +@conference{PaulEMcKenney2009DeterministicRCU +,Author="Paul E. McKenney" +,Title="Deterministic Synchronization in Multicore Systems: the Role of {RCU}" +,Booktitle="Eleventh Real Time Linux Workshop" +,month="September" +,year="2009" +,address="Dresden, Germany" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/realtime/paper/DetSyncRCU.2009.08.18a.pdf} +[Viewed January 14, 2009]" +} + +@unpublished{PaulEMcKenney2009HuntingHeisenbugs +,Author="Paul E. McKenney" +,Title="Hunting Heisenbugs" +,month="November" +,year="2009" +,day="1" +,note="Available: +\url{http://paulmck.livejournal.com/14639.html} +[Viewed June 4, 2010]" +,annotation=" + Day-one bug in Tree RCU that took forever to track down. +" +} + +@unpublished{MathieuDesnoyers2009defer:rcu +,Author="Mathieu Desnoyers" +,Title="Kernel RCU: shrink the size of the struct rcu\_head" +,month="December" +,year="2009" +,note="Available: +\url{http://lkml.org/lkml/2009/10/18/129} +[Viewed December 29, 2009]" +,annotation=" + Mathieu proposed defer_rcu() with fixed-size per-thread pool + of RCU callbacks. +" +} + +@unpublished{MathieuDesnoyers2009VerifPrePub +,Author="Mathieu Desnoyers and Paul E. McKenney and Michel R. Dagenais" +,Title="Multi-Core Systems Modeling for Formal Verification of Parallel Algorithms" +,month="December" +,year="2009" +,note="Submitted to IEEE TPDS" +,annotation=" + OOMem model for Mathieu's user-level RCU mechanical proof of + correctness. +" +} + +@unpublished{MathieuDesnoyers2009URCUPrePub +,Author="Mathieu Desnoyers and Paul E. McKenney and Alan Stern and Michel R. Dagenais and Jonathan Walpole" +,Title="User-Level Implementations of Read-Copy Update" +,month="December" +,year="2010" +,url=\url{http://www.computer.org/csdl/trans/td/2012/02/ttd2012020375-abs.html} +,annotation=" + RCU overview, desiderata, semi-formal semantics, user-level RCU + usage scenarios, three classes of RCU implementation, wait-free + RCU updates, RCU grace-period batching, update overhead, + http://www.rdrop.com/users/paulmck/RCU/urcu-main-accepted.2011.08.30a.pdf + http://www.rdrop.com/users/paulmck/RCU/urcu-supp-accepted.2011.08.30a.pdf + Superseded by MathieuDesnoyers2012URCU. +" +} + +@inproceedings{HariKannan2009DynamicAnalysisRCU +,author = {Kannan, Hari} +,title = {Ordering decoupled metadata accesses in multiprocessors} +,booktitle = {MICRO 42: Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture} +,year = {2009} +,isbn = {978-1-60558-798-1} +,pages = {381--390} +,location = {New York, New York} +,doi = {http://doi.acm.org/10.1145/1669112.1669161} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation={ + Uses RCU to protect metadata used in dynamic analysis. +}} + +@conference{PaulEMcKenney2010SimpleOptRCU +,Author="Paul E. McKenney" +,Title="Simplicity Through Optimization" +,Booktitle="linux.conf.au 2010" +,month="January" +,year="2010" +,address="Wellington, New Zealand" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/SimplicityThruOptimization.2010.01.21f.pdf} +[Viewed October 10, 2010]" +,annotation=" + TREE_PREEMPT_RCU optimizations greatly simplified the old + PREEMPT_RCU implementation. +" +} + +@unpublished{PaulEMcKenney2010LockdepRCU +,Author="Paul E. McKenney" +,Title="Lockdep-{RCU}" +,month="February" +,year="2010" +,day="1" +,note="Available: +\url{https://lwn.net/Articles/371986/} +[Viewed June 4, 2010]" +,annotation=" + CONFIG_PROVE_RCU, or at least an early version. +" +} + +@unpublished{AviKivity2010KVM2RCU +,Author="Avi Kivity" +,Title="[{PATCH} 37/40] {KVM}: Bump maximum vcpu count to 64" +,month="February" +,year="2010" +,note="Available: +\url{http://www.mail-archive.com/[email protected]/msg28640.html} +[Viewed March 20, 2010]" +,annotation=" + Use of RCU permits KVM to increase the size of guest OSes from + 16 CPUs to 64 CPUs. +" +} + +@unpublished{HerbertXu2010RCUResizeHash +,Author="Herbert Xu" +,Title="bridge: Add core IGMP snooping support" +,month="February" +,year="2010" +,note="Available: +\url{http://kerneltrap.com/mailarchive/linux-netdev/2010/2/26/6270589} +[Viewed March 20, 2011]" +,annotation={ + Use a pair of list_head structures to support RCU-protected + resizable hash tables. +}} + +@article{JoshTriplett2010RPHash +,author="Josh Triplett and Paul E. McKenney and Jonathan Walpole" +,title="Scalable Concurrent Hash Tables via Relativistic Programming" +,journal="ACM Operating Systems Review" +,year=2010 +,volume=44 +,number=3 +,month="July" +,annotation={ + RP fun with hash tables. + http://portal.acm.org/citation.cfm?id=1842733.1842750 +}} + +@unpublished{PaulEMcKenney2010RCUAPI +,Author="Paul E. McKenney" +,Title="The {RCU} {API}, 2010 Edition" +,month="December" +,day="8" +,year="2010" +,note="Available: +\url{http://lwn.net/Articles/418853/} +[Viewed December 8, 2010]" +,annotation=" + Includes updated software-engineering features. +" +} + +@mastersthesis{AndrejPodzimek2010masters +,author="Andrej Podzimek" +,title="Read-Copy-Update for OpenSolaris" +,school="Charles University in Prague" +,year="2010" +,note="Available: +\url{https://andrej.podzimek.org/thesis.pdf} +[Viewed January 31, 2011]" +,annotation={ + Reviews RCU implementations and creates a few for OpenSolaris. + Drives quiescent-state detection from RCU read-side primitives, + in a manner roughly similar to that of Jim Houston. +}} + +@unpublished{LinusTorvalds2011Linux2:6:38:rc1:NPigginVFS +,Author="Linus Torvalds" +,Title="Linux 2.6.38-rc1" +,month="January" +,year="2011" +,note="Available: +\url{https://lkml.org/lkml/2011/1/18/322} +[Viewed March 4, 2011]" +,annotation={ + "The RCU-based name lookup is at the other end of the spectrum - the + absolute anti-gimmick. It's some seriously good stuff, and gets rid of + the last main global lock that really tends to hurt some kernel loads. + The dentry lock is no longer a big serializing issue. What's really + nice about it is that it actually improves performance a lot even for + single-threaded loads (on an SMP kernel), because it gets rid of some + of the most expensive parts of path component lookup, which was the + d_lock on every component lookup. So I'm seeing improvements of 30-50% + on some seriously pathname-lookup intensive loads." +}} + +@techreport{JoshTriplett2011RPScalableCorrectOrdering +,author = {Josh Triplett and Philip W. Howard and Paul E. McKenney and Jonathan Walpole} +,title = {Scalable Correct Memory Ordering via Relativistic Programming} +,year = {2011} +,number = {11-03} +,institution = {Portland State University} +,note = {\url{http://www.cs.pdx.edu/pdfs/tr1103.pdf}} +} + +@inproceedings{PhilHoward2011RCUTMRBTree +,author = {Philip W. Howard and Jonathan Walpole} +,title = {A Relativistic Enhancement to Software Transactional Memory} +,booktitle = {Proceedings of the 3rd USENIX conference on Hot topics in parallelism} +,series = {HotPar'11} +,year = {2011} +,location = {Berkeley, CA} +,pages = {1--6} +,numpages = {6} +,url = {http://www.usenix.org/event/hotpar11/tech/final_files/Howard.pdf} +,publisher = {USENIX Association} +,address = {Berkeley, CA, USA} +} + +@techreport{PaulEMcKenney2011cyclicparallelRCU +,author="Paul E. McKenney and Jonathan Walpole" +,title="Efficient Support of Consistent Cyclic Search With Read-Copy Update and Parallel Updates" +,institution="US Patent and Trademark Office" +,address="Washington, DC" +,year="2011" +,number="US Patent 7,953,778" +,month="May" +,pages="34" +,annotation=" + Maintains an array of generation numbers to track in-flight + updates and keeps an additional level of indirection to allow + readers to confine themselves to the desired snapshot of the + data structure. +" +} + +@inproceedings{Triplett:2011:RPHash +,author = {Triplett, Josh and McKenney, Paul E. and Walpole, Jonathan} +,title = {Resizable, Scalable, Concurrent Hash Tables via Relativistic Programming} +,booktitle = {Proceedings of the 2011 USENIX Annual Technical Conference} +,month = {June} +,year = {2011} +,pages = {145--158} +,numpages = {14} +,url={http://www.usenix.org/event/atc11/tech/final_files/atc11_proceedings.pdf} +,publisher = {The USENIX Association} +,address = {Portland, OR USA} +} + +@unpublished{PaulEMcKenney2011RCU3.0trainwreck +,Author="Paul E. McKenney" +,Title="3.0 and {RCU:} what went wrong" +,month="July" +,day="27" +,year="2011" +,note="Available: +\url{http://lwn.net/Articles/453002/} +[Viewed July 27, 2011]" +,annotation=" + Analysis of the RCU trainwreck in Linux kernel 3.0. +" +} + +@unpublished{NeilBrown2011MeetTheLockers +,Author="Neil Brown" +,Title="Meet the Lockers" +,month="August" +,day="3" +,year="2011" +,note="Available: +\url{http://lwn.net/Articles/453685/} +[Viewed September 2, 2011]" +,annotation=" + The Locker family as an analogy for locking, reference counting, + RCU, and seqlock. +" +} + +@article{MathieuDesnoyers2012URCU +,Author="Mathieu Desnoyers and Paul E. McKenney and Alan Stern and Michel R. Dagenais and Jonathan Walpole" +,Title="User-Level Implementations of Read-Copy Update" +,journal="IEEE Transactions on Parallel and Distributed Systems" +,volume={23} +,year="2012" +,issn="1045-9219" +,pages="375-382" +,doi="http://doi.ieeecomputersociety.org/10.1109/TPDS.2011.159" +,publisher="IEEE Computer Society" +,address="Los Alamitos, CA, USA" +,annotation={ + RCU overview, desiderata, semi-formal semantics, user-level RCU + usage scenarios, three classes of RCU implementation, wait-free + RCU updates, RCU grace-period batching, update overhead, + http://www.rdrop.com/users/paulmck/RCU/urcu-main-accepted.2011.08.30a.pdf + http://www.rdrop.com/users/paulmck/RCU/urcu-supp-accepted.2011.08.30a.pdf +} }
Documentation/RCU/checklist.txt +14 -0
@@ -180,6 +180,20 @@ over a rather long period of time, but improvements are always welcome! operations that would not normally be undertaken while a real-time workload is running. + In particular, if you find yourself invoking one of the expedited + primitives repeatedly in a loop, please do everyone a favor: + Restructure your code so that it batches the updates, allowing + a single non-expedited primitive to cover the entire batch. + This will very likely be faster than the loop containing the + expedited primitive, and will be much much easier on the rest + of the system, especially to real-time workloads running on + the rest of the system. + + In addition, it is illegal to call the expedited forms from + a CPU-hotplug notifier, or while holding a lock that is acquired + by a CPU-hotplug notifier. Failing to observe this restriction + will result in deadlock. + 7. If the updater uses call_rcu() or synchronize_rcu(), then the corresponding readers must use rcu_read_lock() and rcu_read_unlock(). If the updater uses call_rcu_bh() or
Documentation/RCU/stallwarn.txt +80 -7
104 lines not on this page · raw message
Documentation/RCU/torture.txt +30 -3
@@ -69,6 +69,13 @@ onoff_interval CPU-hotplug operations regardless of what value is specified for onoff_interval. +onoff_holdoff The number of seconds to wait until starting CPU-hotplug + operations. This would normally only be used when + rcutorture was built into the kernel and started + automatically at boot time, in which case it is useful + in order to avoid confusing boot-time code with CPUs + coming and going. + shuffle_interval The number of seconds to keep the test threads affinitied to a particular subset of the CPUs, defaults to 3 seconds.
@@ -79,6 +86,24 @@ shutdown_secs The number of seconds to run the test before terminating zero, which disables test termination and system shutdown. This capability is useful for automated testing. +stall_cpu The number of seconds that a CPU should be stalled while + within both an rcu_read_lock() and a preempt_disable(). + This stall happens only once per rcutorture run. + If you need multiple stalls, use modprobe and rmmod to + repeatedly run rcutorture. The default for stall_cpu + is zero, which prevents rcutorture from stalling a CPU. + + Note that attempts to rmmod rcutorture while the stall + is ongoing will hang, so be careful what value you + choose for this module parameter! In addition, too-large + values for stall_cpu might well induce failures and + warnings in other parts of the kernel. You have been + warned! + +stall_cpu_holdoff + The number of seconds to wait after rcutorture starts + before stalling a CPU. Defaults to 10 seconds. + stat_interval The number of seconds between output of torture statistics (via printk()). Regardless of the interval, statistics are printed when the module is unloaded.
@@ -271,11 +296,13 @@ The following script may be used to torture RCU: #!/bin/sh modprobe rcutorture - sleep 100 + sleep 3600 rmmod rcutorture dmesg | grep torture: The output can be manually inspected for the error flag of "!!!". One could of course create a more elaborate script that automatically -checked for such errors. The "rmmod" command forces a "SUCCESS" or -"FAILURE" indication to be printk()ed. +checked for such errors. The "rmmod" command forces a "SUCCESS", +"FAILURE", or "RCU_HOTPLUG" indication to be printk()ed. The first +two are self-explanatory, while the last indicates that while there +were no RCU failures, CPU-hotplug problems were detected.
Documentation/RCU/trace.txt +16 -20
@@ -33,23 +33,23 @@ rcu/rcuboost: The output of "cat rcu/rcudata" looks as follows: rcu_sched: - 0 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=545/1/0 df=50 of=0 ri=0 ql=163 qs=NRW. kt=0/W/0 ktl=ebc3 b=10 ci=153737 co=0 ca=0 - 1 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=967/1/0 df=58 of=0 ri=0 ql=634 qs=NRW. kt=0/W/1 ktl=58c b=10 ci=191037 co=0 ca=0 - 2 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=1081/1/0 df=175 of=0 ri=0 ql=74 qs=N.W. kt=0/W/2 ktl=da94 b=10 ci=75991 co=0 ca=0 - 3 c=20942 g=20943 pq=1 pgp=20942 qp=1 dt=1846/0/0 df=404 of=0 ri=0 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=72261 co=0 ca=0 - 4 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=369/1/0 df=83 of=0 ri=0 ql=48 qs=N.W. kt=0/W/4 ktl=e0e7 b=10 ci=128365 co=0 ca=0 - 5 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=381/1/0 df=64 of=0 ri=0 ql=169 qs=NRW. kt=0/W/5 ktl=fb2f b=10 ci=164360 co=0 ca=0 - 6 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=1037/1/0 df=183 of=0 ri=0 ql=62 qs=N.W. kt=0/W/6 ktl=d2ad b=10 ci=65663 co=0 ca=0 - 7 c=20897 g=20897 pq=1 pgp=20896 qp=0 dt=1572/0/0 df=382 of=0 ri=0 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=75006 co=0 ca=0 + 0 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=545/1/0 df=50 of=0 ql=163 qs=NRW. kt=0/W/0 ktl=ebc3 b=10 ci=153737 co=0 ca=0 + 1 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=967/1/0 df=58 of=0 ql=634 qs=NRW. kt=0/W/1 ktl=58c b=10 ci=191037 co=0 ca=0 + 2 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=1081/1/0 df=175 of=0 ql=74 qs=N.W. kt=0/W/2 ktl=da94 b=10 ci=75991 co=0 ca=0 + 3 c=20942 g=20943 pq=1 pgp=20942 qp=1 dt=1846/0/0 df=404 of=0 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=72261 co=0 ca=0 + 4 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=369/1/0 df=83 of=0 ql=48 qs=N.W. kt=0/W/4 ktl=e0e7 b=10 ci=128365 co=0 ca=0 + 5 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=381/1/0 df=64 of=0 ql=169 qs=NRW. kt=0/W/5 ktl=fb2f b=10 ci=164360 co=0 ca=0 + 6 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=1037/1/0 df=183 of=0 ql=62 qs=N.W. kt=0/W/6 ktl=d2ad b=10 ci=65663 co=0 ca=0 + 7 c=20897 g=20897 pq=1 pgp=20896 qp=0 dt=1572/0/0 df=382 of=0 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=75006 co=0 ca=0 rcu_bh: - 0 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=545/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/0 ktl=ebc3 b=10 ci=0 co=0 ca=0 - 1 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=967/1/0 df=3 of=0 ri=1 ql=0 qs=.... kt=0/W/1 ktl=58c b=10 ci=151 co=0 ca=0 - 2 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1081/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/2 ktl=da94 b=10 ci=0 co=0 ca=0 - 3 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1846/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=0 co=0 ca=0 - 4 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=369/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/4 ktl=e0e7 b=10 ci=0 co=0 ca=0 - 5 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=381/1/0 df=4 of=0 ri=1 ql=0 qs=.... kt=0/W/5 ktl=fb2f b=10 ci=0 co=0 ca=0 - 6 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1037/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/6 ktl=d2ad b=10 ci=0 co=0 ca=0 - 7 c=1474 g=1474 pq=1 pgp=1473 qp=0 dt=1572/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=0 co=0 ca=0 + 0 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=545/1/0 df=6 of=0 ql=0 qs=.... kt=0/W/0 ktl=ebc3 b=10 ci=0 co=0 ca=0 + 1 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=967/1/0 df=3 of=0 ql=0 qs=.... kt=0/W/1 ktl=58c b=10 ci=151 co=0 ca=0 + 2 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1081/1/0 df=6 of=0 ql=0 qs=.... kt=0/W/2 ktl=da94 b=10 ci=0 co=0 ca=0 + 3 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1846/0/0 df=8 of=0 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=0 co=0 ca=0 + 4 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=369/1/0 df=6 of=0 ql=0 qs=.... kt=0/W/4 ktl=e0e7 b=10 ci=0 co=0 ca=0 + 5 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=381/1/0 df=4 of=0 ql=0 qs=.... kt=0/W/5 ktl=fb2f b=10 ci=0 co=0 ca=0 + 6 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1037/1/0 df=6 of=0 ql=0 qs=.... kt=0/W/6 ktl=d2ad b=10 ci=0 co=0 ca=0 + 7 c=1474 g=1474 pq=1 pgp=1473 qp=0 dt=1572/0/0 df=8 of=0 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=0 co=0 ca=0 The first section lists the rcu_data structures for rcu_sched, the second for rcu_bh. Note that CONFIG_TREE_PREEMPT_RCU kernels will have an
@@ -119,10 +119,6 @@ o "of" is the number of times that some other CPU has forced a CPU is offline when it is really alive and kicking) is a fatal error, so it makes sense to err conservatively. -o "ri" is the number of times that RCU has seen fit to send a - reschedule IPI to this CPU in order to get it to report a - quiescent state. - o "ql" is the number of RCU callbacks currently residing on this CPU. This is the total number of callbacks, regardless of what state they are in (new, waiting for grace period to
arch/s390/kernel/irq.c +1 -8
@@ -165,13 +165,6 @@ static inline int ext_hash(u16 code)return (code + (code >>9 )) &0xff ; } -static void ext_int_hash_update (struct rcu_head *head) -{ - struct ext_int_info *p = container_of(head, struct ext_int_info, rcu); - -kfree (p); -} -int register_external_interrupt (u16 code,ext_int_handler_t handler) {struct ext_int_info *p;
@@ -202,7 +195,7 @@ int unregister_external_interrupt(u16 code, ext_int_handler_t handler)list_for_each_entry_rcu (p, &ext_int_hash[index], entry)if (p->code == code && p->handler == handler) {list_del_rcu (&p->entry); -call_rcu (&p->rcu, ext_int_hash_update); +kfree_rcu (p, rcu); }spin_unlock_irqrestore (&ext_int_hash_lock, flags);return 0 ;
drivers/target/tcm_fc/tfc_sess.c +1 -11
@@ -86,16 +86,6 @@ static struct ft_tport *ft_tport_create(struct fc_lport *lport) }/* -* Free tport via RCU. -*/ -static void ft_tport_rcu_free (struct rcu_head *rcu) -{ - struct ft_tport *tport = container_of(rcu, struct ft_tport, rcu); - -kfree (tport); -} - -/* * Delete a target local port. * Caller holds ft_lport_lock. */
@@ -114,7 +104,7 @@ static void ft_tport_delete(struct ft_tport *tport) tpg->tport =NULL ; tport->tpg =NULL ; } -call_rcu (&tport->rcu, ft_tport_rcu_free); +kfree_rcu (tport, rcu); }/*
include/linux/rcupdate.h +80 -3
186 lines not on this page · raw message
include/linux/rcutiny.h +6 -4
@@ -27,13 +27,9 @@#include <linux/cache.h> -#ifdef CONFIG_RCU_BOOST static inline void rcu_init (void ) { } -#else /* #ifdef CONFIG_RCU_BOOST */ -void rcu_init (void ); -#endif /* #else #ifdef CONFIG_RCU_BOOST */ static inline void rcu_barrier_bh (void ) {
@@ -83,6 +79,12 @@ static inline void synchronize_sched_expedited(void)synchronize_sched (); } +static inline void kfree_call_rcu (struct rcu_head *head, +void (*func)(struct rcu_head *rcu)) +{ +call_rcu (head, func); +} +#ifdef CONFIG_TINY_RCU static inline void rcu_preempt_note_context_switch (void )
include/linux/rcutree.h +19 -0
@@ -61,6 +61,24 @@ extern void synchronize_rcu_bh(void);extern void synchronize_sched_expedited (void );extern void synchronize_rcu_expedited (void ); +void kfree_call_rcu (struct rcu_head *head,void (*func)(struct rcu_head *rcu)); + +/** +* synchronize_rcu_bh_expedited - Brute-force RCU-bh grace period +* +* Wait for an RCU-bh grace period to elapse, but use a "big hammer" +* approach to force the grace period to end quickly. This consumes +* significant time on all CPUs and is unfriendly to real-time workloads, +* so is thus not recommended for any sort of common-case code. In fact, +* if you are using synchronize_rcu_bh_expedited() in a loop, please +* restructure your code to batch your updates, and then use a single +* synchronize_rcu_bh() instead. +* +* Note that it is illegal to call this function while holding any lock +* that is acquired by a CPU-hotplug notifier. And yes, it is also illegal +* to call this function from a CPU-hotplug notifier. Failing to observe +* these restriction will result in deadlock. +*/ static inline void synchronize_rcu_bh_expedited (void ) {synchronize_sched_expedited ();
@@ -83,6 +101,7 @@ extern void rcu_sched_force_quiescent_state(void);/* A context switch is a grace period for RCU-sched and RCU-bh. */ static inline int rcu_blocking_is_gp (void ) { +might_sleep ();/* Check for RCU read-side critical section. */ return num_online_cpus () ==1 ; }
include/linux/sched.h +1 -2
@@ -1864,8 +1864,7 @@ extern void task_clear_jobctl_pending(struct task_struct *task,#ifdef CONFIG_PREEMPT_RCU #define RCU_READ_UNLOCK_BLOCKED (1 << 0)/* blocked while in RCU read-side. */ -#define RCU_READ_UNLOCK_BOOSTED (1 << 1)/* boosted while in RCU read-side. */ -#define RCU_READ_UNLOCK_NEED_QS (1 << 2)/* RCU core needs CPU response. */ +#define RCU_READ_UNLOCK_NEED_QS (1 << 1)/* RCU core needs CPU response. */ static inline void rcu_copy_process (struct task_struct *p) {
include/linux/srcu.h +11 -4
@@ -99,15 +99,18 @@ long srcu_batches_completed(struct srcu_struct *sp);* power mode. This way we can notice an extended quiescent state to * other CPUs that started a grace period. Otherwise we would delay any * grace period as long as we run in the idle task. +* +* Similarly, we avoid claiming an SRCU read lock held if the current +* CPU is offline. */ static inline int srcu_read_lock_held (struct srcu_struct *sp) { -if (rcu_is_cpu_idle ()) -return 0 ; -if (!debug_lockdep_rcu_enabled ())return 1 ; - +if (rcu_is_cpu_idle ()) +return 0 ; +if (!rcu_lockdep_current_cpu_online ()) +return 0 ;return lock_is_held (&sp->dep_map); }
@@ -169,6 +172,8 @@ static inline int srcu_read_lock(struct srcu_struct *sp) __acquires(sp)int retval =__srcu_read_lock (sp);rcu_lock_acquire (&(sp)->dep_map); +rcu_lockdep_assert (!rcu_is_cpu_idle (), +"srcu_read_lock() used illegally while idle" );return retval; }
@@ -182,6 +187,8 @@ static inline int srcu_read_lock(struct srcu_struct *sp) __acquires(sp)static inline void srcu_read_unlock (struct srcu_struct *sp,int idx)__releases (sp) { +rcu_lockdep_assert (!rcu_is_cpu_idle (), +"srcu_read_unlock() used illegally while idle" );rcu_lock_release (&(sp)->dep_map);__srcu_read_unlock (sp, idx); }
include/trace/events/rcu.h +39 -24
146 lines not on this page · raw message
init/Kconfig +0 -9
@@ -438,15 +438,6 @@ config PREEMPT_RCU This option enables preemptible-RCU code that is common between the TREE_PREEMPT_RCU and TINY_PREEMPT_RCU implementations. -config RCU_TRACE - bool "Enable tracing for RCU" - help - This option provides tracing in RCU which presents stats - in debugfs for debugging RCU implementation. - - Say Y here if you want to enable RCU tracing - Say N if you are unsure. - config RCU_FANOUT int "Tree-based hierarchical RCU fanout value" range 2 64 if 64BIT
kernel/lockdep.c +7 -1
@@ -4176,7 +4176,13 @@ void lockdep_rcu_suspicious(const char *file, const int line, const char *s)printk ("-------------------------------\n" );printk ("%s:%d %s!\n" , file, line, s);printk ("\nother info that might help us debug this:\n\n" ); -printk ("\nrcu_scheduler_active = %d, debug_locks = %d\n" , rcu_scheduler_active, debug_locks); +printk ("\n%srcu_scheduler_active = %d, debug_locks = %d\n" , + !rcu_lockdep_current_cpu_online () + ?"RCU used illegally from offline CPU!\n" + :rcu_is_cpu_idle () + ?"RCU used illegally from idle CPU!\n" + :"" , + rcu_scheduler_active, debug_locks);/* * If a CPU is in the RCU-free window in idle (ie: in the section
kernel/rcu.h +23 -3
@@ -33,8 +33,27 @@* Process-level increment to ->dynticks_nesting field. This allows for * architectures that use half-interrupts and half-exceptions from * process context. +* +* DYNTICK_TASK_NEST_MASK defines a field of width DYNTICK_TASK_NEST_WIDTH +* that counts the number of process-based reasons why RCU cannot +* consider the corresponding CPU to be idle, and DYNTICK_TASK_NEST_VALUE +* is the value used to increment or decrement this field. +* +* The rest of the bits could in principle be used to count interrupts, +* but this would mean that a negative-one value in the interrupt +* field could incorrectly zero out the DYNTICK_TASK_NEST_MASK field. +* We therefore provide a two-bit guard field defined by DYNTICK_TASK_MASK +* that is set to DYNTICK_TASK_FLAG upon initial exit from idle. +* The DYNTICK_TASK_EXIT_IDLE value is thus the combined value used upon +* initial exit from idle. */ -#define DYNTICK_TASK_NESTING (LLONG_MAX / 2 - 1) +#define DYNTICK_TASK_NEST_WIDTH 7 +#define DYNTICK_TASK_NEST_VALUE ((LLONG_MAX >> DYNTICK_TASK_NEST_WIDTH) + 1) +#define DYNTICK_TASK_NEST_MASK (LLONG_MAX - DYNTICK_TASK_NEST_VALUE + 1) +#define DYNTICK_TASK_FLAG ((DYNTICK_TASK_NEST_VALUE / 8) * 2) +#define DYNTICK_TASK_MASK ((DYNTICK_TASK_NEST_VALUE / 8) * 3) +#define DYNTICK_TASK_EXIT_IDLE (DYNTICK_TASK_NEST_VALUE + \ + DYNTICK_TASK_FLAG)/* * debug_rcu_head_queue()/debug_rcu_head_unqueue() are used internally
@@ -50,7 +69,6 @@ extern struct debug_obj_descr rcuhead_debug_descr;static inline void debug_rcu_head_queue (struct rcu_head *head) { -WARN_ON_ONCE ((unsigned long )head &0x3 );debug_object_activate (head, &rcuhead_debug_descr); debug_object_active_state(head, &rcuhead_debug_descr, STATE_RCU_HEAD_READY,
@@ -76,16 +94,18 @@ static inline void debug_rcu_head_unqueue(struct rcu_head *head)extern void kfree (const void *); -static inline void __rcu_reclaim (char *rn,struct rcu_head *head) +static inline bool __rcu_reclaim (char *rn,struct rcu_head *head) {unsigned long offset = (unsigned long )head->func;if (__is_kfree_rcu_offset (offset)) {RCU_TRACE (trace_rcu_invoke_kfree_callback (rn, head, offset));kfree ((void *)head - offset); +return 1 ; }else {RCU_TRACE (trace_rcu_invoke_callback (rn, head)); head->func (head);
Lines of this file are not shown. View the raw message.
kernel/rcutiny_plugin.h +60 -17
77 lines not on this page · raw message
kernel/rcutorture.c +86 -5
91 lines not on this page · raw message
kernel/rcutree.c +371 -136
507 lines not on this page · raw message
kernel/rcutree_plugin.h +276 -174
450 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
Quoted 4 lines · Ingo Molnar
Please pull the latest core-rcu-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitcore-rcu-for-linus
So Ingo, Paul - I'd really have liked to have some high-level user-readable explanation of the changes that I could have put in the merge message.
This is especially true for something like RCU, which (a) seems to be continually growing and (b) has had cases of "oops, now it stopped working" or "uhhuh, new rules due to new rcu implementations interacting differently with spinlocks" etc.
So when the diffstat summary looks like this:
29 files changed, 2965 insertions(+), 642 deletions(-)
with basically 2000+ new lines (yes, many of them in Documebntation, happily, but rcutree grew noticeably too), it really would have been nice to have some blurb about what the heck actually happened, ok?
I'm starting to get explanations in signed tags or in the "please pull" messages from a fairly large number of maintainers now, and the pulls from Ingo are beginning to look sad and unexplained in contrast.
Linus
Quoted 15 lines · Linus Torvalds
Quoted 4 lines · Ingo Molnar
Please pull the latest core-rcu-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitcore-rcu-for-linusSo Ingo, Paul - I'd really have liked to have some high-level user-readable explanation of the changes that I could have put in the merge message.
This is especially true for something like RCU, which (a) seems to be continually growing and (b) has had cases of "oops, now it stopped working" or "uhhuh, new rules due to new rcu implementations interacting differently with spinlocks" etc.
Quoted 7 lines
So when the diffstat summary looks like this:
29 files changed, 2965 insertions(+), 642 deletions(-)with basically 2000+ new lines (yes, many of them in Documebntation, happily, but rcutree grew noticeably too), it really would have been nice to have some blurb about what the heck actually happened, ok?
Yep, 2072 of the net additions were in Documentation.
Quoted 3 lines
I'm starting to get explanations in signed tags or in the "please pull" messages from a fairly large number of maintainers now, and the pulls from Ingo are beginning to look sad and unexplained in contrast.
Many of the change to rcutree.c were due to additions of checks for illegal use of RCU from idle or offlined CPUs, reworking of the CPU-offline code path, and fixes to allow use of RCU from idle CPUs (which has always been illegal, but many uses have nevertheless slipped in over the years).
If it would help, I would be happy to put together an itemized list. I will of course do so for the next merge window.
Thanx, Paul
Quoted 10 lines · Paul E. McKenney
Quoted 9 lines · Linus Torvalds
Quoted 4 lines · Ingo Molnar
Please pull the latest core-rcu-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.gitcore-rcu-for-linusSo Ingo, Paul - I'd really have liked to have some high-level user-readable explanation of the changes that I could have put in the merge message.
Yeah, good point - I distilled that list in the merge commit itself (see commit bdd4431c8d071491a68a65d9457996f222b5ecd3) - should have exported that to the pull request itself:
Quoted 1 line
From
bdd4431c8d071491a68a65d9457996f222b5ecd3Mon Sep 17 00:00:00 2001
From: Ingo Molnar <[email protected]>
Date: Tue, 28 Feb 2012 10:16:01 +0100
Subject: [PATCH] Merge branch 'rcu/next' of
git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu
The major features of this series are:
- making RCU more aggressive about entering dyntick-idle mode in order to improve energy efficiency
- converting a few more call_rcu()s to kfree_rcu()s
- applying a number of rcutree fixes and cleanups to rcutiny
- removing CONFIG_SMP #ifdefs from treercu
- allowing RCU CPU stall times to be set via sysfs
- adding CPU-stall capability to rcutorture
- adding more RCU-abuse diagnostics
- updating documentation
- fixing yet more issues located by the still-ongoing top-to-bottom inspection of RCU, this time with a special focus on the CPU-hotplug code path.
Thanks,
Ingo
Quoted 3 lines · Paul E. McKenney
If it would help, I would be happy to put together an itemized list. I will of course do so for the next merge window.
Ok. I would like to get an itemized list next time, now I ended up re-doing the merges with the one Ingo sent me.
However, looking at the current state of RCU, the thing I would *really* like to *finally* be fixed is that total disaster called __rcu_read_lock() (and to a lesser degree __rcu_read_unlock).
Why do I call it a total disaster?
Simple: there are two versions of that function (not counting the inlined trivial non-preempt version that just disables preemption), AND THEY ARE BOTH IDENTICAL.
More importantly, they are both IDENTICALLY BAD.
They are crap because:
- they shouldn't be out-of-line to begin with.
Doing a function call for these things is stupid. It's not like
it's even "oh, there are two versions of it, so the linker picks one over the other". Sure, there are two versions of it, but they are the same stupid code just duplicated.
- the rcu counter should be a per-cpu counter, not a per-thread one
Right now that function ends up being two instructions:
mov %gs:0xb700,%rax
incl 0x100(%rax)and dammit, using a function call to do that is pretty much borderline. But it should be *one* instruction that just increments the percpu variable:
incl %gs:rcu_read_lock_nesting
and it shouldn't be out-of-line.
Because wouldn't it be nice to just make the scheduler save/restore the rcu read lock depth for the rcu preemption case.
Yeah, we should do the same thing with the preempt count. It shouldn't be in the thread structure, it should be per-cpu.
Please? Every time I look at some profiles, that silly rcu_read_lock is there in the profile. It's annoying. I'd rather see it in the function that invokes it.
Linus
Quoted 8 lines · Linus Torvalds
Quoted 3 lines · Paul E. McKenney
If it would help, I would be happy to put together an itemized list. I will of course do so for the next merge window.
Ok. I would like to get an itemized list next time, now I ended up re-doing the merges with the one Ingo sent me.
Will do!
Quoted 32 lines
However, looking at the current state of RCU, the thing I would *really* like to *finally* be fixed is that total disaster called
__rcu_read_lock()(and to a lesser degree__rcu_read_unlock).Why do I call it a total disaster?
Simple: there are two versions of that function (not counting the inlined trivial non-preempt version that just disables preemption), AND THEY ARE BOTH IDENTICAL.
More importantly, they are both IDENTICALLY BAD.
They are crap because:
- they shouldn't be out-of-line to begin with.
Doing a function call for these things is stupid. It's not like
it's even "oh, there are two versions of it, so the linker picks one over the other". Sure, there are two versions of it, but they are the same stupid code just duplicated.- the rcu counter should be a per-cpu counter, not a per-thread one
Right now that function ends up being two instructions:
mov %gs:0xb700,%rax incl 0x100(%rax)and dammit, using a function call to do that is pretty much borderline. But it should be *one* instruction that just increments the percpu variable:
Quoted 6 lines
incl %gs:rcu_read_lock_nesting
and it shouldn't be out-of-line.
Because wouldn't it be nice to just make the scheduler save/restore the rcu read lock depth for the rcu preemption case.
Good point, especially given that there already is an RCU hook in the scheduler entry path that could be used for this purpose, namely rcu_note_context_switch(). I would need a hook in the scheduler exit path. My guess is that this should go right after post_schedule(). (Or am I missing an early exit in there somewhere, Peter?)
The nice thing about this is that it avoids include-file issues.
Other architectures would have a load-add-store sequence, but assuming that no RCU read-side critical sections in the scheduler extend over the rcu_note_context_switch() and rcu_note_context_switch_end() (or whatever), this should be OK -- the value will be properly saved and restored.
I don't believe that I would have thought of this approach, so I don't feel quite as guilty about being lazy on inlining __rcu_read_lock() and __rcu_read_unlock(). Thank you!
Quoted 6 lines
Yeah, we should do the same thing with the preempt count. It shouldn't be in the thread structure, it should be per-cpu.
Please? Every time I look at some profiles, that silly
rcu_read_lockis there in the profile. It's annoying. I'd rather see it in the function that invokes it.
Let me see what I can do...
Thanx, Paul
Quoted 6 lines · Paul E. McKenney
Quoted 4 lines
Please? Every time I look at some profiles, that silly
rcu_read_lockis there in the profile. It's annoying. I'd rather see it in the function that invokes it.Let me see what I can do...
Thanks. To some degree, rcu_read_lock() is the more critical one, because it is often in the much more critical path in the caller. In particular, it's often at the beginning of a function, where a number of arguments are "live", and calling it out-of-line also forces the compiler to then save/restore those arguments (because they are clobbered by the function call).
rcu_read_unlock() is *usually* not as critical, and is obviously much harder to inline anyway due to the whole complexity with needing to check if an RCU sequence has ended. It often is at the end of the function call in the caller, when the only thing like is often just the single return value (if that). So it seldom looks nearly as bad in any profiles, because it doesn't tend to have the same kind of bad impact on the call site.
Linus
Quoted 16 lines · Linus Torvalds
Quoted 6 lines · Paul E. McKenney
Quoted 4 lines
Please? Every time I look at some profiles, that silly
rcu_read_lockis there in the profile. It's annoying. I'd rather see it in the function that invokes it.Let me see what I can do...
Thanks. To some degree,
rcu_read_lock()is the more critical one, because it is often in the much more critical path in the caller. In particular, it's often at the beginning of a function, where a number of arguments are "live", and calling it out-of-line also forces the compiler to then save/restore those arguments (because they are clobbered by the function call).
Quoted 7 lines
rcu_read_unlock()is *usually* not as critical, and is obviously much harder to inline anyway due to the whole complexity with needing to check if an RCU sequence has ended. It often is at the end of the function call in the caller, when the only thing like is often just the single return value (if that). So it seldom looks nearly as bad in any profiles, because it doesn't tend to have the same kind of bad impact on the call site.
Very good to hear! Especially since I am not seeing how to move ->rcu_read_unlock_special to a per-CPU variable given that rcu_boost() needs cross-task access to it. There is probably some obvious trick, but I will start with just __rcu_read_lock() for now.
Thanx, Paul
Quoted 25 lines · Paul E. McKenney
Quoted 16 lines · Linus Torvalds
Quoted 6 lines · Paul E. McKenney
Quoted 4 lines
Please? Every time I look at some profiles, that silly
rcu_read_lockis there in the profile. It's annoying. I'd rather see it in the function that invokes it.Let me see what I can do...
Thanks. To some degree,
rcu_read_lock()is the more critical one, because it is often in the much more critical path in the caller. In particular, it's often at the beginning of a function, where a number of arguments are "live", and calling it out-of-line also forces the compiler to then save/restore those arguments (because they are clobbered by the function call).Quoted 7 lines
rcu_read_unlock()is *usually* not as critical, and is obviously much harder to inline anyway due to the whole complexity with needing to check if an RCU sequence has ended. It often is at the end of the function call in the caller, when the only thing like is often just the single return value (if that). So it seldom looks nearly as bad in any profiles, because it doesn't tend to have the same kind of bad impact on the call site.
Quoted 4 lines
Very good to hear! Especially since I am not seeing how to move ->rcu_read_unlock_special to a per-CPU variable given that
rcu_boost()needs cross-task access to it. There is probably some obvious trick, but I will start with just__rcu_read_lock()for now.
And one obvious trick is a per-CPU pointer to the task-structure variable. But __rcu_read_lock() first.
Thanx, Paul
Quoted 26 lines · Paul E. McKenney
Quoted 25 lines · Paul E. McKenney
Quoted 23 lines · Linus Torvalds
Quoted 6 lines · Paul E. McKenney
Quoted 4 lines
Please? Every time I look at some profiles, that silly
rcu_read_lockis there in the profile. It's annoying. I'd rather see it in the function that invokes it.Let me see what I can do...
Thanks. To some degree,
rcu_read_lock()is the more critical one, because it is often in the much more critical path in the caller. In particular, it's often at the beginning of a function, where a number of arguments are "live", and calling it out-of-line also forces the compiler to then save/restore those arguments (because they are clobbered by the function call).
rcu_read_unlock()is *usually* not as critical, and is obviously much harder to inline anyway due to the whole complexity with needing to check if an RCU sequence has ended. It often is at the end of the function call in the caller, when the only thing like is often just the single return value (if that). So it seldom looks nearly as bad in any profiles, because it doesn't tend to have the same kind of bad impact on the call site.
Quoted 7 lines
Quoted 4 lines
Very good to hear! Especially since I am not seeing how to move ->rcu_read_unlock_special to a per-CPU variable given that
rcu_boost()needs cross-task access to it. There is probably some obvious trick, but I will start with just__rcu_read_lock()for now.And one obvious trick is a per-CPU pointer to the task-structure variable. But
__rcu_read_lock()first.
And here is a preliminary patch with some known bugs (probably among others), but useful for checking the __rcu_read_lock() code generation.
The known bugs include one where ->rcu_read_unlock_special can get paired with the wrong call to rcu_read_unlock_special(), which can happen due to the scheduler using RCU read-side primitives. (I am currently leaning towards an "ignore me" flag in ->rcu_read_unlock_special, but will see how it goes.) Another possible bug could occur if the scheduler enables preemption on any code path reachable by __schedule().
Thoughts on code generation while I track down the other bugs?
This is what I get on a 32-bit build, according to objdump:
24e3: 64 ff 05 00 00 00 00 incl %fs:0x0 Thanx, Paul
------------------------------------------------------------------------
rcu: Make __rcu_read_lock() inlinable
The preemptible-RCU implementations of __rcu_read_lock() have not been inlinable due to task_struct references that ran afoul of include-file dependencies. Fix this by moving the task_struct ->rcu_read_lock_nesting field to a per-CPU variable that is saved and restored at context-switch time. With this change, __rcu_read_lock() references only that new per-CPU variable, and so may be safely inlined. This change also allows some code that was duplicated in kernel/rcutree_plugin.h and kernel/rcutiny_plugin.h to be merged into include/linux/rcupdate.h.
This same approach unfortunately cannot be used on __rcu_read_unlock() because it also references the task_struct ->rcu_read_unlock_special field, to which cross-task access is required by rcu_boost(). This function will be handled in a separate commit, if need be.
include/linux/init_task.h +1 -1
@@ -105,7 +105,7 @@ extern struct group_info init_groups;#endif #ifdef CONFIG_PREEMPT_RCU #define INIT_TASK_RCU_PREEMPT (tsk) \ - .rcu_read_lock_nesting = 0, \ + .rcu_read_lock_nesting_save = 0, \ .rcu_read_unlock_special = 0, \ .rcu_node_entry = LIST_HEAD_INIT(tsk.rcu_node_entry), \ INIT_TASK_RCU_TREE_PREEMPT() \
include/linux/rcupdate.h +51 -2
74 lines not on this page · raw message
include/linux/rcutiny.h +0 -5
@@ -91,10 +91,6 @@ static inline void rcu_preempt_note_context_switch(void) { } -static inline void exit_rcu (void ) -{ -} -static inline int rcu_needs_cpu (int cpu) {return 0 ;
@@ -103,7 +99,6 @@ static inline int rcu_needs_cpu(int cpu)#else /* #ifdef CONFIG_TINY_RCU */ void rcu_preempt_note_context_switch (void ); -extern void exit_rcu (void );int rcu_preempt_needs_cpu (void );static inline int rcu_needs_cpu (int cpu)
include/linux/rcutree.h +0 -12
@@ -45,18 +45,6 @@ static inline void rcu_virt_note_context_switch(int cpu)rcu_note_context_switch (cpu); } -#ifdef CONFIG_TREE_PREEMPT_RCU - -extern void exit_rcu (void ); - -#else /* #ifdef CONFIG_TREE_PREEMPT_RCU */ - -static inline void exit_rcu (void ) -{ -} - -#endif /* #else #ifdef CONFIG_TREE_PREEMPT_RCU */ -extern void synchronize_rcu_bh (void );extern void synchronize_sched_expedited (void );extern void synchronize_rcu_expedited (void );
include/linux/sched.h +2 -2
@@ -1275,7 +1275,7 @@ struct task_struct {cpumask_t cpus_allowed;#ifdef CONFIG_PREEMPT_RCU -int rcu_read_lock_nesting; +int rcu_read_lock_nesting_save;char rcu_read_unlock_special;struct list_head rcu_node_entry;#endif /* #ifdef CONFIG_PREEMPT_RCU */
@@ -1868,7 +1868,7 @@ extern void task_clear_jobctl_pending(struct task_struct *task,static inline void rcu_copy_process (struct task_struct *p) { - p->rcu_read_lock_nesting =0 ; + p->rcu_read_lock_nesting_save =0 ; p->rcu_read_unlock_special =0 ;#ifdef CONFIG_TREE_PREEMPT_RCU p->rcu_blocked_node =NULL ;
kernel/rcupdate.c +4 -0
@@ -51,6 +51,10 @@#include "rcu.h" +#ifdef CONFIG_PREEMPT_RCU +DEFINE_PER_CPU (int, rcu_read_lock_nesting); +#endif /* #ifdef CONFIG_PREEMPT_RCU */ +#ifdef CONFIG_DEBUG_LOCK_ALLOC static struct lock_class_key rcu_lock_key; struct lockdep_map rcu_lock_map =
kernel/rcutiny_plugin.h +16 -49
112 lines not on this page · raw message
kernel/rcutree_plugin.h +20 -34
117 lines not on this page · raw message
kernel/sched/core.c +2 -0
@@ -3221,6 +3221,8 @@ need_resched:post_schedule (rq); +rcu_note_context_switch_end (); +preempt_enable_no_resched ();if (need_resched ()) goto need_resched;
Quoted 4 lines · Paul E. McKenney
This is what I get on a 32-bit build, according to objdump:
24e3: 64 ff 05 00 00 00 00 incl %fs:0x0
You do? I'm surprised:
include/linux/rcupdate.h extern void synchronize_sched(void); · line 158
Quoted 5 lines
include/linux/rcupdate.h +5
-0+static inline void __rcu_read_lock (void ) +{ + __raw_get_cpu_var(rcu_read_lock_nesting)++; + barrier(); /* Keep code within RCU read-side critical section. */ +}
This looks wrong.
It should use "this_cpu_inc(rcu_read_lock_nesting)", I don't even see how you get gcc to generate that "inc %fs:" without it.
In general, I don't think you should ever use the __raw_get_cpu_var() things.
Linus
Quoted 8 lines · Linus Torvalds
Quoted 4 lines · Paul E. McKenney
This is what I get on a 32-bit build, according to objdump:
24e3: 64 ff 05 00 00 00 00 incl %fs:0x0
You do? I'm surprised:
That would be because I forgot the needed "git commit --amend" before doing "git show HEAD". :-/
See below for the patch including the most recent changes.
Quoted 8 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+static inline void __rcu_read_lock(void) +{ + __raw_get_cpu_var(rcu_read_lock_nesting)++; + barrier(); /* Keep code within RCU read-side critical section. */ +}This looks wrong.
Quoted 4 lines
It should use "
this_cpu_inc(rcu_read_lock_nesting)", I don't even see how you get gcc to generate that "inc %fs:" without it.In general, I don't think you should ever use the
__raw_get_cpu_var()things.
I must admit that __this_cpu_inc() would be nicer than __this_cpu_add(), though, will fix. I need the leading "__" to avoid disabling preemption needlessly on non-x86 platforms. The reason that the "__raw" forms are safe in this case is because the per-CPU variable is saved and restored at context-switch time.
Or am I still missing something here?
Thanx, Paul
------------------------------------------------------------------------
include/linux/init_task.h +1 -1
@@ -105,7 +105,7 @@ extern struct group_info init_groups;#endif #ifdef CONFIG_PREEMPT_RCU #define INIT_TASK_RCU_PREEMPT (tsk) \ - .rcu_read_lock_nesting = 0, \ + .rcu_read_lock_nesting_save = 0, \ .rcu_read_unlock_special = 0, \ .rcu_node_entry = LIST_HEAD_INIT(tsk.rcu_node_entry), \ INIT_TASK_RCU_TREE_PREEMPT() \
include/linux/rcupdate.h +52 -2
81 lines not on this page · raw message
include/linux/rcutiny.h +0 -5
@@ -91,10 +91,6 @@ static inline void rcu_preempt_note_context_switch(void) { } -static inline void exit_rcu (void ) -{ -} -static inline int rcu_needs_cpu (int cpu) {return 0 ;
@@ -103,7 +99,6 @@ static inline int rcu_needs_cpu(int cpu)#else /* #ifdef CONFIG_TINY_RCU */ void rcu_preempt_note_context_switch (void ); -extern void exit_rcu (void );int rcu_preempt_needs_cpu (void );static inline int rcu_needs_cpu (int cpu)
include/linux/rcutree.h +0 -12
@@ -45,18 +45,6 @@ static inline void rcu_virt_note_context_switch(int cpu)rcu_note_context_switch (cpu); } -#ifdef CONFIG_TREE_PREEMPT_RCU - -extern void exit_rcu (void ); - -#else /* #ifdef CONFIG_TREE_PREEMPT_RCU */ - -static inline void exit_rcu (void ) -{ -} - -#endif /* #else #ifdef CONFIG_TREE_PREEMPT_RCU */ -extern void synchronize_rcu_bh (void );extern void synchronize_sched_expedited (void );extern void synchronize_rcu_expedited (void );
include/linux/sched.h +2 -2
@@ -1275,7 +1275,7 @@ struct task_struct {cpumask_t cpus_allowed;#ifdef CONFIG_PREEMPT_RCU -int rcu_read_lock_nesting; +int rcu_read_lock_nesting_save;char rcu_read_unlock_special;struct list_head rcu_node_entry;#endif /* #ifdef CONFIG_PREEMPT_RCU */
@@ -1868,7 +1868,7 @@ extern void task_clear_jobctl_pending(struct task_struct *task,static inline void rcu_copy_process (struct task_struct *p) { - p->rcu_read_lock_nesting =0 ; + p->rcu_read_lock_nesting_save =0 ; p->rcu_read_unlock_special =0 ;#ifdef CONFIG_TREE_PREEMPT_RCU p->rcu_blocked_node =NULL ;
kernel/rcupdate.c +4 -0
@@ -51,6 +51,10 @@#include "rcu.h" +#ifdef CONFIG_PREEMPT_RCU +DEFINE_PER_CPU (int, rcu_read_lock_nesting); +#endif /* #ifdef CONFIG_PREEMPT_RCU */ +#ifdef CONFIG_DEBUG_LOCK_ALLOC static struct lock_class_key rcu_lock_key; struct lockdep_map rcu_lock_map =
kernel/rcutiny_plugin.h +18 -49
114 lines not on this page · raw message
kernel/rcutree_plugin.h +23 -34
120 lines not on this page · raw message
kernel/sched/core.c +2 -0
@@ -3221,6 +3221,8 @@ need_resched:post_schedule (rq); +rcu_note_context_switch_end (); +preempt_enable_no_resched ();if (need_resched ()) goto need_resched;
Quoted 4 lines · Paul E. McKenney
I must admit that
__this_cpu_inc()would be nicer than__this_cpu_add(), though, will fix. I need the leading "__" to avoid disabling preemption needlessly on non-x86 platforms.
Yeah, that's just bogus. Ok on that.
Quoted 5 lines
The reason that the "
__raw" forms are
safe in this case is because the per-CPU variable is saved and restored at context-switch time.Or am I still missing something here?
It's not that the "__raw" forms are "safe". It's that they are SH*T.
Don't use them. They are crap. Why would you do
+ __raw_get_cpu_var(rcu_read_lock_nesting) =
+ current->rcu_read_lock_nesting_save;
which is just crazy and cannot use the actual sane "%fs:" segment overrides, but instead has to do idiotic "ready the per-cpu offset pointer and add it in".
We've got "__this_cpu_write()" which generates the correct code.
Rule of thumb: there is _never_ any good reason to use __raw_get_cpu_var. It's a broken interface.
Linus
Quoted 22 lines · Linus Torvalds
Quoted 4 lines · Paul E. McKenney
I must admit that
__this_cpu_inc()would be nicer than__this_cpu_add(), though, will fix. I need the leading "__" to avoid disabling preemption needlessly on non-x86 platforms.Yeah, that's just bogus. Ok on that.
Quoted 5 lines
The reason that the "
__raw" forms are
safe in this case is because the per-CPU variable is saved and restored at context-switch time.Or am I still missing something here?
It's not that the "
__raw" forms are "safe". It's that they are SH*T.Don't use them. They are crap. Why would you do
(quoted diff) +2
-0+ __raw_get_cpu_var(rcu_read_lock_nesting) = + current->rcu_read_lock_nesting_save;
Quoted 8 lines
which is just crazy and cannot use the actual sane "%fs:" segment overrides, but instead has to do idiotic "ready the per-cpu offset pointer and add it in".
We've got "
__this_cpu_write()" which generates the correct code.Rule of thumb: there is _never_ any good reason to use
__raw_get_cpu_var. It's a broken interface.
OK, I will switch to __this_cpu_write(), thank you for the tip.
Thanx, Paul