RCU changes for v4.3
Subsystems (21)
Linus,
Please pull the latest core-rcu-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-rcu-for-linus
# HEAD: f612a7b1a7f1b5139f228724ce340aac24720591 Merge branch 'for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcu
The main RCU changes in this cycle are:
- The combination of tree geometry-initialization simplifications and OS-jitter-reduction changes to expedited grace periods. These two are stacked due to the large number of conflicts that would otherwise result.
- Privatize smp_mb__after_unlock_lock(). This commit moves the definition of smp_mb__after_unlock_lock() to kernel/rcu/tree.h, in recognition of the fact that RCU is the only thing using this, that nothing else is likely to use it, and that it is likely to go away completely.
- Documentation updates.
- Torture-test updates.
- Misc fixes.
Thanks,
Ingo
------------------>
Alexander Gordeev (10):
rcu: Panic if RCU tree can not accommodate all CPUs
rcu: Remove superfluous local variable in rcu_init_geometry()
rcu: Cleanup rcu_init_geometry() code and arithmetics
rcu: Simplify rcu_init_geometry() capacity arithmetics
rcu: Limit rcu_state::levelcnt[] to RCU_NUM_LVLS items
rcu: Limit rcu_capacity[] size to RCU_NUM_LVLS items
rcu: Remove unnecessary fields from rcu_state structure
rcu: Limit count of static data to the number of RCU levels
rcu: Simplify arithmetic to calculate number of RCU nodes
rcu: Shut up bogus gcc array bounds warning
Alexei Starovoitov (1):
rcu: Make rcu_is_watching() really notrace
Denys Vlasenko (1):
rcu: Deinline rcu_read_lock_sched_held() if DEBUG_LOCK_ALLOC
Lai Jiangshan (1):
rcu: Update MAINTAINERS entry
Nicholas Mc Guire (1):
rcu: Change return type to bool
Paul E. McKenney (43):
rcu: Drop RCU_USER_QS in favor of NO_HZ_FULL
documentation: Bring rcutorture parameters up to date
documentation: Fix spelling of "operators"
documentation: Fix variable-name typo in memory-barriers.txt
documentation: Replace ACCESS_ONCE() by READ_ONCE() and WRITE_ONCE()
doc: Call out smp_mb__after_unlock_lock() transitivity
rcu: Provide more diagnostics for stalled GP kthread
rcutorture: Better bounds checking for n_barrier_cbs
rcutorture: Check nfakewriters parameter
rcutorture: Bounds-check rcutorture.shuffle_interval
rcutorture: Fix rcu_torture_cbflood() for callback-free RCU
rcutorture: Add RCU-tasks qualifier to dereference
rcutorture: Enable lockdep-RCU on TASKS01
rcu: Reset rcu_fanout_leaf if out of bounds
rcu: Stop disabling CPU hotplug in synchronize_rcu_expedited()
rcu: Remove CONFIG_RCU_CPU_STALL_INFO
rcu: Rework synchronize_sched_expedited() counter handling
rcu: Get rid of synchronize_sched_expedited()'s polling loop
rcu: Abstract sequence counting from synchronize_sched_expedited()
rcu: Make synchronize_rcu_expedited() use sequence-counter scheme
rcu: Abstract funnel locking from synchronize_sched_expedited()
rcu: Fix synchronize_sched_expedited() type error for "s"
rcu: Use funnel locking for synchronize_rcu_expedited()'s polling loop
rcu: Apply rcu_seq operations to _rcu_barrier()
rcu: Consolidate last open-coded expedited memory barrier
rcu: Extend expedited funnel locking to rcu_data structure
rcu: Add stall warnings to synchronize_sched_expedited()
documentation: Describe new expedited stall warnings
rcu: Pull out wait_event*() condition into helper function
rcu: Rename RCU_GP_DONE_FQS to RCU_GP_DOING_FQS
rcu: Add fastpath bypassing funnel locking
rcu: Add RCU-sched flavors of get-state and cond-sync
rcu: Hide RCU_NOCB_CPU behind RCU_EXPERT
rcu: Fix obsolete priority-boosting comment
rcu: Create a synchronize_rcu_mult()
cpu: Wait for RCU grace periods concurrently
rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()
rcu: Fix backwards RCU_LOCKDEP_WARN() in synchronize_rcu_tasks()
rcu: Clarify CONFIG_RCU_EQS_DEBUG help text
scripts: Make checkpatch.pl warn on expedited RCU grace periods
rcu: Don't disable CPU hotplug during OOM notifiers
rcu: Silence lockdep false positive for expedited grace periods rcu,locking: Privatize smp_mb__after_unlock_lock()
Peter Zijlstra (3):
rcu: Switch synchronize_sched_expedited() to stop_one_cpu()
rcu: Make expedited GP CPU stoppage asynchronous
rcu: Use WRITE_ONCE in RCU_INIT_POINTER
Documentation/RCU/rcu_dereference.txt | 2 +- Documentation/RCU/stallwarn.txt | 29 +- Documentation/RCU/trace.txt | 36 +- Documentation/RCU/whatisRCU.txt | 2 +- Documentation/kernel-parameters.txt | 35 +- Documentation/memory-barriers.txt | 404 ++++++------ MAINTAINERS | 6 +- arch/powerpc/include/asm/spinlock.h | 2 - arch/x86/kernel/cpu/mcheck/mce.c | 6 +- arch/x86/kernel/traps.c | 2 +- drivers/base/power/opp.c | 4 +- include/linux/fdtable.h | 4 +- include/linux/rcupdate.h | 144 +++-- include/linux/rcutiny.h | 10 + include/linux/rcutree.h | 2 + include/linux/spinlock.h | 10 - include/linux/types.h | 3 + include/trace/events/rcu.h | 1 - init/Kconfig | 10 +- kernel/cgroup.c | 4 +- kernel/cpu.c | 10 +- kernel/pid.c | 5 +- kernel/rcu/rcutorture.c | 42 +- kernel/rcu/srcu.c | 15 +- kernel/rcu/tiny.c | 8 +- kernel/rcu/tree.c | 681 ++++++++++++--------- kernel/rcu/tree.h | 96 ++- kernel/rcu/tree_plugin.h | 130 +--- kernel/rcu/tree_trace.c | 19 +- kernel/rcu/update.c | 90 ++- kernel/sched/core.c | 8 +- kernel/time/Kconfig | 2 - kernel/workqueue.c | 20 +- lib/Kconfig.debug | 16 +- scripts/checkpatch.pl | 19 + security/device_cgroup.c | 6 +- .../selftests/rcutorture/configs/rcu/TASKS01 | 4 +- .../selftests/rcutorture/configs/rcu/TREE01 | 1 - .../selftests/rcutorture/configs/rcu/TREE02 | 1 - .../selftests/rcutorture/configs/rcu/TREE02-T | 1 - .../selftests/rcutorture/configs/rcu/TREE03 | 1 - .../selftests/rcutorture/configs/rcu/TREE04 | 1 - .../selftests/rcutorture/configs/rcu/TREE05 | 1 - .../selftests/rcutorture/configs/rcu/TREE06 | 1 - .../selftests/rcutorture/configs/rcu/TREE07 | 1 - .../selftests/rcutorture/configs/rcu/TREE08 | 1 - .../selftests/rcutorture/configs/rcu/TREE08-T | 1 - .../selftests/rcutorture/configs/rcu/TREE09 | 1 - .../selftests/rcutorture/doc/TREE_RCU-kconfig.txt | 1 - 49 files changed, 992 insertions(+), 907 deletions(-)
Documentation/RCU/rcu_dereference.txt +1 -1
@@ -28,7 +28,7 @@ o You must use one of the rcu_dereference() family of primitives o Avoid cancellation when using the "+" and "-" infix arithmetic operators. For example, for a given variable "x", avoid "(x-x)". There are similar arithmetic pitfalls from other - arithmetic operatiors, such as "(x*0)", "(x/(x+1))" or "(x%1)". + arithmetic operators, such as "(x*0)", "(x/(x+1))" or "(x%1)". The compiler is within its rights to substitute zero for all of these expressions, so that subsequent accesses no longer depend on the rcu_dereference(), again possibly resulting in bugs due
Documentation/RCU/stallwarn.txt +19 -10
@@ -26,12 +26,6 @@ CONFIG_RCU_CPU_STALL_TIMEOUT Stall-warning messages may be enabled and disabled completely via /sys/module/rcupdate/parameters/rcu_cpu_stall_suppress. -CONFIG_RCU_CPU_STALL_INFO - - This kernel configuration parameter causes the stall warning to - print out additional per-CPU diagnostic information, including - information on scheduling-clock ticks and RCU's idle-CPU tracking. - RCU_STALL_DELAY_DELTA Although the lockdep facility is extremely useful, it does add
@@ -101,15 +95,13 @@ interact. Please note that it is not possible to entirely eliminate this sort of false positive without resorting to things like stop_machine(), which is overkill for this sort of problem. -If the CONFIG_RCU_CPU_STALL_INFO kernel configuration parameter is set, -more information is printed with the stall-warning message, for example: +Recent kernels will print a long form of the stall-warning message: INFO: rcu_preempt detected stall on CPU 0: (63959 ticks this GP) idle=241/3fffffffffffffff/0 softirq=82/543 (t=65000 jiffies) -In kernels with CONFIG_RCU_FAST_NO_HZ, even more information is -printed: +In kernels with CONFIG_RCU_FAST_NO_HZ, more information is printed: INFO: rcu_preempt detected stall on CPU 0: (64628 ticks this GP) idle=dd5/3fffffffffffffff/0 softirq=82/543 last_accelerate: a345/d342 nonlazy_posted: 25 .D
@@ -171,6 +163,23 @@ message will be about three times the interval between the beginning of the stall and the first message. +Stall Warnings for Expedited Grace Periods + +If an expedited grace period detects a stall, it will place a message +like the following in dmesg: + + INFO: rcu_sched detected expedited stalls on CPUs: { 1 2 6 } 26009 jiffies s: 1043 + +This indicates that CPUs 1, 2, and 6 have failed to respond to a +reschedule IPI, that the expedited grace period has been going on for +26,009 jiffies, and that the expedited grace-period sequence counter is +1043. The fact that this last value is odd indicates that an expedited +grace period is in flight. + +It is entirely possible to see stall warnings from normal and from +expedited grace periods at about the same time from the same run. + + What Causes RCU CPU Stall Warnings? So your kernel printed an RCU CPU stall warning. The next question is
Documentation/RCU/trace.txt +10 -26
@@ -237,42 +237,26 @@ o "ktl" is the low-order 16 bits (in hexadecimal) of the count of The output of "cat rcu/rcu_preempt/rcuexp" looks as follows: -s=21872 d=21872 w=0 tf=0 wd1=0 wd2=0 n=0 sc=21872 dt=21872 dl=0 dx=21872 +s=21872 wd0=0 wd1=0 wd2=0 wd3=5 n=0 enq=0 sc=21872 These fields are as follows: -o "s" is the starting sequence number. +o "s" is the sequence number, with an odd number indicating that + an expedited grace period is in progress. -o "d" is the ending sequence number. When the starting and ending - numbers differ, there is an expedited grace period in progress. - -o "w" is the number of times that the sequence numbers have been - in danger of wrapping. - -o "tf" is the number of times that contention has resulted in a - failure to begin an expedited grace period. - -o "wd1" and "wd2" are the number of times that an attempt to - start an expedited grace period found that someone else had - completed an expedited grace period that satisfies the +o "wd0", "wd1", "wd2", and "wd3" are the number of times that an + attempt to start an expedited grace period found that someone + else had completed an expedited grace period that satisfies the attempted request. "Our work is done." -o "n" is number of times that contention was so great that - the request was demoted from an expedited grace period to - a normal grace period. +o "n" is number of times that a concurrent CPU-hotplug operation + forced a fallback to a normal grace period. + +o "enq" is the number of quiescent states still outstanding. o "sc" is the number of times that the attempt to start a new expedited grace period succeeded. -o "dt" is the number of times that we attempted to update - the "d" counter. - -o "dl" is the number of times that we failed to update the "d" - counter. - -o "dx" is the number of times that we succeeded in updating - the "d" counter. - The output of "cat rcu/rcu_preempt/rcugp" looks as follows:
Documentation/RCU/whatisRCU.txt +1 -1
@@ -883,7 +883,7 @@ All: lockdep-checked RCU-protected pointer access rcu_access_pointer rcu_dereference_raw - rcu_lockdep_assert + RCU_LOCKDEP_WARN rcu_sleep_check RCU_NONIDLE
Documentation/kernel-parameters.txt +24 -11
@@ -3135,22 +3135,35 @@ bytes respectively. Such letter suffixes can also be entirely omitted. in a given burst of a callback-flood test. rcutorture.fqs_duration= [KNL] - Set duration of force_quiescent_state bursts. + Set duration of force_quiescent_state bursts + in microseconds. rcutorture.fqs_holdoff= [KNL] - Set holdoff time within force_quiescent_state bursts. + Set holdoff time within force_quiescent_state bursts + in microseconds. rcutorture.fqs_stutter= [KNL] - Set wait time between force_quiescent_state bursts. + Set wait time between force_quiescent_state bursts + in seconds. + + rcutorture.gp_cond= [KNL] + Use conditional/asynchronous update-side + primitives, if available. rcutorture.gp_exp= [KNL] - Use expedited update-side primitives. + Use expedited update-side primitives, if available. rcutorture.gp_normal= [KNL] - Use normal (non-expedited) update-side primitives. - If both gp_exp and gp_normal are set, do both. - If neither gp_exp nor gp_normal are set, still - do both. + Use normal (non-expedited) asynchronous + update-side primitives, if available. + + rcutorture.gp_sync= [KNL] + Use normal (non-expedited) synchronous + update-side primitives, if available. If all + of rcutorture.gp_cond=, rcutorture.gp_exp=, + rcutorture.gp_normal=, and rcutorture.gp_sync= + are zero, rcutorture acts as if is interpreted + they are all non-zero. rcutorture.n_barrier_cbs= [KNL] Set callbacks/threads for rcu_barrier() testing.
@@ -3177,9 +3190,6 @@ bytes respectively. Such letter suffixes can also be entirely omitted. Set time (s) between CPU-hotplug operations, or zero to disable CPU-hotplug testing. - rcutorture.torture_runnable= [BOOT] - Start rcutorture running at boot time. - rcutorture.shuffle_interval= [KNL] Set task-shuffle interval (s). Shuffling tasks allows some CPUs to go into dyntick-idle mode
@@ -3220,6 +3230,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted. Test RCU's dyntick-idle handling. See also the rcutorture.shuffle_interval parameter. + rcutorture.torture_runnable= [BOOT] + Start rcutorture running at boot time. + rcutorture.torture_type= [KNL] Specify the RCU implementation to test.
Documentation/memory-barriers.txt +175 -229
755 lines not on this page · raw message
MAINTAINERS +3 -3
@@ -8472,7 +8472,7 @@ M: "Paul E. McKenney" <[email protected]> M: Josh Triplett <[email protected]> R: Steven Rostedt <[email protected]> R: Mathieu Desnoyers <[email protected]> -R: Lai Jiangshan <[email protected]> +R: Lai Jiangshan <[email protected]> L: [email protected] S: Supported T: git git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
@@ -8499,7 +8499,7 @@ M: "Paul E. McKenney" <[email protected]> M: Josh Triplett <[email protected]> R: Steven Rostedt <[email protected]> R: Mathieu Desnoyers <[email protected]> -R: Lai Jiangshan <[email protected]> +R: Lai Jiangshan <[email protected]> L: [email protected] W: http://www.rdrop.com/users/paulmck/RCU/ S: Supported
@@ -9367,7 +9367,7 @@ F: include/linux/sl?b*.h F: mm/sl?b* SLEEPABLE READ-COPY UPDATE (SRCU) -M: Lai Jiangshan <[email protected]> +M: Lai Jiangshan <[email protected]> M: "Paul E. McKenney" <[email protected]> M: Josh Triplett <[email protected]> R: Steven Rostedt <[email protected]>
arch/powerpc/include/asm/spinlock.h +0 -2
@@ -28,8 +28,6 @@#include <asm/synch.h> #include <asm/ppc-opcode.h> -#define smp_mb__after_unlock_lock () smp_mb()/* Full ordering for lock. */ -#ifdef CONFIG_PPC64 /* use 0x800000yy when locked, where yy == CPU number */ #ifdef __BIG_ENDIAN__
arch/x86/kernel/cpu/mcheck/mce.c +3 -3
@@ -54,9 +54,9 @@ static DEFINE_MUTEX(mce_chrdev_read_mutex);#define rcu_dereference_check_mce (p) \ ({ \ - rcu_lockdep_assert(rcu_read_lock_sched_held() || \ - lockdep_is_held(&mce_chrdev_read_mutex), \ - "suspicious rcu_dereference_check_mce() usage"); \ + RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held() && \ + !lockdep_is_held(&mce_chrdev_read_mutex), \ + "suspicious rcu_dereference_check_mce() usage"); \ smp_load_acquire(&(p)); \ })
arch/x86/kernel/traps.c +1 -1
@@ -136,7 +136,7 @@ enum ctx_state ist_enter(struct pt_regs *regs)preempt_count_add (HARDIRQ_OFFSET );/* This code is a bit fragile. Test it. */ -rcu_lockdep_assert (rcu_is_watching (),"ist_enter didn't work" ); +RCU_LOCKDEP_WARN (!rcu_is_watching (),"ist_enter didn't work" );return prev_state; }
drivers/base/power/opp.c +2 -2
@@ -110,8 +110,8 @@ static DEFINE_MUTEX(dev_opp_list_lock);#define opp_rcu_lockdep_assert () \ do { \ - rcu_lockdep_assert(rcu_read_lock_held() || \ - lockdep_is_held(&dev_opp_list_lock), \ + RCU_LOCKDEP_WARN(!rcu_read_lock_held() && \ + !lockdep_is_held(&dev_opp_list_lock), \ "Missing rcu_read_lock() or " \ "dev_opp_list_lock protection"); \ } while (0)
include/linux/fdtable.h +2 -2
@@ -86,8 +86,8 @@ static inline struct file *__fcheck_files(struct files_struct *files, unsigned istatic inline struct file *fcheck_files (struct files_struct *files,unsigned int fd) { -rcu_lockdep_assert (rcu_read_lock_held () || -lockdep_is_held (&files->file_lock), +RCU_LOCKDEP_WARN (!rcu_read_lock_held () && + !lockdep_is_held (&files->file_lock),"suspicious rcu_dereference_check() usage" );return __fcheck_files (files, fd); }
include/linux/rcupdate.h +78 -66
260 lines not on this page · raw message
include/linux/rcutiny.h +10 -0
@@ -37,6 +37,16 @@ static inline void cond_synchronize_rcu(unsigned long oldstate)might_sleep (); } +static inline unsigned long get_state_synchronize_sched (void ) +{ +return 0 ; +} + +static inline void cond_synchronize_sched (unsigned long oldstate) +{ +might_sleep (); +} +static inline void rcu_barrier_bh (void ) {wait_rcu_gp (call_rcu_bh);
include/linux/rcutree.h +2 -0
@@ -76,6 +76,8 @@ void rcu_barrier_bh(void);void rcu_barrier_sched (void );unsigned long get_state_synchronize_rcu (void );void cond_synchronize_rcu (unsigned long oldstate); +unsigned long get_state_synchronize_sched (void ); +void cond_synchronize_sched (unsigned long oldstate);extern unsigned long rcutorture_testseq;extern unsigned long rcutorture_vernum;
include/linux/spinlock.h +0 -10
@@ -130,16 +130,6 @@ do { \#define smp_mb__before_spinlock () smp_wmb()#endif -/* -* Place this after a lock-acquisition primitive to guarantee that -* an UNLOCK+LOCK pair act as a full barrier. This guarantee applies -* if the UNLOCK and LOCK are executed by the same CPU or if the -* UNLOCK and LOCK operate on the same lock variable. -*/ -#ifndef smp_mb__after_unlock_lock -#define smp_mb__after_unlock_lock () do { } while (0) -#endif -/** * raw_spin_unlock_wait - wait until the spinlock gets unlocked * @lock: the spinlock in question.
include/linux/types.h +3 -0
@@ -212,6 +212,9 @@ struct callback_head { };#define rcu_head callback_head +typedef void (*rcu_callback_t )(struct rcu_head *head); +typedef void (*call_rcu_func_t )(struct rcu_head *head,rcu_callback_t func); +/* clocksource cycle base type */ typedef u64 cycle_t ;
include/trace/events/rcu.h +0 -1
@@ -661,7 +661,6 @@ TRACE_EVENT(rcu_torture_read,* Tracepoint for _rcu_barrier() execution. The string "s" describes * the _rcu_barrier phase: * "Begin": _rcu_barrier() started. -* "Check": _rcu_barrier() checking for piggybacking. * "EarlyExit": _rcu_barrier() piggybacked, thus early exit. * "Inc1": _rcu_barrier() piggyback check counter incremented. * "OfflineNoCB": _rcu_barrier() found callback on never-online CPU
init/Kconfig +1 -9
@@ -538,15 +538,6 @@ config RCU_STALL_COMMON config CONTEXT_TRACKING bool -config RCU_USER_QS - bool - help - This option sets hooks on kernel / userspace boundaries and - puts RCU in extended quiescent state when the CPU runs in - userspace. It means that when a CPU runs in userspace, it is - excluded from the global RCU state machine and thus doesn't - try to keep the timer tick on for RCU. - config CONTEXT_TRACKING_FORCE bool "Force context tracking" depends on CONTEXT_TRACKING
@@ -707,6 +698,7 @@ config RCU_BOOST_DELAY config RCU_NOCB_CPU bool "Offload RCU callback processing from boot-selected CPUs" depends on TREE_RCU || PREEMPT_RCU + depends on RCU_EXPERT || NO_HZ_FULL default n help Use this option to reduce OS jitter for aggressive HPC or
kernel/cgroup.c +2 -2
@@ -107,8 +107,8 @@ static DEFINE_SPINLOCK(release_agent_path_lock);struct percpu_rw_semaphore cgroup_threadgroup_rwsem;#define cgroup_assert_mutex_or_rcu_locked () \ - rcu_lockdep_assert(rcu_read_lock_held() || \ - lockdep_is_held(&cgroup_mutex), \ + RCU_LOCKDEP_WARN(!rcu_read_lock_held() && \ + !lockdep_is_held(&cgroup_mutex), \ "cgroup_mutex or RCU read lock required");/*
kernel/cpu.c +5 -5
@@ -381,14 +381,14 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)* will observe it. * * For CONFIG_PREEMPT we have preemptible RCU and its sync_rcu() might -* not imply sync_sched(), so explicitly call both. +* not imply sync_sched(), so wait for both. * * Do sync before park smpboot threads to take care the rcu boost case. */ -#ifdef CONFIG_PREEMPT -synchronize_sched (); -#endif -synchronize_rcu (); +if (IS_ENABLED (CONFIG_PREEMPT )) +synchronize_rcu_mult (call_rcu, call_rcu_sched); +else +synchronize_rcu ();smpboot_park_threads (cpu);
kernel/pid.c +2 -3
@@ -451,9 +451,8 @@ EXPORT_SYMBOL(pid_task);*/ struct task_struct *find_task_by_pid_ns (pid_t nr,struct pid_namespace *ns) { -rcu_lockdep_assert (rcu_read_lock_held (), -"find_task_by_pid_ns() needs rcu_read_lock()" -" protection" ); +RCU_LOCKDEP_WARN (!rcu_read_lock_held (), +"find_task_by_pid_ns() needs rcu_read_lock() protection" );return pid_task (find_pid_ns (nr, ns),PIDTYPE_PID ); }
kernel/rcu/rcutorture.c +29 -13
100 lines not on this page · raw message
kernel/rcu/srcu.c +8 -7
@@ -252,14 +252,15 @@ static bool srcu_readers_active_idx_check(struct srcu_struct *sp, int idx) }/** -* srcu_readers_active - returns approximate number of readers. +* srcu_readers_active - returns true if there are readers. and false +* otherwise * @sp: which srcu_struct to count active readers (holding srcu_read_lock). * * Note that this is not an atomic primitive, and can therefore suffer * severe errors when invoked on an active srcu_struct. That said, it * can be useful as an error check at cleanup time. */ -static int srcu_readers_active (struct srcu_struct *sp) +static bool srcu_readers_active (struct srcu_struct *sp) {int cpu;unsigned long sum =0 ;
@@ -414,11 +415,11 @@ static void __synchronize_srcu(struct srcu_struct *sp, int trycount)struct rcu_head *head = &rcu.head;bool done = false; -rcu_lockdep_assert (!lock_is_held (&sp->dep_map) && - !lock_is_held (&rcu_bh_lock_map) && - !lock_is_held (&rcu_lock_map) && - !lock_is_held (&rcu_sched_lock_map), -"Illegal synchronize_srcu() in same-type SRCU (or RCU) read-side critical section" ); +RCU_LOCKDEP_WARN (lock_is_held (&sp->dep_map) || +lock_is_held (&rcu_bh_lock_map) || +lock_is_held (&rcu_lock_map) || +lock_is_held (&rcu_sched_lock_map), +"Illegal synchronize_srcu() in same-type SRCU (or in RCU) read-side critical section" );might_sleep ();init_completion (&rcu.completion);
kernel/rcu/tiny.c +4 -4
@@ -191,10 +191,10 @@ static void rcu_process_callbacks(struct softirq_action *unused)*/ void synchronize_sched (void ) { -rcu_lockdep_assert (!lock_is_held (&rcu_bh_lock_map) && - !lock_is_held (&rcu_lock_map) && - !lock_is_held (&rcu_sched_lock_map), -"Illegal synchronize_sched() in RCU read-side critical section" ); +RCU_LOCKDEP_WARN (lock_is_held (&rcu_bh_lock_map) || +lock_is_held (&rcu_lock_map) || +lock_is_held (&rcu_sched_lock_map), +"Illegal synchronize_sched() in RCU read-side critical section" );cond_resched (); }EXPORT_SYMBOL_GPL (synchronize_sched);
kernel/rcu/tree.c +399 -282
956 lines not on this page · raw message
kernel/rcu/tree.h +63 -33
193 lines not on this page · raw message
kernel/rcu/tree_plugin.h +20 -110
234 lines not on this page · raw message
kernel/rcu/tree_trace.c +8 -11
@@ -81,9 +81,9 @@ static void r_stop(struct seq_file *m, void *v)static int show_rcubarrier (struct seq_file *m,void *v) {struct rcu_state *rsp = (struct rcu_state *)m->private; -seq_printf (m,"bcc: %d nbd: %lu\n" , +seq_printf (m,"bcc: %d bseq: %lu\n" ,atomic_read (&rsp->barrier_cpu_count), - rsp->n_barrier_done); + rsp->barrier_sequence);return 0 ; }
@@ -185,18 +185,15 @@ static int show_rcuexp(struct seq_file *m, void *v) {struct rcu_state *rsp = (struct rcu_state *)m->private; -seq_printf (m,"s=%lu d=%lu w=%lu tf=%lu wd1=%lu wd2=%lu n=%lu sc=%lu dt=%lu dl=%lu dx=%lu\n" , -atomic_long_read (&rsp->expedited_start), -atomic_long_read (&rsp->expedited_done), -atomic_long_read (&rsp->expedited_wrap), -atomic_long_read (&rsp->expedited_tryfail), +seq_printf (m,"s=%lu wd0=%lu wd1=%lu wd2=%lu wd3=%lu n=%lu enq=%d sc=%lu\n" , + rsp->expedited_sequence, +atomic_long_read (&rsp->expedited_workdone0),atomic_long_read (&rsp->expedited_workdone1),atomic_long_read (&rsp->expedited_workdone2), +atomic_long_read (&rsp->expedited_workdone3),atomic_long_read (&rsp->expedited_normal), -atomic_long_read (&rsp->expedited_stoppedcpus), -atomic_long_read (&rsp->expedited_done_tries), -atomic_long_read (&rsp->expedited_done_lost), -atomic_long_read (&rsp->expedited_done_exit)); +atomic_read (&rsp->expedited_need_qs), + rsp->expedited_sequence /2 );return 0 ; }
kernel/rcu/update.c +78 -12
49 lines not on this page · raw message
tools/testing/selftests/rcutorture/configs/rcu/TASKS01 +2 -2
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE01 +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE02 +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE02-T +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE03 +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE04 +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE05 +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE06 +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE07 +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE08 +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE08-T +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/configs/rcu/TREE09 +0 -1
Lines of this file are not shown. View the raw message.
tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt +0 -1
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
Quoted 9 lines · Ingo Molnar
Linus,
Please pull the latest core-rcu-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-rcu-for-linus # HEAD: f612a7b1a7f1b5139f228724ce340aac24720591 Merge branch 'for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into core/rcuThe main RCU changes in this cycle are:
JFYI, note that if you pull these RCU changes plus then pending x86/asm changes, you will see new but harmless build time deprecation warnings from new lockdep_assert() usages.
This will go away once I send you:
80008e1af691 ("rcu: Convert leftover rcu_lockdep_assert() call sites to RCU_LOCKDEP_WARN()")
Which I'll do once/when/if all the dependent changes hit your tree.
Thanks,
Ingo