RCU changes for v4.1
Subsystems (15)
Linus,
Please pull the latest core-rcu-for-linus git tree from:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git core-rcu-for-linus
# HEAD: 590ee7dbd569a012df705a5204fc5f1066f52b8c cpu: Provide smpboot_thread_init() on !CONFIG_SMP kernels as well
The main changes in this cycle were:
- Changes permitting use of call_rcu() and friends very early in boot, for example, before rcu_init() is invoked.
- Add in-kernel API to enable and disable expediting of normal RCU grace periods.
- Improve RCU's handling of (hotplug-) outgoing CPUs.
- NO_HZ_FULL_SYSIDLE fixes.
- Tiny RCU updates to make it more tiny.
- Documentation updates.
- Miscellaneous fixes.
Thanks,
Ingo
------------------>
Alexander Gordeev (2):
rcu: Remove unnecessary condition check in rcu_qsctr_help()
rcu: Remove fastpath from __rcu_process_callbacks()
Ingo Molnar (1):
cpu: Provide smpboot_thread_init() on !CONFIG_SMP kernels as well
Lai Jiangshan (1):
rcu_tree: Avoid touching rnp->completed when a new GP is started
Paul E. McKenney (53):
rcu: Consolidate rcu_synchronize and wakeme_after_rcu()
documentation: Update rcutree.kthread_prio for grace-period kthread use
documentation: Update based on on-demand vmstat workers
documentation: Update NO_HZ_FULL interaction with POSIX timers
documentation: Update per-CPU kthreads documentation
documentation: Clarify memory-barrier semantics of atomic operations
documentation: Clarify control-dependency pairing
rcu: Abstract default callback-list initialization from init_callback_list()
rcu: Wire ->rda pointers at compile time
rcu: Prevent early-boot RCU callbacks from splatting
rcu: Refine diagnostics for lacking kthread for no-CBs callbacks
rcu: Avoid clobbering early boot callbacks
rcu: Fix a couple of typos in rcu_all_qs() comment header
rcu: Drive PROVE_RCU directly off of PROVE_LOCKING
rcu: Provide rcu_expedite_gp() and rcu_unexpedite_gp()
rcu: Add rcu_expedite_gp() and rcu_unexpedite_gp() to rcutorture
rcu: Update from rcu_expedited variable to rcu_gp_is_expedited()
rcu: Add Kconfig option to expedite grace periods during boot
rcutorture: Make consistent use of variables
rcu: Fixes to NO_HZ_FULL sysidle accounting
rcu: Tighten up affinity and check for sysidle
rcu: Move early-boot callbacks to no-CBs lists for no-CBs CPUs
rcu: Move early boot callback tests earlier
rcu: Use IS_ENABLED() to CONFIG_RCU_FANOUT_EXACT #ifdef
rcu: Improve diagnostics for blocked critical sections in irq
rcu: Use IS_ENABLED() to simplify rcu_bootup_announce_oddness()
rcu: Add boot-up check for non-default CONFIG_RCU_FANOUT_LEAF values
rcu: Reverse rcu_dereference_check() conditions
torture: Avoid script syntax error when insufficient CPUs
rcu: Get rcu_sched_force_quiescent_state() where it belongs
smpboot: Add common code for notification from dying CPU
x86: Use common outgoing-CPU-notification code
blackfin: Use common outgoing-CPU-notification code
metag: Use common outgoing-CPU-notification code
rcu: Consolidate offline-CPU callback initialization
rcu: Put all orphan-callback-related code under same comment
rcu: Simplify sync_rcu_preempt_exp_init()
rcu: Eliminate empty HOTPLUG_CPU ifdef
rcu: Detect stalls caused by failure to propagate up rcu_node tree
rcu: Provide diagnostic option to slow down grace-period initialization
rcutorture: Enable slow grace-period initializations
rcu: Remove event tracing from rcu_cpu_notify(), used by offline CPUs
rcu: Rework preemptible expedited bitmask handling
rcu: Move rcu_report_unblock_qs_rnp() to common code
rcu: Process offlining and onlining only at grace-period start
rcu: Eliminate ->onoff_mutex from rcu_node structure
cpu: Make CPU-offline idle-loop transition point more precise
rcu: Handle outgoing CPUs on exit from idle loop
rcutorture: Default to grace-period-initialization delays
rcu: Add diagnostics to grace-period cleanup
rcu: Yet another fix for preemption and CPU hotplug
rcu: Associate quiescent-state reports with grace period
cpu: Defer smpboot kthread unparking until CPU known to scheduler
Yao Dongdong (1):
rcu: Remove redundant check of cpu_online()
Documentation/atomic_ops.txt | 45 +-- Documentation/kernel-parameters.txt | 20 +- Documentation/kernel-per-CPU-kthreads.txt | 34 +- Documentation/memory-barriers.txt | 42 +- Documentation/timers/NO_HZ.txt | 10 +- arch/blackfin/mach-common/smp.c | 6 +- arch/metag/kernel/smp.c | 5 +- arch/x86/include/asm/cpu.h | 2 - arch/x86/include/asm/smp.h | 2 +- arch/x86/kernel/smpboot.c | 39 +- arch/x86/xen/smp.c | 46 ++- include/linux/cpu.h | 20 + include/linux/lockdep.h | 7 +- include/linux/rcupdate.h | 40 +- include/linux/srcu.h | 2 +- init/Kconfig | 13 + init/main.c | 1 + kernel/cpu.c | 38 +- kernel/rcu/rcutorture.c | 27 +- kernel/rcu/srcu.c | 19 +- kernel/rcu/tiny.c | 14 +- kernel/rcu/tree.c | 437 +++++++++++++++------ kernel/rcu/tree.h | 11 +- kernel/rcu/tree_plugin.h | 267 +++++++------ kernel/rcu/tree_trace.c | 4 +- kernel/rcu/update.c | 72 +++- kernel/sched/idle.c | 9 +- kernel/smpboot.c | 156 ++++++++ lib/Kconfig.debug | 35 +- tools/testing/selftests/rcutorture/bin/kvm.sh | 2 +- .../selftests/rcutorture/configs/rcu/CFcommon | 1 + 31 files changed, 986 insertions(+), 440 deletions(-)
Documentation/atomic_ops.txt +23 -22
@@ -201,11 +201,11 @@ These routines add 1 and subtract 1, respectively, from the given atomic_t and return the new counter value after the operation is performed. -Unlike the above routines, it is required that explicit memory -barriers are performed before and after the operation. It must be -done such that all memory operations before and after the atomic -operation calls are strongly ordered with respect to the atomic -operation itself. +Unlike the above routines, it is required that these primitives +include explicit memory barriers that are performed before and after +the operation. It must be done such that all memory operations before +and after the atomic operation calls are strongly ordered with respect +to the atomic operation itself. For example, it should behave as if a smp_mb() call existed both before and after the atomic operation.
@@ -233,21 +233,21 @@ These two routines increment and decrement by 1, respectively, the given atomic counter. They return a boolean indicating whether the resulting counter value was zero or not. -It requires explicit memory barrier semantics around the operation as -above. +Again, these primitives provide explicit memory barrier semantics around +the atomic operation. int atomic_sub_and_test(int i, atomic_t *v); This is identical to atomic_dec_and_test() except that an explicit -decrement is given instead of the implicit "1". It requires explicit -memory barrier semantics around the operation. +decrement is given instead of the implicit "1". This primitive must +provide explicit memory barrier semantics around the operation. int atomic_add_negative(int i, atomic_t *v); -The given increment is added to the given atomic counter value. A -boolean is return which indicates whether the resulting counter value -is negative. It requires explicit memory barrier semantics around the -operation. +The given increment is added to the given atomic counter value. A boolean +is return which indicates whether the resulting counter value is negative. +This primitive must provide explicit memory barrier semantics around +the operation. Then:
@@ -257,7 +257,7 @@ This performs an atomic exchange operation on the atomic variable v, setting the given new value. It returns the old value that the atomic variable v had just before the operation. -atomic_xchg requires explicit memory barriers around the operation. +atomic_xchg must provide explicit memory barriers around the operation. int atomic_cmpxchg(atomic_t *v, int old, int new);
@@ -266,7 +266,7 @@ with the given old and new values. Like all atomic_xxx operations, atomic_cmpxchg will only satisfy its atomicity semantics as long as all other accesses of *v are performed through atomic_xxx operations. -atomic_cmpxchg requires explicit memory barriers around the operation. +atomic_cmpxchg must provide explicit memory barriers around the operation. The semantics for atomic_cmpxchg are the same as those defined for 'cas' below.
@@ -279,8 +279,8 @@ If the atomic value v is not equal to u, this function adds a to v, and returns non zero. If v is equal to u then it returns zero. This is done as an atomic operation. -atomic_add_unless requires explicit memory barriers around the operation -unless it fails (returns 0). +atomic_add_unless must provide explicit memory barriers around the +operation unless it fails (returns 0). atomic_inc_not_zero, equivalent to atomic_add_unless(v, 1, 0)
@@ -460,9 +460,9 @@ the return value into an int. There are other places where things like this occur as well. These routines, like the atomic_t counter operations returning values, -require explicit memory barrier semantics around their execution. All -memory operations before the atomic bit operation call must be made -visible globally before the atomic bit operation is made visible. +must provide explicit memory barrier semantics around their execution. +All memory operations before the atomic bit operation call must be +made visible globally before the atomic bit operation is made visible. Likewise, the atomic bit operation must be visible globally before any subsequent memory operation is made visible. For example:
@@ -536,8 +536,9 @@ except that two underscores are prefixed to the interface name. These non-atomic variants also do not require any special memory barrier semantics. -The routines xchg() and cmpxchg() need the same exact memory barriers -as the atomic and bit operations returning values. +The routines xchg() and cmpxchg() must provide the same exact +memory-barrier semantics as the atomic and bit operations returning +values. Spinlocks and rwlocks have memory barrier expectations as well. The rule to follow is simple:
Documentation/kernel-parameters.txt +15 -5
@@ -2968,6 +2968,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. Set maximum number of finished RCU callbacks to process in one batch. + rcutree.gp_init_delay= [KNL] + Set the number of jiffies to delay each step of + RCU grace-period initialization. This only has + effect when CONFIG_RCU_TORTURE_TEST_SLOW_INIT is + set. + rcutree.rcu_fanout_leaf= [KNL] Increase the number of CPUs assigned to each leaf rcu_node structure. Useful for very large
@@ -2991,11 +2997,15 @@ bytes respectively. Such letter suffixes can also be entirely omitted. value is one, and maximum value is HZ. rcutree.kthread_prio= [KNL,BOOT] - Set the SCHED_FIFO priority of the RCU - per-CPU kthreads (rcuc/N). This value is also - used for the priority of the RCU boost threads - (rcub/N). Valid values are 1-99 and the default - is 1 (the least-favored priority). + Set the SCHED_FIFO priority of the RCU per-CPU + kthreads (rcuc/N). This value is also used for + the priority of the RCU boost threads (rcub/N) + and for the RCU grace-period kthreads (rcu_bh, + rcu_preempt, and rcu_sched). If RCU_BOOST is + set, valid values are 1-99 and the default is 1 + (the least-favored priority). Otherwise, when + RCU_BOOST is not set, valid values are 0-99 and + the default is zero (non-realtime operation). rcutree.rcu_nocb_leader_stride= [KNL] Set the number of NOCB kthread groups, which
Documentation/kernel-per-CPU-kthreads.txt +21 -13
@@ -190,20 +190,24 @@ Purpose: Execute workqueue requests on each CPU, including cs_dbs_timer() and od_dbs_timer(). WARNING: Please check your CPU specifications to make sure that this is safe on your particular system. - d. It is not possible to entirely get rid of OS jitter - from vmstat_update() on CONFIG_SMP=y systems, but you - can decrease its frequency by writing a large value - to /proc/sys/vm/stat_interval. The default value is - HZ, for an interval of one second. Of course, larger - values will make your virtual-memory statistics update - more slowly. Of course, you can also run your workload - at a real-time priority, thus preempting vmstat_update(), + d. As of v3.18, Christoph Lameter's on-demand vmstat workers + commit prevents OS jitter due to vmstat_update() on + CONFIG_SMP=y systems. Before v3.18, is not possible + to entirely get rid of the OS jitter, but you can + decrease its frequency by writing a large value to + /proc/sys/vm/stat_interval. The default value is HZ, + for an interval of one second. Of course, larger values + will make your virtual-memory statistics update more + slowly. Of course, you can also run your workload at + a real-time priority, thus preempting vmstat_update(), but if your workload is CPU-bound, this is a bad idea. However, there is an RFC patch from Christoph Lameter (based on an earlier one from Gilad Ben-Yossef) that reduces or even eliminates vmstat overhead for some workloads at https://lkml.org/lkml/2013/9/4/379. - e. If running on high-end powerpc servers, build with + e. Boot with "elevator=noop" to avoid workqueue use by + the block layer. + f. If running on high-end powerpc servers, build with CONFIG_PPC_RTAS_DAEMON=n. This prevents the RTAS daemon from running on each CPU every second or so. (This will require editing Kconfig files and will defeat
@@ -211,12 +215,12 @@ Purpose: Execute workqueue requests due to the rtas_event_scan() function. WARNING: Please check your CPU specifications to make sure that this is safe on your particular system. - f. If running on Cell Processor, build your kernel with + g. If running on Cell Processor, build your kernel with CBE_CPUFREQ_SPU_GOVERNOR=n to avoid OS jitter from spu_gov_work(). WARNING: Please check your CPU specifications to make sure that this is safe on your particular system. - g. If running on PowerMAC, build your kernel with + h. If running on PowerMAC, build your kernel with CONFIG_PMAC_RACKMETER=n to disable the CPU-meter, avoiding OS jitter from rackmeter_do_timer().
@@ -258,8 +262,12 @@ Purpose: Detect software lockups on each CPU. To reduce its OS jitter, do at least one of the following: 1. Build with CONFIG_LOCKUP_DETECTOR=n, which will prevent these kthreads from being created in the first place. -2. Echo a zero to /proc/sys/kernel/watchdog to disable the +2. Boot with "nosoftlockup=0", which will also prevent these kthreads + from being created. Other related watchdog and softlockup boot + parameters may be found in Documentation/kernel-parameters.txt + and Documentation/watchdog/watchdog-parameters.txt. +3. Echo a zero to /proc/sys/kernel/watchdog to disable the watchdog timer. -3. Echo a large number of /proc/sys/kernel/watchdog_thresh in +4. Echo a large number of /proc/sys/kernel/watchdog_thresh in order to reduce the frequency of OS jitter due to the watchdog timer down to a level that is acceptable for your workload.
Documentation/memory-barriers.txt +29 -13
78 lines not on this page · raw message
Documentation/timers/NO_HZ.txt +3 -7
@@ -158,13 +158,9 @@ Just as with dyntick-idle mode, the benefits of adaptive-tick mode do to the need to inform kernel subsystems (such as RCU) about the change in mode. -3. POSIX CPU timers on adaptive-tick CPUs may miss their deadlines - (perhaps indefinitely) because they currently rely on - scheduling-tick interrupts. This will likely be fixed in - one of two ways: (1) Prevent CPUs with POSIX CPU timers from - entering adaptive-tick mode, or (2) Use hrtimers or other - adaptive-ticks-immune mechanism to cause the POSIX CPU timer to - fire properly. +3. POSIX CPU timers prevent CPUs from entering adaptive-tick mode. + Real-time applications needing to take actions based on CPU time + consumption need to use other means of doing so. 4. If there are more perf events pending than the hardware can accommodate, they are normally round-robined so as to collect
arch/blackfin/mach-common/smp.c +2 -4
@@ -413,16 +413,14 @@ int __cpu_disable(void)return 0 ; } -static DECLARE_COMPLETION (cpu_killed); -int __cpu_die (unsigned int cpu) { -return wait_for_completion_timeout (&cpu_killed,5000 ); +return cpu_wait_death (cpu,5 ); }void cpu_die (void ) { -complete (&cpu_killed); + (void )cpu_report_death ();atomic_dec (&init_mm.mm_users);atomic_dec (&init_mm.mm_count);
arch/metag/kernel/smp.c +2 -3
@@ -261,7 +261,6 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) }#ifdef CONFIG_HOTPLUG_CPU -static DECLARE_COMPLETION (cpu_killed);/* * __cpu_disable runs on the processor to be shutdown.
@@ -299,7 +298,7 @@ int __cpu_disable(void)*/ void __cpu_die (unsigned int cpu) { -if (!wait_for_completion_timeout (&cpu_killed,msecs_to_jiffies (1 ))) +if (!cpu_wait_death (cpu,1 ))pr_err ("CPU%u: unable to kill\n" , cpu); }
@@ -314,7 +313,7 @@ void cpu_die(void)local_irq_disable ();idle_task_exit (); -complete (&cpu_killed); + (void )cpu_report_death (); asm ("XOR TXENABLE, D0Re0,D0Re0\n" ); }
arch/x86/include/asm/cpu.h +0 -2
@@ -34,8 +34,6 @@ extern int _debug_hotplug_cpu(int cpu, int action);#endif #endif -DECLARE_PER_CPU (int, cpu_state); -int mwait_usable (const struct cpuinfo_x86 *);#endif /* _ASM_X86_CPU_H */
arch/x86/include/asm/smp.h +1 -1
@@ -150,12 +150,12 @@ static inline void arch_send_call_function_ipi_mask(const struct cpumask *mask) }void cpu_disable_common (void ); -void cpu_die_common (unsigned int cpu);void native_smp_prepare_boot_cpu (void );void native_smp_prepare_cpus (unsigned int max_cpus);void native_smp_cpus_done (unsigned int max_cpus);int native_cpu_up (unsigned int cpunum,struct task_struct *tidle);int native_cpu_disable (void ); +int common_cpu_die (unsigned int cpu);void native_cpu_die (unsigned int cpu);void native_play_dead (void );void play_dead_common (void );
arch/x86/kernel/smpboot.c +18 -21
@@ -77,9 +77,6 @@#include <asm/realmode.h> #include <asm/misc.h> -/* State of each CPU */ -DEFINE_PER_CPU(int, cpu_state) = { 0 }; -/* Number of siblings per CPU package */ int smp_num_siblings =1 ;EXPORT_SYMBOL (smp_num_siblings);
@@ -257,7 +254,7 @@ static void notrace start_secondary(void *unused)lock_vector_lock ();set_cpu_online (smp_processor_id (), true);unlock_vector_lock (); -per_cpu (cpu_state,smp_processor_id ()) =CPU_ONLINE ; +cpu_set_state_online (smp_processor_id ()); x86_platform.nmi_init ();/* enable local interrupts */
@@ -948,7 +945,10 @@ int native_cpu_up(unsigned int cpu, struct task_struct *tidle)*/ mtrr_save_state (); -per_cpu (cpu_state, cpu) =CPU_UP_PREPARE ; +/* x86 CPUs take themselves offline, so delayed offline is OK. */ + err =cpu_check_up_prepare (cpu); +if (err && err != -EBUSY ) +return err;/* the FPU context is blank, nobody can own it */ __cpu_disable_lazy_restore (cpu);
@@ -1191,7 +1191,7 @@ void __init native_smp_prepare_boot_cpu(void)switch_to_new_gdt (me);/* already set me in cpu_online_mask in boot_cpu_init() */ cpumask_set_cpu (me, cpu_callout_mask); -per_cpu (cpu_state, me) =CPU_ONLINE ; +cpu_set_state_online (me); }void __init native_smp_cpus_done (unsigned int max_cpus)
@@ -1318,14 +1318,10 @@ static void __ref remove_cpu_from_maps(int cpu)numa_remove_cpu (cpu); } -static DEFINE_PER_CPU(struct completion, die_complete); -void cpu_disable_common (void ) {int cpu =smp_processor_id (); -init_completion (&per_cpu (die_complete,smp_processor_id ())); -remove_siblinginfo (cpu);/* It's now safe to remove this processor from the online map */
@@ -1349,24 +1345,27 @@ int native_cpu_disable(void)return 0 ; } -void cpu_die_common (unsigned int cpu) +int common_cpu_die (unsigned int cpu) { -wait_for_completion_timeout (&per_cpu (die_complete, cpu),HZ ); -} +int ret =0 ; -void native_cpu_die (unsigned int cpu) -{/* We don't do anything here: idle task is faking death itself. */ -cpu_die_common (cpu); -/* They ack this in play_dead() by setting CPU_DEAD */ -if (per_cpu (cpu_state, cpu) ==CPU_DEAD ) { +if (cpu_wait_death (cpu,5 )) {if (system_state ==SYSTEM_RUNNING )pr_info ("CPU %u is now offline\n" , cpu); }else {pr_err ("CPU %u didn't die...\n" , cpu); + ret =-1 ; } + +return ret; +} + +void native_cpu_die (unsigned int cpu) +{ +common_cpu_die (cpu); }void play_dead_common (void )
@@ -1375,10 +1374,8 @@ void play_dead_common(void)reset_lazy_tlbstate ();amd_e400_remove_cpu (raw_smp_processor_id ()); -mb ();/* Ack it */ -__this_cpu_write (cpu_state,CPU_DEAD ); -complete (&per_cpu (die_complete,smp_processor_id ())); + (void )cpu_report_death ();/* * With physical CPU hotplug, we should halt the cpu
arch/x86/xen/smp.c +25 -21
92 lines not on this page · raw message
include/linux/cpu.h +20 -0
@@ -73,6 +73,7 @@ enum {/* migration should happen before other stuff but after perf */ CPU_PRI_PERF =20 ,CPU_PRI_MIGRATION =10 , +CPU_PRI_SMPBOOT =9 ,/* bring up workqueues before normal notifiers and down after */ CPU_PRI_WORKQUEUE_UP =5 , CPU_PRI_WORKQUEUE_DOWN = -5,
@@ -95,6 +96,10 @@ enum {* Called on the new cpu, just before * enabling interrupts. Must not sleep, * must not fail */ +#define CPU_DYING_IDLE 0x000B/* CPU (unsigned)v dying, reached +* idle loop. */ +#define CPU_BROKEN 0x000C/* CPU (unsigned)v did not die properly, +* perhaps due to preemption. */ /* Used for CPU hotplug events occurring while tasks are frozen due to a suspend * operation in progress
@@ -161,6 +166,7 @@ static inline void __unregister_cpu_notifier(struct notifier_block *nb) }#endif +void smpboot_thread_init (void );int cpu_up (unsigned int cpu);void notify_cpu_starting (unsigned int cpu);extern void cpu_maps_update_begin (void );
@@ -208,6 +214,10 @@ static inline void cpu_notifier_register_done(void) { } +static inline void smpboot_thread_init (void ) +{ +} +#endif /* CONFIG_SMP */ extern struct bus_type cpu_subsys;
@@ -271,4 +281,14 @@ void arch_cpu_idle_enter(void);void arch_cpu_idle_exit (void );void arch_cpu_idle_dead (void ); +DECLARE_PER_CPU (bool, cpu_dead_idle); + +int cpu_report_state (int cpu); +int cpu_check_up_prepare (int cpu); +void cpu_set_state_online (int cpu); +#ifdef CONFIG_HOTPLUG_CPU +bool cpu_wait_death (unsigned int cpu,int seconds); +bool cpu_report_death (void ); +#endif /* #ifdef CONFIG_HOTPLUG_CPU */ +#endif /* _LINUX_CPU_H_ */
include/linux/lockdep.h +6 -1
@@ -531,8 +531,13 @@ do { \# define might_lock_read (lock) do { } while (0)#endif -#ifdef CONFIG_PROVE_RCU +#ifdef CONFIG_LOCKDEP void lockdep_rcu_suspicious (const char *file,const int line,const char *s); +#else +static inline void +lockdep_rcu_suspicious (const char *file,const int line,const char *s) +{ +}#endif #endif /* __LINUX_LOCKDEP_H */
include/linux/rcupdate.h +36 -4
@@ -48,6 +48,26 @@extern int rcu_expedited;/* for sysctl */ +#ifdef CONFIG_TINY_RCU +/* Tiny RCU doesn't expedite, as its purpose in life is instead to be tiny. */ +static inline bool rcu_gp_is_expedited (void )/* Internal RCU use. */ +{ +return false; +} + +static inline void rcu_expedite_gp (void ) +{ +} + +static inline void rcu_unexpedite_gp (void ) +{ +} +#else /* #ifdef CONFIG_TINY_RCU */ +bool rcu_gp_is_expedited (void );/* Internal RCU use. */ +void rcu_expedite_gp (void ); +void rcu_unexpedite_gp (void ); +#endif /* #else #ifdef CONFIG_TINY_RCU */ +enum rcutorture_type {RCU_FLAVOR ,RCU_BH_FLAVOR ,
@@ -195,6 +215,15 @@ void call_rcu_sched(struct rcu_head *head,void synchronize_sched (void ); +/* +* Structure allowing asynchronous waiting on RCU. +*/ +struct rcu_synchronize { +struct rcu_head head; +struct completion completion; +}; +void wakeme_after_rcu (struct rcu_head *head); +/** * call_rcu_tasks() - Queue an RCU for invocation task-based grace period * @head: structure to be used for queueing the RCU updates.
@@ -258,6 +287,7 @@ static inline int rcu_preempt_depth(void)/* Internal to kernel */ void rcu_init (void ); +void rcu_end_inkernel_boot (void );void rcu_sched_qs (void );void rcu_bh_qs (void );void rcu_check_callbacks (int user);
@@ -266,6 +296,8 @@ void rcu_idle_enter(void);void rcu_idle_exit (void );void rcu_irq_enter (void );void rcu_irq_exit (void ); +int rcu_cpu_notify (struct notifier_block *self, +unsigned long action,void *hcpu);#ifdef CONFIG_RCU_STALL_COMMON void rcu_sysrq_start (void );
@@ -720,7 +752,7 @@ static inline void rcu_preempt_sleep_check(void)* annotated as __rcu. */ #define rcu_dereference_check (p, c) \ - __rcu_dereference_check((p), rcu_read_lock_held() || (c), __rcu) + __rcu_dereference_check((p), (c) || rcu_read_lock_held(), __rcu)/** * rcu_dereference_bh_check() - rcu_dereference_bh with debug checking
@@ -730,7 +762,7 @@ static inline void rcu_preempt_sleep_check(void)* This is the RCU-bh counterpart to rcu_dereference_check(). */ #define rcu_dereference_bh_check (p, c) \ - __rcu_dereference_check((p), rcu_read_lock_bh_held() || (c), __rcu) + __rcu_dereference_check((p), (c) || rcu_read_lock_bh_held(), __rcu)/** * rcu_dereference_sched_check() - rcu_dereference_sched with debug checking
@@ -740,7 +772,7 @@ static inline void rcu_preempt_sleep_check(void)* This is the RCU-sched counterpart to rcu_dereference_check(). */ #define rcu_dereference_sched_check (p, c) \ - __rcu_dereference_check((p), rcu_read_lock_sched_held() || (c), \ + __rcu_dereference_check((p), (c) || rcu_read_lock_sched_held(), \ __rcu)#define rcu_dereference_raw (p) rcu_dereference_check(p, 1)/*@@@ needed? @@@*/
@@ -933,9 +965,9 @@ static inline void rcu_read_unlock(void) {rcu_lockdep_assert (rcu_is_watching (),"rcu_read_unlock() used illegally while idle" ); -rcu_lock_release (&rcu_lock_map);__release (RCU );__rcu_read_unlock (); +rcu_lock_release (&rcu_lock_map);/* Keep acq info for rls diags. */ }/**
include/linux/srcu.h +1 -1
@@ -182,7 +182,7 @@ static inline int srcu_read_lock_held(struct srcu_struct *sp)* lockdep_is_held() calls. */ #define srcu_dereference_check (p, sp, c) \ - __rcu_dereference_check((p), srcu_read_lock_held(sp) || (c), __rcu) + __rcu_dereference_check((p), (c) || srcu_read_lock_held(sp), __rcu)/** * srcu_dereference - fetch SRCU-protected pointer for later dereferencing
init/Kconfig +13 -0
@@ -791,6 +791,19 @@ config RCU_NOCB_CPU_ALL endchoice +config RCU_EXPEDITE_BOOT + bool + default n + help + This option enables expedited grace periods at boot time, + as if rcu_expedite_gp() had been invoked early in boot. + The corresponding rcu_unexpedite_gp() is invoked from + rcu_end_inkernel_boot(), which is intended to be invoked + at the end of the kernel-only boot sequence, just before + init is exec'ed. + + Accept the default if unsure. + endmenu # "RCU Subsystem" config BUILD_BIN2C
init/main.c +1 -0
@@ -384,6 +384,7 @@ static noinline void __init_refok rest_init(void)int pid;rcu_scheduler_starting (); +smpboot_thread_init ();/* * We need to spawn init first so that it obtains pid 1, however * the init task will end up wanting to create kthreads, which, if
kernel/cpu.c +34 -4
@@ -408,8 +408,10 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)* * Wait for the stop thread to go away. */ -while (!idle_cpu (cpu)) +while (!per_cpu (cpu_dead_idle, cpu))cpu_relax (); +smp_mb ();/* Read from cpu_dead_idle before __cpu_die(). */ +per_cpu (cpu_dead_idle, cpu) = false;/* This actually kills the CPU. */ __cpu_die (cpu);
@@ -446,6 +448,37 @@ int __ref cpu_down(unsigned int cpu)EXPORT_SYMBOL (cpu_down);#endif /*CONFIG_HOTPLUG_CPU*/ +/* +* Unpark per-CPU smpboot kthreads at CPU-online time. +*/ +static int smpboot_thread_call (struct notifier_block *nfb, +unsigned long action,void *hcpu) +{ +int cpu = (long )hcpu; + +switch (action & ~CPU_TASKS_FROZEN ) { + +case CPU_ONLINE : +smpboot_unpark_threads (cpu); +break ; + +default : +break ; + } + +return NOTIFY_OK ; +} + +static struct notifier_block smpboot_thread_notifier = { + .notifier_call = smpboot_thread_call, + .priority =CPU_PRI_SMPBOOT , +}; + +void __cpuinit smpboot_thread_init (void ) +{ +register_cpu_notifier (&smpboot_thread_notifier); +} +/* Requires cpu_add_remove_lock to be held */ static int _cpu_up (unsigned int cpu,int tasks_frozen) {
@@ -485,9 +518,6 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen) goto out_notify;BUG_ON (!cpu_online (cpu)); -/* Wake the per cpu threads */ -smpboot_unpark_threads (cpu); -/* Now call notifier in preparation. */ cpu_notify (CPU_ONLINE | mod, hcpu);
kernel/rcu/rcutorture.c +26 -1
@@ -853,6 +853,8 @@ rcu_torture_fqs(void *arg)static int rcu_torture_writer (void *arg) { +bool can_expedite = !rcu_gp_is_expedited (); +int expediting =0 ;unsigned long gp_snap;bool gp_cond1 = gp_cond, gp_exp1 = gp_exp, gp_normal1 = gp_normal;bool gp_sync1 = gp_sync;
@@ -865,9 +867,15 @@ rcu_torture_writer(void *arg)int nsynctypes =0 ;VERBOSE_TOROUT_STRING ("rcu_torture_writer task started" ); +pr_alert ("%s" TORTURE_FLAG +" Grace periods expedited from boot/sysfs for %s,\n" , + torture_type, cur_ops->name); +pr_alert ("%s" TORTURE_FLAG +" Testing of dynamic grace-period expediting diabled.\n" , + torture_type);/* Initialize synctype[] array. If none set, take default. */ -if (!gp_cond1 && !gp_exp1 && !gp_normal1 && !gp_sync) +if (!gp_cond1 && !gp_exp1 && !gp_normal1 && !gp_sync1) gp_cond1 = gp_exp1 = gp_normal1 = gp_sync1 = true;if (gp_cond1 && cur_ops->get_state && cur_ops->cond_sync) synctype[nsynctypes++] =RTWS_COND_GET ;
@@ -949,9 +957,26 @@ rcu_torture_writer(void *arg) } }rcutorture_record_progress (++rcu_torture_current_version); +/* Cycle through nesting levels of rcu_expedite_gp() calls. */ +if (can_expedite && + !(torture_random (&rand) &0xff & (!!expediting -1 ))) { +WARN_ON_ONCE (expediting ==0 &&rcu_gp_is_expedited ()); +if (expediting >=0 ) +rcu_expedite_gp (); +else +rcu_unexpedite_gp (); +if (++expediting >3 ) + expediting = -expediting; + } rcu_torture_writer_state =RTWS_STUTTER ;stutter_wait ("rcu_torture_writer" ); }while (!torture_must_stop ()); +/* Reset expediting back to unexpedited. */ +if (expediting >0 ) + expediting = -expediting; +while (can_expedite && expediting++ <0 ) +rcu_unexpedite_gp (); +WARN_ON_ONCE (can_expedite &&rcu_gp_is_expedited ()); rcu_torture_writer_state =RTWS_STOPPING ;torture_kthread_stopping ("rcu_torture_writer" );return 0 ;
kernel/rcu/srcu.c +1 -18
@@ -402,23 +402,6 @@ void call_srcu(struct srcu_struct *sp, struct rcu_head *head, }EXPORT_SYMBOL_GPL (call_srcu); -struct rcu_synchronize { -struct rcu_head head; -struct completion completion; -}; - -/* -* Awaken the corresponding synchronize_srcu() instance now that a -* grace period has elapsed. -*/ -static void wakeme_after_rcu (struct rcu_head *head) -{ -struct rcu_synchronize *rcu; - - rcu = container_of(head, struct rcu_synchronize, head); -complete (&rcu->completion); -} -static void srcu_advance_batches (struct srcu_struct *sp,int trycount);static void srcu_reschedule (struct srcu_struct *sp);
@@ -507,7 +490,7 @@ static void __synchronize_srcu(struct srcu_struct *sp, int trycount)*/ void synchronize_srcu (struct srcu_struct *sp) { -__synchronize_srcu (sp, rcu_expedited +__synchronize_srcu (sp,rcu_gp_is_expedited () ?SYNCHRONIZE_SRCU_EXP_TRYCOUNT :SYNCHRONIZE_SRCU_TRYCOUNT ); }
kernel/rcu/tiny.c +1 -13
@@ -103,8 +103,7 @@ EXPORT_SYMBOL(__rcu_is_watching);static int rcu_qsctr_help (struct rcu_ctrlblk *rcp) {RCU_TRACE (reset_cpu_stall_ticks (rcp)); -if (rcp->rcucblist !=NULL && - rcp->donetail != rcp->curtail) { +if (rcp->donetail != rcp->curtail) { rcp->donetail = rcp->curtail;return 1 ; }
@@ -169,17 +168,6 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)unsigned long flags; RCU_TRACE(int cb_count = 0); -/* If no RCU callbacks ready to invoke, just return. */ -if (&rcp->rcucblist == rcp->donetail) { -RCU_TRACE (trace_rcu_batch_start (rcp->name,0 ,0 ,-1 )); -RCU_TRACE (trace_rcu_batch_end (rcp->name,0 , - !!ACCESS_ONCE (rcp->rcucblist), -need_resched (), -is_idle_task (current), - false)); -return ; - } -/* Move the ready-to-invoke callbacks to a local list. */ local_irq_save (flags);RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , rcp->qlen,-1 ));
kernel/rcu/tree.c +311 -126
774 lines not on this page · raw message
kernel/rcu/tree.h +9 -2
@@ -141,12 +141,20 @@ struct rcu_node {/* complete (only for PREEMPT_RCU). */ unsigned long qsmaskinit;/* Per-GP initial value for qsmask & expmask. */ +/* Initialized from ->qsmaskinitnext at the */ +/* beginning of each grace period. */ +unsigned long qsmaskinitnext; +/* Online CPUs for next grace period. */ unsigned long grpmask;/* Mask to apply to parent qsmask. */ /* Only one bit will be set in this mask. */ int grplo;/* lowest-numbered CPU or group here. */ int grphi;/* highest-numbered CPU or group here. */ u8 grpnum;/* CPU/group number for next level up. */ u8 level;/* root is at level 0. */ +bool wait_blkd_tasks;/* Necessary to wait for blocked tasks to */ +/* exit RCU read-side critical sections */ +/* before propagating offline up the */ +/* rcu_node tree? */ struct rcu_node *parent;struct list_head blkd_tasks;/* Tasks blocked in RCU read-side critical */
@@ -448,8 +456,6 @@ struct rcu_state {long qlen;/* Total number of callbacks. */ /* End of fields guarded by orphan_lock. */ -struct mutex onoff_mutex;/* Coordinate hotplug & GPs. */ -struct mutex barrier_mutex;/* Guards barrier fields. */ atomic_t barrier_cpu_count;/* # CPUs waiting on. */ struct completion barrier_completion;/* Wake at barrier end. */
@@ -559,6 +565,7 @@ static void rcu_prepare_kthreads(int cpu);static void rcu_cleanup_after_idle (void );static void rcu_prepare_for_idle (void );static void rcu_idle_count_callbacks_posted (void ); +static bool rcu_preempt_has_tasks (struct rcu_node *rnp);static void print_cpu_stall_info_begin (void );static void print_cpu_stall_info (struct rcu_state *rsp,int cpu);static void print_cpu_stall_info_end (void );
kernel/rcu/tree_plugin.h +132 -135
437 lines not on this page · raw message
kernel/rcu/tree_trace.c +2 -2
@@ -283,8 +283,8 @@ static void print_one_rcu_state(struct seq_file *m, struct rcu_state *rsp)seq_puts (m,"\n" ); level = rnp->level; } - seq_printf(m, "%lx/%lx %c%c>%c %d:%d ^%d ", - rnp->qsmask, rnp->qsmaskinit, + seq_printf(m, "%lx/%lx->%lx %c%c>%c %d:%d ^%d ", + rnp->qsmask, rnp->qsmaskinit, rnp->qsmaskinitnext, ".G"[rnp->gp_tasks != NULL], ".E"[rnp->exp_tasks != NULL], ".T"[!list_empty(&rnp->blkd_tasks)],
kernel/rcu/update.c +63 -9
85 lines not on this page · raw message
kernel/sched/idle.c +8 -1
@@ -210,6 +210,8 @@ static void cpuidle_idle_call(void) goto exit_idle; } +DEFINE_PER_CPU (bool, cpu_dead_idle); +/* * Generic idle loop implementation *
@@ -234,8 +236,13 @@ static void cpu_idle_loop(void)check_pgt_cache ();rmb (); -if (cpu_is_offline (smp_processor_id ())) +if (cpu_is_offline (smp_processor_id ())) { +rcu_cpu_notify (NULL ,CPU_DYING_IDLE , + (void *)(long )smp_processor_id ()); +smp_mb ();/* all activity before dead. */ +this_cpu_write (cpu_dead_idle, true);arch_cpu_idle_dead (); + }local_irq_disable ();arch_cpu_idle_enter ();
kernel/smpboot.c +156 -0
165 lines not on this page · raw message
lib/Kconfig.debug +25 -10
@@ -1180,16 +1180,7 @@ config DEBUG_CREDENTIALS menu "RCU Debugging" config PROVE_RCU - bool "RCU debugging: prove RCU correctness" - depends on PROVE_LOCKING - default n - help - This feature enables lockdep extensions that check for correct - use of RCU APIs. This is currently under development. Say Y - if you want to debug RCU usage or help work on the PROVE_RCU - feature. - - Say N if you are unsure. + def_bool PROVE_LOCKING config PROVE_RCU_REPEATEDLY bool "RCU debugging: don't disable PROVE_RCU on first splat"
@@ -1257,6 +1248,30 @@ config RCU_TORTURE_TEST_RUNNABLE Say N here if you want the RCU torture tests to start only after being manually enabled via /proc. +config RCU_TORTURE_TEST_SLOW_INIT + bool "Slow down RCU grace-period initialization to expose races" + depends on RCU_TORTURE_TEST + help + This option makes grace-period initialization block for a + few jiffies between initializing each pair of consecutive + rcu_node structures. This helps to expose races involving + grace-period initialization, in other words, it makes your + kernel less stable. It can also greatly increase grace-period + latency, especially on systems with large numbers of CPUs. + This is useful when torture-testing RCU, but in almost no + other circumstance. + + Say Y here if you want your system to crash and hang more often. + Say N if you want a sane system. + +config RCU_TORTURE_TEST_SLOW_INIT_DELAY + int "How much to slow down RCU grace-period initialization" + range 0 5 + default 3 + help + This option specifies the number of jiffies to wait between + each rcu_node structure initialization. + config RCU_CPU_STALL_TIMEOUT int "RCU CPU stall timeout in seconds" depends on RCU_STALL_COMMON
tools/testing/selftests/rcutorture/bin/kvm.sh +1 -1
@@ -310,7 +310,7 @@ function dump(first, pastlast) cfr[jn] = cf[j] "." cfrep[cf[j]]; } if (cpusr[jn] > ncpus && ncpus != 0) - ovf = "(!)"; + ovf = "-ovf"; else ovf = ""; print "echo ", cfr[jn], cpusr[jn] ovf ": Starting build. `date`";
tools/testing/selftests/rcutorture/configs/rcu/CFcommon +1 -0
@@ -1,2 +1,3 @@ CONFIG_RCU_TORTURE_TEST=y CONFIG_PRINTK_TIME=y +CONFIG_RCU_TORTURE_TEST_SLOW_INIT=y
Quoted 2 lines · Ingo Molnar
Please pull the latest core-rcu-for-linus git tree from:
This is very annoying:
...
torture tests for RCU (RCU_TORTURE_TEST) [N/m/y/?] n How much to slow down RCU grace-period initialization
(RCU_TORTURE_TEST_SLOW_INIT_DELAY) [3] (NEW)
...
please make things like this pointless value depend on the option that makes them relevant! So a simple
depends on RCU_TORTURE_TEST_SLOW_INIT
would seem to be called for.
It's not like we don't already have annoying and unanswerable questions for RCU (RCU_KTHREAD_PRIO), let's not add the completely stupid *pointless* ones too.
Linus
Quoted 4 lines · Linus Torvalds
depends on
RCU_TORTURE_TEST_SLOW_INITwould seem to be called for.
Side note, you'll obviously also need to fix the actual bogus 'gp_init_delay' use in kernel/rcu/tree.c.
That code is horrible.
Linus
Quoted 11 lines · Linus Torvalds
Quoted 4 lines · Linus Torvalds
depends on
RCU_TORTURE_TEST_SLOW_INITwould seem to be called for.
Side note, you'll obviously also need to fix the actual bogus '
gp_init_delay' use inkernel/rcu/tree.c.That code is horrible.
I was getting too cute in an attempt to avoid a #ifdef. :-/
Does the (currently being tested) patch below fix things up? If not, please fill me in on the further error of my ways.
Thanx, Paul "it seemed like at good idea a the time" McKenney
------------------------------------------------------------------------
rcu: Control grace-period delays directly from value
In a misguided attempt to avoid an #ifdef, the use of the gp_init_delay module parameter was conditioned on the corresponding RCU_TORTURE_TEST_SLOW_INIT Kconfig variable, using IS_ENABLED() at the point of use in the code. This meant that the compiler always saw the delay, which meant that RCU_TORTURE_TEST_SLOW_INIT_DELAY had to be unconditionally defined. This in turn caused "make oldconfig" to ask pointless questions about the value of RCU_TORTURE_TEST_SLOW_INIT_DELAY in cases where it was not even used.
This commit avoids these pointless questions by defining gp_init_delay under #ifdef. In one branch, gp_init_delay is initialized to RCU_TORTURE_TEST_SLOW_INIT_DELAY and is also a module parameter (thus allowing boot-time modification), and in the other branch gp_init_delay is a const variable initialized by default to zero.
This approach also simplifies the code at the delay point by eliminating the IS_DEFINED(). Because gp_init_delay is constant zero in the no-delay case intended for production use, the "gp_init_delay > 0" check causes the delay to become dead code, as desired in this case. In addition, this commit replaces magic constant "10" with the preprocessor variable PER_RCU_NODE_PERIOD, which controls the number of grace periods that are allowed to elapse at full speed before a delay is inserted.
Paul E. McKenney <[email protected]>
kernel/rcu/tree.c +9 -7
@@ -162,11 +162,14 @@ static void invoke_rcu_callbacks(struct rcu_state *rsp, struct rcu_data *rdp);static int kthread_prio =CONFIG_RCU_KTHREAD_PRIO ;module_param (kthread_prio, int,0644 ); -/* Delay in jiffies for grace-period initialization delays. */ -static int gp_init_delay =IS_ENABLED (CONFIG_RCU_TORTURE_TEST_SLOW_INIT ) - ?CONFIG_RCU_TORTURE_TEST_SLOW_INIT_DELAY - :0 ; +/* Delay in jiffies for grace-period initialization delays, debug only. */ +#ifdef CONFIG_RCU_TORTURE_TEST_SLOW_INIT +static int gp_init_delay =CONFIG_RCU_TORTURE_TEST_SLOW_INIT_DELAY ;module_param (gp_init_delay, int,0644 ); +#else /* #ifdef CONFIG_RCU_TORTURE_TEST_SLOW_INIT */ +static const int gp_init_delay; +#endif /* #else #ifdef CONFIG_RCU_TORTURE_TEST_SLOW_INIT */ +#define PER_RCU_NODE_PERIOD 10/* Number of grace periods between delays. */ /* * Track the rcutorture test sequence number and the update version
@@ -1843,9 +1846,8 @@ static int rcu_gp_init(struct rcu_state *rsp)raw_spin_unlock_irq (&rnp->lock);cond_resched_rcu_qs ();ACCESS_ONCE (rsp->gp_activity) = jiffies; -if (IS_ENABLED (CONFIG_RCU_TORTURE_TEST_SLOW_INIT ) && - gp_init_delay >0 && - !(rsp->gpnum % (rcu_num_nodes *10 ))) +if (gp_init_delay >0 && + !(rsp->gpnum % (rcu_num_nodes *PER_RCU_NODE_PERIOD )))schedule_timeout_uninterruptible (gp_init_delay); }
lib/Kconfig.debug +1 -0
@@ -1268,6 +1268,7 @@ config RCU_TORTURE_TEST_SLOW_INIT_DELAY int "How much to slow down RCU grace-period initialization" range 0 5 default 3 + depends on RCU_TORTURE_TEST_SLOW_INIT help This option specifies the number of jiffies to wait between each rcu_node structure initialization.
Quoted 3 lines · Paul E. McKenney
Does the (currently being tested) patch below fix things up? If not, please fill me in on the further error of my ways.
Looks ok.
That said, couldn't that last dummy gp_init_delay variable:
kernel/rcu/tree.c static void invoke_rcu_callbacks(struct rcu_state *rsp, struct rcu_data *rdp); · line 171
Quoted 7 lines
kernel/rcu/tree.c +6
-0+/* Delay in jiffies for grace-period initialization delays, debug only. */ +#ifdef CONFIG_RCU_TORTURE_TEST_SLOW_INIT +static int gp_init_delay =CONFIG_RCU_TORTURE_TEST_SLOW_INIT_DELAY ;module_param (gp_init_delay, int,0644 ); +#else /* #ifdef CONFIG_RCU_TORTURE_TEST_SLOW_INIT */ +static const int gp_init_delay; +#endif /* #else #ifdef CONFIG_RCU_TORTURE_TEST_SLOW_INIT */
be just a
#define gp_init_delay 0
for the non-CONFIG_RCU_TORTURE_TEST_SLOW_INIT case, so that the code that then does
+ if (gp_init_delay > 0 &&
+ !(rsp->gpnum % (rcu_num_nodes * PER_RCU_NODE_PERIOD)))
would just trivially compile away.
I guess the compiler *might* see a 'static const int' that is never touched and realize it's always zero, but it's not obvious that will be the case.
Linus
Linus
Quoted 28 lines · Linus Torvalds
Quoted 3 lines · Paul E. McKenney
Does the (currently being tested) patch below fix things up? If not, please fill me in on the further error of my ways.
Looks ok.
That said, couldn't that last dummy
gp_init_delayvariable:kernel/rcu/tree.c static void invoke_rcu_callbacks(struct rcu_state *rsp, struct rcu_data *rdp); · line 171
Quoted 7 lines
kernel/rcu/tree.c +6
-0+/* Delay in jiffies for grace-period initialization delays, debug only. */ +#ifdef CONFIG_RCU_TORTURE_TEST_SLOW_INIT +static int gp_init_delay =CONFIG_RCU_TORTURE_TEST_SLOW_INIT_DELAY ;module_param (gp_init_delay, int,0644 ); +#else /* #ifdef CONFIG_RCU_TORTURE_TEST_SLOW_INIT */ +static const int gp_init_delay; +#endif /* #else #ifdef CONFIG_RCU_TORTURE_TEST_SLOW_INIT */ be just a
#define gp_init_delay 0for the non-CONFIG_RCU_TORTURE_TEST_SLOW_INIT case, so that the code that then does
kernel/rcu/tree.c +2
-0+if (gp_init_delay >0 && + !(rsp->gpnum % (rcu_num_nodes *PER_RCU_NODE_PERIOD )))
Quoted 5 lines
would just trivially compile away.
I guess the compiler *might* see a 'static const int' that is never touched and realize it's always zero, but it's not obvious that will be the case.
Well, if I attempt to modify gp_init_delay in its const guise, for example, using gp_init_delay++, I get the following:
/media/homes/git/linux-2.6-tip/kernel/rcu/tree.c: In function ‘rcu_gp_init’: /media/homes/git/linux-2.6-tip/kernel/rcu/tree.c:1849:3: error: increment of read-only variable ‘gp_init_delay’ make[2]: *** [kernel/rcu/tree.o] Error 1 make[1]: *** [kernel/rcu/tree.o] Error 2 make[1]: Leaving directory `/tmp/b' make: *** [sub-make] Error 2
So the compiler knows that it cannot change. And if I compile with CONFIG_RCU_TORTURE_TEST_SLOW_INIT=y, then:
$ nm /tmp/b/kernel/rcu/tree.o | grep gp_init_delay 00000060 r __param_gp_init_delay 000000e3 r __param_str_gp_init_delay 00000444 d gp_init_delay
On the other hand, compiling with CONFIG_RCU_TORTURE_TEST_SLOW_INIT=n results in:
$ nm /tmp/b/kernel/rcu/tree.o | grep gp_init_delay
So the compiler isn't creating a variable in the const case.
Quoted 1 line
From what I can see, at -O1 or better, gcc optimizes the const variable
away. At -O0, gcc allocates space for it and actually tests it.
But if you prefer #define, not a problem.
Thanx, Paul