Josh and Ingo review feedback and bloatwatch RCU
cover Josh and Ingo review feedback and bloatwatch RCU
This patch set applies two sets of review feedback from Josh Triplett and checkpatch feedback from Ingo Molnar. It also contains a forward-port of Bloatwatch RCU, courtesy of David Howells.
b/include/linux/hardirq.h | 24 ++++ b/include/linux/rcupdate.h | 4 b/include/linux/rcutiny.h | 103 ++++++++++++++++++ b/include/linux/rcutree.h | 1 b/init/Kconfig | 9 + b/init/main.c | 1 b/kernel/Makefile | 1 b/kernel/rcupdate.c | 3 b/kernel/rcutiny.c | 246 ++++++++++++++++++++++++++++++++++++++++++++- b/kernel/rcutorture.c | 4 b/kernel/rcutree.c | 29 +++-- b/kernel/rcutree.h | 6 - b/kernel/rcutree_plugin.h | 51 ++++----- b/kernel/rcutree_trace.c | 13 +- include/linux/rcupdate.h | 6 + include/linux/rcutree.h | 12 +- kernel/rcupdate.c | 4 kernel/rcutree.c | 95 ++++++----------- kernel/rcutree.h | 57 ++++++++-- kernel/rcutree_plugin.h | 24 +++- 20 files changed, 557 insertions(+), 136 deletions(-)
PATCH 1/4 Apply review feedback from Josh Triplett, part 1
--- include/linux/rcutree.h | 1 - kernel/rcutree.c | 29 ++++++++++++++++---------- kernel/rcutree.h | 6 ++-- kernel/rcutree_plugin.h | 50 +++++++++++++++++++++++----------------------- 4 files changed, 46 insertions(+), 40 deletions(-)
include/linux/rcutree.h +0 -1
@@ -85,7 +85,6 @@ static inline void synchronize_rcu_bh_expedited(void)extern void __rcu_init (void );extern void rcu_check_callbacks (int cpu,int user); -extern void rcu_restart_cpu (int cpu);extern long rcu_batches_completed (void );extern long rcu_batches_completed_bh (void );
kernel/rcutree.c +18 -11
@@ -100,6 +100,16 @@ static void __cpuinit rcu_init_percpu_data(int cpu, struct rcu_state *rsp,#include "rcutree_plugin.h" /* +* Return true if an RCU grace period is in progress. The ACCESS_ONCE()s +* permit this function to be invoked without holding the root rcu_node +* structure's ->lock, but of course results can be subject to change. +*/ +static int rcu_gp_in_progress (struct rcu_state *rsp) +{ +return ACCESS_ONCE (rsp->completed) !=ACCESS_ONCE (rsp->gpnum); +} + +/* * Note a quiescent state. Because we do not need to know * how many quiescent states passed, just if there was at least * one since the start of the grace period, this just sets a flag.
@@ -172,9 +182,7 @@ cpu_has_callbacks_ready_to_invoke(struct rcu_data *rdp)static int cpu_needs_another_gp (struct rcu_state *rsp,struct rcu_data *rdp) { -/* ACCESS_ONCE() because we are accessing outside of lock. */ -return *rdp->nxttail[RCU_DONE_TAIL ] && -ACCESS_ONCE (rsp->completed) ==ACCESS_ONCE (rsp->gpnum); +return *rdp->nxttail[RCU_DONE_TAIL ] && !rcu_gp_in_progress (rsp); }/*
@@ -481,7 +489,7 @@ static void print_other_cpu_stall(struct rcu_state *rsp)spin_lock_irqsave (&rnp->lock, flags); delta = jiffies - rsp->jiffies_stall; -if (delta <RCU_STALL_RAT_DELAY || rsp->gpnum == rsp->completed) { +if (delta <RCU_STALL_RAT_DELAY || !rcu_gp_in_progress (rsp)) {spin_unlock_irqrestore (&rnp->lock, flags);return ; }
@@ -532,8 +540,7 @@ static void check_cpu_stall(struct rcu_state *rsp, struct rcu_data *rdp)/* We haven't checked in, so go dump stack. */ print_cpu_stall (rsp); - }else if (rsp->gpnum != rsp->completed && - delta >=RCU_STALL_RAT_DELAY ) { + }else if (rcu_gp_in_progress (rsp) && delta >=RCU_STALL_RAT_DELAY ) {/* They had two time units to dump stack, so complain. */ print_other_cpu_stall (rsp);
@@ -698,9 +705,9 @@ rcu_process_gp_end(struct rcu_state *rsp, struct rcu_data *rdp)* hold rnp->lock, as required by rcu_start_gp(), which will release it. */ static void cpu_quiet_msk_finish (struct rcu_state *rsp,unsigned long flags) - __releases(rnp->lock) + __releases(rcu_get_root(rsp)->lock) { -WARN_ON_ONCE (rsp->completed == rsp->gpnum); +WARN_ON_ONCE (!rcu_gp_in_progress (rsp)); rsp->completed = rsp->gpnum;rcu_process_gp_end (rsp, rsp->rda[smp_processor_id ()]);rcu_start_gp (rsp, flags);/* releases root node's rnp->lock. */
@@ -1087,7 +1094,7 @@ static void force_quiescent_state(struct rcu_state *rsp, int relaxed)struct rcu_node *rnp =rcu_get_root (rsp);u8 signaled; -if (ACCESS_ONCE (rsp->completed) ==ACCESS_ONCE (rsp->gpnum)) +if (!rcu_gp_in_progress (rsp))return ;/* No grace period in progress, nothing to force. */ if (!spin_trylock_irqsave (&rsp->fqslock, flags)) { rsp->n_force_qs_lh++;/* Inexact, can lose counts. Tough! */
@@ -1246,7 +1253,7 @@ __call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *rcu), rdp->nxttail[RCU_NEXT_TAIL ] = &head->next;/* Start a new grace period if one not already started. */ -if (ACCESS_ONCE (rsp->completed) ==ACCESS_ONCE (rsp->gpnum)) { +if (!rcu_gp_in_progress (rsp)) {unsigned long nestflag;struct rcu_node *rnp_root =rcu_get_root (rsp);
@@ -1326,7 +1333,7 @@ static int __rcu_pending(struct rcu_state *rsp, struct rcu_data *rdp) }/* Has an RCU GP gone long enough to send resched IPIs &c? */ -if (ACCESS_ONCE (rsp->completed) !=ACCESS_ONCE (rsp->gpnum) && +if (rcu_gp_in_progress (rsp) && ((long )(ACCESS_ONCE (rsp->jiffies_force_qs) - jiffies) <0 )) { rdp->n_rp_need_fqs++;return 1 ;
kernel/rcutree.h +3 -3
@@ -48,14 +48,14 @@#elif NR_CPUS <=RCU_FANOUT_SQ # define NUM_RCU_LVLS 2# define NUM_RCU_LVL_0 1 -# define NUM_RCU_LVL_1 (((NR_CPUS) + RCU_FANOUT - 1) / RCU_FANOUT) +# define NUM_RCU_LVL_1 DIV_ROUND_UP(NR_CPUS, RCU_FANOUT)# define NUM_RCU_LVL_2 (NR_CPUS)# define NUM_RCU_LVL_3 0#elif NR_CPUS <=RCU_FANOUT_CUBE # define NUM_RCU_LVLS 3# define NUM_RCU_LVL_0 1 -# define NUM_RCU_LVL_1 (((NR_CPUS) + RCU_FANOUT_SQ - 1) / RCU_FANOUT_SQ) -# define NUM_RCU_LVL_2 (((NR_CPUS) + (RCU_FANOUT) - 1) / (RCU_FANOUT)) +# define NUM_RCU_LVL_1 DIV_ROUND_UP(NR_CPUS, RCU_FANOUT_SQ) +# define NUM_RCU_LVL_2 DIV_ROUND_UP(NR_CPUS, RCU_FANOUT)# define NUM_RCU_LVL_3 NR_CPUS#else # error "CONFIG_RCU_FANOUT insufficient for NR_CPUS"
kernel/rcutree_plugin.h +25 -25
98 lines not on this page · raw message
Signature · 1 line
1.5.2.5
PATCH 2/4 Apply review feedback from Josh Triplett, part 2
In reply to Paul E. McKenneyFrom: Paul E. McKenney <[email protected]>
--- include/linux/rcupdate.h | 4 ++ include/linux/rcutree.h | 6 +--- init/main.c | 1 - kernel/rcutree.c | 72 +++++++++++++++++++++------------------------- kernel/rcutree.h | 31 ++++++++++++++----- kernel/rcutree_plugin.h | 23 +++++++++++++- 6 files changed, 82 insertions(+), 55 deletions(-)
include/linux/rcupdate.h +4 -0
@@ -196,6 +196,8 @@ static inline void rcu_read_lock_sched(void)__acquire (RCU_SCHED );rcu_read_acquire (); } + +/* Used by lockdep and tracing: cannot be traced, cannot call lockdep. */ static inline notrace void rcu_read_lock_sched_notrace(void) {preempt_disable_notrace ();
@@ -213,6 +215,8 @@ static inline void rcu_read_unlock_sched(void)__release (RCU_SCHED );preempt_enable (); } + +/* Used by lockdep and tracing: cannot be traced, cannot call lockdep. */ static inline notrace void rcu_read_unlock_sched_notrace(void) {__release (RCU_SCHED );
include/linux/rcutree.h +1 -5
@@ -90,10 +90,6 @@ extern long rcu_batches_completed(void);extern long rcu_batches_completed_bh (void );extern long rcu_batches_completed_sched (void ); -static inline void rcu_init_sched (void ) -{ -} -#ifdef CONFIG_NO_HZ void rcu_enter_nohz (void );void rcu_exit_nohz (void );
@@ -106,7 +102,7 @@ static inline void rcu_exit_nohz(void) }#endif /* CONFIG_NO_HZ */ -/* A context switch is a grace period for rcutree. */ +/* A context switch is a grace period for RCU-sched and RCU-bh. */ static inline int rcu_blocking_is_gp (void ) {return num_online_cpus () ==1 ;
init/main.c +0 -1
@@ -804,7 +804,6 @@ static void __init do_initcalls(void)*/ static void __init do_basic_setup (void ) { -rcu_init_sched ();/* needed by module_init stage. */ init_workqueues ();cpuset_init_smp ();usermodehelper_init ();
kernel/rcutree.c +33 -39
111 lines not on this page · raw message
kernel/rcutree.h +23 -8
@@ -79,15 +79,21 @@ struct rcu_dynticks {* Definition for node within the RCU grace-period-detection hierarchy. */ struct rcu_node { -spinlock_t lock; +spinlock_t lock;/* Root rcu_node's lock protects some */ +/* rcu_state fields as well as following. */ long gpnum;/* Current grace period for this node. */ /* This will either be equal to or one */ /* behind the root rcu_node's gpnum. */ unsigned long qsmask;/* CPUs or groups that need to switch in */ /* order for current grace period to proceed.*/ +/* In leaf rcu_node, each bit corresponds to */ +/* an rcu_data structure, otherwise, each */ +/* bit corresponds to a child rcu_node */ +/* structure. */ unsigned long qsmaskinit;/* Per-GP initialization for qsmask. */ unsigned long grpmask;/* Mask to apply to parent qsmask. */ +/* Only one bit will be set in this mask. */ int grplo;/* lowest-numbered CPU or group here. */ int grphi;/* highest-numbered CPU or group here. */ u8 grpnum;/* CPU/group number for next level up. */
@@ -95,6 +101,9 @@ struct rcu_node {struct rcu_node *parent;struct list_head blocked_tasks[2 ];/* Tasks blocked in RCU read-side critsect. */ +/* Grace period number (->gpnum) x blocked */ +/* by tasks on the (x & 0x1) element of the */ +/* blocked_tasks[] array. */ }____cacheline_internodealigned_in_smp ;/* Index values for nxttail array in struct rcu_data. */
@@ -126,19 +135,22 @@ struct rcu_data {* Any of the partitions might be empty, in which case the * pointer to that partition will be equal to the pointer for * the following partition. When the list is empty, all of -* the nxttail elements point to nxtlist, which is NULL. +* the nxttail elements point to the ->nxtlist pointer itself, +* which in that case is NULL. * -* [*nxttail[RCU_NEXT_READY_TAIL], NULL = *nxttail[RCU_NEXT_TAIL]): -* Entries that might have arrived after current GP ended -* [*nxttail[RCU_WAIT_TAIL], *nxttail[RCU_NEXT_READY_TAIL]): -* Entries known to have arrived before current GP ended -* [*nxttail[RCU_DONE_TAIL], *nxttail[RCU_WAIT_TAIL]): -* Entries that batch # <= ->completed - 1: waiting for current GP * [nxtlist, *nxttail[RCU_DONE_TAIL]): * Entries that batch # <= ->completed * The grace period for these entries has completed, and * the other grace-period-completed entries may be moved * here temporarily in rcu_process_callbacks(). +* [*nxttail[RCU_DONE_TAIL], *nxttail[RCU_WAIT_TAIL]): +* Entries that batch # <= ->completed - 1: waiting for current GP +* [*nxttail[RCU_WAIT_TAIL], *nxttail[RCU_NEXT_READY_TAIL]): +* Entries known to have arrived before current GP ended +* [*nxttail[RCU_NEXT_READY_TAIL], *nxttail[RCU_NEXT_TAIL]): +* Entries that might have arrived after current GP ended +* Note that the value of *nxttail[RCU_NEXT_TAIL] will +* always be NULL, as this is the end of the list. */ struct rcu_head *nxtlist;struct rcu_head **nxttail[RCU_NEXT_SIZE ];
@@ -216,6 +228,9 @@ struct rcu_state {/* Force QS state. */ long gpnum;/* Current gp number. */ long completed;/* # of last completed gp. */ + +/* End of fields guarded by root rcu_node's lock. */ +spinlock_t onofflock;/* exclude on/offline and */ /* starting new GP. */ spinlock_t fqslock;/* Only one task forcing */
kernel/rcutree_plugin.h +21 -2
@@ -420,6 +420,18 @@ static void __cpuinit rcu_preempt_init_percpu_data(int cpu) }/* +* Initialize preemptable RCU's state structures. +*/ +static void __init __rcu_init_preempt (void ) +{ +int i;/* All used by RCU_INIT_FLAVOR(). */ +int j; +struct rcu_node *rnp; + +RCU_INIT_FLAVOR (&rcu_preempt_state, rcu_preempt_data); +} + +/* * Check for a task exiting while in a preemptable-RCU read-side * critical section, clean up if so. No need to issue warnings, * as debug_check_no_locks_held() already does this if lockdep
@@ -519,7 +531,7 @@ static void rcu_preempt_offline_cpu(int cpu)* Because preemptable RCU does not exist, it never has any callbacks * to check. */ -void rcu_preempt_check_callbacks (int cpu) +static void rcu_preempt_check_callbacks (int cpu) { }
@@ -527,7 +539,7 @@ void rcu_preempt_check_callbacks(int cpu)* Because preemptable RCU does not exist, it never has any callbacks * to process. */ -void rcu_preempt_process_callbacks (void ) +static void rcu_preempt_process_callbacks (void ) { }
@@ -564,4 +576,11 @@ static void __cpuinit rcu_preempt_init_percpu_data(int cpu) { } +/* +* Because preemptable RCU does not exist, it need not be initialized. +*/ +static void __init __rcu_init_preempt (void ) +{ +} +#endif /* #else #ifdef CONFIG_TREE_PREEMPT_RCU */
Signature · 1 line
1.5.2.5
PATCH 4/4 v6 RCU: the bloatwatch edition
In reply to Paul E. McKenneyFrom: Paul E. McKenney <[email protected]>
This patch is a version of RCU designed for !SMP provided for a small-footprint RCU implementation. In particular, the implementation of synchronize_rcu() is extremely lightweight and high performance. It passes rcutorture testing in each of the four relevant configurations (combinations of NO_HZ and PREEMPT) on x86. This saves about 1K bytes compared to old Classic RCU (which is no longer in mainline), and more than three kilobytes compared to Hierarchical RCU (updated to 2.6.30):
CONFIG_TREE_RCU:
text data bss dec filename
663 32 20 715 kernel/rcupdate.o
3278 528 44 3850 kernel/rcutree.o
4565 Total (vs 4045 for v4)
CONFIG_TREE_PREEMPT_RCU:
text data bss dec filename
743 32 20 795 kernel/rcupdate.o
4548 752 60 5360 kernel/rcutree.o
6155 Total (N/A for v4)
CONFIG_TINY_RCU:
text data bss dec filename
574 32 20 626 kernel/rcupdate.o
486 28 0 514 kernel/rcutiny.o
1140 Total (vs 1024 for v5)
The above is for x86. Your mileage may vary on other platforms.
Changes from v5 (http://lkml.org/lkml/2009/6/23/12).
o Ported to latest pre-2.6.32 merge window kernel.
- Renamed rcu_qsctr_inc() to rcu_sched_qs().
- Renamed rcu_bh_qsctr_inc() to rcu_bh_qs().
- Provided trivial rcu_cpu_notify().
- Provided trivial exit_rcu().
- Provided trivial rcu_needs_cpu().
- Fixed up the rcu_*_enter/exit() functions in linux/hardirq.h.
o Removed the dependence on EMBEDDED, with a view to making
TINY_RCU default for !SMP at some time in the future.
o Added (trivial) support for expedited grace periods.
Changes from v4 (http://lkml.org/lkml/2009/5/2/91) include:
o Squeeze the size down a bit further by removing the
->completed field from struct rcu_ctrlblk.
o This permits synchronize_rcu() to become the empty function.
Previous concerns about rcutorture were unfounded, as rcutorture correctly handles a constant value from rcu_batches_completed() and rcu_batches_completed_bh().
Changes from v3 (http://lkml.org/lkml/2009/3/29/221) include:
o Changed rcu_batches_completed(), rcu_batches_completed_bh()
rcu_enter_nohz(), rcu_exit_nohz(), rcu_nmi_enter(), and rcu_nmi_exit(), to be static inlines, as suggested by David Howells. Doing this saves about 100 bytes from rcutiny.o. (The numbers between v3 and this v4 of the patch are not directly comparable, since they are against different versions of Linux.)
Changes from v2 (http://lkml.org/lkml/2009/2/3/333) include:
o Fix whitespace issues.
o Change short-circuit "||" operator to instead be "+" in order to
fix performance bug noted by "kraai" on LWN. (http://lwn.net/Articles/324348/)
Changes from v1 (http://lkml.org/lkml/2009/1/13/440) include:
o This version depends on EMBEDDED as well as !SMP, as suggested
by Ingo.
o Updated rcu_needs_cpu() to unconditionally return zero,
permitting the CPU to enter dynticks-idle mode at any time. This works because callbacks can be invoked upon entry to dynticks-idle mode.
o I am now OK with this being included, based on a poll at the
Kernel Miniconf at linux.conf.au, where about ten people said that they cared about saving 900 bytes on single-CPU systems.
o Applies to both mainline and tip/core/rcu.
--- include/linux/hardirq.h | 24 +++++ include/linux/rcupdate.h | 6 + include/linux/rcutiny.h | 103 +++++++++++++++++++ init/Kconfig | 9 ++ kernel/Makefile | 1 + kernel/rcupdate.c | 4 + kernel/rcutiny.c | 245 ++++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 392 insertions(+), 0 deletions(-) create mode 100644 include/linux/rcutiny.h create mode 100644 kernel/rcutiny.c
include/linux/hardirq.h +24 -0
@@ -133,10 +133,34 @@ static inline void account_system_vtime(struct task_struct *tsk)#endif #if defined(CONFIG_NO_HZ ) +#if defined(CONFIG_TINY_RCU ) +extern void rcu_enter_nohz (void ); +extern void rcu_exit_nohz (void ); + +static inline void rcu_irq_enter (void ) +{ +rcu_exit_nohz (); +} + +static inline void rcu_irq_exit (void ) +{ +rcu_enter_nohz (); +} + +static inline void rcu_nmi_enter (void ) +{ +} + +static inline void rcu_nmi_exit (void ) +{ +} + +#else extern void rcu_irq_enter (void );extern void rcu_irq_exit (void );extern void rcu_nmi_enter (void );extern void rcu_nmi_exit (void ); +#endif #else # define rcu_irq_enter () do { } while (0)# define rcu_irq_exit () do { } while (0)
include/linux/rcupdate.h +6 -0
@@ -68,11 +68,17 @@ extern int sched_expedited_torture_stats(char *page);/* Internal to kernel */ extern void rcu_init (void );extern void rcu_scheduler_starting (void ); +#ifndef CONFIG_TINY_RCU extern int rcu_needs_cpu (int cpu); +#else +static inline int rcu_needs_cpu (int cpu) {return 0 ; } +#endif extern int rcu_scheduler_active;#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_TREE_PREEMPT_RCU )#include <linux/rcutree.h> +#elif CONFIG_TINY_RCU +#include <linux/rcutiny.h> #else #error "Unknown RCU implementation specified to kernel configuration"#endif
include/linux/rcutiny.h new file +103 -0
103 lines not on this page · raw message
init/Kconfig +9 -0
@@ -334,6 +334,15 @@ config TREE_PREEMPT_RCU is also required. It also scales down nicely to smaller systems. +config TINY_RCU + bool "UP-only small-memory-footprint RCU" + depends on !SMP + help + This option selects the RCU implementation that is + designed for UP systems from which real-time response + is not required. This option greatly reduces the + memory footprint of RCU. + endchoice config RCU_TRACE
kernel/Makefile +1 -0
@@ -83,6 +83,7 @@ obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o obj-$(CONFIG_TREE_RCU) += rcutree.o obj-$(CONFIG_TREE_PREEMPT_RCU) += rcutree.o obj-$(CONFIG_TREE_RCU_TRACE) += rcutree_trace.o +obj-$(CONFIG_TINY_RCU) += rcutiny.o obj-$(CONFIG_RELAY) += relay.o obj-$(CONFIG_SYSCTL) += utsname_sysctl.o obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o
kernel/rcupdate.c +4 -0
@@ -74,6 +74,8 @@ void wakeme_after_rcu(struct rcu_head *head)complete (&rcu->completion); } +#ifndef CONFIG_TINY_RCU +#ifdef CONFIG_TREE_PREEMPT_RCU /**
@@ -164,6 +166,8 @@ void synchronize_rcu_bh(void) }EXPORT_SYMBOL_GPL (synchronize_rcu_bh); +#endif /* #ifndef CONFIG_TINY_RCU */ +static void rcu_barrier_callback (struct rcu_head *notused) {if (atomic_dec_and_test (&rcu_barrier_cpu_count))
kernel/rcutiny.c new file +245 -0
245 lines not on this page · raw message
Signature · 1 line
1.5.2.5
PATCH 3/4 Apply Ingo's checkpatch feedback
In reply to Paul E. McKenneyFrom: Paul E. McKenney <[email protected]>
Move declarations and update storage classes to make checkpatch happy.
--- include/linux/rcutree.h | 6 +++++- kernel/rcupdate.c | 3 --- kernel/rcutorture.c | 4 +--- kernel/rcutree.c | 23 ----------------------- kernel/rcutree.h | 26 +++++++++++++++++++++++++- kernel/rcutree_trace.c | 12 ++++++------ 6 files changed, 37 insertions(+), 37 deletions(-)
include/linux/rcutree.h +5 -1
@@ -30,10 +30,14 @@#ifndef __LINUX_RCUTREE_H#define __LINUX_RCUTREE_H +struct notifier_block ; +extern void rcu_sched_qs (int cpu);extern void rcu_bh_qs (int cpu); - +extern int rcu_cpu_notify (struct notifier_block *self, +unsigned long action,void *hcpu);extern int rcu_needs_cpu (int cpu); +extern int rcu_expedited_torture_stats (char *page);#ifdef CONFIG_TREE_PREEMPT_RCU
kernel/rcupdate.c +0 -3
@@ -259,9 +259,6 @@ static void rcu_migrate_callback(struct rcu_head *notused)wake_up (&rcu_migrate_wq); } -extern int rcu_cpu_notify (struct notifier_block *self, -unsigned long action,void *hcpu); -static int __cpuinit rcu_barrier_cpu_hotplug (struct notifier_block *self,unsigned long action,void *hcpu) {
kernel/rcutorture.c +1 -3
@@ -605,8 +605,6 @@ static struct rcu_torture_ops sched_ops_sync = { .name = "sched_sync" }; -extern int rcu_expedited_torture_stats (char *page); -static struct rcu_torture_ops sched_expedited_ops = { .init = rcu_sync_torture_init, .cleanup =NULL ,
@@ -648,7 +646,7 @@ rcu_torture_writer(void *arg) old_rp = rcu_torture_current; rp->rtort_mbtest =1 ;rcu_assign_pointer (rcu_torture_current, rp); -smp_wmb (); +smp_wmb ();/* Mods to old_rp must follow rcu_assign_pointer() */ if (old_rp) { i = old_rp->rtort_pipe_count;if (i >RCU_TORTURE_PIPE_LEN )
kernel/rcutree.c +0 -23
@@ -80,29 +80,6 @@ DEFINE_PER_CPU(struct rcu_data, rcu_sched_data);struct rcu_state rcu_bh_state =RCU_STATE_INITIALIZER (rcu_bh_state); DEFINE_PER_CPU(struct rcu_data, rcu_bh_data); -/* Forward declarations for rcutree_plugin.h */ -static inline void rcu_bootup_announce (void ); -long rcu_batches_completed (void ); -static void rcu_preempt_note_context_switch (int cpu); -static int rcu_preempted_readers (struct rcu_node *rnp); -#ifdef CONFIG_RCU_CPU_STALL_DETECTOR -static void rcu_print_task_stall (struct rcu_node *rnp); -#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */ -static void rcu_preempt_check_blocked_tasks (struct rcu_node *rnp); -#ifdef CONFIG_HOTPLUG_CPU -static void rcu_preempt_offline_tasks (struct rcu_state *rsp, -struct rcu_node *rnp, -struct rcu_data *rdp); -static void rcu_preempt_offline_cpu (int cpu); -#endif /* #ifdef CONFIG_HOTPLUG_CPU */ -static void rcu_preempt_check_callbacks (int cpu); -static void rcu_preempt_process_callbacks (void ); -void call_rcu (struct rcu_head *head,void (*func)(struct rcu_head *rcu)); -static int rcu_preempt_pending (int cpu); -static int rcu_preempt_needs_cpu (int cpu); -static void __cpuinit rcu_preempt_init_percpu_data (int cpu); -static void __init __rcu_init_preempt (void ); -/* * Return true if an RCU grace period is in progress. The ACCESS_ONCE()s
kernel/rcutree.h +25 -1
@@ -270,5 +270,29 @@ extern struct rcu_state rcu_preempt_state; DECLARE_PER_CPU(struct rcu_data, rcu_preempt_data);#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ -#endif /* #ifdef RCU_TREE_NONCORE */ +#else /* #ifdef RCU_TREE_NONCORE */ +/* Forward declarations for rcutree_plugin.h */ +static inline void rcu_bootup_announce (void ); +long rcu_batches_completed (void ); +static void rcu_preempt_note_context_switch (int cpu); +static int rcu_preempted_readers (struct rcu_node *rnp); +#ifdef CONFIG_RCU_CPU_STALL_DETECTOR +static void rcu_print_task_stall (struct rcu_node *rnp); +#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */ +static void rcu_preempt_check_blocked_tasks (struct rcu_node *rnp); +#ifdef CONFIG_HOTPLUG_CPU +static void rcu_preempt_offline_tasks (struct rcu_state *rsp, +struct rcu_node *rnp, +struct rcu_data *rdp); +static void rcu_preempt_offline_cpu (int cpu); +#endif /* #ifdef CONFIG_HOTPLUG_CPU */ +static void rcu_preempt_check_callbacks (int cpu); +static void rcu_preempt_process_callbacks (void ); +void call_rcu (struct rcu_head *head,void (*func)(struct rcu_head *rcu)); +static int rcu_preempt_pending (int cpu); +static int rcu_preempt_needs_cpu (int cpu); +static void __cpuinit rcu_preempt_init_percpu_data (int cpu); +static void __init __rcu_init_preempt (void ); + +#endif /* #else #ifdef RCU_TREE_NONCORE */
kernel/rcutree_trace.c +6 -6
@@ -93,7 +93,7 @@ static int rcudata_open(struct inode *inode, struct file *file)return single_open (file, show_rcudata,NULL ); } -static struct file_operations rcudata_fops = { +static const struct file_operations rcudata_fops = { .owner =THIS_MODULE , .open = rcudata_open, .read = seq_read,
@@ -145,7 +145,7 @@ static int rcudata_csv_open(struct inode *inode, struct file *file)return single_open (file, show_rcudata_csv,NULL ); } -static struct file_operations rcudata_csv_fops = { +static const struct file_operations rcudata_csv_fops = { .owner =THIS_MODULE , .open = rcudata_csv_open, .read = seq_read,
@@ -159,7 +159,7 @@ static void print_one_rcu_state(struct seq_file *m, struct rcu_state *rsp)struct rcu_node *rnp; seq_printf(m, "c=%ld g=%ld s=%d jfq=%ld j=%x " - "nfqs=%lu/nfqsng=%lu(%lu) fqlh=%lu\n", + "nfqs=%lu/nfqsng=%lu(%lu) fqlh=%lu\n", rsp->completed, rsp->gpnum, rsp->signaled, (long)(rsp->jiffies_force_qs - jiffies), (int)(jiffies & 0xffff),
@@ -196,7 +196,7 @@ static int rcuhier_open(struct inode *inode, struct file *file)return single_open (file, show_rcuhier,NULL ); } -static struct file_operations rcuhier_fops = { +static const struct file_operations rcuhier_fops = { .owner =THIS_MODULE , .open = rcuhier_open, .read = seq_read,
@@ -222,7 +222,7 @@ static int rcugp_open(struct inode *inode, struct file *file)return single_open (file, show_rcugp,NULL ); } -static struct file_operations rcugp_fops = { +static const struct file_operations rcugp_fops = { .owner =THIS_MODULE , .open = rcugp_open, .read = seq_read,
@@ -276,7 +276,7 @@ static int rcu_pending_open(struct inode *inode, struct file *file)return single_open (file, show_rcu_pending,NULL ); } -static struct file_operations rcu_pending_fops = { +static const struct file_operations rcu_pending_fops = { .owner =THIS_MODULE , .open = rcu_pending_open, .read = seq_read,
Signature · 1 line
1.5.2.5
Quoted 3 lines · Paul E. McKenney
This patch set applies two sets of review feedback from Josh Triplett and checkpatch feedback from Ingo Molnar. It also contains a forward-port of Bloatwatch RCU, courtesy of David Howells.
It would be much easier to review if these patches had a changelog describing the problem they address :-)
The first one seems to do:
A)
- aggregate 'rsp->completed == rsp->gpnum' expressions into a common function.
- consistently use ACCESS_ONCE() in the above mentioned function
B)
- use DIV_ROUND_UP()
C)
- aggregate list_empty(&rnp->blocked_tasks[rnp->gpnum & 0x01])
expressions into a common function.
Could have been three patches, but ACK.
The second patch seems to mostly add comments, but also moves code around and makes it static, which could have been split in two patches.
The purpose of the move code around bit could be a cleanup?
Looks to preserve the logic, but didn't go out on a limb to verify, ACK.
The third and fourth do have an adequate changelog :-)
Quoted 19 lines · Peter Zijlstra
Quoted 3 lines · Paul E. McKenney
This patch set applies two sets of review feedback from Josh Triplett and checkpatch feedback from Ingo Molnar. It also contains a forward-port of Bloatwatch RCU, courtesy of David Howells.
It would be much easier to review if these patches had a changelog describing the problem they address :-)
The first one seems to do:
A)
- aggregate 'rsp->completed==rsp->gpnum' expressions into a common function.- consistently use
ACCESS_ONCE()in the above mentioned functionB)
- useDIV_ROUND_UP()
Quoted 6 lines
C)
- aggregatelist_empty(&rnp->blocked_tasks[rnp->gpnum & 0x01])
expressions into a common function.Could have been three patches, but ACK.
Good point, I will re-submit with the changelog and your Acked-by.
Quoted 4 lines
The second patch seems to mostly add comments, but also moves code around and makes it static, which could have been split in two patches.
The purpose of the move code around bit could be a cleanup?
Yep. The main point was that the old way had a bunch of random forward declarations, while the new way has the plugins themselves forward-declared, which (to me at least) makes a lot more sense.
Quoted 1 line
Looks to preserve the logic, but didn't go out on a limb to verify, ACK.
Again, will re-submit with your acked-by.
Quoted 1 line
The third and fourth do have an adequate changelog :-)
;-)
Thank you for looking this over!!!
Thanx, Paul
Quoted 17 lines · Paul E. McKenney
This patch set applies two sets of review feedback from Josh Triplett and checkpatch feedback from Ingo Molnar. It also contains a forward-port of Bloatwatch RCU, courtesy of David Howells.
b/include/linux/hardirq.h | 24 ++++ b/include/linux/rcupdate.h | 4 b/include/linux/rcutiny.h | 103 ++++++++++++++++++ b/include/linux/rcutree.h | 1 b/init/Kconfig | 9 + b/init/main.c | 1 b/kernel/Makefile | 1 b/kernel/rcupdate.c | 3 b/kernel/rcutiny.c | 246 ++++++++++++++++++++++++++++++++++++++++++++- b/kernel/rcutorture.c | 4 b/kernel/rcutree.c | 29 +++-- b/kernel/rcutree.h | 6 - b/kernel/rcutree_plugin.h | 51 ++++-----
Quoted 8 lines
b/kernel/rcutree_trace.c | 13 +- include/linux/rcupdate.h | 6 + include/linux/rcutree.h | 12 +- kernel/rcupdate.c | 4 kernel/rcutree.c | 95 ++++++----------- kernel/rcutree.h | 57 ++++++++-- kernel/rcutree_plugin.h | 24 +++- 20 files changed, 557 insertions(+), 136 deletions(-)
For all four patches:
- Josh Triplett