rcu: Tiny RCU, expedited RCU, rcutorture fixes, lockdep simplification
Subsystems (3)
cover rcu: Tiny RCU, expedited RCU, rcutorture fixes, lockdep simplification
Updated patches for post 2.6.32:
o RCU the Bloatwatch Edition, v8
o Add synchronize_rcu_expedited().
o Add rcutorture tests for synchronize_rcu_expedited().
o Update RCU documentation for synchronize_rcu_expedited().
o Make rcutorture list the torture types it knows about when
complaining about the one specified.
o Simplify creation of the separate lockdep class for the
root rcu_node structure's lock.
b/Documentation/RCU/whatisRCU.txt | 3 b/include/linux/hardirq.h | 24 +++ b/include/linux/rcupdate.h | 6 b/include/linux/rcutiny.h | 97 ++++++++++++ b/include/linux/srcu.h | 1 b/init/Kconfig | 9 + b/kernel/Makefile | 1 b/kernel/rcupdate.c | 4 b/kernel/rcutiny.c | 295 +++++++++++++++++++++++++++++++++++++- b/kernel/rcutorture.c | 26 ++- b/kernel/rcutree.c | 5 b/kernel/rcutree.h | 3 b/kernel/srcu.c | 75 ++++++--- kernel/rcutorture.c | 7 14 files changed, 519 insertions(+), 37 deletions(-)
PATCH 5/6 rcu: Improve rcutorture diagnostics when bad torture_type specified
From: Paul E. McKenney <[email protected]>
Make rcutorture list the available torture_type values when it doesn't like the one specified.
--- kernel/rcutorture.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-)
kernel/rcutorture.c +5 -1
@@ -1129,8 +1129,12 @@ rcu_torture_init(void)break ; }if (i ==ARRAY_SIZE (torture_ops)) { -printk (KERN_ALERT "rcutorture: invalid torture type: \"%s\"\n" , +printk (KERN_ALERT "rcu-torture: invalid torture type: \"%s\"\n" , torture_type); +printk (KERN_ALERT "rcu-torture types:" ); +for (i =0 ; i <ARRAY_SIZE (torture_ops); i++) +printk (KERN_ALERT " %s" , torture_ops[i]->name); +printk (KERN_ALERT "\n" );mutex_unlock (&fullstop_mutex);return -EINVAL ; }
Signature · 1 line
1.5.2.5
PATCH 2/6 rcu: Add synchronize_srcu_expedited()
In reply to Paul E. McKenneyFrom: Paul E. McKenney <[email protected]>
This patch creates a synchronize_srcu_expedited() that uses synchronize_sched_expedited() where synchronize_srcu() uses synchronize_sched(). The synchronize_srcu() and synchronize_srcu_expedited() functions become one-liners that pass synchronize_sched() or synchronize_sched_expedited(), repectively, to a new __synchronize_srcu() function.
While in the file, move the EXPORT_SYMBOL_GPL()s to immediately follow the corresponding functions.
--- include/linux/srcu.h | 1 + kernel/srcu.c | 74 ++++++++++++++++++++++++++++++++++--------------- 2 files changed, 52 insertions(+), 23 deletions(-)
include/linux/srcu.h +1 -0
@@ -48,6 +48,7 @@ void cleanup_srcu_struct(struct srcu_struct *sp); int srcu_read_lock(struct srcu_struct *sp) __acquires(sp); void srcu_read_unlock(struct srcu_struct *sp, int idx) __releases(sp);void synchronize_srcu (struct srcu_struct *sp); +void synchronize_srcu_expedited (struct srcu_struct *sp);long srcu_batches_completed (struct srcu_struct *sp);#endif
kernel/srcu.c +51 -23
128 lines not on this page · raw message
Signature · 1 line
1.5.2.5
PATCH 3/6 rcu: add synchronize_srcu_expedited() to the rcutorture test suite
In reply to Paul E. McKenneyFrom: Paul E. McKenney <[email protected]>
Adds the "srcu_expedited" torture type, and also renames sched_ops_sync to sched_sync_ops for consistency while we are in this file.
--- kernel/rcutorture.c | 25 ++++++++++++++++++++++--- 1 files changed, 22 insertions(+), 3 deletions(-)
kernel/rcutorture.c +22 -3
@@ -547,6 +547,25 @@ static struct rcu_torture_ops srcu_ops = { .name = "srcu" }; +static void srcu_torture_synchronize_expedited (void ) +{ +synchronize_srcu_expedited (&srcu_ctl); +} + +static struct rcu_torture_ops srcu_expedited_ops = { + .init = srcu_torture_init, + .cleanup = srcu_torture_cleanup, + .readlock = srcu_torture_read_lock, + .read_delay = srcu_read_delay, + .readunlock = srcu_torture_read_unlock, + .completed = srcu_torture_completed, + .deferred_free = rcu_sync_torture_deferred_free, + .sync = srcu_torture_synchronize_expedited, + .cb_barrier =NULL , + .stats = srcu_torture_stats, + .name ="srcu_expedited" +}; +/* * Definitions for sched torture testing. */
@@ -592,7 +611,7 @@ static struct rcu_torture_ops sched_ops = { .name = "sched" }; -static struct rcu_torture_ops sched_ops_sync = { +static struct rcu_torture_ops sched_sync_ops = { .init = rcu_sync_torture_init, .cleanup =NULL , .readlock = sched_torture_read_lock,
@@ -1098,8 +1117,8 @@ rcu_torture_init(void)int firsterr =0 ;static struct rcu_torture_ops *torture_ops[] = { &rcu_ops, &rcu_sync_ops, &rcu_bh_ops, &rcu_bh_sync_ops, - &sched_expedited_ops, - &srcu_ops, &sched_ops, &sched_ops_sync, }; + &srcu_ops, &srcu_expedited_ops, + &sched_ops, &sched_sync_ops, &sched_expedited_ops, };mutex_lock (&fullstop_mutex);
Signature · 1 line
1.5.2.5
PATCH 1/6 rcu: The Bloatwatch Edition, v8
In reply to Paul E. McKenneyThis patch is a version of RCU designed for !SMP provided for a small-footprint RCU implementation. In particular, the implementation of synchronize_rcu() is extremely lightweight and high performance. It passes rcutorture testing in each of the four relevant configurations (combinations of NO_HZ and PREEMPT) on x86. This saves about 1K bytes compared to old Classic RCU (which is no longer in mainline), and more than three kilobytes compared to Hierarchical RCU (updated to 2.6.30):
CONFIG_TREE_RCU:
text data bss dec filename
183 4 0 187 kernel/rcupdate.o
2783 520 36 3339 kernel/rcutree.o
3526 Total (vs 4565 for v7)
CONFIG_TREE_PREEMPT_RCU:
text data bss dec filename
263 4 0 267 kernel/rcupdate.o
4594 776 52 5422 kernel/rcutree.o
5689 Total (6155 for v7)
CONFIG_TINY_RCU:
text data bss dec filename
96 4 0 100 kernel/rcupdate.o
734 24 0 758 kernel/rcutiny.o
858 Total (vs 848 for v7)
The above is for x86. Your mileage may vary on other platforms. Further compression is possible, but is being procrastinated.
Changes from v7 (http://lkml.org/lkml/2009/10/9/388)
o Apply Lai Jiangshan's review comments (aside from might_sleep()
in synchronize_sched(), which is covered by SMP builds).
o Fix up expedited primitives.
Changes from v6 (http://lkml.org/lkml/2009/9/23/293).
o Forward ported to put it into the 2.6.33 stream.
o Added lockdep support.
o Make lightweight rcu_barrier.
Changes from v5 (http://lkml.org/lkml/2009/6/23/12).
o Ported to latest pre-2.6.32 merge window kernel.
- Renamed rcu_qsctr_inc() to rcu_sched_qs().
- Renamed rcu_bh_qsctr_inc() to rcu_bh_qs().
- Provided trivial rcu_cpu_notify().
- Provided trivial exit_rcu().
- Provided trivial rcu_needs_cpu().
- Fixed up the rcu_*_enter/exit() functions in linux/hardirq.h.
o Removed the dependence on EMBEDDED, with a view to making
TINY_RCU default for !SMP at some time in the future.
o Added (trivial) support for expedited grace periods.
Changes from v4 (http://lkml.org/lkml/2009/5/2/91) include:
o Squeeze the size down a bit further by removing the
->completed field from struct rcu_ctrlblk.
o This permits synchronize_rcu() to become the empty function.
Previous concerns about rcutorture were unfounded, as rcutorture correctly handles a constant value from rcu_batches_completed() and rcu_batches_completed_bh().
Changes from v3 (http://lkml.org/lkml/2009/3/29/221) include:
o Changed rcu_batches_completed(), rcu_batches_completed_bh()
rcu_enter_nohz(), rcu_exit_nohz(), rcu_nmi_enter(), and rcu_nmi_exit(), to be static inlines, as suggested by David Howells. Doing this saves about 100 bytes from rcutiny.o. (The numbers between v3 and this v4 of the patch are not directly comparable, since they are against different versions of Linux.)
Changes from v2 (http://lkml.org/lkml/2009/2/3/333) include:
o Fix whitespace issues.
o Change short-circuit "||" operator to instead be "+" in order to
fix performance bug noted by "kraai" on LWN. (http://lwn.net/Articles/324348/)
Changes from v1 (http://lkml.org/lkml/2009/1/13/440) include:
o This version depends on EMBEDDED as well as !SMP, as suggested
by Ingo.
o Updated rcu_needs_cpu() to unconditionally return zero,
permitting the CPU to enter dynticks-idle mode at any time. This works because callbacks can be invoked upon entry to dynticks-idle mode.
o Paul is now OK with this being included, based on a poll at the
Kernel Miniconf at linux.conf.au, where about ten people said that they cared about saving 900 bytes on single-CPU systems.
o Applies to both mainline and tip/core/rcu.
--- include/linux/hardirq.h | 24 ++++ include/linux/rcupdate.h | 6 + include/linux/rcutiny.h | 97 +++++++++++++++ init/Kconfig | 9 ++ kernel/Makefile | 1 + kernel/rcupdate.c | 4 + kernel/rcutiny.c | 294 ++++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 435 insertions(+), 0 deletions(-) create mode 100644 include/linux/rcutiny.h create mode 100644 kernel/rcutiny.c
include/linux/hardirq.h +24 -0
@@ -139,10 +139,34 @@ static inline void account_system_vtime(struct task_struct *tsk)#endif #if defined(CONFIG_NO_HZ ) +#if defined(CONFIG_TINY_RCU ) +extern void rcu_enter_nohz (void ); +extern void rcu_exit_nohz (void ); + +static inline void rcu_irq_enter (void ) +{ +rcu_exit_nohz (); +} + +static inline void rcu_irq_exit (void ) +{ +rcu_enter_nohz (); +} + +static inline void rcu_nmi_enter (void ) +{ +} + +static inline void rcu_nmi_exit (void ) +{ +} + +#else extern void rcu_irq_enter (void );extern void rcu_irq_exit (void );extern void rcu_nmi_enter (void );extern void rcu_nmi_exit (void ); +#endif #else # define rcu_irq_enter () do { } while (0)# define rcu_irq_exit () do { } while (0)
include/linux/rcupdate.h +6 -0
@@ -68,11 +68,17 @@ extern int sched_expedited_torture_stats(char *page);/* Internal to kernel */ extern void rcu_init (void );extern void rcu_scheduler_starting (void ); +#ifndef CONFIG_TINY_RCU extern int rcu_needs_cpu (int cpu); +#else +static inline int rcu_needs_cpu (int cpu) {return 0 ; } +#endif extern int rcu_scheduler_active;#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_TREE_PREEMPT_RCU )#include <linux/rcutree.h> +#elif CONFIG_TINY_RCU +#include <linux/rcutiny.h> #else #error "Unknown RCU implementation specified to kernel configuration"#endif
include/linux/rcutiny.h new file +97 -0
97 lines not on this page · raw message
init/Kconfig +9 -0
@@ -334,6 +334,15 @@ config TREE_PREEMPT_RCU is also required. It also scales down nicely to smaller systems. +config TINY_RCU + bool "UP-only small-memory-footprint RCU" + depends on !SMP + help + This option selects the RCU implementation that is + designed for UP systems from which real-time response + is not required. This option greatly reduces the + memory footprint of RCU. + endchoice config RCU_TRACE
kernel/Makefile +1 -0
@@ -82,6 +82,7 @@ obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o obj-$(CONFIG_TREE_RCU) += rcutree.o obj-$(CONFIG_TREE_PREEMPT_RCU) += rcutree.o obj-$(CONFIG_TREE_RCU_TRACE) += rcutree_trace.o +obj-$(CONFIG_TINY_RCU) += rcutiny.o obj-$(CONFIG_RELAY) += relay.o obj-$(CONFIG_SYSCTL) += utsname_sysctl.o obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o
kernel/rcupdate.c +4 -0
@@ -67,6 +67,8 @@ void wakeme_after_rcu(struct rcu_head *head)complete (&rcu->completion); } +#ifndef CONFIG_TINY_RCU +#ifdef CONFIG_TREE_PREEMPT_RCU /**
@@ -157,6 +159,8 @@ void synchronize_rcu_bh(void) }EXPORT_SYMBOL_GPL (synchronize_rcu_bh); +#endif /* #ifndef CONFIG_TINY_RCU */ +static int __cpuinit rcu_barrier_cpu_hotplug (struct notifier_block *self,unsigned long action,void *hcpu) {
kernel/rcutiny.c new file +294 -0
294 lines not on this page · raw message
Signature · 1 line
1.5.2.5
PATCH 4/6 rcu: add synchronize_srcu_expedited() to the documentation
In reply to Paul E. McKenneyFrom: Paul E. McKenney <[email protected]>
--- Documentation/RCU/whatisRCU.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
Documentation/RCU/whatisRCU.txt +1 -1
@@ -830,7 +830,7 @@ sched: Critical sections Grace period Barrier SRCU: Critical sections Grace period Barrier srcu_read_lock synchronize_srcu N/A - srcu_read_unlock + srcu_read_unlock synchronize_srcu_expedited SRCU: Initialization/cleanup init_srcu_struct
Signature · 1 line
1.5.2.5
PATCH 6/6 rcu: Simplify creating of lockdep class for root rcu_node
In reply to Paul E. McKenneyFrom: Paul E. McKenney <[email protected]>
Use lockdep_set_class() to simplify the code and to avoid any additional overhead in the !LOCKDEP case.
--- kernel/rcutree.c | 5 ++--- kernel/rcutree.h | 2 ++ 2 files changed, 4 insertions(+), 3 deletions(-)
kernel/rcutree.c +2 -3
@@ -1666,8 +1666,7 @@ static void __init rcu_init_one(struct rcu_state *rsp) cpustride *= rsp->levelspread[i]; rnp = rsp->level[i];for (j =0 ; j < rsp->levelcnt[i]; j++, rnp++) { -if (rnp !=rcu_get_root (rsp)) -spin_lock_init (&rnp->lock); +spin_lock_init (&rnp->lock); rnp->gpnum =0 ; rnp->qsmask =0 ; rnp->qsmaskinit =0 ;
@@ -1690,7 +1689,7 @@ static void __init rcu_init_one(struct rcu_state *rsp)INIT_LIST_HEAD (&rnp->blocked_tasks[1 ]); } } -spin_lock_init (&rcu_get_root (rsp)->lock); +lockdep_set_class (&rcu_get_root (rsp)->lock, &rcu_root_class); }/*
kernel/rcutree.h +2 -0
@@ -296,6 +296,8 @@ DECLARE_PER_CPU(struct rcu_data, rcu_preempt_data);#else /* #ifdef RCU_TREE_NONCORE */ +static struct lock_class_key rcu_root_class; +/* Forward declarations for rcutree_plugin.h */ static inline void rcu_bootup_announce (void );long rcu_batches_completed (void );
Signature · 1 line
1.5.2.5
Quoted 17 lines · Paul E. McKenney
Updated patches for post 2.6.32:
o RCU the Bloatwatch Edition, v8
o Add
synchronize_rcu_expedited().o Add rcutorture tests for
synchronize_rcu_expedited().o Update RCU documentation for
synchronize_rcu_expedited().o Make rcutorture list the torture types it knows about when
complaining about the one specified.o Simplify creation of the separate lockdep class for the
rootrcu_nodestructure's lock.
Quoted 15 lines
b/Documentation/RCU/whatisRCU.txt | 3 b/include/linux/hardirq.h | 24 +++ b/include/linux/rcupdate.h | 6 b/include/linux/rcutiny.h | 97 ++++++++++++ b/include/linux/srcu.h | 1 b/init/Kconfig | 9 + b/kernel/Makefile | 1 b/kernel/rcupdate.c | 4 b/kernel/rcutiny.c | 295 +++++++++++++++++++++++++++++++++++++- b/kernel/rcutorture.c | 26 ++- b/kernel/rcutree.c | 5 b/kernel/rcutree.h | 3 b/kernel/srcu.c | 75 ++++++--- kernel/rcutorture.c | 7 14 files changed, 519 insertions(+), 37 deletions(-)
For all 6 patches:
Re: [PATCH tip/core/rcu 6/6] rcu: Simplify creating of lockdep class for root rcu_node
In reply to Paul E. McKenneyQuoted 7 lines · Paul E. McKenney
From: Paul E. McKenney <[email protected]>
Use
lockdep_set_class()to simplify the code and to avoid any additional overhead in the !LOCKDEP case.Signed-off-by Peter Zijlstra <[email protected]>Signed-off-by Paul E. McKenney <[email protected]>
This signoff order is wrong - first signoff line must be the author of the patch. (which is a pretty general rule with only rare exceptions)
This patch was from Peter originally, right? In such cases please add:
From: Peter Zijlstra <[email protected]>
To the head of the patch. Or, if Peter acked it, add:
SOB lines are strictly only added by people who send patches to you - i.e. it records the patch handling pathway.
Thanks,
Ingo
Re: [PATCH tip/core/rcu 6/6] rcu: Simplify creating of lockdep class for root rcu_node
In reply to Paul E. McKenneykernel/rcutree.h DECLARE_PER_CPU(struct rcu_data, rcu_preempt_data); · line 300
Quoted 10 lines · Paul E. McKenney
kernel/rcutree.h +2
-0@@ -296,6 +296,8 @@ DECLARE_PER_CPU(struct rcu_data, rcu_preempt_data);#else /* #ifdef RCU_TREE_NONCORE */ +static struct lock_class_key rcu_root_class; +
I think this line needs to be moved into rcutree.c.
kernel/rcutree.h DECLARE_PER_CPU(struct rcu_data, rcu_preempt_data); · line 303
Quoted 3 lines
kernel/rcutree.h +0
-0/* Forward declarations for rcutree_plugin.h */ static inline void rcu_bootup_announce (void );long rcu_batches_completed (void );
For all 6 patches:
[tip:core/rcu] rcu: Add synchronize_srcu_expedited()
In reply to Paul E. McKenneyCommit-ID: 0cd397d33608ae6c97d2ee6c8c43462b419b7e26 Gitweb: http://git.kernel.org/tip/0cd397d33608ae6c97d2ee6c8c43462b419b7e26 Author: Paul E. McKenney <[email protected]> AuthorDate: Sun, 25 Oct 2009 19:03:51 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 26 Oct 2009 09:40:30 +0100
rcu: Add synchronize_srcu_expedited()
This patch creates a synchronize_srcu_expedited() that uses synchronize_sched_expedited() where synchronize_srcu() uses synchronize_sched(). The synchronize_srcu() and synchronize_srcu_expedited() functions become one-liners that pass synchronize_sched() or synchronize_sched_expedited(), repectively, to a new __synchronize_srcu() function.
While in the file, move the EXPORT_SYMBOL_GPL()s to immediately follow the corresponding functions.
LKML-Reference: <12565226354038-git-send-email->
--- include/linux/srcu.h | 1 + kernel/srcu.c | 74 ++++++++++++++++++++++++++++++++++--------------- 2 files changed, 52 insertions(+), 23 deletions(-)
include/linux/srcu.h +1 -0
@@ -48,6 +48,7 @@ void cleanup_srcu_struct(struct srcu_struct *sp); int srcu_read_lock(struct srcu_struct *sp) __acquires(sp); void srcu_read_unlock(struct srcu_struct *sp, int idx) __releases(sp);void synchronize_srcu (struct srcu_struct *sp); +void synchronize_srcu_expedited (struct srcu_struct *sp);long srcu_batches_completed (struct srcu_struct *sp);#endif
kernel/srcu.c +51 -23
128 lines not on this page · raw message
[tip:core/rcu] rcu: "Tiny RCU", The Bloatwatch Edition
In reply to Paul E. McKenneyCommit-ID: 9b1d82fa1611706fa7ee1505f290160a18caf95d Gitweb: http://git.kernel.org/tip/9b1d82fa1611706fa7ee1505f290160a18caf95d Author: Paul E. McKenney <[email protected]> AuthorDate: Sun, 25 Oct 2009 19:03:50 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 26 Oct 2009 09:40:29 +0100
rcu: "Tiny RCU", The Bloatwatch Edition
This patch is a version of RCU designed for !SMP provided for a small-footprint RCU implementation. In particular, the implementation of synchronize_rcu() is extremely lightweight and high performance. It passes rcutorture testing in each of the four relevant configurations (combinations of NO_HZ and PREEMPT) on x86. This saves about 1K bytes compared to old Classic RCU (which is no longer in mainline), and more than three kilobytes compared to Hierarchical RCU (updated to 2.6.30):
CONFIG_TREE_RCU:
text data bss dec filename
183 4 0 187 kernel/rcupdate.o
2783 520 36 3339 kernel/rcutree.o
3526 Total (vs 4565 for v7)
CONFIG_TREE_PREEMPT_RCU:
text data bss dec filename
263 4 0 267 kernel/rcupdate.o
4594 776 52 5422 kernel/rcutree.o
5689 Total (6155 for v7)
CONFIG_TINY_RCU:
text data bss dec filename
96 4 0 100 kernel/rcupdate.o
734 24 0 758 kernel/rcutiny.o
858 Total (vs 848 for v7)
The above is for x86. Your mileage may vary on other platforms. Further compression is possible, but is being procrastinated.
Changes from v7 (http://lkml.org/lkml/2009/10/9/388)
o Apply Lai Jiangshan's review comments (aside frommight_sleep() in synchronize_sched(), which is covered by SMP builds).
o Fix up expedited primitives.
Changes from v6 (http://lkml.org/lkml/2009/9/23/293).
o Forward ported to put it into the 2.6.33 stream.
o Added lockdep support.
o Make lightweight rcu_barrier.
Changes from v5 (http://lkml.org/lkml/2009/6/23/12).
o Ported to latest pre-2.6.32 merge window kernel.
- Renamed rcu_qsctr_inc() to rcu_sched_qs().
- Renamed rcu_bh_qsctr_inc() to rcu_bh_qs().
- Provided trivial rcu_cpu_notify().
- Provided trivial exit_rcu().
- Provided trivial rcu_needs_cpu().
- Fixed up the rcu_*_enter/exit() functions in linux/hardirq.h.
o Removed the dependence on EMBEDDED, with a view to making
TINY_RCU default for !SMP at some time in the future.
o Added (trivial) support for expedited grace periods.
Changes from v4 (http://lkml.org/lkml/2009/5/2/91) include:
o Squeeze the size down a bit further by removing the
->completed field from struct rcu_ctrlblk.
o This permits synchronize_rcu() to become the empty function.
Previous concerns about rcutorture were unfounded, as rcutorture correctly handles a constant value from rcu_batches_completed() and rcu_batches_completed_bh().
Changes from v3 (http://lkml.org/lkml/2009/3/29/221) include:
o Changed rcu_batches_completed(), rcu_batches_completed_bh()
rcu_enter_nohz(), rcu_exit_nohz(), rcu_nmi_enter(), and rcu_nmi_exit(), to be static inlines, as suggested by David Howells. Doing this saves about 100 bytes from rcutiny.o. (The numbers between v3 and this v4 of the patch are not directly comparable, since they are against different versions of Linux.)
Changes from v2 (http://lkml.org/lkml/2009/2/3/333) include:
o Fix whitespace issues.
o Change short-circuit "||" operator to instead be "+" in order
to fix performance bug noted by "kraai" on LWN.
(http://lwn.net/Articles/324348/)
Changes from v1 (http://lkml.org/lkml/2009/1/13/440) include:
o This version depends on EMBEDDED as well as !SMP, as suggested
by Ingo.
o Updated rcu_needs_cpu() to unconditionally return zero,
permitting the CPU to enter dynticks-idle mode at any time. This works because callbacks can be invoked upon entry to dynticks-idle mode.
o Paul is now OK with this being included, based on a poll at
the Kernel Miniconf at linux.conf.au, where about ten people said
that they cared about saving 900 bytes on single-CPU systems.
o Applies to both mainline and tip/core/rcu.
LKML-Reference: <12565226351355-git-send-email->
--- include/linux/hardirq.h | 24 ++++ include/linux/rcupdate.h | 6 + include/linux/rcutiny.h | 97 +++++++++++++++ init/Kconfig | 9 ++ kernel/Makefile | 1 + kernel/rcupdate.c | 4 + kernel/rcutiny.c | 294 ++++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 435 insertions(+), 0 deletions(-)
include/linux/hardirq.h +24 -0
@@ -139,10 +139,34 @@ static inline void account_system_vtime(struct task_struct *tsk)#endif #if defined(CONFIG_NO_HZ ) +#if defined(CONFIG_TINY_RCU ) +extern void rcu_enter_nohz (void ); +extern void rcu_exit_nohz (void ); + +static inline void rcu_irq_enter (void ) +{ +rcu_exit_nohz (); +} + +static inline void rcu_irq_exit (void ) +{ +rcu_enter_nohz (); +} + +static inline void rcu_nmi_enter (void ) +{ +} + +static inline void rcu_nmi_exit (void ) +{ +} + +#else extern void rcu_irq_enter (void );extern void rcu_irq_exit (void );extern void rcu_nmi_enter (void );extern void rcu_nmi_exit (void ); +#endif #else # define rcu_irq_enter () do { } while (0)# define rcu_irq_exit () do { } while (0)
include/linux/rcupdate.h +6 -0
@@ -68,11 +68,17 @@ extern int sched_expedited_torture_stats(char *page);/* Internal to kernel */ extern void rcu_init (void );extern void rcu_scheduler_starting (void ); +#ifndef CONFIG_TINY_RCU extern int rcu_needs_cpu (int cpu); +#else +static inline int rcu_needs_cpu (int cpu) {return 0 ; } +#endif extern int rcu_scheduler_active;#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_TREE_PREEMPT_RCU )#include <linux/rcutree.h> +#elif CONFIG_TINY_RCU +#include <linux/rcutiny.h> #else #error "Unknown RCU implementation specified to kernel configuration"#endif
include/linux/rcutiny.h new file +97 -0
97 lines not on this page · raw message
init/Kconfig +9 -0
@@ -334,6 +334,15 @@ config TREE_PREEMPT_RCU is also required. It also scales down nicely to smaller systems. +config TINY_RCU + bool "UP-only small-memory-footprint RCU" + depends on !SMP + help + This option selects the RCU implementation that is + designed for UP systems from which real-time response + is not required. This option greatly reduces the + memory footprint of RCU. + endchoice config RCU_TRACE
kernel/Makefile +1 -0
@@ -82,6 +82,7 @@ obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o obj-$(CONFIG_TREE_RCU) += rcutree.o obj-$(CONFIG_TREE_PREEMPT_RCU) += rcutree.o obj-$(CONFIG_TREE_RCU_TRACE) += rcutree_trace.o +obj-$(CONFIG_TINY_RCU) += rcutiny.o obj-$(CONFIG_RELAY) += relay.o obj-$(CONFIG_SYSCTL) += utsname_sysctl.o obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o
kernel/rcupdate.c +4 -0
@@ -67,6 +67,8 @@ void wakeme_after_rcu(struct rcu_head *head)complete (&rcu->completion); } +#ifndef CONFIG_TINY_RCU +#ifdef CONFIG_TREE_PREEMPT_RCU /**
@@ -157,6 +159,8 @@ void synchronize_rcu_bh(void) }EXPORT_SYMBOL_GPL (synchronize_rcu_bh); +#endif /* #ifndef CONFIG_TINY_RCU */ +static int __cpuinit rcu_barrier_cpu_hotplug (struct notifier_block *self,unsigned long action,void *hcpu) {
kernel/rcutiny.c new file +294 -0
294 lines not on this page · raw message
[tip:core/rcu] rcu: Add synchronize_srcu_expedited() to the rcutorture test suite
In reply to Paul E. McKenneyCommit-ID: 804bb8370522a569bd3a732b9de5fbd55e26f155 Gitweb: http://git.kernel.org/tip/804bb8370522a569bd3a732b9de5fbd55e26f155 Author: Paul E. McKenney <[email protected]> AuthorDate: Sun, 25 Oct 2009 19:03:52 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 26 Oct 2009 09:40:30 +0100
rcu: Add synchronize_srcu_expedited() to the rcutorture test suite
Adds the "srcu_expedited" torture type, and also renames sched_ops_sync to sched_sync_ops for consistency while we are in this file.
LKML-Reference: <12565226353636-git-send-email->
--- kernel/rcutorture.c | 25 ++++++++++++++++++++++--- 1 files changed, 22 insertions(+), 3 deletions(-)
kernel/rcutorture.c +22 -3
@@ -547,6 +547,25 @@ static struct rcu_torture_ops srcu_ops = { .name = "srcu" }; +static void srcu_torture_synchronize_expedited (void ) +{ +synchronize_srcu_expedited (&srcu_ctl); +} + +static struct rcu_torture_ops srcu_expedited_ops = { + .init = srcu_torture_init, + .cleanup = srcu_torture_cleanup, + .readlock = srcu_torture_read_lock, + .read_delay = srcu_read_delay, + .readunlock = srcu_torture_read_unlock, + .completed = srcu_torture_completed, + .deferred_free = rcu_sync_torture_deferred_free, + .sync = srcu_torture_synchronize_expedited, + .cb_barrier =NULL , + .stats = srcu_torture_stats, + .name ="srcu_expedited" +}; +/* * Definitions for sched torture testing. */
@@ -592,7 +611,7 @@ static struct rcu_torture_ops sched_ops = { .name = "sched" }; -static struct rcu_torture_ops sched_ops_sync = { +static struct rcu_torture_ops sched_sync_ops = { .init = rcu_sync_torture_init, .cleanup =NULL , .readlock = sched_torture_read_lock,
@@ -1098,8 +1117,8 @@ rcu_torture_init(void)int firsterr =0 ;static struct rcu_torture_ops *torture_ops[] = { &rcu_ops, &rcu_sync_ops, &rcu_bh_ops, &rcu_bh_sync_ops, - &sched_expedited_ops, - &srcu_ops, &sched_ops, &sched_ops_sync, }; + &srcu_ops, &srcu_expedited_ops, + &sched_ops, &sched_sync_ops, &sched_expedited_ops, };mutex_lock (&fullstop_mutex);
[tip:core/rcu] rcu: Add synchronize_srcu_expedited() to the documentation
In reply to Paul E. McKenneyCommit-ID: 64179861cb801eac4f00c79f39a29ea5ac9470d7 Gitweb: http://git.kernel.org/tip/64179861cb801eac4f00c79f39a29ea5ac9470d7 Author: Paul E. McKenney <[email protected]> AuthorDate: Sun, 25 Oct 2009 19:03:53 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 26 Oct 2009 09:40:31 +0100
rcu: Add synchronize_srcu_expedited() to the documentation
LKML-Reference: <12565226354176-git-send-email->
--- Documentation/RCU/whatisRCU.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
Documentation/RCU/whatisRCU.txt +1 -1
@@ -830,7 +830,7 @@ sched: Critical sections Grace period Barrier SRCU: Critical sections Grace period Barrier srcu_read_lock synchronize_srcu N/A - srcu_read_unlock + srcu_read_unlock synchronize_srcu_expedited SRCU: Initialization/cleanup init_srcu_struct
[tip:core/rcu] rcu: Do tiny cleanups in rcutiny
In reply to Paul E. McKenneyCommit-ID: 4ce5b90340879ce93d169b7b523c2cbbe7c45843 Gitweb: http://git.kernel.org/tip/4ce5b90340879ce93d169b7b523c2cbbe7c45843 Author: Ingo Molnar <[email protected]> AuthorDate: Mon, 26 Oct 2009 07:55:55 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 26 Oct 2009 09:40:40 +0100
rcu: Do tiny cleanups in rcutiny
No change in functionality - just straighten out a few small stylistic details.
Cc: Paul E. McKenney <[email protected]>
Cc: David Howells <[email protected]>
Cc: Josh Triplett <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
LKML-Reference: <12565226351355-git-send-email->
--- include/linux/rcutiny.h | 6 +--- kernel/rcutiny.c | 49 ++++++++++++++++++++++------------------------ 2 files changed, 25 insertions(+), 30 deletions(-)
include/linux/rcutiny.h +2 -4
@@ -20,9 +20,8 @@* Author: Paul E. McKenney <[email protected]> * * For detailed explanation of Read-Copy Update mechanism see - -* Documentation/RCU +* Documentation/RCU */ -#ifndef __LINUX_TINY_H#define __LINUX_TINY_H
@@ -70,8 +69,7 @@ static inline void synchronize_rcu_bh_expedited(void) }struct notifier_block ; -extern int rcu_cpu_notify (struct notifier_block *self, -unsigned long action,void *hcpu); +extern int rcu_cpu_notify (struct notifier_block *self,unsigned long action,void *hcpu);#ifdef CONFIG_NO_HZ
kernel/rcutiny.c +23 -26
111 lines not on this page · raw message
[tip:core/rcu] rcu: Improve rcutorture diagnostics when bad torture_type specified
In reply to Paul E. McKenneyCommit-ID: cf886c44ec418a01b2c52493465accb81acbf930 Gitweb: http://git.kernel.org/tip/cf886c44ec418a01b2c52493465accb81acbf930 Author: Paul E. McKenney <[email protected]> AuthorDate: Sun, 25 Oct 2009 19:03:54 -0700 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 26 Oct 2009 09:40:31 +0100
rcu: Improve rcutorture diagnostics when bad torture_type specified
Make rcutorture list the available torture_type values when it doesn't like the one specified.
LKML-Reference: <12565226351868-git-send-email->
--- kernel/rcutorture.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-)
kernel/rcutorture.c +5 -1
@@ -1129,8 +1129,12 @@ rcu_torture_init(void)break ; }if (i ==ARRAY_SIZE (torture_ops)) { -printk (KERN_ALERT "rcutorture: invalid torture type: \"%s\"\n" , +printk (KERN_ALERT "rcu-torture: invalid torture type: \"%s\"\n" , torture_type); +printk (KERN_ALERT "rcu-torture types:" ); +for (i =0 ; i <ARRAY_SIZE (torture_ops); i++) +printk (KERN_ALERT " %s" , torture_ops[i]->name); +printk (KERN_ALERT "\n" );mutex_unlock (&fullstop_mutex);return -EINVAL ; }
Paul E. McKenney
Reviewed-byRe: [tip:core/rcu] rcu: Do tiny cleanups in rcutiny
In reply to tip-bot for Ingo MolnarQuoted 11 lines · tip-bot for Ingo Molnar
Commit-ID: 4ce5b90340879ce93d169b7b523c2cbbe7c45843 Gitweb: http://git.kernel.org/tip/4ce5b90340879ce93d169b7b523c2cbbe7c45843 Author: Ingo Molnar <[email protected]> AuthorDate: Mon, 26 Oct 2009 07:55:55 +0100 Committer: Ingo Molnar <[email protected]> CommitDate: Mon, 26 Oct 2009 09:40:40 +0100rcu: Do tiny cleanups in rcutiny
No change in functionality - just straighten out a few small stylistic details.
These all look good to me.
Quoted 174 lines
Cc: Paul E. McKenney <[email protected]>
Cc: David Howells <[email protected]>
Cc: Josh Triplett <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
LKML-Reference: <12565226351355-git-send-email->Signed-off-by Ingo Molnar <[email protected]>--- include/linux/rcutiny.h | 6 +--- kernel/rcutiny.c | 49 ++++++++++++++++++++++------------------------ 2 files changed, 25 insertions(+), 30 deletions(-)include/linux/rcutiny.h +2
-4@@ -20,9 +20,8 @@* Author: Paul E. McKenney <[email protected]> * * For detailed explanation of Read-Copy Update mechanism see - -* Documentation/RCU +* Documentation/RCU */ -#ifndef __LINUX_TINY_H#define __LINUX_TINY_H@@ -70,8 +69,7 @@ static inline void synchronize_rcu_bh_expedited(void) }struct notifier_block ; -extern int rcu_cpu_notify (struct notifier_block *self, -unsigned long action,void *hcpu); +extern int rcu_cpu_notify (struct notifier_block *self,unsigned long action,void *hcpu);#ifdef CONFIG_NO_HZ kernel/rcutiny.c +23
-27@@ -20,22 +20,21 @@* Author: Paul E. McKenney <[email protected]> * * For detailed explanation of Read-Copy Update mechanism see - -* Documentation/RCU +* Documentation/RCU */ - -#include <linux/types.h> -#include <linux/kernel.h> -#include <linux/init.h> -#include <linux/rcupdate.h> -#include <linux/interrupt.h> -#include <linux/sched.h> -#include <linux/module.h> -#include <linux/completion.h> #include <linux/moduleparam.h> +#include <linux/completion.h> +#include <linux/interrupt.h> #include <linux/notifier.h> -#include <linux/cpu.h> +#include <linux/rcupdate.h> +#include <linux/kernel.h> +#include <linux/module.h> #include <linux/mutex.h> +#include <linux/sched.h> +#include <linux/types.h> +#include <linux/init.h> #include <linux/time.h> +#include <linux/cpu.h> /* Global control variables for rcupdate callback mechanism. */ struct rcu_ctrlblk {@@ -46,14 +45,13 @@ struct rcu_ctrlblk {/* Definition for rcupdate control block. */ static struct rcu_ctrlblk rcu_ctrlblk = { - .rcucblist =NULL , - .donetail = &rcu_ctrlblk.rcucblist, - .curtail = &rcu_ctrlblk.rcucblist, + .donetail = &rcu_ctrlblk.rcucblist, + .curtail = &rcu_ctrlblk.rcucblist, }; +static struct rcu_ctrlblk rcu_bh_ctrlblk = { - .rcucblist =NULL , - .donetail = &rcu_bh_ctrlblk.rcucblist, - .curtail = &rcu_bh_ctrlblk.rcucblist, + .donetail = &rcu_bh_ctrlblk.rcucblist, + .curtail = &rcu_bh_ctrlblk.rcucblist, };#ifdef CONFIG_NO_HZ @@ -84,8 +82,8 @@ void rcu_exit_nohz(void)/* * Helper function for rcu_qsctr_inc() and rcu_bh_qsctr_inc(). -* Also disable irqs to avoid confusion due to interrupt handlers invoking -* call_rcu(). +* Also disable irqs to avoid confusion due to interrupt handlers +* invoking call_rcu(). */ static int rcu_qsctr_help (struct rcu_ctrlblk *rcp) {@@ -99,6 +97,7 @@ static int rcu_qsctr_help(struct rcu_ctrlblk *rcp)return 1 ; }local_irq_restore (flags); +return 0 ; }@@ -143,8 +142,8 @@ void rcu_check_callbacks(int cpu, int user)*/ static void __rcu_process_callbacks (struct rcu_ctrlblk *rcp) { -unsigned long flags;struct rcu_head *next, *list; +unsigned long flags;/* If no RCU callbacks ready to invoke, just return. */ if (&rcp->rcucblist == rcp->donetail)@@ -182,8 +181,7 @@ static void rcu_process_callbacks(struct softirq_action *unused)* Null function to handle CPU being onlined. Longer term, we want to * make TINY_RCU avoid using rcupdate.c, but later... */ -int rcu_cpu_notify (struct notifier_block *self, -unsigned long action,void *hcpu) +int rcu_cpu_notify (struct notifier_block *self,unsigned long action,void *hcpu) {return NOTIFY_OK ; }@@ -223,6 +221,7 @@ static void __call_rcu(struct rcu_head *head, head->func = func; head->next =NULL ; +local_irq_save (flags); *rcp->curtail = head; rcp->curtail = &head->next;@@ -234,8 +233,7 @@ static void __call_rcu(struct rcu_head *head,* period. But since we have but one CPU, that would be after any * quiescent state. */ -void call_rcu (struct rcu_head *head, -void (*func)(struct rcu_head *rcu)) +void call_rcu (struct rcu_head *head,void (*func)(struct rcu_head *rcu)) {__call_rcu (head, func, &rcu_ctrlblk); }@@ -245,8 +243,7 @@ EXPORT_SYMBOL_GPL(call_rcu);* Post an RCU bottom-half callback to be invoked after any subsequent * quiescent state. */ -void call_rcu_bh (struct rcu_head *head, -void (*func)(struct rcu_head *rcu)) +void call_rcu_bh (struct rcu_head *head,void (*func)(struct rcu_head *rcu)) {__call_rcu (head, func, &rcu_bh_ctrlblk); } --To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH tip/core/rcu 6/6] rcu: Simplify creating of lockdep class for root rcu_node
In reply to Lai JiangshanQuoted 7 lines · Lai Jiangshan
Quoted 6 lines · Paul E. McKenney
kernel/rcutree.h +0
-0@@ -296,6 +296,8 @@ DECLARE_PER_CPU(struct rcu_data, rcu_preempt_data);
Quoted 6 lines
kernel/rcutree.h DECLARE_PER_CPU(struct rcu_data, rcu_preempt_data); · line 300
Quoted 4 lines
kernel/rcutree.h +2
-0#else /* #ifdef RCU_TREE_NONCORE */ +static struct lock_class_key rcu_root_class; +I think this line needs to be moved into
rcutree.c.
Good point, I will submit a cleanup patch.
Quoted 6 lines
kernel/rcutree.h DECLARE_PER_CPU(struct rcu_data, rcu_preempt_data); · line 303
Quoted 3 lines
kernel/rcutree.h +0
-0/* Forward declarations for rcutree_plugin.h */ static inline void rcu_bootup_announce (void );long rcu_batches_completed (void );For all 6 patches:
Reviewed-by Lai Jiangshan <[email protected]>
Thank you!!!
Thanx, Paul
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH tip/core/rcu 6/6] rcu: Simplify creating of lockdep class for root rcu_node
In reply to Ingo MolnarQuoted 16 lines · Ingo Molnar
Quoted 7 lines · Paul E. McKenney
From: Paul E. McKenney <[email protected]>
Use
lockdep_set_class()to simplify the code and to avoid any additional overhead in the !LOCKDEP case.Signed-off-by Peter Zijlstra <[email protected]>Signed-off-by Paul E. McKenney <[email protected]>This signoff order is wrong - first signoff line must be the author of the patch. (which is a pretty general rule with only rare exceptions)
This patch was from Peter originally, right? In such cases please add:
Quoted 8 lines
From: Peter Zijlstra <[email protected]>
To the head of the patch. Or, if Peter acked it, add:
Acked-by Peter Zijlstra <[email protected]>SOB lines are strictly only added by people who send patches to you - i.e. it records the patch handling pathway.
I will resubmit with this fixed up, apologies for my confusion.
Thanx, Paul