RCU: Preemptible-RCU
Subsystems (5)
cover RCU: Preemptible-RCU
Hello everyone,
This patchset is an updated version of the preemptible RCU patchset that Paul McKenney had posted it in September earlier this year that can be found here --> http://lkml.org/lkml/2007/9/10/213
This patchset incorporates the review comments from Oleg Nesterov and Steven Rostedt.
The testing report of the patchset is as follows:
====================================================================
Patch-stack: 2.6.23-rc3 + cpu-hotplug patches from
http://lkml.org/lkml/2007/11/15/239 + Preempt-RCU patches.
Test: RCU-Torture running parallelly with CPU-Hotplug
operations.
Duration: 24 hours.
Architectures: x86,x86_64, ppc64.
====================================================================
Currently it is based against the latest linux-2.6-sched-devel.git
Awaiting your feedback!
Thanks and Regards
gautham.
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
RFC PATCH 1/6 Preempt-RCU: Use softirq instead of tasklets for RCU
Preempt-RCU: Use softirq instead of tasklets for RCU
From: Dipankar Sarma <[email protected]>
This patch makes RCU use softirq instead of tasklets.
It also adds a memory barrier after raising the softirq inorder to ensure that the cpu sees the most recently updated value of rcu->cur while processing callbacks.
The discussion of the related theoretical race pointed out by James Huang can be found here --> http://lkml.org/lkml/2007/11/20/603
---
include/linux/interrupt.h | 1 + kernel/rcupdate.c | 25 +++++++++++++++++-------- 2 files changed, 18 insertions(+), 8 deletions(-)
include/linux/interrupt.h +1 -0
@@ -256,6 +256,7 @@ enum#ifdef CONFIG_HIGH_RES_TIMERS HRTIMER_SOFTIRQ , #endif + RCU_SOFTIRQ, /* Preferable RCU should always be the last softirq */ };/* softirq mask and active fields moved to irq_cpustat_t in
kernel/rcupdate.c +17 -8
@@ -73,8 +73,6 @@ static struct rcu_ctrlblk rcu_bh_ctrlblk = { DEFINE_PER_CPU(struct rcu_data, rcu_data) = { 0L }; DEFINE_PER_CPU(struct rcu_data, rcu_bh_data) = { 0L }; -/* Fake initialization required by compiler */ -static DEFINE_PER_CPU(struct tasklet_struct, rcu_tasklet) = {NULL};static int blimit =10 ;static int qhimark =10000 ;static int qlowmark =100 ;
@@ -231,6 +229,18 @@ void rcu_barrier(void) }EXPORT_SYMBOL_GPL (rcu_barrier); +/* Raises the softirq for processing rcu_callbacks. */ +static inline void raise_rcu_softirq (void ) +{ +raise_softirq (RCU_SOFTIRQ ); +/* +* The smp_mb() here is required to ensure that this cpu's +* __rcu_process_callbacks() reads the most recently updated +* value of rcu->cur. +*/ +smp_mb (); +} +/* * Invoke the completed RCU callbacks. They are expected to be in * a per-cpu list.
@@ -260,7 +270,7 @@ static void rcu_do_batch(struct rcu_data *rdp)if (!rdp->donelist) rdp->donetail = &rdp->donelist;else -tasklet_schedule (&per_cpu (rcu_tasklet, rdp->cpu)); +raise_rcu_softirq (); }/*
@@ -412,7 +422,6 @@ static void rcu_offline_cpu(int cpu) &per_cpu(rcu_bh_data, cpu));put_cpu_var (rcu_data); put_cpu_var(rcu_bh_data); - tasklet_kill_immediate(&per_cpu(rcu_tasklet, cpu), cpu); }#else
@@ -424,7 +433,7 @@ static void rcu_offline_cpu(int cpu)#endif /* -* This does the RCU processing work from tasklet context. +* This does the RCU processing work from softirq context. */ static void __rcu_process_callbacks (struct rcu_ctrlblk *rcp,struct rcu_data *rdp)
@@ -469,7 +478,7 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp,rcu_do_batch (rdp); } -static void rcu_process_callbacks (unsigned long unused) +static void rcu_process_callbacks (struct softirq_action *unused) {__rcu_process_callbacks (&rcu_ctrlblk, &__get_cpu_var (rcu_data));__rcu_process_callbacks (&rcu_bh_ctrlblk, &__get_cpu_var (rcu_bh_data));
@@ -533,7 +542,7 @@ void rcu_check_callbacks(int cpu, int user)rcu_bh_qsctr_inc (cpu); }else if (!in_softirq ())rcu_bh_qsctr_inc (cpu); -tasklet_schedule (&per_cpu (rcu_tasklet, cpu)); +raise_rcu_softirq (); } static void rcu_init_percpu_data(int cpu, struct rcu_ctrlblk *rcp,
@@ -556,7 +565,7 @@ static void __devinit rcu_online_cpu(int cpu)rcu_init_percpu_data (cpu, &rcu_ctrlblk, rdp);rcu_init_percpu_data (cpu, &rcu_bh_ctrlblk, bh_rdp); -tasklet_init (&per_cpu (rcu_tasklet, cpu), rcu_process_callbacks,0UL ); +open_softirq (RCU_SOFTIRQ , rcu_process_callbacks,NULL ); } static int __cpuinit rcu_cpu_notify(struct notifier_block *self,
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
RFC PATCH 2/6 Preempt-RCU: Reorganize RCU code into rcuclassic.c and rcupdate.c
In reply to Gautham R ShenoyPreempt-RCU: Reorganize RCU code into rcuclassic.c and rcupdate.c
From: Paul E. McKenney <[email protected]>
This patch re-organizes the RCU code to enable multiple implementations of RCU. Users of RCU continues to include rcupdate.h and the RCU interfaces remain the same. This is in preparation for subsequently merging the preemptible RCU implementation.
---
include/linux/rcuclassic.h | 161 ++++++++++++ include/linux/rcupdate.h | 168 ++++--------- kernel/Makefile | 2 kernel/rcuclassic.c | 576 ++++++++++++++++++++++++++++++++++++++++++++ kernel/rcupdate.c | 575 ++------------------------------------------ 5 files changed, 812 insertions(+), 670 deletions(-)
include/linux/rcuclassic.h new file +161 -0
@@ -0,0 +1,161 @@ +/* +* Read-Copy Update mechanism for mutual exclusion (classic version) +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +* +* Copyright IBM Corporation, 2001 +* +* Author: Dipankar Sarma <[email protected]> +* +* Based on the original work by Paul McKenney <[email protected]> +* and inputs from Rusty Russell, Andrea Arcangeli and Andi Kleen. +* Papers: +* http://www.rdrop.com/users/paulmck/paper/rclockpdcsproof.pdf +* http://lse.sourceforge.net/locking/rclock_OLS.2001.05.01c.sc.pdf (OLS2001) +* +* For detailed explanation of Read-Copy Update mechanism see - +* Documentation/RCU +* +*/ + +#ifndef __LINUX_RCUCLASSIC_H +#define __LINUX_RCUCLASSIC_H + +#ifdef __KERNEL__ + +#include <linux/cache.h> +#include <linux/spinlock.h> +#include <linux/threads.h> +#include <linux/percpu.h> +#include <linux/cpumask.h> +#include <linux/seqlock.h> + + +/* Global control variables for rcupdate callback mechanism. */ +struct rcu_ctrlblk { +long cur;/* Current batch number. */ +long completed;/* Number of the last completed batch */ +int next_pending;/* Is the next batch already waiting? */ + +int signaled; + +spinlock_t lock____cacheline_internodealigned_in_smp ; +cpumask_t cpumask;/* CPUs that need to switch in order */ +/* for current batch to proceed. */ +}____cacheline_internodealigned_in_smp ; + +/* Is batch a before batch b ? */ +static inline int rcu_batch_before (long a,long b) +{ +return (a - b) <0 ; +} + +/* Is batch a after batch b ? */ +static inline int rcu_batch_after (long a,long b) +{ +return (a - b) >0 ; +} + +/* +* Per-CPU data for Read-Copy UPdate. +* nxtlist - new callbacks are added here +* curlist - current batch for which quiescent cycle started if any +*/ +struct rcu_data { +/* 1) quiescent state handling : */ +long quiescbatch;/* Batch # for grace period */ +int passed_quiesc;/* User-mode/idle loop etc. */ +int qs_pending;/* core waits for quiesc state */ + +/* 2) batch handling */ +long batch;/* Batch # for current RCU batch */ +struct rcu_head *nxtlist; +struct rcu_head **nxttail; +long qlen;/* # of queued callbacks */ +struct rcu_head *curlist; +struct rcu_head **curtail; +struct rcu_head *donelist; +struct rcu_head **donetail; +long blimit;/* Upper limit on a processed batch */ +int cpu; +struct rcu_head barrier; +}; + +DECLARE_PER_CPU(struct rcu_data, rcu_data); +DECLARE_PER_CPU(struct rcu_data, rcu_bh_data); + +/* +* Increment the quiescent state counter. +* The counter is a bit degenerated: We do not need to know +* how many quiescent states passed, just if there was at least +* one since the start of the grace period. Thus just a flag. +*/ +static inline void rcu_qsctr_inc (int cpu) +{ +struct rcu_data *rdp = &per_cpu (rcu_data, cpu); + rdp->passed_quiesc =1 ; +} +static inline void rcu_bh_qsctr_inc (int cpu) +{ +struct rcu_data *rdp = &per_cpu (rcu_bh_data, cpu); + rdp->passed_quiesc =1 ; +} + +extern int rcu_pending (int cpu); +extern int rcu_needs_cpu (int cpu); + +#ifdef CONFIG_DEBUG_LOCK_ALLOC +extern struct lockdep_map rcu_lock_map; +# define rcu_read_acquire () \ + lock_acquire(&rcu_lock_map, 0, 0, 2, 1, _THIS_IP_) +# define rcu_read_release () lock_release(&rcu_lock_map, 1, _THIS_IP_) +#else +# define rcu_read_acquire () do { } while (0) +# define rcu_read_release () do { } while (0) +#endif + +#define __rcu_read_lock () \ + do { \ + preempt_disable(); \ + __acquire(RCU); \ + rcu_read_acquire(); \ + } while (0) +#define __rcu_read_unlock () \ + do { \ + rcu_read_release(); \ + __release(RCU); \ + preempt_enable(); \ + } while (0) +#define __rcu_read_lock_bh () \ + do { \ + local_bh_disable(); \ + __acquire(RCU_BH); \ + rcu_read_acquire(); \ + } while (0) +#define __rcu_read_unlock_bh () \ + do { \ + rcu_read_release(); \ + __release(RCU_BH); \ + local_bh_enable(); \ + } while (0) + +#define __synchronize_sched () synchronize_rcu() + +extern void __rcu_init (void ); +extern void rcu_check_callbacks (int cpu,int user); +extern void rcu_restart_cpu (int cpu); + +#endif /* __KERNEL__ */ +#endif /* __LINUX_RCUCLASSIC_H */
include/linux/rcupdate.h +49 -119
227 lines not on this page · raw message
kernel/Makefile +1 -1
@@ -6,7 +6,7 @@ obj-y = sched.o fork.o exec_domain.o panic.o printk.o profile.o \ exit.o itimer.o time.o softirq.o resource.o \ sysctl.o capability.o ptrace.o timer.o user.o user_namespace.o \ signal.o sys.o kmod.o workqueue.o pid.o \ - rcupdate.o extable.o params.o posix-timers.o \ + rcupdate.o rcuclassic.o extable.o params.o posix-timers.o \ kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o mutex.o \ hrtimer.o rwsem.o latency.o nsproxy.o srcu.o \ utsname.o notifier.o
kernel/rcuclassic.c new file +576 -0
576 lines not on this page · raw message
kernel/rcupdate.c +25 -550
630 lines not on this page · raw message
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
Re: [RFC PATCH 2/6] Preempt-RCU: Reorganize RCU code into rcuclassic.c and rcupdate.c
Hi,
Quoted 63 lines · Gautham R Shenoy
kernel/rcuclassic.c new file +58
-0+/** +* call_rcu - Queue an RCU callback for invocation after a grace period. +* @head: structure to be used for queueing the RCU updates. +* @func: actual update function to be invoked after the grace period +* +* The update function will be invoked some time after a full grace +* period elapses, in other words after all currently executing RCU +* read-side critical sections have completed. RCU read-side critical +* sections are delimited by rcu_read_lock() and rcu_read_unlock(), +* and may be nested. +*/ +void call_rcu (struct rcu_head *head, +void (*func)(struct rcu_head *rcu)) +{ +unsigned long flags; +struct rcu_data *rdp; + + head->func = func; + head->next =NULL ; +local_irq_save (flags); + rdp = &__get_cpu_var (rcu_data); + *rdp->nxttail = head; + rdp->nxttail = &head->next; +if (unlikely (++rdp->qlen > qhimark)) { + rdp->blimit =INT_MAX ; +force_quiescent_state (rdp, &rcu_ctrlblk); + } +local_irq_restore (flags); +} +EXPORT_SYMBOL_GPL (call_rcu); + +/** +* call_rcu_bh - Queue an RCU for invocation after a quicker grace period. +* @head: structure to be used for queueing the RCU updates. +* @func: actual update function to be invoked after the grace period +* +* The update function will be invoked some time after a full grace +* period elapses, in other words after all currently executing RCU +* read-side critical sections have completed. call_rcu_bh() assumes +* that the read-side critical sections end on completion of a softirq +* handler. This means that read-side critical sections in process +* context must not be interrupted by softirqs. This interface is to be +* used when most of the read-side critical sections are in softirq context. +* RCU read-side critical sections are delimited by rcu_read_lock() and +* rcu_read_unlock(), * if in interrupt context or rcu_read_lock_bh() +* and rcu_read_unlock_bh(), if in process context. These may be nested. +*/ +void call_rcu_bh (struct rcu_head *head, +void (*func)(struct rcu_head *rcu)) +{ +unsigned long flags; +struct rcu_data *rdp; + + head->func = func; + head->next =NULL ; +local_irq_save (flags); + rdp = &__get_cpu_var (rcu_bh_data); + *rdp->nxttail = head;
kernel/rcuclassic.c line 175
Quoted 10 lines
kernel/rcuclassic.c new file +10
-0+ rdp->nxttail = &head->next; + +if (unlikely (++rdp->qlen > qhimark)) { + rdp->blimit =INT_MAX ; +force_quiescent_state (rdp, &rcu_bh_ctrlblk); + } + +local_irq_restore (flags); +} +EXPORT_SYMBOL_GPL (call_rcu_bh);
Those two functions are identical beside the difference of rcu_data' <-> rcu_bh_data' and rcu_ctrlblk' <-> rcu_bh_ctrlblk'.
Is there a way to collapse the code into one helper function or would that effect performance too much?
Hannes
Re: [RFC PATCH 2/6] Preempt-RCU: Reorganize RCU code into rcuclassic.c and rcupdate.c
Quoted 81 lines · Johannes Weiner
Hi,
Quoted 63 lines · Gautham R Shenoy
kernel/rcuclassic.c new file +58
-0+/** +* call_rcu - Queue an RCU callback for invocation after a grace period. +* @head: structure to be used for queueing the RCU updates. +* @func: actual update function to be invoked after the grace period +* +* The update function will be invoked some time after a full grace +* period elapses, in other words after all currently executing RCU +* read-side critical sections have completed. RCU read-side critical +* sections are delimited by rcu_read_lock() and rcu_read_unlock(), +* and may be nested. +*/ +void call_rcu (struct rcu_head *head, +void (*func)(struct rcu_head *rcu)) +{ +unsigned long flags; +struct rcu_data *rdp; + + head->func = func; + head->next =NULL ; +local_irq_save (flags); + rdp = &__get_cpu_var (rcu_data); + *rdp->nxttail = head; + rdp->nxttail = &head->next; +if (unlikely (++rdp->qlen > qhimark)) { + rdp->blimit =INT_MAX ; +force_quiescent_state (rdp, &rcu_ctrlblk); + } +local_irq_restore (flags); +} +EXPORT_SYMBOL_GPL (call_rcu); + +/** +* call_rcu_bh - Queue an RCU for invocation after a quicker grace period. +* @head: structure to be used for queueing the RCU updates. +* @func: actual update function to be invoked after the grace period +* +* The update function will be invoked some time after a full grace +* period elapses, in other words after all currently executing RCU +* read-side critical sections have completed. call_rcu_bh() assumes +* that the read-side critical sections end on completion of a softirq +* handler. This means that read-side critical sections in process +* context must not be interrupted by softirqs. This interface is to be +* used when most of the read-side critical sections are in softirq context. +* RCU read-side critical sections are delimited by rcu_read_lock() and +* rcu_read_unlock(), * if in interrupt context or rcu_read_lock_bh() +* and rcu_read_unlock_bh(), if in process context. These may be nested. +*/ +void call_rcu_bh (struct rcu_head *head, +void (*func)(struct rcu_head *rcu)) +{ +unsigned long flags; +struct rcu_data *rdp; + + head->func = func; + head->next =NULL ; +local_irq_save (flags); + rdp = &__get_cpu_var (rcu_bh_data); + *rdp->nxttail = head;kernel/rcuclassic.c line 175
Quoted 10 lines
kernel/rcuclassic.c new file +10
-0+ rdp->nxttail = &head->next; + +if (unlikely (++rdp->qlen > qhimark)) { + rdp->blimit =INT_MAX ; +force_quiescent_state (rdp, &rcu_bh_ctrlblk); + } + +local_irq_restore (flags); +} +EXPORT_SYMBOL_GPL (call_rcu_bh);Those two functions are identical beside the difference of
rcu_data' <->rcu_bh_data' andrcu_ctrlblk' <->rcu_bh_ctrlblk'.
Quoted 2 lines
Is there a way to collapse the code into one helper function or would that effect performance too much?
This code is performance critical, but on the other hand, perhaps gcc has gotten smarter in the years since this was written. (This patch is just moving this code from one file to another, rather than writing it from scratch.) Would current gcc implementations be able to do the inlining required?
Thanx, Paul
RFC PATCH 3/6 Preempt-RCU: Fix rcu_barrier for preemptive environment
In reply to Gautham R ShenoyPreempt-RCU: Fix rcu_barrier for preemptive environment.
From: Paul E. McKenney <[email protected]>
Fix rcu_barrier() to work properly in preemptive kernel environment. Also, the ordering of callback must be preserved while moving callbacks to another CPU during CPU hotplug.
---
kernel/rcuclassic.c | 2 +- kernel/rcupdate.c | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletions(-)
kernel/rcuclassic.c +1 -1
@@ -371,9 +371,9 @@ static void __rcu_offline_cpu(struct rcu_data *this_rdp,if (rcp->cur != rcp->completed)cpu_quiet (rdp->cpu, rcp);spin_unlock_bh (&rcp->lock); +rcu_move_batch (this_rdp, rdp->donelist, rdp->donetail);rcu_move_batch (this_rdp, rdp->curlist, rdp->curtail);rcu_move_batch (this_rdp, rdp->nxtlist, rdp->nxttail); -rcu_move_batch (this_rdp, rdp->donelist, rdp->donetail); }static void rcu_offline_cpu (int cpu)
kernel/rcupdate.c +10 -0
@@ -115,7 +115,17 @@ void rcu_barrier(void)mutex_lock (&rcu_barrier_mutex);init_completion (&rcu_barrier_completion);atomic_set (&rcu_barrier_cpu_count,0 ); +/* +* The queueing of callbacks in all CPUs must be atomic with +* respect to RCU, otherwise one CPU may queue a callback, +* wait for a grace period, decrement barrier count and call +* complete(), while other CPUs have not yet queued anything. +* So, we need to make sure that grace periods cannot complete +* until all the callbacks are queued. +*/ +rcu_read_lock ();on_each_cpu (rcu_barrier_func,NULL ,0 ,1 ); +rcu_read_unlock ();wait_for_completion (&rcu_barrier_completion);mutex_unlock (&rcu_barrier_mutex); }
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
RFC PATCH 4/6 Preempt-RCU: Implementation
In reply to Gautham R ShenoyPreempt-RCU: Implementation
From: Paul E. McKenney <[email protected]>
This patch implements a new version of RCU which allows its read-side critical sections to be preempted. It uses a set of counter pairs to keep track of the read-side critical sections and flips them when all tasks exit read-side critical section. The details of this implementation can be found in this paper -
http://www.rdrop.com/users/paulmck/RCU/OLSrtRCU.2006.08.11a.pdf
and the article-
http://lwn.net/Articles/253651/
This patch was developed as a part of the -rt kernel development and meant to provide better latencies when read-side critical sections of RCU don't disable preemption. As a consequence of keeping track of RCU readers, the readers have a slight overhead (optimizations in the paper). This implementation co-exists with the "classic" RCU implementations and can be switched to at compiler.
Also includes RCU tracing summarized in debugfs.
---
include/linux/rcuclassic.h | 3 include/linux/rcupdate.h | 11 - include/linux/rcupreempt.h | 86 ++++ include/linux/rcupreempt_trace.h | 99 +++++ include/linux/sched.h | 5 kernel/Kconfig.preempt | 38 ++ kernel/Makefile | 7 kernel/fork.c | 4 kernel/rcuclassic.c | 1 kernel/rcupreempt.c | 816 ++++++++++++++++++++++++++++++++++++++ kernel/rcupreempt_trace.c | 330 +++++++++++++++ 11 files changed, 1394 insertions(+), 6 deletions(-)
include/linux/rcuclassic.h +3 -0
@@ -157,5 +157,8 @@ extern void __rcu_init(void);extern void rcu_check_callbacks (int cpu,int user);extern void rcu_restart_cpu (int cpu); +extern long rcu_batches_completed (void ); +extern long rcu_batches_completed_bh (void ); +#endif /* __KERNEL__ */ #endif /* __LINUX_RCUCLASSIC_H */
include/linux/rcupdate.h +7 -4
@@ -53,7 +53,11 @@ struct rcu_head {void (*func)(struct rcu_head *head); }; +#ifdef CONFIG_CLASSIC_RCU #include <linux/rcuclassic.h> +#else /* #ifdef CONFIG_CLASSIC_RCU */ +#include <linux/rcupreempt.h> +#endif /* #else #ifdef CONFIG_CLASSIC_RCU */ #define RCU_HEAD_INIT { .next = NULL, .func = NULL }#define RCU_HEAD (head) struct rcu_head head = RCU_HEAD_INIT
@@ -231,13 +235,12 @@ extern void call_rcu_bh(struct rcu_head *head,/* Exported common interfaces */ extern void synchronize_rcu (void );extern void rcu_barrier (void ); +extern long rcu_batches_completed (void ); +extern long rcu_batches_completed_bh (void );/* Internal to kernel */ extern void rcu_init (void ); -extern void rcu_check_callbacks (int cpu,int user); - -extern long rcu_batches_completed (void ); -extern long rcu_batches_completed_bh (void ); +extern int rcu_needs_cpu (int cpu);#endif /* __KERNEL__ */ #endif /* __LINUX_RCUPDATE_H */
include/linux/rcupreempt.h new file +86 -0
86 lines not on this page · raw message
include/linux/rcupreempt_trace.h new file +99 -0
99 lines not on this page · raw message
include/linux/sched.h +5 -0
@@ -979,6 +979,11 @@ struct task_struct {int nr_cpus_allowed;unsigned int time_slice; +#ifdef CONFIG_PREEMPT_RCU +int rcu_read_lock_nesting; +int rcu_flipctr_idx; +#endif /* #ifdef CONFIG_PREEMPT_RCU */ +#if defined(CONFIG_SCHEDSTATS ) || defined(CONFIG_TASK_DELAY_ACCT )struct sched_info sched_info;#endif
kernel/Kconfig.preempt +38 -0
@@ -63,3 +63,41 @@ config PREEMPT_BKL Say Y here if you are building a kernel for a desktop system. Say N if you are unsure. +choice + prompt "RCU implementation type:" + default CLASSIC_RCU + +config CLASSIC_RCU + bool "Classic RCU" + help + This option selects the classic RCU implementation that is + designed for best read-side performance on non-realtime + systems. + + Say Y if you are unsure. + +config PREEMPT_RCU + bool "Preemptible RCU" + depends on PREEMPT + help + This option reduces the latency of the kernel by making certain + RCU sections preemptible. Normally RCU code is non-preemptible, if + this option is selected then read-only RCU sections become + preemptible. This helps latency, but may expose bugs due to + now-naive assumptions about each RCU read-side critical section + remaining on a given CPU through its execution. + + Say N if you are unsure. + +endchoice + +config RCU_TRACE + bool "Enable tracing for RCU - currently stats in debugfs" + select DEBUG_FS + default y + help + This option provides tracing in RCU which presents stats + in debugfs for debugging RCU implementation. + + Say Y here if you want to enable RCU tracing + Say N if you are unsure.
kernel/Makefile +6 -1
@@ -6,7 +6,7 @@ obj-y = sched.o fork.o exec_domain.o panic.o printk.o profile.o \ exit.o itimer.o time.o softirq.o resource.o \ sysctl.o capability.o ptrace.o timer.o user.o user_namespace.o \ signal.o sys.o kmod.o workqueue.o pid.o \ - rcupdate.o rcuclassic.o extable.o params.o posix-timers.o \ + rcupdate.o extable.o params.o posix-timers.o \ kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o mutex.o \ hrtimer.o rwsem.o latency.o nsproxy.o srcu.o \ utsname.o notifier.o
@@ -52,6 +52,11 @@ obj-$(CONFIG_DETECT_SOFTLOCKUP) += softlockup.o obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ obj-$(CONFIG_SECCOMP) += seccomp.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o +obj-$(CONFIG_CLASSIC_RCU) += rcuclassic.o +obj-$(CONFIG_PREEMPT_RCU) += rcupreempt.o +ifeq ($(CONFIG_PREEMPT_RCU),y) +obj-$(CONFIG_RCU_TRACE) += rcupreempt_trace.o +endif obj-$(CONFIG_RELAY) += relay.o obj-$(CONFIG_SYSCTL) += utsname_sysctl.o obj-$(CONFIG_TASK_DELAY_ACCT) += delayacct.o
kernel/fork.c +4 -0
@@ -1045,6 +1045,10 @@ static struct task_struct *copy_process(unsigned long clone_flags,copy_flags (clone_flags, p);INIT_LIST_HEAD (&p->children);INIT_LIST_HEAD (&p->sibling); +#ifdef CONFIG_PREEMPT_RCU + p->rcu_read_lock_nesting =0 ; + p->rcu_flipctr_idx =0 ; +#endif /* #ifdef CONFIG_PREEMPT_RCU */ p->vfork_done =NULL ;spin_lock_init (&p->alloc_lock);
kernel/rcuclassic.c +0 -1
@@ -45,7 +45,6 @@#include <linux/moduleparam.h> #include <linux/percpu.h> #include <linux/notifier.h> -/* #include <linux/rcupdate.h> @@@ */ #include <linux/cpu.h> #include <linux/mutex.h>
kernel/rcupreempt.c new file +816 -0
816 lines not on this page · raw message
kernel/rcupreempt_trace.c new file +29 -0
@@ -0,0 +1,330 @@ +/* +* Read-Copy Update tracing for realtime implementation +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +* GNU General Public License for more details. +* +* You should have received a copy of the GNU General Public License +* along with this program; if not, write to the Free Software +* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +* +* Copyright IBM Corporation, 2006 +* +* Papers: http://www.rdrop.com/users/paulmck/RCU +* +* For detailed explanation of Read-Copy Update mechanism see - +* Documentation/RCU/ *.txt +* +*/ +#include <linux/types.h> +#include <linux/kernel.h> +#include <linux/init.h> +#include <linux/spinlock.h>
+#include <linux/smp.h>
+#include <linux/rcupdate.h>
+#include <linux/interrupt.h>
+#include <linux/sched.h>
+#include <asm/atomic.h>
+#include <linux/bitops.h>
+#include <linux/module.h>
+#include <linux/completion.h>
+#include <linux/moduleparam.h>
+#include <linux/percpu.h>
+#include <linux/notifier.h>
+#include <linux/rcupdate.h>
+#include <linux/cpu.h>
+#include <linux/mutex.h>
+#include <linux/rcupreempt_trace.h>
+#include <linux/debugfs.h>
+
+static struct mutex rcupreempt_trace_mutex;
+static char *rcupreempt_trace_buf;
+#define RCUPREEMPT_TRACE_BUF_SIZE 4096
+
+void rcupreempt_trace_move2done(struct rcupreempt_trace *trace) +{
+ trace->done_length += trace->wait_length;
+ trace->done_add += trace->wait_length;
+ trace->wait_length = 0;
+}
+void rcupreempt_trace_move2wait(struct rcupreempt_trace *trace) +{
+ trace->wait_length += trace->next_length;
+ trace->wait_add += trace->next_length;
+ trace->next_length = 0;
+}
+void rcupreempt_trace_try_flip_1(struct rcupreempt_trace *trace) +{
+ atomic_inc(&trace->rcu_try_flip_1);
+}
+void rcupreempt_trace_try_flip_e1(struct rcupreempt_trace *trace) +{
+ atomic_inc(&trace->rcu_try_flip_e1);
+}
+void rcupreempt_trace_try_flip_i1(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_i1++;
+}
+void rcupreempt_trace_try_flip_ie1(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_ie1++;
+}
+void rcupreempt_trace_try_flip_g1(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_g1++;
+}
+void rcupreempt_trace_try_flip_a1(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_a1++;
+}
+void rcupreempt_trace_try_flip_ae1(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_ae1++;
+}
+void rcupreempt_trace_try_flip_a2(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_a2++;
+}
+void rcupreempt_trace_try_flip_z1(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_z1++;
+}
+void rcupreempt_trace_try_flip_ze1(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_ze1++;
+}
+void rcupreempt_trace_try_flip_z2(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_z2++;
+}
+void rcupreempt_trace_try_flip_m1(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_m1++;
+}
+void rcupreempt_trace_try_flip_me1(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_me1++;
+}
+void rcupreempt_trace_try_flip_m2(struct rcupreempt_trace *trace) +{
+ trace->rcu_try_flip_m2++;
+}
+void rcupreempt_trace_check_callbacks(struct rcupreempt_trace *trace) +{
+ trace->rcu_check_callbacks++;
+}
+void rcupreempt_trace_done_remove(struct rcupreempt_trace *trace) +{
+ trace->done_remove += trace->done_length;
+ trace->done_length = 0;
+}
+void rcupreempt_trace_invoke(struct rcupreempt_trace *trace) +{
+ atomic_inc(&trace->done_invoked);
+}
+void rcupreempt_trace_next_add(struct rcupreempt_trace *trace) +{
+ trace->next_add++;
+ trace->next_length++;
+}
+
+static void rcupreempt_trace_sum(struct rcupreempt_trace *sp) +{
+ struct rcupreempt_trace *cp;
+ int cpu;
+
+ memset(sp, 0, sizeof(*sp));
+ for_each_possible_cpu(cpu) {
+ cp = rcupreempt_trace_cpu(cpu);
+ sp->next_length += cp->next_length;
+ sp->next_add += cp->next_add;
+ sp->wait_length += cp->wait_length;
+ sp->wait_add += cp->wait_add;
+ sp->done_length += cp->done_length;
+ sp->done_add += cp->done_add;
+ sp->done_remove += cp->done_remove;
+ atomic_set(&sp->done_invoked, atomic_read(&cp->done_invoked));
+ sp->rcu_check_callbacks += cp->rcu_check_callbacks;
+ atomic_set(&sp->rcu_try_flip_1,
+ atomic_read(&cp->rcu_try_flip_1));
+ atomic_set(&sp->rcu_try_flip_e1,
+ atomic_read(&cp->rcu_try_flip_e1));
+ sp->rcu_try_flip_i1 += cp->rcu_try_flip_i1;
+ sp->rcu_try_flip_ie1 += cp->rcu_try_flip_ie1;
+ sp->rcu_try_flip_g1 += cp->rcu_try_flip_g1;
+ sp->rcu_try_flip_a1 += cp->rcu_try_flip_a1;
+ sp->rcu_try_flip_ae1 += cp->rcu_try_flip_ae1;
+ sp->rcu_try_flip_a2 += cp->rcu_try_flip_a2;
+ sp->rcu_try_flip_z1 += cp->rcu_try_flip_z1;
+ sp->rcu_try_flip_ze1 += cp->rcu_try_flip_ze1;
+ sp->rcu_try_flip_z2 += cp->rcu_try_flip_z2;
+ sp->rcu_try_flip_m1 += cp->rcu_try_flip_m1;
+ sp->rcu_try_flip_me1 += cp->rcu_try_flip_me1;
+ sp->rcu_try_flip_m2 += cp->rcu_try_flip_m2;
+ }
+}
+
+static ssize_t rcustats_read(struct file *filp, char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ struct rcupreempt_trace trace;
+ ssize_t bcount;
+ int cnt = 0;
+
+ rcupreempt_trace_sum(&trace);
+ mutex_lock(&rcupreempt_trace_mutex);
+ snprintf(&rcupreempt_trace_buf[cnt], RCUPREEMPT_TRACE_BUF_SIZE - cnt,
+ "ggp=%ld rcc=%ld\n",
+ rcu_batches_completed(),
+ trace.rcu_check_callbacks);
+ snprintf(&rcupreempt_trace_buf[cnt], RCUPREEMPT_TRACE_BUF_SIZE - cnt,
+ "na=%ld nl=%ld wa=%ld wl=%ld da=%ld dl=%ld dr=%ld di=%d\n"
+ "1=%d e1=%d i1=%ld ie1=%ld g1=%ld a1=%ld ae1=%ld a2=%ld\n"
+ "z1=%ld ze1=%ld z2=%ld m1=%ld me1=%ld m2=%ld\n",
+
+ trace.next_add, trace.next_length,
+ trace.wait_add, trace.wait_length,
+ trace.done_add, trace.done_length,
+ trace.done_remove, atomic_read(&trace.done_invoked),
+ atomic_read(&trace.rcu_try_flip_1),
+ atomic_read(&trace.rcu_try_flip_e1),
+ trace.rcu_try_flip_i1, trace.rcu_try_flip_ie1,
+ trace.rcu_try_flip_g1,
+ trace.rcu_try_flip_a1, trace.rcu_try_flip_ae1,
+ trace.rcu_try_flip_a2,
+ trace.rcu_try_flip_z1, trace.rcu_try_flip_ze1,
+ trace.rcu_try_flip_z2,
+ trace.rcu_try_flip_m1, trace.rcu_try_flip_me1,
+ trace.rcu_try_flip_m2);
+ bcount = simple_read_from_buffer(buffer, count, ppos,
+ rcupreempt_trace_buf, strlen(rcupreempt_trace_buf));
+ mutex_unlock(&rcupreempt_trace_mutex);
+ return bcount;
+}
+
+static ssize_t rcugp_read(struct file *filp, char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ long oldgp = rcu_batches_completed();
+ ssize_t bcount;
+
+ mutex_lock(&rcupreempt_trace_mutex);
+ synchronize_rcu();
+ snprintf(rcupreempt_trace_buf, RCUPREEMPT_TRACE_BUF_SIZE,
+ "oldggp=%ld newggp=%ld\n", oldgp, rcu_batches_completed());
+ bcount = simple_read_from_buffer(buffer, count, ppos,
+ rcupreempt_trace_buf, strlen(rcupreempt_trace_buf));
+ mutex_unlock(&rcupreempt_trace_mutex);
+ return bcount;
+}
+
+static ssize_t rcuctrs_read(struct file *filp, char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ int cnt = 0;
+ int cpu;
+ int f = rcu_batches_completed() & 0x1;
+ ssize_t bcount;
+
+ mutex_lock(&rcupreempt_trace_mutex);
+
+ cnt += snprintf(&rcupreempt_trace_buf[cnt], RCUPREEMPT_TRACE_BUF_SIZE,
+ "CPU last cur F M\n");
+ for_each_online_cpu(cpu) {
+ long *flipctr = rcupreempt_flipctr(cpu);
+ cnt += snprintf(&rcupreempt_trace_buf[cnt],
+ RCUPREEMPT_TRACE_BUF_SIZE - cnt,
+ "%3d %4ld %3ld %d %d\n",
+ cpu, + flipctr[!f], + flipctr[f], + rcupreempt_flip_flag(cpu), + rcupreempt_mb_flag(cpu));
+ }
+ cnt += snprintf(&rcupreempt_trace_buf[cnt],
+ RCUPREEMPT_TRACE_BUF_SIZE - cnt,
+ "ggp = %ld, state = %s\n",
+ rcu_batches_completed(),
+ rcupreempt_try_flip_state_name());
+ cnt += snprintf(&rcupreempt_trace_buf[cnt],
+ RCUPREEMPT_TRACE_BUF_SIZE - cnt,
+ "\n");
+ bcount = simple_read_from_buffer(buffer, count, ppos,
+ rcupreempt_trace_buf, strlen(rcupreempt_trace_buf));
+ mutex_unlock(&rcupreempt_trace_mutex);
+ return bcount;
+}
+
+static struct file_operations rcustats_fops = {
+ .owner = THIS_MODULE,
+ .read = rcustats_read,
+};
+
+static struct file_operations rcugp_fops = {
+ .owner = THIS_MODULE,
+ .read = rcugp_read,
+};
+
+static struct file_operations rcuctrs_fops = {
+ .owner = THIS_MODULE,
+ .read = rcuctrs_read,
+};
+
+static struct dentry *rcudir, *statdir, *ctrsdir, *gpdir; +static int rcupreempt_debugfs_init(void)
+{
+ rcudir = debugfs_create_dir("rcu", NULL);
+ if (!rcudir)
+ goto out;
+ statdir = debugfs_create_file("rcustats", 0444, rcudir,
+ NULL, &rcustats_fops);
+ if (!statdir)
+ goto free_out;
+
+ gpdir = debugfs_create_file("rcugp", 0444, rcudir, NULL, &rcugp_fops);
+ if (!gpdir)
+ goto free_out;
+
+ ctrsdir = debugfs_create_file("rcuctrs", 0444, rcudir,
+ NULL, &rcuctrs_fops);
+ if (!ctrsdir)
+ goto free_out;
+ return 0;
+free_out:
+ if (statdir)
+ debugfs_remove(statdir);
+ if (gpdir)
+ debugfs_remove(gpdir);
+ debugfs_remove(rcudir);
+out:
+ return 1;
+}
+
+static int __init rcupreempt_trace_init(void) +{
+ mutex_init(&rcupreempt_trace_mutex);
+ rcupreempt_trace_buf = kmalloc(RCUPREEMPT_TRACE_BUF_SIZE, GFP_KERNEL);
+ if (!rcupreempt_trace_buf)
+ return 1;
+ return rcupreempt_debugfs_init();
+}
+
+static void __exit rcupreempt_trace_cleanup(void) +{
+ debugfs_remove(statdir);
+ debugfs_remove(gpdir);
+ debugfs_remove(ctrsdir);
+ debugfs_remove(rcudir);
+ kfree(rcupreempt_trace_buf);
+}
+
+
+module_init(rcupreempt_trace_init); +module_exit(rcupreempt_trace_cleanup);
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
Re: [RFC PATCH 4/6] Preempt-RCU: Implementation
Hi,
Quoted 25 lines · Gautham R Shenoy
kernel/Kconfig.preempt +17
-0@@ -63,3 +63,41 @@ config PREEMPT_BKL Say Y here if you are building a kernel for a desktop system. Say N if you are unsure. +choice + prompt "RCU implementation type:" + default CLASSIC_RCU + +config CLASSIC_RCU + bool "Classic RCU" + help + This option selects the classic RCU implementation that is + designed for best read-side performance on non-realtime + systems. + + Say Y if you are unsure. + +config PREEMPT_RCU + bool "Preemptible RCU" + depends on PREEMPT + help
kernel/Kconfig.preempt config PREEMPT_BKL · line 92
Quoted 10 lines
kernel/Kconfig.preempt +10
-0+ This option reduces the latency of the kernel by making certain + RCU sections preemptible. Normally RCU code is non-preemptible, if + this option is selected then read-only RCU sections become + preemptible. This helps latency, but may expose bugs due to + now-naive assumptions about each RCU read-side critical section + remaining on a given CPU through its execution. + + Say N if you are unsure. + +endchoice
Why got this moved into init/Kconfig?
Now it's somewhere in the root menu, not really belonging to anything. Also why is this a choice? Are more RCU types planned?
bye, Roman
Re: [RFC PATCH 4/6] Preempt-RCU: Implementation
Quoted 33 lines · Roman Zippel
Hi,
Quoted 29 lines · Gautham R Shenoy
kernel/Kconfig.preempt +21
-0@@ -63,3 +63,41 @@ config PREEMPT_BKL Say Y here if you are building a kernel for a desktop system. Say N if you are unsure. +choice + prompt "RCU implementation type:" + default CLASSIC_RCU + +config CLASSIC_RCU + bool "Classic RCU" + help + This option selects the classic RCU implementation that is + designed for best read-side performance on non-realtime + systems. + + Say Y if you are unsure. + +config PREEMPT_RCU + bool "Preemptible RCU" + depends on PREEMPT + help + This option reduces the latency of the kernel by making certain + RCU sections preemptible. Normally RCU code is non-preemptible, if + this option is selected then read-only RCU sections become + preemptible. This helps latency, but may expose bugs due to
Quoted 8 lines
kernel/Kconfig.preempt config PREEMPT_BKL · line 92
Quoted 6 lines
kernel/Kconfig.preempt +6
-0+ now-naive assumptions about each RCU read-side critical section + remaining on a given CPU through its execution. + + Say N if you are unsure. + +endchoiceWhy got this moved into init/Kconfig?
Because there are some arches that don't have kernel/Kconfig.preempt, its earlier home. Therefore, putting it into kernel/Kconfig.preempt broke those arches' builds by supplying neither PREEMPT_RCU nor CLASSIC_RCU.
Quoted 1 line
Now it's somewhere in the root menu, not really belonging to anything.
Do you have a suggested location?
Quoted 1 line
Also why is this a choice? Are more RCU types planned?
I don't expect additional drop-in replacements for RCU, though people are certainly free to experiment if they wish. It is a choice because this gives people a very clear idea of the two options and because it makes the implementation a bit cleaner.
Thanx, Paul
Re: [RFC PATCH 4/6] Preempt-RCU: Implementation
Hi
Quoted 9 lines · Paul E. McKenney
Quoted 1 line
Why got this moved into init/Kconfig?
Because there are some arches that don't have
kernel/Kconfig.preempt, its earlier home. Therefore, putting it intokernel/Kconfig.preemptbroke those arches' builds by supplying neitherPREEMPT_RCUnorCLASSIC_RCU.Quoted 1 line
Now it's somewhere in the root menu, not really belonging to anything.
Quoted 8 lines
Do you have a suggested location?
Quoted 1 line
Also why is this a choice? Are more RCU types planned?
I don't expect additional drop-in replacements for RCU, though people are certainly free to experiment if they wish. It is a choice because this gives people a very clear idea of the two options and because it makes the implementation a bit cleaner.
I'd suggest to move PREEMPT_RCU back to Kconfig.preempt and if you really need the second symbol leave this behind (maybe with a comment):
config CLASSIC_RCU
def_bool !PREEMPT_RCU
Once there are more options, we can still look for a better place...
Also could you please add a proper dependency to RCU_TRACE on PREEMPT_RCU, so that this condition isn't needed anymore:
ifeq ($(CONFIG_PREEMPT_RCU),y)
obj-$(CONFIG_RCU_TRACE) += rcupreempt_trace.o
endif
Thanks.
bye, Roman
Steven Rostedt
reviewerRe: [RFC PATCH 4/6] Preempt-RCU: Implementation
Quoted 6 lines · Roman Zippel
I'd suggest to move
PREEMPT_RCUback toKconfig.preemptand if you really need the second symbol leave this behind (maybe with a comment):config
CLASSIC_RCU
def_bool!PREEMPT_RCU
Ah, OK, you are saying that we should only show the option for PREEMPT_RCU? If selected, we use that and if not, we default to CLASSIC_RCU.
I guess that can work.
-- Steve
Quoted 2 lines
Once there are more options, we can still look for a better place...
Re: [RFC PATCH 4/6] Preempt-RCU: Implementation
In reply to Roman ZippelQuoted 28 lines · Roman Zippel
Hi
Quoted 9 lines · Paul E. McKenney
Quoted 1 line
Why got this moved into init/Kconfig?
Because there are some arches that don't have
kernel/Kconfig.preempt, its earlier home. Therefore, putting it intokernel/Kconfig.preemptbroke those arches' builds by supplying neitherPREEMPT_RCUnorCLASSIC_RCU.Quoted 1 line
Now it's somewhere in the root menu, not really belonging to anything.
Quoted 8 lines
Do you have a suggested location?
Quoted 1 line
Also why is this a choice? Are more RCU types planned?
I don't expect additional drop-in replacements for RCU, though people are certainly free to experiment if they wish. It is a choice because this gives people a very clear idea of the two options and because it makes the implementation a bit cleaner.
I'd suggest to move
PREEMPT_RCUback toKconfig.preemptand if you really need the second symbol leave this behind (maybe with a comment):config
CLASSIC_RCU
def_bool!PREEMPT_RCU
Quoted 8 lines
Once there are more options, we can still look for a better place...
Also could you please add a proper dependency to
RCU_TRACEonPREEMPT_RCU, so that this condition isn't needed anymore:ifeq ($(CONFIG_PREEMPT_RCU),y)
obj-$(CONFIG_RCU_TRACE) += rcupreempt_trace.o
endif
Is this what you had in mind? I don't have any way to test on a system not supporting CONFIG_PREEMPT, but seems to work on x86.
---
init/Kconfig | 34 +++------------------------------- kernel/Kconfig.preempt | 15 +++++++++++++++ 2 files changed, 18 insertions(+), 31 deletions(-)
init/Kconfig +3 -31
@@ -860,38 +860,10 @@ source "block/Kconfig" config PREEMPT_NOTIFIERS bool -choice - prompt "RCU implementation type:" - default CLASSIC_RCU - help - This allows you to choose either the classic RCU implementation - that is designed for best read-side performance on non-realtime - systems, or the preemptible RCU implementation for best latency - on realtime systems. Note that some kernel preemption modes - will restrict your choice. - - Select the default if you are unsure. - config CLASSIC_RCU - bool "Classic RCU" + def_bool !PREEMPT_RCU help This option selects the classic RCU implementation that is designed for best read-side performance on non-realtime - systems. - - Say Y if you are unsure. - -config PREEMPT_RCU - bool "Preemptible RCU" - depends on PREEMPT - help - This option reduces the latency of the kernel by making certain - RCU sections preemptible. Normally RCU code is non-preemptible, if - this option is selected then read-only RCU sections become - preemptible. This helps latency, but may expose bugs due to - now-naive assumptions about each RCU read-side critical section - remaining on a given CPU through its execution. - - Say N if you are unsure. - -endchoice + systems. Classic RCU is the default. Note that the + PREEMPT_RCU symbol is used to select/deselect this option.
kernel/Kconfig.preempt +15 -0
@@ -52,8 +52,23 @@ config PREEMPT endchoice +config PREEMPT_RCU + bool "Preemptible RCU" + depends on PREEMPT + default n + help + This option reduces the latency of the kernel by making certain + RCU sections preemptible. Normally RCU code is non-preemptible, if + this option is selected then read-only RCU sections become + preemptible. This helps latency, but may expose bugs due to + now-naive assumptions about each RCU read-side critical section + remaining on a given CPU through its execution. + + Say N if you are unsure. + config RCU_TRACE bool "Enable tracing for RCU - currently stats in debugfs" + depends on PREEMPT_RCU select DEBUG_FS default y help
Steven Rostedt
reviewerRe: [RFC PATCH 4/6] Preempt-RCU: Implementation
Quoted 26 lines · Paul E. McKenney
Is this what you had in mind? I don't have any way to test on a system not supporting
CONFIG_PREEMPT, but seems to work on x86.Signed-off-by Paul E. McKenney <[email protected]>---
init/Kconfig | 34 +++------------------------------- kernel/Kconfig.preempt | 15 +++++++++++++++ 2 files changed, 18 insertions(+), 31 deletions(-)diff -urpNa -X dontdiff linux-2.6.25-rc3/init/Kconfig linux-2.6.25-rc3-preempt_rcu/init/Kconfig
init/Kconfig +0
-9@@ -860,38 +860,10 @@ source "block/Kconfig" config PREEMPT_NOTIFIERS bool -choice - prompt "RCU implementation type:" - default CLASSIC_RCU - help - This allows you to choose either the classic RCU implementation - that is designed for best read-side performance on non-realtime - systems, or the preemptible RCU implementation for best latency - on realtime systems. Note that some kernel preemption modes - will restrict your choice.
init/Kconfig source "block/Kconfig" · line 880
Quoted 10 lines
init/Kconfig +1
-5- - Select the default if you are unsure. - config CLASSIC_RCU - bool "Classic RCU" + def_bool !PREEMPT_RCU help This option selects the classic RCU implementation that is designed for best read-side performance on non-realtime - systems.
Actually Paul, you don't need to make this an "option". Simply the default if PREEMPT_RCU is not selected. In otherwords, no prompt.
config CLASSIC_RCU
bool
depends on !PREEMPT_RCU
default y
Then, have the PREEMPT_RCU defined down below (as you already did), and have it be the only option. That is, you can have normal RCU or select to have PREEMPT_RCU.
If the user selects PREEMPT_RCU, CLASSIC_RCU will be unselected.
-- Steve
Quoted 47 lines
init/Kconfig +2
-17- - Say Y if you are unsure. - -config PREEMPT_RCU - bool "Preemptible RCU" - depends on PREEMPT - help - This option reduces the latency of the kernel by making certain - RCU sections preemptible. Normally RCU code is non-preemptible, if - this option is selected then read-only RCU sections become - preemptible. This helps latency, but may expose bugs due to - now-naive assumptions about each RCU read-side critical section - remaining on a given CPU through its execution. - - Say N if you are unsure. - -endchoice + systems. Classic RCU is the default. Note that the + PREEMPT_RCU symbol is used to select/deselect this option.diff -urpNa -X dontdiff linux-2.6.25-rc3/kernel/Kconfig.preempt linux-2.6.25-rc3-preempt_rcu/kernel/Kconfig.preempt
kernel/Kconfig.preempt +15
-0@@ -52,8 +52,23 @@ config PREEMPT endchoice +config PREEMPT_RCU + bool "Preemptible RCU" + depends on PREEMPT + default n + help + This option reduces the latency of the kernel by making certain + RCU sections preemptible. Normally RCU code is non-preemptible, if + this option is selected then read-only RCU sections become + preemptible. This helps latency, but may expose bugs due to + now-naive assumptions about each RCU read-side critical section + remaining on a given CPU through its execution. + + Say N if you are unsure. + config RCU_TRACE bool "Enable tracing for RCU - currently stats in debugfs" + depends on PREEMPT_RCU select DEBUG_FS default y help
Re: [RFC PATCH 4/6] Preempt-RCU: Implementation
Hi,
Quoted 11 lines · Steven Rostedt
init/Kconfig source "block/Kconfig" · line 880
Quoted 7 lines
config
CLASSIC_RCUinit/Kconfig +1
-2- bool "Classic RCU" + def_bool !PREEMPT_RCU help This option selects the classic RCU implementation that is designed for best read-side performance on non-realtime - systems.Actually Paul, you don't need to make this an "option". Simply the default if
PREEMPT_RCUis not selected. In otherwords, no prompt.
Quoted 4 lines
config
CLASSIC_RCU
bool
depends on !PREEMPT_RCU
default y
There is no prompt. :)
"def_bool !PREEMPT_RCU" does the same as this.
bye, Roman
Re: [RFC PATCH 4/6] Preempt-RCU: Implementation
In reply to Paul E. McKenneyHi,
Quoted 2 lines · Paul E. McKenney
Is this what you had in mind? I don't have any way to test on a system not supporting
CONFIG_PREEMPT, but seems to work on x86.
Yes, looks fine.
kernel/Kconfig.preempt config PREEMPT · line 58
Quoted 4 lines
kernel/Kconfig.preempt +4
-0+config PREEMPT_RCU + bool "Preemptible RCU" + depends on PREEMPT + default n
"default n" isn't really necessary, it's already the default.
bye, Roman
Re: [RFC PATCH 4/6] Preempt-RCU: Implementation
Quoted 7 lines · Roman Zippel
Hi,
Quoted 2 lines · Paul E. McKenney
Is this what you had in mind? I don't have any way to test on a system not supporting
CONFIG_PREEMPT, but seems to work on x86.
Quoted 8 lines
Yes, looks fine.
kernel/Kconfig.preempt config PREEMPT · line 58
Quoted 4 lines
kernel/Kconfig.preempt +4
-0+config PREEMPT_RCU + bool "Preemptible RCU" + depends on PREEMPT + default n"default n" isn't really necessary, it's already the default.
Fair enough. But something like 125 Kconfig files in 2.6.25-rc3 have at least one "default n" in them, so is it worth getting rid of it? Seems to me that the explicit "default n" has some substantial readability advantages.
Thanx, Paul
PATCH 1/1 move PREEMPT_RCU config option back under PREEMPT
Hello!
The original preemptible-RCU patch put the choice between classic and preemptible RCU into kernel/Kconfig.preempt, which resulted in build failures on machines not supporting CONFIG_PREEMPT. This choice was therefore moved to init/Kconfig, which worked, but placed the choice between classic and preemptible RCU at the top level, a very obtuse choice indeed. This patch changes from the Kconfig "choice" mechanism to a pair of booleans, only one of which (CONFIG_PREEMPT_RCU) is user-visible, and is located in kernel/Kconfig.preempt, where one would expect it to be. The other (CONFIG_CLASSIC_RCU) is in init/Kconfig so that it is available to all architectures, hopefully avoiding build breakage. Thanks to Roman Zippel for suggesting this approach.
I have tested this, but sadly do not have access to a machine that does not support CONFIG_PREEMPT. However, I did edit my config in an attempt to simulate this situation.
---
init/Kconfig | 34 +++------------------------------- kernel/Kconfig.preempt | 15 +++++++++++++++ 2 files changed, 18 insertions(+), 31 deletions(-)
init/Kconfig +3 -31
@@ -860,38 +860,10 @@ source "block/Kconfig" config PREEMPT_NOTIFIERS bool -choice - prompt "RCU implementation type:" - default CLASSIC_RCU - help - This allows you to choose either the classic RCU implementation - that is designed for best read-side performance on non-realtime - systems, or the preemptible RCU implementation for best latency - on realtime systems. Note that some kernel preemption modes - will restrict your choice. - - Select the default if you are unsure. - config CLASSIC_RCU - bool "Classic RCU" + def_bool !PREEMPT_RCU help This option selects the classic RCU implementation that is designed for best read-side performance on non-realtime - systems. - - Say Y if you are unsure. - -config PREEMPT_RCU - bool "Preemptible RCU" - depends on PREEMPT - help - This option reduces the latency of the kernel by making certain - RCU sections preemptible. Normally RCU code is non-preemptible, if - this option is selected then read-only RCU sections become - preemptible. This helps latency, but may expose bugs due to - now-naive assumptions about each RCU read-side critical section - remaining on a given CPU through its execution. - - Say N if you are unsure. - -endchoice + systems. Classic RCU is the default. Note that the + PREEMPT_RCU symbol is used to select/deselect this option.
kernel/Kconfig.preempt +15 -0
@@ -52,8 +52,23 @@ config PREEMPT endchoice +config PREEMPT_RCU + bool "Preemptible RCU" + depends on PREEMPT + default n + help + This option reduces the latency of the kernel by making certain + RCU sections preemptible. Normally RCU code is non-preemptible, if + this option is selected then read-only RCU sections become + preemptible. This helps latency, but may expose bugs due to + now-naive assumptions about each RCU read-side critical section + remaining on a given CPU through its execution. + + Say N if you are unsure. + config RCU_TRACE bool "Enable tracing for RCU - currently stats in debugfs" + depends on PREEMPT_RCU select DEBUG_FS default y help
Re: [PATCH] move PREEMPT_RCU config option back under PREEMPT
Quoted 13 lines · Paul E. McKenney
The original preemptible-RCU patch put the choice between classic and preemptible RCU into
kernel/Kconfig.preempt, which resulted in build failures on machines not supportingCONFIG_PREEMPT. This choice was therefore moved to init/Kconfig, which worked, but placed the choice between classic and preemptible RCU at the top level, a very obtuse choice indeed. This patch changes from theKconfig"choice" mechanism to a pair of booleans, only one of which (CONFIG_PREEMPT_RCU) is user-visible, and is located inkernel/Kconfig.preempt, where one would expect it to be. The other (CONFIG_CLASSIC_RCU) is in init/Kconfig so that it is available to all architectures, hopefully avoiding build breakage. Thanks to Roman Zippel for suggesting this approach.
Quoted 3 lines
I have tested this, but sadly do not have access to a machine that does not support
CONFIG_PREEMPT. However, I did edit my config in an attempt to simulate this situation.
One minor comment, but otherwise:
Quoted 5 lines
Signed-off-by Paul E. McKenney <[email protected]>config
CLASSIC_RCU
init/Kconfig source "block/Kconfig" · line 883
Quoted 9 lines
init/Kconfig +1
-5- bool "Classic RCU" + def_bool !PREEMPT_RCU help This option selects the classic RCU implementation that is designed for best read-side performance on non-realtime - systems. - - Say Y if you are unsure. -
You can get rid of the "help" part since it isn't visible to users.
-- Steve
Re: [PATCH] move PREEMPT_RCU config option back under PREEMPT
Quoted 27 lines · Steven Rostedt
Quoted 13 lines · Paul E. McKenney
The original preemptible-RCU patch put the choice between classic and preemptible RCU into
kernel/Kconfig.preempt, which resulted in build failures on machines not supportingCONFIG_PREEMPT. This choice was therefore moved to init/Kconfig, which worked, but placed the choice between classic and preemptible RCU at the top level, a very obtuse choice indeed. This patch changes from theKconfig"choice" mechanism to a pair of booleans, only one of which (CONFIG_PREEMPT_RCU) is user-visible, and is located inkernel/Kconfig.preempt, where one would expect it to be. The other (CONFIG_CLASSIC_RCU) is in init/Kconfig so that it is available to all architectures, hopefully avoiding build breakage. Thanks to Roman Zippel for suggesting this approach.Quoted 3 lines
I have tested this, but sadly do not have access to a machine that does not support
CONFIG_PREEMPT. However, I did edit my config in an attempt to simulate this situation.One minor comment, but otherwise:
Acked-by Steven Rostedt <[email protected]>Quoted 2 lines
Signed-off-by Paul E. McKenney <[email protected]>
Quoted 12 lines
init/Kconfig source "block/Kconfig" · line 883
Quoted 10 lines
config
CLASSIC_RCUinit/Kconfig +1
-5- bool "Classic RCU" + def_bool !PREEMPT_RCU help This option selects the classic RCU implementation that is designed for best read-side performance on non-realtime - systems. - - Say Y if you are unsure. -You can get rid of the "help" part since it isn't visible to users.
So how about if I replace it with comment lines (starting with "#", not with "comment")?
Thanx, Paul
Re: [PATCH] move PREEMPT_RCU config option back under PREEMPT
Hi,
Quoted 12 lines · Paul E. McKenney
Quoted 11 lines
init/Kconfig source "block/Kconfig" · line 883
Quoted 9 lines
init/Kconfig +1
-5- bool "Classic RCU" + def_bool !PREEMPT_RCU help This option selects the classic RCU implementation that is designed for best read-side performance on non-realtime - systems. - - Say Y if you are unsure. -You can get rid of the "help" part since it isn't visible to users.
Quoted 2 lines
So how about if I replace it with comment lines (starting with "#", not with "comment")?
Actually xconfig can display this text, so using the help like this is fine.
bye, Roman
Re: [PATCH] move PREEMPT_RCU config option back under PREEMPT
Quoted 16 lines · Roman Zippel
Hi,
Quoted 12 lines · Paul E. McKenney
Quoted 11 lines
init/Kconfig source "block/Kconfig" · line 883
Quoted 9 lines
init/Kconfig +1
-5- bool "Classic RCU" + def_bool !PREEMPT_RCU help This option selects the classic RCU implementation that is designed for best read-side performance on non-realtime - systems. - - Say Y if you are unsure. -You can get rid of the "help" part since it isn't visible to users.
Quoted 5 lines
Quoted 2 lines
So how about if I replace it with comment lines (starting with "#", not with "comment")?
Actually xconfig can display this text, so using the help like this is fine.
Cool, I will stick with the "help" clause, then.
Thanx, Paul
Re: [RFC PATCH 4/6] Preempt-RCU: Implementation
In reply to Paul E. McKenneyHi,
Quoted 6 lines · Paul E. McKenney
Quoted 1 line
"default n" isn't really necessary, it's already the default.
Fair enough. But something like 125
Kconfigfiles in 2.6.25-rc3 have at least one "default n" in them, so is it worth getting rid of it? Seems to me that the explicit "default n" has some substantial readability advantages.
The inverse would mean all the other configs have a readability disadvantage.
In most cases they can be simply removed, only in form of 'def_bool n' it makes somewhat sense.
bye, Roman
Steven Rostedt
reviewerRe: [RFC PATCH 4/6] Preempt-RCU: Implementation
In reply to Roman ZippelQuoted 2 lines · Roman Zippel
Also why is this a choice? Are more RCU types planned?
Why shouldn't it be a choice? You can have CLASSIC_RCU or PREEMPT_RCU, one or the other and not both. Sounds perfect for being a choice.
-- Steve
Re: [RFC PATCH 4/6] Preempt-RCU: Implementation
Hi,
Quoted 4 lines · Steven Rostedt
Quoted 1 line
Also why is this a choice? Are more RCU types planned?
Why shouldn't it be a choice? You can have
CLASSIC_RCUorPREEMPT_RCU, one or the other and not both. Sounds perfect for being a choice.
With this logic almost every bool could be implemented via choice. A simple bool perfectly gives you two choices too.
bye, Roman
RFC PATCH 5/6 Preempt-RCU: CPU Hotplug handling
In reply to Gautham R ShenoyPreempt-RCU: CPU Hotplug handling
From: Paul E. McKenney <[email protected]>
This patch allows preemptible RCU to tolerate CPU-hotplug operations. It accomplishes this by maintaining a local copy of a map of online CPUs, which it accesses under its own lock.
---
kernel/rcupreempt.c | 147 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 142 insertions(+), 5 deletions(-)
kernel/rcupreempt.c +142 -5
@@ -147,6 +147,8 @@ static char *rcu_try_flip_state_names[] = { "idle", "waitack", "waitzero", "waitmb" };#endif /* #ifdef CONFIG_RCU_TRACE */ +static cpumask_t rcu_cpu_online_map__read_mostly =CPU_MASK_NONE ; +/* * Enum and per-CPU flag to determine when each CPU has seen * the most recent counter flip.
@@ -445,7 +447,7 @@ rcu_try_flip_idle(void)/* Now ask each CPU for acknowledgement of the flip. */ -for_each_possible_cpu (cpu) +for_each_cpu_mask (cpu, rcu_cpu_online_map)per_cpu (rcu_flip_flag, cpu) = rcu_flipped;return 1 ;
@@ -461,7 +463,7 @@ rcu_try_flip_waitack(void)int cpu;RCU_TRACE_ME (rcupreempt_trace_try_flip_a1); -for_each_possible_cpu (cpu) +for_each_cpu_mask (cpu, rcu_cpu_online_map)if (per_cpu (rcu_flip_flag, cpu) != rcu_flip_seen) {RCU_TRACE_ME (rcupreempt_trace_try_flip_ae1);return 0 ;
@@ -492,7 +494,7 @@ rcu_try_flip_waitzero(void)/* Check to see if the sum of the "last" counters is zero. */ RCU_TRACE_ME (rcupreempt_trace_try_flip_z1); -for_each_possible_cpu (cpu) +for_each_cpu_mask (cpu, rcu_cpu_online_map) sum +=RCU_DATA_CPU (cpu)->rcu_flipctr[lastidx];if (sum !=0 ) {RCU_TRACE_ME (rcupreempt_trace_try_flip_ze1);
@@ -507,7 +509,7 @@ rcu_try_flip_waitzero(void)smp_mb ();/* ^^^^^^^^^^^^ */ /* Call for a memory barrier from each CPU. */ -for_each_possible_cpu (cpu) +for_each_cpu_mask (cpu, rcu_cpu_online_map)per_cpu (rcu_mb_flag, cpu) = rcu_mb_needed;RCU_TRACE_ME (rcupreempt_trace_try_flip_z2);
@@ -525,7 +527,7 @@ rcu_try_flip_waitmb(void)int cpu;RCU_TRACE_ME (rcupreempt_trace_try_flip_m1); -for_each_possible_cpu (cpu) +for_each_cpu_mask (cpu, rcu_cpu_online_map)if (per_cpu (rcu_mb_flag, cpu) != rcu_mb_done) {RCU_TRACE_ME (rcupreempt_trace_try_flip_me1);return 0 ;
@@ -637,6 +639,98 @@ void rcu_advance_callbacks(int cpu, int user)spin_unlock_irqrestore (&rdp->lock, flags); } +#ifdef CONFIG_HOTPLUG_CPU +#define rcu_offline_cpu_enqueue (srclist, srctail, dstlist, dsttail) do { \ + *dsttail = srclist; \ + if (srclist != NULL) { \ + dsttail = srctail; \ + srclist = NULL; \ + srctail = &srclist;\ + } \ + } while (0) + +void rcu_offline_cpu (int cpu) +{ +int i; +struct rcu_head *list =NULL ; +unsigned long flags; +struct rcu_data *rdp =RCU_DATA_CPU (cpu); +struct rcu_head **tail = &list; + +/* +* Remove all callbacks from the newly dead CPU, retaining order. +* Otherwise rcu_barrier() will fail +*/ + +spin_lock_irqsave (&rdp->lock, flags); +rcu_offline_cpu_enqueue (rdp->donelist, rdp->donetail, list, tail); +for (i =GP_STAGES -1 ; i >=0 ; i--) +rcu_offline_cpu_enqueue (rdp->waitlist[i], rdp->waittail[i], + list, tail); +rcu_offline_cpu_enqueue (rdp->nextlist, rdp->nexttail, list, tail); +spin_unlock_irqrestore (&rdp->lock, flags); + rdp->waitlistcount =0 ; + +/* Disengage the newly dead CPU from the grace-period computation. */ + +spin_lock_irqsave (&rcu_ctrlblk.fliplock, flags); +rcu_check_mb (cpu); +if (per_cpu (rcu_flip_flag, cpu) == rcu_flipped) { +smp_mb ();/* Subsequent counter accesses must see new value */ +per_cpu (rcu_flip_flag, cpu) = rcu_flip_seen; +smp_mb ();/* Subsequent RCU read-side critical sections */ +/* seen -after- acknowledgement. */ + } + +RCU_DATA_ME ()->rcu_flipctr[0 ] +=RCU_DATA_CPU (cpu)->rcu_flipctr[0 ]; +RCU_DATA_ME ()->rcu_flipctr[1 ] +=RCU_DATA_CPU (cpu)->rcu_flipctr[1 ]; + +RCU_DATA_CPU (cpu)->rcu_flipctr[0 ] =0 ; +RCU_DATA_CPU (cpu)->rcu_flipctr[1 ] =0 ; + +cpu_clear (cpu, rcu_cpu_online_map); + +spin_unlock_irqrestore (&rcu_ctrlblk.fliplock, flags); + +/* +* Place the removed callbacks on the current CPU's queue. +* Make them all start a new grace period: simple approach, +* in theory could starve a given set of callbacks, but +* you would need to be doing some serious CPU hotplugging +* to make this happen. If this becomes a problem, adding +* a synchronize_rcu() to the hotplug path would be a simple +* fix. +*/ + + rdp =RCU_DATA_ME (); +spin_lock_irqsave (&rdp->lock, flags); + *rdp->nexttail = list; +if (list) + rdp->nexttail = tail; +spin_unlock_irqrestore (&rdp->lock, flags); +} + +void __devinit rcu_online_cpu (int cpu) +{ +unsigned long flags; + +spin_lock_irqsave (&rcu_ctrlblk.fliplock, flags); +cpu_set (cpu, rcu_cpu_online_map); +spin_unlock_irqrestore (&rcu_ctrlblk.fliplock, flags); +} + +#else /* #ifdef CONFIG_HOTPLUG_CPU */ + +void rcu_offline_cpu (int cpu) +{ +} + +void __devinit rcu_online_cpu (int cpu) +{ +} + +#endif /* #else #ifdef CONFIG_HOTPLUG_CPU */ +static void rcu_process_callbacks (struct softirq_action *unused) {unsigned long flags;
@@ -746,6 +840,32 @@ int rcu_pending(int cpu)return 0 ; } +static int __cpuinit rcu_cpu_notify (struct notifier_block *self, +unsigned long action,void *hcpu) +{ +long cpu = (long )hcpu; + +switch (action) { +case CPU_UP_PREPARE : +case CPU_UP_PREPARE_FROZEN : +rcu_online_cpu (cpu); +break ; +case CPU_UP_CANCELED : +case CPU_UP_CANCELED_FROZEN : +case CPU_DEAD : +case CPU_DEAD_FROZEN : +rcu_offline_cpu (cpu); +break ; +default : +break ; + } +return NOTIFY_OK ; +} + +static struct notifier_block __cpuinitdata rcu_nb = { + .notifier_call = rcu_cpu_notify, +}; +void __init __rcu_init (void ) {int cpu;
@@ -769,6 +889,23 @@ void __init __rcu_init(void) rdp->rcu_flipctr[0 ] =0 ; rdp->rcu_flipctr[1 ] =0 ; } +register_cpu_notifier (&rcu_nb); + +/* +* We don't need protection against CPU-Hotplug here +* since +* a) If a CPU comes online while we are iterating over the +* cpu_online_map below, we would only end up making a +* duplicate call to rcu_online_cpu() which sets the corresponding +* CPU's mask in the rcu_cpu_online_map. +* +* b) A CPU cannot go offline at this point in time since the user +* does not have access to the sysfs interface, nor do we +* suspend the system. +*/ +for_each_online_cpu (cpu) +rcu_cpu_notify (&rcu_nb,CPU_UP_PREPARE , (void *)(long ) cpu); +open_softirq (RCU_SOFTIRQ , rcu_process_callbacks,NULL ); }
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
RFC PATCH 6/6 Preempt-RCU: Update RCU Documentation.
In reply to Gautham R ShenoyPreempt-RCU: Update RCU Documentation.
From: Paul E. McKenney <[email protected]>
This patch updates the RCU documentation to reflect preemptible RCU as well as recent publications.
---
Documentation/RCU/RTFP.txt | 210 +++++++++++++++++++++++++++++++++++++++-- Documentation/RCU/rcu.txt | 19 +++- Documentation/RCU/torture.txt | 11 +- 3 files changed, 221 insertions(+), 19 deletions(-)
Documentation/RCU/RTFP.txt +199 -11
@@ -9,8 +9,8 @@ The first thing resembling RCU was published in 1980, when Kung and Lehman [Kung80] recommended use of a garbage collector to defer destruction of nodes in a parallel binary search tree in order to simplify its implementation. This works well in environments that have garbage -collectors, but current production garbage collectors incur significant -read-side overhead. +collectors, but most production garbage collectors incur significant +overhead. In 1982, Manber and Ladner [Manber82,Manber84] recommended deferring destruction until all threads running at that time have terminated, again
@@ -99,16 +99,25 @@ locking, reduces contention, reduces memory latency for readers, and parallelizes pipeline stalls and memory latency for writers. However, these techniques still impose significant read-side overhead in the form of memory barriers. Researchers at Sun worked along similar lines -in the same timeframe [HerlihyLM02,HerlihyLMS03]. These techniques -can be thought of as inside-out reference counts, where the count is -represented by the number of hazard pointers referencing a given data -structure (rather than the more conventional counter field within the -data structure itself). +in the same timeframe [HerlihyLM02]. These techniques can be thought +of as inside-out reference counts, where the count is represented by the +number of hazard pointers referencing a given data structure (rather than +the more conventional counter field within the data structure itself). + +By the same token, RCU can be thought of as a "bulk reference count", +where some form of reference counter covers all reference by a given CPU +or thread during a set timeframe. This timeframe is related to, but +not necessarily exactly the same as, an RCU grace period. In classic +RCU, the reference counter is the per-CPU bit in the "bitmask" field, +and each such bit covers all references that might have been made by +the corresponding CPU during the prior grace period. Of course, RCU +can be thought of in other terms as well. In 2003, the K42 group described how RCU could be used to create -hot-pluggable implementations of operating-system functions. Later that -year saw a paper describing an RCU implementation of System V IPC -[Arcangeli03], and an introduction to RCU in Linux Journal [McKenney03a]. +hot-pluggable implementations of operating-system functions [Appavoo03a]. +Later that year saw a paper describing an RCU implementation of System +V IPC [Arcangeli03], and an introduction to RCU in Linux Journal +[McKenney03a]. 2004 has seen a Linux-Journal article on use of RCU in dcache [McKenney04a], a performance comparison of locking to RCU on several
@@ -117,10 +126,19 @@ number of operating-system kernels [PaulEdwardMcKenneyPhD], a paper describing how to make RCU safe for soft-realtime applications [Sarma04c], and a paper describing SELinux performance with RCU [JamesMorris04b]. -2005 has seen further adaptation of RCU to realtime use, permitting +2005 brought further adaptation of RCU to realtime use, permitting preemption of RCU realtime critical sections [PaulMcKenney05a, PaulMcKenney05b]. +2006 saw the first best-paper award for an RCU paper [ThomasEHart2006a], +as well as further work on efficient implementations of preemptible +RCU [PaulEMcKenney2006b], but priority-boosting of RCU read-side critical +sections proved elusive. An RCU implementation permitting general +blocking in read-side critical sections appeared [PaulEMcKenney2006c], +Robert Olsson described an RCU-protected trie-hash combination +[RobertOlsson2006a]. + + Bibtex Entries @article{Kung80
@@ -203,6 +221,41 @@ Bibtex Entries ,Address="New Orleans, LA" } +@conference{Pu95a, +Author = "Calton Pu and Tito Autrey and Andrew Black and Charles Consel and +Crispin Cowan and Jon Inouye and Lakshmi Kethana and Jonathan Walpole and +Ke Zhang", +Title = "Optimistic Incremental Specialization: Streamlining a Commercial +Operating System", +Booktitle = "15\textsuperscript{th} ACM Symposium on +Operating Systems Principles (SOSP'95)", +address = "Copper Mountain, CO", +month="December", +year="1995", +pages="314-321", +annotation=" + Uses a replugger, but with a flag to signal when people are + using the resource at hand. Only one reader at a time. +" +} + +@conference{Cowan96a, +Author = "Crispin Cowan and Tito Autrey and Charles Krasic and +Calton Pu and Jonathan Walpole", +Title = "Fast Concurrent Dynamic Linking for an Adaptive Operating System", +Booktitle = "International Conference on Configurable Distributed Systems +(ICCDS'96)", +address = "Annapolis, MD", +month="May", +year="1996", +pages="108", +isbn="0-8186-7395-8", +annotation=" + Uses a replugger, but with a counter to signal when people are + using the resource at hand. Allows multiple readers. +" +} + @techreport{Slingwine95 ,author="John D. Slingwine and Paul E. McKenney" ,title="Apparatus and Method for Achieving Reduced Overhead Mutual
@@ -312,6 +365,49 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell" [Viewed June 23, 2004]" } +@conference{Michael02a +,author="Maged M. Michael" +,title="Safe Memory Reclamation for Dynamic Lock-Free Objects Using Atomic +Reads and Writes" +,Year="2002" +,Month="August" +,booktitle="{Proceedings of the 21\textsuperscript{st} Annual ACM +Symposium on Principles of Distributed Computing}" +,pages="21-30" +,annotation=" + Each thread keeps an array of pointers to items that it is + currently referencing. Sort of an inside-out garbage collection + mechanism, but one that requires the accessing code to explicitly + state its needs. Also requires read-side memory barriers on + most architectures. +" +} + +@conference{Michael02b +,author="Maged M. Michael" +,title="High Performance Dynamic Lock-Free Hash Tables and List-Based Sets" +,Year="2002" +,Month="August" +,booktitle="{Proceedings of the 14\textsuperscript{th} Annual ACM +Symposium on Parallel +Algorithms and Architecture}" +,pages="73-82" +,annotation=" + Like the title says... +" +} + +@InProceedings{HerlihyLM02 +,author={Maurice Herlihy and Victor Luchangco and Mark Moir} +,title="The Repeat Offender Problem: A Mechanism for Supporting Dynamic-Sized, +Lock-Free Data Structures" +,booktitle={Proceedings of 16\textsuperscript{th} International +Symposium on Distributed Computing} +,year=2002 +,month="October" +,pages="339-353" +} + @article{Appavoo03a ,author="J. Appavoo and K. Hui and C. A. N. Soules and R. W. Wisniewski and D. M. {Da Silva} and O. Krieger and M. A. Auslander and D. J. Edelsohn and
@@ -447,3 +543,95 @@ Oregon Health and Sciences University" Realtime turns into making RCU yet more realtime friendly. " } + +@conference{ThomasEHart2006a +,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown" +,Title="Making Lockless Synchronization Fast: Performance Implications +of Memory Reclamation" +,Booktitle="20\textsuperscript{th} {IEEE} International Parallel and +Distributed Processing Symposium" +,month="April" +,year="2006" +,day="25-29" +,address="Rhodes, Greece" +,annotation=" + Compares QSBR (AKA "classic RCU"), HPBR, EBR, and lock-free + reference counting. +" +} + +@Conference{PaulEMcKenney2006b +,Author="Paul E. McKenney and Dipankar Sarma and Ingo Molnar and +Suparna Bhattacharya" +,Title="Extending RCU for Realtime and Embedded Workloads" +,Booktitle="{Ottawa Linux Symposium}" +,Month="July" +,Year="2006" +,pages="v2 123-138" +,note="Available: +\url{http://www.linuxsymposium.org/2006/view_abstract.php?content_key=184} +\url{http://www.rdrop.com/users/paulmck/RCU/OLSrtRCU.2006.08.11a.pdf} +[Viewed January 1, 2007]" +,annotation=" + Described how to improve the -rt implementation of realtime RCU. +" +} + +@unpublished{PaulEMcKenney2006c +,Author="Paul E. McKenney" +,Title="Sleepable {RCU}" +,month="October" +,day="9" +,year="2006" +,note="Available: +\url{http://lwn.net/Articles/202847/} +Revised: +\url{http://www.rdrop.com/users/paulmck/RCU/srcu.2007.01.14a.pdf} +[Viewed August 21, 2006]" +,annotation=" + LWN article introducing SRCU. +" +} + +@unpublished{RobertOlsson2006a +,Author="Robert Olsson and Stefan Nilsson" +,Title="{TRASH}: A dynamic {LC}-trie and hash data structure" +,month="August" +,day="18" +,year="2006" +,note="Available: +\url{http://www.nada.kth.se/~snilsson/public/papers/trash/trash.pdf} +[Viewed February 24, 2007]" +,annotation=" + RCU-protected dynamic trie-hash combination. +" +} + +@unpublished{ThomasEHart2007a +,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown and Jonathan Walpole" +,Title="Performance of memory reclamation for lockless synchronization" +,journal="J. Parallel Distrib. Comput." +,year="2007" +,note="To appear in J. Parallel Distrib. Comput. + \url{doi=10.1016/j.jpdc.2007.04.010}" +,annotation={ + Compares QSBR (AKA "classic RCU"), HPBR, EBR, and lock-free + reference counting. Journal version of ThomasEHart2006a. +} +} + +@unpublished{PaulEMcKenney2007QRCUspin +,Author="Paul E. McKenney" +,Title="Using Promela and Spin to verify parallel algorithms" +,month="August" +,day="1" +,year="2007" +,note="Available: +\url{http://lwn.net/Articles/243851/} +[Viewed September 8, 2007]" +,annotation=" + LWN article describing Promela and spin, and also using Oleg + Nesterov's QRCU as an example (with Paul McKenney's fastpath). +" +} +
Documentation/RCU/rcu.txt +17 -2
@@ -36,6 +36,14 @@ o How can the updater tell when a grace period has completed executed in user mode, or executed in the idle loop, we can safely free up that item. + Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the + same effect, but require that the readers manipulate CPU-local + counters. These counters allow limited types of blocking + within RCU read-side critical sections. SRCU also uses + CPU-local counters, and permits general blocking within + RCU read-side critical sections. These two variants of + RCU detect grace periods by sampling these counters. + o If I am running on a uniprocessor kernel, which can only do one thing at a time, why should I wait for a grace period?
@@ -46,7 +54,10 @@ o How can I see where RCU is currently used in the Linux kernel? Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu", "rcu_read_lock_bh", "rcu_read_unlock_bh", "call_rcu_bh", "srcu_read_lock", "srcu_read_unlock", "synchronize_rcu", - "synchronize_net", and "synchronize_srcu". + "synchronize_net", "synchronize_srcu", and the other RCU + primitives. Or grab one of the cscope databases from: + + http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html o What guidelines should I follow when writing code that uses RCU?
@@ -67,7 +78,11 @@ o I hear that RCU is patented? What is with that? o I hear that RCU needs work in order to support realtime kernels? - Yes, work in progress. + This work is largely completed. Realtime-friendly RCU can be + enabled via the CONFIG_PREEMPT_RCU kernel configuration parameter. + However, work is in progress for enabling priority boosting of + preempted RCU read-side critical sections.This is needed if you + have CPU-bound realtime threads. o Where can I find more information on RCU?
Documentation/RCU/torture.txt +3 -3
@@ -46,12 +46,13 @@ stat_interval The number of seconds between output of torture shuffle_interval The number of seconds to keep the test threads affinitied - to a particular subset of the CPUs. Used in conjunction - with test_no_idle_hz. + to a particular subset of the CPUs, defaults to 5 seconds. + Used in conjunction with test_no_idle_hz. test_no_idle_hz Whether or not to test the ability of RCU to operate in a kernel that disables the scheduling-clock interrupt to idle CPUs. Boolean parameter, "1" to test, "0" otherwise. + Defaults to omitting this test. torture_type The type of RCU to test: "rcu" for the rcu_read_lock() API, "rcu_sync" for rcu_read_lock() with synchronous reclamation,
@@ -82,8 +83,6 @@ be evident. ;-) The entries are as follows: -o "ggp": The number of counter flips (or batches) since boot.
Re: [RFC PATCH 6/6] Preempt-RCU: Update RCU Documentation.
Quoted 8 lines · Gautham R Shenoy
Preempt-RCU: Update RCU Documentation.
From: Paul E. McKenney <[email protected]>
This patch updates the RCU documentation to reflect preemptible RCU as well as recent publications.
Signed-off-by Paul E. McKenney <[email protected]>
Gautham, this patch is missing your SoB line - could you please send it?
Ingo
Re: [RFC PATCH 6/6] Preempt-RCU: Update RCU Documentation.
Quoted 5 lines · Ingo Molnar
Quoted 2 lines · Gautham R Shenoy
Preempt-RCU: Update RCU Documentation.
Quoted 8 lines
Quoted 6 lines
From: Paul E. McKenney <[email protected]>
This patch updates the RCU documentation to reflect preemptible RCU as well as recent publications.
Signed-off-by Paul E. McKenney <[email protected]>Gautham, this patch is missing your SoB line - could you please send it?
Sorry about that. I do take responsibility of this patch too!
Patch Appended below.
Quoted 2 lines
Ingo
Thanks and Regards
gautham.
---->
Preempt-RCU: Update RCU Documentation.
From: Paul E. McKenney <[email protected]>
This patch updates the RCU documentation to reflect preemptible RCU as well as recent publications.
---
Documentation/RCU/RTFP.txt | 210 +++++++++++++++++++++++++++++++++++++++-- Documentation/RCU/rcu.txt | 19 +++- Documentation/RCU/torture.txt | 11 +- 3 files changed, 221 insertions(+), 19 deletions(-)
Documentation/RCU/RTFP.txt +199 -11
@@ -9,8 +9,8 @@ The first thing resembling RCU was published in 1980, when Kung and Lehman [Kung80] recommended use of a garbage collector to defer destruction of nodes in a parallel binary search tree in order to simplify its implementation. This works well in environments that have garbage -collectors, but current production garbage collectors incur significant -read-side overhead. +collectors, but most production garbage collectors incur significant +overhead. In 1982, Manber and Ladner [Manber82,Manber84] recommended deferring destruction until all threads running at that time have terminated, again
@@ -99,16 +99,25 @@ locking, reduces contention, reduces memory latency for readers, and parallelizes pipeline stalls and memory latency for writers. However, these techniques still impose significant read-side overhead in the form of memory barriers. Researchers at Sun worked along similar lines -in the same timeframe [HerlihyLM02,HerlihyLMS03]. These techniques -can be thought of as inside-out reference counts, where the count is -represented by the number of hazard pointers referencing a given data -structure (rather than the more conventional counter field within the -data structure itself). +in the same timeframe [HerlihyLM02]. These techniques can be thought +of as inside-out reference counts, where the count is represented by the +number of hazard pointers referencing a given data structure (rather than +the more conventional counter field within the data structure itself). + +By the same token, RCU can be thought of as a "bulk reference count", +where some form of reference counter covers all reference by a given CPU +or thread during a set timeframe. This timeframe is related to, but +not necessarily exactly the same as, an RCU grace period. In classic +RCU, the reference counter is the per-CPU bit in the "bitmask" field, +and each such bit covers all references that might have been made by +the corresponding CPU during the prior grace period. Of course, RCU +can be thought of in other terms as well. In 2003, the K42 group described how RCU could be used to create -hot-pluggable implementations of operating-system functions. Later that -year saw a paper describing an RCU implementation of System V IPC -[Arcangeli03], and an introduction to RCU in Linux Journal [McKenney03a]. +hot-pluggable implementations of operating-system functions [Appavoo03a]. +Later that year saw a paper describing an RCU implementation of System +V IPC [Arcangeli03], and an introduction to RCU in Linux Journal +[McKenney03a]. 2004 has seen a Linux-Journal article on use of RCU in dcache [McKenney04a], a performance comparison of locking to RCU on several
@@ -117,10 +126,19 @@ number of operating-system kernels [PaulEdwardMcKenneyPhD], a paper describing how to make RCU safe for soft-realtime applications [Sarma04c], and a paper describing SELinux performance with RCU [JamesMorris04b]. -2005 has seen further adaptation of RCU to realtime use, permitting +2005 brought further adaptation of RCU to realtime use, permitting preemption of RCU realtime critical sections [PaulMcKenney05a, PaulMcKenney05b]. +2006 saw the first best-paper award for an RCU paper [ThomasEHart2006a], +as well as further work on efficient implementations of preemptible +RCU [PaulEMcKenney2006b], but priority-boosting of RCU read-side critical +sections proved elusive. An RCU implementation permitting general +blocking in read-side critical sections appeared [PaulEMcKenney2006c], +Robert Olsson described an RCU-protected trie-hash combination +[RobertOlsson2006a]. + + Bibtex Entries @article{Kung80
@@ -203,6 +221,41 @@ Bibtex Entries ,Address="New Orleans, LA" } +@conference{Pu95a, +Author = "Calton Pu and Tito Autrey and Andrew Black and Charles Consel and +Crispin Cowan and Jon Inouye and Lakshmi Kethana and Jonathan Walpole and +Ke Zhang", +Title = "Optimistic Incremental Specialization: Streamlining a Commercial +Operating System", +Booktitle = "15\textsuperscript{th} ACM Symposium on +Operating Systems Principles (SOSP'95)", +address = "Copper Mountain, CO", +month="December", +year="1995", +pages="314-321", +annotation=" + Uses a replugger, but with a flag to signal when people are + using the resource at hand. Only one reader at a time. +" +} + +@conference{Cowan96a, +Author = "Crispin Cowan and Tito Autrey and Charles Krasic and +Calton Pu and Jonathan Walpole", +Title = "Fast Concurrent Dynamic Linking for an Adaptive Operating System", +Booktitle = "International Conference on Configurable Distributed Systems +(ICCDS'96)", +address = "Annapolis, MD", +month="May", +year="1996", +pages="108", +isbn="0-8186-7395-8", +annotation=" + Uses a replugger, but with a counter to signal when people are + using the resource at hand. Allows multiple readers. +" +} + @techreport{Slingwine95 ,author="John D. Slingwine and Paul E. McKenney" ,title="Apparatus and Method for Achieving Reduced Overhead Mutual
@@ -312,6 +365,49 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell" [Viewed June 23, 2004]" } +@conference{Michael02a +,author="Maged M. Michael" +,title="Safe Memory Reclamation for Dynamic Lock-Free Objects Using Atomic +Reads and Writes" +,Year="2002" +,Month="August" +,booktitle="{Proceedings of the 21\textsuperscript{st} Annual ACM +Symposium on Principles of Distributed Computing}" +,pages="21-30" +,annotation=" + Each thread keeps an array of pointers to items that it is + currently referencing. Sort of an inside-out garbage collection + mechanism, but one that requires the accessing code to explicitly + state its needs. Also requires read-side memory barriers on + most architectures. +" +} + +@conference{Michael02b +,author="Maged M. Michael" +,title="High Performance Dynamic Lock-Free Hash Tables and List-Based Sets" +,Year="2002" +,Month="August" +,booktitle="{Proceedings of the 14\textsuperscript{th} Annual ACM +Symposium on Parallel +Algorithms and Architecture}" +,pages="73-82" +,annotation=" + Like the title says... +" +} + +@InProceedings{HerlihyLM02 +,author={Maurice Herlihy and Victor Luchangco and Mark Moir} +,title="The Repeat Offender Problem: A Mechanism for Supporting Dynamic-Sized, +Lock-Free Data Structures" +,booktitle={Proceedings of 16\textsuperscript{th} International +Symposium on Distributed Computing} +,year=2002 +,month="October" +,pages="339-353" +} + @article{Appavoo03a ,author="J. Appavoo and K. Hui and C. A. N. Soules and R. W. Wisniewski and D. M. {Da Silva} and O. Krieger and M. A. Auslander and D. J. Edelsohn and
@@ -447,3 +543,95 @@ Oregon Health and Sciences University" Realtime turns into making RCU yet more realtime friendly. " } + +@conference{ThomasEHart2006a +,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown" +,Title="Making Lockless Synchronization Fast: Performance Implications +of Memory Reclamation" +,Booktitle="20\textsuperscript{th} {IEEE} International Parallel and +Distributed Processing Symposium" +,month="April" +,year="2006" +,day="25-29" +,address="Rhodes, Greece" +,annotation=" + Compares QSBR (AKA "classic RCU"), HPBR, EBR, and lock-free + reference counting. +" +} + +@Conference{PaulEMcKenney2006b +,Author="Paul E. McKenney and Dipankar Sarma and Ingo Molnar and +Suparna Bhattacharya" +,Title="Extending RCU for Realtime and Embedded Workloads" +,Booktitle="{Ottawa Linux Symposium}" +,Month="July" +,Year="2006" +,pages="v2 123-138" +,note="Available: +\url{http://www.linuxsymposium.org/2006/view_abstract.php?content_key=184} +\url{http://www.rdrop.com/users/paulmck/RCU/OLSrtRCU.2006.08.11a.pdf} +[Viewed January 1, 2007]" +,annotation=" + Described how to improve the -rt implementation of realtime RCU. +" +} + +@unpublished{PaulEMcKenney2006c +,Author="Paul E. McKenney" +,Title="Sleepable {RCU}" +,month="October" +,day="9" +,year="2006" +,note="Available: +\url{http://lwn.net/Articles/202847/} +Revised: +\url{http://www.rdrop.com/users/paulmck/RCU/srcu.2007.01.14a.pdf} +[Viewed August 21, 2006]" +,annotation=" + LWN article introducing SRCU. +" +} + +@unpublished{RobertOlsson2006a +,Author="Robert Olsson and Stefan Nilsson" +,Title="{TRASH}: A dynamic {LC}-trie and hash data structure" +,month="August" +,day="18" +,year="2006" +,note="Available: +\url{http://www.nada.kth.se/~snilsson/public/papers/trash/trash.pdf} +[Viewed February 24, 2007]" +,annotation=" + RCU-protected dynamic trie-hash combination. +" +} + +@unpublished{ThomasEHart2007a +,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown and Jonathan Walpole" +,Title="Performance of memory reclamation for lockless synchronization" +,journal="J. Parallel Distrib. Comput." +,year="2007" +,note="To appear in J. Parallel Distrib. Comput. + \url{doi=10.1016/j.jpdc.2007.04.010}" +,annotation={ + Compares QSBR (AKA "classic RCU"), HPBR, EBR, and lock-free + reference counting. Journal version of ThomasEHart2006a. +} +} + +@unpublished{PaulEMcKenney2007QRCUspin +,Author="Paul E. McKenney" +,Title="Using Promela and Spin to verify parallel algorithms" +,month="August" +,day="1" +,year="2007" +,note="Available: +\url{http://lwn.net/Articles/243851/} +[Viewed September 8, 2007]" +,annotation=" + LWN article describing Promela and spin, and also using Oleg + Nesterov's QRCU as an example (with Paul McKenney's fastpath). +" +} +
Documentation/RCU/rcu.txt +17 -2
@@ -36,6 +36,14 @@ o How can the updater tell when a grace period has completed executed in user mode, or executed in the idle loop, we can safely free up that item. + Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the + same effect, but require that the readers manipulate CPU-local + counters. These counters allow limited types of blocking + within RCU read-side critical sections. SRCU also uses + CPU-local counters, and permits general blocking within + RCU read-side critical sections. These two variants of + RCU detect grace periods by sampling these counters. + o If I am running on a uniprocessor kernel, which can only do one thing at a time, why should I wait for a grace period?
@@ -46,7 +54,10 @@ o How can I see where RCU is currently used in the Linux kernel? Search for "rcu_read_lock", "rcu_read_unlock", "call_rcu", "rcu_read_lock_bh", "rcu_read_unlock_bh", "call_rcu_bh", "srcu_read_lock", "srcu_read_unlock", "synchronize_rcu", - "synchronize_net", and "synchronize_srcu". + "synchronize_net", "synchronize_srcu", and the other RCU + primitives. Or grab one of the cscope databases from: + + http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html o What guidelines should I follow when writing code that uses RCU?
@@ -67,7 +78,11 @@ o I hear that RCU is patented? What is with that? o I hear that RCU needs work in order to support realtime kernels? - Yes, work in progress. + This work is largely completed. Realtime-friendly RCU can be + enabled via the CONFIG_PREEMPT_RCU kernel configuration parameter. + However, work is in progress for enabling priority boosting of + preempted RCU read-side critical sections.This is needed if you + have CPU-bound realtime threads. o Where can I find more information on RCU?
Documentation/RCU/torture.txt +5 -6
@@ -46,12 +46,13 @@ stat_interval The number of seconds between output of torture shuffle_interval The number of seconds to keep the test threads affinitied - to a particular subset of the CPUs. Used in conjunction - with test_no_idle_hz. + to a particular subset of the CPUs, defaults to 5 seconds. + Used in conjunction with test_no_idle_hz. test_no_idle_hz Whether or not to test the ability of RCU to operate in a kernel that disables the scheduling-clock interrupt to idle CPUs. Boolean parameter, "1" to test, "0" otherwise. + Defaults to omitting this test. torture_type The type of RCU to test: "rcu" for the rcu_read_lock() API, "rcu_sync" for rcu_read_lock() with synchronous reclamation,
@@ -82,8 +83,6 @@ be evident. ;-) The entries are as follows: -o "ggp": The number of counter flips (or batches) since boot. - o "rtc": The hexadecimal address of the structure currently visible to readers.
@@ -117,8 +116,8 @@ o "Reader Pipe": Histogram of "ages" of structures seen by readers. o "Reader Batch": Another histogram of "ages" of structures seen by readers, but in terms of counter flips (or batches) rather than in terms of grace periods. The legal number of non-zero - entries is again two. The reason for this separate view is - that it is easier to get the third entry to show up in the + entries is again two. The reason for this separate view is that + it is sometimes easier to get the third entry to show up in the "Reader Batch" list than in the "Reader Pipe" list. o "Free-Block Circulation": Shows the number of torture structures
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
Steven Rostedt
reviewerQuoted 4 lines · Gautham R Shenoy
Currently it is based against the latest linux-2.6-sched-devel.git
Awaiting your feedback!
Hi Gautham,
Thanks for posting this. I believe this is the same version of preempt RCU as we have in the RT patch. It seems to be very stable. I ran the RT patch version of the RCU Preempt (just the Preempt RCU patches without RT on latest git) on a 64way box and the results seems just as good (if not slightly better) than classic RCU! I'll rerun this patch series on that box and post the results.
Quoted 1 line
From what I'm seening with this, is that it is ready for mainline. These
patches should probably go into -mm and be ready for 2.6.25. If Andrew wants to wait for my results, I'll run them tonight.
Thanks Gautham, Paul and Dipankar for all this great work!
-- Steve
Quoted 16 lines · Steven Rostedt
Quoted 4 lines · Gautham R Shenoy
Currently it is based against the latest linux-2.6-sched-devel.git
Awaiting your feedback!
Hi Gautham,
Thanks for posting this. I believe this is the same version of preempt RCU as we have in the RT patch. It seems to be very stable. I ran the RT patch version of the RCU Preempt (just the Preempt RCU patches without RT on latest git) on a 64way box and the results seems just as good (if not slightly better) than classic RCU! I'll rerun this patch series on that box and post the results.
Quoted 6 lines
Quoted 2 lines
From what I'm seening with this, is that it is ready for mainline. These
patches should probably go into -mm and be ready for 2.6.25. If Andrew wants to wait for my results, I'll run them tonight.
Thanks Gautham, Paul and Dipankar for all this great work!
Steve, if you went with a fine comb over the code and have done a thorough review, could you send me your Reviewed-by line for this patchset?
Ingo
Quoted 3 lines · Ingo Molnar
Steve, if you went with a fine comb over the code and have done a thorough review, could you send me your Reviewed-by line for this patchset?
Sure thing. And here's the thread that contains a lot of my review (and I had some with Paul on IRC).
http://lkml.org/lkml/2007/9/10/213
Hi Steve,
Quoted 7 lines · Steven Rostedt
Quoted 4 lines · Gautham R Shenoy
Currently it is based against the latest linux-2.6-sched-devel.git
Awaiting your feedback!
Quoted 8 lines
Hi Gautham,
Thanks for posting this. I believe this is the same version of preempt RCU as we have in the RT patch. It seems to be very stable. I ran the RT patch version of the RCU Preempt (just the Preempt RCU patches without RT on latest git) on a 64way box and the results seems just as good (if not slightly better) than classic RCU! I'll rerun this patch series on that box and post the results.
The version of synchronize_sched() that's implemented in this patchset requires these CPU-Hotplug patches -->
http://lkml.org/lkml/2007/11/15/239.
This is the reason why I rebased it against linux-2.6-sched-devel.git since it already has those patches.
Quoted 7 lines
Quoted 2 lines
From what I'm seening with this, is that it is ready for mainline. These
patches should probably go into -mm and be ready for 2.6.25. If Andrew wants to wait for my results, I'll run them tonight.
Thanks Gautham, Paul and Dipankar for all this great work!
Anytime! That probably the only way I could teach myself how RCU works!
Quoted 2 lines
-- Steve
Thanks and Regards
gautham.
Signature · 5 lines
Gautham R Shenoy Linux Technology Center IBM India. "Freedom comes with a price tag of responsibility, which is still a bargain, because Freedom is priceless!"
Quoted 21 lines · Gautham R Shenoy
Hello everyone,
This patchset is an updated version of the preemptible RCU patchset that Paul McKenney had posted it in September earlier this year that can be found here --> http://lkml.org/lkml/2007/9/10/213
This patchset incorporates the review comments from Oleg Nesterov and Steven Rostedt.
The testing report of the patchset is as follows:
====================================================================
Patch-stack: 2.6.23-rc3 + cpu-hotplug patches fromhttp://lkml.org/lkml/2007/11/15/239 + Preempt-RCU patches.Test: RCU-Torture running parallelly with CPU-Hotplug
operations.
Duration: 24 hours.
Architectures: x86,x86_64, ppc64.
====================================================================
Quoted 3 lines
Currently it is based against the latest linux-2.6-sched-devel.git
Awaiting your feedback!
thanks Gautham, the patchset from you and Paul looks good to me and i've applied it to sched-devel.git to get it tested and reviewed some more.
from the Nitpicking Police, there are a couple of minor style problems/warnings with the code, you can see it via:
scripts/checkpatch.pl --file kernel/rcu*.c
nothing serious - RCU is still one of the cleanest subsystems in the kernel:
errors lines of code errors/KLOC
kernel/rcuclassic.c 0 575 0 kernel/rcupdate.c 1 138 7.2 kernel/rcupreempt.c 0 953 0 kernel/rcupreempt_trace.c 0 330 0 kernel/rcutorture.c 8 995 8.0
the eventual goal would be to match:
scripts/checkpatch.pl --file kernel/sched*.[ch]
output ;-)
Ingo
Quoted 34 lines · Ingo Molnar
Quoted 21 lines · Gautham R Shenoy
Hello everyone,
This patchset is an updated version of the preemptible RCU patchset that Paul McKenney had posted it in September earlier this year that can be found here --> http://lkml.org/lkml/2007/9/10/213
This patchset incorporates the review comments from Oleg Nesterov and Steven Rostedt.
The testing report of the patchset is as follows:
====================================================================
Patch-stack: 2.6.23-rc3 + cpu-hotplug patches fromhttp://lkml.org/lkml/2007/11/15/239 + Preempt-RCU patches.Test: RCU-Torture running parallelly with CPU-Hotplug
operations.
Duration: 24 hours.
Architectures: x86,x86_64, ppc64.
====================================================================Quoted 3 lines
Currently it is based against the latest linux-2.6-sched-devel.git
Awaiting your feedback!
thanks Gautham, the patchset from you and Paul looks good to me and i've applied it to sched-devel.git to get it tested and reviewed some more.
from the Nitpicking Police, there are a couple of minor style problems/warnings with the code, you can see it via:
Quoted 8 lines
scripts/checkpatch.pl --file kernel/rcu*.cnothing serious - RCU is still one of the cleanest subsystems in the kernel:
errors lines of code errors/KLOC
kernel/rcuclassic.c 0 575 0 kernel/rcupdate.c 1 138 7.2
This one is the exception to checkpatch.pl's rule against "volatile". ;-)
The volatile declaration is within the ACCESS_ONCE() macro that is used within rcu_read_lock() and rcu_read_unlock() to force the compiler to maintain ordering with respect to interrupt handler running only on that same CPU.
Quoted 3 lines
kernel/rcupreempt.c 0 953 0 kernel/rcupreempt_trace.c 0 330 0 kernel/rcutorture.c 8 995 8.0
Hmmm... Definitely some old whitespace issues here...
Quoted 5 lines
the eventual goal would be to match:
scripts/checkpatch.pl --file kernel/sched*.[ch]output ;-)
We should be able to make some progress in that direction. ;-)
Thanx, Paul