RCU commits queued for 2.6.36/7
Subsystems (29)
cover RCU commits queued for 2.6.36/7
Hello!
This patchset shows the patches queued for 2.6.36/7. These are not yet ready to move upstream due to dependencies on commits moving up other trees. The patches are as follows:
1-19. These are the remaining patches implementing Arnd's sparse-checking
work. #1 needs to go up the networking tree, but will cause regressions until f5155b33277 ("rcu: add an rcu_dereference_index_check()") reaches mainline. #2 depends on #1, and many of the later patches (#3-#30) depend on #2. Many of #3-#19 will go up non-tip trees.
20. Update Documentation/RCU to remove the now-deprecated rcu_head
initialization macros.
21. Fix long-standing DocBook example showing long-dead three-argument
variant of call_rcu().
22. Make the CPU stall warning timeout configurable at build time
with new RCU_CPU_STALL_TIMEOUT config parameter.
23. Remove the now-deprecated rcu_head initialization macros.
This depends on a couple of patches removing their use that are making their way up other maintainer trees.
24. Document the new debug assists from Arnd, Mathieu, and myself.
25. Add random preemption to rcutorture testing of preemptible
RCU, courtesy of Lai Jiangshan.
26. Remove TREE_RCU's ->rda[] array in favor of the new __percpu
facility, also courtesy of Lai Jiangshan.
27. Fixlet to #26 for RCU tracing.
28. Rename __do_rcu_dereference_check() to rcu_lockdep_assert(),
given that Tetsuo Handa has found outside-of-RCU use for this function.
29. Add a boot parameter to suppress RCU CPU stall warnings.
30. Update kerneldoc or rcu_read_lock(), call_rcu(), and
synchronize_rcu().
For a testing-only git-tree version of this patchset, see:
git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git rcu/testing
By "testing-only" I mean that any of the commits in the rcu/testing branch might be folded, spindled, or otherwise mutilated at any time. ;-)
Thanx, Paul
Documentation/DocBook/kernel-locking.tmpl | 6 b/Documentation/DocBook/kernel-locking.tmpl | 8 b/Documentation/RCU/checklist.txt | 23 + b/arch/x86/include/asm/kvm_host.h | 2 b/drivers/input/evdev.c | 2 b/drivers/net/bnx2.h | 2 b/drivers/net/bnx2x.h | 2 b/drivers/net/cnic.h | 2 b/drivers/net/macvtap.c | 2 b/drivers/vhost/net.c | 15 - b/drivers/vhost/vhost.c | 29 +- b/drivers/vhost/vhost.h | 11 b/include/linux/cgroup.h | 4 b/include/linux/compiler.h | 4 b/include/linux/cred.h | 2 b/include/linux/fdtable.h | 6 b/include/linux/fs.h | 2 b/include/linux/genhd.h | 6 b/include/linux/idr.h | 4 b/include/linux/if_macvlan.h | 2 b/include/linux/igmp.h | 4 b/include/linux/init_task.h | 4 b/include/linux/input.h | 2 b/include/linux/iocontext.h | 2 b/include/linux/key.h | 3 b/include/linux/kvm_host.h | 2 b/include/linux/mm_types.h | 2 b/include/linux/netdevice.h | 12 b/include/linux/nfs_fs.h | 2 b/include/linux/notifier.h | 10 b/include/linux/perf_event.h | 6 b/include/linux/radix-tree.h | 4 b/include/linux/rculist.h | 53 ++-- b/include/linux/rculist_nulls.h | 16 - b/include/linux/rcupdate.h | 352 ++++++++++++++++------------ b/include/linux/sched.h | 2 b/include/linux/srcu.h | 27 +- b/include/linux/sunrpc/auth_gss.h | 4 b/include/net/cls_cgroup.h | 3 b/include/net/dst.h | 2 b/include/net/fib_rules.h | 2 b/include/net/garp.h | 2 b/include/net/inet_sock.h | 2 b/include/net/ip6_tunnel.h | 2 b/include/net/ipip.h | 6 b/include/net/net_namespace.h | 2 b/include/net/netfilter/nf_conntrack.h | 2 b/include/net/netns/xfrm.h | 2 b/include/net/sock.h | 4 b/kernel/cgroup.c | 2 b/kernel/pid.c | 2 b/kernel/rcupdate.c | 6 b/kernel/rcutorture.c | 6 b/kernel/rcutree.c | 42 +-- b/kernel/rcutree.h | 5 b/kernel/rcutree_plugin.h | 4 b/kernel/rcutree_trace.c | 2 b/kernel/sched.c | 2 b/lib/Kconfig.debug | 13 + b/lib/radix-tree.c | 2 b/net/802/stp.c | 4 b/net/ipv4/ip_gre.c | 2 b/net/ipv4/ipip.c | 10 b/net/ipv4/netfilter/nf_nat_core.c | 2 b/net/ipv4/protocol.c | 2 b/net/ipv4/route.c | 2 b/net/ipv4/tcp.c | 4 b/net/ipv6/ip6_tunnel.c | 6 b/net/ipv6/protocol.c | 2 b/net/ipv6/sit.c | 10 b/net/mac80211/ieee80211_i.h | 15 - b/net/mac80211/sta_info.h | 4 b/net/netfilter/core.c | 2 b/net/netfilter/nf_conntrack_ecache.c | 4 b/net/netfilter/nf_conntrack_extend.c | 2 b/net/netfilter/nf_conntrack_proto.c | 4 b/net/netfilter/nf_log.c | 2 b/net/netfilter/nf_queue.c | 2 b/net/netlabel/netlabel_domainhash.c | 4 b/net/netlabel/netlabel_unlabeled.c | 4 b/net/netlink/af_netlink.c | 2 b/net/phonet/af_phonet.c | 2 b/net/phonet/pn_dev.c | 2 b/net/socket.c | 2 include/linux/kvm_host.h | 4 include/linux/rcupdate.h | 36 +- include/linux/sched.h | 6 kernel/rcutree.c | 11 kernel/rcutree.h | 2 kernel/rcutree_plugin.h | 8 lib/Kconfig.debug | 11 91 files changed, 563 insertions(+), 366 deletions(-)
RFC PATCH 2/30 rcu: define __rcu address space modifier for sparse
This commit provides definitions for the __rcu annotation defined earlier. This annotation permits sparse to check for correct use of RCU-protected pointers. If a pointer that is annotated with __rcu is accessed directly (as opposed to via rcu_dereference(), rcu_assign_pointer(), or one of their variants), sparse can be made to complain. To enable such complaints, use the new default-disabled CONFIG_SPARSE_RCU_POINTER kernel configuration option. Please note that these sparse complaints are intended to be a debugging aid, -not- a code-style-enforcement mechanism.
There are special rcu_dereference_protected() and rcu_access_pointer() accessors for use when RCU read-side protection is not required, for example, when no other CPU has access to the data structure in question or while the current CPU hold the update-side lock.
This patch also updates a number of docbook comments that were showing their age.
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Christopher Li <[email protected]>
Cc Josh Triplett <[email protected]>
--- include/linux/compiler.h | 4 + include/linux/rcupdate.h | 352 ++++++++++++++++++++++++++++------------------ include/linux/srcu.h | 27 +++- kernel/rcupdate.c | 6 +- lib/Kconfig.debug | 13 ++ 5 files changed, 257 insertions(+), 145 deletions(-)
include/linux/compiler.h +4 -0
@@ -16,7 +16,11 @@# define __release (x) __context__(x,-1)# define __cond_lock (x,c) ((c) ? ({ __acquire(x); 1; }) : 0)# define __percpu __attribute__((noderef, address_space(3))) +#ifdef CONFIG_SPARSE_RCU_POINTER +# define __rcu __attribute__((noderef, address_space(4))) +#else # define __rcu +#endif extern void __chk_user_ptr (const volatile void __user *);extern void __chk_io_ptr (const volatile void __iomem *);#else
include/linux/rcupdate.h +213 -139
477 lines not on this page · raw message
include/linux/srcu.h +23 -4
@@ -108,12 +108,31 @@ static inline int srcu_read_lock_held(struct srcu_struct *sp)#endif /* #else #ifdef CONFIG_DEBUG_LOCK_ALLOC */ /** -* srcu_dereference - fetch SRCU-protected pointer with checking +* srcu_dereference_check - fetch SRCU-protected pointer for later dereferencing +* @p: the pointer to fetch and protect for later dereferencing +* @sp: pointer to the srcu_struct, which is used to check that we +* really are in an SRCU read-side critical section. +* @c: condition to check for update-side use * -* Makes rcu_dereference_check() do the dirty work. +* If PROVE_RCU is enabled, invoking this outside of an RCU read-side +* critical section will result in an RCU-lockdep splat, unless @c evaluates +* to 1. The @c argument will normally be a logical expression containing +* lockdep_is_held() calls. */ -#define srcu_dereference (p, sp) \ - rcu_dereference_check(p, srcu_read_lock_held(sp)) +#define srcu_dereference_check (p, sp, c) \ + __rcu_dereference_check((p), srcu_read_lock_held(sp) || (c), __rcu) + +/** +* srcu_dereference - fetch SRCU-protected pointer for later dereferencing +* @p: the pointer to fetch and protect for later dereferencing +* @sp: pointer to the srcu_struct, which is used to check that we +* really are in an SRCU read-side critical section. +* +* Makes rcu_dereference_check() do the dirty work. If PROVE_RCU +* is enabled, invoking this outside of an RCU read-side critical +* section will result in an RCU-lockdep splat. +*/ +#define srcu_dereference (p, sp) srcu_dereference_check((p), (sp), 0)/** * srcu_read_lock - register a new reader for an SRCU-protected structure.
kernel/rcupdate.c +4 -2
@@ -73,12 +73,14 @@ int debug_lockdep_rcu_enabled(void)EXPORT_SYMBOL_GPL (debug_lockdep_rcu_enabled);/** -* rcu_read_lock_bh_held - might we be in RCU-bh read-side critical section? +* rcu_read_lock_bh_held() - might we be in RCU-bh read-side critical section? * * Check for bottom half being disabled, which covers both the * CONFIG_PROVE_RCU and not cases. Note that if someone uses * rcu_read_lock_bh(), but then later enables BH, lockdep (if enabled) -* will show the situation. +* will show the situation. This is useful for debug checks in functions +* that require that they be called within an RCU read-side critical +* section. * * Check debug_lockdep_rcu_enabled() to prevent false positives during boot. */
lib/Kconfig.debug +13 -0
@@ -528,6 +528,19 @@ config PROVE_RCU_REPEATEDLY disabling, allowing multiple RCU-lockdep warnings to be printed on a single reboot. +config SPARSE_RCU_POINTER + bool "RCU debugging: sparse-based checks for pointer usage" + default n + help + This feature enables the __rcu sparse annotation for + RCU-protected pointers. This annotation will cause sparse + to flag any non-RCU used of annotated pointers. This can be + helpful when debugging RCU usage. Please note that this feature + is not intended to enforce code cleanliness; it is instead merely + a debugging aid. + + Say Y to make sparse flag questionable use of RCU-protected pointers + Say N if you are unsure. config LOCKDEP
Signature · 1 line
1.7.0.6
RFC PATCH 5/30 credentials: rcu annotation
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Acked-by David Howells <[email protected]>
--- include/linux/sched.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
include/linux/sched.h +2 -2
@@ -1297,9 +1297,9 @@ struct task_struct {struct list_head cpu_timers[3 ];/* process credentials */ -const struct cred *real_cred;/* objective and real subjective task +const struct cred __rcu *real_cred;/* objective and real subjective task * credentials (COW) */ -const struct cred *cred;/* effective (overridable) subjective task +const struct cred __rcu *cred;/* effective (overridable) subjective task * credentials (COW) */ struct mutex cred_guard_mutex;/* guard against foreign influences on * credential calculations
Signature · 1 line
1.7.0.6
RFC PATCH 3/30 rculist: avoid __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
This avoids warnings from missing __rcu annotations in the rculist implementation, making it possible to use the same lists in both RCU and non-RCU cases.
We can add rculist annotations later, together with lockdep support for rculist, which is missing as well, but that may involve changing all the users.
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Pavel Emelyanov <[email protected]>
Cc Sukadev Bhattiprolu <[email protected]>
--- include/linux/rculist.h | 53 ++++++++++++++++++++++++++-------------- include/linux/rculist_nulls.h | 16 ++++++++---- kernel/pid.c | 2 +- 3 files changed, 46 insertions(+), 25 deletions(-)
include/linux/rculist.h +34 -19
@@ -10,6 +10,12 @@#include <linux/rcupdate.h> /* +* return the ->next pointer of a list_head in an rcu safe +* way, we must not access it directly +*/ +#define list_next_rcu (list) (*((struct list_head __rcu **)(&(list)->next))) + +/* * Insert a new entry between two known consecutive entries. * * This is only for internal list manipulation where we know
@@ -20,7 +26,7 @@ static inline void __list_add_rcu(struct list_head *new, { new->next = next; new->prev = prev; -rcu_assign_pointer (prev->next, new); +rcu_assign_pointer (list_next_rcu (prev), new); next->prev = new; }
@@ -138,7 +144,7 @@ static inline void list_replace_rcu(struct list_head *old, { new->next = old->next; new->prev = old->prev; -rcu_assign_pointer (new->prev->next, new); +rcu_assign_pointer (list_next_rcu (new->prev), new); new->next->prev = new; old->prev =LIST_POISON2 ; }
@@ -193,7 +199,7 @@ static inline void list_splice_init_rcu(struct list_head *list,*/ last->next = at; -rcu_assign_pointer (head->next, first); +rcu_assign_pointer (list_next_rcu (head), first); first->prev = head; at->prev = last; }
@@ -208,7 +214,9 @@ static inline void list_splice_init_rcu(struct list_head *list,* primitives such as list_add_rcu() as long as it's guarded by rcu_read_lock(). */ #define list_entry_rcu (ptr, type, member) \ - container_of(rcu_dereference_raw(ptr), type, member) + ({typeof (*ptr) __rcu *__ptr = (typeof (*ptr) __rcu __force *)ptr; \ + container_of((typeof(ptr))rcu_dereference_raw(__ptr), type, member); \ + })/** * list_first_entry_rcu - get the first element from a list
@@ -225,9 +233,9 @@ static inline void list_splice_init_rcu(struct list_head *list, list_entry_rcu((ptr)->next, type, member)#define __list_for_each_rcu (pos, head) \ - for (pos = rcu_dereference_raw((head)->next); \ + for (pos = rcu_dereference_raw(list_next_rcu(head)); \ pos != (head); \ - pos = rcu_dereference_raw(pos->next)) + pos = rcu_dereference_raw(list_next_rcu((pos)))/** * list_for_each_entry_rcu - iterate over rcu list of given type
@@ -257,9 +265,9 @@ static inline void list_splice_init_rcu(struct list_head *list,* as long as the traversal is guarded by rcu_read_lock(). */ #define list_for_each_continue_rcu (pos, head) \ - for ((pos) = rcu_dereference_raw((pos)->next); \ + for ((pos) = rcu_dereference_raw(list_next_rcu(pos)); \ prefetch((pos)->next), (pos) != (head); \ - (pos) = rcu_dereference_raw((pos)->next)) + (pos) = rcu_dereference_raw(list_next_rcu(pos)))/** * list_for_each_entry_continue_rcu - continue iteration over list of given type
@@ -314,12 +322,19 @@ static inline void hlist_replace_rcu(struct hlist_node *old, new->next = next; new->pprev = old->pprev; -rcu_assign_pointer (*new->pprev, new); +rcu_assign_pointer (*(struct hlist_node __rcu **)new->pprev, new);if (next) new->next->pprev = &new->next; old->pprev =LIST_POISON2 ; } +/* +* return the first or the next element in an RCU protected hlist +*/ +#define hlist_first_rcu (head) (*((struct hlist_node __rcu **)(&(head)->first))) +#define hlist_next_rcu (node) (*((struct hlist_node __rcu **)(&(node)->next))) +#define hlist_pprev_rcu (node) (*((struct hlist_node __rcu **)((node)->pprev))) +/** * hlist_add_head_rcu * @n: the element to add to the hash list.
@@ -346,7 +361,7 @@ static inline void hlist_add_head_rcu(struct hlist_node *n, n->next = first; n->pprev = &h->first; -rcu_assign_pointer (h->first, n); +rcu_assign_pointer (hlist_first_rcu (h), n);if (first) first->pprev = &n->next; }
@@ -374,7 +389,7 @@ static inline void hlist_add_before_rcu(struct hlist_node *n, { n->pprev = next->pprev; n->next = next; -rcu_assign_pointer (*(n->pprev), n); +rcu_assign_pointer (hlist_pprev_rcu (n), n); next->pprev = &n->next; }
@@ -401,15 +416,15 @@ static inline void hlist_add_after_rcu(struct hlist_node *prev, { n->next = prev->next; n->pprev = &prev->next; -rcu_assign_pointer (prev->next, n); +rcu_assign_pointer (hlist_next_rcu (prev), n);if (n->next) n->next->pprev = &n->next; } -#define __hlist_for_each_rcu (pos, head) \ - for (pos = rcu_dereference((head)->first); \ - pos && ({ prefetch(pos->next); 1; }); \ - pos = rcu_dereference(pos->next)) +#define __hlist_for_each_rcu (pos, head) \ + for (pos = rcu_dereference(hlist_first_rcu(head)); \ + pos && ({ prefetch(pos->next); 1; }); \ + pos = rcu_dereference(hlist_next_rcu(pos)))/** * hlist_for_each_entry_rcu - iterate over rcu list of given type
@@ -422,11 +437,11 @@ static inline void hlist_add_after_rcu(struct hlist_node *prev,* the _rcu list-mutation primitives such as hlist_add_head_rcu() * as long as the traversal is guarded by rcu_read_lock(). */ -#define hlist_for_each_entry_rcu (tpos, pos, head, member) \ - for (pos = rcu_dereference_raw((head)->first); \ +#define hlist_for_each_entry_rcu (tpos, pos, head, member) \ + for (pos = rcu_dereference_raw(hlist_first_rcu(head)); \ pos && ({ prefetch(pos->next); 1; }) && \ ({ tpos = hlist_entry(pos, typeof(*tpos), member); 1; }); \ - pos = rcu_dereference_raw(pos->next)) + pos = rcu_dereference_raw(hlist_next_rcu(pos)))/** * hlist_for_each_entry_rcu_bh - iterate over rcu list of given type
include/linux/rculist_nulls.h +11 -5
@@ -37,6 +37,12 @@ static inline void hlist_nulls_del_init_rcu(struct hlist_nulls_node *n) } } +#define hlist_nulls_first_rcu (head) \ + (*((struct hlist_nulls_node __rcu __force **)&(head)->first)) + +#define hlist_nulls_next_rcu (node) \ + (*((struct hlist_nulls_node __rcu __force **)&(node)->next)) +/** * hlist_nulls_del_rcu - deletes entry from hash list without re-initialization * @n: the element to delete from the hash list.
@@ -88,7 +94,7 @@ static inline void hlist_nulls_add_head_rcu(struct hlist_nulls_node *n, n->next = first; n->pprev = &h->first; -rcu_assign_pointer (h->first, n); +rcu_assign_pointer (hlist_nulls_first_rcu (h), n);if (!is_a_nulls (first)) first->pprev = &n->next; }
@@ -100,11 +106,11 @@ static inline void hlist_nulls_add_head_rcu(struct hlist_nulls_node *n,* @member: the name of the hlist_nulls_node within the struct. * */ -#define hlist_nulls_for_each_entry_rcu (tpos, pos, head, member) \ - for (pos = rcu_dereference_raw((head)->first); \ - (!is_a_nulls(pos)) && \ +#define hlist_nulls_for_each_entry_rcu (tpos, pos, head, member) \ + for (pos = rcu_dereference_raw(hlist_nulls_first_rcu(head)); \ + (!is_a_nulls(pos)) && \ ({ tpos = hlist_nulls_entry(pos, typeof(*tpos), member); 1; }); \ - pos = rcu_dereference_raw(pos->next)) + pos = rcu_dereference_raw(hlist_nulls_next_rcu(pos)))#endif #endif
kernel/pid.c +1 -1
@@ -367,7 +367,7 @@ struct task_struct *pid_task(struct pid *pid, enum pid_type type)struct task_struct *result =NULL ;if (pid) {struct hlist_node *first; - first =rcu_dereference_check (pid->tasks[type].first, + first =rcu_dereference_check (hlist_first_rcu (&pid->tasks[type]),rcu_read_lock_held () ||lockdep_tasklist_lock_is_held ());if (first)
Signature · 1 line
1.7.0.6
RFC PATCH 6/30 keys: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Acked-by David Howells <[email protected]>
--- include/linux/cred.h | 2 +- include/linux/key.h | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-)
include/linux/cred.h +1 -1
@@ -84,7 +84,7 @@ struct thread_group_cred {atomic_t usage;pid_t tgid;/* thread group process ID */ spinlock_t lock; -struct key *session_keyring;/* keyring inherited over fork */ +struct key __rcu *session_keyring;/* keyring inherited over fork */ struct key *process_keyring;/* keyring private to this process */ struct rcu_head rcu;/* RCU deletion hook */ };
include/linux/key.h +2 -1
@@ -178,8 +178,9 @@ struct key {*/ union {unsigned long value; +void __rcu *rcudata;void *data; -struct keyring_list *subscriptions; +struct keyring_list __rcu *subscriptions; } payload; };
Signature · 1 line
1.7.0.6
RFC PATCH 8/30 net: __rcu annotations for drivers
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Eric Dumazet <[email protected]>
--- drivers/net/bnx2.h | 2 +- drivers/net/bnx2x.h | 2 +- drivers/net/cnic.h | 2 +- drivers/net/macvtap.c | 2 +- include/linux/if_macvlan.h | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-)
drivers/net/bnx2.h +1 -1
@@ -6749,7 +6749,7 @@ struct bnx2 {u32 tx_wake_thresh;#ifdef BCM_CNIC -struct cnic_ops *cnic_ops; +struct cnic_ops __rcu *cnic_ops;void *cnic_data;#endif
drivers/net/bnx2x.h +1 -1
@@ -1036,7 +1036,7 @@ struct bnx2x {dma_addr_t timers_mapping;void *qm;dma_addr_t qm_mapping; -struct cnic_ops *cnic_ops; +struct cnic_ops __rcu *cnic_ops;void *cnic_data;u32 cnic_tag;struct cnic_eth_dev cnic_eth_dev;
drivers/net/cnic.h +1 -1
@@ -177,7 +177,7 @@ struct cnic_local {#define ULP_F_INIT 0#define ULP_F_START 1#define ULP_F_CALL_PENDING 2 -struct cnic_ulp_ops *ulp_ops[MAX_CNIC_ULP_TYPE ]; +struct cnic_ulp_ops __rcu *ulp_ops[MAX_CNIC_ULP_TYPE ];unsigned long cnic_local_flags;#define CNIC_LCL_FL_KWQ_INIT 0x0
drivers/net/macvtap.c +1 -1
@@ -39,7 +39,7 @@ struct macvtap_queue {struct socket sock;struct socket_wq wq;int vnet_hdr_sz; -struct macvlan_dev *vlan; +struct macvlan_dev __rcu *vlan;struct file *file;unsigned int flags; };
include/linux/if_macvlan.h +1 -1
@@ -47,7 +47,7 @@ struct macvlan_dev {enum macvlan_mode mode;int (*receive)(struct sk_buff *skb);int (*forward)(struct net_device *dev,struct sk_buff *skb); -struct macvtap_queue *tap; +struct macvtap_queue __rcu *tap; }; static inline void macvlan_count_rx(const struct macvlan_dev *vlan,
Signature · 1 line
1.7.0.6
RFC PATCH 20/30 Update documentation to note the passage of INIT_RCU_HEAD()
In reply to Paul E. McKenneySigned-off-by Alexey Dobriyan <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
--- Documentation/DocBook/kernel-locking.tmpl | 8 -------- 1 files changed, 0 insertions(+), 8 deletions(-)
Documentation/DocBook/kernel-locking.tmpl +0 -8
@@ -1725,14 +1725,6 @@ the amount of locking which needs to be done. if (++cache_num > MAX_CACHE_SIZE) { struct object *i, *outcast = NULL; list_for_each_entry(i, &cache, list) { -@@ -85,6 +94,7 @@ - obj->popularity = 0; - atomic_set(&obj->refcnt, 1); /* The cache holds a reference */ - spin_lock_init(&obj->lock); -+ INIT_RCU_HEAD(&obj->rcu); - - spin_lock_irqsave(&cache_lock, flags); - __cache_add(obj); @@ -104,12 +114,11 @@ struct object *cache_find(int id) {
Signature · 1 line
1.7.0.6
RFC PATCH 16/30 kernel: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
This adds annotations for RCU operations in core kernel components
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Al Viro <[email protected]>
Cc Jens Axboe <[email protected]>
Cc Andrew Morton <[email protected]>
--- include/linux/fdtable.h | 6 +++--- include/linux/fs.h | 2 +- include/linux/genhd.h | 6 +++--- include/linux/init_task.h | 4 ++-- include/linux/iocontext.h | 2 +- include/linux/mm_types.h | 2 +- 6 files changed, 11 insertions(+), 11 deletions(-)
include/linux/fdtable.h +3 -3
@@ -31,7 +31,7 @@ struct embedded_fd_set {struct fdtable {unsigned int max_fds; -struct file ** fd;/* current fd array */ +struct file __rcu **fd;/* current fd array */ fd_set *close_on_exec;fd_set *open_fds;struct rcu_head rcu;
@@ -46,7 +46,7 @@ struct files_struct {* read mostly part */ atomic_t count; -struct fdtable *fdt; +struct fdtable __rcu *fdt;struct fdtable fdtab;/* * written part on a separate cache line in SMP
@@ -55,7 +55,7 @@ struct files_struct {int next_fd;struct embedded_fd_set close_on_exec_init;struct embedded_fd_set open_fds_init; -struct file * fd_array[NR_OPEN_DEFAULT ]; +struct file __rcu * fd_array[NR_OPEN_DEFAULT ]; };#define rcu_dereference_check_fdtable (files, fdtfd) \
include/linux/fs.h +1 -1
@@ -1381,7 +1381,7 @@ struct super_block {* Saved mount options for lazy filesystems using * generic_show_options() */ -char *s_options; +char __rcu *s_options; };extern struct timespec current_fs_time (struct super_block *sb);
include/linux/genhd.h +3 -3
@@ -129,8 +129,8 @@ struct blk_scsi_cmd_filter {struct disk_part_tbl {struct rcu_head rcu_head;int len; -struct hd_struct *last_lookup; -struct hd_struct *part[]; +struct hd_struct __rcu *last_lookup; +struct hd_struct __rcu *part[]; };struct gendisk {
@@ -149,7 +149,7 @@ struct gendisk {* non-critical accesses use RCU. Always access through * helpers. */ -struct disk_part_tbl *part_tbl; +struct disk_part_tbl __rcu *part_tbl;struct hd_struct part0;const struct block_device_operations *fops;
include/linux/init_task.h +2 -2
@@ -137,8 +137,8 @@ extern struct cred init_cred; .children = LIST_HEAD_INIT(tsk.children), \ .sibling = LIST_HEAD_INIT(tsk.sibling), \ .group_leader = &tsk, \ - .real_cred = &init_cred, \ - .cred = &init_cred, \ + RCU_INIT_POINTER(.real_cred, &init_cred), \ + RCU_INIT_POINTER(.cred, &init_cred), \ .cred_guard_mutex = \__MUTEX_INITIALIZER (tsk.cred_guard_mutex), \ .comm = "swapper", \
include/linux/iocontext.h +1 -1
@@ -53,7 +53,7 @@ struct io_context {struct radix_tree_root radix_root;struct hlist_head cic_list; -void *ioc_data; +void __rcu *ioc_data; };static inline struct io_context *ioc_task_link (struct io_context *ioc)
include/linux/mm_types.h +1 -1
@@ -299,7 +299,7 @@ struct mm_struct {* new_owner->mm == mm * new_owner->alloc_lock is held */ -struct task_struct *owner; +struct task_struct __rcu *owner;#endif #ifdef CONFIG_PROC_FS
Signature · 1 line
1.7.0.6
RFC PATCH 4/30 cgroups: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Acked-by Paul Menage <[email protected]>
Cc Li Zefan <[email protected]>
--- include/linux/cgroup.h | 4 ++-- include/linux/sched.h | 2 +- kernel/cgroup.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
include/linux/cgroup.h +2 -2
@@ -75,7 +75,7 @@ struct cgroup_subsys_state {unsigned long flags;/* ID for this css, if possible */ -struct css_id *id; +struct css_id __rcu *id; };/* bits in struct cgroup_subsys_state flags field */
@@ -205,7 +205,7 @@ struct cgroup {struct list_head children;/* my children */ struct cgroup *parent;/* my parent */ -struct dentry *dentry;/* cgroup fs entry, RCU protected */ +struct dentry __rcu *dentry;/* cgroup fs entry, RCU protected */ /* Private pointers for each registered subsystem */ struct cgroup_subsys_state *subsys[CGROUP_SUBSYS_COUNT ];
include/linux/sched.h +1 -1
@@ -1427,7 +1427,7 @@ struct task_struct {#endif #ifdef CONFIG_CGROUPS /* Control Group info protected by css_set_lock */ -struct css_set *cgroups; +struct css_set __rcu *cgroups;/* cg_list protected by css_set_lock and tsk->alloc_lock */ struct list_head cg_list;#endif
kernel/cgroup.c +1 -1
@@ -138,7 +138,7 @@ struct css_id {* is called after synchronize_rcu(). But for safe use, css_is_removed() * css_tryget() should be used for avoiding race. */ -struct cgroup_subsys_state *css; +struct cgroup_subsys_state __rcu *css;/* * ID of this css. */
Signature · 1 line
1.7.0.6
RFC PATCH 28/30 rcu: rename __do_rcu_dereference_check() to rcu_lockdep_assert()
In reply to Paul E. McKenneyThe __do_rcu_dereference_check() functionality needs to be exported, so make a real name for it. This allows RCU-lockdep assertions to be used independent of rcu_dereference() and friends.
Signed-off-by Paul E. McKenney <[email protected]>
--- include/linux/rcupdate.h | 14 +++++++++----- 1 files changed, 9 insertions(+), 5 deletions(-)
include/linux/rcupdate.h +9 -5
@@ -209,7 +209,11 @@ static inline int rcu_read_lock_sched_held(void)extern int rcu_my_thread_group_empty (void ); -#define __do_rcu_dereference_check (c) \ +/** +* rcu_lockdep_assert - emit lockdep splat if specified condition not met +* @c: condition to check +*/ +#define rcu_lockdep_assert (c) \ do { \ static bool __warned; \ if (debug_lockdep_rcu_enabled() && !__warned && !(c)) { \
@@ -220,7 +224,7 @@ extern int rcu_my_thread_group_empty(void);#else /* #ifdef CONFIG_PROVE_RCU */ -#define __do_rcu_dereference_check (c) do { } while (0) +#define rcu_lockdep_assert (c) do { } while (0)#endif /* #else #ifdef CONFIG_PROVE_RCU */
@@ -241,14 +245,14 @@ extern int rcu_my_thread_group_empty(void);#define __rcu_dereference_check (p, c, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p)*__force )ACCESS_ONCE(p); \ - __do_rcu_dereference_check(c); \ + rcu_lockdep_assert(c); \ (void) (((typeof (*p) space *)p) == p); \ smp_read_barrier_depends(); \ ((typeof(*p) __force __kernel *)(_________p1)); \ })#define __rcu_dereference_protected (p, c, space) \ ({ \ - __do_rcu_dereference_check(c); \ + rcu_lockdep_assert(c); \ (void) (((typeof (*p) space *)p) == p); \ ((typeof(*p) __force __kernel *)(p)); \ })
@@ -256,7 +260,7 @@ extern int rcu_my_thread_group_empty(void);#define __rcu_dereference_index_check (p, c) \ ({ \ typeof(p) _________p1 = ACCESS_ONCE(p); \ - __do_rcu_dereference_check(c); \ + rcu_lockdep_assert(c); \ smp_read_barrier_depends(); \ (_________p1); \ })
Signature · 1 line
1.7.0.6
RFC PATCH 12/30 idr: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Manfred Spraul <[email protected]>
--- include/linux/idr.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
include/linux/idr.h +2 -2
@@ -50,14 +50,14 @@struct idr_layer {unsigned long bitmap;/* A zero bit means "space here" */ -struct idr_layer *ary[1 <<IDR_BITS ]; +struct idr_layer __rcu *ary[1 <<IDR_BITS ];int count;/* When zero, we can release it */ int layer;/* distance from leaf */ struct rcu_head rcu_head; };struct idr { -struct idr_layer *top; +struct idr_layer __rcu *top;struct idr_layer *id_free;int layers;/* only valid without concurrent changes */ int id_free_cnt;
Signature · 1 line
1.7.0.6
RFC PATCH 9/30 perf_event: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Paul Mackerras <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
--- include/linux/perf_event.h | 6 +++--- include/linux/sched.h | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-)
include/linux/perf_event.h +3 -3
@@ -645,7 +645,7 @@ struct perf_event {struct hlist_node hlist_entry;int nr_siblings;int group_flags; -struct perf_event *group_leader; +struct perf_event __rcu *group_leader;const struct pmu *pmu;enum perf_event_active_state state;
@@ -709,7 +709,7 @@ struct perf_event {atomic_t mmap_count;int mmap_locked;struct user_struct *mmap_user; -struct perf_mmap_data *data; +struct perf_mmap_data __rcu *data;/* poll related */ wait_queue_head_t waitq;
@@ -777,7 +777,7 @@ struct perf_event_context {* These fields let us detect when two contexts have both * been cloned (inherited) from a common ancestor. */ -struct perf_event_context *parent_ctx; +struct perf_event_context __rcu *parent_ctx;u64 parent_gen;u64 generation;int pin_count;
include/linux/sched.h +1 -1
@@ -1440,7 +1440,7 @@ struct task_struct {struct futex_pi_state *pi_state_cache;#endif #ifdef CONFIG_PERF_EVENTS -struct perf_event_context *perf_event_ctxp; +struct perf_event_context __rcu *perf_event_ctxp;struct mutex perf_event_mutex;struct list_head perf_event_list;#endif
Signature · 1 line
1.7.0.6
RFC PATCH 19/30 vhost: add __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Also add rcu_dreference_protected() for code paths where locks are held.
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc "Michael S. Tsirkin" <[email protected]>
--- drivers/vhost/net.c | 15 +++++++++++---- drivers/vhost/vhost.c | 29 +++++++++++++++++++++++------ drivers/vhost/vhost.h | 11 ++++++++--- 3 files changed, 42 insertions(+), 13 deletions(-)
drivers/vhost/net.c +11 -4
@@ -110,7 +110,9 @@ static void handle_tx(struct vhost_net *net)size_t len, total_len =0 ;int err, wmem;size_t hdr_size; -struct socket *sock =rcu_dereference (vq->private_data); +struct socket *sock; + + sock =rcu_dereference_check (vq->private_data,in_vhost_workqueue ());if (!sock)return ;
@@ -364,7 +366,10 @@ static void vhost_net_disable_vq(struct vhost_net *n,static void vhost_net_enable_vq (struct vhost_net *n,struct vhost_virtqueue *vq) { -struct socket *sock = vq->private_data; +struct socket *sock; + + sock =rcu_dereference_protected (vq->private_data, +lockdep_is_held (&vq->mutex));if (!sock)return ;if (vq == n->vqs +VHOST_NET_VQ_TX ) {
@@ -380,7 +385,8 @@ static struct socket *vhost_net_stop_vq(struct vhost_net *n,struct socket *sock;mutex_lock (&vq->mutex); - sock = vq->private_data; + sock =rcu_dereference_protected (vq->private_data, +lockdep_is_held (&vq->mutex));vhost_net_disable_vq (n, vq);rcu_assign_pointer (vq->private_data,NULL );mutex_unlock (&vq->mutex);
@@ -518,7 +524,8 @@ static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd) }/* start polling new socket */ - oldsock = vq->private_data; + oldsock =rcu_dereference_protected (vq->private_data, +lockdep_is_held (&vq->mutex));if (sock == oldsock) goto done;
drivers/vhost/vhost.c +23 -6
@@ -39,6 +39,13 @@ enum {static struct workqueue_struct *vhost_workqueue; +#ifdef CONFIG_PROVE_RCU +int in_vhost_workqueue (void ) +{ +return in_workqueue_context (vhost_workqueue); +} +#endif /* #ifdef CONFIG_PROVE_RCU */ +static void vhost_poll_func (struct file *file,wait_queue_head_t *wqh,poll_table *pt) {
@@ -180,7 +187,7 @@ long vhost_dev_reset_owner(struct vhost_dev *dev)vhost_dev_cleanup (dev); memory->nregions =0 ; - dev->memory = memory; +RCU_INIT_POINTER (dev->memory, memory);return 0 ; }
@@ -212,8 +219,9 @@ void vhost_dev_cleanup(struct vhost_dev *dev)fput (dev->log_file); dev->log_file =NULL ;/* No one will access memory at this point */ -kfree (dev->memory); - dev->memory =NULL ; +kfree (rcu_dereference_protected (dev->memory, +lockdep_is_held (&dev->mutex))); +RCU_INIT_POINTER (dev->memory,NULL );if (dev->mm)mmput (dev->mm); dev->mm =NULL ;
@@ -294,14 +302,22 @@ static int vq_access_ok(unsigned int num, /* Caller should have device mutex but not vq mutex */ int vhost_log_access_ok(struct vhost_dev *dev) { - return memory_access_ok(dev, dev->memory, 1); + struct vhost_memory *mp; + + mp = rcu_dereference_protected(dev->memory, + lockdep_is_held(&dev->mutex)); + return memory_access_ok(dev, mp, 1); } /* Verify access for write logging. */ /* Caller should have vq mutex and device mutex */ static int vq_log_access_ok(struct vhost_virtqueue *vq, void __user *log_base) { - return vq_memory_access_ok(log_base, vq->dev->memory, + struct vhost_memory *mp; + + mp = rcu_dereference_protected(vq->dev->memory, + lockdep_is_held(&vq->mutex)); + return vq_memory_access_ok(log_base, mp, vhost_has_feature(vq->dev, VHOST_F_LOG_ALL)) && (!vq->log_used || log_access_ok(log_base, vq->log_addr, sizeof *vq->used +
@@ -341,7 +357,8 @@ static long vhost_set_memory(struct vhost_dev *d, struct vhost_memory __user *m)kfree (newmem);return -EFAULT ; } - oldmem = d->memory; + oldmem =rcu_dereference_protected (d->memory, +lockdep_is_held (&d->mutex));rcu_assign_pointer (d->memory, newmem);synchronize_rcu ();kfree (oldmem);
drivers/vhost/vhost.h +8 -3
@@ -92,7 +92,7 @@ struct vhost_virtqueue {* work item execution acts instead of rcu_read_lock() and the end of * work item execution acts instead of rcu_read_lock(). * Writers use virtqueue mutex. */ -void *private_data; +void __rcu *private_data;/* Log write descriptors */ void __user *log_base;struct vhost_log log[VHOST_NET_MAX_SG ];
@@ -102,7 +102,7 @@ struct vhost_dev {/* Readers use RCU to access memory table pointer * log base pointer and features. * Writers use mutex below.*/ -struct vhost_memory *memory; +struct vhost_memory __rcu *memory;struct mm_struct *mm;struct mutex mutex;unsigned acked_features;
@@ -112,6 +112,7 @@ struct vhost_dev {struct eventfd_ctx *log_ctx; }; +int in_vhost_workqueue (void );long vhost_dev_init (struct vhost_dev *,struct vhost_virtqueue *vqs,int nvqs);long vhost_dev_check_owner (struct vhost_dev *);long vhost_dev_reset_owner (struct vhost_dev *);
@@ -154,7 +155,11 @@ enum {static inline int vhost_has_feature (struct vhost_dev *dev,int bit) { -unsigned acked_features =rcu_dereference (dev->acked_features); +unsigned acked_features; + + acked_features = +rcu_dereference_index_check (dev->acked_features, +lockdep_is_held (&dev->mutex));return acked_features & (1 << bit); }
Signature · 1 line
1.7.0.6
RFC PATCH 7/30 nfs: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Trond Myklebust <[email protected]>
--- include/linux/nfs_fs.h | 2 +- include/linux/sunrpc/auth_gss.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
include/linux/nfs_fs.h +1 -1
@@ -178,7 +178,7 @@ struct nfs_inode {struct nfs4_cached_acl *nfs4_acl;/* NFSv4 state */ struct list_head open_states; -struct nfs_delegation *delegation; +struct nfs_delegation __rcu *delegation;fmode_t delegation_state;struct rw_semaphore rwsem;#endif /* CONFIG_NFS_V4*/
include/linux/sunrpc/auth_gss.h +2 -2
@@ -69,7 +69,7 @@ struct gss_cl_ctx {enum rpc_gss_proc gc_proc;u32 gc_seq;spinlock_t gc_seq_lock; -struct gss_ctx *gc_gss_ctx; +struct gss_ctx __rcu *gc_gss_ctx;struct xdr_netobj gc_wire_ctx;u32 gc_win;unsigned long gc_expiry;
@@ -80,7 +80,7 @@ struct gss_upcall_msg;struct gss_cred {struct rpc_cred gc_base;enum rpc_gss_svc gc_service; -struct gss_cl_ctx *gc_ctx; +struct gss_cl_ctx __rcu *gc_ctx;struct gss_upcall_msg *gc_upcall;unsigned long gc_upcall_timestamp;unsigned char gc_machine_cred :1 ;
Signature · 1 line
1.7.0.6
RFC PATCH 17/30 net: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Eric Dumazet <[email protected]>
--- include/linux/igmp.h | 4 ++-- include/linux/netdevice.h | 12 ++++++------ include/net/dst.h | 2 +- include/net/fib_rules.h | 2 +- include/net/garp.h | 2 +- include/net/inet_sock.h | 2 +- include/net/ip6_tunnel.h | 2 +- include/net/ipip.h | 6 +++--- include/net/net_namespace.h | 2 +- include/net/netns/xfrm.h | 2 +- include/net/sock.h | 4 ++-- kernel/sched.c | 2 +- net/802/stp.c | 4 ++-- net/ipv4/ip_gre.c | 2 +- net/ipv4/ipip.c | 10 +++++----- net/ipv4/protocol.c | 2 +- net/ipv4/route.c | 2 +- net/ipv4/tcp.c | 4 ++-- net/ipv6/ip6_tunnel.c | 6 +++--- net/ipv6/protocol.c | 2 +- net/ipv6/sit.c | 10 +++++----- net/mac80211/ieee80211_i.h | 15 ++++++++------- net/mac80211/sta_info.h | 4 ++-- net/netlabel/netlabel_domainhash.c | 4 ++-- net/netlabel/netlabel_unlabeled.c | 4 ++-- net/netlink/af_netlink.c | 2 +- net/phonet/af_phonet.c | 2 +- net/phonet/pn_dev.c | 2 +- net/socket.c | 2 +- 29 files changed, 60 insertions(+), 59 deletions(-)
include/linux/igmp.h +2 -2
@@ -167,10 +167,10 @@ struct ip_sf_socklist {*/ struct ip_mc_socklist { -struct ip_mc_socklist *next; +struct ip_mc_socklist __rcu *next;struct ip_mreqn multi;unsigned int sfmode;/* MCAST_{INCLUDE,EXCLUDE} */ -struct ip_sf_socklist *sflist; +struct ip_sf_socklist __rcu *sflist;struct rcu_head rcu; };
include/linux/netdevice.h +6 -6
@@ -924,10 +924,10 @@ struct net_device {#ifdef CONFIG_NET_DSA void *dsa_ptr;/* dsa specific data */ #endif -void *atalk_ptr;/* AppleTalk link */ -void *ip_ptr;/* IPv4 specific data */ +void *atalk_ptr;/* AppleTalk link */ +void __rcu *ip_ptr;/* IPv4 specific data */ void *dn_ptr;/* DECnet specific data */ -void *ip6_ptr;/* IPv6 specific data */ +void __rcu *ip6_ptr;/* IPv6 specific data */ void *ec_ptr;/* Econet specific data */ void *ax25_ptr;/* AX.25 specific data */ struct wireless_dev *ieee80211_ptr;/* IEEE 802.11 specific data,
@@ -1025,11 +1025,11 @@ struct net_device {void *ml_priv;/* bridge stuff */ -struct net_bridge_port *br_port; +void __rcu *br_port;/* macvlan */ -struct macvlan_port *macvlan_port; +struct macvlan_port __rcu *macvlan_port;/* GARP */ -struct garp_port *garp_port; +struct garp_port __rcu *garp_port;/* class/net/name entry */ struct device dev;
include/net/dst.h +1 -1
@@ -94,7 +94,7 @@ struct dst_entry {unsigned long lastuse;union {struct dst_entry *next; -struct rtable *rt_next; +struct rtable __rcu *rt_next;struct rt6_info *rt6_next;struct dn_route *dn_next; };
include/net/fib_rules.h +1 -1
@@ -20,7 +20,7 @@ struct fib_rule {u32 table;u8 action;u32 target; -struct fib_rule * ctarget; +struct fib_rule __rcu *ctarget;char iifname[IFNAMSIZ ];char oifname[IFNAMSIZ ];struct rcu_head rcu;
include/net/garp.h +1 -1
@@ -107,7 +107,7 @@ struct garp_applicant { };struct garp_port { -struct garp_applicant *applicants[GARP_APPLICATION_MAX +1 ]; +struct garp_applicant __rcu *applicants[GARP_APPLICATION_MAX +1 ]; };extern int garp_register_application (struct garp_application *app);
include/net/inet_sock.h +1 -1
@@ -140,7 +140,7 @@ struct inet_sock { mc_all:1 ;int mc_index;__be32 mc_addr; -struct ip_mc_socklist *mc_list; +struct ip_mc_socklist __rcu *mc_list;struct {unsigned int flags;unsigned int fragsize;
include/net/ip6_tunnel.h +1 -1
@@ -13,7 +13,7 @@/* IPv6 tunnel */ struct ip6_tnl { -struct ip6_tnl *next;/* next tunnel in list */ +struct ip6_tnl __rcu *next;/* next tunnel in list */ struct net_device *dev;/* virtual device associated with tunnel */ struct ip6_tnl_parm parms;/* tunnel configuration parameters */ struct flowi fl;/* flowi template for xmit */
include/net/ipip.h +3 -3
@@ -16,7 +16,7 @@ struct ip_tunnel_6rd_parm { };struct ip_tunnel { -struct ip_tunnel *next; +struct ip_tunnel __rcu *next;struct net_device *dev;int err_count;/* Number of arrived ICMP errors */
@@ -34,12 +34,12 @@ struct ip_tunnel {#ifdef CONFIG_IPV6_SIT_6RD struct ip_tunnel_6rd_parm ip6rd;#endif -struct ip_tunnel_prl_entry *prl;/* potential router list */ +struct ip_tunnel_prl_entry __rcu *prl;/* potential router list */ unsigned int prl_count;/* # of entries in PRL */ };struct ip_tunnel_prl_entry { -struct ip_tunnel_prl_entry *next; +struct ip_tunnel_prl_entry __rcu *next;__be32 addr;u16 flags;struct rcu_head rcu_head;
include/net/net_namespace.h +1 -1
@@ -90,7 +90,7 @@ struct net {#ifdef CONFIG_WEXT_CORE struct sk_buff_head wext_nlevents;#endif -struct net_generic *gen; +struct net_generic __rcu *gen; };
include/net/netns/xfrm.h +1 -1
@@ -48,7 +48,7 @@ struct netns_xfrm {struct dst_ops xfrm6_dst_ops;#endif -struct sock *nlsk; +struct sock __rcu *nlsk;struct sock *nlsk_stash;u32 sysctl_aevent_etime;
include/net/sock.h +2 -2
@@ -148,7 +148,7 @@ struct sock_common { };struct proto *skc_prot;#ifdef CONFIG_NET_NS -struct net *skc_net; +struct net __rcu *skc_net;#endif };
@@ -298,7 +298,7 @@ struct sock {struct ucred sk_peercred;long sk_rcvtimeo;long sk_sndtimeo; -struct sk_filter *sk_filter; +struct sk_filter __rcu *sk_filter;void *sk_protinfo;struct timer_list sk_timer;ktime_t sk_stamp;
kernel/sched.c +1 -1
@@ -496,7 +496,7 @@ struct rq {#ifdef CONFIG_SMP struct root_domain *rd; -struct sched_domain *sd; +struct sched_domain __rcu *sd;unsigned long cpu_power;
net/802/stp.c +2 -2
@@ -21,8 +21,8 @@#define GARP_ADDR_MAX 0x2F#define GARP_ADDR_RANGE (GARP_ADDR_MAX - GARP_ADDR_MIN) -static const struct stp_proto *garp_protos[GARP_ADDR_RANGE +1 ]__read_mostly ; -static const struct stp_proto *stp_proto__read_mostly ; +static const struct stp_proto __rcu *garp_protos[GARP_ADDR_RANGE +1 ]__read_mostly ; +static const struct stp_proto __rcu *stp_proto__read_mostly ;static struct llc_sap *sap__read_mostly ;static unsigned int sap_registered;
net/ipv4/ip_gre.c +1 -1
@@ -128,7 +128,7 @@ static int ipgre_tunnel_bind_dev(struct net_device *dev);static int ipgre_net_id__read_mostly ;struct ipgre_net { -struct ip_tunnel *tunnels[4 ][HASH_SIZE ]; +struct ip_tunnel __rcu *tunnels[4 ][HASH_SIZE ];struct net_device *fb_tunnel_dev; };
net/ipv4/ipip.c +5 -5
@@ -122,11 +122,11 @@static int ipip_net_id__read_mostly ;struct ipip_net { -struct ip_tunnel *tunnels_r_l[HASH_SIZE ]; -struct ip_tunnel *tunnels_r[HASH_SIZE ]; -struct ip_tunnel *tunnels_l[HASH_SIZE ]; -struct ip_tunnel *tunnels_wc[1 ]; -struct ip_tunnel **tunnels[4 ]; +struct ip_tunnel __rcu *tunnels_r_l[HASH_SIZE ]; +struct ip_tunnel __rcu *tunnels_r[HASH_SIZE ]; +struct ip_tunnel __rcu *tunnels_l[HASH_SIZE ]; +struct ip_tunnel __rcu *tunnels_wc[1 ]; +struct ip_tunnel __rcu **tunnels[4 ];struct net_device *fb_tunnel_dev; };
net/ipv4/protocol.c +1 -1
@@ -28,7 +28,7 @@#include <linux/spinlock.h> #include <net/protocol.h> -const struct net_protocol *inet_protos[MAX_INET_PROTOS ]____cacheline_aligned_in_smp ; +const struct net_protocol __rcu *inet_protos[MAX_INET_PROTOS ]____cacheline_aligned_in_smp ;static DEFINE_SPINLOCK (inet_proto_lock);/*
net/ipv4/route.c +1 -1
@@ -199,7 +199,7 @@ const __u8 ip_tos2prio[16] = {*/ struct rt_hash_bucket { -struct rtable *chain; +struct rtable __rcu *chain; }; #if defined(CONFIG_SMP) || defined(CONFIG_DEBUG_SPINLOCK) || \
net/ipv4/tcp.c +2 -2
@@ -3072,9 +3072,9 @@ static struct tcp_cookie_secret tcp_secret_one;static struct tcp_cookie_secret tcp_secret_two;/* Essentially a circular list, without dynamic allocation. */ -static struct tcp_cookie_secret *tcp_secret_generating; +static struct tcp_cookie_secret __rcu *tcp_secret_generating;static struct tcp_cookie_secret *tcp_secret_primary; -static struct tcp_cookie_secret *tcp_secret_retiring; +static struct tcp_cookie_secret __rcu *tcp_secret_retiring;static struct tcp_cookie_secret *tcp_secret_secondary;static DEFINE_SPINLOCK (tcp_secret_locker);
net/ipv6/ip6_tunnel.c +3 -3
@@ -83,9 +83,9 @@ struct ip6_tnl_net {/* the IPv6 tunnel fallback device */ struct net_device *fb_tnl_dev;/* lists for storing tunnels in use */ -struct ip6_tnl *tnls_r_l[HASH_SIZE ]; -struct ip6_tnl *tnls_wc[1 ]; -struct ip6_tnl **tnls[2 ]; +struct ip6_tnl __rcu *tnls_r_l[HASH_SIZE ]; +struct ip6_tnl __rcu *tnls_wc[1 ]; +struct ip6_tnl __rcu **tnls[2 ]; };/*
net/ipv6/protocol.c +1 -1
@@ -25,7 +25,7 @@#include <linux/spinlock.h> #include <net/protocol.h> -const struct inet6_protocol *inet6_protos[MAX_INET_PROTOS ]; +const struct inet6_protocol __rcu *inet6_protos[MAX_INET_PROTOS ];static DEFINE_SPINLOCK (inet6_proto_lock);
net/ipv6/sit.c +5 -5
@@ -68,11 +68,11 @@ static void ipip6_tunnel_setup(struct net_device *dev);static int sit_net_id__read_mostly ;struct sit_net { -struct ip_tunnel *tunnels_r_l[HASH_SIZE ]; -struct ip_tunnel *tunnels_r[HASH_SIZE ]; -struct ip_tunnel *tunnels_l[HASH_SIZE ]; -struct ip_tunnel *tunnels_wc[1 ]; -struct ip_tunnel **tunnels[4 ]; +struct ip_tunnel __rcu *tunnels_r_l[HASH_SIZE ]; +struct ip_tunnel __rcu *tunnels_r[HASH_SIZE ]; +struct ip_tunnel __rcu *tunnels_l[HASH_SIZE ]; +struct ip_tunnel __rcu *tunnels_wc[1 ]; +struct ip_tunnel __rcu **tunnels[4 ];struct net_device *fb_tunnel_dev; };
net/mac80211/ieee80211_i.h +8 -7
@@ -192,7 +192,7 @@ struct beacon_data { };struct ieee80211_if_ap { -struct beacon_data *beacon; +struct beacon_data __rcu *beacon;struct list_head vlans;
@@ -214,7 +214,7 @@ struct ieee80211_if_vlan {struct list_head list;/* used for all tx if the VLAN is configured to 4-addr mode */ -struct sta_info *sta; +struct sta_info __rcu *sta; };struct mesh_stats {
@@ -407,7 +407,8 @@ struct ieee80211_if_ibss {unsigned long ibss_join_req;/* probe response/beacon for IBSS */ -struct sk_buff *presp, *skb; +struct sk_buff __rcu *presp; +struct sk_buff *skb;enum {IEEE80211_IBSS_MLME_SEARCH ,
@@ -511,9 +512,9 @@ struct ieee80211_sub_if_data {#define NUM_DEFAULT_KEYS 4#define NUM_DEFAULT_MGMT_KEYS 2 -struct ieee80211_key *keys[NUM_DEFAULT_KEYS +NUM_DEFAULT_MGMT_KEYS ]; -struct ieee80211_key *default_key; -struct ieee80211_key *default_mgmt_key; +struct ieee80211_key __rcu *keys[NUM_DEFAULT_KEYS +NUM_DEFAULT_MGMT_KEYS ]; +struct ieee80211_key __rcu *default_key; +struct ieee80211_key __rcu *default_mgmt_key;u16 sequence_number;
@@ -716,7 +717,7 @@ struct ieee80211_local {spinlock_t sta_lock;unsigned long num_sta;struct list_head sta_list, sta_pending_list; -struct sta_info *sta_hash[STA_HASH_SIZE ]; +struct sta_info __rcu *sta_hash[STA_HASH_SIZE ];struct timer_list sta_cleanup;struct work_struct sta_finish_work;int sta_generation;
net/mac80211/sta_info.h +2 -2
@@ -226,10 +226,10 @@ struct sta_ampdu_mlme {struct sta_info {/* General information, mostly static */ struct list_head list; -struct sta_info *hnext; +struct sta_info __rcu *hnext;struct ieee80211_local *local;struct ieee80211_sub_if_data *sdata; -struct ieee80211_key *key; +struct ieee80211_key __rcu *key;struct rate_control_ref *rate_ctrl;void *rate_ctrl_priv;spinlock_t lock;
net/netlabel/netlabel_domainhash.c +2 -2
@@ -57,8 +57,8 @@ static DEFINE_SPINLOCK(netlbl_domhsh_lock);#define netlbl_domhsh_rcu_deref (p) \ rcu_dereference_check(p, rcu_read_lock_held() || \ lockdep_is_held(&netlbl_domhsh_lock)) -static struct netlbl_domhsh_tbl *netlbl_domhsh =NULL ; -static struct netlbl_dom_map *netlbl_domhsh_def =NULL ; +static struct netlbl_domhsh_tbl __rcu *netlbl_domhsh =NULL ; +static struct netlbl_dom_map __rcu *netlbl_domhsh_def =NULL ;/* * Domain Hash Table Helper Functions
net/netlabel/netlabel_unlabeled.c +2 -2
@@ -118,8 +118,8 @@ static DEFINE_SPINLOCK(netlbl_unlhsh_lock);#define netlbl_unlhsh_rcu_deref (p) \ rcu_dereference_check(p, rcu_read_lock_held() || \ lockdep_is_held(&netlbl_unlhsh_lock)) -static struct netlbl_unlhsh_tbl *netlbl_unlhsh =NULL ; -static struct netlbl_unlhsh_iface *netlbl_unlhsh_def =NULL ; +static struct netlbl_unlhsh_tbl __rcu *netlbl_unlhsh =NULL ; +static struct netlbl_unlhsh_iface __rcu *netlbl_unlhsh_def =NULL ;/* Accept unlabeled packets flag */ static u8 netlabel_unlabel_acceptflg =0 ;
net/netlink/af_netlink.c +1 -1
@@ -119,7 +119,7 @@ struct nl_pid_hash {struct netlink_table {struct nl_pid_hash hash;struct hlist_head mc_list; -unsigned long *listeners; +unsigned long __rcu *listeners;unsigned int nl_nonroot;unsigned int groups;struct mutex *cb_mutex;
net/phonet/af_phonet.c +1 -1
@@ -35,7 +35,7 @@#include <net/phonet/pn_dev.h> /* Transport protocol registration */ -static struct phonet_protocol *proto_tab[PHONET_NPROTO ]__read_mostly ; +static struct phonet_protocol __rcu *proto_tab[PHONET_NPROTO ]__read_mostly ;static struct phonet_protocol *phonet_proto_get (int protocol) {
net/phonet/pn_dev.c +1 -1
@@ -36,7 +36,7 @@struct phonet_routes {struct mutex lock; -struct net_device *table[64 ]; +struct net_device __rcu *table[64 ]; };struct phonet_net {
net/socket.c +1 -1
@@ -156,7 +156,7 @@ static const struct file_operations socket_file_ops = {*/ static DEFINE_SPINLOCK (net_family_lock); -static const struct net_proto_family *net_families[NPROTO ]__read_mostly ; +static const struct net_proto_family __rcu *net_families[NPROTO ]__read_mostly ;/* * Statistics counters of the socket lists
Signature · 1 line
1.7.0.6
RFC PATCH 26/30 rcu: simplify the usage of percpu data
In reply to Paul E. McKenneyFrom: Lai Jiangshan <[email protected]>
&percpu_data is compatible with allocated percpu data.
And we use it and remove the "->rda[NR_CPUS]" array, saving significant storage on systems with large numbers of CPUs. This does add an additional level of indirection and thus an additional cache line referenced, but because ->rda is not used on the read side, this is OK.
Signed-off-by Lai Jiangshan <[email protected]>
Reviewed-by Tejun Heo <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
--- kernel/rcutree.c | 42 +++++++++++++++--------------------------- kernel/rcutree.h | 2 +- kernel/rcutree_plugin.h | 4 ++-- 3 files changed, 18 insertions(+), 30 deletions(-)
kernel/rcutree.c +15 -27
@@ -712,7 +712,7 @@ static void rcu_start_gp(struct rcu_state *rsp, unsigned long flags) __releases(rcu_get_root(rsp)->lock) { - struct rcu_data *rdp = rsp->rda[smp_processor_id()]; + struct rcu_data *rdp = this_cpu_ptr(rsp->rda);struct rcu_node *rnp =rcu_get_root (rsp);if (!cpu_needs_another_gp (rsp, rdp) || rsp->fqs_active) {
@@ -960,7 +960,7 @@ rcu_check_quiescent_state(struct rcu_state *rsp, struct rcu_data *rdp)static void rcu_send_cbs_to_orphanage (struct rcu_state *rsp) {int i; -struct rcu_data *rdp = rsp->rda[smp_processor_id ()]; +struct rcu_data *rdp =this_cpu_ptr (rsp->rda);if (rdp->nxtlist ==NULL )return ;/* irqs disabled, so comparison is stable. */
@@ -984,7 +984,7 @@ static void rcu_adopt_orphan_cbs(struct rcu_state *rsp)struct rcu_data *rdp;raw_spin_lock_irqsave (&rsp->onofflock, flags); - rdp = rsp->rda[smp_processor_id ()]; + rdp =this_cpu_ptr (rsp->rda);if (rsp->orphan_cbs_list ==NULL ) {raw_spin_unlock_irqrestore (&rsp->onofflock, flags);return ;
@@ -1007,7 +1007,7 @@ static void __rcu_offline_cpu(int cpu, struct rcu_state *rsp)unsigned long flags;unsigned long mask;int need_report =0 ; -struct rcu_data *rdp = rsp->rda[cpu]; +struct rcu_data *rdp =per_cpu_ptr (rsp->rda, cpu);struct rcu_node *rnp;/* Exclude any attempts to start a new grace period. */
@@ -1226,7 +1226,8 @@ static void force_qs_rnp(struct rcu_state *rsp, int (*f)(struct rcu_data *)) cpu = rnp->grplo; bit =1 ;for (; cpu <= rnp->grphi; cpu++, bit <<=1 ) { -if ((rnp->qsmask & bit) !=0 &&f (rsp->rda[cpu])) +if ((rnp->qsmask & bit) !=0 && +f (per_cpu_ptr (rsp->rda, cpu))) mask |= bit; }if (mask !=0 ) {
@@ -1402,7 +1403,7 @@ __call_rcu(struct rcu_head *head, void (*func)(struct rcu_head *rcu),* a quiescent state betweentimes. */ local_irq_save (flags); - rdp = rsp->rda[smp_processor_id ()]; + rdp =this_cpu_ptr (rsp->rda);rcu_process_gp_end (rsp, rdp);check_for_new_grace_period (rsp, rdp);
@@ -1701,7 +1702,7 @@ rcu_boot_init_percpu_data(int cpu, struct rcu_state *rsp) {unsigned long flags;int i; -struct rcu_data *rdp = rsp->rda[cpu]; +struct rcu_data *rdp =per_cpu_ptr (rsp->rda, cpu);struct rcu_node *rnp =rcu_get_root (rsp);/* Set up local state, ensuring consistent view of global state. */
@@ -1729,7 +1730,7 @@ rcu_init_percpu_data(int cpu, struct rcu_state *rsp, int preemptable) {unsigned long flags;unsigned long mask; -struct rcu_data *rdp = rsp->rda[cpu]; +struct rcu_data *rdp =per_cpu_ptr (rsp->rda, cpu);struct rcu_node *rnp =rcu_get_root (rsp);/* Set up local state, ensuring consistent view of global state. */
@@ -1865,7 +1866,8 @@ static void __init rcu_init_levelspread(struct rcu_state *rsp)/* * Helper function for rcu_init() that initializes one rcu_state structure. */ -static void __init rcu_init_one (struct rcu_state *rsp) +static void __init rcu_init_one (struct rcu_state *rsp, +struct rcu_data __percpu *rda) {static char *buf[] = {"rcu_node_level_0" ,"rcu_node_level_1" ,
@@ -1918,37 +1920,23 @@ static void __init rcu_init_one(struct rcu_state *rsp) } } + rsp->rda = rda; rnp = rsp->level[NUM_RCU_LVLS -1 ];for_each_possible_cpu (i) {while (i > rnp->grphi) rnp++; - rsp->rda[i]->mynode = rnp; +per_cpu_ptr (rsp->rda, i)->mynode = rnp;rcu_boot_init_percpu_data (i, rsp); } } -/* -* Helper macro for __rcu_init() and __rcu_init_preempt(). To be used -* nowhere else! Assigns leaf node pointers into each CPU's rcu_data -* structure. -*/ -#define RCU_INIT_FLAVOR (rsp, rcu_data) \ -do { \ - int i; \ - \ - for_each_possible_cpu(i) { \ - (rsp)->rda[i] = &per_cpu(rcu_data, i); \ - } \ - rcu_init_one(rsp); \ -} while (0) -void __init rcu_init (void ) {int cpu;rcu_bootup_announce (); -RCU_INIT_FLAVOR (&rcu_sched_state, rcu_sched_data); -RCU_INIT_FLAVOR (&rcu_bh_state, rcu_bh_data); +rcu_init_one (&rcu_sched_state, &rcu_sched_data); +rcu_init_one (&rcu_bh_state, &rcu_bh_data);__rcu_init_preempt ();open_softirq (RCU_SOFTIRQ , rcu_process_callbacks);
kernel/rcutree.h +1 -1
@@ -284,7 +284,7 @@ struct rcu_state {struct rcu_node *level[NUM_RCU_LVLS ];/* Hierarchy levels. */ u32 levelcnt[MAX_RCU_LVLS +1 ];/* # nodes in each level. */ u8 levelspread[NUM_RCU_LVLS ];/* kids/node in each level. */ -struct rcu_data *rda[NR_CPUS ];/* array of rdp pointers. */ +struct rcu_data __percpu *rda;/* pointer of percu rcu_data. */ /* The following fields are guarded by the root rcu_node's lock. */
kernel/rcutree_plugin.h +2 -2
@@ -154,7 +154,7 @@ static void rcu_preempt_note_context_switch(int cpu) (t->rcu_read_unlock_special & RCU_READ_UNLOCK_BLOCKED) == 0) {/* Possibly blocking in an RCU read-side critical section. */ - rdp = rcu_preempt_state.rda[cpu]; + rdp =per_cpu_ptr (rcu_preempt_state.rda, cpu); rnp = rdp->mynode;raw_spin_lock_irqsave (&rnp->lock, flags); t->rcu_read_unlock_special |=RCU_READ_UNLOCK_BLOCKED ;
@@ -771,7 +771,7 @@ static void rcu_preempt_send_cbs_to_orphanage(void)*/ static void __init __rcu_init_preempt (void ) { -RCU_INIT_FLAVOR (&rcu_preempt_state, rcu_preempt_data); +rcu_init_one (&rcu_preempt_state, &rcu_preempt_data); }/*
Signature · 1 line
1.7.0.6
RFC PATCH 22/30 rcu: make CPU stall warning timeout configurable
In reply to Paul E. McKenneyAlso set the default to 60 seconds, up from the previous hard-coded timeout of 10 seconds. This allows people who care to set short timeouts, while avoiding people with unusual configurations (make randconfig!!!) from being bothered with spurious CPU stall warnings.
Signed-off-by Paul E. McKenney <[email protected]>
--- kernel/rcutree.h | 5 +++-- lib/Kconfig.debug | 11 +++++++++++ 2 files changed, 14 insertions(+), 2 deletions(-)
kernel/rcutree.h +3 -2
@@ -254,9 +254,10 @@ struct rcu_data {#define RCU_STALL_DELAY_DELTA 0#endif -#define RCU_SECONDS_TILL_STALL_CHECK (10 * HZ + RCU_STALL_DELAY_DELTA) +#define RCU_SECONDS_TILL_STALL_CHECK (CONFIG_RCU_CPU_STALL_TIMEOUT * HZ + \ + RCU_STALL_DELAY_DELTA)/* for rsp->jiffies_stall */ -#define RCU_SECONDS_TILL_STALL_RECHECK (30 * HZ + RCU_STALL_DELAY_DELTA) +#define RCU_SECONDS_TILL_STALL_RECHECK (3 * RCU_SECONDS_TILL_STALL_CHECK + 30)/* for rsp->jiffies_stall */ #define RCU_STALL_RAT_DELAY 2/* Allow other CPUs time */ /* to take at least one */
lib/Kconfig.debug +11 -0
@@ -821,6 +821,17 @@ config RCU_CPU_STALL_DETECTOR Say Y if you are unsure. +config RCU_CPU_STALL_TIMEOUT + int "RCU CPU stall timeout in seconds" + depends on RCU_CPU_STALL_DETECTOR + range 3 300 + default 60 + help + If a given RCU grace period extends more than the specified + number of seconds, a CPU stall warning is printed. If the + RCU grace period persists, additional CPU stall warnings are + printed at more widely spaced intervals. + config RCU_CPU_STALL_VERBOSE bool "Print additional per-task information for RCU_CPU_STALL_DETECTOR" depends on RCU_CPU_STALL_DETECTOR && TREE_PREEMPT_RCU
Signature · 1 line
1.7.0.6
RFC PATCH 15/30 kvm: add __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Avi Kivity <[email protected]>
Cc Marcelo Tosatti <[email protected]>
--- include/linux/kvm_host.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
include/linux/kvm_host.h +1 -1
@@ -203,7 +203,7 @@ struct kvm {struct mutex irq_lock;#ifdef CONFIG_HAVE_KVM_IRQCHIP -struct kvm_irq_routing_table *irq_routing; +struct kvm_irq_routing_table __rcu *irq_routing;struct hlist_head mask_notifier_list;struct hlist_head irq_ack_notifier_list;#endif
Signature · 1 line
1.7.0.6
RFC PATCH 30/30 rcu: improve kerneldoc for rcu_read_lock(), call_rcu(), and synchronize_rcu()
In reply to Paul E. McKenneyMake it explicit that new RCU read-side critical sections that start after call_rcu() and synchronize_rcu() start might still be running after the end of the relevant grace period.
Signed-off-by Paul E. McKenney <[email protected]>
--- include/linux/rcupdate.h | 16 +++++++++------- kernel/rcutree_plugin.h | 8 +++++--- 2 files changed, 14 insertions(+), 10 deletions(-)
include/linux/rcupdate.h +9 -7
@@ -444,7 +444,7 @@ extern int rcu_my_thread_group_empty(void);* until after the all the other CPUs exit their critical sections. * * Note, however, that RCU callbacks are permitted to run concurrently -* with RCU read-side critical sections. One way that this can happen +* with new RCU read-side critical sections. One way that this can happen * is via the following sequence of events: (1) CPU 0 enters an RCU * read-side critical section, (2) CPU 1 invokes call_rcu() to register * an RCU callback, (3) CPU 0 exits the RCU read-side critical section,
@@ -602,11 +602,13 @@ extern void wakeme_after_rcu(struct rcu_head *head);/** * call_rcu() - Queue an RCU callback for invocation after a grace period. * @head: structure to be used for queueing the RCU updates. -* @func: actual update function to be invoked after the grace period +* @func: actual callback function to be invoked after the grace period * -* The update function will be invoked some time after a full grace -* period elapses, in other words after all currently executing RCU -* read-side critical sections have completed. RCU read-side critical +* The callback function will be invoked some time after a full grace +* period elapses, in other words after all pre-existing RCU read-side +* critical sections have completed. However, the callback function +* might well execute concurrently with RCU read-side critical sections +* that started after call_rcu() was invoked. RCU read-side critical * sections are delimited by rcu_read_lock() and rcu_read_unlock(), * and may be nested. */
@@ -616,9 +618,9 @@ extern void call_rcu(struct rcu_head *head,/** * call_rcu_bh() - Queue an RCU for invocation after a quicker grace period. * @head: structure to be used for queueing the RCU updates. -* @func: actual update function to be invoked after the grace period +* @func: actual callback function to be invoked after the grace period * -* The update function will be invoked some time after a full grace +* The callback function will be invoked some time after a full grace * period elapses, in other words after all currently executing RCU * read-side critical sections have completed. call_rcu_bh() assumes * that the read-side critical sections end on completion of a softirq
kernel/rcutree_plugin.h +5 -3
@@ -546,9 +546,11 @@ EXPORT_SYMBOL_GPL(call_rcu);* * Control will return to the caller some time after a full grace * period has elapsed, in other words after all currently executing RCU -* read-side critical sections have completed. RCU read-side critical -* sections are delimited by rcu_read_lock() and rcu_read_unlock(), -* and may be nested. +* read-side critical sections have completed. Note, however, that +* upon return from synchronize_rcu(), the caller might well be executing +* concurrently with new RCU read-side critical sections that began while +* synchronize_rcu() was waiting. RCU read-side critical sections are +* delimited by rcu_read_lock() and rcu_read_unlock(), and may be nested. */ void synchronize_rcu (void ) {
Signature · 1 line
1.7.0.6
RFC PATCH 27/30 rcu: apply ->rda changes to rcutree_trace.c
In reply to Paul E. McKenneyThe print_rcu_pendings() function used the ->rda[] array, so this commit makes it instead use per_cpu_ptr().
Signed-off-by Paul E. McKenney <[email protected]>
--- kernel/rcutree_trace.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/rcutree_trace.c +1 -1
@@ -262,7 +262,7 @@ static void print_rcu_pendings(struct seq_file *m, struct rcu_state *rsp)struct rcu_data *rdp;for_each_possible_cpu (cpu) { - rdp = rsp->rda[cpu]; + rdp =per_cpu_ptr (rsp->rda, cpu);if (rdp->beenonline)print_one_rcu_pending (m, rdp); }
Signature · 1 line
1.7.0.6
RFC PATCH 23/30 rcu head remove init
In reply to Paul E. McKenneyFrom: Mathieu Desnoyers <[email protected]>
RCU heads really don't need to be initialized. Their state before call_rcu() really does not matter.
We need to keep init/destroy_rcu_head_on_stack() though, since we want debugobjects to be able to keep track of these objects.
Signed-off-by Alexey Dobriyan <[email protected]>
Signed-off-by Mathieu Desnoyers <[email protected]>
CC David S. Miller <[email protected]>
CC "Paul E. McKenney" <[email protected]>
CC Alexey Dobriyan <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
--- include/linux/rcupdate.h | 6 ------ 1 files changed, 0 insertions(+), 6 deletions(-)
include/linux/rcupdate.h +0 -6
@@ -75,12 +75,6 @@ extern void rcu_init(void);#error "Unknown RCU implementation specified to kernel configuration"#endif -#define RCU_HEAD_INIT { .next = NULL, .func = NULL } -#define RCU_HEAD (head) struct rcu_head head = RCU_HEAD_INIT -#define INIT_RCU_HEAD (ptr) do { \ - (ptr)->next = NULL; (ptr)->func = NULL; \ -} while (0) -/* * init_rcu_head_on_stack()/destroy_rcu_head_on_stack() are needed for dynamic * initialization and destruction of rcu_head on the stack. rcu_head structures
Signature · 1 line
1.7.0.6
RFC PATCH 29/30 rcu: add boot parameter to suppress RCU CPU stall warning messages
In reply to Paul E. McKenneyAlthough the RCU CPU stall warning messages are a very good way to alert people to a problem, once alerted, it is sometimes helpful to shut them off in order to avoid obscuring other messages that might be being used to track down the problem. Although you can rebuild the kernel with CONFIG_RCU_CPU_STALL_DETECTOR=n, this is sometimes inconvenient. This commit therefore adds a boot parameter named "rcu_cpu_stall_suppress" that shuts these messages off without requiring a rebuild (though a reboot might be needed for those not brave enough to patch their kernel while it is running).
This message-suppression was already in place for the panic case, so this commit need only rename the variable and export it via module_param().
Signed-off-by Paul E. McKenney <[email protected]>
--- kernel/rcutree.c | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-)
kernel/rcutree.c +8 -3
@@ -143,6 +143,11 @@ module_param(blimit, int, 0);module_param (qhimark, int,0 );module_param (qlowmark, int,0 ); +#ifdef CONFIG_RCU_CPU_STALL_DETECTOR +int rcu_cpu_stall_suppress__read_mostly ; +module_param (rcu_cpu_stall_suppress, int,0 ); +#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */ +static void force_quiescent_state (struct rcu_state *rsp,int relaxed);static int rcu_pending (int cpu);
@@ -450,7 +455,7 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)#ifdef CONFIG_RCU_CPU_STALL_DETECTOR -int rcu_cpu_stall_panicking__read_mostly ; +int rcu_cpu_stall_suppress__read_mostly ;static void record_gp_stall_check_time (struct rcu_state *rsp) {
@@ -530,7 +535,7 @@ static void check_cpu_stall(struct rcu_state *rsp, struct rcu_data *rdp)long delta;struct rcu_node *rnp; -if (rcu_cpu_stall_panicking) +if (rcu_cpu_stall_suppress)return ; delta = jiffies - rsp->jiffies_stall; rnp = rdp->mynode;
@@ -548,7 +553,7 @@ static void check_cpu_stall(struct rcu_state *rsp, struct rcu_data *rdp)static int rcu_panic (struct notifier_block *this,unsigned long ev,void *ptr) { - rcu_cpu_stall_panicking =1 ; + rcu_cpu_stall_suppress =1 ;return NOTIFY_DONE ; }
Signature · 1 line
1.7.0.6
RFC PATCH 25/30 rcutorture: add random preemption
In reply to Paul E. McKenneyFrom: Lai Jiangshan <[email protected]>
Add random preemption to help we to torture the preemptable rcu.
srcu_read_delay() also calls rcu_read_delay() for shorter delays.
Added comment to preempt_schedule() call indicating that no quiescent states happen if preemption is disabled.
Signed-off-by Lai Jiangshan <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
--- kernel/rcutorture.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
kernel/rcutorture.c +6 -0
@@ -304,6 +304,10 @@ static void rcu_read_delay(struct rcu_random_state *rrsp)mdelay (longdelay_ms);if (!(rcu_random (rrsp) % (nrealreaders *2 * shortdelay_us)))udelay (shortdelay_us); +#ifdef CONFIG_PREEMPT +if (!preempt_count () && !(rcu_random (rrsp) % (nrealreaders *20000 ))) +preempt_schedule ();/* No QS if preempt_disable() in effect */ +#endif } static void rcu_torture_read_unlock(int idx) __releases(RCU)
@@ -537,6 +541,8 @@ static void srcu_read_delay(struct rcu_random_state *rrsp) delay =rcu_random (rrsp) % (nrealreaders *2 * longdelay * uspertick);if (!delay)schedule_timeout_interruptible (longdelay); +else +rcu_read_delay (rrsp); } static void srcu_torture_read_unlock(int idx) __releases(&srcu_ctl)
Signature · 1 line
1.7.0.6
RFC PATCH 11/30 radix-tree: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Nick Piggin <[email protected]>
--- include/linux/radix-tree.h | 4 +++- lib/radix-tree.c | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-)
include/linux/radix-tree.h +3 -1
@@ -47,6 +47,8 @@ static inline void *radix_tree_indirect_to_ptr(void *ptr) {return (void *)((unsigned long )ptr & ~RADIX_TREE_INDIRECT_PTR ); } +#define radix_tree_indirect_to_ptr (ptr) \ + radix_tree_indirect_to_ptr((void __force *)(ptr))static inline int radix_tree_is_indirect_ptr (void *ptr) {
@@ -61,7 +63,7 @@ static inline int radix_tree_is_indirect_ptr(void *ptr)struct radix_tree_root {unsigned int height;gfp_t gfp_mask; -struct radix_tree_node *rnode; +struct radix_tree_node __rcu *rnode; };#define RADIX_TREE_INIT (mask) { \
lib/radix-tree.c +1 -1
@@ -49,7 +49,7 @@ struct radix_tree_node {unsigned int height;/* Height from the bottom */ unsigned int count;struct rcu_head rcu_head; -void *slots[RADIX_TREE_MAP_SIZE ]; +void __rcu *slots[RADIX_TREE_MAP_SIZE ];unsigned long tags[RADIX_TREE_MAX_TAGS ][RADIX_TREE_TAG_LONGS ]; };
Signature · 1 line
1.7.0.6
RFC PATCH 13/30 input: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Dmitry Torokhov <[email protected]>
Acked-by Dmitry Torokhov <[email protected]>
--- drivers/input/evdev.c | 2 +- include/linux/input.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/input/evdev.c +1 -1
@@ -28,7 +28,7 @@ struct evdev {int minor;struct input_handle handle;wait_queue_head_t wait; -struct evdev_client *grab; +struct evdev_client __rcu *grab;struct list_head client_list;spinlock_t client_lock;/* protects client_list */ struct mutex mutex;
include/linux/input.h +1 -1
@@ -1174,7 +1174,7 @@ struct input_dev {int (*flush)(struct input_dev *dev,struct file *file);int (*event)(struct input_dev *dev,unsigned int type,unsigned int code,int value); -struct input_handle *grab; +struct input_handle __rcu *grab;spinlock_t event_lock;struct mutex mutex;
Signature · 1 line
1.7.0.6
RFC PATCH 14/30 net/netfilter: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Acked-by Patrick McHardy <[email protected]>
Cc "David S. Miller" <[email protected]>
Cc Eric Dumazet <[email protected]>
--- include/net/netfilter/nf_conntrack.h | 2 +- net/ipv4/netfilter/nf_nat_core.c | 2 +- net/netfilter/core.c | 2 +- net/netfilter/nf_conntrack_ecache.c | 4 ++-- net/netfilter/nf_conntrack_extend.c | 2 +- net/netfilter/nf_conntrack_proto.c | 4 ++-- net/netfilter/nf_log.c | 2 +- net/netfilter/nf_queue.c | 2 +- 8 files changed, 10 insertions(+), 10 deletions(-)
include/net/netfilter/nf_conntrack.h +1 -1
@@ -75,7 +75,7 @@ struct nf_conntrack_helper;/* nf_conn feature for connections that have a helper */ struct nf_conn_help {/* Helper. if any */ -struct nf_conntrack_helper *helper; +struct nf_conntrack_helper __rcu *helper;union nf_conntrack_help help;
net/ipv4/netfilter/nf_nat_core.c +1 -1
@@ -38,7 +38,7 @@ static DEFINE_SPINLOCK(nf_nat_lock);static struct nf_conntrack_l3proto *l3proto__read_mostly ;#define MAX_IP_NAT_PROTO 256 -static const struct nf_nat_protocol *nf_nat_protos[MAX_IP_NAT_PROTO ] +static const struct nf_nat_protocol __rcu *nf_nat_protos[MAX_IP_NAT_PROTO ]__read_mostly ; static inline const struct nf_nat_protocol *
net/netfilter/core.c +1 -1
@@ -27,7 +27,7 @@static DEFINE_MUTEX (afinfo_mutex); -const struct nf_afinfo *nf_afinfo[NFPROTO_NUMPROTO ]__read_mostly ; +const struct nf_afinfo __rcu *nf_afinfo[NFPROTO_NUMPROTO ]__read_mostly ;EXPORT_SYMBOL (nf_afinfo);int nf_register_afinfo (const struct nf_afinfo *afinfo)
net/netfilter/nf_conntrack_ecache.c +2 -2
@@ -26,10 +26,10 @@static DEFINE_MUTEX (nf_ct_ecache_mutex); -struct nf_ct_event_notifier *nf_conntrack_event_cb__read_mostly ; +struct nf_ct_event_notifier __rcu *nf_conntrack_event_cb__read_mostly ;EXPORT_SYMBOL_GPL (nf_conntrack_event_cb); -struct nf_exp_event_notifier *nf_expect_event_cb__read_mostly ; +struct nf_exp_event_notifier __rcu *nf_expect_event_cb__read_mostly ;EXPORT_SYMBOL_GPL (nf_expect_event_cb);/* deliver cached events and clear cache entry - must be called with locally
net/netfilter/nf_conntrack_extend.c +1 -1
@@ -16,7 +16,7 @@#include <linux/skbuff.h> #include <net/netfilter/nf_conntrack_extend.h> -static struct nf_ct_ext_type *nf_ct_ext_types[NF_CT_EXT_NUM ]; +static struct nf_ct_ext_type __rcu *nf_ct_ext_types[NF_CT_EXT_NUM ];static DEFINE_MUTEX (nf_ct_ext_type_mutex);void __nf_ct_ext_destroy (struct nf_conn *ct)
net/netfilter/nf_conntrack_proto.c +2 -2
@@ -28,8 +28,8 @@#include <net/netfilter/nf_conntrack_l4proto.h> #include <net/netfilter/nf_conntrack_core.h> -static struct nf_conntrack_l4proto **nf_ct_protos[PF_MAX ]__read_mostly ; -struct nf_conntrack_l3proto *nf_ct_l3protos[AF_MAX ]__read_mostly ; +static struct nf_conntrack_l4proto __rcu **nf_ct_protos[PF_MAX ]__read_mostly ; +struct nf_conntrack_l3proto __rcu *nf_ct_l3protos[AF_MAX ]__read_mostly ;EXPORT_SYMBOL_GPL (nf_ct_l3protos);static DEFINE_MUTEX (nf_ct_proto_mutex);
net/netfilter/nf_log.c +1 -1
@@ -16,7 +16,7 @@#define NF_LOG_PREFIXLEN 128#define NFLOGGER_NAME_LEN 64 -static const struct nf_logger *nf_loggers[NFPROTO_NUMPROTO ]__read_mostly ; +static const struct nf_logger __rcu *nf_loggers[NFPROTO_NUMPROTO ]__read_mostly ;static struct list_head nf_loggers_l[NFPROTO_NUMPROTO ]__read_mostly ;static DEFINE_MUTEX (nf_log_mutex);
net/netfilter/nf_queue.c +1 -1
@@ -18,7 +18,7 @@* long term mutex. The handler must provide an an outfn() to accept packets * for queueing and must reinject all packets it receives, no matter what. */ -static const struct nf_queue_handler *queue_handler[NFPROTO_NUMPROTO ]__read_mostly ; +static const struct nf_queue_handler __rcu *queue_handler[NFPROTO_NUMPROTO ]__read_mostly ;static DEFINE_MUTEX (queue_handler_mutex);
Signature · 1 line
1.7.0.6
RFC PATCH 18/30 kvm: more __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Avi Kivity <[email protected]>
Cc Marcelo Tosatti <[email protected]>
--- arch/x86/include/asm/kvm_host.h | 2 +- include/linux/kvm_host.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -1
@@ -378,7 +378,7 @@ struct kvm_mem_aliases { };struct kvm_arch { -struct kvm_mem_aliases *aliases; +struct kvm_mem_aliases __rcu *aliases;unsigned int n_free_mmu_pages;unsigned int n_requested_mmu_pages;
include/linux/kvm_host.h +2 -2
@@ -175,7 +175,7 @@ struct kvm {raw_spinlock_t requests_lock;struct mutex slots_lock;struct mm_struct *mm;/* userspace tied to this vm */ -struct kvm_memslots *memslots; +struct kvm_memslots __rcu *memslots;struct srcu_struct srcu;#ifdef CONFIG_KVM_APIC_ARCHITECTURE u32 bsp_vcpu_id;
@@ -185,7 +185,7 @@ struct kvm {atomic_t online_vcpus;struct list_head vm_list;struct mutex lock; -struct kvm_io_bus *buses[KVM_NR_BUSES ]; +struct kvm_io_bus __rcu *buses[KVM_NR_BUSES ]; #ifdef CONFIG_HAVE_KVM_EVENTFD struct { spinlock_t lock;
Signature · 1 line
1.7.0.6
RFC PATCH 21/30 Update call_rcu() usage, add synchronize_rcu()
In reply to Paul E. McKenneyReported-by Kyle Hubert <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
--- Documentation/DocBook/kernel-locking.tmpl | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-)
Documentation/DocBook/kernel-locking.tmpl +4 -2
@@ -1645,7 +1645,9 @@ the amount of locking which needs to be done. all the readers who were traversing the list when we deleted the element are finished. We use <function>call_rcu()</function> to register a callback which will actually destroy the object once - the readers are finished. + all pre-existing readers are finished. Alternatively, + <function>synchronize_rcu()</function> may be used to block until + all pre-existing are finished. </para> <para> But how does Read Copy Update know when the readers are
@@ -1714,7 +1716,7 @@ the amount of locking which needs to be done. - object_put(obj); + list_del_rcu(&obj->list); cache_num--; -+ call_rcu(&obj->rcu, cache_delete_rcu, obj); ++ call_rcu(&obj->rcu, cache_delete_rcu); } /* Must be holding cache_lock */
Signature · 1 line
1.7.0.6
RFC PATCH 24/30 rcu: add shiny new debug assists to Documentation/RCU/checklist.txt
In reply to Paul E. McKenneyAdd a section describing PROVE_RCU, DEBUG_OBJECTS_RCU_HEAD, and the __rcu sparse checking to the RCU checklist.
Suggested-by David Miller <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
--- Documentation/RCU/checklist.txt | 23 +++++++++++++++++++++++ 1 files changed, 23 insertions(+), 0 deletions(-)
Documentation/RCU/checklist.txt +23 -0
@@ -365,3 +365,26 @@ over a rather long period of time, but improvements are always welcome! and the compiler to freely reorder code into and out of RCU read-side critical sections. It is the responsibility of the RCU update-side primitives to deal with this. + +17. Use CONFIG_PROVE_RCU, CONFIG_DEBUG_OBJECTS_RCU_HEAD, and + the __rcu sparse checks to validate your RCU code. These + can help find problems as follows: + + CONFIG_PROVE_RCU: check that accesses to RCU-protected data + structures are carried out under the proper RCU + read-side critical section, while holding the right + combination of locks, or whatever other conditions + are appropriate. + + CONFIG_DEBUG_OBJECTS_RCU_HEAD: check that you don't pass the + same object to call_rcu() (or friends) before an RCU + grace period has elapsed since the last time that you + passed that same object to call_rcu() (or friends). + + __rcu sparse checks: tag the pointer to the RCU-protected data + structure with __rcu, and sparse will warn you if you + access that pointer without the services of one of the + variants of rcu_dereference(). + + These debugging aids can help you find problems that are + otherwise extremely difficult to spot.
Signature · 1 line
1.7.0.6
RFC PATCH 1/30 net: convert to rcu_dereference_index_check()
In reply to Paul E. McKenneyThe task_cls_classid() function applies rcu_dereference() to integers, which does not work with the shiny new sparse-based checking in rcu_dereference(). This commit therefore moves to the new RCU API rcu_dereference_index_check().
Signed-off-by Paul E. McKenney <[email protected]>
--- include/net/cls_cgroup.h | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
include/net/cls_cgroup.h +2 -1
@@ -45,7 +45,8 @@ static inline u32 task_cls_classid(struct task_struct *p)return 0 ;rcu_read_lock (); - id =rcu_dereference (net_cls_subsys_id); + id =rcu_dereference_index_check (net_cls_subsys_id, +rcu_read_lock_held ());if (id >=0 ) classid =container_of (task_subsys_state (p, id), struct cgroup_cls_state, css)->classid;
Signature · 1 line
1.7.0.6
RFC PATCH 10/30 notifiers: __rcu annotations
In reply to Paul E. McKenneyFrom: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
Cc Alan Cox <[email protected]>
--- include/linux/notifier.h | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-)
include/linux/notifier.h +5 -5
@@ -49,28 +49,28 @@struct notifier_block {int (*notifier_call)(struct notifier_block *,unsigned long ,void *); -struct notifier_block *next; +struct notifier_block __rcu *next;int priority; };struct atomic_notifier_head {spinlock_t lock; -struct notifier_block *head; +struct notifier_block __rcu *head; };struct blocking_notifier_head {struct rw_semaphore rwsem; -struct notifier_block *head; +struct notifier_block __rcu *head; };struct raw_notifier_head { -struct notifier_block *head; +struct notifier_block __rcu *head; };struct srcu_notifier_head {struct mutex mutex;struct srcu_struct srcu; -struct notifier_block *head; +struct notifier_block __rcu *head; };#define ATOMIC_INIT_NOTIFIER_HEAD (name) do { \
Signature · 1 line
1.7.0.6
Josh Triplett
maintainerRe: [PATCH RFC tip/core/rcu 27/30] rcu: apply ->rda changes to rcutree_trace.c
In reply to Paul E. McKenneyQuoted 13 lines · Paul E. McKenney
The
print_rcu_pendings()function used the ->rda[] array, so this commit makes it instead useper_cpu_ptr().Signed-off-by Paul E. McKenney <[email protected]>--- kernel/rcutree_trace.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)kernel/rcutree_trace.c +0
-0
kernel/rcutree_trace.c static void print_rcu_pendings(struct seq_file *m, struct rcu_state *rsp) · line 268
Quoted 8 lines
kernel/rcutree_trace.c +1
-1@@ -262,7 +262,7 @@ static void print_rcu_pendings(struct seq_file *m, struct rcu_state *rsp)struct rcu_data *rdp;for_each_possible_cpu (cpu) { - rdp = rsp->rda[cpu]; + rdp =per_cpu_ptr (rsp->rda, cpu);if (rdp->beenonline)print_one_rcu_pending (m, rdp); }
For the final version of this patchset, this should get squashed into the previous patch, right? Otherwise, the state after the previous patch but before this one won't compile.
- Josh Triplett
Re: [PATCH RFC tip/core/rcu 27/30] rcu: apply ->rda changes to rcutree_trace.c
Quoted 16 lines · Josh Triplett
Quoted 15 lines · Paul E. McKenney
The
print_rcu_pendings()function used the ->rda[] array, so this commit makes it instead useper_cpu_ptr().Signed-off-by Paul E. McKenney <[email protected]>--- kernel/rcutree_trace.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)kernel/rcutree_trace.c +0
-0@@ -262,7 +262,7 @@ static void print_rcu_pendings(struct seq_file *m, struct rcu_state *rsp)struct rcu_data *rdp;
Quoted 10 lines
kernel/rcutree_trace.c static void print_rcu_pendings(struct seq_file *m, struct rcu_state *rsp) · line 268
Quoted 6 lines
kernel/rcutree_trace.c +1
-1for_each_possible_cpu (cpu) { - rdp = rsp->rda[cpu]; + rdp =per_cpu_ptr (rsp->rda, cpu);if (rdp->beenonline)print_one_rcu_pending (m, rdp); }For the final version of this patchset, this should get squashed into the previous patch, right? Otherwise, the state after the previous patch but before this one won't compile.
Makes sense, given that the earlier patch has not yet gone upstream.
Thanx, Paul
Quoted 23 lines · Paul E. McKenney
This patchset shows the patches queued for 2.6.36/7. These are not yet ready to move upstream due to dependencies on commits moving up other trees. The patches are as follows:
1-19. These are the remaining patches implementing Arnd's sparse-checking
work. #1 needs to go up the networking tree, but will cause regressions untilf5155b33277("rcu: add anrcu_dereference_index_check()") reaches mainline. #2 depends on #1, and many of the later patches (#3-#30) depend on #2. Many of #3-#19 will go up non-tip trees.
20. UpdateDocumentation/RCUto remove the now-deprecatedrcu_head
initialization macros.
21. Fix long-standing DocBook example showing long-dead three-argument
variant ofcall_rcu().
22. Make the CPU stall warning timeout configurable at build time
with newRCU_CPU_STALL_TIMEOUTconfig parameter.
23. Remove the now-deprecatedrcu_headinitialization macros.
This depends on a couple of patches removing their use that are making their way up other maintainer trees.
24. Document the new debug assists from Arnd, Mathieu, and myself.
25. Add random preemption to rcutorture testing of preemptible
RCU, courtesy of Lai Jiangshan.
26. Remove TREE_RCU's ->rda[] array in favor of the new__percpu
Quoted 8 lines
facility, also courtesy of Lai Jiangshan.
27. Fixlet to #26 for RCU tracing.
28. Rename__do_rcu_dereference_check()torcu_lockdep_assert(),
given that Tetsuo Handa has found outside-of-RCU use for this function.
29. Add a boot parameter to suppress RCU CPU stall warnings.
30. Update kerneldoc orrcu_read_lock(),call_rcu(), and
synchronize_rcu().
I replied to patches 27 and 29 with suggested changes. For all the rest:
Reviewed-by Josh Triplett <[email protected]>
- Josh Triplett
Josh Triplett
maintainerRe: [PATCH RFC tip/core/rcu 29/30] rcu: add boot parameter to suppress RCU CPU stall warning messages
In reply to Paul E. McKenneyQuoted 23 lines · Paul E. McKenney
Although the RCU CPU stall warning messages are a very good way to alert people to a problem, once alerted, it is sometimes helpful to shut them off in order to avoid obscuring other messages that might be being used to track down the problem. Although you can rebuild the kernel with
CONFIG_RCU_CPU_STALL_DETECTOR=n, this is sometimes inconvenient. This commit therefore adds a boot parameter named "rcu_cpu_stall_suppress" that shuts these messages off without requiring a rebuild (though a reboot might be needed for those not brave enough to patch their kernel while it is running).This message-suppression was already in place for the panic case, so this commit need only rename the variable and export it via
module_param().Signed-off-by Paul E. McKenney <[email protected]>--- kernel/rcutree.c | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-)kernel/rcutree.c +0
-0
kernel/rcutree.c module_param(blimit, int, 0); · line 149
Quoted 7 lines
kernel/rcutree.c +4
-0@@ -143,6 +143,11 @@ module_param(blimit, int, 0);module_param (qhimark, int,0 );module_param (qlowmark, int,0 ); +#ifdef CONFIG_RCU_CPU_STALL_DETECTOR +int rcu_cpu_stall_suppress__read_mostly ; +module_param (rcu_cpu_stall_suppress, int,0 ); +#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */
In theory, you could change the permissions on this module_param from 0 to 0644, making it appear under /sys as a writable file. That would allow changing it at runtime rather than just at boot time. Looking at the use of this variable, changing it dynamically should work.
- Josh Triplett
Re: [PATCH RFC tip/core/rcu 07/30] nfs: __rcu annotations
In reply to Paul E. McKenneyQuoted 5 lines · Paul E. McKenney
From: Arnd Bergmann <[email protected]>
Signed-off-by Arnd Bergmann <[email protected]>Signed-off-by Paul E. McKenney <[email protected]>Cc Trond Myklebust <[email protected]>
Acked-by Trond Myklebust <[email protected]>
Quoted 40 lines
--- include/linux/nfs_fs.h | 2 +- include/linux/sunrpc/auth_gss.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)include/linux/nfs_fs.h +1
-1@@ -178,7 +178,7 @@ struct nfs_inode {struct nfs4_cached_acl *nfs4_acl;/* NFSv4 state */ struct list_head open_states; -struct nfs_delegation *delegation; +struct nfs_delegation __rcu *delegation;fmode_t delegation_state;struct rw_semaphore rwsem;#endif /* CONFIG_NFS_V4*/ include/linux/sunrpc/auth_gss.h +2
-2@@ -69,7 +69,7 @@ struct gss_cl_ctx {enum rpc_gss_proc gc_proc;u32 gc_seq;spinlock_t gc_seq_lock; -struct gss_ctx *gc_gss_ctx; +struct gss_ctx __rcu *gc_gss_ctx;struct xdr_netobj gc_wire_ctx;u32 gc_win;unsigned long gc_expiry;@@ -80,7 +80,7 @@ struct gss_upcall_msg;struct gss_cred {struct rpc_cred gc_base;enum rpc_gss_svc gc_service; -struct gss_cl_ctx *gc_ctx; +struct gss_cl_ctx __rcu *gc_ctx;struct gss_upcall_msg *gc_upcall;unsigned long gc_upcall_timestamp;unsigned char gc_machine_cred :1 ;
Quoted 33 lines · Josh Triplett
Quoted 23 lines · Paul E. McKenney
This patchset shows the patches queued for 2.6.36/7. These are not yet ready to move upstream due to dependencies on commits moving up other trees. The patches are as follows:
1-19. These are the remaining patches implementing Arnd's sparse-checking
work. #1 needs to go up the networking tree, but will cause regressions untilf5155b33277("rcu: add anrcu_dereference_index_check()") reaches mainline. #2 depends on #1, and many of the later patches (#3-#30) depend on #2. Many of #3-#19 will go up non-tip trees.
20. UpdateDocumentation/RCUto remove the now-deprecatedrcu_head
initialization macros.
21. Fix long-standing DocBook example showing long-dead three-argument
variant ofcall_rcu().
22. Make the CPU stall warning timeout configurable at build time
with newRCU_CPU_STALL_TIMEOUTconfig parameter.
23. Remove the now-deprecatedrcu_headinitialization macros.
This depends on a couple of patches removing their use that are making their way up other maintainer trees.
24. Document the new debug assists from Arnd, Mathieu, and myself.
25. Add random preemption to rcutorture testing of preemptible
RCU, courtesy of Lai Jiangshan.
26. Remove TREE_RCU's ->rda[] array in favor of the new__percpuQuoted 8 lines
facility, also courtesy of Lai Jiangshan.
27. Fixlet to #26 for RCU tracing.
28. Rename__do_rcu_dereference_check()torcu_lockdep_assert(),
given that Tetsuo Handa has found outside-of-RCU use for this function.
29. Add a boot parameter to suppress RCU CPU stall warnings.
30. Update kerneldoc orrcu_read_lock(),call_rcu(), and
synchronize_rcu().
Quoted 4 lines
I replied to patches 27 and 29 with suggested changes. For all the rest:
Reviewed-by Josh Triplett <[email protected]>
Thank you very much for looking this over!
Thanx, Paul
Re: [PATCH RFC tip/core/rcu 29/30] rcu: add boot parameter to suppress RCU CPU stall warning messages
In reply to Josh TriplettQuoted 27 lines · Josh Triplett
Quoted 26 lines · Paul E. McKenney
Although the RCU CPU stall warning messages are a very good way to alert people to a problem, once alerted, it is sometimes helpful to shut them off in order to avoid obscuring other messages that might be being used to track down the problem. Although you can rebuild the kernel with
CONFIG_RCU_CPU_STALL_DETECTOR=n, this is sometimes inconvenient. This commit therefore adds a boot parameter named "rcu_cpu_stall_suppress" that shuts these messages off without requiring a rebuild (though a reboot might be needed for those not brave enough to patch their kernel while it is running).This message-suppression was already in place for the panic case, so this commit need only rename the variable and export it via
module_param().Signed-off-by Paul E. McKenney <[email protected]>--- kernel/rcutree.c | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-)kernel/rcutree.c +0
-0@@ -143,6 +143,11 @@ module_param(blimit, int, 0);module_param (qhimark, int,0 );module_param (qlowmark, int,0 );
Quoted 9 lines
kernel/rcutree.c module_param(blimit, int, 0); · line 149
Quoted 4 lines
kernel/rcutree.c +4
-0+#ifdef CONFIG_RCU_CPU_STALL_DETECTOR +int rcu_cpu_stall_suppress__read_mostly ; +module_param (rcu_cpu_stall_suppress, int,0 ); +#endif /* #ifdef CONFIG_RCU_CPU_STALL_DETECTOR */ In theory, you could change the permissions on this
module_paramfrom 0 to 0644, making it appear under /sys as a writable file. That would allow changing it at runtime rather than just at boot time. Looking at the use of this variable, changing it dynamically should work.
Will give it a try!
Thanx, Paul