Skip to contentSkip to conversation

Miscellaneous fixes for v5.3

PATCH In review Paul E. McKenney 2 participants 12 messages May 30, 2019 – Jun 13, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover Miscellaneous fixes for v5.3

Hello!

This series provides miscellaneous fixes:

1. Dump specified number of blocked tasks rather than rely on
serendipity, courtesy of Neeraj Upadhyay.

2. Correctly unlock root node in rcu_check_gp_start_stall(),
courtesy of Neeraj Upadhyay.

3. Make kfree_rcu() ignore NULL pointers.

4. Set a maximum limit for back-to-back callback invocation.

5. Remove ".vnet" from paulmck email addresses.

6. Upgrade sync_exp_work_done() to smp_mb().

7. Fix irritating whitespace error in rcu_assign_pointer().

8. Force inlining of rcu_read_lock(), courtesy of Waiman Long.

9. Don't return a value from rcu_assign_pointer(), courtesy of
Andrea Parri.

Thanx, Paul

------------------------------------------------------------------------

 Documentation/RCU/whatisRCU.txt                      |    8 +++----
 Documentation/core-api/circular-buffers.rst          |    2 -
 Documentation/memory-barriers.txt                    |    2 -
 Documentation/translations/ko_KR/memory-barriers.txt |    2 -
 include/linux/rcupdate.h                             |   21 ++++++++++---------
 kernel/rcu/tree.c                                    |    7 +++---
 kernel/rcu/tree_exp.h                                |    3 --
 kernel/rcu/tree_plugin.h                             |    2 -
 kernel/rcu/tree_stall.h                              |    4 ++-
 tools/include/linux/rcu.h                            |    4 +--
 tools/testing/radix-tree/linux/rcupdate.h            |    2 -
 11 files changed, 31 insertions(+), 26 deletions(-)
#1 lore raw

PATCH 1/9 rcu: Dump specified number of blocked tasks

From: Neeraj Upadhyay <[email protected]>

The dump_blkd_tasks() function dumps at most 10 blocked tasks, ignoring the value of the ncheck parameter. This commit therefore substitutes the value of ncheck for the hard-coded value of 10. Because all callers currently pass 10 as the number, this patch does not change behavior, but it is clearly an accident waiting to happen.

---
 kernel/rcu/tree_plugin.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/tree_plugin.h +1 -1
index 1102765f91fd..3a9891a74ead 100644
@@ -760,7 +760,7 @@ dump_blkd_tasks(struct rcu_node *rnp, int ncheck)
 	i = 0;
 	list_for_each(lhp, &rnp->blkd_tasks) {
 		pr_cont(" %p", lhp);
-		if (++i >= 10)
+		if (++i >= ncheck)
 			break;
 	}
 	pr_cont("\n");
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/9 rcu: Correctly unlock root node in rcu_check_gp_start_stall()

In reply to Paul E. McKenney

From: Neeraj Upadhyay <[email protected]>

On systems whose rcu_node tree has only one node, the rcu_check_gp_start_stall() function's values of rnp and rnp_root will be identical. In this case, it clearly does not make sense to release both rnp->lock and rnp_root->lock, but that is exactly what this function does in the last early exit. This commit therefore unlocks only rnp->lock when rnp and rnp_root are equal.

---
 kernel/rcu/tree_stall.h | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/rcu/tree_stall.h +3 -1
index f65a73a97323..065183391f75 100644
@@ -630,7 +630,9 @@ static void rcu_check_gp_start_stall(struct rcu_node *rnp, struct rcu_data *rdp,
 	    time_before(j, rcu_state.gp_req_activity + gpssdelay) ||
 	    time_before(j, rcu_state.gp_activity + gpssdelay) ||
 	    atomic_xchg(&warned, 1)) {
-		raw_spin_unlock_rcu_node(rnp_root); /* irqs remain disabled. */
+		if (rnp_root != rnp)
+			/* irqs remain disabled. */
+			raw_spin_unlock_rcu_node(rnp_root);
 		raw_spin_unlock_irqrestore_rcu_node(rnp, flags);
 		return;
 	}
Signature · 1 line
2.17.1
#3 lore raw

PATCH 3/9 rcu: Make kfree_rcu() ignore NULL pointers

In reply to Paul E. McKenney

This commit makes the kfree_rcu() macro's semantics be consistent with the likes of kfree() by adding a check for NULL pointers, so that kfree_rcu(NULL, ...) is a no-op.

---
 include/linux/rcupdate.h | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
include/linux/rcupdate.h +8 -4
index 922bb6848813..915460ec0872 100644
@@ -805,7 +805,7 @@ static inline notrace void rcu_read_unlock_sched_notrace(void)
 /**
  * kfree_rcu() - kfree an object after a grace period.
  * @ptr:	pointer to kfree
- * @rcu_head:	the name of the struct rcu_head within the type of @ptr.
+ * @rhf:	the name of the struct rcu_head within the type of @ptr.
  *
  * Many rcu callbacks functions just call kfree() on the base structure.
  * These functions are trivial, but their size adds up, and furthermore
@@ -828,9 +828,13 @@ static inline notrace void rcu_read_unlock_sched_notrace(void)
  * The BUILD_BUG_ON check must not involve any function calls, hence the
  * checks are done in macros here.
  */
-#define kfree_rcu(ptr, rcu_head)					\
-	__kfree_rcu(&((ptr)->rcu_head), offsetof(typeof(*(ptr)), rcu_head))
-
+#define kfree_rcu(ptr, rhf)						\
+do {									\
+	typeof (ptr) ___p = (ptr);					\
+									\
+	if (___p)							\
+		__kfree_rcu(&((___p)->rhf), offsetof(typeof(*(ptr)), rhf)); \
+} while (0)
 
 /*
  * Place this after a lock-acquisition primitive to guarantee that
Signature · 1 line
2.17.1
#4 lore raw

PATCH 4/9 rcu: Set a maximum limit for back-to-back callback invocation

In reply to Paul E. McKenney

Currently, if a CPU has more than 10,000 callbacks pending, it will increase rdp->blimit to LONG_MAX. If you are lucky, LONG_MAX is only about two billion, but this is still a bit too many callbacks to invoke back-to-back while otherwise ignoring the world.

This commit therefore sets a maximum limit of DEFAULT_MAX_RCU_BLIMIT, which is set to 10,000, for rdp->blimit.

---
 kernel/rcu/tree.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
kernel/rcu/tree.c +4 -3
index 980ca3ca643f..f888a76673da 100644
@@ -380,7 +380,8 @@ static int rcu_is_cpu_rrupt_from_idle(void)
 	       __this_cpu_read(rcu_data.dynticks_nmi_nesting) <= 1;
 }
 
-#define DEFAULT_RCU_BLIMIT 10     /* Maximum callbacks per rcu_do_batch. */
+#define DEFAULT_RCU_BLIMIT 10     /* Maximum callbacks per rcu_do_batch ... */
+#define DEFAULT_MAX_RCU_BLIMIT 10000 /* ... even during callback flood. */
 static long blimit = DEFAULT_RCU_BLIMIT;
 #define DEFAULT_RCU_QHIMARK 10000 /* If this many pending, ignore blimit. */
 static long qhimark = DEFAULT_RCU_QHIMARK;
@@ -2113,7 +2114,7 @@ static void rcu_do_batch(struct rcu_data *rdp)
 
 	/* Reinstate batch limit if we have worked down the excess. */
 	count = rcu_segcblist_n_cbs(&rdp->cblist);
-	if (rdp->blimit == LONG_MAX && count <= qlowmark)
+	if (rdp->blimit >= DEFAULT_MAX_RCU_BLIMIT && count <= qlowmark)
 		rdp->blimit = blimit;
 
 	/* Reset ->qlen_last_fqs_check trigger if enough CBs have drained. */
@@ -2354,7 +2355,7 @@ static void __call_rcu_core(struct rcu_data *rdp, struct rcu_head *head,
 			rcu_accelerate_cbs_unlocked(rdp->mynode, rdp);
 		} else {
 			/* Give the grace period a kick. */
-			rdp->blimit = LONG_MAX;
+			rdp->blimit = DEFAULT_MAX_RCU_BLIMIT;
 			if (rcu_state.n_force_qs == rdp->n_force_qs_snap &&
 			    rcu_segcblist_first_pend_cb(&rdp->cblist) != head)
 				rcu_force_quiescent_state();
Signature · 1 line
2.17.1
#5 lore raw

PATCH 5/9 doc: Remove ".vnet" from paulmck email addresses

In reply to Paul E. McKenney
---
 Documentation/core-api/circular-buffers.rst          | 2 +-
 Documentation/memory-barriers.txt                    | 2 +-
 Documentation/translations/ko_KR/memory-barriers.txt | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
Documentation/core-api/circular-buffers.rst +1 -1
index 53e51caa3347..50966f66e398 100644
@@ -3,7 +3,7 @@ Circular Buffers
 ================
 
 :Author: David Howells <[email protected]>
-:Author: Paul E. McKenney <[email protected]>
+:Author: Paul E. McKenney <[email protected]>
 
 
 Linux provides a number of features that can be used to implement circular
Documentation/memory-barriers.txt +1 -1
index f70ebcdfe592..e4e07c8ab89e 100644
@@ -3,7 +3,7 @@
 			 ============================
 
 By: David Howells <[email protected]>
-    Paul E. McKenney <[email protected]>
+    Paul E. McKenney <[email protected]>
     Will Deacon <[email protected]>
     Peter Zijlstra <[email protected]>
 
Documentation/translations/ko_KR/memory-barriers.txt +1 -1
index db0b9d8619f1..5f3c74dcad43 100644
@@ -24,7 +24,7 @@ Documentation/memory-barriers.txt
 			 =========================
 
 저자: David Howells <[email protected]>
-      Paul E. McKenney <[email protected]>
+      Paul E. McKenney <[email protected]>
       Will Deacon <[email protected]>
       Peter Zijlstra <[email protected]>
 
Signature · 1 line
2.17.1
#6 lore raw

PATCH 6/9 rcu: Upgrade sync_exp_work_done() to smp_mb()

In reply to Paul E. McKenney

The sync_exp_work_done() function uses smp_mb__before_atomic(), but there is no obvious atomic in the ensuing code. The ordering is absolutely required for grace periods to work correctly, so this commit upgrades the smp_mb__before_atomic() to smp_mb().

---
 kernel/rcu/tree_exp.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
kernel/rcu/tree_exp.h +1 -2
index 9c990df880d1..d969650a72c6 100644
@@ -259,8 +259,7 @@ static bool sync_exp_work_done(unsigned long s)
 {
 	if (rcu_exp_gp_seq_done(s)) {
 		trace_rcu_exp_grace_period(rcu_state.name, s, TPS("done"));
-		/* Ensure test happens before caller kfree(). */
-		smp_mb__before_atomic(); /* ^^^ */
+		smp_mb(); /* Ensure test happens before caller kfree(). */
 		return true;
 	}
 	return false;
Signature · 1 line
2.17.1
maintainer
#7 lore raw

Re: [PATCH tip/core/rcu 6/9] rcu: Upgrade sync_exp_work_done() to smp_mb()

Quoted 5 lines · Paul E. McKenney

The sync_exp_work_done() function uses smp_mb__before_atomic(), but there is no obvious atomic in the ensuing code. The ordering is absolutely required for grace periods to work correctly, so this commit upgrades the smp_mb__before_atomic() to smp_mb().

Did this commit want a Fixes: line? Such that robots can find the right kernels to backport this to?

Quoted 23 lines
---
 kernel/rcu/tree_exp.h | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
kernel/rcu/tree_exp.h +1 -2
index 9c990df880d1..d969650a72c6 100644
@@ -259,8 +259,7 @@ static bool sync_exp_work_done(unsigned long s)
 {
 	if (rcu_exp_gp_seq_done(s)) {
 		trace_rcu_exp_grace_period(rcu_state.name, s, TPS("done"));
-		/* Ensure test happens before caller kfree(). */
-		smp_mb__before_atomic(); /* ^^^ */
+		smp_mb(); /* Ensure test happens before caller kfree(). */
 		return true;
 	}
 	return false;
Signature · 1 line
2.17.1
#8 lore raw

Re: [PATCH tip/core/rcu 6/9] rcu: Upgrade sync_exp_work_done() to smp_mb()

Quoted 9 lines · Peter Zijlstra
Quoted 5 lines · Paul E. McKenney

The sync_exp_work_done() function uses smp_mb__before_atomic(), but there is no obvious atomic in the ensuing code. The ordering is absolutely required for grace periods to work correctly, so this commit upgrades the smp_mb__before_atomic() to smp_mb().

Did this commit want a Fixes: line? Such that robots can find the right kernels to backport this to?

Indeed it does, and thanks to Andrea for finding the commit.

Thanx, Paul

------------------------------------------------------------------------

commit 96050c68be33edef18800ad6748f61f81db81a20

Author: Paul E. McKenney <[email protected]>
Date:   Sat Apr 20 01:40:54 2019 -0700

rcu: Upgrade sync_exp_work_done() to smp_mb()

The sync_exp_work_done() function uses smp_mb__before_atomic(), but there is no obvious atomic in the ensuing code. The ordering is absolutely required for grace periods to work correctly, so this commit upgrades the smp_mb__before_atomic() to smp_mb().

Fixes 6fba2b3767ea ("rcu: Remove deprecated RCU debugfs tracing code")
kernel/rcu/tree_exp.h +1 -2
index 9c990df880d1..d969650a72c6 100644
@@ -259,8 +259,7 @@ static bool sync_exp_work_done(unsigned long s)
 {
 	if (rcu_exp_gp_seq_done(s)) {
 		trace_rcu_exp_grace_period(rcu_state.name, s, TPS("done"));
-		/* Ensure test happens before caller kfree(). */
-		smp_mb__before_atomic(); /* ^^^ */
+		smp_mb(); /* Ensure test happens before caller kfree(). */
 		return true;
 	}
 	return false;
#9 lore raw

PATCH 7/9 rcu: Fix irritating whitespace error in rcu_assign_pointer()

In reply to Paul E. McKenney
---
 include/linux/rcupdate.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/rcupdate.h +1 -1
index 915460ec0872..534c05d529b5 100644
@@ -369,7 +369,7 @@ static inline void rcu_preempt_sleep_check(void) { }
 #define rcu_assign_pointer(p, v)					      \
 ({									      \
 	uintptr_t _r_a_p__v = (uintptr_t)(v);				      \
-	rcu_check_sparse(p, __rcu);				      \
+	rcu_check_sparse(p, __rcu);					      \
 									      \
 	if (__builtin_constant_p(v) && (_r_a_p__v) == (uintptr_t)NULL)	      \
 		WRITE_ONCE((p), (typeof(p))(_r_a_p__v));		      \
Signature · 1 line
2.17.1
#10 lore raw

PATCH 8/9 rcu: Force inlining of rcu_read_lock()

In reply to Paul E. McKenney

From: Waiman Long <[email protected]>

When debugging options are turned on, the rcu_read_lock() function might not be inlined. This results in lockdep's print_lock() function printing "rcu_read_lock+0x0/0x70" instead of rcu_read_lock()'s caller. For example:

[   10.579995] =============================
[   10.584033] WARNING: suspicious RCU usage
[   10.588074] 4.18.0.memcg_v2+ #1 Not tainted
[   10.593162] -----------------------------
[   10.597203] include/linux/rcupdate.h:281 Illegal context switch in
RCU read-side critical section!
[   10.606220]
[   10.606220] other info that might help us debug this:
[   10.606220]
[   10.614280]
[   10.614280] rcu_scheduler_active = 2, debug_locks = 1
[   10.620853] 3 locks held by systemd/1:
[   10.624632]  #0: (____ptrval____) (&type->i_mutex_dir_key#5){.+.+}, at: lookup_slow+0x42/0x70
[   10.633232]  #1: (____ptrval____) (rcu_read_lock){....}, at: rcu_read_lock+0x0/0x70
[   10.640954]  #2: (____ptrval____) (rcu_read_lock){....}, at: rcu_read_lock+0x0/0x70

These "rcu_read_lock+0x0/0x70" strings are not providing any useful information. This commit therefore forces inlining of the rcu_read_lock() function so that rcu_read_lock()'s caller is instead shown.

---
 include/linux/rcupdate.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/rcupdate.h +1 -1
index 534c05d529b5..a8ed624da555 100644
@@ -588,7 +588,7 @@ static inline void rcu_preempt_sleep_check(void) { }
  * read-side critical sections may be preempted and they may also block, but
  * only when acquiring spinlocks that are subject to priority inheritance.
  */
-static inline void rcu_read_lock(void)
+static __always_inline void rcu_read_lock(void)
 {
 	__rcu_read_lock();
 	__acquire(RCU);
Signature · 1 line
2.17.1
#11 lore raw

PATCH 9/9 rcu: Don't return a value from rcu_assign_pointer()

In reply to Paul E. McKenney

From: Andrea Parri <[email protected]>

Quoting Paul [1]:

"Given that a quick (and perhaps error-prone) search of the uses
of rcu_assign_pointer() in v5.1 didn't find a single use of the return value, let's please instead change the documentation and implementation to eliminate the return value."

[1] https://lkml.kernel.org/r/[email protected]

Cc "Paul E. McKenney" <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc Lai Jiangshan <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Will Deacon <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Matthew Wilcox <[email protected]>
Cc Sasha Levin <[email protected]>
---
 Documentation/RCU/whatisRCU.txt           | 8 ++++----
 include/linux/rcupdate.h                  | 5 ++---
 tools/include/linux/rcu.h                 | 4 ++--
 tools/testing/radix-tree/linux/rcupdate.h | 2 +-
 4 files changed, 9 insertions(+), 10 deletions(-)
Documentation/RCU/whatisRCU.txt +4 -4
index 981651a8b65d..7e1a8721637a 100644
@@ -212,7 +212,7 @@ synchronize_rcu()
 
 rcu_assign_pointer()
 
-	typeof(p) rcu_assign_pointer(p, typeof(p) v);
+	void rcu_assign_pointer(p, typeof(p) v);
 
 	Yes, rcu_assign_pointer() -is- implemented as a macro, though it
 	would be cool to be able to declare a function in this manner.
@@ -220,9 +220,9 @@ rcu_assign_pointer()
 
 	The updater uses this function to assign a new value to an
 	RCU-protected pointer, in order to safely communicate the change
-	in value from the updater to the reader.  This function returns
-	the new value, and also executes any memory-barrier instructions
-	required for a given CPU architecture.
+	in value from the updater to the reader.  This macro does not
+	evaluate to an rvalue, but it does execute any memory-barrier
+	instructions required for a given CPU architecture.
 
 	Perhaps just as important, it serves to document (1) which
 	pointers are protected by RCU and (2) the point at which a
include/linux/rcupdate.h +2 -3
index a8ed624da555..0c9b92799abc 100644
@@ -367,7 +367,7 @@ static inline void rcu_preempt_sleep_check(void) { }
  * other macros that it invokes.
  */
 #define rcu_assign_pointer(p, v)					      \
-({									      \
+do {									      \
 	uintptr_t _r_a_p__v = (uintptr_t)(v);				      \
 	rcu_check_sparse(p, __rcu);					      \
 									      \
@@ -375,8 +375,7 @@ static inline void rcu_preempt_sleep_check(void) { }
 		WRITE_ONCE((p), (typeof(p))(_r_a_p__v));		      \
 	else								      \
 		smp_store_release(&p, RCU_INITIALIZER((typeof(p))_r_a_p__v)); \
-	_r_a_p__v;							      \
-})
+} while (0)
 
 /**
  * rcu_swap_protected() - swap an RCU and a regular pointer
tools/include/linux/rcu.h +2 -2
index 7d02527e5bce..9554d3fa54f3 100644
@@ -19,7 +19,7 @@ static inline bool rcu_is_watching(void)
 	return false;
 }
 
-#define rcu_assign_pointer(p, v) ((p) = (v))
-#define RCU_INIT_POINTER(p, v) p=(v)
+#define rcu_assign_pointer(p, v)	do { (p) = (v); } while (0)
+#define RCU_INIT_POINTER(p, v)	do { (p) = (v); } while (0)
 
 #endif
tools/testing/radix-tree/linux/rcupdate.h +1 -1
index fd280b070fdb..fed468fb0c78 100644
@@ -7,6 +7,6 @@
 #define rcu_dereference_raw(p) rcu_dereference(p)
 #define rcu_dereference_protected(p, cond) rcu_dereference(p)
 #define rcu_dereference_check(p, cond) rcu_dereference(p)
-#define RCU_INIT_POINTER(p, v)	(p) = (v)
+#define RCU_INIT_POINTER(p, v)	do { (p) = (v); } while (0)
 
 #endif
Signature · 1 line
2.17.1