Miscellaneous fixes for v5.3
Subsystems (4)
cover Miscellaneous fixes for v5.3
Hello!
This series provides miscellaneous fixes:
1. Dump specified number of blocked tasks rather than rely on
serendipity, courtesy of Neeraj Upadhyay.
2. Correctly unlock root node in rcu_check_gp_start_stall(),
courtesy of Neeraj Upadhyay.
3. Make kfree_rcu() ignore NULL pointers.
4. Set a maximum limit for back-to-back callback invocation.
5. Remove ".vnet" from paulmck email addresses.
6. Upgrade sync_exp_work_done() to smp_mb().
7. Fix irritating whitespace error in rcu_assign_pointer().
8. Force inlining of rcu_read_lock(), courtesy of Waiman Long.
9. Don't return a value from rcu_assign_pointer(), courtesy of
Andrea Parri.
Thanx, Paul
------------------------------------------------------------------------
Documentation/RCU/whatisRCU.txt | 8 +++---- Documentation/core-api/circular-buffers.rst | 2 - Documentation/memory-barriers.txt | 2 - Documentation/translations/ko_KR/memory-barriers.txt | 2 - include/linux/rcupdate.h | 21 ++++++++++--------- kernel/rcu/tree.c | 7 +++--- kernel/rcu/tree_exp.h | 3 -- kernel/rcu/tree_plugin.h | 2 - kernel/rcu/tree_stall.h | 4 ++- tools/include/linux/rcu.h | 4 +-- tools/testing/radix-tree/linux/rcupdate.h | 2 - 11 files changed, 31 insertions(+), 26 deletions(-)
PATCH 3/9 rcu: Make kfree_rcu() ignore NULL pointers
This commit makes the kfree_rcu() macro's semantics be consistent with the likes of kfree() by adding a check for NULL pointers, so that kfree_rcu(NULL, ...) is a no-op.
--- include/linux/rcupdate.h | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
include/linux/rcupdate.h +8 -4
@@ -805,7 +805,7 @@ static inline notrace void rcu_read_unlock_sched_notrace(void)/** * kfree_rcu() - kfree an object after a grace period. * @ptr: pointer to kfree -* @rcu_head: the name of the struct rcu_head within the type of @ptr. +* @rhf: the name of the struct rcu_head within the type of @ptr. * * Many rcu callbacks functions just call kfree() on the base structure. * These functions are trivial, but their size adds up, and furthermore
@@ -828,9 +828,13 @@ static inline notrace void rcu_read_unlock_sched_notrace(void)* The BUILD_BUG_ON check must not involve any function calls, hence the * checks are done in macros here. */ -#define kfree_rcu (ptr, rcu_head) \ - __kfree_rcu(&((ptr)->rcu_head), offsetof(typeof(*(ptr)), rcu_head)) - +#define kfree_rcu (ptr, rhf) \ +do { \ + typeof (ptr) ___p = (ptr); \ + \ + if (___p) \ + __kfree_rcu(&((___p)->rhf), offsetof(typeof(*(ptr)), rhf)); \ +} while (0)/* * Place this after a lock-acquisition primitive to guarantee that
Signature · 1 line
2.17.1
PATCH 2/9 rcu: Correctly unlock root node in rcu_check_gp_start_stall()
In reply to Paul E. McKenneyFrom: Neeraj Upadhyay <[email protected]>
On systems whose rcu_node tree has only one node, the rcu_check_gp_start_stall() function's values of rnp and rnp_root will be identical. In this case, it clearly does not make sense to release both rnp->lock and rnp_root->lock, but that is exactly what this function does in the last early exit. This commit therefore unlocks only rnp->lock when rnp and rnp_root are equal.
--- kernel/rcu/tree_stall.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/rcu/tree_stall.h +3 -1
@@ -630,7 +630,9 @@ static void rcu_check_gp_start_stall(struct rcu_node *rnp, struct rcu_data *rdp, time_before(j, rcu_state.gp_req_activity + gpssdelay) || time_before(j, rcu_state.gp_activity + gpssdelay) || atomic_xchg(&warned, 1)) { -raw_spin_unlock_rcu_node (rnp_root);/* irqs remain disabled. */ +if (rnp_root != rnp) +/* irqs remain disabled. */ +raw_spin_unlock_rcu_node (rnp_root);raw_spin_unlock_irqrestore_rcu_node (rnp, flags);return ; }
Signature · 1 line
2.17.1
PATCH 6/9 rcu: Upgrade sync_exp_work_done() to smp_mb()
In reply to Paul E. McKenneyThe sync_exp_work_done() function uses smp_mb__before_atomic(), but there is no obvious atomic in the ensuing code. The ordering is absolutely required for grace periods to work correctly, so this commit upgrades the smp_mb__before_atomic() to smp_mb().
--- kernel/rcu/tree_exp.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
kernel/rcu/tree_exp.h +1 -2
@@ -259,8 +259,7 @@ static bool sync_exp_work_done(unsigned long s) {if (rcu_exp_gp_seq_done (s)) {trace_rcu_exp_grace_period (rcu_state.name, s,TPS ("done" )); -/* Ensure test happens before caller kfree(). */ -smp_mb__before_atomic ();/* ^^^ */ +smp_mb ();/* Ensure test happens before caller kfree(). */ return true; }return false;
Signature · 1 line
2.17.1
PATCH 7/9 rcu: Fix irritating whitespace error in rcu_assign_pointer()
In reply to Paul E. McKenney--- include/linux/rcupdate.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/rcupdate.h +1 -1
@@ -369,7 +369,7 @@ static inline void rcu_preempt_sleep_check(void) { }#define rcu_assign_pointer (p, v) \ ({ \ uintptr_t _r_a_p__v = (uintptr_t)(v); \ - rcu_check_sparse(p, __rcu); \ + rcu_check_sparse(p, __rcu); \ \ if (__builtin_constant_p(v) && (_r_a_p__v) == (uintptr_t)NULL) \ WRITE_ONCE((p), (typeof(p))(_r_a_p__v)); \
Signature · 1 line
2.17.1
PATCH 9/9 rcu: Don't return a value from rcu_assign_pointer()
In reply to Paul E. McKenneyFrom: Andrea Parri <[email protected]>
Quoting Paul [1]:
"Given that a quick (and perhaps error-prone) search of the uses
of rcu_assign_pointer() in v5.1 didn't find a single use of the return value, let's please instead change the documentation and implementation to eliminate the return value."
[1] https://lkml.kernel.org/r/[email protected]
--- Documentation/RCU/whatisRCU.txt | 8 ++++---- include/linux/rcupdate.h | 5 ++--- tools/include/linux/rcu.h | 4 ++-- tools/testing/radix-tree/linux/rcupdate.h | 2 +- 4 files changed, 9 insertions(+), 10 deletions(-)
Documentation/RCU/whatisRCU.txt +4 -4
@@ -212,7 +212,7 @@ synchronize_rcu() rcu_assign_pointer() - typeof(p) rcu_assign_pointer(p, typeof(p) v); + void rcu_assign_pointer(p, typeof(p) v); Yes, rcu_assign_pointer() -is- implemented as a macro, though it would be cool to be able to declare a function in this manner.
@@ -220,9 +220,9 @@ rcu_assign_pointer() The updater uses this function to assign a new value to an RCU-protected pointer, in order to safely communicate the change - in value from the updater to the reader. This function returns - the new value, and also executes any memory-barrier instructions - required for a given CPU architecture. + in value from the updater to the reader. This macro does not + evaluate to an rvalue, but it does execute any memory-barrier + instructions required for a given CPU architecture. Perhaps just as important, it serves to document (1) which pointers are protected by RCU and (2) the point at which a
include/linux/rcupdate.h +2 -3
@@ -367,7 +367,7 @@ static inline void rcu_preempt_sleep_check(void) { }* other macros that it invokes. */ #define rcu_assign_pointer (p, v) \ -({ \ +do { \ uintptr_t _r_a_p__v = (uintptr_t)(v); \ rcu_check_sparse(p, __rcu); \ \
@@ -375,8 +375,7 @@ static inline void rcu_preempt_sleep_check(void) { }WRITE_ONCE ((p), (typeof (p))(_r_a_p__v)); \ else \ smp_store_release(&p, RCU_INITIALIZER((typeof(p))_r_a_p__v)); \ - _r_a_p__v; \ -}) +}while (0 )/** * rcu_swap_protected() - swap an RCU and a regular pointer
tools/include/linux/rcu.h +2 -2
@@ -19,7 +19,7 @@ static inline bool rcu_is_watching(void) return false; } -#define rcu_assign_pointer (p, v) ((p) = (v)) -#define RCU_INIT_POINTER (p, v) p=(v) +#define rcu_assign_pointer (p, v) do { (p) = (v); } while (0) +#define RCU_INIT_POINTER (p, v) do { (p) = (v); } while (0)#endif
tools/testing/radix-tree/linux/rcupdate.h +1 -1
@@ -7,6 +7,6 @@#define rcu_dereference_raw (p) rcu_dereference(p)#define rcu_dereference_protected (p, cond) rcu_dereference(p)#define rcu_dereference_check (p, cond) rcu_dereference(p) -#define RCU_INIT_POINTER (p, v) (p) = (v) +#define RCU_INIT_POINTER (p, v) do { (p) = (v); } while (0)#endif
Signature · 1 line
2.17.1
PATCH 5/9 doc: Remove ".vnet" from paulmck email addresses
In reply to Paul E. McKenney--- Documentation/core-api/circular-buffers.rst | 2 +- Documentation/memory-barriers.txt | 2 +- Documentation/translations/ko_KR/memory-barriers.txt | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
Documentation/core-api/circular-buffers.rst +1 -1
@@ -3,7 +3,7 @@ Circular Buffers ================ :Author: David Howells <[email protected]> -:Author: Paul E. McKenney <[email protected]> +:Author: Paul E. McKenney <[email protected]> Linux provides a number of features that can be used to implement circular
Documentation/memory-barriers.txt +1 -1
@@ -3,7 +3,7 @@ ============================ By: David Howells <[email protected]> - Paul E. McKenney <[email protected]> + Paul E. McKenney <[email protected]> Will Deacon <[email protected]> Peter Zijlstra <[email protected]>
Documentation/translations/ko_KR/memory-barriers.txt +1 -1
@@ -24,7 +24,7 @@ Documentation/memory-barriers.txt ========================= 저자: David Howells <[email protected]> - Paul E. McKenney <[email protected]> + Paul E. McKenney <[email protected]> Will Deacon <[email protected]> Peter Zijlstra <[email protected]>
Signature · 1 line
2.17.1
PATCH 4/9 rcu: Set a maximum limit for back-to-back callback invocation
In reply to Paul E. McKenneyCurrently, if a CPU has more than 10,000 callbacks pending, it will increase rdp->blimit to LONG_MAX. If you are lucky, LONG_MAX is only about two billion, but this is still a bit too many callbacks to invoke back-to-back while otherwise ignoring the world.
This commit therefore sets a maximum limit of DEFAULT_MAX_RCU_BLIMIT, which is set to 10,000, for rdp->blimit.
--- kernel/rcu/tree.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
kernel/rcu/tree.c +4 -3
@@ -380,7 +380,8 @@ static int rcu_is_cpu_rrupt_from_idle(void)__this_cpu_read (rcu_data.dynticks_nmi_nesting) <=1 ; } -#define DEFAULT_RCU_BLIMIT 10/* Maximum callbacks per rcu_do_batch. */ +#define DEFAULT_RCU_BLIMIT 10/* Maximum callbacks per rcu_do_batch ... */ +#define DEFAULT_MAX_RCU_BLIMIT 10000/* ... even during callback flood. */ static long blimit =DEFAULT_RCU_BLIMIT ;#define DEFAULT_RCU_QHIMARK 10000/* If this many pending, ignore blimit. */ static long qhimark =DEFAULT_RCU_QHIMARK ;
@@ -2113,7 +2114,7 @@ static void rcu_do_batch(struct rcu_data *rdp)/* Reinstate batch limit if we have worked down the excess. */ count =rcu_segcblist_n_cbs (&rdp->cblist); -if (rdp->blimit ==LONG_MAX && count <= qlowmark) +if (rdp->blimit >=DEFAULT_MAX_RCU_BLIMIT && count <= qlowmark) rdp->blimit = blimit;/* Reset ->qlen_last_fqs_check trigger if enough CBs have drained. */
@@ -2354,7 +2355,7 @@ static void __call_rcu_core(struct rcu_data *rdp, struct rcu_head *head,rcu_accelerate_cbs_unlocked (rdp->mynode, rdp); }else {/* Give the grace period a kick. */ - rdp->blimit =LONG_MAX ; + rdp->blimit =DEFAULT_MAX_RCU_BLIMIT ;if (rcu_state.n_force_qs == rdp->n_force_qs_snap &&rcu_segcblist_first_pend_cb (&rdp->cblist) != head)rcu_force_quiescent_state ();
Signature · 1 line
2.17.1
PATCH 8/9 rcu: Force inlining of rcu_read_lock()
In reply to Paul E. McKenneyFrom: Waiman Long <[email protected]>
When debugging options are turned on, the rcu_read_lock() function might not be inlined. This results in lockdep's print_lock() function printing "rcu_read_lock+0x0/0x70" instead of rcu_read_lock()'s caller. For example:
[ 10.579995] =============================
[ 10.584033] WARNING: suspicious RCU usage
[ 10.588074] 4.18.0.memcg_v2+ #1 Not tainted
[ 10.593162] -----------------------------
[ 10.597203] include/linux/rcupdate.h:281 Illegal context switch in
RCU read-side critical section!
[ 10.606220]
[ 10.606220] other info that might help us debug this:
[ 10.606220]
[ 10.614280]
[ 10.614280] rcu_scheduler_active = 2, debug_locks = 1
[ 10.620853] 3 locks held by systemd/1:
[ 10.624632] #0: (____ptrval____) (&type->i_mutex_dir_key#5){.+.+}, at: lookup_slow+0x42/0x70
[ 10.633232] #1: (____ptrval____) (rcu_read_lock){....}, at: rcu_read_lock+0x0/0x70
[ 10.640954] #2: (____ptrval____) (rcu_read_lock){....}, at: rcu_read_lock+0x0/0x70These "rcu_read_lock+0x0/0x70" strings are not providing any useful information. This commit therefore forces inlining of the rcu_read_lock() function so that rcu_read_lock()'s caller is instead shown.
--- include/linux/rcupdate.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/rcupdate.h +1 -1
@@ -588,7 +588,7 @@ static inline void rcu_preempt_sleep_check(void) { }* read-side critical sections may be preempted and they may also block, but * only when acquiring spinlocks that are subject to priority inheritance. */ -static inline void rcu_read_lock (void ) +static __always_inline void rcu_read_lock(void) {__rcu_read_lock ();__acquire (RCU );
Signature · 1 line
2.17.1
PATCH 1/9 rcu: Dump specified number of blocked tasks
In reply to Paul E. McKenneyFrom: Neeraj Upadhyay <[email protected]>
The dump_blkd_tasks() function dumps at most 10 blocked tasks, ignoring the value of the ncheck parameter. This commit therefore substitutes the value of ncheck for the hard-coded value of 10. Because all callers currently pass 10 as the number, this patch does not change behavior, but it is clearly an accident waiting to happen.
--- kernel/rcu/tree_plugin.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/tree_plugin.h +1 -1
@@ -760,7 +760,7 @@ dump_blkd_tasks(struct rcu_node *rnp, int ncheck) i =0 ;list_for_each (lhp, &rnp->blkd_tasks) {pr_cont (" %p" , lhp); -if (++i >=10 ) +if (++i >= ncheck)break ; }pr_cont ("\n" );
Signature · 1 line
2.17.1
Peter Zijlstra
maintainerRe: [PATCH tip/core/rcu 6/9] rcu: Upgrade sync_exp_work_done() to smp_mb()
In reply to Paul E. McKenneyQuoted 5 lines · Paul E. McKenney
The
sync_exp_work_done()function usessmp_mb__before_atomic(), but there is no obvious atomic in the ensuing code. The ordering is absolutely required for grace periods to work correctly, so this commit upgrades thesmp_mb__before_atomic()tosmp_mb().
Did this commit want a Fixes: line? Such that robots can find the right kernels to backport this to?
Quoted 23 lines
Reported-by Andrea Parri <[email protected]>Signed-off-by Paul E. McKenney <[email protected]>--- kernel/rcu/tree_exp.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)kernel/rcu/tree_exp.h +1
-2@@ -259,8 +259,7 @@ static bool sync_exp_work_done(unsigned long s) {if (rcu_exp_gp_seq_done (s)) {trace_rcu_exp_grace_period (rcu_state.name, s,TPS ("done" )); -/* Ensure test happens before caller kfree(). */ -smp_mb__before_atomic ();/* ^^^ */ +smp_mb ();/* Ensure test happens before caller kfree(). */ return true; }return false;Signature · 1 line
2.17.1
Re: [PATCH tip/core/rcu 6/9] rcu: Upgrade sync_exp_work_done() to smp_mb()
Quoted 9 lines · Peter Zijlstra
Quoted 5 lines · Paul E. McKenney
The
sync_exp_work_done()function usessmp_mb__before_atomic(), but there is no obvious atomic in the ensuing code. The ordering is absolutely required for grace periods to work correctly, so this commit upgrades thesmp_mb__before_atomic()tosmp_mb().Did this commit want a Fixes: line? Such that robots can find the right kernels to backport this to?
Indeed it does, and thanks to Andrea for finding the commit.
Thanx, Paul
------------------------------------------------------------------------
commit 96050c68be33edef18800ad6748f61f81db81a20
Author: Paul E. McKenney <[email protected]> Date: Sat Apr 20 01:40:54 2019 -0700
rcu: Upgrade sync_exp_work_done() to smp_mb()
The sync_exp_work_done() function uses smp_mb__before_atomic(), but there is no obvious atomic in the ensuing code. The ordering is absolutely required for grace periods to work correctly, so this commit upgrades the smp_mb__before_atomic() to smp_mb().
kernel/rcu/tree_exp.h +1 -2
@@ -259,8 +259,7 @@ static bool sync_exp_work_done(unsigned long s) {if (rcu_exp_gp_seq_done (s)) {trace_rcu_exp_grace_period (rcu_state.name, s,TPS ("done" )); -/* Ensure test happens before caller kfree(). */ -smp_mb__before_atomic ();/* ^^^ */ +smp_mb ();/* Ensure test happens before caller kfree(). */ return true; }return false;