RCU torture changes for v6.16
Subsystems (7)
Joel Fernandes
maintainercover RCU torture changes for v6.16
Hi,
Please find the upcoming miscellaneous RCU changes. The changes can also be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git torture.2025.04.18a
Thanks.
Joel Fernandes (2):
rcutorture: Perform more frequent testing of ->gpwrap
rcutorture: Fix issue with re-using old images on ARM64
Paul E. McKenney (12):
rcutorture: Make srcu_lockdep.sh check kernel Kconfig
rcutorture: Make srcu_lockdep.sh check reader-conflict handling
rcutorture: Split out beginning and end from rcu_torture_one_read()
rcutorture: Make torture.sh --do-rt use CONFIG_PREEMPT_RT
rcutorture: Add tests for SRCU up/down reader primitives
rcutorture: Pull rcu_torture_updown() loop body into new function
rcutorture: Comment invocations of tick_dep_set_task()
rcutorture: Complain if an ->up_read() is delayed more than 10 seconds
rcutorture: Check for ->up_read() without matching ->down_read()
checkpatch: Deprecate srcu_read_lock_lite() and
srcu_read_unlock_lite()
torture: Add --do-{,no-}normal to torture.sh
torture: Add testing of RCU's Rust bindings to torture.sh
kernel/rcu/rcu.h | 4 + kernel/rcu/rcutorture.c | 440 +++++++++++++++--- kernel/rcu/tree.c | 34 +- kernel/rcu/tree.h | 1 + scripts/checkpatch.pl | 2 + .../rcutorture/bin/kvm-test-1-run.sh | 2 +- .../selftests/rcutorture/bin/srcu_lockdep.sh | 42 +- .../selftests/rcutorture/bin/torture.sh | 89 +++- 8 files changed, 537 insertions(+), 77 deletions(-)
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 1/14 rcutorture: Make srcu_lockdep.sh check kernel Kconfig
From: "Paul E. McKenney" <[email protected]>
The srcu_lockdep.sh currently blindly trusts the rcutorture SRCU-P scenario to build its kernel with lockdep enabled. Of course, this dependency might not be obvious to someone rebalancing SRCU scenarios. This commit therefore adds code to srcu_lockdep.sh that verifies that the .config file has lockdep enabled.
--- .../testing/selftests/rcutorture/bin/srcu_lockdep.sh | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
tools/testing/selftests/rcutorture/bin/srcu_lockdep.sh +10 -1
@@ -39,8 +39,9 @@ do shift done -err= nerrs=0 + +# Test lockdep's handling of deadlocks. for d in 0 1 do for t in 0 1 2
@@ -52,6 +53,12 @@ do tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 5s --configs "SRCU-P" --kconfig "CONFIG_FORCE_NEED_SRCU_NMI_SAFE=y" --bootargs "rcutorture.test_srcu_lockdep=$val rcutorture.reader_flavor=0x2" --trust-make --datestamp "$ds/$val" > "$T/kvm.sh.out" 2>&1 ret=$? mv "$T/kvm.sh.out" "$RCUTORTURE/res/$ds/$val" + if ! grep -q '^CONFIG_PROVE_LOCKING=y' .config + then + echo "rcu_torture_init_srcu_lockdep:Error: CONFIG_PROVE_LOCKING disabled in rcutorture SRCU-P scenario" + nerrs=$((nerrs+1)) + err=1 + fi if test "$d" -ne 0 && test "$ret" -eq 0 then err=1
@@ -71,6 +78,8 @@ do done done done + +# Set up exit code. if test "$nerrs" -ne 0 then exit 1
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 2/14 rcutorture: Make srcu_lockdep.sh check reader-conflict handling
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
Mixing different flavors of RCU readers is forbidden, for example, you should not use srcu_read_lock() and srcu_read_lock_nmisafe() on the same srcu_struct structure. There are checks for this, but these checks are not tested on a regular basis. This commit therefore adds such tests to srcu_lockdep.sh.
--- .../selftests/rcutorture/bin/srcu_lockdep.sh | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+)
tools/testing/selftests/rcutorture/bin/srcu_lockdep.sh +31 -0
@@ -79,6 +79,37 @@ do done done +# Test lockdep-enabled testing of mixed SRCU readers. +for val in 0x1 0xf +do + err= + tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 5s --configs "SRCU-P" --kconfig "CONFIG_FORCE_NEED_SRCU_NMI_SAFE=y" --bootargs "rcutorture.reader_flavor=$val" --trust-make --datestamp "$ds/$val" > "$T/kvm.sh.out" 2>&1 + ret=$? + mv "$T/kvm.sh.out" "$RCUTORTURE/res/$ds/$val" + if ! grep -q '^CONFIG_PROVE_LOCKING=y' .config + then + echo "rcu_torture_init_srcu_lockdep:Error: CONFIG_PROVE_LOCKING disabled in rcutorture SRCU-P scenario" + nerrs=$((nerrs+1)) + err=1 + fi + if test "$val" -eq 0xf && test "$ret" -eq 0 + then + err=1 + echo -n Unexpected success for > "$RCUTORTURE/res/$ds/$val/kvm.sh.err" + fi + if test "$val" -eq 0x1 && test "$ret" -ne 0 + then + err=1 + echo -n Unexpected failure for > "$RCUTORTURE/res/$ds/$val/kvm.sh.err" + fi + if test -n "$err" + then + grep "rcu_torture_init_srcu_lockdep: test_srcu_lockdep = " "$RCUTORTURE/res/$ds/$val/SRCU-P/console.log" | sed -e 's/^.*rcu_torture_init_srcu_lockdep://' >> "$RCUTORTURE/res/$ds/$val/kvm.sh.err" + cat "$RCUTORTURE/res/$ds/$val/kvm.sh.err" + nerrs=$((nerrs+1)) + fi +done + # Set up exit code. if test "$nerrs" -ne 0 then
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 3/14 rcutorture: Split out beginning and end from rcu_torture_one_read()
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
The rcu_torture_one_read() function is designed for RCU readers that are confined to a task, such that a single thread of control extends from the beginning of a given RCU read-side critical section to its end. This does not suffice for things like srcu_down_read() and srcu_up_read(), where the critical section might start at task level and end in a timer handler.
This commit therefore creates separate init_rcu_torture_one_read_state(), rcu_torture_one_read_start(), and rcu_torture_one_read_end() functions, along with a rcu_torture_one_read_state structure to coordinate their actions. These will be used to create tests for srcu_down_read() and friends.
One caution: The caller to rcu_torture_one_read_start() must enter the initial read-side critical section prior to the call. This enables use of non-standard primitives such as srcu_down_read() while still using the same validation code.
--- kernel/rcu/rcutorture.c | 124 ++++++++++++++++++++++++++-------------- 1 file changed, 81 insertions(+), 43 deletions(-)
kernel/rcu/rcutorture.c +81 -43
@@ -2164,53 +2164,70 @@ rcutorture_loop_extend(int *readstate, bool insoftirq, struct torture_random_stareturn &rtrsp[j]; } -/* -* Do one read-side critical section, returning false if there was -* no data to read. Can be invoked both from process context and -* from a timer handler. -*/ -static bool rcu_torture_one_read (struct torture_random_state *trsp,long myid) -{ -bool checkpolling = !(torture_random (trsp) &0xfff ); +struct rcu_torture_one_read_state { +bool checkpolling;unsigned long cookie;struct rcu_gp_oldstate cookie_full; -int i;unsigned long started; -unsigned long completed; -int newstate;struct rcu_torture *p; -int pipe_count; -bool preempted = false; -int readstate =0 ; -struct rt_read_seg rtseg[RCUTORTURE_RDR_MAX_SEGS ] = { {0 } }; -struct rt_read_seg *rtrsp = &rtseg[0 ]; -struct rt_read_seg *rtrsp1; +int readstate; +struct rt_read_seg rtseg[RCUTORTURE_RDR_MAX_SEGS ]; +struct rt_read_seg *rtrsp;unsigned long long ts; +}; -WARN_ON_ONCE (!rcu_is_watching ()); - newstate =rcutorture_extend_mask (readstate, trsp); -rcutorture_one_extend (&readstate, newstate, myid <0 , trsp, rtrsp++); -if (checkpolling) { +static void init_rcu_torture_one_read_state (struct rcu_torture_one_read_state *rtorsp, +struct torture_random_state *trsp) +{ +memset (rtorsp,0 ,sizeof (*rtorsp)); + rtorsp->checkpolling = !(torture_random (trsp) &0xfff ); + rtorsp->rtrsp = &rtorsp->rtseg[0 ]; +} + +/* +* Set up the first segment of a series of overlapping read-side +* critical sections. The caller must have actually initiated the +* outermost read-side critical section. +*/ +static bool rcu_torture_one_read_start (struct rcu_torture_one_read_state *rtorsp, +struct torture_random_state *trsp,long myid) +{ +if (rtorsp->checkpolling) {if (cur_ops->get_gp_state && cur_ops->poll_gp_state) - cookie = cur_ops->get_gp_state (); + rtorsp->cookie = cur_ops->get_gp_state ();if (cur_ops->get_gp_state_full && cur_ops->poll_gp_state_full) - cur_ops->get_gp_state_full (&cookie_full); + cur_ops->get_gp_state_full (&rtorsp->cookie_full); } - started = cur_ops->get_gp_seq (); - ts =rcu_trace_clock_local (); - p =rcu_dereference_check (rcu_torture_current, + rtorsp->started = cur_ops->get_gp_seq (); + rtorsp->ts =rcu_trace_clock_local (); + rtorsp->p =rcu_dereference_check (rcu_torture_current, !cur_ops->readlock_held || cur_ops->readlock_held ()); -if (p ==NULL ) { +if (rtorsp->p ==NULL ) {/* Wait for rcu_torture_writer to get underway */ -rcutorture_one_extend (&readstate,0 , myid <0 , trsp, rtrsp); +rcutorture_one_extend (&rtorsp->readstate,0 , myid <0 , trsp, rtorsp->rtrsp);return false; } -if (p->rtort_mbtest ==0 ) +if (rtorsp->p->rtort_mbtest ==0 )atomic_inc (&n_rcu_torture_mberror); -rcu_torture_reader_do_mbchk (myid, p, trsp); - rtrsp =rcutorture_loop_extend (&readstate, myid <0 , trsp, rtrsp); +rcu_torture_reader_do_mbchk (myid, rtorsp->p, trsp); +return true; +} + +/* +* Complete the last segment of a series of overlapping read-side +* critical sections and check for errors. +*/ +static void rcu_torture_one_read_end (struct rcu_torture_one_read_state *rtorsp, +struct torture_random_state *trsp,long myid) +{ +int i; +unsigned long completed; +int pipe_count; +bool preempted = false; +struct rt_read_seg *rtrsp1; +preempt_disable (); - pipe_count =READ_ONCE (p->rtort_pipe_count); + pipe_count =READ_ONCE (rtorsp->p->rtort_pipe_count);if (pipe_count >RCU_TORTURE_PIPE_LEN ) {// Should not happen in a correct RCU implementation, // happens quite often for torture_type=busted.
@@ -2218,28 +2235,28 @@ static bool rcu_torture_one_read(struct torture_random_state *trsp, long myid) } completed = cur_ops->get_gp_seq ();if (pipe_count >1 ) { -do_trace_rcu_torture_read (cur_ops->name, &p->rtort_rcu, - ts, started, completed); +do_trace_rcu_torture_read (cur_ops->name, &rtorsp->p->rtort_rcu, + rtorsp->ts, rtorsp->started, completed);rcu_ftrace_dump (DUMP_ALL ); }__this_cpu_inc (rcu_torture_count[pipe_count]); - completed =rcutorture_seq_diff (completed, started); + completed =rcutorture_seq_diff (completed, rtorsp->started);if (completed >RCU_TORTURE_PIPE_LEN ) {/* Should not happen, but... */ completed =RCU_TORTURE_PIPE_LEN ; }__this_cpu_inc (rcu_torture_batch[completed]);preempt_enable (); -if (checkpolling) { +if (rtorsp->checkpolling) {if (cur_ops->get_gp_state && cur_ops->poll_gp_state) -WARN_ONCE (cur_ops->poll_gp_state (cookie), +WARN_ONCE (cur_ops->poll_gp_state (rtorsp->cookie),"%s: Cookie check 2 failed %s(%d) %lu->%lu\n" , __func__,rcu_torture_writer_state_getname (), rcu_torture_writer_state, - cookie, cur_ops->get_gp_state ()); + rtorsp->cookie, cur_ops->get_gp_state ());if (cur_ops->get_gp_state_full && cur_ops->poll_gp_state_full) - WARN_ONCE(cur_ops->poll_gp_state_full(&cookie_full), + WARN_ONCE(cur_ops->poll_gp_state_full(&rtorsp->cookie_full), "%s: Cookie check 6 failed %s(%d) online %*pbl\n", __func__, rcu_torture_writer_state_getname(),
@@ -2248,21 +2265,42 @@ static bool rcu_torture_one_read(struct torture_random_state *trsp, long myid) }if (cur_ops->reader_blocked) preempted = cur_ops->reader_blocked (); -rcutorture_one_extend (&readstate,0 , myid <0 , trsp, rtrsp); -WARN_ON_ONCE (readstate); +rcutorture_one_extend (&rtorsp->readstate,0 , myid <0 , trsp, rtorsp->rtrsp); +WARN_ON_ONCE (rtorsp->readstate);// This next splat is expected behavior if leakpointer, especially // for CONFIG_RCU_STRICT_GRACE_PERIOD=y kernels. -WARN_ON_ONCE (leakpointer &&READ_ONCE (p->rtort_pipe_count) >1 ); +WARN_ON_ONCE (leakpointer &&READ_ONCE (rtorsp->p->rtort_pipe_count) >1 );/* If error or close call, record the sequence of reader protections. */ if ((pipe_count >1 || completed >1 ) && !xchg (&err_segs_recorded,1 )) { i =0 ; -for (rtrsp1 = &rtseg[0 ]; rtrsp1 < rtrsp; rtrsp1++) +for (rtrsp1 = &rtorsp->rtseg[0 ]; rtrsp1 < rtorsp->rtrsp; rtrsp1++) err_segs[i++] = *rtrsp1; rt_read_nsegs = i; rt_read_preempted = preempted; } +} +/* +* Do one read-side critical section, returning false if there was +* no data to read. Can be invoked both from process context and +* from a timer handler. +*/ +static bool rcu_torture_one_read (struct torture_random_state *trsp,long myid) +{ +int newstate; +struct rcu_torture_one_read_state rtors; + +WARN_ON_ONCE (!rcu_is_watching ()); +init_rcu_torture_one_read_state (&rtors, trsp); + newstate =rcutorture_extend_mask (rtors.readstate, trsp); +rcutorture_one_extend (&rtors.readstate, newstate, myid <0 , trsp, rtors.rtrsp++); +if (!rcu_torture_one_read_start (&rtors, trsp, myid)) { +rcutorture_one_extend (&rtors.readstate,0 , myid <0 , trsp, rtors.rtrsp); +return false; + } + rtors.rtrsp =rcutorture_loop_extend (&rtors.readstate, myid <0 , trsp, rtors.rtrsp); +rcu_torture_one_read_end (&rtors, trsp, myid);return true; }
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 4/14 rcutorture: Make torture.sh --do-rt use CONFIG_PREEMPT_RT
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
The torture.sh --do-rt command-line parameter is intended to mimic -rt kernels. Now that CONFIG_PREEMPT_RT is upstream, this commit makes this mimicking more precise.
Note that testing of RCU priority boosting is disabled in favor of forward-progress testing of RCU callbacks. If it turns out to be possible to make kernels built with CONFIG_PREEMPT_RT=y to tolerate testing of both, both will be enabled.
[ paulmck: Apply Sebastian Siewior feedback. ]
--- tools/testing/selftests/rcutorture/bin/torture.sh | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
tools/testing/selftests/rcutorture/bin/torture.sh +9 -5
@@ -448,13 +448,17 @@ fi if test "$do_rt" = "yes" then - # With all post-boot grace periods forced to normal. - torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcupdate.rcu_normal=1" - torture_set "rcurttorture" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --trust-make + # In both runs, disable testing of RCU priority boosting because + # -rt doesn't like its interaction with testing of callback + # flooding. + + # With all post-boot grace periods forced to normal (default for PREEMPT_RT). + torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcutorture.test_boost=0 rcutorture.preempt_duration=0" + torture_set "rcurttorture" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=y CONFIG_RCU_NOCB_CPU=y" --trust-make # With all post-boot grace periods forced to expedited. - torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcupdate.rcu_expedited=1" - torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --trust-make + torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcutorture.test_boost=0 rcupdate.rcu_normal_after_boot=0 rcupdate.rcu_expedited=1 rcutorture.preempt_duration=0" + torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_FULL=y CONFIG_RCU_NOCB_CPU=y" --trust-make fi if test "$do_srcu_lockdep" = "yes"
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 5/14 rcutorture: Add tests for SRCU up/down reader primitives
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
This commit adds a new rcutorture.n_up_down kernel boot parameter that specifies the number of outstanding SRCU up/down readers, which begin in kthread context and end in an hrtimer handler. There is a new kthread ("rcu_torture_updown") that scans an per-reader array looking for elements whose readers have ended. This kthread sleeps between one and two milliseconds between consecutive scans.
[ paulmck: Apply kernel test robot feedback. ]
[ paulmck: Apply Z qiang feedback. ]
[ joel: Fix build error: hrtimer_init is replaced by hrtimer_setup. ] [ joel: Apply Boqun bug fix to drop extra up_read() call in
rcu_torture_updown()].
--- kernel/rcu/rcutorture.c | 225 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 206 insertions(+), 19 deletions(-)
kernel/rcu/rcutorture.c +206 -19
@@ -55,22 +55,24 @@ MODULE_DESCRIPTION("Read-Copy Update module-based torture test facility");MODULE_LICENSE ("GPL" );MODULE_AUTHOR ("Paul E. McKenney <[email protected]> and Josh Triplett <[email protected]>" ); -/* Bits for ->extendables field, extendables param, and related definitions. */ -#define RCUTORTURE_RDR_SHIFT_1 8/* Put SRCU index in upper bits. */ -#define RCUTORTURE_RDR_MASK_1 (0xff << RCUTORTURE_RDR_SHIFT_1) -#define RCUTORTURE_RDR_SHIFT_2 16/* Put SRCU index in upper bits. */ -#define RCUTORTURE_RDR_MASK_2 (0xff << RCUTORTURE_RDR_SHIFT_2) -#define RCUTORTURE_RDR_BH 0x01/* Extend readers by disabling bh. */ -#define RCUTORTURE_RDR_IRQ 0x02/* ... disabling interrupts. */ -#define RCUTORTURE_RDR_PREEMPT 0x04/* ... disabling preemption. */ -#define RCUTORTURE_RDR_RBH 0x08/* ... rcu_read_lock_bh(). */ -#define RCUTORTURE_RDR_SCHED 0x10/* ... rcu_read_lock_sched(). */ -#define RCUTORTURE_RDR_RCU_1 0x20/* ... entering another RCU reader. */ -#define RCUTORTURE_RDR_RCU_2 0x40/* ... entering another RCU reader. */ -#define RCUTORTURE_RDR_NBITS 7/* Number of bits defined above. */ -#define RCUTORTURE_MAX_EXTEND \ +// Bits for ->extendables field, extendables param, and related definitions. +#define RCUTORTURE_RDR_SHIFT_1 8 // Put SRCU index in upper bits. +#define RCUTORTURE_RDR_MASK_1 (0xff << RCUTORTURE_RDR_SHIFT_1) +#define RCUTORTURE_RDR_SHIFT_2 16 // Put SRCU index in upper bits. +#define RCUTORTURE_RDR_MASK_2 (0xff << RCUTORTURE_RDR_SHIFT_2) +#define RCUTORTURE_RDR_BH 0x01 // Extend readers by disabling bh. +#define RCUTORTURE_RDR_IRQ 0x02 // ... disabling interrupts. +#define RCUTORTURE_RDR_PREEMPT 0x04 // ... disabling preemption. +#define RCUTORTURE_RDR_RBH 0x08 // ... rcu_read_lock_bh(). +#define RCUTORTURE_RDR_SCHED 0x10 // ... rcu_read_lock_sched(). +#define RCUTORTURE_RDR_RCU_1 0x20 // ... entering another RCU reader. +#define RCUTORTURE_RDR_RCU_2 0x40 // ... entering another RCU reader. +#define RCUTORTURE_RDR_UPDOWN 0x80 // ... up-read from task, down-read from timer. +// Note: Manual start, automatic end. +#define RCUTORTURE_RDR_NBITS 8 // Number of bits defined above. +#define RCUTORTURE_MAX_EXTEND \ (RCUTORTURE_RDR_BH | RCUTORTURE_RDR_IRQ | RCUTORTURE_RDR_PREEMPT | \ - RCUTORTURE_RDR_RBH | RCUTORTURE_RDR_SCHED) + RCUTORTURE_RDR_RBH | RCUTORTURE_RDR_SCHED) // Intentionally omit RCUTORTURE_RDR_UPDOWN.#define RCUTORTURE_RDR_ALLBITS \ (RCUTORTURE_MAX_EXTEND | RCUTORTURE_RDR_RCU_1 | RCUTORTURE_RDR_RCU_2 | \ RCUTORTURE_RDR_MASK_1 | RCUTORTURE_RDR_MASK_2)
@@ -110,6 +112,7 @@ torture_param(bool, gp_sync, false, "Use synchronous GP wait primitives");torture_param (int, irqreader,1 ,"Allow RCU readers from irq handlers" );torture_param (int, leakpointer,0 ,"Leak pointer dereferences from readers" );torture_param (int, n_barrier_cbs,0 ,"# of callbacks/kthreads for barrier testing" ); +torture_param (int, n_up_down,32 ,"# of concurrent up/down hrtimer-based RCU readers" );torture_param (int, nfakewriters,4 ,"Number of RCU fake writer threads" );torture_param (int, nreaders,-1 ,"Number of RCU reader threads" );torture_param (int, object_debug,0 ,"Enable debug-object double call_rcu() testing" );
@@ -152,6 +155,7 @@ static int nrealfakewriters;static struct task_struct *writer_task;static struct task_struct **fakewriter_tasks;static struct task_struct **reader_tasks; +static struct task_struct *updown_task;static struct task_struct **nocb_tasks;static struct task_struct *stats_task;static struct task_struct *fqs_task;
@@ -374,6 +378,8 @@ struct rcu_torture_ops {void (*readunlock)(int idx);int (*readlock_held)(void );// lockdep. int (*readlock_nesting)(void );// actual nesting, if available, -1 if not. +int (*down_read)(void ); +void (*up_read)(int idx);unsigned long (*get_gp_seq)(void );unsigned long (*gp_diff)(unsigned long new,unsigned long old);void (*deferred_free)(struct rcu_torture *p);
@@ -421,6 +427,7 @@ struct rcu_torture_ops {int no_pi_lock;int debug_objects;int start_poll_irqsoff; +int have_up_down;const char *name; };
@@ -754,6 +761,50 @@ static int torture_srcu_read_lock_held(void)return srcu_read_lock_held (srcu_ctlp); } +static bool srcu_torture_have_up_down (void ) +{ +int rf = reader_flavor; + +if (!rf) + rf =SRCU_READ_FLAVOR_NORMAL ; +return !!(cur_ops->have_up_down & rf); +} + +static int srcu_torture_down_read (void ) +{ +int idx; +struct srcu_ctr __percpu *scp; + +WARN_ON_ONCE (reader_flavor & ~SRCU_READ_FLAVOR_ALL ); +WARN_ON_ONCE (reader_flavor & (reader_flavor -1 )); + +if ((reader_flavor &SRCU_READ_FLAVOR_NORMAL ) || !(reader_flavor &SRCU_READ_FLAVOR_ALL )) { + idx =srcu_down_read (srcu_ctlp); +WARN_ON_ONCE (idx & ~0x1 ); +return idx; + } +if (reader_flavor &SRCU_READ_FLAVOR_FAST ) { + scp =srcu_down_read_fast (srcu_ctlp); + idx =__srcu_ptr_to_ctr (srcu_ctlp, scp); +WARN_ON_ONCE (idx & ~0x1 ); +return idx <<3 ; + } +WARN_ON_ONCE (1 ); +return 0 ; +} + +static void srcu_torture_up_read (int idx) +{ +WARN_ON_ONCE ((reader_flavor && (idx & ~reader_flavor)) || (!reader_flavor && (idx & ~0x1 ))); +if (reader_flavor &SRCU_READ_FLAVOR_FAST ) +srcu_up_read_fast (srcu_ctlp,__srcu_ctr_to_ptr (srcu_ctlp, (idx &0x8 ) >>3 )); +else if ((reader_flavor &SRCU_READ_FLAVOR_NORMAL ) || + !(reader_flavor &SRCU_READ_FLAVOR_ALL )) +srcu_up_read (srcu_ctlp, idx &0x1 ); +else +WARN_ON_ONCE (1 ); +} +static unsigned long srcu_torture_completed (void ) {return srcu_batches_completed (srcu_ctlp);
@@ -811,6 +862,8 @@ static struct rcu_torture_ops srcu_ops = { .readlock = srcu_torture_read_lock, .read_delay = srcu_read_delay, .readunlock = srcu_torture_read_unlock, + .down_read = srcu_torture_down_read, + .up_read = srcu_torture_up_read, .readlock_held = torture_srcu_read_lock_held, .get_gp_seq = srcu_torture_completed, .gp_diff = rcu_seq_diff,
@@ -831,6 +884,8 @@ static struct rcu_torture_ops srcu_ops = { .irq_capable = 1, .no_pi_lock = IS_ENABLED(CONFIG_TINY_SRCU), .debug_objects = 1, + .have_up_down =IS_ENABLED (CONFIG_TINY_SRCU ) + ? 0 : SRCU_READ_FLAVOR_NORMAL | SRCU_READ_FLAVOR_FAST, .name ="srcu" };
@@ -856,6 +911,8 @@ static struct rcu_torture_ops srcud_ops = { .read_delay = srcu_read_delay, .readunlock = srcu_torture_read_unlock, .readlock_held = torture_srcu_read_lock_held, + .down_read = srcu_torture_down_read, + .up_read = srcu_torture_up_read, .get_gp_seq = srcu_torture_completed, .gp_diff = rcu_seq_diff, .deferred_free = srcu_torture_deferred_free,
@@ -875,6 +932,8 @@ static struct rcu_torture_ops srcud_ops = { .irq_capable = 1, .no_pi_lock = IS_ENABLED(CONFIG_TINY_SRCU), .debug_objects = 1, + .have_up_down =IS_ENABLED (CONFIG_TINY_SRCU ) + ? 0 : SRCU_READ_FLAVOR_NORMAL | SRCU_READ_FLAVOR_FAST, .name ="srcud" };
@@ -1985,7 +2044,7 @@ static void rcutorture_one_extend(int *readstate, int newstate, bool insoftirq, first = idxold1 ==0 ;WARN_ON_ONCE (idxold2 <0 ); -WARN_ON_ONCE (idxold2 & ~RCUTORTURE_RDR_ALLBITS ); +WARN_ON_ONCE (idxold2 & ~(RCUTORTURE_RDR_ALLBITS |RCUTORTURE_RDR_UPDOWN ));rcutorture_one_extend_check ("before change" , idxold1, statesnew, statesold, insoftirq); rtrsp->rt_readstate = newstate;
@@ -2061,6 +2120,11 @@ static void rcutorture_one_extend(int *readstate, int newstate, bool insoftirq,if (lockit)raw_spin_unlock_irqrestore (¤t->pi_lock, flags); } +if (statesold &RCUTORTURE_RDR_UPDOWN ) { + cur_ops->up_read ((idxold1 &RCUTORTURE_RDR_MASK_1 ) >>RCUTORTURE_RDR_SHIFT_1 ); +WARN_ON_ONCE (idxnew1 !=-1 ); + idxold1 =0 ; + }/* Delay if neither beginning nor end and there was a change. */ if ((statesnew || statesold) && *readstate && newstate)
@@ -2201,7 +2265,8 @@ static bool rcu_torture_one_read_start(struct rcu_torture_one_read_state *rtorsp rtorsp->started = cur_ops->get_gp_seq (); rtorsp->ts =rcu_trace_clock_local (); rtorsp->p =rcu_dereference_check (rcu_torture_current, - !cur_ops->readlock_held || cur_ops->readlock_held ()); + !cur_ops->readlock_held || cur_ops->readlock_held () || + (rtorsp->readstate &RCUTORTURE_RDR_UPDOWN ));if (rtorsp->p ==NULL ) {/* Wait for rcu_torture_writer to get underway */ rcutorture_one_extend (&rtorsp->readstate,0 , myid <0 , trsp, rtorsp->rtrsp);
@@ -2370,6 +2435,121 @@ rcu_torture_reader(void *arg)return 0 ; } +struct rcu_torture_one_read_state_updown { +struct hrtimer rtorsu_hrt; +bool rtorsu_inuse; +struct torture_random_state rtorsu_trs; +struct rcu_torture_one_read_state rtorsu_rtors; +}; + +static struct rcu_torture_one_read_state_updown *updownreaders; +static DEFINE_TORTURE_RANDOM (rcu_torture_updown_rand); +static int rcu_torture_updown (void *arg); + +static enum hrtimer_restart rcu_torture_updown_hrt (struct hrtimer *hrtp) +{ +struct rcu_torture_one_read_state_updown *rtorsup; + + rtorsup = container_of(hrtp, struct rcu_torture_one_read_state_updown, rtorsu_hrt); +rcu_torture_one_read_end (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 ); +smp_store_release (&rtorsup->rtorsu_inuse, false); +return HRTIMER_NORESTART ; +} + +static int rcu_torture_updown_init (void ) +{ +int i; +struct torture_random_state *rand = &rcu_torture_updown_rand; +int ret; + +if (n_up_down <0 ) +return 0 ; +if (!srcu_torture_have_up_down ()) { +VERBOSE_TOROUT_STRING ("rcu_torture_updown_init: Disabling up/down reader tests due to lack of primitives" ); +return 0 ; + } + updownreaders =kcalloc (n_up_down,sizeof (*updownreaders),GFP_KERNEL ); +if (!updownreaders) { +VERBOSE_TOROUT_STRING ("rcu_torture_updown_init: Out of memory, disabling up/down reader tests" ); +return -ENOMEM ; + } +for (i =0 ; i < n_up_down; i++) { +init_rcu_torture_one_read_state (&updownreaders[i].rtorsu_rtors, rand); +hrtimer_setup (&updownreaders[i].rtorsu_hrt, rcu_torture_updown_hrt,CLOCK_MONOTONIC , +HRTIMER_MODE_REL |HRTIMER_MODE_SOFT ); +torture_random_init (&updownreaders[i].rtorsu_trs); +init_rcu_torture_one_read_state (&updownreaders[i].rtorsu_rtors, + &updownreaders[i].rtorsu_trs); + } + ret =torture_create_kthread (rcu_torture_updown, rand, updown_task); +if (ret) { +kfree (updownreaders); + updownreaders =NULL ; + } +return ret; +} + +static void rcu_torture_updown_cleanup (void ) +{ +struct rcu_torture_one_read_state_updown *rtorsup; + +for (rtorsup = updownreaders; rtorsup < &updownreaders[n_up_down]; rtorsup++) { +if (!smp_load_acquire (&rtorsup->rtorsu_inuse)) +continue ; +if (!hrtimer_cancel (&rtorsup->rtorsu_hrt)) +WARN_ON_ONCE (rtorsup->rtorsu_inuse); + + } +kfree (updownreaders); + updownreaders =NULL ; +} + +/* +* RCU torture up/down reader kthread, starting RCU readers in kthread +* context and ending them in hrtimer handlers. Otherwise similar to +* rcu_torture_reader(). +*/ +static int +rcu_torture_updown (void *arg) +{ +int idx; +int rawidx; +struct rcu_torture_one_read_state_updown *rtorsup; +ktime_t t; + +VERBOSE_TOROUT_STRING ("rcu_torture_updown task started" ); +do { +for (rtorsup = updownreaders; rtorsup < &updownreaders[n_up_down]; rtorsup++) { +if (torture_must_stop ()) +break ; +if (smp_load_acquire (&rtorsup->rtorsu_inuse)) +continue ; +init_rcu_torture_one_read_state (&rtorsup->rtorsu_rtors, + &rtorsup->rtorsu_trs); + rawidx = cur_ops->down_read (); + idx = (rawidx <<RCUTORTURE_RDR_SHIFT_1 ) &RCUTORTURE_RDR_MASK_1 ; + rtorsup->rtorsu_rtors.readstate = idx |RCUTORTURE_RDR_UPDOWN ; + rtorsup->rtorsu_rtors.rtrsp++; +if (!rcu_torture_one_read_start (&rtorsup->rtorsu_rtors, + &rtorsup->rtorsu_trs,-1 )) { +schedule_timeout_idle (HZ ); +continue ; + } +smp_store_release (&rtorsup->rtorsu_inuse, true); + t =torture_random (&rtorsup->rtorsu_trs) &0xfffff ;// One per million. +if (t <10 *1000 ) + t =200 *1000 *1000 ; +hrtimer_start (&rtorsup->rtorsu_hrt, t, +HRTIMER_MODE_REL |HRTIMER_MODE_SOFT ); + } +torture_hrtimeout_ms (1 ,1000 , &rcu_torture_updown_rand); +stutter_wait ("rcu_torture_updown" ); + }while (!torture_must_stop ()); +rcu_torture_updown_cleanup (); +torture_kthread_stopping ("rcu_torture_updown" ); +return 0 ; +} +/* * Randomly Toggle CPUs' callback-offload state. This uses hrtimers to * increase race probabilities and fuzzes the interval between toggling.
@@ -2620,7 +2800,7 @@ rcu_torture_print_module_parms(struct rcu_torture_ops *cur_ops, const char *tag)"reader_flavor=%x " "nocbs_nthreads=%d nocbs_toggle=%d " "test_nmis=%d " -"preempt_duration=%d preempt_interval=%d\n" , +"preempt_duration=%d preempt_interval=%d n_up_down=%d\n" , torture_type, tag, nrealreaders, nrealfakewriters, stat_interval, verbose, test_no_idle_hz, shuffle_interval, stutter, irqreader, fqs_duration, fqs_holdoff, fqs_stutter,
@@ -2634,7 +2814,7 @@ rcu_torture_print_module_parms(struct rcu_torture_ops *cur_ops, const char *tag) reader_flavor, nocbs_nthreads, nocbs_toggle, test_nmis, - preempt_duration, preempt_interval); + preempt_duration, preempt_interval, n_up_down); }static int rcutorture_booster_cleanup (unsigned int cpu)
@@ -3686,6 +3866,10 @@ rcu_torture_cleanup(void) nocb_tasks =NULL ; } +if (updown_task) { +torture_stop_kthread (rcu_torture_updown, updown_task); + updown_task =NULL ; + }if (reader_tasks) {for (i =0 ; i < nrealreaders; i++) torture_stop_kthread(rcu_torture_reader,
@@ -4216,6 +4400,9 @@ rcu_torture_init(void)if (torture_init_error (firsterr)) goto unwind; } + firsterr =rcu_torture_updown_init (); +if (torture_init_error (firsterr)) + goto unwind; nrealnocbers = nocbs_nthreads;if (WARN_ON (nrealnocbers <0 )) nrealnocbers =1 ;
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 6/14 rcutorture: Pull rcu_torture_updown() loop body into new function
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
This is strictly a code-movement commit, pulling that part of the rcu_torture_updown() function's loop body that processes one rcu_torture_one_read_state_updown structure into a new rcu_torture_updown_one() function. The checks for the end of the torture test and the current structure being in use remain in the rcu_torture_updown() function.
--- kernel/rcu/rcutorture.c | 44 ++++++++++++++++++++++------------------- 1 file changed, 24 insertions(+), 20 deletions(-)
kernel/rcu/rcutorture.c +24 -20
@@ -2504,6 +2504,29 @@ static void rcu_torture_updown_cleanup(void) updownreaders =NULL ; } +// Do one reader for rcu_torture_updown(). +static void rcu_torture_updown_one (struct rcu_torture_one_read_state_updown *rtorsup) +{ +int idx; +int rawidx; +ktime_t t; + +init_rcu_torture_one_read_state (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs); + rawidx = cur_ops->down_read (); + idx = (rawidx <<RCUTORTURE_RDR_SHIFT_1 ) &RCUTORTURE_RDR_MASK_1 ; + rtorsup->rtorsu_rtors.readstate = idx |RCUTORTURE_RDR_UPDOWN ; + rtorsup->rtorsu_rtors.rtrsp++; +if (!rcu_torture_one_read_start (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 )) { +schedule_timeout_idle (HZ ); +return ; + } +smp_store_release (&rtorsup->rtorsu_inuse, true); + t =torture_random (&rtorsup->rtorsu_trs) &0xfffff ;// One per million. +if (t <10 *1000 ) + t =200 *1000 *1000 ; +hrtimer_start (&rtorsup->rtorsu_hrt, t,HRTIMER_MODE_REL |HRTIMER_MODE_SOFT ); +} +/* * RCU torture up/down reader kthread, starting RCU readers in kthread * context and ending them in hrtimer handlers. Otherwise similar to
@@ -2512,10 +2535,7 @@ static void rcu_torture_updown_cleanup(void)static int rcu_torture_updown (void *arg) { -int idx; -int rawidx;struct rcu_torture_one_read_state_updown *rtorsup; -ktime_t t;VERBOSE_TOROUT_STRING ("rcu_torture_updown task started" ); do {
@@ -2524,23 +2544,7 @@ rcu_torture_updown(void *arg)break ;if (smp_load_acquire (&rtorsup->rtorsu_inuse))continue ; -init_rcu_torture_one_read_state (&rtorsup->rtorsu_rtors, - &rtorsup->rtorsu_trs); - rawidx = cur_ops->down_read (); - idx = (rawidx <<RCUTORTURE_RDR_SHIFT_1 ) &RCUTORTURE_RDR_MASK_1 ; - rtorsup->rtorsu_rtors.readstate = idx |RCUTORTURE_RDR_UPDOWN ; - rtorsup->rtorsu_rtors.rtrsp++; -if (!rcu_torture_one_read_start (&rtorsup->rtorsu_rtors, - &rtorsup->rtorsu_trs,-1 )) { -schedule_timeout_idle (HZ ); -continue ; - } -smp_store_release (&rtorsup->rtorsu_inuse, true); - t =torture_random (&rtorsup->rtorsu_trs) &0xfffff ;// One per million. -if (t <10 *1000 ) - t =200 *1000 *1000 ; -hrtimer_start (&rtorsup->rtorsu_hrt, t, -HRTIMER_MODE_REL |HRTIMER_MODE_SOFT ); +rcu_torture_updown_one (rtorsup); }torture_hrtimeout_ms (1 ,1000 , &rcu_torture_updown_rand);stutter_wait ("rcu_torture_updown" );
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 7/14 rcutorture: Comment invocations of tick_dep_set_task()
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
The rcu_torture_reader() and rcu_torture_fwd_prog_cr() functions run CPU-bound for extended periods of time (tens or even hundreds of milliseconds), so they invoke tick_dep_set_task() and tick_dep_clear_task() to ensure that the scheduling-clock tick helps move grace periods forward.
So why doesn't rcu_torture_fwd_prog_nr() also invoke tick_dep_set_task() and tick_dep_clear_task()? Because the point of this function is to test RCU's ability to (eventually) force grace periods forward even when the tick has been disabled during long CPU-bound kernel execution.
--- kernel/rcu/rcutorture.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/rcu/rcutorture.c +2 -2
@@ -2410,7 +2410,7 @@ rcu_torture_reader(void *arg)set_user_nice (current,MAX_NICE );if (irqreader && cur_ops->irq_capable)timer_setup_on_stack (&t, rcu_torture_timer,0 ); -tick_dep_set_task (current,TICK_DEP_BIT_RCU ); +tick_dep_set_task (current,TICK_DEP_BIT_RCU );// CPU bound, so need tick. do { if (irqreader && cur_ops->irq_capable) { if (!timer_pending(&t))
@@ -3258,7 +3258,7 @@ static void rcu_torture_fwd_prog_cr(struct rcu_fwd *rfp) cver =READ_ONCE (rcu_torture_current_version); gps = cur_ops->get_gp_seq (); rfp->rcu_launder_gp_seq_start = gps; -tick_dep_set_task (current,TICK_DEP_BIT_RCU ); +tick_dep_set_task (current,TICK_DEP_BIT_RCU );// CPU bound, so need tick. while (time_before (jiffies, stopat) && !shutdown_time_arrived () && !READ_ONCE (rcu_fwd_emergency_stop) && !torture_must_stop ()) {
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 8/14 rcutorture: Complain if an ->up_read() is delayed more than 10 seconds
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
The down/up SRCU reader testing uses an hrtimer handler to exit the SRCU read-side critical section. This might be delayed, and if delayed for too long, it can prevent the rcutorture run from completing. This commit therefore complains if the hrtimer handler is delayed for more than ten seconds.
--- kernel/rcu/rcutorture.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/rcu/rcutorture.c +5 -1
@@ -2438,6 +2438,7 @@ rcu_torture_reader(void *arg)struct rcu_torture_one_read_state_updown {struct hrtimer rtorsu_hrt;bool rtorsu_inuse; +unsigned long rtorsu_j;struct torture_random_state rtorsu_trs;struct rcu_torture_one_read_state rtorsu_rtors; };
@@ -2520,6 +2521,7 @@ static void rcu_torture_updown_one(struct rcu_torture_one_read_state_updown *rtoschedule_timeout_idle (HZ );return ; } + rtorsup->rtorsu_j = jiffies;smp_store_release (&rtorsup->rtorsu_inuse, true); t =torture_random (&rtorsup->rtorsu_trs) &0xfffff ;// One per million. if (t <10 *1000 )
@@ -2542,8 +2544,10 @@ rcu_torture_updown(void *arg)for (rtorsup = updownreaders; rtorsup < &updownreaders[n_up_down]; rtorsup++) {if (torture_must_stop ())break ; -if (smp_load_acquire (&rtorsup->rtorsu_inuse)) +if (smp_load_acquire (&rtorsup->rtorsu_inuse)) { +WARN_ON_ONCE (time_after (jiffies, rtorsup->rtorsu_j +10 *HZ ));continue ; + }rcu_torture_updown_one (rtorsup); }torture_hrtimeout_ms (1 ,1000 , &rcu_torture_updown_rand);
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 9/14 rcutorture: Check for ->up_read() without matching ->down_read()
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
This commit creates counters in the rcu_torture_one_read_state_updown structure that check for a call to ->up_read() that lacks a matching call to ->down_read().
While in the area, add end-of-run cleanup code that prevents calls to rcu_torture_updown_hrt() from happening after the test has moved on. Yes, the srcu_barrier() at the end of the test will wait for them, but this could result in confusing states, statistics, and diagnostic information. So explicitly wait for them before we get to the end-of-test output.
[ paulmck: Apply kernel test robot feedback. ]
--- kernel/rcu/rcutorture.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)
kernel/rcu/rcutorture.c +13 -2
@@ -2439,6 +2439,8 @@ struct rcu_torture_one_read_state_updown {struct hrtimer rtorsu_hrt;bool rtorsu_inuse;unsigned long rtorsu_j; +unsigned long rtorsu_ndowns; +unsigned long rtorsu_nups;struct torture_random_state rtorsu_trs;struct rcu_torture_one_read_state rtorsu_rtors; };
@@ -2453,6 +2455,8 @@ static enum hrtimer_restart rcu_torture_updown_hrt(struct hrtimer *hrtp) rtorsup = container_of(hrtp, struct rcu_torture_one_read_state_updown, rtorsu_hrt);rcu_torture_one_read_end (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 ); +WARN_ONCE (rtorsup->rtorsu_nups >= rtorsup->rtorsu_ndowns,"%s: Up without matching down #%zu.\n" , __func__, rtorsup - updownreaders); + rtorsup->rtorsu_nups++;smp_store_release (&rtorsup->rtorsu_inuse, false);return HRTIMER_NORESTART ; }
@@ -2497,8 +2501,12 @@ static void rcu_torture_updown_cleanup(void)for (rtorsup = updownreaders; rtorsup < &updownreaders[n_up_down]; rtorsup++) {if (!smp_load_acquire (&rtorsup->rtorsu_inuse))continue ; -if (!hrtimer_cancel (&rtorsup->rtorsu_hrt)) -WARN_ON_ONCE (rtorsup->rtorsu_inuse); +if (hrtimer_cancel (&rtorsup->rtorsu_hrt) ||WARN_ON_ONCE (rtorsup->rtorsu_inuse)) { +rcu_torture_one_read_end (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 ); +WARN_ONCE (rtorsup->rtorsu_nups >= rtorsup->rtorsu_ndowns,"%s: Up without matching down #%zu.\n" , __func__, rtorsup - updownreaders); + rtorsup->rtorsu_nups++; +smp_store_release (&rtorsup->rtorsu_inuse, false); + } }kfree (updownreaders);
@@ -2514,10 +2522,13 @@ static void rcu_torture_updown_one(struct rcu_torture_one_read_state_updown *rtoinit_rcu_torture_one_read_state (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs); rawidx = cur_ops->down_read (); + rtorsup->rtorsu_ndowns++; idx = (rawidx <<RCUTORTURE_RDR_SHIFT_1 ) &RCUTORTURE_RDR_MASK_1 ; rtorsup->rtorsu_rtors.readstate = idx |RCUTORTURE_RDR_UPDOWN ; rtorsup->rtorsu_rtors.rtrsp++;if (!rcu_torture_one_read_start (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 )) { +WARN_ONCE (rtorsup->rtorsu_nups >= rtorsup->rtorsu_ndowns,"%s: Up without matching down #%zu.\n" , __func__, rtorsup - updownreaders); + rtorsup->rtorsu_nups++;schedule_timeout_idle (HZ );return ; }
Signature · 1 line
2.43.0
Joel Fernandes
maintainerRe: [PATCH 09/14] rcutorture: Check for ->up_read() without matching ->down_read()
Quoted 6 lines · Joel Fernandes
From: "Paul E. McKenney" <[email protected]>
This commit creates counters in the
rcu_torture_one_read_state_updownstructure that check for a call to ->up_read() that lacks a matching call to ->down_read().
Quoted 7 lines
While in the area, add end-of-run cleanup code that prevents calls to
rcu_torture_updown_hrt()from happening after the test has moved on. Yes, thesrcu_barrier()at the end of the test will wait for them, but this could result in confusing states, statistics, and diagnostic information. So explicitly wait for them before we get to the end-of-test output.[ paulmck: Apply kernel test robot feedback. ]
Fyi, I just updated this commit message:
[ joel: Apply Boqun's fix for counter increment ordering. ]
Hope it looks good now [1], thank you Boqun for the fix!
- Joel
Joel Fernandes
maintainerPATCH 10/14 checkpatch: Deprecate srcu_read_lock_lite() and srcu_read_unlock_lite()
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
Uses of srcu_read_lock_lite() and srcu_read_unlock_lite() are better served by the new srcu_read_lock_fast() and srcu_read_unlock_fast() APIs. As in srcu_read_lock_lite() and srcu_read_unlock_lite() would never have happened had I thought a bit harder a few months ago. Therefore, mark them deprecated.
--- scripts/checkpatch.pl | 2 ++ 1 file changed, 2 insertions(+)
scripts/checkpatch.pl +2 -0
@@ -839,6 +839,8 @@ our %deprecated_apis = ( "kunmap" => "kunmap_local", "kmap_atomic" => "kmap_local_page", "kunmap_atomic" => "kunmap_local", + "srcu_read_lock_lite" => "srcu_read_lock_fast", + "srcu_read_unlock_lite" => "srcu_read_unlock_fast", ); #Create a search pattern for all these strings to speed up a loop below
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 11/14 torture: Add --do-{,no-}normal to torture.sh
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
Right now, torture.sh runs normal runs unconditionally, which can be slow and thus annoying when you only want to test --kcsan or --kasan runs. This commit therefore adds a --do-normal argument so that "--kcsan --do-no-kasan --do-no-normal" runs only KCSAN runs. Note that specifying "--do-no-kasan --do-no-kcsan --do-no-normal" gets normal runs, so you should not try to use this as a synonym for --do-none.
--- .../selftests/rcutorture/bin/torture.sh | 30 +++++++++++++++++-- 1 file changed, 27 insertions(+), 3 deletions(-)
tools/testing/selftests/rcutorture/bin/torture.sh +27 -3
@@ -51,6 +51,8 @@ do_scftorture=yes do_rcuscale=yes do_refscale=yes do_kvfree=yes +do_normal=yes +explicit_normal=no do_kasan=yes do_kcsan=no do_clocksourcewd=yes
@@ -128,6 +130,8 @@ do do_refscale=yes do_rt=yes do_kvfree=yes + do_normal=yes + explicit_normal=no do_kasan=yes do_kcsan=yes do_clocksourcewd=yes
@@ -161,11 +165,17 @@ do do_refscale=no do_rt=no do_kvfree=no + do_normal=no + explicit_normal=no do_kasan=no do_kcsan=no do_clocksourcewd=no do_srcu_lockdep=no ;; + --do-normal|--do-no-normal|--no-normal) + do_normal=`doyesno "$1" --do-normal` + explicit_normal=yes + ;; --do-rcuscale|--do-no-rcuscale|--no-rcuscale) do_rcuscale=`doyesno "$1" --do-rcuscale` ;;
@@ -242,6 +252,17 @@ trap 'rm -rf $T' 0 2 echo " --- " $scriptname $args | tee -a $T/log echo " --- Results directory: " $ds | tee -a $T/log +if test "$do_normal" = "no" && test "$do_kasan" = "no" && test "$do_kcsan" = "no" +then + # Match old scripts so that "--do-none --do-rcutorture" does + # normal rcutorture testing, but no KASAN or KCSAN testing. + if test $explicit_normal = yes + then + echo " --- Everything disabled, so explicit --do-normal overridden" | tee -a $T/log + fi + do_normal=yes +fi + # Calculate rcutorture defaults and apportion time if test -z "$configs_rcutorture" then
@@ -332,9 +353,12 @@ function torture_set { local kcsan_kmake_tag= local flavor=$1 shift - curflavor=$flavor - torture_one "$@" - mv $T/last-resdir $T/last-resdir-nodebug || : + if test "$do_normal" = "yes" + then + curflavor=$flavor + torture_one "$@" + mv $T/last-resdir $T/last-resdir-nodebug || : + fi if test "$do_kasan" = "yes" then curflavor=${flavor}-kasan
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 12/14 torture: Add testing of RCU's Rust bindings to torture.sh
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
This commit adds a --do-rcu-rust parameter to torture.sh, which invokes a rust_doctests_kernel kunit run. Note that kunit wants a clean source tree, so this runs "make mrproper", which might come as a surprise to some users. Should there be a --mrproper parameter to torture.sh to make the user explicitly ask for it?
--- .../selftests/rcutorture/bin/torture.sh | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+)
tools/testing/selftests/rcutorture/bin/torture.sh +45 -0
@@ -59,6 +59,7 @@ do_clocksourcewd=yes do_rt=yes do_rcutasksflavors=yes do_srcu_lockdep=yes +do_rcu_rust=no # doyesno - Helper function for yes/no arguments function doyesno () {
@@ -89,6 +90,7 @@ usage () { echo " --do-rcutorture / --do-no-rcutorture / --no-rcutorture" echo " --do-refscale / --do-no-refscale / --no-refscale" echo " --do-rt / --do-no-rt / --no-rt" + echo " --do-rcu-rust / --do-no-rcu-rust / --no-rcu-rust" echo " --do-scftorture / --do-no-scftorture / --no-scftorture" echo " --do-srcu-lockdep / --do-no-srcu-lockdep / --no-srcu-lockdep" echo " --duration [ <minutes> | <hours>h | <days>d ]"
@@ -191,6 +193,9 @@ do --do-rt|--do-no-rt|--no-rt) do_rt=`doyesno "$1" --do-rt` ;; + --do-rcu-rust|--do-no-rcu-rust|--no-rcu-rust) + do_rcu_rust=`doyesno "$1" --do-rcu-rust` + ;; --do-scftorture|--do-no-scftorture|--no-scftorture) do_scftorture=`doyesno "$1" --do-scftorture` ;;
@@ -485,6 +490,46 @@ then torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_FULL=y CONFIG_RCU_NOCB_CPU=y" --trust-make fi +if test "$do_rcu_rust" = "yes" +then + echo " --- do-rcu-rust:" Start `date` | tee -a $T/log + rrdir="tools/testing/selftests/rcutorture/res/$ds/results-rcu-rust" + mkdir -p "$rrdir" + echo " --- make LLVM=1 rustavailable " | tee -a $rrdir/log > $rrdir/rustavailable.out + make LLVM=1 rustavailable > $T/rustavailable.out 2>&1 + retcode=$? + echo $retcode > $rrdir/rustavailable.exitcode + cat $T/rustavailable.out | tee -a $rrdir/log >> $rrdir/rustavailable.out 2>&1 + buildphase=rustavailable + if test "$retcode" -eq 0 + then + echo " --- Running 'make mrproper' in order to run kunit." | tee -a $rrdir/log > $rrdir/mrproper.out + make mrproper > $rrdir/mrproper.out 2>&1 + retcode=$? + echo $retcode > $rrdir/mrproper.exitcode + buildphase=mrproper + fi + if test "$retcode" -eq 0 + then + echo " --- Running rust_doctests_kernel." | tee -a $rrdir/log > $rrdir/rust_doctests_kernel.out + ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --make_options CLIPPY=1 --arch arm64 --kconfig_add CONFIG_SMP=y --kconfig_add CONFIG_WERROR=y --kconfig_add CONFIG_RUST=y rust_doctests_kernel >> $rrdir/rust_doctests_kernel.out 2>&1 + # @@@ Remove "--arch arm64" in order to test on native architecture? + # @@@ Analyze $rrdir/rust_doctests_kernel.out contents? + retcode=$? + echo $retcode > $rrdir/rust_doctests_kernel.exitcode + buildphase=rust_doctests_kernel + fi + if test "$retcode" -eq 0 + then + echo "rcu-rust($retcode)" $rrdir >> $T/successes + echo Success >> $rrdir/log + else + echo "rcu-rust($retcode)" $rrdir >> $T/failures + echo " --- rcu-rust Test summary:" >> $rrdir/log + echo " --- Summary: Exit code $retcode from $buildphase, see $rrdir/$buildphase.out" >> $rrdir/log + fi +fi + if test "$do_srcu_lockdep" = "yes" then echo " --- do-srcu-lockdep:" Start `date` | tee -a $T/log
Signature · 1 line
2.43.0
Re: [PATCH 12/14] torture: Add testing of RCU's Rust bindings to torture.sh
Quoted 5 lines · Joel Fernandes
a
rust_doctests_kernelkunit run. Note that kunit wants a clean source tree, so this runs "make mrproper", which might come as a surprise to some users. Should there be a --mrproper parameter totorture.shto make the user explicitly ask for it?
One may run KUnit without using kunit.py, i.e. by enabling the Kconfigs and booting the kernel. Would that help?
Cheers,
Miguel
Paul E. McKenney
maintainerRe: [PATCH 12/14] torture: Add testing of RCU's Rust bindings to torture.sh
Quoted 9 lines · Miguel Ojeda
Quoted 5 lines · Joel Fernandes
a
rust_doctests_kernelkunit run. Note that kunit wants a clean source tree, so this runs "make mrproper", which might come as a surprise to some users. Should there be a --mrproper parameter totorture.shto make the user explicitly ask for it?One may run KUnit without using
kunit.py, i.e. by enabling the Kconfigs and booting the kernel. Would that help?
Potentially?
Suppose we fired up a guest OS and captured the console output. Is there a way to make that guest OS shut down automatically at the end of the test and to extract the test results?
In the short term, I am good with this the current state of this being default-disabled and those of us wishing to run it regularly adjusting our scripts accordingly.
But, longer term, if there is a better way...
Thanx, Paul
Re: [PATCH 12/14] torture: Add testing of RCU's Rust bindings to torture.sh
Quoted 4 lines · Paul E. McKenney
Suppose we fired up a guest OS and captured the console output. Is there a way to make that guest OS shut down automatically at the end of the test and to extract the test results?
Ah, sorry, I thought you were already doing something like that, i.e. that perhaps you could reuse some kernel build you already had and avoiding a full rebuild/mrproper. The KUnit Python script uses QEMU and parses the results; e.g. you could look for the results lines like:
# Totals: pass:133 fail:0 skip:0 total:133
ok 2 rust_doctests_kernel
Cheers,
Miguel
Paul E. McKenney
maintainerRe: [PATCH 12/14] torture: Add testing of RCU's Rust bindings to torture.sh
Quoted 6 lines · Miguel Ojeda
Quoted 4 lines · Paul E. McKenney
Suppose we fired up a guest OS and captured the console output. Is there a way to make that guest OS shut down automatically at the end of the test and to extract the test results?
Quoted 8 lines
Ah, sorry, I thought you were already doing something like that, i.e. that perhaps you could reuse some kernel build you already had and avoiding a full rebuild/
mrproper. The KUnit Python script uses QEMU and parses the results; e.g. you could look for the results lines like:# Totals: pass:133 fail:0 skip:0 total:133
ok 2rust_doctests_kernel
Alternatively, I could clone a copy of the current archive into a temporary directory, "make mrproper" there, run kunit normally, then clean up the temporary directory. Extra storage, but quite a bit more robust and user-friendly.
Other thoughts?
Thanx, Paul
Joel Fernandes
maintainerRe: [12/14] torture: Add testing of RCU's Rust bindings to torture.sh
Hello, Paul,
Quoted 15 lines · Paul E. McKenney
Quoted 1 line · Miguel Ojeda
On Fri, Apr 18, 2025 at 8:04 PM Paul E. McKenney <[email protected]> wrot
e:
Quoted 2 lines
Quoted 2 lines
Suppose we fired up a guest OS and captured the console output. Is ther
e
Quoted 9 lines
Quoted 2 lines
a way to make that guest OS shut down automatically at the end of the test and to extract the test results?
Ah, sorry, I thought you were already doing something like that, i.e. that perhaps you could reuse some kernel build you already had and avoiding a full rebuild/mrproper. The KUnit Python script uses QEMU and parses the results; e.g. you could look for the results lines like:
Quoted 8 lines
Quoted 2 lines
# Totals: pass:133 fail:0 skip:0 total:133
ok 2rust_doctests_kernelAlternatively, I could clone a copy of the current archive into a temporary directory, "make mrproper" there, run kunit normally, then clean up the temporary directory. Extra storage, but quite a bit more robust and user-friendly.
Just to be on the same page, is the concern about the slowness of mrproper or that it kills the kernel build artifacts requiring a clean build?
What kind of improvement are we looking for and why would this patch in its current form not work?
thanks,
- Joel
Quoted 3 lines
Other thoughts?
Thanx, Paul
Paul E. McKenney
maintainerRe: [12/14] torture: Add testing of RCU's Rust bindings to torture.sh
Quoted 21 lines · Joel Fernandes
Hello, Paul,
Quoted 18 lines · Paul E. McKenney
Quoted 1 line · Miguel Ojeda
On Fri, Apr 18, 2025 at 8:04 PM Paul E. McKenney <[email protected]> wrot
e:
Quoted 2 lines
Quoted 2 lines
Suppose we fired up a guest OS and captured the console output. Is ther
e
Quoted 11 lines
Quoted 2 lines
a way to make that guest OS shut down automatically at the end of the test and to extract the test results?
Ah, sorry, I thought you were already doing something like that, i.e. that perhaps you could reuse some kernel build you already had and avoiding a full rebuild/mrproper. The KUnit Python script uses QEMU and parses the results; e.g. you could look for the results lines like:
# Totals: pass:133 fail:0 skip:0 total:133
ok 2rust_doctests_kernel
Quoted 8 lines
Quoted 4 lines
Alternatively, I could clone a copy of the current archive into a temporary directory, "make mrproper" there, run kunit normally, then clean up the temporary directory. Extra storage, but quite a bit more robust and user-friendly.
Just to be on the same page, is the concern about the slowness of mrproper or that it kills the kernel build artifacts requiring a clean build?
It blows away things like tags and cscope databases. Me, I have my cscope databases elsewhere, but lots of people build them for their live git repos. And they are (quite understandably) unhappy when their source-code browsing tools are blown away by some random test doing an unsuspected "make mrproper". ;-)
Quoted 2 lines
What kind of improvement are we looking for and why would this patch in its current form not work?
For the near term, I am OK with its current form because it is non-default. Longer term, though, we need it to be tested by default, and that means making it avoid undoing people's work. My short-term approach is to enable this test on my employer's test systems, which have Rust set up correctly, and skip it on my laptop, which has a strange FrankenRust due to my early playing around with that language.
Thanx, Paul
Joel Fernandes
maintainerRe: [12/14] torture: Add testing of RCU's Rust bindings to torture.sh
Hi Paul,
Quoted 21 lines · Paul E. McKenney
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 3 lines
Suppose we fired up a guest OS and captured the console output. Is there a way to make that guest OS shut down automatically at the end of the test and to extract the test results?
Ah, sorry, I thought you were already doing something like that, i.e. that perhaps you could reuse some kernel build you already had and avoiding a full rebuild/mrproper. The KUnit Python script uses QEMU and parses the results; e.g. you could look for the results lines like:
# Totals: pass:133 fail:0 skip:0 total:133
ok 2rust_doctests_kernelQuoted 8 lines
Quoted 5 lines
Alternatively, I could clone a copy of the current archive into a temporary directory, "make mrproper" there, run kunit normally, then clean up the temporary directory. Extra storage, but quite a bit more robust and user-friendly.
Just to be on the same page, is the concern about the slowness of mrproper or that it kills the kernel build artifacts requiring a clean build?
Quoted 5 lines
It blows away things like tags and cscope databases. Me, I have my cscope databases elsewhere, but lots of people build them for their live git repos. And they are (quite understandably) unhappy when their source-code browsing tools are blown away by some random test doing an unsuspected "make mrproper". 😉
Cool. One thing is, running other test modes in torture.sh also reconfigures the kernel and potentially recompiles the entire kernel as a result. So if someone is already having their own kernel build, running torture.sh already may cause them to have to do a full rebuild, AFAICS. But yes, and to your point, the cscope DB and stuff may get blown away for additional disruption.
Quoted 10 lines
Quoted 3 lines
What kind of improvement are we looking for and why would this patch in its current form not work?
For the near term, I am OK with its current form because it is non-default. Longer term, though, we need it to be tested by default, and that means making it avoid undoing people's work. My short-term approach is to enable this test on my employer's test systems, which have Rust set up correctly, and skip it on my laptop, which has a strange FrankenRust due to my early playing around with that language.
Or we teach kunit.py to not require a mrproper? :-) I wonder why it needs to do that. I may run into that too considering my other kernel project requires me to mess around with rust.
thanks,
- Joel
Paul E. McKenney
maintainerRe: [12/14] torture: Add testing of RCU's Rust bindings to torture.sh
Quoted 46 lines · Joel Fernandes
Hi Paul,
Quoted 21 lines · Paul E. McKenney
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 3 lines
Suppose we fired up a guest OS and captured the console output. Is there a way to make that guest OS shut down automatically at the end of the test and to extract the test results?
Ah, sorry, I thought you were already doing something like that, i.e. that perhaps you could reuse some kernel build you already had and avoiding a full rebuild/mrproper. The KUnit Python script uses QEMU and parses the results; e.g. you could look for the results lines like:
# Totals: pass:133 fail:0 skip:0 total:133
ok 2rust_doctests_kernelQuoted 8 lines
Quoted 5 lines
Alternatively, I could clone a copy of the current archive into a temporary directory, "make mrproper" there, run kunit normally, then clean up the temporary directory. Extra storage, but quite a bit more robust and user-friendly.
Just to be on the same page, is the concern about the slowness of mrproper or that it kills the kernel build artifacts requiring a clean build?
Quoted 5 lines
It blows away things like tags and cscope databases. Me, I have my cscope databases elsewhere, but lots of people build them for their live git repos. And they are (quite understandably) unhappy when their source-code browsing tools are blown away by some random test doing an unsuspected "make mrproper". 😉
Cool. One thing is, running other test modes in
torture.shalso reconfigures the kernel and potentially recompiles the entire kernel as a result. So if someone is already having their own kernel build, runningtorture.shalready may cause them to have to do a full rebuild, AFAICS. But yes, and to your point, the cscope DB and stuff may get blown away for additional disruption.Quoted 10 lines
Quoted 3 lines
What kind of improvement are we looking for and why would this patch in its current form not work?
For the near term, I am OK with its current form because it is non-default. Longer term, though, we need it to be tested by default, and that means making it avoid undoing people's work. My short-term approach is to enable this test on my employer's test systems, which have Rust set up correctly, and skip it on my laptop, which has a strange FrankenRust due to my early playing around with that language.
Quoted 3 lines
Or we teach
kunit.pyto not require a mrproper? :-) I wonder why it needs to do that. I may run into that too considering my other kernel project requires me to mess around with rust.
They might have a good reason for that "make mrproper", but it does sound like a good avenue to investigate. Otherwise, as noted earlier, my thought is to clone the repo into a temp directory, run "make mrproper" there, continue being a non-special user of kunit, and avoid messing up people's databases and tags.
Thanx, Paul
Joel Fernandes
maintainerRe: [12/14] torture: Add testing of RCU's Rust bindings to torture.sh
In reply to Paul E. McKenneyHello, Paul,
Quoted 15 lines · Paul E. McKenney
Quoted 1 line · Miguel Ojeda
On Fri, Apr 18, 2025 at 8:04 PM Paul E. McKenney <[email protected]> wrot
e:
Quoted 2 lines
Quoted 2 lines
Suppose we fired up a guest OS and captured the console output. Is ther
e
Quoted 9 lines
Quoted 2 lines
a way to make that guest OS shut down automatically at the end of the test and to extract the test results?
Ah, sorry, I thought you were already doing something like that, i.e. that perhaps you could reuse some kernel build you already had and avoiding a full rebuild/mrproper. The KUnit Python script uses QEMU and parses the results; e.g. you could look for the results lines like:
Quoted 8 lines
Quoted 2 lines
# Totals: pass:133 fail:0 skip:0 total:133
ok 2rust_doctests_kernelAlternatively, I could clone a copy of the current archive into a temporary directory, "make mrproper" there, run kunit normally, then clean up the temporary directory. Extra storage, but quite a bit more robust and user-friendly.
Just to be on the same page, is the concern about the slowness of mrproper or that it kills the kernel build artifacts requiring a clean build?
What kind of improvement are we looking for and why would this patch in its current form not work?
thanks,
- Joel
Quoted 3 lines
Other thoughts?
Thanx, Paul
Joel Fernandes
maintainerPATCH 13/14 rcutorture: Perform more frequent testing of ->gpwrap
In reply to Joel FernandesCurrently, the ->gpwrap is not tested (at all per my testing) due to the requirement of a large delta between a CPU's rdp->gp_seq and its node's rnp->gpseq.
This results in no testing of ->gpwrap being set. This patch by default adds 5 minutes of testing with ->gpwrap forced by lowering the delta between rdp->gp_seq and rnp->gp_seq to just 8 GPs. All of this is configurable, including the active time for the setting and a full testing cycle.
By default, the first 25 minutes of a test will have the _default_ behavior there is right now (ULONG_MAX / 4) delta. Then for 5 minutes, we switch to a smaller delta causing 1-2 wraps in 5 minutes. I believe this is reasonable since we at least add a little bit of testing for usecases where ->gpwrap is set.
--- kernel/rcu/rcu.h | 4 +++ kernel/rcu/rcutorture.c | 68 ++++++++++++++++++++++++++++++++++++++++- kernel/rcu/tree.c | 34 +++++++++++++++++++-- kernel/rcu/tree.h | 1 + 4 files changed, 104 insertions(+), 3 deletions(-)
kernel/rcu/rcu.h +4 -0
@@ -572,6 +572,8 @@ void do_trace_rcu_torture_read(const char *rcutorturename,unsigned long c_old, unsigned long c);void rcu_gp_set_torture_wait (int duration); +void rcu_set_gpwrap_lag (unsigned long lag); +int rcu_get_gpwrap_count (int cpu);#else static inline void rcutorture_get_gp_data (int *flags,unsigned long *gp_seq) {
@@ -589,6 +591,8 @@ void do_trace_rcu_torture_read(const char *rcutorturename, do { } while (0)#endif static inline void rcu_gp_set_torture_wait (int duration) { } +static inline void rcu_set_gpwrap_lag (unsigned long lag) { } +static inline int rcu_get_gpwrap_count (int cpu) {return 0 ; }#endif unsigned long long rcutorture_gather_gp_seqs (void );void rcutorture_format_gp_seqs (unsigned long long seqs,char *cp,size_t len);
kernel/rcu/rcutorture.c +67 -1
123 lines not on this page · raw message
kernel/rcu/tree.c +32 -2
@@ -80,6 +80,15 @@ static void rcu_sr_normal_gp_cleanup_work(struct work_struct *); static DEFINE_PER_CPU_SHARED_ALIGNED(struct rcu_data, rcu_data) = { .gpwrap = true, }; + +int rcu_get_gpwrap_count (int cpu) +{ +struct rcu_data *rdp =per_cpu_ptr (&rcu_data, cpu); + +return READ_ONCE (rdp->gpwrap_count); +} +EXPORT_SYMBOL_GPL (rcu_get_gpwrap_count); +static struct rcu_state rcu_state = { .level = { &rcu_state.node[0 ] }, .gp_state =RCU_GP_IDLE ,
@@ -757,6 +766,25 @@ void rcu_request_urgent_qs_task(struct task_struct *t)smp_store_release (per_cpu_ptr (&rcu_data.rcu_urgent_qs, cpu), true); } +/** +* rcu_set_gpwrap_lag - Set RCU GP sequence overflow lag value. +* @lag_gps: Set overflow lag to this many grace period worth of counters +* which is used by rcutorture to quickly force a gpwrap situation. +* @lag_gps = 0 means we reset it back to the boot-time value. +*/ +static unsigned long seq_gpwrap_lag =ULONG_MAX /4 ; + +void rcu_set_gpwrap_lag (unsigned long lag_gps) +{ +unsigned long lag_seq_count; + + lag_seq_count = (lag_gps ==0 ) + ?ULONG_MAX /4 + : lag_gps <<RCU_SEQ_CTR_SHIFT ; +WRITE_ONCE (seq_gpwrap_lag, lag_seq_count); +} +EXPORT_SYMBOL_GPL (rcu_set_gpwrap_lag); +/* * When trying to report a quiescent state on behalf of some other CPU, * it is our responsibility to check for and handle potential overflow
@@ -767,9 +795,11 @@ void rcu_request_urgent_qs_task(struct task_struct *t)static void rcu_gpnum_ovf (struct rcu_node *rnp,struct rcu_data *rdp) {raw_lockdep_assert_held_rcu_node (rnp); -if (ULONG_CMP_LT (rcu_seq_current (&rdp->gp_seq) +ULONG_MAX /4 , - rnp->gp_seq)) +if (ULONG_CMP_LT (rcu_seq_current (&rdp->gp_seq) + seq_gpwrap_lag, + rnp->gp_seq)) {WRITE_ONCE (rdp->gpwrap, true); +WRITE_ONCE (rdp->gpwrap_count,READ_ONCE (rdp->gpwrap_count) +1 ); + }if (ULONG_CMP_LT (rdp->rcu_iw_gp_seq +ULONG_MAX /4 , rnp->gp_seq)) rdp->rcu_iw_gp_seq = rnp->gp_seq +ULONG_MAX /4 ; }
kernel/rcu/tree.h +1 -0
@@ -183,6 +183,7 @@ struct rcu_data {bool core_needs_qs;/* Core waits for quiescent state. */ bool beenonline;/* CPU online at least once. */ bool gpwrap;/* Possible ->gp_seq wrap. */ +unsigned int gpwrap_count;/* Count of GP sequence wrap. */ bool cpu_started;/* RCU watching this onlining CPU. */ struct rcu_node *mynode;/* This CPU's leaf of hierarchy */ unsigned long grpmask;/* Mask to apply to leaf qsmask. */
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 14/14 rcutorture: Fix issue with re-using old images on ARM64
In reply to Joel FernandesOn ARM64, when running with --configs '36*SRCU-P', I noticed that only 1 instance instead of 36 for starting.
Fix it by checking for Image files, instead of bzImage which ARM does not seem to have. With this I see all 36 instances running at the same time in the batch.
--- tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh +1 -1
@@ -73,7 +73,7 @@ config_override_param "$config_dir/CFcommon.$(uname -m)" KcList \ cp $T/KcList $resdir/ConfigFragment base_resdir=`echo $resdir | sed -e 's/\.[0-9]\+$//'` -if test "$base_resdir" != "$resdir" && test -f $base_resdir/bzImage && test -f $base_resdir/vmlinux +if test "$base_resdir" != "$resdir" && (test -f $base_resdir/bzImage || test -f $base_resdir/Image) && test -f $base_resdir/vmlinux then # Rerunning previous test, so use that test's kernel. QEMU="`identify_qemu $base_resdir/vmlinux`"
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 1/12 rcutorture: Make srcu_lockdep.sh check kernel Kconfig
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
The srcu_lockdep.sh currently blindly trusts the rcutorture SRCU-P scenario to build its kernel with lockdep enabled. Of course, this dependency might not be obvious to someone rebalancing SRCU scenarios. This commit therefore adds code to srcu_lockdep.sh that verifies that the .config file has lockdep enabled.
--- .../testing/selftests/rcutorture/bin/srcu_lockdep.sh | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
tools/testing/selftests/rcutorture/bin/srcu_lockdep.sh +10 -1
@@ -39,8 +39,9 @@ do shift done -err= nerrs=0 + +# Test lockdep's handling of deadlocks. for d in 0 1 do for t in 0 1 2
@@ -52,6 +53,12 @@ do tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 5s --configs "SRCU-P" --kconfig "CONFIG_FORCE_NEED_SRCU_NMI_SAFE=y" --bootargs "rcutorture.test_srcu_lockdep=$val rcutorture.reader_flavor=0x2" --trust-make --datestamp "$ds/$val" > "$T/kvm.sh.out" 2>&1 ret=$? mv "$T/kvm.sh.out" "$RCUTORTURE/res/$ds/$val" + if ! grep -q '^CONFIG_PROVE_LOCKING=y' .config + then + echo "rcu_torture_init_srcu_lockdep:Error: CONFIG_PROVE_LOCKING disabled in rcutorture SRCU-P scenario" + nerrs=$((nerrs+1)) + err=1 + fi if test "$d" -ne 0 && test "$ret" -eq 0 then err=1
@@ -71,6 +78,8 @@ do done done done + +# Set up exit code. if test "$nerrs" -ne 0 then exit 1
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 2/12 rcutorture: Make srcu_lockdep.sh check reader-conflict handling
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
Mixing different flavors of RCU readers is forbidden, for example, you should not use srcu_read_lock() and srcu_read_lock_nmisafe() on the same srcu_struct structure. There are checks for this, but these checks are not tested on a regular basis. This commit therefore adds such tests to srcu_lockdep.sh.
--- .../selftests/rcutorture/bin/srcu_lockdep.sh | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+)
tools/testing/selftests/rcutorture/bin/srcu_lockdep.sh +31 -0
@@ -79,6 +79,37 @@ do done done +# Test lockdep-enabled testing of mixed SRCU readers. +for val in 0x1 0xf +do + err= + tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 5s --configs "SRCU-P" --kconfig "CONFIG_FORCE_NEED_SRCU_NMI_SAFE=y" --bootargs "rcutorture.reader_flavor=$val" --trust-make --datestamp "$ds/$val" > "$T/kvm.sh.out" 2>&1 + ret=$? + mv "$T/kvm.sh.out" "$RCUTORTURE/res/$ds/$val" + if ! grep -q '^CONFIG_PROVE_LOCKING=y' .config + then + echo "rcu_torture_init_srcu_lockdep:Error: CONFIG_PROVE_LOCKING disabled in rcutorture SRCU-P scenario" + nerrs=$((nerrs+1)) + err=1 + fi + if test "$val" -eq 0xf && test "$ret" -eq 0 + then + err=1 + echo -n Unexpected success for > "$RCUTORTURE/res/$ds/$val/kvm.sh.err" + fi + if test "$val" -eq 0x1 && test "$ret" -ne 0 + then + err=1 + echo -n Unexpected failure for > "$RCUTORTURE/res/$ds/$val/kvm.sh.err" + fi + if test -n "$err" + then + grep "rcu_torture_init_srcu_lockdep: test_srcu_lockdep = " "$RCUTORTURE/res/$ds/$val/SRCU-P/console.log" | sed -e 's/^.*rcu_torture_init_srcu_lockdep://' >> "$RCUTORTURE/res/$ds/$val/kvm.sh.err" + cat "$RCUTORTURE/res/$ds/$val/kvm.sh.err" + nerrs=$((nerrs+1)) + fi +done + # Set up exit code. if test "$nerrs" -ne 0 then
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 3/12 rcutorture: Split out beginning and end from rcu_torture_one_read()
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
The rcu_torture_one_read() function is designed for RCU readers that are confined to a task, such that a single thread of control extends from the beginning of a given RCU read-side critical section to its end. This does not suffice for things like srcu_down_read() and srcu_up_read(), where the critical section might start at task level and end in a timer handler.
This commit therefore creates separate init_rcu_torture_one_read_state(), rcu_torture_one_read_start(), and rcu_torture_one_read_end() functions, along with a rcu_torture_one_read_state structure to coordinate their actions. These will be used to create tests for srcu_down_read() and friends.
One caution: The caller to rcu_torture_one_read_start() must enter the initial read-side critical section prior to the call. This enables use of non-standard primitives such as srcu_down_read() while still using the same validation code.
--- kernel/rcu/rcutorture.c | 124 ++++++++++++++++++++++++++-------------- 1 file changed, 81 insertions(+), 43 deletions(-)
kernel/rcu/rcutorture.c +81 -43
@@ -2164,53 +2164,70 @@ rcutorture_loop_extend(int *readstate, bool insoftirq, struct torture_random_stareturn &rtrsp[j]; } -/* -* Do one read-side critical section, returning false if there was -* no data to read. Can be invoked both from process context and -* from a timer handler. -*/ -static bool rcu_torture_one_read (struct torture_random_state *trsp,long myid) -{ -bool checkpolling = !(torture_random (trsp) &0xfff ); +struct rcu_torture_one_read_state { +bool checkpolling;unsigned long cookie;struct rcu_gp_oldstate cookie_full; -int i;unsigned long started; -unsigned long completed; -int newstate;struct rcu_torture *p; -int pipe_count; -bool preempted = false; -int readstate =0 ; -struct rt_read_seg rtseg[RCUTORTURE_RDR_MAX_SEGS ] = { {0 } }; -struct rt_read_seg *rtrsp = &rtseg[0 ]; -struct rt_read_seg *rtrsp1; +int readstate; +struct rt_read_seg rtseg[RCUTORTURE_RDR_MAX_SEGS ]; +struct rt_read_seg *rtrsp;unsigned long long ts; +}; -WARN_ON_ONCE (!rcu_is_watching ()); - newstate =rcutorture_extend_mask (readstate, trsp); -rcutorture_one_extend (&readstate, newstate, myid <0 , trsp, rtrsp++); -if (checkpolling) { +static void init_rcu_torture_one_read_state (struct rcu_torture_one_read_state *rtorsp, +struct torture_random_state *trsp) +{ +memset (rtorsp,0 ,sizeof (*rtorsp)); + rtorsp->checkpolling = !(torture_random (trsp) &0xfff ); + rtorsp->rtrsp = &rtorsp->rtseg[0 ]; +} + +/* +* Set up the first segment of a series of overlapping read-side +* critical sections. The caller must have actually initiated the +* outermost read-side critical section. +*/ +static bool rcu_torture_one_read_start (struct rcu_torture_one_read_state *rtorsp, +struct torture_random_state *trsp,long myid) +{ +if (rtorsp->checkpolling) {if (cur_ops->get_gp_state && cur_ops->poll_gp_state) - cookie = cur_ops->get_gp_state (); + rtorsp->cookie = cur_ops->get_gp_state ();if (cur_ops->get_gp_state_full && cur_ops->poll_gp_state_full) - cur_ops->get_gp_state_full (&cookie_full); + cur_ops->get_gp_state_full (&rtorsp->cookie_full); } - started = cur_ops->get_gp_seq (); - ts =rcu_trace_clock_local (); - p =rcu_dereference_check (rcu_torture_current, + rtorsp->started = cur_ops->get_gp_seq (); + rtorsp->ts =rcu_trace_clock_local (); + rtorsp->p =rcu_dereference_check (rcu_torture_current, !cur_ops->readlock_held || cur_ops->readlock_held ()); -if (p ==NULL ) { +if (rtorsp->p ==NULL ) {/* Wait for rcu_torture_writer to get underway */ -rcutorture_one_extend (&readstate,0 , myid <0 , trsp, rtrsp); +rcutorture_one_extend (&rtorsp->readstate,0 , myid <0 , trsp, rtorsp->rtrsp);return false; } -if (p->rtort_mbtest ==0 ) +if (rtorsp->p->rtort_mbtest ==0 )atomic_inc (&n_rcu_torture_mberror); -rcu_torture_reader_do_mbchk (myid, p, trsp); - rtrsp =rcutorture_loop_extend (&readstate, myid <0 , trsp, rtrsp); +rcu_torture_reader_do_mbchk (myid, rtorsp->p, trsp); +return true; +} + +/* +* Complete the last segment of a series of overlapping read-side +* critical sections and check for errors. +*/ +static void rcu_torture_one_read_end (struct rcu_torture_one_read_state *rtorsp, +struct torture_random_state *trsp,long myid) +{ +int i; +unsigned long completed; +int pipe_count; +bool preempted = false; +struct rt_read_seg *rtrsp1; +preempt_disable (); - pipe_count =READ_ONCE (p->rtort_pipe_count); + pipe_count =READ_ONCE (rtorsp->p->rtort_pipe_count);if (pipe_count >RCU_TORTURE_PIPE_LEN ) {// Should not happen in a correct RCU implementation, // happens quite often for torture_type=busted.
@@ -2218,28 +2235,28 @@ static bool rcu_torture_one_read(struct torture_random_state *trsp, long myid) } completed = cur_ops->get_gp_seq ();if (pipe_count >1 ) { -do_trace_rcu_torture_read (cur_ops->name, &p->rtort_rcu, - ts, started, completed); +do_trace_rcu_torture_read (cur_ops->name, &rtorsp->p->rtort_rcu, + rtorsp->ts, rtorsp->started, completed);rcu_ftrace_dump (DUMP_ALL ); }__this_cpu_inc (rcu_torture_count[pipe_count]); - completed =rcutorture_seq_diff (completed, started); + completed =rcutorture_seq_diff (completed, rtorsp->started);if (completed >RCU_TORTURE_PIPE_LEN ) {/* Should not happen, but... */ completed =RCU_TORTURE_PIPE_LEN ; }__this_cpu_inc (rcu_torture_batch[completed]);preempt_enable (); -if (checkpolling) { +if (rtorsp->checkpolling) {if (cur_ops->get_gp_state && cur_ops->poll_gp_state) -WARN_ONCE (cur_ops->poll_gp_state (cookie), +WARN_ONCE (cur_ops->poll_gp_state (rtorsp->cookie),"%s: Cookie check 2 failed %s(%d) %lu->%lu\n" , __func__,rcu_torture_writer_state_getname (), rcu_torture_writer_state, - cookie, cur_ops->get_gp_state ()); + rtorsp->cookie, cur_ops->get_gp_state ());if (cur_ops->get_gp_state_full && cur_ops->poll_gp_state_full) - WARN_ONCE(cur_ops->poll_gp_state_full(&cookie_full), + WARN_ONCE(cur_ops->poll_gp_state_full(&rtorsp->cookie_full), "%s: Cookie check 6 failed %s(%d) online %*pbl\n", __func__, rcu_torture_writer_state_getname(),
@@ -2248,21 +2265,42 @@ static bool rcu_torture_one_read(struct torture_random_state *trsp, long myid) }if (cur_ops->reader_blocked) preempted = cur_ops->reader_blocked (); -rcutorture_one_extend (&readstate,0 , myid <0 , trsp, rtrsp); -WARN_ON_ONCE (readstate); +rcutorture_one_extend (&rtorsp->readstate,0 , myid <0 , trsp, rtorsp->rtrsp); +WARN_ON_ONCE (rtorsp->readstate);// This next splat is expected behavior if leakpointer, especially // for CONFIG_RCU_STRICT_GRACE_PERIOD=y kernels. -WARN_ON_ONCE (leakpointer &&READ_ONCE (p->rtort_pipe_count) >1 ); +WARN_ON_ONCE (leakpointer &&READ_ONCE (rtorsp->p->rtort_pipe_count) >1 );/* If error or close call, record the sequence of reader protections. */ if ((pipe_count >1 || completed >1 ) && !xchg (&err_segs_recorded,1 )) { i =0 ; -for (rtrsp1 = &rtseg[0 ]; rtrsp1 < rtrsp; rtrsp1++) +for (rtrsp1 = &rtorsp->rtseg[0 ]; rtrsp1 < rtorsp->rtrsp; rtrsp1++) err_segs[i++] = *rtrsp1; rt_read_nsegs = i; rt_read_preempted = preempted; } +} +/* +* Do one read-side critical section, returning false if there was +* no data to read. Can be invoked both from process context and +* from a timer handler. +*/ +static bool rcu_torture_one_read (struct torture_random_state *trsp,long myid) +{ +int newstate; +struct rcu_torture_one_read_state rtors; + +WARN_ON_ONCE (!rcu_is_watching ()); +init_rcu_torture_one_read_state (&rtors, trsp); + newstate =rcutorture_extend_mask (rtors.readstate, trsp); +rcutorture_one_extend (&rtors.readstate, newstate, myid <0 , trsp, rtors.rtrsp++); +if (!rcu_torture_one_read_start (&rtors, trsp, myid)) { +rcutorture_one_extend (&rtors.readstate,0 , myid <0 , trsp, rtors.rtrsp); +return false; + } + rtors.rtrsp =rcutorture_loop_extend (&rtors.readstate, myid <0 , trsp, rtors.rtrsp); +rcu_torture_one_read_end (&rtors, trsp, myid);return true; }
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 4/12 rcutorture: Make torture.sh --do-rt use CONFIG_PREEMPT_RT
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
The torture.sh --do-rt command-line parameter is intended to mimic -rt kernels. Now that CONFIG_PREEMPT_RT is upstream, this commit makes this mimicking more precise.
Note that testing of RCU priority boosting is disabled in favor of forward-progress testing of RCU callbacks. If it turns out to be possible to make kernels built with CONFIG_PREEMPT_RT=y to tolerate testing of both, both will be enabled.
[ paulmck: Apply Sebastian Siewior feedback. ]
--- tools/testing/selftests/rcutorture/bin/torture.sh | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
tools/testing/selftests/rcutorture/bin/torture.sh +9 -5
@@ -448,13 +448,17 @@ fi if test "$do_rt" = "yes" then - # With all post-boot grace periods forced to normal. - torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcupdate.rcu_normal=1" - torture_set "rcurttorture" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --trust-make + # In both runs, disable testing of RCU priority boosting because + # -rt doesn't like its interaction with testing of callback + # flooding. + + # With all post-boot grace periods forced to normal (default for PREEMPT_RT). + torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcutorture.test_boost=0 rcutorture.preempt_duration=0" + torture_set "rcurttorture" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=y" --trust-make # With all post-boot grace periods forced to expedited. - torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcupdate.rcu_expedited=1" - torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --trust-make + torture_bootargs="rcupdate.rcu_cpu_stall_suppress_at_boot=1 torture.disable_onoff_at_boot rcupdate.rcu_task_stall_timeout=30000 rcutorture.test_boost=0 rcupdate.rcu_normal_after_boot=0 rcupdate.rcu_expedited=1 rcutorture.preempt_duration=0" + torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_FULL=y" --trust-make fi if test "$do_srcu_lockdep" = "yes"
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 5/12 rcutorture: Add tests for SRCU up/down reader primitives
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
This commit adds a new rcutorture.n_up_down kernel boot parameter that specifies the number of outstanding SRCU up/down readers, which begin in kthread context and end in an hrtimer handler. There is a new kthread ("rcu_torture_updown") that scans an per-reader array looking for elements whose readers have ended. This kthread sleeps between one and two milliseconds between consecutive scans.
[ paulmck: Apply kernel test robot feedback. ]
[ paulmck: Apply Z qiang feedback. ]
--- kernel/rcu/rcutorture.c | 227 ++++++++++++++++++++++++++++++++++++---- 1 file changed, 208 insertions(+), 19 deletions(-)
kernel/rcu/rcutorture.c +208 -19
@@ -55,22 +55,24 @@ MODULE_DESCRIPTION("Read-Copy Update module-based torture test facility");MODULE_LICENSE ("GPL" );MODULE_AUTHOR ("Paul E. McKenney <[email protected]> and Josh Triplett <[email protected]>" ); -/* Bits for ->extendables field, extendables param, and related definitions. */ -#define RCUTORTURE_RDR_SHIFT_1 8/* Put SRCU index in upper bits. */ -#define RCUTORTURE_RDR_MASK_1 (0xff << RCUTORTURE_RDR_SHIFT_1) -#define RCUTORTURE_RDR_SHIFT_2 16/* Put SRCU index in upper bits. */ -#define RCUTORTURE_RDR_MASK_2 (0xff << RCUTORTURE_RDR_SHIFT_2) -#define RCUTORTURE_RDR_BH 0x01/* Extend readers by disabling bh. */ -#define RCUTORTURE_RDR_IRQ 0x02/* ... disabling interrupts. */ -#define RCUTORTURE_RDR_PREEMPT 0x04/* ... disabling preemption. */ -#define RCUTORTURE_RDR_RBH 0x08/* ... rcu_read_lock_bh(). */ -#define RCUTORTURE_RDR_SCHED 0x10/* ... rcu_read_lock_sched(). */ -#define RCUTORTURE_RDR_RCU_1 0x20/* ... entering another RCU reader. */ -#define RCUTORTURE_RDR_RCU_2 0x40/* ... entering another RCU reader. */ -#define RCUTORTURE_RDR_NBITS 7/* Number of bits defined above. */ -#define RCUTORTURE_MAX_EXTEND \ +// Bits for ->extendables field, extendables param, and related definitions. +#define RCUTORTURE_RDR_SHIFT_1 8 // Put SRCU index in upper bits. +#define RCUTORTURE_RDR_MASK_1 (0xff << RCUTORTURE_RDR_SHIFT_1) +#define RCUTORTURE_RDR_SHIFT_2 16 // Put SRCU index in upper bits. +#define RCUTORTURE_RDR_MASK_2 (0xff << RCUTORTURE_RDR_SHIFT_2) +#define RCUTORTURE_RDR_BH 0x01 // Extend readers by disabling bh. +#define RCUTORTURE_RDR_IRQ 0x02 // ... disabling interrupts. +#define RCUTORTURE_RDR_PREEMPT 0x04 // ... disabling preemption. +#define RCUTORTURE_RDR_RBH 0x08 // ... rcu_read_lock_bh(). +#define RCUTORTURE_RDR_SCHED 0x10 // ... rcu_read_lock_sched(). +#define RCUTORTURE_RDR_RCU_1 0x20 // ... entering another RCU reader. +#define RCUTORTURE_RDR_RCU_2 0x40 // ... entering another RCU reader. +#define RCUTORTURE_RDR_UPDOWN 0x80 // ... up-read from task, down-read from timer. +// Note: Manual start, automatic end. +#define RCUTORTURE_RDR_NBITS 8 // Number of bits defined above. +#define RCUTORTURE_MAX_EXTEND \ (RCUTORTURE_RDR_BH | RCUTORTURE_RDR_IRQ | RCUTORTURE_RDR_PREEMPT | \ - RCUTORTURE_RDR_RBH | RCUTORTURE_RDR_SCHED) + RCUTORTURE_RDR_RBH | RCUTORTURE_RDR_SCHED) // Intentionally omit RCUTORTURE_RDR_UPDOWN.#define RCUTORTURE_RDR_ALLBITS \ (RCUTORTURE_MAX_EXTEND | RCUTORTURE_RDR_RCU_1 | RCUTORTURE_RDR_RCU_2 | \ RCUTORTURE_RDR_MASK_1 | RCUTORTURE_RDR_MASK_2)
@@ -110,6 +112,7 @@ torture_param(bool, gp_sync, false, "Use synchronous GP wait primitives");torture_param (int, irqreader,1 ,"Allow RCU readers from irq handlers" );torture_param (int, leakpointer,0 ,"Leak pointer dereferences from readers" );torture_param (int, n_barrier_cbs,0 ,"# of callbacks/kthreads for barrier testing" ); +torture_param (int, n_up_down,32 ,"# of concurrent up/down hrtimer-based RCU readers" );torture_param (int, nfakewriters,4 ,"Number of RCU fake writer threads" );torture_param (int, nreaders,-1 ,"Number of RCU reader threads" );torture_param (int, object_debug,0 ,"Enable debug-object double call_rcu() testing" );
@@ -152,6 +155,7 @@ static int nrealfakewriters;static struct task_struct *writer_task;static struct task_struct **fakewriter_tasks;static struct task_struct **reader_tasks; +static struct task_struct *updown_task;static struct task_struct **nocb_tasks;static struct task_struct *stats_task;static struct task_struct *fqs_task;
@@ -374,6 +378,8 @@ struct rcu_torture_ops {void (*readunlock)(int idx);int (*readlock_held)(void );// lockdep. int (*readlock_nesting)(void );// actual nesting, if available, -1 if not. +int (*down_read)(void ); +void (*up_read)(int idx);unsigned long (*get_gp_seq)(void );unsigned long (*gp_diff)(unsigned long new,unsigned long old);void (*deferred_free)(struct rcu_torture *p);
@@ -421,6 +427,7 @@ struct rcu_torture_ops {int no_pi_lock;int debug_objects;int start_poll_irqsoff; +int have_up_down;const char *name; };
@@ -754,6 +761,50 @@ static int torture_srcu_read_lock_held(void)return srcu_read_lock_held (srcu_ctlp); } +static bool srcu_torture_have_up_down (void ) +{ +int rf = reader_flavor; + +if (!rf) + rf =SRCU_READ_FLAVOR_NORMAL ; +return !!(cur_ops->have_up_down & rf); +} + +static int srcu_torture_down_read (void ) +{ +int idx; +struct srcu_ctr __percpu *scp; + +WARN_ON_ONCE (reader_flavor & ~SRCU_READ_FLAVOR_ALL ); +WARN_ON_ONCE (reader_flavor & (reader_flavor -1 )); + +if ((reader_flavor &SRCU_READ_FLAVOR_NORMAL ) || !(reader_flavor &SRCU_READ_FLAVOR_ALL )) { + idx =srcu_down_read (srcu_ctlp); +WARN_ON_ONCE (idx & ~0x1 ); +return idx; + } +if (reader_flavor &SRCU_READ_FLAVOR_FAST ) { + scp =srcu_down_read_fast (srcu_ctlp); + idx =__srcu_ptr_to_ctr (srcu_ctlp, scp); +WARN_ON_ONCE (idx & ~0x1 ); +return idx <<3 ; + } +WARN_ON_ONCE (1 ); +return 0 ; +} + +static void srcu_torture_up_read (int idx) +{ +WARN_ON_ONCE ((reader_flavor && (idx & ~reader_flavor)) || (!reader_flavor && (idx & ~0x1 ))); +if (reader_flavor &SRCU_READ_FLAVOR_FAST ) +srcu_up_read_fast (srcu_ctlp,__srcu_ctr_to_ptr (srcu_ctlp, (idx &0x8 ) >>3 )); +else if ((reader_flavor &SRCU_READ_FLAVOR_NORMAL ) || + !(reader_flavor &SRCU_READ_FLAVOR_ALL )) +srcu_up_read (srcu_ctlp, idx &0x1 ); +else +WARN_ON_ONCE (1 ); +} +static unsigned long srcu_torture_completed (void ) {return srcu_batches_completed (srcu_ctlp);
@@ -811,6 +862,8 @@ static struct rcu_torture_ops srcu_ops = { .readlock = srcu_torture_read_lock, .read_delay = srcu_read_delay, .readunlock = srcu_torture_read_unlock, + .down_read = srcu_torture_down_read, + .up_read = srcu_torture_up_read, .readlock_held = torture_srcu_read_lock_held, .get_gp_seq = srcu_torture_completed, .gp_diff = rcu_seq_diff,
@@ -831,6 +884,8 @@ static struct rcu_torture_ops srcu_ops = { .irq_capable = 1, .no_pi_lock = IS_ENABLED(CONFIG_TINY_SRCU), .debug_objects = 1, + .have_up_down =IS_ENABLED (CONFIG_TINY_SRCU ) + ? 0 : SRCU_READ_FLAVOR_NORMAL | SRCU_READ_FLAVOR_FAST, .name ="srcu" };
@@ -856,6 +911,8 @@ static struct rcu_torture_ops srcud_ops = { .read_delay = srcu_read_delay, .readunlock = srcu_torture_read_unlock, .readlock_held = torture_srcu_read_lock_held, + .down_read = srcu_torture_down_read, + .up_read = srcu_torture_up_read, .get_gp_seq = srcu_torture_completed, .gp_diff = rcu_seq_diff, .deferred_free = srcu_torture_deferred_free,
@@ -875,6 +932,8 @@ static struct rcu_torture_ops srcud_ops = { .irq_capable = 1, .no_pi_lock = IS_ENABLED(CONFIG_TINY_SRCU), .debug_objects = 1, + .have_up_down =IS_ENABLED (CONFIG_TINY_SRCU ) + ? 0 : SRCU_READ_FLAVOR_NORMAL | SRCU_READ_FLAVOR_FAST, .name ="srcud" };
@@ -1985,7 +2044,7 @@ static void rcutorture_one_extend(int *readstate, int newstate, bool insoftirq, first = idxold1 ==0 ;WARN_ON_ONCE (idxold2 <0 ); -WARN_ON_ONCE (idxold2 & ~RCUTORTURE_RDR_ALLBITS ); +WARN_ON_ONCE (idxold2 & ~(RCUTORTURE_RDR_ALLBITS |RCUTORTURE_RDR_UPDOWN ));rcutorture_one_extend_check ("before change" , idxold1, statesnew, statesold, insoftirq); rtrsp->rt_readstate = newstate;
@@ -2061,6 +2120,11 @@ static void rcutorture_one_extend(int *readstate, int newstate, bool insoftirq,if (lockit)raw_spin_unlock_irqrestore (¤t->pi_lock, flags); } +if (statesold &RCUTORTURE_RDR_UPDOWN ) { + cur_ops->up_read ((idxold1 &RCUTORTURE_RDR_MASK_1 ) >>RCUTORTURE_RDR_SHIFT_1 ); +WARN_ON_ONCE (idxnew1 !=-1 ); + idxold1 =0 ; + }/* Delay if neither beginning nor end and there was a change. */ if ((statesnew || statesold) && *readstate && newstate)
@@ -2201,7 +2265,8 @@ static bool rcu_torture_one_read_start(struct rcu_torture_one_read_state *rtorsp rtorsp->started = cur_ops->get_gp_seq (); rtorsp->ts =rcu_trace_clock_local (); rtorsp->p =rcu_dereference_check (rcu_torture_current, - !cur_ops->readlock_held || cur_ops->readlock_held ()); + !cur_ops->readlock_held || cur_ops->readlock_held () || + (rtorsp->readstate &RCUTORTURE_RDR_UPDOWN ));if (rtorsp->p ==NULL ) {/* Wait for rcu_torture_writer to get underway */ rcutorture_one_extend (&rtorsp->readstate,0 , myid <0 , trsp, rtorsp->rtrsp);
@@ -2370,6 +2435,123 @@ rcu_torture_reader(void *arg)return 0 ; } +struct rcu_torture_one_read_state_updown { +struct hrtimer rtorsu_hrt; +bool rtorsu_inuse; +struct torture_random_state rtorsu_trs; +struct rcu_torture_one_read_state rtorsu_rtors; +}; + +static struct rcu_torture_one_read_state_updown *updownreaders; +static DEFINE_TORTURE_RANDOM (rcu_torture_updown_rand); +static int rcu_torture_updown (void *arg); + +static enum hrtimer_restart rcu_torture_updown_hrt (struct hrtimer *hrtp) +{ +struct rcu_torture_one_read_state_updown *rtorsup; + + rtorsup = container_of(hrtp, struct rcu_torture_one_read_state_updown, rtorsu_hrt); +rcu_torture_one_read_end (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 ); +smp_store_release (&rtorsup->rtorsu_inuse, false); +return HRTIMER_NORESTART ; +} + +static int rcu_torture_updown_init (void ) +{ +int i; +struct torture_random_state *rand = &rcu_torture_updown_rand; +int ret; + +if (n_up_down <0 ) +return 0 ; +if (!srcu_torture_have_up_down ()) { +VERBOSE_TOROUT_STRING ("rcu_torture_updown_init: Disabling up/down reader tests due to lack of primitives" ); +return 0 ; + } + updownreaders =kcalloc (n_up_down,sizeof (*updownreaders),GFP_KERNEL ); +if (!updownreaders) { +VERBOSE_TOROUT_STRING ("rcu_torture_updown_init: Out of memory, disabling up/down reader tests" ); +return -ENOMEM ; + } +for (i =0 ; i < n_up_down; i++) { +init_rcu_torture_one_read_state (&updownreaders[i].rtorsu_rtors, rand); +hrtimer_init (&updownreaders[i].rtorsu_hrt,CLOCK_MONOTONIC , +HRTIMER_MODE_REL |HRTIMER_MODE_SOFT ); + updownreaders[i].rtorsu_hrt.function = rcu_torture_updown_hrt; +torture_random_init (&updownreaders[i].rtorsu_trs); +init_rcu_torture_one_read_state (&updownreaders[i].rtorsu_rtors, + &updownreaders[i].rtorsu_trs); + } + ret =torture_create_kthread (rcu_torture_updown, rand, updown_task); +if (ret) { +kfree (updownreaders); + updownreaders =NULL ; + } +return ret; +} + +static void rcu_torture_updown_cleanup (void ) +{ +struct rcu_torture_one_read_state_updown *rtorsup; + +for (rtorsup = updownreaders; rtorsup < &updownreaders[n_up_down]; rtorsup++) { +if (!smp_load_acquire (&rtorsup->rtorsu_inuse)) +continue ; +if (!hrtimer_cancel (&rtorsup->rtorsu_hrt)) +WARN_ON_ONCE (rtorsup->rtorsu_inuse); + + } +kfree (updownreaders); + updownreaders =NULL ; +} + +/* +* RCU torture up/down reader kthread, starting RCU readers in kthread +* context and ending them in hrtimer handlers. Otherwise similar to +* rcu_torture_reader(). +*/ +static int +rcu_torture_updown (void *arg) +{ +int idx; +int rawidx; +struct rcu_torture_one_read_state_updown *rtorsup; +ktime_t t; + +VERBOSE_TOROUT_STRING ("rcu_torture_updown task started" ); +do { +for (rtorsup = updownreaders; rtorsup < &updownreaders[n_up_down]; rtorsup++) { +if (torture_must_stop ()) +break ; +if (smp_load_acquire (&rtorsup->rtorsu_inuse)) +continue ; +init_rcu_torture_one_read_state (&rtorsup->rtorsu_rtors, + &rtorsup->rtorsu_trs); + rawidx = cur_ops->down_read (); + idx = (rawidx <<RCUTORTURE_RDR_SHIFT_1 ) &RCUTORTURE_RDR_MASK_1 ; + rtorsup->rtorsu_rtors.readstate = idx |RCUTORTURE_RDR_UPDOWN ; + rtorsup->rtorsu_rtors.rtrsp++; +if (!rcu_torture_one_read_start (&rtorsup->rtorsu_rtors, + &rtorsup->rtorsu_trs,-1 )) { + cur_ops->up_read (rawidx); +schedule_timeout_idle (HZ ); +continue ; + } +smp_store_release (&rtorsup->rtorsu_inuse, true); + t =torture_random (&rtorsup->rtorsu_trs) &0xfffff ;// One per million. +if (t <10 *1000 ) + t =200 *1000 *1000 ; +hrtimer_start (&rtorsup->rtorsu_hrt, t, +HRTIMER_MODE_REL |HRTIMER_MODE_SOFT ); + } +torture_hrtimeout_ms (1 ,1000 , &rcu_torture_updown_rand); +stutter_wait ("rcu_torture_updown" ); + }while (!torture_must_stop ()); +rcu_torture_updown_cleanup (); +torture_kthread_stopping ("rcu_torture_updown" ); +return 0 ; +} +/* * Randomly Toggle CPUs' callback-offload state. This uses hrtimers to * increase race probabilities and fuzzes the interval between toggling.
@@ -2620,7 +2802,7 @@ rcu_torture_print_module_parms(struct rcu_torture_ops *cur_ops, const char *tag)"reader_flavor=%x " "nocbs_nthreads=%d nocbs_toggle=%d " "test_nmis=%d " -"preempt_duration=%d preempt_interval=%d\n" , +"preempt_duration=%d preempt_interval=%d n_up_down=%d\n" , torture_type, tag, nrealreaders, nrealfakewriters, stat_interval, verbose, test_no_idle_hz, shuffle_interval, stutter, irqreader, fqs_duration, fqs_holdoff, fqs_stutter,
@@ -2634,7 +2816,7 @@ rcu_torture_print_module_parms(struct rcu_torture_ops *cur_ops, const char *tag) reader_flavor, nocbs_nthreads, nocbs_toggle, test_nmis, - preempt_duration, preempt_interval); + preempt_duration, preempt_interval, n_up_down); }static int rcutorture_booster_cleanup (unsigned int cpu)
@@ -3686,6 +3868,10 @@ rcu_torture_cleanup(void) nocb_tasks =NULL ; } +if (updown_task) { +torture_stop_kthread (rcu_torture_updown, updown_task); + updown_task =NULL ; + }if (reader_tasks) {for (i =0 ; i < nrealreaders; i++) torture_stop_kthread(rcu_torture_reader,
@@ -4216,6 +4402,9 @@ rcu_torture_init(void)if (torture_init_error (firsterr)) goto unwind; } + firsterr =rcu_torture_updown_init (); +if (torture_init_error (firsterr)) + goto unwind; nrealnocbers = nocbs_nthreads;if (WARN_ON (nrealnocbers <0 )) nrealnocbers =1 ;
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 6/12 rcutorture: Pull rcu_torture_updown() loop body into new function
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
This is strictly a code-movement commit, pulling that part of the rcu_torture_updown() function's loop body that processes one rcu_torture_one_read_state_updown structure into a new rcu_torture_updown_one() function. The checks for the end of the torture test and the current structure being in use remain in the rcu_torture_updown() function.
--- kernel/rcu/rcutorture.c | 46 ++++++++++++++++++++++------------------- 1 file changed, 25 insertions(+), 21 deletions(-)
kernel/rcu/rcutorture.c +25 -21
@@ -2505,6 +2505,30 @@ static void rcu_torture_updown_cleanup(void) updownreaders =NULL ; } +// Do one reader for rcu_torture_updown(). +static void rcu_torture_updown_one (struct rcu_torture_one_read_state_updown *rtorsup) +{ +int idx; +int rawidx; +ktime_t t; + +init_rcu_torture_one_read_state (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs); + rawidx = cur_ops->down_read (); + idx = (rawidx <<RCUTORTURE_RDR_SHIFT_1 ) &RCUTORTURE_RDR_MASK_1 ; + rtorsup->rtorsu_rtors.readstate = idx |RCUTORTURE_RDR_UPDOWN ; + rtorsup->rtorsu_rtors.rtrsp++; +if (!rcu_torture_one_read_start (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 )) { + cur_ops->up_read (rawidx); +schedule_timeout_idle (HZ ); +return ; + } +smp_store_release (&rtorsup->rtorsu_inuse, true); + t =torture_random (&rtorsup->rtorsu_trs) &0xfffff ;// One per million. +if (t <10 *1000 ) + t =200 *1000 *1000 ; +hrtimer_start (&rtorsup->rtorsu_hrt, t,HRTIMER_MODE_REL |HRTIMER_MODE_SOFT ); +} +/* * RCU torture up/down reader kthread, starting RCU readers in kthread * context and ending them in hrtimer handlers. Otherwise similar to
@@ -2513,10 +2537,7 @@ static void rcu_torture_updown_cleanup(void)static int rcu_torture_updown (void *arg) { -int idx; -int rawidx;struct rcu_torture_one_read_state_updown *rtorsup; -ktime_t t;VERBOSE_TOROUT_STRING ("rcu_torture_updown task started" ); do {
@@ -2525,24 +2546,7 @@ rcu_torture_updown(void *arg)break ;if (smp_load_acquire (&rtorsup->rtorsu_inuse))continue ; -init_rcu_torture_one_read_state (&rtorsup->rtorsu_rtors, - &rtorsup->rtorsu_trs); - rawidx = cur_ops->down_read (); - idx = (rawidx <<RCUTORTURE_RDR_SHIFT_1 ) &RCUTORTURE_RDR_MASK_1 ; - rtorsup->rtorsu_rtors.readstate = idx |RCUTORTURE_RDR_UPDOWN ; - rtorsup->rtorsu_rtors.rtrsp++; -if (!rcu_torture_one_read_start (&rtorsup->rtorsu_rtors, - &rtorsup->rtorsu_trs,-1 )) { - cur_ops->up_read (rawidx); -schedule_timeout_idle (HZ ); -continue ; - } -smp_store_release (&rtorsup->rtorsu_inuse, true); - t =torture_random (&rtorsup->rtorsu_trs) &0xfffff ;// One per million. -if (t <10 *1000 ) - t =200 *1000 *1000 ; -hrtimer_start (&rtorsup->rtorsu_hrt, t, -HRTIMER_MODE_REL |HRTIMER_MODE_SOFT ); +rcu_torture_updown_one (rtorsup); }torture_hrtimeout_ms (1 ,1000 , &rcu_torture_updown_rand);stutter_wait ("rcu_torture_updown" );
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 7/12 rcutorture: Comment invocations of tick_dep_set_task()
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
The rcu_torture_reader() and rcu_torture_fwd_prog_cr() functions run CPU-bound for extended periods of time (tens or even hundreds of milliseconds), so they invoke tick_dep_set_task() and tick_dep_clear_task() to ensure that the scheduling-clock tick helps move grace periods forward.
So why doesn't rcu_torture_fwd_prog_nr() also invoke tick_dep_set_task() and tick_dep_clear_task()? Because the point of this function is to test RCU's ability to (eventually) force grace periods forward even when the tick has been disabled during long CPU-bound kernel execution.
--- kernel/rcu/rcutorture.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/rcu/rcutorture.c +2 -2
@@ -2410,7 +2410,7 @@ rcu_torture_reader(void *arg)set_user_nice (current,MAX_NICE );if (irqreader && cur_ops->irq_capable)timer_setup_on_stack (&t, rcu_torture_timer,0 ); -tick_dep_set_task (current,TICK_DEP_BIT_RCU ); +tick_dep_set_task (current,TICK_DEP_BIT_RCU );// CPU bound, so need tick. do { if (irqreader && cur_ops->irq_capable) { if (!timer_pending(&t))
@@ -3260,7 +3260,7 @@ static void rcu_torture_fwd_prog_cr(struct rcu_fwd *rfp) cver =READ_ONCE (rcu_torture_current_version); gps = cur_ops->get_gp_seq (); rfp->rcu_launder_gp_seq_start = gps; -tick_dep_set_task (current,TICK_DEP_BIT_RCU ); +tick_dep_set_task (current,TICK_DEP_BIT_RCU );// CPU bound, so need tick. while (time_before (jiffies, stopat) && !shutdown_time_arrived () && !READ_ONCE (rcu_fwd_emergency_stop) && !torture_must_stop ()) {
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 8/12 rcutorture: Complain if an ->up_read() is delayed more than 10 seconds
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
The down/up SRCU reader testing uses an hrtimer handler to exit the SRCU read-side critical section. This might be delayed, and if delayed for too long, it can prevent the rcutorture run from completing. This commit therefore complains if the hrtimer handler is delayed for more than ten seconds.
--- kernel/rcu/rcutorture.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/rcu/rcutorture.c +5 -1
@@ -2438,6 +2438,7 @@ rcu_torture_reader(void *arg)struct rcu_torture_one_read_state_updown {struct hrtimer rtorsu_hrt;bool rtorsu_inuse; +unsigned long rtorsu_j;struct torture_random_state rtorsu_trs;struct rcu_torture_one_read_state rtorsu_rtors; };
@@ -2522,6 +2523,7 @@ static void rcu_torture_updown_one(struct rcu_torture_one_read_state_updown *rtoschedule_timeout_idle (HZ );return ; } + rtorsup->rtorsu_j = jiffies;smp_store_release (&rtorsup->rtorsu_inuse, true); t =torture_random (&rtorsup->rtorsu_trs) &0xfffff ;// One per million. if (t <10 *1000 )
@@ -2544,8 +2546,10 @@ rcu_torture_updown(void *arg)for (rtorsup = updownreaders; rtorsup < &updownreaders[n_up_down]; rtorsup++) {if (torture_must_stop ())break ; -if (smp_load_acquire (&rtorsup->rtorsu_inuse)) +if (smp_load_acquire (&rtorsup->rtorsu_inuse)) { +WARN_ON_ONCE (time_after (jiffies, rtorsup->rtorsu_j +10 *HZ ));continue ; + }rcu_torture_updown_one (rtorsup); }torture_hrtimeout_ms (1 ,1000 , &rcu_torture_updown_rand);
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 9/12 rcutorture: Check for ->up_read() without matching ->down_read()
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
This commit creates counters in the rcu_torture_one_read_state_updown structure that check for a call to ->up_read() that lacks a matching call to ->down_read().
While in the area, add end-of-run cleanup code that prevents calls to rcu_torture_updown_hrt() from happening after the test has moved on. Yes, the srcu_barrier() at the end of the test will wait for them, but this could result in confusing states, statistics, and diagnostic information. So explicitly wait for them before we get to the end-of-test output.
[ paulmck: Apply kernel test robot feedback. ]
--- kernel/rcu/rcutorture.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)
kernel/rcu/rcutorture.c +13 -2
@@ -2439,6 +2439,8 @@ struct rcu_torture_one_read_state_updown {struct hrtimer rtorsu_hrt;bool rtorsu_inuse;unsigned long rtorsu_j; +unsigned long rtorsu_ndowns; +unsigned long rtorsu_nups;struct torture_random_state rtorsu_trs;struct rcu_torture_one_read_state rtorsu_rtors; };
@@ -2453,6 +2455,8 @@ static enum hrtimer_restart rcu_torture_updown_hrt(struct hrtimer *hrtp) rtorsup = container_of(hrtp, struct rcu_torture_one_read_state_updown, rtorsu_hrt);rcu_torture_one_read_end (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 ); +WARN_ONCE (rtorsup->rtorsu_nups >= rtorsup->rtorsu_ndowns,"%s: Up without matching down #%zu.\n" , __func__, rtorsup - updownreaders); + rtorsup->rtorsu_nups++;smp_store_release (&rtorsup->rtorsu_inuse, false);return HRTIMER_NORESTART ; }
@@ -2498,8 +2502,12 @@ static void rcu_torture_updown_cleanup(void)for (rtorsup = updownreaders; rtorsup < &updownreaders[n_up_down]; rtorsup++) {if (!smp_load_acquire (&rtorsup->rtorsu_inuse))continue ; -if (!hrtimer_cancel (&rtorsup->rtorsu_hrt)) -WARN_ON_ONCE (rtorsup->rtorsu_inuse); +if (hrtimer_cancel (&rtorsup->rtorsu_hrt) ||WARN_ON_ONCE (rtorsup->rtorsu_inuse)) { +rcu_torture_one_read_end (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 ); +WARN_ONCE (rtorsup->rtorsu_nups >= rtorsup->rtorsu_ndowns,"%s: Up without matching down #%zu.\n" , __func__, rtorsup - updownreaders); + rtorsup->rtorsu_nups++; +smp_store_release (&rtorsup->rtorsu_inuse, false); + } }kfree (updownreaders);
@@ -2515,11 +2523,14 @@ static void rcu_torture_updown_one(struct rcu_torture_one_read_state_updown *rtoinit_rcu_torture_one_read_state (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs); rawidx = cur_ops->down_read (); + rtorsup->rtorsu_ndowns++; idx = (rawidx <<RCUTORTURE_RDR_SHIFT_1 ) &RCUTORTURE_RDR_MASK_1 ; rtorsup->rtorsu_rtors.readstate = idx |RCUTORTURE_RDR_UPDOWN ; rtorsup->rtorsu_rtors.rtrsp++;if (!rcu_torture_one_read_start (&rtorsup->rtorsu_rtors, &rtorsup->rtorsu_trs,-1 )) { cur_ops->up_read (rawidx); + rtorsup->rtorsu_nups++; +WARN_ONCE (rtorsup->rtorsu_nups >= rtorsup->rtorsu_ndowns,"%s: Up without matching down #%zu.\n" , __func__, rtorsup - updownreaders);schedule_timeout_idle (HZ );return ; }
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 10/12 checkpatch: Deprecate srcu_read_lock_lite() and srcu_read_unlock_lite()
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
Uses of srcu_read_lock_lite() and srcu_read_unlock_lite() are better served by the new srcu_read_lock_fast() and srcu_read_unlock_fast() APIs. As in srcu_read_lock_lite() and srcu_read_unlock_lite() would never have happened had I thought a bit harder a few months ago. Therefore, mark them deprecated.
--- scripts/checkpatch.pl | 2 ++ 1 file changed, 2 insertions(+)
scripts/checkpatch.pl +2 -0
@@ -838,6 +838,8 @@ our %deprecated_apis = ( "kunmap" => "kunmap_local", "kmap_atomic" => "kmap_local_page", "kunmap_atomic" => "kunmap_local", + "srcu_read_lock_lite" => "srcu_read_lock_fast", + "srcu_read_unlock_lite" => "srcu_read_unlock_fast", ); #Create a search pattern for all these strings to speed up a loop below
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 11/12 torture: Add --do-{,no-}normal to torture.sh
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
Right now, torture.sh runs normal runs unconditionally, which can be slow and thus annoying when you only want to test --kcsan or --kasan runs. This commit therefore adds a --do-normal argument so that "--kcsan --do-no-kasan --do-no-normal" runs only KCSAN runs. Note that specifying "--do-no-kasan --do-no-kcsan --do-no-normal" gets normal runs, so you should not try to use this as a synonym for --do-none.
--- .../selftests/rcutorture/bin/torture.sh | 30 +++++++++++++++++-- 1 file changed, 27 insertions(+), 3 deletions(-)
tools/testing/selftests/rcutorture/bin/torture.sh +27 -3
@@ -51,6 +51,8 @@ do_scftorture=yes do_rcuscale=yes do_refscale=yes do_kvfree=yes +do_normal=yes +explicit_normal=no do_kasan=yes do_kcsan=no do_clocksourcewd=yes
@@ -128,6 +130,8 @@ do do_refscale=yes do_rt=yes do_kvfree=yes + do_normal=yes + explicit_normal=no do_kasan=yes do_kcsan=yes do_clocksourcewd=yes
@@ -161,11 +165,17 @@ do do_refscale=no do_rt=no do_kvfree=no + do_normal=no + explicit_normal=no do_kasan=no do_kcsan=no do_clocksourcewd=no do_srcu_lockdep=no ;; + --do-normal|--do-no-normal|--no-normal) + do_normal=`doyesno "$1" --do-normal` + explicit_normal=yes + ;; --do-rcuscale|--do-no-rcuscale|--no-rcuscale) do_rcuscale=`doyesno "$1" --do-rcuscale` ;;
@@ -242,6 +252,17 @@ trap 'rm -rf $T' 0 2 echo " --- " $scriptname $args | tee -a $T/log echo " --- Results directory: " $ds | tee -a $T/log +if test "$do_normal" = "no" && test "$do_kasan" = "no" && test "$do_kcsan" = "no" +then + # Match old scripts so that "--do-none --do-rcutorture" does + # normal rcutorture testing, but no KASAN or KCSAN testing. + if test $explicit_normal = yes + then + echo " --- Everything disabled, so explicit --do-normal overridden" | tee -a $T/log + fi + do_normal=yes +fi + # Calculate rcutorture defaults and apportion time if test -z "$configs_rcutorture" then
@@ -332,9 +353,12 @@ function torture_set { local kcsan_kmake_tag= local flavor=$1 shift - curflavor=$flavor - torture_one "$@" - mv $T/last-resdir $T/last-resdir-nodebug || : + if test "$do_normal" = "yes" + then + curflavor=$flavor + torture_one "$@" + mv $T/last-resdir $T/last-resdir-nodebug || : + fi if test "$do_kasan" = "yes" then curflavor=${flavor}-kasan
Signature · 1 line
2.43.0
Joel Fernandes
maintainerPATCH 12/12 torture: Add testing of RCU's Rust bindings to torture.sh
In reply to Joel FernandesFrom: "Paul E. McKenney" <[email protected]>
This commit adds a --do-rcu-rust parameter to torture.sh, which invokes a rust_doctests_kernel kunit run. Note that kunit wants a clean source tree, so this runs "make mrproper", which might come as a surprise to some users. Should there be a --mrproper parameter to torture.sh to make the user explicitly ask for it?
--- .../selftests/rcutorture/bin/torture.sh | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+)
tools/testing/selftests/rcutorture/bin/torture.sh +45 -0
@@ -59,6 +59,7 @@ do_clocksourcewd=yes do_rt=yes do_rcutasksflavors=yes do_srcu_lockdep=yes +do_rcu_rust=no # doyesno - Helper function for yes/no arguments function doyesno () {
@@ -89,6 +90,7 @@ usage () { echo " --do-rcutorture / --do-no-rcutorture / --no-rcutorture" echo " --do-refscale / --do-no-refscale / --no-refscale" echo " --do-rt / --do-no-rt / --no-rt" + echo " --do-rcu-rust / --do-no-rcu-rust / --no-rcu-rust" echo " --do-scftorture / --do-no-scftorture / --no-scftorture" echo " --do-srcu-lockdep / --do-no-srcu-lockdep / --no-srcu-lockdep" echo " --duration [ <minutes> | <hours>h | <days>d ]"
@@ -191,6 +193,9 @@ do --do-rt|--do-no-rt|--no-rt) do_rt=`doyesno "$1" --do-rt` ;; + --do-rcu-rust|--do-no-rcu-rust|--no-rcu-rust) + do_rcu_rust=`doyesno "$1" --do-rcu-rust` + ;; --do-scftorture|--do-no-scftorture|--no-scftorture) do_scftorture=`doyesno "$1" --do-scftorture` ;;
@@ -485,6 +490,46 @@ then torture_set "rcurttorture-exp" tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration "$duration_rcutorture" --configs "TREE03" --kconfig "CONFIG_PREEMPT_RT=y CONFIG_EXPERT=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_FULL=y" --trust-make fi +if test "$do_rcu_rust" = "yes" +then + echo " --- do-rcu-rust:" Start `date` | tee -a $T/log + rrdir="tools/testing/selftests/rcutorture/res/$ds/results-rcu-rust" + mkdir -p "$rrdir" + echo " --- make LLVM=1 rustavailable " | tee -a $rrdir/log > $rrdir/rustavailable.out + make LLVM=1 rustavailable > $T/rustavailable.out 2>&1 + retcode=$? + echo $retcode > $rrdir/rustavailable.exitcode + cat $T/rustavailable.out | tee -a $rrdir/log >> $rrdir/rustavailable.out 2>&1 + buildphase=rustavailable + if test "$retcode" -eq 0 + then + echo " --- Running 'make mrproper' in order to run kunit." | tee -a $rrdir/log > $rrdir/mrproper.out + make mrproper > $rrdir/mrproper.out 2>&1 + retcode=$? + echo $retcode > $rrdir/mrproper.exitcode + buildphase=mrproper + fi + if test "$retcode" -eq 0 + then + echo " --- Running rust_doctests_kernel." | tee -a $rrdir/log > $rrdir/rust_doctests_kernel.out + ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --make_options CLIPPY=1 --arch arm64 --kconfig_add CONFIG_SMP=y --kconfig_add CONFIG_WERROR=y --kconfig_add CONFIG_RUST=y rust_doctests_kernel >> $rrdir/rust_doctests_kernel.out 2>&1 + # @@@ Remove "--arch arm64" in order to test on native architecture? + # @@@ Analyze $rrdir/rust_doctests_kernel.out contents? + retcode=$? + echo $retcode > $rrdir/rust_doctests_kernel.exitcode + buildphase=rust_doctests_kernel + fi + if test "$retcode" -eq 0 + then + echo "rcu-rust($retcode)" $rrdir >> $T/successes + echo Success >> $rrdir/log + else + echo "rcu-rust($retcode)" $rrdir >> $T/failures + echo " --- rcu-rust Test summary:" >> $rrdir/log + echo " --- Summary: Exit code $retcode from $buildphase, see $rrdir/$buildphase.out" >> $rrdir/log + fi +fi + if test "$do_srcu_lockdep" = "yes" then echo " --- do-srcu-lockdep:" Start `date` | tee -a $T/log
Signature · 1 line
2.43.0
Joel Fernandes
maintainerQuoted 3 lines · Joel Fernandes
Hi,
Please find the upcoming miscellaneous RCU changes. The changes can also
FYI: Sorry, s/miscellaneous/torture-related/ here.
Also please ignore patches with "XX/12" in their subject, that was an error on my part.
The rest of the series stands good for perusal.
thanks,
- Joel