Skip to contentSkip to conversation

torture: Support randomized shuffling for proxy exec testing

PATCH Applied John Stultz 3 participants 5 messages Jun 2, 2023 – Jun 3, 2023 Torture-test modules

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 torture: Support randomized shuffling for proxy exec testing

From: Connor O'Brien <[email protected]>

Currently shuffling sets the same cpu affinities for all tasks, which makes us less likely to hit paths involving migrating blocked tasks onto a cpu where they can't run.

This patch adds an element of randomness to allow affinities of different writer tasks to diverge.

This has helped uncover issues in testing with Proxy Execution

Cc Davidlohr Bueso <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Valentin Schneider <[email protected]>
Cc Dietmar Eggemann <[email protected]>
---
 kernel/torture.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
kernel/torture.c +8 -2
index 1a0519b836ac..8be83fdc6be1 100644
@@ -54,6 +54,9 @@ module_param(verbose_sleep_frequency, int, 0444);
 static int verbose_sleep_duration = 1;
 module_param(verbose_sleep_duration, int, 0444);
 
+static int random_shuffle;
+module_param(random_shuffle, int, 0444);
+
 static char *torture_type;
 static int verbose;
 
@@ -518,6 +521,7 @@ static void torture_shuffle_task_unregister_all(void)
  */
 static void torture_shuffle_tasks(void)
 {
+	DEFINE_TORTURE_RANDOM(rand);
 	struct shuffle_task *stp;
 
 	cpumask_setall(shuffle_tmp_mask);
@@ -537,8 +541,10 @@ static void torture_shuffle_tasks(void)
 		cpumask_clear_cpu(shuffle_idle_cpu, shuffle_tmp_mask);
 
 	mutex_lock(&shuffle_task_mutex);
-	list_for_each_entry(stp, &shuffle_task_list, st_l)
-		set_cpus_allowed_ptr(stp->st_t, shuffle_tmp_mask);
+	list_for_each_entry(stp, &shuffle_task_list, st_l) {
+		if (!random_shuffle || torture_random(&rand) & 0x1)
+			set_cpus_allowed_ptr(stp->st_t, shuffle_tmp_mask);
+	}
 	mutex_unlock(&shuffle_task_mutex);
 
 	cpus_read_unlock();
Signature · 1 line
2.41.0.rc2.161.g9c6817b8e7-goog
#1 lore raw

PATCH 2/2 torture: Add lock_torture_writer_fifo module param

From: Dietmar Eggemann <[email protected]>

Modifies locktorture writer to run as RT task.

To use it:
insmod /lib/modules/torture.ko random_shuffle=1 lock_torture_writer_fifo=1

                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^

insmod /lib/modules/locktorture.ko torture_type=mutex_lock rt_boost=1 rt_boost_factor=50 nested_locks=3

This patch has been helpful to uncover issues with the proxy-execution seires.

Cc Davidlohr Bueso <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Valentin Schneider <[email protected]>
Cc Dietmar Eggemann <[email protected]>
[jstultz: Include header change to build, reword commit message]
---
 kernel/locking/locktorture.c |  3 ++-
 kernel/torture.c             | 11 ++++++++++-
 2 files changed, 12 insertions(+), 2 deletions(-)
kernel/locking/locktorture.c +2 -1
index 153ddc4c47ef..7cb044fc99b2 100644
@@ -816,7 +816,8 @@ static int lock_torture_writer(void *arg)
 	bool skip_main_lock;
 
 	VERBOSE_TOROUT_STRING("lock_torture_writer task started");
-	set_user_nice(current, MAX_NICE);
+	if (!rt_task(current))
+		set_user_nice(current, MAX_NICE);
 
 	do {
 		if ((torture_random(&rand) & 0xfffff) == 0)
kernel/torture.c +10 -1
index 8be83fdc6be1..db79197e257a 100644
@@ -37,6 +37,7 @@
 #include <linux/ktime.h>
 #include <asm/byteorder.h>
 #include <linux/torture.h>
+#include <linux/sched/rt.h>
 #include "rcu/rcu.h"
 
 MODULE_LICENSE("GPL");
@@ -57,6 +58,9 @@ module_param(verbose_sleep_duration, int, 0444);
 static int random_shuffle;
 module_param(random_shuffle, int, 0444);
 
+static int lock_torture_writer_fifo;
+module_param(lock_torture_writer_fifo, int, 0444);
+
 static char *torture_type;
 static int verbose;
 
@@ -734,7 +738,7 @@ bool stutter_wait(const char *title)
 	cond_resched_tasks_rcu_qs();
 	spt = READ_ONCE(stutter_pause_test);
 	for (; spt; spt = READ_ONCE(stutter_pause_test)) {
-		if (!ret) {
+		if (!ret && !rt_task(current)) {
 			sched_set_normal(current, MAX_NICE);
 			ret = true;
 		}
@@ -944,6 +948,11 @@ int _torture_create_kthread(int (*fn)(void *arg), void *arg, char *s, char *m,
 		*tp = NULL;
 		return ret;
 	}
+
+	if (lock_torture_writer_fifo &&
+	    !strncmp(s, "lock_torture_writer", strlen(s)))
+		sched_set_fifo(*tp);
+
 	wake_up_process(*tp);  // Process is sleeping, so ordering provided.
 	torture_shuffle_task_register(*tp);
 	return ret;
Signature · 1 line
2.41.0.rc2.161.g9c6817b8e7-goog
maintainer Applied
#2 lore raw

Re: [PATCH 2/2] torture: Add lock_torture_writer_fifo module param

Quoted 12 lines · John Stultz

From: Dietmar Eggemann <[email protected]>

Modifies locktorture writer to run as RT task.

To use it:
insmod /lib/modules/torture.ko random_shuffle=1 lock_torture_writer_fifo=1

                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^

insmod /lib/modules/locktorture.ko torture_type=mutex_lock rt_boost=1 rt_boost_factor=50 nested_locks=3

This patch has been helpful to uncover issues with the proxy-execution seires.

Quoted 11 lines
Cc Davidlohr Bueso <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Valentin Schneider <[email protected]>
Cc Dietmar Eggemann <[email protected]>
[jstultz: Include header change to build, reword commit message]

Queued and pushed, thank you all!

Thanx, Paul

Quoted 65 lines
---
 kernel/locking/locktorture.c |  3 ++-
 kernel/torture.c             | 11 ++++++++++-
 2 files changed, 12 insertions(+), 2 deletions(-)
kernel/locking/locktorture.c +2 -1
index 153ddc4c47ef..7cb044fc99b2 100644
@@ -816,7 +816,8 @@ static int lock_torture_writer(void *arg)
 	bool skip_main_lock;
 
 	VERBOSE_TOROUT_STRING("lock_torture_writer task started");
-	set_user_nice(current, MAX_NICE);
+	if (!rt_task(current))
+		set_user_nice(current, MAX_NICE);
 
 	do {
 		if ((torture_random(&rand) & 0xfffff) == 0)
kernel/torture.c +10 -1
index 8be83fdc6be1..db79197e257a 100644
@@ -37,6 +37,7 @@
 #include <linux/ktime.h>
 #include <asm/byteorder.h>
 #include <linux/torture.h>
+#include <linux/sched/rt.h>
 #include "rcu/rcu.h"
 
 MODULE_LICENSE("GPL");
@@ -57,6 +58,9 @@ module_param(verbose_sleep_duration, int, 0444);
 static int random_shuffle;
 module_param(random_shuffle, int, 0444);
 
+static int lock_torture_writer_fifo;
+module_param(lock_torture_writer_fifo, int, 0444);
+
 static char *torture_type;
 static int verbose;
 
@@ -734,7 +738,7 @@ bool stutter_wait(const char *title)
 	cond_resched_tasks_rcu_qs();
 	spt = READ_ONCE(stutter_pause_test);
 	for (; spt; spt = READ_ONCE(stutter_pause_test)) {
-		if (!ret) {
+		if (!ret && !rt_task(current)) {
 			sched_set_normal(current, MAX_NICE);
 			ret = true;
 		}
@@ -944,6 +948,11 @@ int _torture_create_kthread(int (*fn)(void *arg), void *arg, char *s, char *m,
 		*tp = NULL;
 		return ret;
 	}
+
+	if (lock_torture_writer_fifo &&
+	    !strncmp(s, "lock_torture_writer", strlen(s)))
+		sched_set_fifo(*tp);
+
 	wake_up_process(*tp);  // Process is sleeping, so ordering provided.
 	torture_shuffle_task_register(*tp);
 	return ret;
Signature · 1 line
2.41.0.rc2.161.g9c6817b8e7-goog
maintainer Acked-by
#3 lore raw

Re: [PATCH 2/2] torture: Add lock_torture_writer_fifo module param

Quoted 25 lines · Paul E. McKenney
Quoted 12 lines · John Stultz

From: Dietmar Eggemann <[email protected]>

Modifies locktorture writer to run as RT task.

To use it:
insmod /lib/modules/torture.ko random_shuffle=1 lock_torture_writer_fifo=1

                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^

insmod /lib/modules/locktorture.ko torture_type=mutex_lock rt_boost=1 rt_boost_factor=50 nested_locks=3

This patch has been helpful to uncover issues with the proxy-execution seires.

Quoted 11 lines
Cc Davidlohr Bueso <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Valentin Schneider <[email protected]>
Cc Dietmar Eggemann <[email protected]>
[jstultz: Include header change to build, reword commit message]
Quoted 1 line

Queued and pushed, thank you all!

Both look good to me. Feel free to add my:

maintainer
#4 lore raw

Re: [PATCH 2/2] torture: Add lock_torture_writer_fifo module param

Quoted 27 lines · Davidlohr Bueso
Quoted 25 lines · Paul E. McKenney
Quoted 23 lines · John Stultz

From: Dietmar Eggemann <[email protected]>

Modifies locktorture writer to run as RT task.

To use it:
insmod /lib/modules/torture.ko random_shuffle=1 lock_torture_writer_fifo=1

                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^

insmod /lib/modules/locktorture.ko torture_type=mutex_lock rt_boost=1 rt_boost_factor=50 nested_locks=3

This patch has been helpful to uncover issues with the proxy-execution seires.

Cc Davidlohr Bueso <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Josh Triplett <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Juri Lelli <[email protected]>
Cc Valentin Schneider <[email protected]>
Cc Dietmar Eggemann <[email protected]>
[jstultz: Include header change to build, reword commit message]
Quoted 5 lines
Quoted 1 line

Queued and pushed, thank you all!

Both look good to me. Feel free to add my:

Thank you! I will apply this on my next rebase.

Thanx, Paul