Skip to contentSkip to conversation

rtla/tests: Extend runtime test coverage

PATCH In review Tomas Glozar 2 participants 11 messages Apr 23 – Apr 27

Subsystems (2)
lore
Share & tools
lore

Tomas Glozar

author maintainer
#0 lore raw

cover rtla/tests: Extend runtime test coverage

This patchset introduces some new tests to cover more options, especially histogram and thread options. Most of the new tests use positive and negative output matches, sometimes in combination with action scripts, to verify that RTLA is applying the settings correctly.

Tests were reorganized a little, adding two new sections: thread tests and histogram tests, next to basic tests.

Additionally, coverage of existing tests is extended by adding new matches and by extending tests to cover both top and hist tools where possible. For the latter, new helpers check_top_hist and check_top_q_hist are added to engine.sh.

As part of the new action scripts, detection of measurement threads is made more robust by following child processes of either RTLA (user workload) or kthreadd (kernel workload) rather than grepping through the comms of all processes, which might have lead to false positives.

These changes significantly improve test coverage and make the test suite more against false positives from unrelated processes.

Tomas Glozar (9):
rtla/tests: Cover both top and hist tools where possible rtla/tests: Add get_workload_pids() helper rtla/tests: Check -c/--cpus thread affinity rtla/tests: Use negative match when testing --aa-only rtla/tests: Extend timerlat top --aa-only coverage rtla/tests: Cover all hist options in runtime tests rtla/tests: Add runtime test for -H/--house-keeping rtla/tests: Add runtime test for -k and -u options rtla/tests: Add runtime tests for -C/--cgroup

 tools/tracing/rtla/tests/engine.sh            |  15 +++
 tools/tracing/rtla/tests/osnoise.t            |  73 +++++++----
 .../rtla/tests/scripts/check-cgroup-match.sh  |  17 +++
 .../tracing/rtla/tests/scripts/check-cpus.sh  |   9 ++
 .../tests/scripts/check-housekeeping-cpus.sh  |   4 +
 .../rtla/tests/scripts/check-priority.sh      |   8 +-
 .../scripts/check-user-kernel-threads.sh      |  16 +++
 .../tests/scripts/lib/get_workload_pids.sh    |  11 ++
 tools/tracing/rtla/tests/timerlat.t           | 113 +++++++++++-------
 9 files changed, 194 insertions(+), 72 deletions(-)
 create mode 100755 tools/tracing/rtla/tests/scripts/check-cgroup-match.sh
 create mode 100755 tools/tracing/rtla/tests/scripts/check-cpus.sh
 create mode 100755 tools/tracing/rtla/tests/scripts/check-housekeeping-cpus.sh
 create mode 100755 tools/tracing/rtla/tests/scripts/check-user-kernel-threads.sh
 create mode 100644 tools/tracing/rtla/tests/scripts/lib/get_workload_pids.sh
Signature · 1 line
2.53.0
author maintainer
#1 lore raw

PATCH 1/9 rtla/tests: Cover both top and hist tools where possible

RTLA runtime tests currently do not cover both tool variants for osnoise and timerlat properly. Many tests applicable to both tools are only tested for one tool, selected randomly.

Introduce two new shell functions, check_top_hist() and check_top_q_hist(). The functions use the same syntax as check() and run check() on the arguments twice: once replacing the "TOOL" string in the command with "top" (or "top -q"), once replacing it with "hist". The top -q variant is used for tests relying on messages printed after aborting the RTLA main loop with a starting new line, which only happens for top tools in quiet mode; without -q, the top output is printed on the same line and the matches would fail.

Tests that are applicable to both top and hist tools were modified to the run for both; additionally, tests that were already done for both tools were migrated to the new shell functions, unless the test command or matches differ between the tools. Additional tests were added to test tool-specific help messages.

---
 tools/tracing/rtla/tests/engine.sh  | 15 ++++++
 tools/tracing/rtla/tests/osnoise.t  | 46 +++++++++--------
 tools/tracing/rtla/tests/timerlat.t | 76 ++++++++++++++---------------
 3 files changed, 73 insertions(+), 64 deletions(-)
tools/tracing/rtla/tests/engine.sh +15 -0
index ed261e07c6d9..27d92f19a322 100644
@@ -112,6 +112,21 @@ check_with_osnoise_options() {
 	NO_RESET_OSNOISE=1 check "$arg1" "$arg2" "$arg3"
 }
 
+check_top_hist() {
+	# Test one command with both "top" and "hist" tools, replacing "TOOL" in
+	# command with either "top" or "hist" respectively, and prefixing the test
+	# names with "top " and "hist ".
+	check "top $1" "$(echo "$2" | sed 's/TOOL/top/g')" "${@:3}"
+	check "hist $1" "$(echo "$2" | sed 's/TOOL/hist/g')" "${@:3}"
+}
+
+check_top_q_hist() {
+	# Same as above, but pass "-q" to top so that strings printed in main
+	# loop are on their own line for top too, not only for hist.
+	check "top $1" "$(echo "$2" | sed 's/TOOL/top -q/g')" "${@:3}"
+	check "hist $1" "$(echo "$2" | sed 's/TOOL/hist/g')" "${@:3}"
+}
+
 set_timeout() {
 	TIMEOUT="timeout -v -k 15s $1"
 }
tools/tracing/rtla/tests/osnoise.t +22 -24

62 lines not on this page · raw message

tools/tracing/rtla/tests/timerlat.t +36 -40

100 lines not on this page · raw message

Signature · 1 line
2.53.0
author maintainer
#2 lore raw

PATCH 2/9 rtla/tests: Add get_workload_pids() helper

In reply to Tomas Glozar

RTLA runtime tests that check workload processes (currently the test case "verify -P/--priority" of timerlat.t and "verify the --priority/-P param" of osnoise.t) use "pgrep timerlatu/" or "pgrep osnoise/" respectively to identify the workload.

Make them more robust by adding a get_workload_pids() helper that finds the main rtla process and returns the PIDs of all siblings other than the test script itself, plus all child processes of kthreadd that have the osnoise/timerlat kthread pattern comm.

This filters out any spurious processes not related to the running test that happen to have "timerlatu/" or "osnoise/" in their command, for example, a user grepping the same names at the time of the running of the test.

---
 tools/tracing/rtla/tests/osnoise.t                    |  2 +-
 tools/tracing/rtla/tests/scripts/check-priority.sh    |  8 ++++----
 .../rtla/tests/scripts/lib/get_workload_pids.sh       | 11 +++++++++++
 tools/tracing/rtla/tests/timerlat.t                   |  2 +-
 4 files changed, 17 insertions(+), 6 deletions(-)
 create mode 100644 tools/tracing/rtla/tests/scripts/lib/get_workload_pids.sh
tools/tracing/rtla/tests/osnoise.t +1 -1
index ce3a448b1f87..ed6ff0cc3329 100644
@@ -10,7 +10,7 @@ check "verify help page" \
 check_top_hist "verify help page" \
 	"osnoise TOOL --help" 0 "rtla osnoise"
 check_top_q_hist "verify the --priority/-P param" \
-	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh osnoise/ SCHED_FIFO 1\"" \
+	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh SCHED_FIFO 1\"" \
 	2 "Priorities are set correctly"
 check_top_q_hist "verify the --stop/-s param" \
 	"osnoise TOOL -s 30 -T 1" 2 "osnoise hit stop tracing"
tools/tracing/rtla/tests/scripts/check-priority.sh +4 -4
index 79b702a34a96..b51d5232a868 100755
@@ -1,8 +1,8 @@
 #!/bin/bash
 # SPDX-License-Identifier: GPL-2.0
-pids="$(pgrep ^$1)" || exit 1
-for pid in $pids
+. "$(dirname $0)/lib/get_workload_pids.sh"
+for pid in $(get_workload_pids)
 do
-  chrt -p $pid | cut -d ':' -f 2 | head -n1 | grep "^ $2\$" >/dev/null
-  chrt -p $pid | cut -d ':' -f 2 | tail -n1 | grep "^ $3\$" >/dev/null
+  chrt -p $pid | cut -d ':' -f 2 | head -n1 | grep "^ $1\$" >/dev/null
+  chrt -p $pid | cut -d ':' -f 2 | tail -n1 | grep "^ $2\$" >/dev/null
 done && echo "Priorities are set correctly"
tools/tracing/rtla/tests/scripts/lib/get_workload_pids.sh new file +11 -0
new file mode 100644
index 000000000000..8aff98cd2c1f
@@ -0,0 +1,11 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+get_workload_pids() {
+    local shell_pid=$$
+    local rtla_pid=$(ps -o ppid= $shell_pid)
+
+    # kernel threads
+    pgrep -P $(pgrep ^kthreadd$) -f '^(osnoise|timerlat)/[0-9]+$'
+    # user threads
+    pgrep -P $rtla_pid | grep -v "^$shell_pid$"
+}
tools/tracing/rtla/tests/timerlat.t +1 -1
index d7944710a859..765dffd9d42a 100644
@@ -27,7 +27,7 @@ check_top_hist "verify help page" \
 check_top_hist "verify -s/--stack" \
 	"timerlat TOOL -s 3 -T 10 -t" 2 "Blocking thread stack trace"
 check_top_hist "verify -P/--priority" \
-	"timerlat TOOL -P F:1 -c 0 -d 10s -T 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh timerlatu/ SCHED_FIFO 1\"" \
+	"timerlat TOOL -P F:1 -c 0 -d 10s -T 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh SCHED_FIFO 1\"" \
 	2 "Priorities are set correctly"
 check_top_hist "test in nanoseconds" \
 	"timerlat TOOL -i 2 -c 0 -n -d 10s" 2 "ns"
Signature · 1 line
2.53.0
author maintainer
#3 lore raw

PATCH 3/9 rtla/tests: Check -c/--cpus thread affinity

In reply to Tomas Glozar

RTLA runtime tests verify the -c/--cpus options, but do not check whether the correct affinity is actually applied.

Add a script named check-cpus.sh that retrieves the affinity of all workload threads and use it to check the -c/--cpus option for both osnoise and timerlat tools.

Also add missing -c/--cpus test for osnoise.

---
 tools/tracing/rtla/tests/osnoise.t             | 2 ++
 tools/tracing/rtla/tests/scripts/check-cpus.sh | 9 +++++++++
 tools/tracing/rtla/tests/timerlat.t            | 4 ++--
 3 files changed, 13 insertions(+), 2 deletions(-)
 create mode 100755 tools/tracing/rtla/tests/scripts/check-cpus.sh
tools/tracing/rtla/tests/osnoise.t +2 -0
index ed6ff0cc3329..5edffb23981b 100644
@@ -18,6 +18,8 @@ check_top_q_hist "verify the --trace param" \
 	"osnoise TOOL -s 30 -T 1 -t" 2 "Saving trace to osnoise_trace.txt"
 check "verify the --entries/-E param" \
 	"osnoise hist -P F:1 -c 0 -r 900000 -d 10s -b 10 -E 25"
+check_top_q_hist "verify the -c/--cpus param" \
+	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
 
 # Test setting default period by putting an absurdly high period
 # and stopping on threshold.
tools/tracing/rtla/tests/scripts/check-cpus.sh new file +9 -0
new file mode 100755
index 000000000000..0b016d4a7945
@@ -0,0 +1,9 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+. "$(dirname $0)/lib/get_workload_pids.sh"
+echo -n "Affinity of threads: "
+for pid in $(get_workload_pids)
+do
+    echo -n $(taskset -c -p $pid | cut -d ':' -f 2)
+done
+echo
tools/tracing/rtla/tests/timerlat.t +2 -2
index 765dffd9d42a..fb60022aaa64 100644
@@ -39,8 +39,8 @@ check "print the auto-analysis if hits the stop tracing condition" \
 	"timerlat top --aa-only 5" 2
 check_top_hist "disable auto-analysis" \
 	"timerlat TOOL -s 3 -T 10 -t --no-aa" 2
-check_top_hist "verify -c/--cpus" \
-	"timerlat TOOL -c 0 -d 10s"
+check_top_q_hist "verify -c/--cpus" \
+	"timerlat TOOL -c 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
 
 # Actions tests
 check_top_q_hist "trace output through -t" \
Signature · 1 line
2.53.0
author maintainer
#4 lore raw

PATCH 4/9 rtla/tests: Use negative match when testing --aa-only

In reply to Tomas Glozar

For testing the -a/--auto option in timerlat tool, the string "analyzing it" is matched against to make sure auto-analysis was triggered.

Use the same string as a negative match for --aa-only option test.

---
 tools/tracing/rtla/tests/timerlat.t | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/tracing/rtla/tests/timerlat.t +1 -1
index fb60022aaa64..f47a82c115c7 100644
@@ -38,7 +38,7 @@ check_top_hist "dump tasks" \
 check "print the auto-analysis if hits the stop tracing condition" \
 	"timerlat top --aa-only 5" 2
 check_top_hist "disable auto-analysis" \
-	"timerlat TOOL -s 3 -T 10 -t --no-aa" 2
+	"timerlat TOOL -s 3 -T 10 -t --no-aa" 2 "" "analyzing it"
 check_top_q_hist "verify -c/--cpus" \
 	"timerlat TOOL -c 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
 
Signature · 1 line
2.53.0
author maintainer
#5 lore raw

PATCH 5/9 rtla/tests: Extend timerlat top --aa-only coverage

In reply to Tomas Glozar

rtla-timerlat-top's --aa-only option is currently only tested for return value.

Extend the tests to also check that only auto-analysis is being done via a negative match for the "Timer Latency" text in the top header, and further split the test case into two:

- one test case for --aa-only stopping on threshold
- one test case for --aa-only exiting without threshold being hit

For both cases, the expected output ("analyzing it" or "Max latency was" respectively) is checked against in addition to the negative match.

---
 tools/tracing/rtla/tests/timerlat.t | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
tools/tracing/rtla/tests/timerlat.t +4 -2
index f47a82c115c7..28c01d8b299d 100644
@@ -35,8 +35,10 @@ check_top_hist "set the automatic trace mode" \
 	"timerlat TOOL -a 5" 2 "analyzing it"
 check_top_hist "dump tasks" \
 	"timerlat TOOL -a 5 --dump-tasks" 2 "Printing CPU tasks"
-check "print the auto-analysis if hits the stop tracing condition" \
-	"timerlat top --aa-only 5" 2
+check "verify --aa-only stop on threshold" \
+	"timerlat top --aa-only 5" 2 "analyzing it" "Timer Latency"
+check "verify --aa-only max latency" \
+	"timerlat top --aa-only 2000000 -d 1s" 0 "^  Max latency was" "Timer Latency"
 check_top_hist "disable auto-analysis" \
 	"timerlat TOOL -s 3 -T 10 -t --no-aa" 2 "" "analyzing it"
 check_top_q_hist "verify -c/--cpus" \
Signature · 1 line
2.53.0
author maintainer
#6 lore raw

PATCH 6/9 rtla/tests: Cover all hist options in runtime tests

In reply to Tomas Glozar

Cover all options regarding histogram formatting for both rtla-osnoise-hist and rtla-timerlat-hist tools. All options also have output checking using positive or negative match, except for -b/--bucket-size and -E/--entries, which cannot be tested in isolated due to the output depending on the actual data collected.

Old -E/--entries test for rtla-osnoise was replaced with a new one equivalent to the timerlat one.

---
 tools/tracing/rtla/tests/osnoise.t  | 18 ++++++++++++++++--
 tools/tracing/rtla/tests/timerlat.t | 20 ++++++++++++++++++++
 2 files changed, 36 insertions(+), 2 deletions(-)
tools/tracing/rtla/tests/osnoise.t +16 -2
index 5edffb23981b..773a46e2dc5f 100644
@@ -16,11 +16,25 @@ check_top_q_hist "verify the --stop/-s param" \
 	"osnoise TOOL -s 30 -T 1" 2 "osnoise hit stop tracing"
 check_top_q_hist "verify the --trace param" \
 	"osnoise TOOL -s 30 -T 1 -t" 2 "Saving trace to osnoise_trace.txt"
-check "verify the --entries/-E param" \
-	"osnoise hist -P F:1 -c 0 -r 900000 -d 10s -b 10 -E 25"
 check_top_q_hist "verify the -c/--cpus param" \
 	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
 
+# Histogram tests
+check "hist with -b/--bucket-size" \
+	"osnoise hist -b 1 -d 1s"
+check "hist with -E/--entries" \
+	"osnoise hist -E 10 -d 1s"
+check "hist with -E/--entries out of range" \
+	"osnoise hist -E 1 -d 1s" 1 "^Entries must be > 10 and < 9999999$"
+check "hist with --no-header" \
+	"osnoise hist --no-header -d 1s" 0 "" "RTLA osnoise histogram"
+check "hist with --with-zeros" \
+	"osnoise hist --with-zeros -b 100000 -E 21 -d 1s" 0 '^2000000\s+0\s+'
+check "hist with --no-index" \
+	"osnoise hist --no-index --with-zeros -d 1s" 0 "" "^count:"
+check "hist with --no-summary" \
+	"osnoise hist --no-summary -d 1s" 0 "" "^count:"
+
 # Test setting default period by putting an absurdly high period
 # and stopping on threshold.
 # If default period is not set, this will time out.
tools/tracing/rtla/tests/timerlat.t +20 -0
index 28c01d8b299d..a14d9ec32ede 100644
@@ -44,6 +44,26 @@ check_top_hist "disable auto-analysis" \
 check_top_q_hist "verify -c/--cpus" \
 	"timerlat TOOL -c 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
 
+# Histogram tests
+check "hist with -b/--bucket-size" \
+	"timerlat hist -b 1 -d 1s"
+check "hist with -E/--entries" \
+	"timerlat hist -E 10 -d 1s"
+check "hist with -E/--entries out of range" \
+	"timerlat hist -E 1 -d 1s" 1 "^Entries must be > 10 and < 9999999$"
+check "hist with --no-header" \
+	"timerlat hist --no-header -d 1s" 0 "" "RTLA timerlat histogram"
+check "hist with --with-zeros" \
+	"timerlat hist --with-zeros -b 100000 -E 21 -d 1s" 0 '^2000000\s+0\s+'
+check "hist with --no-index" \
+	"timerlat hist --no-index --with-zeros -d 1s" 0 "" "^count:"
+check "hist with --no-summary" \
+	"timerlat hist --no-summary -d 1s" 0 "" "^ALL:"
+check "hist with --no-irq" \
+	"timerlat hist --no-irq -d 1s" 0 "" "IRQ-"
+check "hist with --no-thread" \
+	"timerlat hist --no-thread -d 1s" 0 "" "Thr-"
+
 # Actions tests
 check_top_q_hist "trace output through -t" \
 	"timerlat TOOL -T 2 -t" 2 "^  Saving trace to timerlat_trace.txt$"
Signature · 1 line
2.53.0
author maintainer
#7 lore raw

PATCH 7/9 rtla/tests: Add runtime test for -H/--house-keeping

In reply to Tomas Glozar

Add a runtime test for -H/--house-keeping option for both osnoise and timerlat tools, with affinity checking similar to what is done for -c/--cpus.

---
 tools/tracing/rtla/tests/osnoise.t                          | 2 ++
 tools/tracing/rtla/tests/scripts/check-housekeeping-cpus.sh | 4 ++++
 tools/tracing/rtla/tests/timerlat.t                         | 2 ++
 3 files changed, 8 insertions(+)
 create mode 100755 tools/tracing/rtla/tests/scripts/check-housekeeping-cpus.sh
tools/tracing/rtla/tests/osnoise.t +2 -0
index 773a46e2dc5f..cdea84914345 100644
@@ -18,6 +18,8 @@ check_top_q_hist "verify the --trace param" \
 	"osnoise TOOL -s 30 -T 1 -t" 2 "Saving trace to osnoise_trace.txt"
 check_top_q_hist "verify the -c/--cpus param" \
 	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
+check_top_q_hist "verify the -H/--house-keeping param" \
+	"osnoise TOOL -P F:1 -H 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=tests/scripts/check-housekeeping-cpus.sh" 2 "^Affinity of threads: 0$"
 
 # Histogram tests
 check "hist with -b/--bucket-size" \
tools/tracing/rtla/tests/scripts/check-housekeeping-cpus.sh new file +4 -0
new file mode 100755
index 000000000000..4742f34efb49
@@ -0,0 +1,4 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+pid=$(ps -o ppid= $$)
+echo "Affinity of threads:$(taskset -c -p $pid | cut -d ':' -f 2)"
tools/tracing/rtla/tests/timerlat.t +2 -0
index a14d9ec32ede..20f68bcbcb27 100644
@@ -43,6 +43,8 @@ check_top_hist "disable auto-analysis" \
 	"timerlat TOOL -s 3 -T 10 -t --no-aa" 2 "" "analyzing it"
 check_top_q_hist "verify -c/--cpus" \
 	"timerlat TOOL -c 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
+check_top_q_hist "verify -H/--house-keeping" \
+	"timerlat TOOL -H 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-housekeeping-cpus.sh" 2 "^Affinity of threads: 0$"
 
 # Histogram tests
 check "hist with -b/--bucket-size" \
Signature · 1 line
2.53.0
author maintainer
#8 lore raw

PATCH 8/9 rtla/tests: Add runtime test for -k and -u options

In reply to Tomas Glozar

Add runtime test for rtla-timerlat's -k/--kernel-threads and -u/--user-threads options using get_workload_pids.sh to check whether the appropriate threads are being created.

The tests are implemented for both top and hist. Additionally, all tests related to timerlat threads are moved to a separate section in the test files. The latter is also done for rtla-osnoise tests.

---
 tools/tracing/rtla/tests/osnoise.t               |  8 +++++---
 .../tests/scripts/check-user-kernel-threads.sh   | 16 ++++++++++++++++
 tools/tracing/rtla/tests/timerlat.t              | 12 +++++++++---
 3 files changed, 30 insertions(+), 6 deletions(-)
 create mode 100755 tools/tracing/rtla/tests/scripts/check-user-kernel-threads.sh
tools/tracing/rtla/tests/osnoise.t +5 -3
index cdea84914345..d0b623233db5 100644
@@ -9,13 +9,15 @@ check "verify help page" \
 	"osnoise --help" 0 "osnoise version"
 check_top_hist "verify help page" \
 	"osnoise TOOL --help" 0 "rtla osnoise"
-check_top_q_hist "verify the --priority/-P param" \
-	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh SCHED_FIFO 1\"" \
-	2 "Priorities are set correctly"
 check_top_q_hist "verify the --stop/-s param" \
 	"osnoise TOOL -s 30 -T 1" 2 "osnoise hit stop tracing"
 check_top_q_hist "verify the --trace param" \
 	"osnoise TOOL -s 30 -T 1 -t" 2 "Saving trace to osnoise_trace.txt"
+
+# Thread tests
+check_top_q_hist "verify the --priority/-P param" \
+	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh SCHED_FIFO 1\"" \
+	2 "Priorities are set correctly"
 check_top_q_hist "verify the -c/--cpus param" \
 	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
 check_top_q_hist "verify the -H/--house-keeping param" \
tools/tracing/rtla/tests/scripts/check-user-kernel-threads.sh new file +16 -0
new file mode 100755
index 000000000000..bb7ac510a735
@@ -0,0 +1,16 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+. "$(dirname $0)/lib/get_workload_pids.sh"
+kthreadd_pid=$(pgrep ^kthreadd$)
+cnt_kernel=0
+cnt_user=0
+for pid in $(get_workload_pids)
+do
+    if [ "$(echo $(ps -o ppid= $pid))" = "$kthreadd_pid" ]
+    then
+        ((++cnt_kernel))
+    else
+        ((++cnt_user))
+    fi
+done
+echo "$cnt_kernel kernel threads, $cnt_user user threads"
tools/tracing/rtla/tests/timerlat.t +9 -3
index 20f68bcbcb27..3557adbdebae 100644
@@ -26,9 +26,6 @@ check_top_hist "verify help page" \
 	"timerlat TOOL --help" 0 "rtla timerlat"
 check_top_hist "verify -s/--stack" \
 	"timerlat TOOL -s 3 -T 10 -t" 2 "Blocking thread stack trace"
-check_top_hist "verify -P/--priority" \
-	"timerlat TOOL -P F:1 -c 0 -d 10s -T 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh SCHED_FIFO 1\"" \
-	2 "Priorities are set correctly"
 check_top_hist "test in nanoseconds" \
 	"timerlat TOOL -i 2 -c 0 -n -d 10s" 2 "ns"
 check_top_hist "set the automatic trace mode" \
@@ -41,10 +38,19 @@ check "verify --aa-only max latency" \
 	"timerlat top --aa-only 2000000 -d 1s" 0 "^  Max latency was" "Timer Latency"
 check_top_hist "disable auto-analysis" \
 	"timerlat TOOL -s 3 -T 10 -t --no-aa" 2 "" "analyzing it"
+
+# Thread tests
+check_top_hist "verify -P/--priority" \
+	"timerlat TOOL -P F:1 -c 0 -d 10s -T 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh SCHED_FIFO 1\"" \
+	2 "Priorities are set correctly"
 check_top_q_hist "verify -c/--cpus" \
 	"timerlat TOOL -c 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
 check_top_q_hist "verify -H/--house-keeping" \
 	"timerlat TOOL -H 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-housekeeping-cpus.sh" 2 "^Affinity of threads: 0$"
+check_top_q_hist "verify -k/--kernel-threads" \
+	"timerlat TOOL -k -c 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-user-kernel-threads.sh" 2 "1 kernel threads, 0 user threads"
+check_top_q_hist "verify -u/--user-threads" \
+	"timerlat TOOL -u -c 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-user-kernel-threads.sh" 2 "0 kernel threads, 1 user threads"
 
 # Histogram tests
 check "hist with -b/--bucket-size" \
Signature · 1 line
2.53.0
author maintainer
#9 lore raw

PATCH 9/9 rtla/tests: Add runtime tests for -C/--cgroup

In reply to Tomas Glozar

Add a new script check-cgroup-match.sh that retrieves the cgroup of the main rtla process and compares it to the cgroup of the rtla workload threads.

Add a new test based on this script, for both osnoise and timerlat tools, testing the variant of -C without argument (which sets the cgroup of the workload to the cgroup of the rtla main process).

Note that this has to be tested in kernel mode to be significant for timerlat tool, as user workloads inherit the parent rtla process cgroup even without the option.

---
 tools/tracing/rtla/tests/osnoise.t              |  3 +++
 .../rtla/tests/scripts/check-cgroup-match.sh    | 17 +++++++++++++++++
 tools/tracing/rtla/tests/timerlat.t             |  3 +++
 3 files changed, 23 insertions(+)
 create mode 100755 tools/tracing/rtla/tests/scripts/check-cgroup-match.sh
tools/tracing/rtla/tests/osnoise.t +3 -0
index d0b623233db5..06787471d0e8 100644
@@ -18,6 +18,9 @@ check_top_q_hist "verify the --trace param" \
 check_top_q_hist "verify the --priority/-P param" \
 	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh SCHED_FIFO 1\"" \
 	2 "Priorities are set correctly"
+check_top_q_hist "verify the -C/--cgroup param" \
+	"osnoise TOOL -C -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=\"tests/scripts/check-cgroup-match.sh\"" \
+	2 "cgroup matches for all workload PIDs"
 check_top_q_hist "verify the -c/--cpus param" \
 	"osnoise TOOL -P F:1 -c 0 -r 900000 -d 10s -S 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
 check_top_q_hist "verify the -H/--house-keeping param" \
tools/tracing/rtla/tests/scripts/check-cgroup-match.sh new file +17 -0
new file mode 100755
index 000000000000..fdc2c68c5957
@@ -0,0 +1,17 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+. "$(dirname $0)/lib/get_workload_pids.sh"
+rtla_pid=$(echo $(ps -o ppid= $$))
+rtla_cgroup=$(</proc/$rtla_pid/cgroup)
+echo "RTLA cgroup: $rtla_cgroup"
+for pid in $(get_workload_pids)
+do
+    pid_cgroup=$(</proc/$pid/cgroup)
+    echo "PID $pid cgroup: $pid_cgroup"
+    if ! [ "$pid_cgroup" = "$rtla_cgroup" ]
+    then
+        echo "Mismatch!"
+        exit 0
+    fi
+done
+echo "cgroup matches for all workload PIDs"
tools/tracing/rtla/tests/timerlat.t +3 -0
index 3557adbdebae..3ebfe316b39e 100644
@@ -43,6 +43,9 @@ check_top_hist "disable auto-analysis" \
 check_top_hist "verify -P/--priority" \
 	"timerlat TOOL -P F:1 -c 0 -d 10s -T 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh SCHED_FIFO 1\"" \
 	2 "Priorities are set correctly"
+check_top_hist "verify -C/--cgroup" \
+	"timerlat TOOL -k -C -c 0 -d 10s -T 1 --on-threshold shell,command=\"tests/scripts/check-cgroup-match.sh\"" \
+	2 "cgroup matches for all workload PIDs"
 check_top_q_hist "verify -c/--cpus" \
 	"timerlat TOOL -c 0 -d 10s -T 1 --on-threshold shell,command=tests/scripts/check-cpus.sh" 2 "^Affinity of threads: 0$"
 check_top_q_hist "verify -H/--house-keeping" \
Signature · 1 line
2.53.0
Reviewed-by on the v1 series
#10 lore raw
In reply to Tomas Glozar
Quoted 12 lines · Tomas Glozar

This patchset introduces some new tests to cover more options, especially histogram and thread options. Most of the new tests use positive and negative output matches, sometimes in combination with action scripts, to verify that RTLA is applying the settings correctly.

Tests were reorganized a little, adding two new sections: thread tests and histogram tests, next to basic tests.

Additionally, coverage of existing tests is extended by adding new matches and by extending tests to cover both top and hist tools where possible. For the latter, new helpers check_top_hist and check_top_q_hist are added to engine.sh.

Quoted 7 lines

As part of the new action scripts, detection of measurement threads is made more robust by following child processes of either RTLA (user workload) or kthreadd (kernel workload) rather than grepping through the comms of all processes, which might have lead to false positives.

These changes significantly improve test coverage and make the test suite more against false positives from unrelated processes.

Quoted 31 lines

Tomas Glozar (9):
rtla/tests: Cover both top and hist tools where possible rtla/tests: Add get_workload_pids() helper rtla/tests: Check -c/--cpus thread affinity rtla/tests: Use negative match when testing --aa-only rtla/tests: Extend timerlat top --aa-only coverage rtla/tests: Cover all hist options in runtime tests rtla/tests: Add runtime test for -H/--house-keeping rtla/tests: Add runtime test for -k and -u options rtla/tests: Add runtime tests for -C/--cgroup

 tools/tracing/rtla/tests/engine.sh            |  15 +++
 tools/tracing/rtla/tests/osnoise.t            |  73 +++++++----
 .../rtla/tests/scripts/check-cgroup-match.sh  |  17 +++
 .../tracing/rtla/tests/scripts/check-cpus.sh  |   9 ++
 .../tests/scripts/check-housekeeping-cpus.sh  |   4 +
 .../rtla/tests/scripts/check-priority.sh      |   8 +-
 .../scripts/check-user-kernel-threads.sh      |  16 +++
 .../tests/scripts/lib/get_workload_pids.sh    |  11 ++
 tools/tracing/rtla/tests/timerlat.t           | 113 +++++++++++-------
 9 files changed, 194 insertions(+), 72 deletions(-)
 create mode 100755 tools/tracing/rtla/tests/scripts/check-cgroup-match.sh
 create mode 100755 tools/tracing/rtla/tests/scripts/check-cpus.sh
 create mode 100755 tools/tracing/rtla/tests/scripts/check-housekeeping-cpus.sh
 create mode 100755 tools/tracing/rtla/tests/scripts/check-user-kernel-threads.sh
 create mode 100644 tools/tracing/rtla/tests/scripts/lib/get_workload_pids.sh
Signature · 1 line
2.53.0