Skip to contentSkip to conversation

CPU-Idle latency selftest framework

RFC New Pratik R. Sampat 2 participants 4 messages Jun 11, 2021 – Jun 22, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover CPU-Idle latency selftest framework

A kernel module + userspace driver to estimate the wakeup latency caused by going into stop states. The motivation behind this program is to find significant deviations behind advertised latency and residency values.

The patchset measures latencies for two kinds of events. IPIs and Timers As this is a software-only mechanism, there will additional latencies of the kernel-firmware-hardware interactions. To account for that, the program also measures a baseline latency on a 100 percent loaded CPU and the latencies achieved must be in view relative to that.

To achieve this, we introduce a kernel module and expose its control knobs through the debugfs interface that the selftests can engage with.

The kernel module provides the following interfaces within /sys/kernel/debug/latency_test/ for,

IPI test:
ipi_cpu_dest = Destination CPU for the IPI
ipi_cpu_src = Origin of the IPI
ipi_latency_ns = Measured latency time in ns
Timeout test:
timeout_cpu_src = CPU on which the timer to be queued timeout_expected_ns = Timer duration
timeout_diff_ns = Difference of actual duration vs expected timer

Sample output on a POWER9 system is as follows:
# --IPI Latency Test---
# Baseline Average IPI latency(ns): 3114
# Observed Average IPI latency(ns) - Snooze: 3265 # Observed Average IPI latency(ns) - Stop0_lite: 3507 # Observed Average IPI latency(ns) - Stop0: 3739 # Observed Average IPI latency(ns) - Stop2: 3807 # Observed Average IPI latency(ns) - Stop4: 17070 # Observed Average IPI latency(ns) - Stop5: 1038174 #
# --Timeout Latency Test--
# Baseline Average timeout diff(ns): 1420
# Observed Average timeout diff(ns) - Snooze: 1640 # Observed Average timeout diff(ns) - Stop0_lite: 1764 # Observed Average timeout diff(ns) - Stop0: 1715 # Observed Average timeout diff(ns) - Stop2: 1845 # Observed Average timeout diff(ns) - Stop4: 16581 # Observed Average timeout diff(ns) - Stop5: 939977

Pratik R. Sampat (2):
powerpc/cpuidle: Extract IPI based and timer based wakeup latency from
idle states
powerpc/selftest: Add support for cpuidle latency measurement

 arch/powerpc/kernel/Makefile                  |   1 +
 arch/powerpc/kernel/test-cpuidle_latency.c    | 157 +++++++
 lib/Kconfig.debug                             |  10 +
 tools/testing/selftests/powerpc/Makefile      |   1 +
 .../powerpc/cpuidle_latency/.gitignore        |   2 +
 .../powerpc/cpuidle_latency/Makefile          |   6 +
 .../cpuidle_latency/cpuidle_latency.sh        | 419 ++++++++++++++++++
 .../powerpc/cpuidle_latency/settings          |   1 +
 8 files changed, 597 insertions(+)
 create mode 100644 arch/powerpc/kernel/test-cpuidle_latency.c
 create mode 100644 tools/testing/selftests/powerpc/cpuidle_latency/.gitignore
 create mode 100644 tools/testing/selftests/powerpc/cpuidle_latency/Makefile
 create mode 100755 tools/testing/selftests/powerpc/cpuidle_latency/cpuidle_latency.sh
 create mode 100644 tools/testing/selftests/powerpc/cpuidle_latency/settings
Signature · 1 line
2.17.1
#1 lore raw

RFC 1/2 powerpc/cpuidle: Extract IPI based and timer based wakeup latency from idle states

Introduce a mechanism to fire directed IPIs from a specified source CPU to a specified target CPU and measure the difference in time incurred on wakeup.

Also, introduce a mechanism to queue a HR timer on a specified CPU and subsequently measure the time taken to wakeup the CPU.

Finally define a simple debugfs interface to control the knobs to fire the IPI and Timer events on specified CPU and view their incurred idle wakeup latencies.

---
 arch/powerpc/kernel/Makefile               |   1 +
 arch/powerpc/kernel/test-cpuidle_latency.c | 157 +++++++++++++++++++++
 lib/Kconfig.debug                          |  10 ++
 3 files changed, 168 insertions(+)
 create mode 100644 arch/powerpc/kernel/test-cpuidle_latency.c
arch/powerpc/kernel/Makefile +1 -0
index f66b63e81c3b..56e36e797dd4 100644
@@ -56,6 +56,7 @@ obj-$(CONFIG_PPC_WATCHDOG)	+= watchdog.o
 obj-$(CONFIG_HAVE_HW_BREAKPOINT)	+= hw_breakpoint.o
 obj-$(CONFIG_PPC_DAWR)		+= dawr.o
 obj-$(CONFIG_PPC_BOOK3S_64)	+= cpu_setup_ppc970.o cpu_setup_pa6t.o
+obj-$(CONFIG_IDLE_LATENCY_SELFTEST)	  += test-cpuidle_latency.o
 obj-$(CONFIG_PPC_BOOK3S_64)	+= cpu_setup_power.o
 obj-$(CONFIG_PPC_BOOK3S_64)	+= mce.o mce_power.o
 obj-$(CONFIG_PPC_BOOK3E_64)	+= exceptions-64e.o idle_book3e.o
arch/powerpc/kernel/test-cpuidle_latency.c new file +157 -0

157 lines not on this page · raw message

lib/Kconfig.debug +10 -0
index 417c3d3e521b..70264687e0c2 100644
@@ -1513,6 +1513,16 @@ config DEBUG_KOBJECT
 	  If you say Y here, some extra kobject debugging messages will be sent
 	  to the syslog.
 
+config IDLE_LATENCY_SELFTEST
+	tristate "Cpuidle latency selftests"
+	depends on CPU_IDLE
+	help
+	  This option provides a kernel module that runs tests using the IPI and
+	  timers to measure latency.
+
+	  Say M if you want these self tests to build as a module.
+	  Say N if you are unsure.
+
 config DEBUG_KOBJECT_RELEASE
 	bool "kobject release debugging"
 	depends on DEBUG_OBJECTS_TIMERS
Signature · 1 line
2.17.1
#2 lore raw

Re: [RFC 1/2] powerpc/cpuidle: Extract IPI based and timer based wakeup latency from idle states

[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi "Pratik,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on powerpc/next]
[also build test ERROR on linux/master linus/master v5.13-rc7 next-20210621] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Pratik-R-Sampat/CPU-Idle-latency-selftest-framework/20210616-221820
base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: powerpc64-randconfig-r021-20210622 (attached as .config)
compiler: powerpc64-linux-gcc (GCC) 9.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/cdad9e7811445b4042cf43d31a5e07a1e1c0d1f6
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Pratik-R-Sampat/CPU-Idle-latency-selftest-framework/20210616-221820 git checkout cdad9e7811445b4042cf43d31a5e07a1e1c0d1f6
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=powerpc64

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>):

>> arch/powerpc/kernel/test-cpuidle_latency.c:34:6: error: no previous prototype for 'run_smp_call_function_test' [-Werror=missing-prototypes]

34 | void run_smp_call_function_test(unsigned int cpu)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

vim +/run_smp_call_function_test +34 arch/powerpc/kernel/test-cpuidle_latency.c

    33	
  > 34	void run_smp_call_function_test(unsigned int cpu)
    35	{
    36		ipi_wakeup.src_cpu = smp_processor_id();
    37		ipi_wakeup.dest_cpu = cpu;
    38		ipi_wakeup.time_start = ktime_get();
    39		smp_call_function_single(cpu, measure_latency, &ipi_wakeup, 1);
    40	}
    41	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#3 lore raw

RFC 2/2 powerpc/selftest: Add support for cpuidle latency measurement

In reply to Pratik R. Sampat

The cpuidle latency selftest provides support to systematically extract, analyse and present IPI and timer based wakeup latencies for each CPU and each idle state available on the system.

The selftest leverages test-cpuidle_latency module's debugfs interface to interact and extract latency information from the kernel.

The selftest inserts the module if already not inserted, disables all the idle states and enables them one by one testing the following:
1. Keeping source CPU constant, iterate through all the CPUS measuring IPI latency for baseline (CPU is busy with cat /dev/random > /dev/null workload) and then when the CPU is allowed to be at rest
2. Iterating through all the CPUs, sending expected timer durations to be equivalent to the residency of the deepest idle state enabled and extracting the difference in time between the time of wakeup and the expected timer duration

To run this test specifically:

$ sudo make -C tools/testing/selftests \
  TARGETS="powerpc/cpuidle_latency" run_tests

There are a few optional arguments too that the script can take
[-h <help>]
[-m <location of the module>]
[-o <location of the output>]
[-v <verbose> (run on all cpus)]
Default Output location in:
tools/testing/selftests/powerpc/cpuidle_latency/cpuidle_latency.log

To run the test without re-compiling:

$ cd tools/testing/selftest/powerpc/cpuidle_latency/
$ sudo ./cpuidle_latency.sh
---
 tools/testing/selftests/powerpc/Makefile      |   1 +
 .../powerpc/cpuidle_latency/.gitignore        |   2 +
 .../powerpc/cpuidle_latency/Makefile          |   6 +
 .../cpuidle_latency/cpuidle_latency.sh        | 419 ++++++++++++++++++
 .../powerpc/cpuidle_latency/settings          |   1 +
 5 files changed, 429 insertions(+)
 create mode 100644 tools/testing/selftests/powerpc/cpuidle_latency/.gitignore
 create mode 100644 tools/testing/selftests/powerpc/cpuidle_latency/Makefile
 create mode 100755 tools/testing/selftests/powerpc/cpuidle_latency/cpuidle_latency.sh
 create mode 100644 tools/testing/selftests/powerpc/cpuidle_latency/settings
tools/testing/selftests/powerpc/Makefile +1 -0
index 0830e63818c1..71ce6fff867d 100644
@@ -17,6 +17,7 @@ SUB_DIRS = alignment		\
 	   benchmarks		\
 	   cache_shape		\
 	   copyloops		\
+	   cpuidle_latency	\
 	   dscr			\
 	   mm			\
 	   nx-gzip		\
tools/testing/selftests/powerpc/cpuidle_latency/.gitignore new file +2 -0
new file mode 100644
index 000000000000..987f8852dc59
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+cpuidle_latency.log
tools/testing/selftests/powerpc/cpuidle_latency/Makefile new file +6 -0
new file mode 100644
index 000000000000..04492b6d2582
@@ -0,0 +1,6 @@
+# SPDX-License-Identifier: GPL-2.0
+all:
+
+TEST_PROGS := cpuidle_latency.sh
+
+include ../../lib.mk
tools/testing/selftests/powerpc/cpuidle_latency/cpuidle_latency.sh new file +419 -0

419 lines not on this page · raw message

tools/testing/selftests/powerpc/cpuidle_latency/settings new file +1 -0
new file mode 100644
index 000000000000..e7b9417537fb
@@ -0,0 +1 @@
+timeout=0
Signature · 1 line
2.17.1