A simple hardware detector for latency as well as throughtput ver 0.10
Subsystems (3)
cover A simple hardware detector for latency as well as throughtput ver 0.10
Back to July of this year, I sent the first round of the tool. Now I've polished it a little bit.
The patch is the fist step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SMI. Currently the patch tests TSC, CPU Frequency, and RDRAND, which is a new CPU instruction to get random number introudced in new CPU like Intel Ivy Bridge, in stop_machine context.
Some test I did on a old P4 and a latest IvyBridge latpop: X230 as follows:
P4:
[root@p4 hw_latency_test]# modprobe hw_latency_test [root@p4 hw_latency_test]# cd /sys/kernel/debug/hw_latency_test/ [root@p4 hw_latency_test]# ls
available count current enable max sample threshold window [root@p4 hw_latency_test]# cat available
mem random_bytes freq tsc
[root@p4 hw_latency_test]# echo mem > current
[root@p4 hw_latency_test]# echo 1 > enable
[root@p4 hw_latency_test]# cat sample
[0]1061813091.0918683721 176ns [ffffffffc0001329]
[0]1061813092.0419683746 132ns [ffffffffc0002750]
[0]1061813092.0920683770 194ns [ffffffffc0003993]
[0]1061813093.0421683794 194ns [ffffffffc0004bd6]
[0]1061813093.0922683818 175ns [ffffffffc0005f51]
[0]1061813094.0423683843 177ns [ffffffffc000722b]
[0]1061813094.0924683867 178ns [ffffffffc00085a5]
[0]1061813095.0425683891 177ns [ffffffffc0009920]
[0]1061813095.0926683915 179ns [ffffffffc000ac9a]
[0]1061813096.0427683940 224ns [ffffffffc000c013]
[0]1061813096.0928683964 179ns [ffffffffc000d38d]
[0]1061813097.0429683988 175ns [ffffffffc000e708]
[0]1061813097.0930684012 179ns [ffffffffc000fa82]
[0]1061813098.0431684037 194ns [ffffffffc0010dfc]
^C
[root@p4 hw_latency_test]# echo 0 > enable
[root@p4 hw_latency_test]# echo random_bytes > current [root@p4 hw_latency_test]# echo 1 > enable
[0]1061813148.0873686478 247us [0]
[0]1061813149.0874686526 244us [0]
[0]1061813150.0877686575 246us [0]
[0]1061813151.0878686623 245us [0]
^C
[root@p4 hw_latency_test]# echo 0 > enable
[root@p4 hw_latency_test]# echo freq > current
[root@p4 hw_latency_test]# echo 1 > enable
[root@p4 hw_latency_test]# cat sample
[0]1061813198.0701688889 245us [0]
[0]1061813199.0713688938 273us [0]
[0]1061813200.0725688987 232us [0]
[0]1061813201.0736689036 265us [0]
[0]1061813202.0747689085 257us [0]
[0]1061813203.0758689134 265us [0]
^C
[root@p4 hw_latency_test]# echo 0 > enable
[root@p4 hw_latency_test]# echo tsc > current
[root@p4 hw_latency_test]# echo 1 > enable
[0]1061813225.0514690187 138ns [0]
[0]1061813226.0015690211 139ns [0]
[0]1061813226.0516690236 152ns [0]
^C
[root@p4 hw_latency_test]#
IvyBridge (thinkpad x230):
[root@ivb hw_latency_test]# echo mem > current [root@ivb hw_latency_test]# echo 1 > enable [root@ivb hw_latency_test]# cat sample
[0]1352079648.0152669920 190ns [ffff880000002486]
[0]1352079648.0654669947 167ns [ffff88000000490c]
[0]1352079649.0155669973 317ns [ffff880000006d90]
[0]1352079649.0656669999 214ns [ffff880000009215]
[0]1352079650.0157670025 163ns [ffff88000000b72f]
[0]1352079650.0658670052 150ns [ffff88000000dbb5]
[0]1352079651.0159670078 398ns [ffff88000000ffa8]
[0]1352079651.0660670104 367ns [ffff88000001242a]
[0]1352079652.0161670131 193ns [ffff8800000148b0]
[root@ivb hw_latency_test]# echo 0 > enable [root@ivb hw_latency_test]# echo random_bytes > current [root@ivb hw_latency_test]# echo 1 > threshold [root@ivb hw_latency_test]# echo 1 > enable [root@ivb hw_latency_test]# cat sample
[3]1352079707.0143673015 17us [0]
[0]1352079708.0144673068 16us [0]
[1]1352079708.0144673068 16us [0]
[2]1352079708.0144673068 24us [0]
[3]1352079708.0144673068 17us [0]
[0]1352079709.0145673120 24us [0]
[1]1352079709.0145673120 17us [0]
[2]1352079709.0145673120 17us [0]
[3]1352079709.0145673120 22us [0]
^C
ot@ivb hw_latency_test]# echo 0 > enable
[root@ivb hw_latency_test]# echo freq > current
[root@ivb hw_latency_test]# echo 10 > threshold
[root@ivb hw_latency_test]# echo 1 > enable
[root@ivb hw_latency_test]#
[root@ivb hw_latency_test]# cat sample
[0]1352079763.0780675986 195us [0]
[1]1352079764.0285676013 281us [0]
[2]1352079764.0790676039 193us [0]
[3]1352079765.0296676066 225us [0]
[0]1352079766.0303676119 228us [0]
[1]1352079766.0808676145 269us [0]
[2]1352079767.0313676172 195us [0]
[3]1352079767.0819676198 220us [0]
^C
[root@ivb hw_latency_test]# echo 0 > enable
[root@ivb hw_latency_test]# echo tsc > current
[root@ivb hw_latency_test]# echo 1 > enable
[root@ivb hw_latency_test]# cat sample
[0]1352079798.0116677788 76ns [0]
[1]1352079798.0116677788 84ns [0]
[2]1352079798.0116677788 52ns [0]
[3]1352079798.0116677788 52ns [0]
[0]1352079798.0617677814 89ns [0]
[1]1352079798.0617677814 84ns [0]
[2]1352079798.0617677814 74ns [0]
[3]1352079798.0617677814 61ns [0]
^C
Luming Yu (13):
HW-latency: hardware latency test 0.10
HW-latency: Fix a lockdep warnning
HW-latency: Use get_random_bytes_arch
HW-latency: Differentiate three modes to use CPU carry out testing
HW-latency: Add CPU field in sample output
HW-latency: cycle through all online cpus to re-test cpufreq
HW-latency: delete too many "Fast TSC calibration using PIT" in
cpufreq sampling
HW-latency: A stupid memory scanner for raw memory latency test
HW-latency: Fix unwanted crash caused by write to dummy debugfs
interface
HW-latency: add address range for x86-32
HW-latency: fix a warnning in previous patch
HW-latency: Add sample unit in sample data
HW-latency: some sample data format change
arch/x86/kernel/tsc.c | 2 +- drivers/misc/Kconfig | 7 + drivers/misc/Makefile | 1 + drivers/misc/hw_latency_test.c | 922 +++++++++++++++++++++++++++++++++++++++++ fs/libfs.c | 2 +- 5 files changed, 932 insertions(+), 2 deletions(-) create mode 100644 drivers/misc/hw_latency_test.c
Signature · 1 line
1.7.12.1
PATCH 1/13 HW-latency: hardware latency test 0.10
This patch is the first step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SMI. Currently the patch tests TSC, CPU Frequency and RDRAND which is a new CPU instruction to get random number introduced in new CPU like Intel Ivy Bridge in stop_machine context,which is choosen to make sure testers fully control their system under test to rule out some level of unwanted noise.
--- drivers/misc/Kconfig | 7 + drivers/misc/Makefile | 1 + drivers/misc/hw_latency_test.c | 833 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 841 insertions(+) create mode 100644 drivers/misc/hw_latency_test.c
drivers/misc/Kconfig +7 -0
@@ -114,6 +114,13 @@ config IBM_ASM for information on the specific driver level and support statement for your IBM server. +config HW_LATENCY_TEST + tristate "Testing module to detect hardware lattency and throughput" + depends on DEBUG_FS + depends on RING_BUFFER + depends on X86 + default m + config PHANTOM tristate "Sensable PHANToM (PCI)" depends on PCI
drivers/misc/Makefile +1 -0
@@ -49,3 +49,4 @@ obj-y += carma/ obj-$(CONFIG_USB_SWITCH_FSA9480) += fsa9480.o obj-$(CONFIG_ALTERA_STAPL) +=altera-stapl/ obj-$(CONFIG_INTEL_MEI) += mei/ +obj-$(CONFIG_HW_LATENCY_TEST) += hw_latency_test.o
drivers/misc/hw_latency_test.c new file +833 -0
833 lines not on this page · raw message
Signature · 1 line
1.7.12.1
Re: [PATCH 01/13] HW-latency: hardware latency test 0.10
Hey,
Op 05-11-12 02:59, Luming Yu schreef:
Quoted 22 lines
This patch is the first step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SMI. Currently the patch tests TSC, CPU Frequency and RDRAND which is a new CPU instruction to get random number introduced in new CPU like Intel Ivy Bridge in
stop_machinecontext,which is choosen to make sure testers fully control their system under test to rule out some level of unwanted noise.Signed-off-by Luming Yu <[email protected]>--- drivers/misc/Kconfig | 7 + drivers/misc/Makefile | 1 + drivers/misc/hw_latency_test.c | 833 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 841 insertions(+) create mode 100644 drivers/misc/hw_latency_test.cdrivers/misc/Kconfig +0
-0
drivers/misc/Kconfig config IBM_ASM · line 122
Quoted 9 lines
drivers/misc/Kconfig +6
-0@@ -114,6 +114,13 @@ config IBM_ASM for information on the specific driver level and support statement for your IBM server. +config HW_LATENCY_TEST + tristate "Testing module to detect hardware lattency and throughput" + depends on DEBUG_FS + depends on RING_BUFFER + depends on X86 + default m
Is there any reason this tester couldn't easily be made to work for !x86?
Also I think it would make more sense to squash all fixes, and submit fixes for the things like '[PATCH 07/13] HW-latency: delete too many "Fast TSC calibration using PIT" in cpufreq sampling' before the actual patch. It seems this is not necessarily a hw-latency specific patch to me.
~Maarten
Re: [PATCH 01/13] HW-latency: hardware latency test 0.10
Quoted 28 lines · Maarten Lankhorst
Hey,
Op 05-11-12 02:59, Luming Yu schreef:
Quoted 25 lines
This patch is the first step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SMI. Currently the patch tests TSC, CPU Frequency and RDRAND which is a new CPU instruction to get random number introduced in new CPU like Intel Ivy Bridge in
stop_machinecontext,which is choosen to make sure testers fully control their system under test to rule out some level of unwanted noise.Signed-off-by Luming Yu <[email protected]>--- drivers/misc/Kconfig | 7 + drivers/misc/Makefile | 1 + drivers/misc/hw_latency_test.c | 833 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 841 insertions(+) create mode 100644 drivers/misc/hw_latency_test.cdrivers/misc/Kconfig +0
-0@@ -114,6 +114,13 @@ config IBM_ASM for information on the specific driver level and support statement for your IBM server.
Quoted 8 lines
drivers/misc/Kconfig config IBM_ASM · line 122
Quoted 6 lines
drivers/misc/Kconfig +6
-0+config HW_LATENCY_TEST + tristate "Testing module to detect hardware lattency and throughput" + depends on DEBUG_FS + depends on RING_BUFFER + depends on X86 + default mIs there any reason this tester couldn't easily be made to work for !x86?
Only problem is that I don't have platform to test.
Quoted 3 lines
Also I think it would make more sense to squash all fixes, and submit fixes for the things like '[PATCH 07/13] HW-latency: delete too many "Fast TSC calibration using PIT" in cpufreq sampling' before the actual patch. It seems this is not necessarily a hw-latency specific patch to me.
Correct. I can do that later. I don't care it's a single big patch or a patch series. It depends on how to help maintainers who would be interested in taking this feature into upstream. Although I will take all responsibility to fix all bugs coming with it and the future enhancement of the feature.
The biggest problem to me right now for this feature is to find it a way into upstream. I WILL do everything necessary. One big patch instead of 13 patches-set is fine. Making it platform neutral is fine too as long as it's necessary to make it upstream.
Quoted 2 lines
~Maarten
Re: [PATCH 01/13] HW-latency: hardware latency test 0.10
In reply to Luming YuQuoted 75 lines · Luming Yu
This patch is the first step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SMI. Currently the patch tests TSC, CPU Frequency and RDRAND which is a new CPU instruction to get random number introduced in new CPU like Intel Ivy Bridge in
stop_machinecontext,which is choosen to make sure testers fully control their system under test to rule out some level of unwanted noise.Signed-off-by Luming Yu <[email protected]>--- drivers/misc/Kconfig | 7 + drivers/misc/Makefile | 1 + drivers/misc/hw_latency_test.c | 833 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 841 insertions(+) create mode 100644 drivers/misc/hw_latency_test.cdrivers/misc/Kconfig +7
-0@@ -114,6 +114,13 @@ config IBM_ASM for information on the specific driver level and support statement for your IBM server. +config HW_LATENCY_TEST + tristate "Testing module to detect hardware lattency and throughput" + depends on DEBUG_FS + depends on RING_BUFFER + depends on X86 + default m + config PHANTOM tristate "Sensable PHANToM (PCI)" depends on PCIdrivers/misc/Makefile +1
-0@@ -49,3 +49,4 @@ obj-y += carma/ obj-$(CONFIG_USB_SWITCH_FSA9480) += fsa9480.o obj-$(CONFIG_ALTERA_STAPL) +=altera-stapl/ obj-$(CONFIG_INTEL_MEI) += mei/ +obj-$(CONFIG_HW_LATENCY_TEST) += hw_latency_test.odrivers/misc/hw_latency_test.c new file +24
-0@@ -0,0 +1,833 @@ +#include <linux/module.h> +#include <linux/init.h> +#include <linux/ring_buffer.h> +#include <linux/stop_machine.h> +#include <linux/time.h> +#include <linux/hrtimer.h> +#include <linux/kthread.h> +#include <linux/debugfs.h> +#include <linux/seq_file.h> +#include <linux/uaccess.h> +#include <linux/version.h> +#include <linux/delay.h> +#include <linux/slab.h> +#include <linux/random.h> +#include <asm/tlbflush.h> + +#define BUF_SIZE_DEFAULT 262144UL +#define BUF_FLAGS (RB_FL_OVERWRITE) +#define U64STR_SIZE 22 +#define DEBUGFS_BUF_SIZE 1024 +#define DEBUGFS_NAME_SIZE 32 + +#define VERSION "0.1.0" +#define BANNER "hardware latency test"
drivers/misc/hw_latency_test.c line 34
Quoted 10 lines
drivers/misc/hw_latency_test.c new file +10
-0+#define DRVNAME "hw_latency_test" + +#define DEFAULT_SAMPLE_WINDOW 1000000 +#define DEFAULT_SAMPLE_WIDTH 500000 +#define DEFAULT_LAT_THRESHOLD 10 + +MODULE_LICENSE ("GPL" ); +MODULE_AUTHOR ("Luming Yu <[email protected]>" ); +MODULE_DESCRIPTION ("A simple hardware latency test" ); +MODULE_VERSION (VERSION );
-------->o SNIP
Ok, hopefully this is just an unintentional oversight - but I don't see where you are acknowledging that the original author of most of this code is Jon Masters. It's fine for you to work on it, but you have to somewhere acknowledge where it comes from.
Thanks
John
Re: [PATCH 01/13] HW-latency: hardware latency test 0.10
Quoted 90 lines · John Kacur
Quoted 75 lines · Luming Yu
This patch is the first step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SMI. Currently the patch tests TSC, CPU Frequency and RDRAND which is a new CPU instruction to get random number introduced in new CPU like Intel Ivy Bridge in
stop_machinecontext,which is choosen to make sure testers fully control their system under test to rule out some level of unwanted noise.Signed-off-by Luming Yu <[email protected]>--- drivers/misc/Kconfig | 7 + drivers/misc/Makefile | 1 + drivers/misc/hw_latency_test.c | 833 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 841 insertions(+) create mode 100644 drivers/misc/hw_latency_test.cdrivers/misc/Kconfig +7
-0@@ -114,6 +114,13 @@ config IBM_ASM for information on the specific driver level and support statement for your IBM server. +config HW_LATENCY_TEST + tristate "Testing module to detect hardware lattency and throughput" + depends on DEBUG_FS + depends on RING_BUFFER + depends on X86 + default m + config PHANTOM tristate "Sensable PHANToM (PCI)" depends on PCIdrivers/misc/Makefile +1
-0@@ -49,3 +49,4 @@ obj-y += carma/ obj-$(CONFIG_USB_SWITCH_FSA9480) += fsa9480.o obj-$(CONFIG_ALTERA_STAPL) +=altera-stapl/ obj-$(CONFIG_INTEL_MEI) += mei/ +obj-$(CONFIG_HW_LATENCY_TEST) += hw_latency_test.odrivers/misc/hw_latency_test.c new file +24
-0@@ -0,0 +1,833 @@ +#include <linux/module.h> +#include <linux/init.h> +#include <linux/ring_buffer.h> +#include <linux/stop_machine.h> +#include <linux/time.h> +#include <linux/hrtimer.h> +#include <linux/kthread.h> +#include <linux/debugfs.h> +#include <linux/seq_file.h> +#include <linux/uaccess.h> +#include <linux/version.h> +#include <linux/delay.h> +#include <linux/slab.h> +#include <linux/random.h> +#include <asm/tlbflush.h> + +#define BUF_SIZE_DEFAULT 262144UL +#define BUF_FLAGS (RB_FL_OVERWRITE) +#define U64STR_SIZE 22 +#define DEBUGFS_BUF_SIZE 1024 +#define DEBUGFS_NAME_SIZE 32 + +#define VERSION "0.1.0" +#define BANNER "hardware latency test"drivers/misc/hw_latency_test.c line 34
Quoted 10 lines
drivers/misc/hw_latency_test.c new file +10
-0+#define DRVNAME "hw_latency_test" + +#define DEFAULT_SAMPLE_WINDOW 1000000 +#define DEFAULT_SAMPLE_WIDTH 500000 +#define DEFAULT_LAT_THRESHOLD 10 + +MODULE_LICENSE ("GPL" ); +MODULE_AUTHOR ("Luming Yu <[email protected]>" ); +MODULE_DESCRIPTION ("A simple hardware latency test" ); +MODULE_VERSION (VERSION );-------->o SNIP
Quoted 4 lines
Ok, hopefully this is just an unintentional oversight - but I don't see where you are acknowledging that the original author of most of this code is Jon Masters. It's fine for you to work on it, but you have to somewhere acknowledge where it comes from.
Thanks very much for pointing it out.
I did ACK Jon Masters in the first push of this feature in July of this year.
Yes, I will add Jon Master to be the first author of the work in right place, I will
claim secondary place for myself . :-)
Quoted 4 lines
Thanks
John
Re: [PATCH 01/13] HW-latency: hardware latency test 0.10
In reply to Luming YuQuoted 10 lines · Luming Yu
This patch is the first step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SMI. Currently the patch tests TSC, CPU Frequency and RDRAND which is a new CPU instruction to get random number introduced in new CPU like Intel Ivy Bridge in
stop_machinecontext,which is choosen to make sure testers fully control their system under test to rule out some level of unwanted noise.Signed-off-by Luming Yu <[email protected]>
That patch has my full stamp of approval!
Re: [PATCH 01/13] HW-latency: hardware latency test 0.10
In reply to Luming YuQuoted 10 lines · Luming Yu
This patch is the first step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SMI. Currently the patch tests TSC, CPU Frequency and RDRAND which is a new CPU instruction to get random number introduced in new CPU like Intel Ivy Bridge in
stop_machinecontext,which is choosen to make sure testers fully control their system under test to rule out some level of unwanted noise.Signed-off-by Luming Yu <[email protected]>
That patch has my full stamp of approval!
PATCH 2/13 HW-latency: Fix a lockdep warnning
In reply to Luming Yu[ 150.733088] in_atomic(): 1, irqs_disabled(): 1, pid: 19, name: migration/2 [ 150.735456] INFO: lockdep is turned off. [ 150.738903] irq event stamp: 28 [ 150.741255] hardirqs last enabled at (27): [<ffffffff816e6a80>] _raw_spin_unlock_irq+0x30/0x50 [ 150.744884] hardirqs last disabled at (28): [<ffffffff816e616f>] _raw_spin_lock_irq+0x1f/0x90 [ 150.748505] softirqs last enabled at (24): [<ffffffff810724a7>] __do_softirq+0x167/0x3d0 [ 150.752121] softirqs last disabled at (19): [<ffffffff816f143c>] call_softirq+0x1c/0x30 [ 150.755709] Pid: 19, comm: migration/2 Tainted: G W 3.7.0-rc2+ #2 [ 150.759361] Call Trace: [ 150.762952] [<ffffffff810d53e0>] ? print_irqtrace_events+0xd0/0xe0 [ 150.766610] [<ffffffff810a3b4c>] __might_sleep+0x18c/0x250 [ 150.769140] [<ffffffff816e2ab0>] mutex_lock_nested+0x40/0x390 [ 150.772765] [<ffffffff810d853d>] ? trace_hardirqs_on+0xd/0x10 [ 150.776392] [<ffffffff816e6a80>] ? _raw_spin_unlock_irq+0x30/0x50 [ 150.778899] [<ffffffff81021829>] ? read_tsc+0x9/0x20 [ 150.782546] [<ffffffffa05877e3>] get_freq_sample+0x33/0xd0 [hw_latency_test] [ 150.786249] [<ffffffff810fac9b>] stop_machine_cpu_stop+0xab/0x110 [ 150.789964] [<ffffffff810fadd6>] cpu_stopper_thread+0xd6/0x1b0 [ 150.792528] [<ffffffff810fabf0>] ? cpu_stop_queue_work+0x80/0x80 [ 150.795044] [<ffffffff8115798e>] ? perf_event_alloc+0x5e/0x4a0 [ 150.797572] [<ffffffff816e4672>] ? __schedule+0x442/0xa00 [ 150.800079] [<ffffffff810fad00>] ? stop_machine_cpu_stop+0x110/0x110 [ 150.802512] [<ffffffff810938fd>] kthread+0xed/0x100 [ 150.805825] [<ffffffff81093810>] ? flush_kthread_worker+0x190/0x190 [ 150.808051] [<ffffffff816f00ec>] ret_from_fork+0x7c/0xb0 [ 150.810161] [<ffffffff81093810>] ? flush_kthread_worker+0x190/0x190
--- drivers/misc/hw_latency_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/misc/hw_latency_test.c +1 -1
@@ -149,7 +149,7 @@ static int get_random_bytes_sample(void *unused)u64 sample =0 ;int ret =1 ; - buffer =kzalloc (1024 ,GFP_KERNEL ); + buffer =kzalloc (1024 ,GFP_ATOMIC ); start =ktime_get (); do {
Signature · 1 line
1.7.12.1
PATCH 3/13 HW-latency: Use get_random_bytes_arch
In reply to Luming YuTo test rdrand on CPU like Ivy Bridget, we need feature aware random function.
--- drivers/misc/hw_latency_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/misc/hw_latency_test.c +1 -1
@@ -155,7 +155,7 @@ static int get_random_bytes_sample(void *unused) do { t1 = ktime_get(); - get_random_bytes(buffer, 1024); + get_random_bytes_arch(buffer, 1024); t2 = ktime_get(); total = ktime_to_us(ktime_sub(t2, start)); diff = ktime_to_us(ktime_sub(t2, t1));
Signature · 1 line
1.7.12.1
PATCH 4/13 HW-latency: Differentiate three modes to use CPU carry out testing
In reply to Luming Yu0: all online cpus
1: any one of online cpus
2: all online cpus sequentially run test
--- drivers/misc/hw_latency_test.c | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-)
drivers/misc/hw_latency_test.c +32 -5
@@ -47,6 +47,7 @@ static unsigned long buf_size = 262144UL;static struct task_struct *kthread;struct sample { +unsigned int cpu;u64 seqnum;u64 duration;struct timespec timestamp;
@@ -70,6 +71,7 @@ static struct data {static ktime_t now;struct sample_function {const char *name; +u8 type;/* 0=all parallel, 1=anyone, 2=all sequential*/ struct list_head list;int (*get_sample)(void *unused); };
@@ -186,14 +188,11 @@ static int get_freq_sample(void *unused)u32 sample =0 ;int ret =1 ;unsigned int cpu_tsc_freq; -static DEFINE_MUTEX (freq_pit_mutex); start =ktime_get (); do { t1 = ktime_get(); - mutex_lock(&freq_pit_mutex); cpu_tsc_freq = x86_platform.calibrate_tsc(); - mutex_unlock(&freq_pit_mutex); t2 = ktime_get(); total = ktime_to_us(ktime_sub(t2, start)); diff = abs(cpu_tsc_freq - tsc_khz);
@@ -249,23 +248,30 @@ out:struct sample_function tsc_sample = { .name ="tsc" , + .type =0 , .get_sample = get_tsc_sample, };struct sample_function tsc_freq_sample = { .name ="freq" , + .type =2 , .get_sample = get_freq_sample, };struct sample_function random_bytes_sample = { .name ="random_bytes" , + .type =0 , .get_sample = get_random_bytes_sample, }; +static DECLARE_BITMAP(testing_cpu_map, NR_CPUS); +static int kthread_fn (void *unused) {int err =0 ;u64 interval =0 ; +int cpu; +struct cpumask *testing_cpu_mask =to_cpumask (testing_cpu_map);int (*get_sample)(void *unused);mutex_lock (&sample_function_mutex);
@@ -274,10 +280,31 @@ static int kthread_fn(void *unused)else goto out; +cpumask_or (testing_cpu_mask, testing_cpu_mask, cpu_online_mask);while (!kthread_should_stop ()) {mutex_lock (&data.lock); - - err =stop_machine (get_sample, unused, cpu_online_mask); + +switch (current_sample_func->type) { +case 0 : + err =stop_machine (get_sample, unused, testing_cpu_mask); +break ; +case 1 : + err =stop_machine (get_sample, unused,NULL ); +break ; +case 2 : +for_each_cpu (cpu, cpu_online_mask) { +cpumask_clear (testing_cpu_mask); +cpumask_set_cpu (cpu, testing_cpu_mask); + err =stop_machine (get_sample, unused,NULL ); +if (err) +break ; + } +break ; +default : +mutex_unlock (&data.lock); + goto err_out; + } +if (err) {mutex_unlock (&data.lock); goto err_out;
Signature · 1 line
1.7.12.1
PATCH 5/13 HW-latency: Add CPU field in sample output
In reply to Luming YuThe filed tells user the sample is from which cpu.
--- drivers/misc/hw_latency_test.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/misc/hw_latency_test.c +4 -1
@@ -124,11 +124,13 @@ static struct sample *buffer_get_sample(struct sample *sample)static int buffer_add_sample (u64 sample) {int ret =0 ; +unsigned int cpu =smp_processor_id ();if (sample > data.threshold) {struct sample s; data.count++; + s.cpu = cpu; s.seqnum = data.count; s.duration = sample; s.timestamp =CURRENT_TIME ;
@@ -615,7 +617,8 @@ static ssize_t debug_sample_fread(struct file *filp, char __user *ubuf, goto out; } } - len =snprintf (buf,sizeof (buf),"%010lu.%010lu\t%llu\n" , + len =snprintf (buf,sizeof (buf),"[%d]%010lu.%010lu\t%llu\n" , + sample->cpu, sample->timestamp.tv_sec, sample->timestamp.tv_nsec, sample->duration);
Signature · 1 line
1.7.12.1
PATCH 6/13 HW-latency: cycle through all online cpus to re-test cpufreq
In reply to Luming YuFix a typo to enable cycling through all online cpus to get cpufreq sample from all cpus sequentially.
--- drivers/misc/hw_latency_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/misc/hw_latency_test.c +1 -1
@@ -297,7 +297,7 @@ static int kthread_fn(void *unused)for_each_cpu (cpu, cpu_online_mask) {cpumask_clear (testing_cpu_mask);cpumask_set_cpu (cpu, testing_cpu_mask); - err =stop_machine (get_sample, unused,NULL ); + err =stop_machine (get_sample, unused, testing_cpu_mask);if (err)break ; }
Signature · 1 line
1.7.12.1
PATCH 7/13 HW-latency: delete too many "Fast TSC calibration using PIT" in cpufreq sampling
In reply to Luming Yu"Fast TSC calibration using PIT" should be a devel info.
--- arch/x86/kernel/tsc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/tsc.c +1 -1
@@ -393,7 +393,7 @@ success:*/ delta *=PIT_TICK_RATE ;do_div (delta, i*256 *1000 ); -pr_info ("Fast TSC calibration using PIT\n" ); +pr_devel ("Fast TSC calibration using PIT\n" );return delta; }
Signature · 1 line
1.7.12.1
PATCH 8/13 HW-latency: A stupid memory scanner for raw memory latency test
In reply to Luming YuIt's simple memory latency test without any consideration of memory hierarchy and memory model. It just does simply scan byte by byte.
--- drivers/misc/hw_latency_test.c | 53 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 51 insertions(+), 2 deletions(-)
drivers/misc/hw_latency_test.c +51 -2
@@ -46,11 +46,15 @@ static DEFINE_MUTEX(ring_buffer_mutex);static unsigned long buf_size =262144UL ;static struct task_struct *kthread; +static u8 *__start = (u8 *)0xffff880000000000 ; +static u8 *__end = (u8 *)0xffffc7ffffffffff ; +struct sample {unsigned int cpu;u64 seqnum;u64 duration;struct timespec timestamp; +u64 addr;unsigned long lost; };
@@ -134,6 +138,7 @@ static int buffer_add_sample(u64 sample) s.seqnum = data.count; s.duration = sample; s.timestamp =CURRENT_TIME ; + s.addr = (u64 ) __start; ret =__buffer_add_sample (&s);if (sample > data.max_sample)
@@ -247,6 +252,42 @@ out:return ret; } +static int get_mem_sample (void *unused) +{ +ktime_t start, t1, t2; +s64 diff, total =0 ; +u64 sample =0 ; +int ret =1 ; +u8 temp; + + now = start =ktime_get (); +do { + t1 = now; + now = t2 =ktime_get (); + + total =ktime_to_ns (ktime_sub (t2, start)); + temp = *__start++; + diff =ktime_to_ns (ktime_sub (t2, t1)); + +if (diff <0 ) { + printk(KERN_ERR BANNER "time running backwards\n"); + goto out; + } + +if (diff > sample) + sample = diff; + +if (__start == __end) { + __start = (u8 *)0xffff880000000000 ; + printk(KERN_INFO BANNER "one pass finished, jmp to the beginning\n"); + } + + }while (total <= data.sample_width); + + ret =buffer_add_sample (sample); +out: +return ret; +}struct sample_function tsc_sample = { .name ="tsc" ,
@@ -266,6 +307,12 @@ struct sample_function random_bytes_sample = { .get_sample = get_random_bytes_sample, }; +struct sample_function mem_sample = { + .name ="mem" , + .type =1 , + .get_sample = get_mem_sample, +}; + static DECLARE_BITMAP(testing_cpu_map, NR_CPUS);static int kthread_fn (void *unused)
@@ -617,11 +664,12 @@ static ssize_t debug_sample_fread(struct file *filp, char __user *ubuf, goto out; } } - len =snprintf (buf,sizeof (buf),"[%d]%010lu.%010lu\t%llu\n" , + len =snprintf (buf,sizeof (buf),"[%d]%010lu.%010lu\t%llu\t[%llx]\n" , sample->cpu, sample->timestamp.tv_sec, sample->timestamp.tv_nsec, - sample->duration); + sample->duration, + sample->addr);if (len > cnt) goto out;if (copy_to_user (ubuf, buf,len))
@@ -827,6 +875,7 @@ static int hw_test_init(void)sample_function_register (&tsc_sample);sample_function_register (&tsc_freq_sample);sample_function_register (&random_bytes_sample); +sample_function_register (&mem_sample); ret =init_stats ();if (0 != ret)
Signature · 1 line
1.7.12.1
PATCH 9/13 HW-latency: Fix unwanted crash caused by write to dummy debugfs interface
In reply to Luming Yu[ 141.311906] BUG: unable to handle kernel NULL pointer dereference at 0000000000000008 [ 141.314071] IP: [<ffffffff811f8f3c>] simple_attr_write+0x2c/0x100 [ 141.316195] PGD c3bd7067 PUD cb41d067 PMD 0 [ 141.318287] Oops: 0000 [#1] SMP [ 141.320338] Modules linked in: hw_latency_test lockd sunrpc iptable_mangle nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack rfcomm bnep coretemp kvm arc4 iwldvm mac80211 snd_hda_codec_hdmi snd_hda_codec_realtek option usb_wwan snd_hda_intel snd_hda_codec btusb bluetooth snd_hwdep snd_seq snd_seq_device snd_pcm iwlwifi thinkpad_acpi cfg80211 snd_page_alloc snd_timer crc32c_intel snd e1000e tpm_tis ghash_clmulni_intel tpm tpm_bios soundcore iTCO_wdt rfkill joydev microcode i2c_i801 wmi iTCO_vendor_support mei lpc_ich mfd_core pcspkr uinput i915 usb_storage i2c_algo_bit uas drm_kms_helper sdhci_pci sdhci drm mmc_core i2c_core video [ 141.329446] CPU 2 [ 141.329467] Pid: 804, comm: bash Not tainted 3.7.0-rc2+ #5 LENOVO 232045C/232045C [ 141.333922] RIP: 0010:[<ffffffff811f8f3c>] [<ffffffff811f8f3c>] simple_attr_write+0x2c/0x100 [ 141.336173] RSP: 0018:ffff8800cb6c3eb8 EFLAGS: 00010286 [ 141.338377] RAX: ffffffff811f8f10 RBX: ffff8800c4549600 RCX: ffff8800cb6c3f50 [ 141.340573] RDX: 0000000000000002 RSI: 00007fcbf9ef0000 RDI: ffff8800c4549600 [ 141.342744] RBP: ffff8800cb6c3ef8 R08: 000000000000000a R09: 00007fcbf9edd740 [ 141.344896] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000002 [ 141.347017] R13: 00007fcbf9ef0000 R14: ffff8800cb6c3f50 R15: 0000000000000000 [ 141.349115] FS: 00007fcbf9edd740(0000) GS:ffff880119200000(0000) knlGS:0000000000000000 [ 141.351209] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 141.353314] CR2: 0000000000000008 CR3: 00000000c696c000 CR4: 00000000001407e0 [ 141.355457] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 141.357590] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 141.359685] Process bash (pid: 804, threadinfo ffff8800cb6c2000, task ffff8800cb7ccd20) [ 141.361767] Stack: [ 141.363793] ffff8800c4549600 00007fcbf9ef0000 ffff8800cb6c3ef8 ffff8800c4549600 [ 141.365864] 0000000000000002 00007fcbf9ef0000 ffff8800cb6c3f50 0000000000000000 [ 141.367905] ffff8800cb6c3f28 ffffffff811cf27f ffff8800c4549600 00007fcbf9ef0000 [ 141.369924] Call Trace: [ 141.371882] [<ffffffff811cf27f>] vfs_write+0xaf/0x190 [ 141.373827] [<ffffffff811cf5d5>] sys_write+0x55/0xa0 [ 141.375745] [<ffffffff816f0199>] system_call_fastpath+0x16/0x1b [ 141.377661] Code: 1f 44 00 00 55 48 89 e5 48 83 ec 40 48 89 5d d8 4c 89 65 e0 4c 89 6d e8 4c 89 75 f0 4c 89 7d f8 4c 8b bf 28 01 00 00 48 89 75 c8 <49> 83 7f 08 00 0f 84 b1 00 00 00 4d 8d 67 50 31 f6 49 89 d5 4c [ 141.382206] RIP [<ffffffff811f8f3c>] simple_attr_write+0x2c/0x100 [ 141.384326] RSP <ffff8800cb6c3eb8> [ 141.386401] CR2: 0000000000000008 [ 141.388548] ---[ end trace 9c28eee46fcb7871 ]---
--- fs/libfs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
fs/libfs.c +1 -1
@@ -819,7 +819,7 @@ ssize_t simple_attr_write(struct file *file, const char __user *buf,ssize_t ret; attr = file->private_data; -if (!attr->set) +if (!attr || !attr->set)return -EACCES ; ret =mutex_lock_interruptible (&attr->mutex);
Signature · 1 line
1.7.12.1
PATCH 10/13 HW-latency: add address range for x86-32
In reply to Luming YuAdd address range for x86-32
--- drivers/misc/hw_latency_test.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/misc/hw_latency_test.c +5 -0
@@ -46,8 +46,13 @@ static DEFINE_MUTEX(ring_buffer_mutex);static unsigned long buf_size =262144UL ;static struct task_struct *kthread; +#ifdef CONFIG_X86_64 static u8 *__start = (u8 *)0xffff880000000000 ;static u8 *__end = (u8 *)0xffffc7ffffffffff ; +#elif CONFIG_X86_32 +static u8 *__start = (u8 *)0xc0000000 ; +static u8 *__end = (u8 *)0xffffffff ; +#endif struct sample {unsigned int cpu;
Signature · 1 line
1.7.12.1
PATCH 11/13 HW-latency: fix a warnning in previous patch
In reply to Luming Yudrivers/misc/hw_latency_test.c:52:7: warning: "CONFIG_X86_32" is not defined [-Wundef]
--- drivers/misc/hw_latency_test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/misc/hw_latency_test.c +1 -1
@@ -49,7 +49,7 @@ static struct task_struct *kthread;#ifdef CONFIG_X86_64 static u8 *__start = (u8 *)0xffff880000000000 ;static u8 *__end = (u8 *)0xffffc7ffffffffff ; -#elif CONFIG_X86_32 +#else static u8 *__start = (u8 *)0xc0000000 ;static u8 *__end = (u8 *)0xffffffff ;#endif
Signature · 1 line
1.7.12.1
PATCH 12/13 HW-latency: Add sample unit in sample data
In reply to Luming Yuns and us
--- drivers/misc/hw_latency_test.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
drivers/misc/hw_latency_test.c +9 -6
@@ -60,6 +60,7 @@ struct sample {u64 duration;struct timespec timestamp;u64 addr; +u8 unit;/*0: ns 1:us*/ unsigned long lost; };
@@ -130,7 +131,7 @@ static struct sample *buffer_get_sample(struct sample *sample)return sample; } -static int buffer_add_sample (u64 sample) +static int buffer_add_sample (u64 sample,u8 unit) {int ret =0 ;unsigned int cpu =smp_processor_id ();
@@ -144,6 +145,7 @@ static int buffer_add_sample(u64 sample) s.duration = sample; s.timestamp =CURRENT_TIME ; s.addr = (u64 ) __start; + s.unit = unit; ret =__buffer_add_sample (&s);if (sample > data.max_sample)
@@ -184,7 +186,7 @@ static int get_random_bytes_sample(void *unused) }while (total <= data.sample_width); - ret =buffer_add_sample (sample); + ret =buffer_add_sample (sample,1 ); out:kfree (buffer);return ret;
@@ -219,7 +221,7 @@ static int get_freq_sample(void *unused) }while (total <= data.sample_width); - ret =buffer_add_sample (sample); + ret =buffer_add_sample (sample,1 ); out:return ret; }
@@ -252,7 +254,7 @@ static int get_tsc_sample(void *unused) }while (total <= data.sample_width); - ret =buffer_add_sample (sample); + ret =buffer_add_sample (sample,0 ); out:return ret; }
@@ -289,7 +291,7 @@ static int get_mem_sample(void *unused) }while (total <= data.sample_width); - ret =buffer_add_sample (sample); + ret =buffer_add_sample (sample,0 ); out:return ret; }
@@ -669,11 +671,12 @@ static ssize_t debug_sample_fread(struct file *filp, char __user *ubuf, goto out; } } - len =snprintf (buf,sizeof (buf),"[%d]%010lu.%010lu\t%llu\t[%llx]\n" , + len =snprintf (buf,sizeof (buf),"[%d]%010lu.%010lu\t%llu%2s\t[%llx]\n" , sample->cpu, sample->timestamp.tv_sec, sample->timestamp.tv_nsec, sample->duration, + sample->unit ?"us" :"ns" , sample->addr);if (len > cnt) goto out;
Signature · 1 line
1.7.12.1
PATCH 13/13 HW-latency: some sample data format change
In reply to Luming Yusome trival format changes
--- drivers/misc/hw_latency_test.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/misc/hw_latency_test.c +3 -1
@@ -671,13 +671,15 @@ static ssize_t debug_sample_fread(struct file *filp, char __user *ubuf, goto out; } } + len =snprintf (buf,sizeof (buf),"[%d]%010lu.%010lu\t%llu%2s\t[%llx]\n" , sample->cpu, sample->timestamp.tv_sec, sample->timestamp.tv_nsec, sample->duration, sample->unit ?"us" :"ns" , - sample->addr); + (((u64) (current_sample_func->get_sample) + == (u64 ) get_mem_sample) ? sample->addr:0 ));if (len > cnt) goto out;if (copy_to_user (ubuf, buf,len))
Signature · 1 line
1.7.12.1
Quoted 9 lines · Luming Yu
Back to July of this year, I sent the first round of the tool. Now I've polished it a little bit.
The patch is the fist step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SMI. Currently the patch tests TSC, CPU Frequency, and RDRAND, which is a new CPU instruction to get random number introudced in new CPU like Intel Ivy Bridge, in
stop_machinecontext.
You are adding a new file, drivers/misc/hw_latency_test.c to the kernel. I don't think it makes sense to include the file, and then follow it with 12 patches which modify the file. You don't need to "show your work" with the various bug fixes that has been made since July. I'd recommend collapsing all of the collected patches plus the base file into a single commit.
Regards,
- Ted