a simple hardware detector for latency as well as throughput ver. 0.1.0

Hi Jon,

The patch is the fist step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SM. Currently the patch tests hardware features (tsc,freq, and rdrandom whiich is new instruction to get random number) in stop_machine context. I will add more after the first step get merged for those guys who want to directly play with new hardware functions.

I suppose I can add your signed-off-by as the code is derived from your hwlat_dector.

I'm also reuqesting if you are going to queue it up somewhere that can be pulled into 3.5.

Of cause, I will update the patch based upon any comments that you think must be fixed for 3.5 merge.

Thanks,
Luming

Signed-off-by Luming Yu <[email protected]>

 Kconfig   |    7 
 Makefile  |    2 
 hw_test.c |  954 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 963 insertions(+)

index c779509..f66ad56 100644
@@ -123,6 +123,13 @@ config IBM_ASM
 	  for information on the specific driver level and support statement
 	  for your IBM server.
 
+config HW_TEST
+	tristate "Testing module to detect hardware lattency and throughput"
+	depends on DEBUG_FS
+	depends on RING_BUFFER
+	depends on X86
+	default m
+
 config PHANTOM
 	tristate "Sensable PHANToM (PCI)"
 	depends on PCI

index 3e1d801..ec1de86 100644
@@ -48,4 +48,6 @@ obj-y				+= lis3lv02d/
 obj-y				+= carma/
 obj-$(CONFIG_USB_SWITCH_FSA9480) += fsa9480.o
 obj-$(CONFIG_ALTERA_STAPL)	+=altera-stapl/
+obj-$(CONFIG_HW_TEST)		+= hw_test.o
+
 obj-$(CONFIG_MAX8997_MUIC)	+= max8997-muic.o

Hi,

I'd like to know if the patch looks good for linux-next to find its way upstream in 3.6.

Thanks and regards,
Luming

---------- Forwarded message ----------
From: Luming Yu <[email protected]>
Date: Wed, May 30, 2012 at 7:47 AM
Subject: Fwd: [patch] a simple hardware detector for latency as well as throughput ver. 0.1.0
To: Andrew Morton <[email protected]>
Cc: [email protected]

Hello akpm,

I'd like to push the patch to upstream, but I'm not sure if jcm has extra bandwidth although he is also interested in having the tool upstream..So I'd like ping you to check if there is any chance to queue it up in your tree first.I will enhance it further after it's upstream.

Thanks,
Luming

---------- Forwarded message ----------
From: Luming Yu <[email protected]>
Date: Tue, May 29, 2012 at 4:37 AM
Subject: [patch] a simple hardware detector for latency as well as throughput ver. 0.1.0
To: [email protected]
Cc: [email protected]

Hi Jon,

The patch is the fist step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SM. Currently the patch tests hardware features (tsc,freq, and rdrandom whiich is new instruction to get random number) in stop_machine context. I will add more after the first step get merged for those guys who want to directly play with new hardware functions.

I suppose I can add your signed-off-by as the code is derived from your hwlat_dector.

I'm also reuqesting if you are going to queue it up somewhere that can be pulled into 3.5.

Of cause, I will update the patch based upon any comments that you think must be fixed for 3.5 merge.

Thanks,
Luming

Signed-off-by Luming Yu <[email protected]>

 Kconfig   |    7
 Makefile  |    2
 hw_test.c |  954
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 3 files changed, 963 insertions(+)

index c779509..f66ad56 100644
@@ -123,6 +123,13 @@ config IBM_ASM

         for information on the specific driver level and support statement for your IBM server.

+config HW_TEST
+       tristate "Testing module to detect hardware lattency and throughput"
+       depends on DEBUG_FS
+       depends on RING_BUFFER
+       depends on X86
+       default m
+
 config PHANTOM
       tristate "Sensable PHANToM (PCI)"
       depends on PCI

index 3e1d801..ec1de86 100644
@@ -48,4 +48,6 @@ obj-y                         += lis3lv02d/

 obj-y                          += carma/
 obj-$(CONFIG_USB_SWITCH_FSA9480) += fsa9480.o
 obj-$(CONFIG_ALTERA_STAPL)     +=altera-stapl/
+obj-$(CONFIG_HW_TEST)          += hw_test.o
+
 obj-$(CONFIG_MAX8997_MUIC)     += max8997-muic.o

count, (ssize_t) 0),
+                               "\n");
+       count = simple_read_from_buffer(ubuf, cnt, ppos, buf, count);
+       kfree(buf);
+       return count;
+}
+
+static ssize_t debug_available_fwrite(struct file *filp, const char __user *ubuf,
+                                       size_t  cnt, loff_t *ppos) +{
+       return (ssize_t) 0;
+}
+
+static int debug_current_fopen(struct inode *inode, struct file *filp) +{
+       return 0;
+}
+
+static ssize_t debug_current_fread(struct file *filp, char __user *ubuf,
+                                       size_t cnt, loff_t *ppos) +{
+       ssize_t count = 0;
+       char *buf;
+
+       buf = kzalloc(DEBUGFS_NAME_SIZE, GFP_KERNEL);
+       if (!buf)
+               return -ENOMEM;
+
+       count += snprintf(buf + count,
+               max((ssize_t)DEBUGFS_NAME_SIZE - count, (ssize_t)0),
+                       "%s ", current_sample_func->name);
+       count += snprintf(buf + count,
+                       max((ssize_t)DEBUGFS_NAME_SIZE - count, (ssize_t)0),
+                       "\n");
+       count = simple_read_from_buffer(ubuf, cnt, ppos, buf, count);
+       kfree(buf);
+
+       return count;
+}
+
+static ssize_t debug_current_fwrite(struct file *filp, const char __user *ubuf,
+                                       size_t cnt, loff_t *ppos) +{
+       char *buf;
+       ssize_t count;
+       struct sample_function *sf;
+
+       buf = kzalloc(DEBUGFS_NAME_SIZE, GFP_KERNEL);
+       if (!buf)
+               return -ENOMEM;
+       count = simple_write_to_buffer(buf, DEBUGFS_NAME_SIZE, ppos, ubuf, cnt);
+       mutex_lock(&sample_function_mutex);
+       list_for_each_entry(sf, &sample_function_list, list) {
+               if (strncmp(sf->name, buf, count-1) !=0)
+                       continue;
+               current_sample_func = sf;
+               break;
+       }
+       mutex_unlock(&sample_function_mutex);
+       return (ssize_t) count;
+}
+
+static int debug_count_fopen(struct inode *inode, struct file *filp) +{
+       return 0;
+}
+
+static ssize_t debug_count_fread(struct file *filp, char __user *ubuf,
+                                       size_t cnt, loff_t *ppos) +{
+       return simple_data_read(filp, ubuf, cnt, ppos, &data.count); +}
+static ssize_t debug_count_fwrite(struct file *filp, const char __user *ubuf,
+                                       size_t cnt,
+                                       loff_t *ppos)
+{
+       return simple_data_write(filp, ubuf, cnt, ppos, &data.count); +}
+static int debug_enable_fopen(struct inode *inode, struct file *filp) +{
+       return 0;
+}
+static ssize_t debug_enable_fread(struct file *filp, char __user *ubuf,
+                                       size_t cnt, loff_t *ppos) +{
+       char buf[4];
+       if ((cnt < sizeof(buf)) || (*ppos))
+               return 0;
+       buf[0] = enabled ? '1' : '0';
+       buf[1] = '\n';
+       buf[2] = '\0';
+       if (copy_to_user(ubuf, buf, strlen(buf)))
+               return -EFAULT;
+       return *ppos = strlen(buf);
+}
+static ssize_t debug_enable_fwrite(struct file *filp,
+                                       const char __user *ubuf,
+                                       size_t cnt,
+                                       loff_t *ppos)
+{
+       char buf[4];
+       int csize = min(cnt, sizeof(buf));
+       long val = 0;
+       int err = 0;
+
+       memset(buf, '\0', sizeof(buf));
+       if (copy_from_user(buf, ubuf, csize))
+               return -EFAULT;
+       buf[sizeof(buf)-1] = '\0';
+       err = strict_strtoul(buf, 10, &val);
+       if (0 != err)
+               return -EINVAL;
+       if (val) {
+               if (enabled)
+                       goto unlock;
+               enabled = 1;
+               if (start_kthread())
+                       return -EFAULT;
+       } else {
+               if (!enabled)
+                       goto unlock;
+               enabled = 0;
+               err = stop_kthread();
+               if (err) {
+                       printk(KERN_ERR BANNER "cannot stop kthread\n");
+                       return -EFAULT;
+               }
+               wake_up(&data.wq);
+       }
+unlock:
+       return csize;
+}
+static int debug_max_fopen(struct inode *inode, struct file *filp) +{
+       return 0;
+}
+static ssize_t debug_max_fread(struct file *filp, char __user *ubuf,
+                               size_t cnt, loff_t *ppos)
+{
+       return simple_data_read(filp, ubuf, cnt, ppos, &data.max_sample); +}
+static ssize_t debug_max_fwrite(struct file *filp,
+                               const char __user *ubuf,
+                               size_t  cnt,
+                               loff_t  *ppos)
+{
+       return simple_data_write(filp, ubuf, cnt, ppos, &data.max_sample); +}
+static int debug_sample_fopen(struct inode *inode, struct file *filp) +{
+       if (!atomic_add_unless(&data.sample_open, 1, 1))
+               return -EBUSY;
+       else
+               return 0;
+}
+static ssize_t debug_sample_fread(struct file *filp, char __user *ubuf,
+                                       size_t cnt, loff_t *ppos) +{
+       int len = 0;
+       char buf[64];
+       struct sample *sample = NULL;
+
+       if (!enabled)
+               return 0;
+       sample = kzalloc(sizeof(struct sample), GFP_KERNEL);
+       if(!sample)
+               return -ENOMEM;
+
+       while (!buffer_get_sample(sample)) {
+               DEFINE_WAIT(wait);
+               if (filp->f_flags & O_NONBLOCK) {
+                       len = -EAGAIN;
+                       goto out;
+               }
+               prepare_to_wait(&data.wq, &wait, TASK_INTERRUPTIBLE);
+               schedule();
+               finish_wait(&data.wq, &wait);
+               if (signal_pending(current)) {
+                       len = -EINTR;
+                       goto out;
+               }
+               if (!enabled) {
+                       len = 0;
+                       goto out;
+               }
+       }
+       len = snprintf(buf, sizeof(buf), "%010lu.%010lu\t%llu\n",
+                       sample->timestamp.tv_sec,
+                       sample->timestamp.tv_nsec,
+                       sample->duration);
+       if (len > cnt)
+               goto out;
+       if (copy_to_user(ubuf, buf,len))
+               len = -EFAULT;
+out:
+       kfree(sample);
+       return len;
+}
+
+static int debug_sample_release(struct inode *inode, struct file *filp) +{
+       atomic_dec(&data.sample_open);
+       return 0;
+}
+static int debug_threshold_fopen(struct inode *inode, struct file *filp) +{
+       return 0;
+}
+static ssize_t debug_threshold_fread(struct file *filp, char __user *ubuf,
+                                       size_t cnt, loff_t *ppos) +{
+       return simple_data_read(filp, ubuf, cnt, ppos, &data.threshold); +}
+static ssize_t debug_threshold_fwrite(struct file *filp,
+                                       const char __user *ubuf,
+                                       size_t cnt,
+                                       loff_t *ppos)
+{
+       int ret;
+       ret = simple_data_write(filp, ubuf, cnt, ppos, &data.threshold);
+       if (enabled)
+               wake_up_process(kthread);
+       return ret;
+}
+static int debug_width_fopen(struct inode *inode, struct file *filp) +{
+       return 0;
+}
+static ssize_t debug_width_fread(struct file *filp, char __user *ubuf,
+                               size_t cnt, loff_t *ppos)
+{
+       return simple_data_read(filp, ubuf, cnt, ppos, &data.sample_width); +}
+static ssize_t debug_width_fwrite(struct file *filp,
+                                       const char __user *ubuf,
+                                       size_t cnt,
+                                       loff_t *ppos)
+{
+       char buf[U64STR_SIZE];
+       int csize = min(cnt, sizeof(buf));
+       u64 val = 0;
+       int err = 0;
+
+       memset(buf, '\0', sizeof(buf));
+       if (copy_from_user(buf, ubuf, csize))
+               return -EFAULT;
+       buf[U64STR_SIZE-1] = '\0';
+       err = strict_strtoull(buf, 10, &val);
+       if (0 != err)
+               return -EINVAL;
+       mutex_lock(&data.lock);
+       if (val < data.sample_window)
+               data.sample_width = val;
+       else {
+               mutex_unlock(&data.lock);
+               return -EINVAL;
+       }
+       mutex_unlock(&data.lock);
+       if (enabled)
+               wake_up_process(kthread);
+
+       return csize;
+}
+static int debug_window_fopen(struct inode *inode, struct file *filp) +{
+       return 0;
+}
+static ssize_t debug_window_fread(struct file *filp, char __user *ubuf,
+                               size_t cnt, loff_t *ppos)
+{
+       return simple_data_read(filp, ubuf, cnt, ppos, &data.sample_window); +}
+static ssize_t debug_window_fwrite(struct file *filp,
+                                       const char __user *ubuf,
+                                       size_t cnt,
+                                       loff_t *ppos)
+{
+       char buf[U64STR_SIZE];
+       int csize = min(cnt, sizeof(buf));
+       u64 val = 0;
+       int err = 0;
+
+       memset(buf, '\0', sizeof(buf));
+       if (copy_from_user(buf, ubuf, csize))
+               return -EFAULT;
+       buf[U64STR_SIZE-1] = '\0';
+       err = strict_strtoull(buf, 10, &val);
+       if (0 != err)
+               return -EINVAL;
+       mutex_lock(&data.lock);
+       if (data.sample_width < val)
+               data.sample_window = val;
+       else {
+               mutex_unlock(&data.lock);
+               return -EINVAL;
+       }
+       mutex_unlock(&data.lock);
+       return csize;
+}
+
+static const struct file_operations available_fops = {
+       .open   = debug_available_fopen,
+       .read   = debug_available_fread,
+       .write  = debug_available_fwrite,
+       .owner  = THIS_MODULE,
+};
+
+static const struct file_operations current_fops = {
+       .open   = debug_current_fopen,
+       .read   = debug_current_fread,
+       .write  = debug_current_fwrite,
+       .owner  = THIS_MODULE,
+};
+
+static const struct file_operations count_fops = {
+       .open   = debug_count_fopen,
+       .read   = debug_count_fread,
+       .write  = debug_count_fwrite,
+       .owner  = THIS_MODULE,
+};
+
+static const struct file_operations enable_fops = {
+       .open   = debug_enable_fopen,
+       .read   = debug_enable_fread,
+       .write  = debug_enable_fwrite,
+       .owner  = THIS_MODULE,
+};
+
+static const struct file_operations max_fops = {
+       .open   = debug_max_fopen,
+       .read   = debug_max_fread,
+       .write  = debug_max_fwrite,
+       .owner  = THIS_MODULE,
+};
+
+static const struct file_operations sample_fops = {
+       .open   = debug_sample_fopen,
+       .read   = debug_sample_fread,
+       .release = debug_sample_release,
+       .owner  = THIS_MODULE,
+};
+
+static const struct file_operations threshold_fops = {
+       .open   = debug_threshold_fopen,
+       .read   = debug_threshold_fread,
+       .write  = debug_threshold_fwrite,
+       .owner  = THIS_MODULE,
+};
+
+static const struct file_operations width_fops = {
+       .open   = debug_width_fopen,
+       .read   = debug_width_fread,
+       .write  = debug_width_fwrite,
+       .owner  = THIS_MODULE,
+};
+
+static const struct file_operations window_fops = {
+       .open   = debug_window_fopen,
+       .read   = debug_window_fread,
+       .write  = debug_window_fwrite,
+       .owner  = THIS_MODULE,
+};
+static int init_debugfs(void)
+{
+       int ret = -ENOMEM;
+
+       debug_dir = debugfs_create_dir(DRVNAME, NULL);
+       if (!debug_dir)
+               goto err_debug_dir;
+
+       debug_available = debugfs_create_file("available", 0444,
+                                               debug_dir, NULL,
+                                               &available_fops);
+       if (!debug_available)
+               goto err_available;
+
+       debug_current = debugfs_create_file("current", 0444,
+                                               debug_dir, NULL,
+                                               &current_fops);
+       if (!debug_current)
+               goto err_current;
+
+       debug_sample = debugfs_create_file("sample", 0444,
+                                               debug_dir, NULL,
+                                               &sample_fops);
+       if (!debug_sample)
+               goto err_sample;
+
+       debug_count = debugfs_create_file("count", 0444,
+                                               debug_dir, NULL,
+                                               &count_fops);
+       if (!debug_count)
+               goto err_count;
+
+       debug_max = debugfs_create_file("max", 0444,
+                                               debug_dir, NULL,
+                                               &max_fops);
+       if (!debug_max)
+               goto err_max;
+
+       debug_sample_window = debugfs_create_file("window", 0644,
+                                               debug_dir, NULL,
+                                               &window_fops);
+       if (!debug_sample_window)
+               goto err_window;
+
+       debug_sample_width = debugfs_create_file("width", 0644,
+                                               debug_dir, NULL,
+                                               &width_fops);
+       if (!debug_sample_width)
+               goto err_width;
+
+       debug_threshold = debugfs_create_file("threshold", 0644,
+                                               debug_dir, NULL,
+                                               &threshold_fops);
+       if (!debug_threshold)
+               goto err_threshold;
+
+       debug_enable = debugfs_create_file("enable", 0644,
+                                               debug_dir, &enabled,
+                                               &enable_fops);
+       if (!debug_enable)
+               goto err_enable;
+       else {
+               ret = 0;
+               goto out;
+       }
+
+err_enable:
+       debugfs_remove(debug_threshold);
+err_threshold:
+       debugfs_remove(debug_sample_width);
+err_width:
+       debugfs_remove(debug_sample_window);
+err_window:
+       debugfs_remove(debug_max);
+err_max:
+       debugfs_remove(debug_count);
+err_count:
+       debugfs_remove(debug_sample);
+err_sample:
+       debugfs_remove(debug_current);
+err_current:
+       debugfs_remove(debug_available);
+err_available:
+       debugfs_remove(debug_dir);
+err_debug_dir:
+out:
+       return ret;
+}
+
+static void free_debugfs(void)
+{
+       debugfs_remove(debug_enable);
+       debugfs_remove(debug_threshold);
+       debugfs_remove(debug_sample_width);
+       debugfs_remove(debug_sample_window);
+       debugfs_remove(debug_max);
+       debugfs_remove(debug_count);
+       debugfs_remove(debug_sample);
+       debugfs_remove(debug_current);
+       debugfs_remove(debug_available);
+       debugfs_remove(debug_dir);
+}
+
+static int hw_test_init(void)
+{
+       int ret = -ENOMEM;
+
+       printk(KERN_INFO BANNER "version %s\n", VERSION);
+
+       sample_function_register(&tsc_sample);
+       sample_function_register(&tsc_freq_sample);
+       sample_function_register(&random_bytes_sample);
+
+       ret = init_stats();
+       if (0 != ret)
+               goto out;
+       ret = init_debugfs();
+       if (0 != ret)
+               goto err_stats;
+       if (enabled)
+               ret = start_kthread();
+       goto out;
+
+err_stats:
+       ring_buffer_free(ring_buffer);
+out:
+       return ret;
+}
+
+static void hw_test_exit(void)
+{
+       int err;
+
+       if (enabled) {
+               enabled = 0;
+               err = stop_kthread();
+               if (err)
+                       printk(KERN_ERR BANNER "cannot stop kthread\n");
+       }
+
+       free_debugfs();
+       ring_buffer_free(ring_buffer);
+}
+
+module_init(hw_test_init);
+module_exit(hw_test_exit);

Hello everyone,

I'm trying to push a new tool upstream. I'd like to hear back from you what the best practice is to get the job done.

Thanks,
Luming

---------- Forwarded message ----------
From: Luming Yu <[email protected]>
Date: Mon, Jun 11, 2012 at 9:59 PM
Subject: Fwd: [patch] a simple hardware detector for latency as well as throughput ver. 0.1.0
To: [email protected]
Cc: Andrew Morton <[email protected]>, [email protected], [email protected]

Hi,

I'd like to know if the patch looks good for linux-next to find its way upstream in 3.6.

Thanks and regards,
Luming

---------- Forwarded message ----------
From: Luming Yu <[email protected]>
Date: Wed, May 30, 2012 at 7:47 AM
Subject: Fwd: [patch] a simple hardware detector for latency as well as throughput ver. 0.1.0
To: Andrew Morton <[email protected]>
Cc: [email protected]

Hello akpm,

I'd like to push the patch to upstream, but I'm not sure if jcm has extra bandwidth although he is also interested in having the tool upstream..So I'd like ping you to check if there is any chance to queue it up in your tree first.I will enhance it further after it's upstream.

Thanks,
Luming

---------- Forwarded message ----------
From: Luming Yu <[email protected]>
Date: Tue, May 29, 2012 at 4:37 AM
Subject: [patch] a simple hardware detector for latency as well as throughput ver. 0.1.0
To: [email protected]
Cc: [email protected]

Hi Jon,

The patch is the fist step to test some basic hardware functions like TSC to help people understand if there is any hardware latency as well as throughput problem exposed on bare metal or left behind by BIOS or interfered by SM. Currently the patch tests hardware features (tsc,freq, and rdrandom whiich is new instruction to get random number) in stop_machine context. I will add more after the first step get merged for those guys who want to directly play with new hardware functions.

I suppose I can add your signed-off-by as the code is derived from your hwlat_dector.

I'm also reuqesting if you are going to queue it up somewhere that can be pulled into 3.5.

Of cause, I will update the patch based upon any comments that you think must be fixed for 3.5 merge.

Thanks,
Luming

Signed-off-by Luming Yu <[email protected]>

 Kconfig   |    7
 Makefile  |    2
 hw_test.c |  954
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

 3 files changed, 963 insertions(+)

Hi Luming,

Please write the text in the email so that it can be used as a permanent changelog entry for the kernel git history. Everything that you do not want to have in that history can go under the "---" line that separates the Signed-off-by from the diffstat.

...

Done in patch update.

Done in update

It's a placeholder for future use although it's not used right now in current tests.

I've reduced the code redundancy in update.

Done

It is for testing new instruction rdrand for new cpus like Intel Ivy Bridge.

The purpose is to sequentially test all cpu's frequency.

Yes, the only purpose of the tool is to measure various hardware operation's latency
as well as bandwidth. We need the test done with least invasion from others.

It's out the testing loops. I didn't notice it hurts testing results so far. I will change it
in 0.1-> 0.2

There is a data.lock encapsulated in the helper. I can't remove them otherwise the data protection will get lost.

I've simplified syfs interface code in other ways. Basically use define magic.

I've used define magic in patch update.

All empty functions are gone.

Done in update patch.

Thanks for detailed review. An update patch is being prepared right now. Will send in few minutes.

/l