per-CPU vmstat thresholds and vmstat worker disablement

The per-CPU vmstat worker is a problem on -RT workloads (because ideally the CPU is entirely reserved for the -RT app, without interference). The worker transfers accumulated per-CPU vmstat counters to global counters.

To resolve the problem, create two tunables:

* Userspace configurable per-CPU vmstat threshold: by default the VM code calculates the size of the per-CPU vmstat arrays. This tunable allows userspace to configure the values.

* Userspace configurable per-CPU vmstat worker: allow disabling the per-CPU vmstat worker.

The patch below contains documentation which describes the tunables in more detail.

Quoted 14 lines · Marcelo Tosatti

The per-CPU vmstat worker is a problem on -RT workloads (because ideally the CPU is entirely reserved for the -RT app, without interference). The worker transfers accumulated per-CPU vmstat counters to global counters.

To resolve the problem, create two tunables:

* Userspace configurable per-CPU vmstat threshold: by default the VM code calculates the size of the per-CPU vmstat arrays. This tunable allows userspace to configure the values.

* Userspace configurable per-CPU vmstat worker: allow disabling the per-CPU vmstat worker.

Quoted 2 lines

The patch below contains documentation which describes the tunables in more detail.

The documentation says what the tunables do, but not how you should set them in different scenarios, or why.

That could be a little more helpful to sysadmins.

The per-CPU vmstat worker is a problem on -RT workloads (because ideally the CPU is entirely reserved for the -RT app, without interference). The worker transfers accumulated per-CPU vmstat counters to global counters.

To resolve the problem, create two tunables:

* Userspace configurable per-CPU vmstat threshold: by default the VM code calculates the size of the per-CPU vmstat arrays. This tunable allows userspace to configure the values.

* Userspace configurable per-CPU vmstat worker: allow disabling the per-CPU vmstat worker.

The patch below contains documentation which describes the tunables in more detail.

Signed-off-by Marcelo Tosatti <[email protected]>

---
 Documentation/vm/vmstat_thresholds.txt |   38 +++++
 mm/vmstat.c                            |  248 +++++++++++++++++++++++++++++++--
 2 files changed, 272 insertions(+), 14 deletions(-)

Index: linux-2.6-git-disable-vmstat-worker/mm/vmstat.c
===================================================================

mm/vmstat.c +234 -14

@@ -91,8 +91,17 @@
 EXPORT_SYMBOL(vm_zone_stat);
 EXPORT_SYMBOL(vm_node_stat);
 
+struct vmstat_uparam {
+	atomic_t vmstat_work_enabled;
+	atomic_t user_stat_thresh;
+};
+
+static DEFINE_PER_CPU(struct vmstat_uparam, vmstat_uparam);
+
 #ifdef CONFIG_SMP
 
+#define MAX_THRESHOLD 125
+
 int calculate_pressure_threshold(struct zone *zone)
 {
 	int threshold;

@@ -110,9 +119,9 @@
 	threshold = max(1, (int)(watermark_distance / num_online_cpus()));
 
 	/*
-	 * Maximum threshold is 125
+	 * Maximum threshold is MAX_THRESHOLD == 125
 	 */
-	threshold = min(125, threshold);
+	threshold = min(MAX_THRESHOLD, threshold);
 
 	return threshold;
 }

@@ -188,15 +197,31 @@
 		threshold = calculate_normal_threshold(zone);
 
 		for_each_online_cpu(cpu) {
-			int pgdat_threshold;
+			int pgdat_threshold, ustat_thresh;
+			struct vmstat_uparam *vup;
 
-			per_cpu_ptr(zone->pageset, cpu)->stat_threshold
-							= threshold;
+			struct per_cpu_nodestat __percpu *pcp;
+			struct per_cpu_pageset *p;
+
+			p = per_cpu_ptr(zone->pageset, cpu);
+
+			vup = &per_cpu(vmstat_uparam, cpu);
+			ustat_thresh = atomic_read(&vup->user_stat_thresh);
+
+			if (ustat_thresh)
+				p->stat_threshold = ustat_thresh;
+			else
+				p->stat_threshold = threshold;
+
+			pcp = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu);
 
 			/* Base nodestat threshold on the largest populated zone. */
-			pgdat_threshold = per_cpu_ptr(pgdat->per_cpu_nodestats, cpu)->stat_threshold;
-			per_cpu_ptr(pgdat->per_cpu_nodestats, cpu)->stat_threshold
-				= max(threshold, pgdat_threshold);
+			pgdat_threshold = pcp->stat_threshold;
+			if (ustat_thresh)
+				pcp->stat_threshold = ustat_thresh;
+			else
+				pcp->stat_threshold = max(threshold,
+							  pgdat_threshold);
 		}
 
 		/*

@@ -226,9 +251,24 @@
 			continue;
 
 		threshold = (*calculate_pressure)(zone);
-		for_each_online_cpu(cpu)
+		for_each_online_cpu(cpu) {
+			int t, ustat_thresh;
+			struct vmstat_uparam *vup;
+
+			vup = &per_cpu(vmstat_uparam, cpu);
+			ustat_thresh = atomic_read(&vup->user_stat_thresh);
+			t = threshold;
+
+			/*
+			 * min because pressure could cause
+			 * calculate_pressure'ed value to be smaller.
+			 */
+			if (ustat_thresh)
+				t = min(threshold, ustat_thresh);
+
 			per_cpu_ptr(zone->pageset, cpu)->stat_threshold
-							= threshold;
+							= t;
+		}
 	}
 }
 

@@ -1567,6 +1607,9 @@
 	long val;
 	int err;
 	int i;
+	int cpu;
+	struct work_struct __percpu *works;
+	static struct cpumask has_work;
 
 	/*
 	 * The regular update, every sysctl_stat_interval, may come later

@@ -1580,9 +1623,31 @@
 	 * transiently negative values, report an error here if any of
 	 * the stats is negative, so we know to go looking for imbalance.
 	 */
-	err = schedule_on_each_cpu(refresh_vm_stats);
-	if (err)
-		return err;
+
+	works = alloc_percpu(struct work_struct);
+	if (!works)
+		return -ENOMEM;
+
+	cpumask_clear(&has_work);
+	get_online_cpus();
+
+	for_each_online_cpu(cpu) {
+		struct work_struct *work = per_cpu_ptr(works, cpu);
+		struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
+
+		if (atomic_read(&vup->vmstat_work_enabled)) {
+			INIT_WORK(work, refresh_vm_stats);
+			schedule_work_on(cpu, work);
+			cpumask_set_cpu(cpu, &has_work);
+		}
+	}
+
+	for_each_cpu(cpu, &has_work)
+		flush_work(per_cpu_ptr(works, cpu));
+
+	put_online_cpus();
+	free_percpu(works);
+
 	for (i = 0; i < NR_VM_ZONE_STAT_ITEMS; i++) {
 		val = atomic_long_read(&vm_zone_stat[i]);
 		if (val < 0) {

@@ -1674,6 +1739,10 @@
 	/* Check processors whose vmstat worker threads have been disabled */
 	for_each_online_cpu(cpu) {
 		struct delayed_work *dw = &per_cpu(vmstat_work, cpu);
+		struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
+
+		if (atomic_read(&vup->vmstat_work_enabled) == 0)
+			continue;
 
 		if (!delayed_work_pending(dw) && need_update(cpu))
 			queue_delayed_work_on(cpu, mm_percpu_wq, dw, 0);

@@ -1696,6 +1765,135 @@
 		round_jiffies_relative(sysctl_stat_interval));
 }
 
+#ifdef CONFIG_SYSFS
+
+static ssize_t vmstat_worker_show(struct device *dev,
+				  struct device_attribute *attr, char *buf)
+{
+	unsigned int cpu = dev->id;
+	struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
+
+	return sprintf(buf, "%d\n", atomic_read(&vup->vmstat_work_enabled));
+}
+
+static ssize_t vmstat_worker_store(struct device *dev,
+				   struct device_attribute *attr,
+				   const char *buf, size_t count)
+{
+	int ret, val;
+	struct vmstat_uparam *vup;
+	unsigned int cpu = dev->id;
+
+	ret = sscanf(buf, "%d", &val);
+	if (ret != 1 || val > 1 || val < 0)
+		return -EINVAL;
+
+	preempt_disable();
+
+	if (cpu_online(cpu)) {
+		vup = &per_cpu(vmstat_uparam, cpu);
+		atomic_set(&vup->vmstat_work_enabled, val);
+	} else
+		count = -EINVAL;
+
+	preempt_enable();
+
+	return count;
+}
+
+static ssize_t vmstat_thresh_show(struct device *dev,
+				  struct device_attribute *attr, char *buf)
+{
+	int ret;
+	struct vmstat_uparam *vup;
+	unsigned int cpu = dev->id;
+
+	preempt_disable();
+
+	vup = &per_cpu(vmstat_uparam, cpu);
+	ret = sprintf(buf, "%d\n", atomic_read(&vup->user_stat_thresh));
+
+	preempt_enable();
+
+	return ret;
+}
+
+static ssize_t vmstat_thresh_store(struct device *dev,
+				   struct device_attribute *attr,
+				   const char *buf, size_t count)
+{
+	int ret, val;
+	unsigned int cpu = dev->id;
+	struct vmstat_uparam *vup;
+
+	ret = sscanf(buf, "%d", &val);
+	if (ret != 1 || val < 1 || val > MAX_THRESHOLD)
+		return -EINVAL;
+
+	preempt_disable();
+
+	if (cpu_online(cpu)) {
+		vup = &per_cpu(vmstat_uparam, cpu);
+		atomic_set(&vup->user_stat_thresh, val);
+	} else
+		count = -EINVAL;
+
+	preempt_enable();
+
+	return count;
+}
+
+struct device_attribute vmstat_worker_attr =
+	__ATTR(vmstat_worker, 0644, vmstat_worker_show, vmstat_worker_store);
+
+struct device_attribute vmstat_threshold_attr =
+	__ATTR(vmstat_threshold, 0644, vmstat_thresh_show, vmstat_thresh_store);
+
+static struct attribute *vmstat_attrs[] = {
+	&vmstat_worker_attr.attr,
+	&vmstat_threshold_attr.attr,
+	NULL
+};
+
+static struct attribute_group vmstat_attr_group = {
+	.attrs  =  vmstat_attrs,
+	.name   = "vmstat"
+};
+
+static int vmstat_thresh_cpu_online(unsigned int cpu)
+{
+	struct device *dev = get_cpu_device(cpu);
+	int ret;
+
+	ret = sysfs_create_group(&dev->kobj, &vmstat_attr_group);
+	if (ret)
+		return ret;
+
+	return 0;
+}
+
+static int vmstat_thresh_cpu_down_prep(unsigned int cpu)
+{
+	struct device *dev = get_cpu_device(cpu);
+
+	sysfs_remove_group(&dev->kobj, &vmstat_attr_group);
+	return 0;
+}
+
+static void init_vmstat_sysfs(void)
+{
+	int cpu;
+
+	for_each_possible_cpu(cpu) {
+		struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
+
+		atomic_set(&vup->user_stat_thresh, 0);
+		atomic_set(&vup->vmstat_work_enabled, 1);
+	}
+}
+
+#endif /* CONFIG_SYSFS */
+
 static void __init init_cpu_node_state(void)
 {
 	int node;

@@ -1723,9 +1921,13 @@
 {
 	const struct cpumask *node_cpus;
 	int node;
+	struct vmstat_uparam *vup = &per_cpu(vmstat_uparam, cpu);
 
 	node = cpu_to_node(cpu);
 
+	atomic_set(&vup->user_stat_thresh, 0);
+	atomic_set(&vup->vmstat_work_enabled, 1);
+
 	refresh_zone_stat_thresholds();
 	node_cpus = cpumask_of_node(node);
 	if (cpumask_weight(node_cpus) > 0)

@@ -1735,7 +1937,7 @@
 	return 0;
 }
 
-#endif
+#endif /* CONFIG_SMP */
 
 struct workqueue_struct *mm_percpu_wq;
 

@@ -1772,6 +1974,24 @@
 #endif
 }
 
+static int __init init_mm_internals_late(void)
+{
+#ifdef CONFIG_SYSFS
+	int ret;
+
+	init_vmstat_sysfs();
+
+	ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "mm/vmstat_thresh:online",
+					vmstat_thresh_cpu_online,
+					vmstat_thresh_cpu_down_prep);
+	if (ret < 0)
+		pr_err("vmstat_thresh: failed to register 'online' hotplug state\n");
+#endif
+	return 0;
+}
+
+late_initcall(init_mm_internals_late);
+
 #if defined(CONFIG_DEBUG_FS) && defined(CONFIG_COMPACTION)
 
 /*

Index: linux-2.6-git-disable-vmstat-worker/Documentation/vm/vmstat_thresholds.txt
===================================================================

Documentation/vm/vmstat_thresholds.txt new file +28 -0

@@ -0,0 +1,38 @@
+Userspace configurable vmstat thresholds
+========================================
+
+This document describes the tunables to control
+per-CPU vmstat threshold and per-CPU vmstat worker
+thread.
+
+/sys/devices/system/cpu/cpuN/vmstat/vmstat_threshold:
+
+This file contains the per-CPU vmstat threshold.
+This value is the maximum that a single per-CPU vmstat statistic
+can accumulate before transferring to the global counters.
+
+A value of 0 indicates that the value is set
+by the in kernel algorithm.
+
+A value different than 0 indicates that particular
+value is used for vmstat_threshold.
+
+/sys/devices/system/cpu/cpuN/vmstat/vmstat_worker:
+
+Enable/disable the per-CPU vmstat worker.
+
+Usage example:
+=============
+
+To disable vmstat_update worker for cpu1:
+

Quoted 4 lines · On Tue, 25 Apr 2017 10:57:19 -0300 Marcelo Tosatti <[email protected]> wrote:

The per-CPU vmstat worker is a problem on -RT workloads (because ideally the CPU is entirely reserved for the -RT app, without interference). The worker transfers accumulated per-CPU vmstat counters to global counters.

This is a problem for non-RT too. Any task pinned to an isolated CPU that doesn't want to be ever interrupted will be interrupted by the vmstat kworker.

Quoted 6 lines

To resolve the problem, create two tunables:

* Userspace configurable per-CPU vmstat threshold: by default the VM code calculates the size of the per-CPU vmstat arrays. This tunable allows userspace to configure the values.

Quoted 2 lines

* Userspace configurable per-CPU vmstat worker: allow disabling the per-CPU vmstat worker.

I have several questions about the tunables:

- What does the vmstat_threshold value mean? What are the implications of changing this value? What's the difference in choosing 1, 2, 3 or 500?

- If the purpose of having vmstat_threshold is to allow disabling the vmstat kworker, why can't the kernel pick a value automatically?

- What are the implications of disabling the vmstat kworker? Will vm stats still be collected someway or will it be completely off for the CPU?

Also, shouldn't this patch be split into two?

Quoted 5 lines

I have several questions about the tunables:

- What does the vmstat_threshold value mean? What are the implications of changing this value? What's the difference in choosing 1, 2, 3 or 500?

Its the maximum value for a vmstat statistics counter to hold. After that value, the statistics are transferred to the global counter:

void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item,
long delta)

{
        struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats;
        s8 __percpu *p = pcp->vm_node_stat_diff + item;
        long x;
        long t;

        x = delta + __this_cpu_read(*p);

        t = __this_cpu_read(pcp->stat_threshold);

        if (unlikely(x > t || x < -t)) {
                node_page_state_add(x, pgdat, item);
                x = 0;
        }
        __this_cpu_write(*p, x);
}

EXPORT_SYMBOL(__mod_node_page_state);

BTW, there is a bug there, should change that to:

        if (unlikely(x >= t || x <= -t)) {

Increasing the threshold value does two things:
1) It decreases the number of inter-processor accesses.
2) It increases how much the global counters stay out of sync relative to actual current values.

Quoted 2 lines

- If the purpose of having vmstat_threshold is to allow disabling the vmstat kworker, why can't the kernel pick a value automatically?

Because it might be acceptable for the user to accept a small out of syncedness of the global counters in favour of performance (one would have to analyze the situation).

Setting vmstat_threshold == 1 means the global counter is always in sync with the page counter state of the pCPU.

Quoted 4 lines

- What are the implications of disabling the vmstat kworker? Will vm stats still be collected someway or will it be completely off for the CPU?

It will not be necessary to collect vmstats because at every modification of the vm statistics, pCPUs with vmstat_threshold=1 transfer their values to the global counters (that is, there is no queueing of statistics locally to improve performance).

Quoted 1 line

Also, shouldn't this patch be split into two?

First add one sysfs file, then add another sysfs file, you mean?

Quoted 33 lines · On Tue, 2 May 2017 13:52:00 -0300 Marcelo Tosatti <[email protected]> wrote:

Quoted 5 lines

I have several questions about the tunables:

- What does the vmstat_threshold value mean? What are the implications of changing this value? What's the difference in choosing 1, 2, 3 or 500?

Its the maximum value for a vmstat statistics counter to hold. After that value, the statistics are transferred to the global counter:

void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item,
long delta)

{
        struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats;
        s8 __percpu *p = pcp->vm_node_stat_diff + item;
        long x;
        long t;

        x = delta + __this_cpu_read(*p);

        t = __this_cpu_read(pcp->stat_threshold);

        if (unlikely(x > t || x < -t)) {
                node_page_state_add(x, pgdat, item);
                x = 0;
        }
        __this_cpu_write(*p, x);
}

EXPORT_SYMBOL(__mod_node_page_state);

BTW, there is a bug there, should change that to:

        if (unlikely(x >= t || x <= -t)) {

Quoted 4 lines

Increasing the threshold value does two things:
1) It decreases the number of inter-processor accesses.
2) It increases how much the global counters stay out of sync relative to actual current values.

OK, but I'm mostly concerned with the sysadmin who will have to change the tunable. So, I think it's a good idea to improve the doc to contain that information.

Quoted 7 lines

Quoted 2 lines

- If the purpose of having vmstat_threshold is to allow disabling the vmstat kworker, why can't the kernel pick a value automatically?

Because it might be acceptable for the user to accept a small out of syncedness of the global counters in favour of performance (one would have to analyze the situation).

Quoted 2 lines

Setting vmstat_threshold == 1 means the global counter is always in sync with the page counter state of the pCPU.

IMHO, if vmstat_threshold == 1 is the required setting for disabling the vmstat kworker then I'd go with only one tunable for now. But that's just a suggestion.

Quoted 4 lines

Quoted 3 lines

- What are the implications of disabling the vmstat kworker? Will vm stats still be collected someway or will it be completely off for the CPU?

Quoted 4 lines

It will not be necessary to collect vmstats because at every modification of the vm statistics, pCPUs with vmstat_threshold=1 transfer their values to the global counters (that is, there is no queueing of statistics locally to improve performance).

Ah, OK. Got this now. I'll give this patch a try. But I think we want to hear from Christoph (who worked on reducing the vmstat interruptions in the past).

Quoted 3 lines

Quoted 1 line

Also, shouldn't this patch be split into two?

First add one sysfs file, then add another sysfs file, you mean?

Yes, one tunable per patch.

Ah, OK. Got this now. I'll give this patch a try. But I think we want to hear from Christoph (who worked on reducing the vmstat interruptions in the past).

Quoted 3 lines · Luiz Capitulino

Ah, OK. Got this now. I'll give this patch a try. But I think we want to hear from Christoph (who worked on reducing the vmstat interruptions in the past).

A bit confused by this one. The vmstat worker is already disabled if there are no updates. Also the patches by Chris Metcalf on data plane mode add a prctl to quiet the vmstat workers.

Why do we need more than this?

Quoted 5 lines

A bit confused by this one. The vmstat worker is already disabled if there are no updates. Also the patches by Chris Metcalf on data plane mode add a prctl to quiet the vmstat workers.

Why do we need more than this?

If there are vmstat statistic updates on a given CPU, and you don't want intervention from the vmstat worker, you change the behaviour of stat data collection to directly write to the global structures (which disables the performance optimization of collecting data in per-cpu counters).

This way you can disable vmstat worker (because it causes undesired latencies), while allowing vmstatistics to function properly.

The prctl from Chris Metcalf patchset allows one to disable vmstat worker per CPU? If so, they replace the functionality of the patch "[patch 3/3] MM: allow per-cpu vmstat_worker configuration" of the -v2 series of my patchset, and we can use it instead.

Is it integrated already?

Quoted 6 lines · Marcelo Tosatti

Quoted 5 lines

A bit confused by this one. The vmstat worker is already disabled if there are no updates. Also the patches by Chris Metcalf on data plane mode add a prctl to quiet the vmstat workers.

Why do we need more than this?

Quoted 5 lines

If there are vmstat statistic updates on a given CPU, and you don't want intervention from the vmstat worker, you change the behaviour of stat data collection to directly write to the global structures (which disables the performance optimization of collecting data in per-cpu counters).

Hmmm.... Ok. That is going to be expensive if you do this for each individual vmstat update.

Quoted 2 lines

This way you can disable vmstat worker (because it causes undesired latencies), while allowing vmstatistics to function properly.

Best then to run the vmstat update mechanism when you leave kernel mode to get all the updates in one go.

Quoted 6 lines

The prctl from Chris Metcalf patchset allows one to disable vmstat worker per CPU? If so, they replace the functionality of the patch "[patch 3/3] MM: allow per-cpu vmstat_worker configuration" of the -v2 series of my patchset, and we can use it instead.

Is it integrated already?

The data plane mode patches disables vmstat processing by updating the vmstats immediately if necessary and switching off the kworker thread.

So the kworker wont be running until the next time statistics are checked by the shepherd task from a remote cpu.

If the counters have been updated
then the shepherd task will reenable the kworker. This is already merged and has been working for a long time. Data plan mode has not been merged yet but the infrastructure in vmstat.c is there because NOHZ needs it too.

See linux/vmstat.c:quiet_vmstat()

It would be easy to add a /proc file that allows the quieting of the vmstat workers for a certain cpu. Just make it call the quiet_vmstat() on the right cpu.

This will quiet vmstat down. The shepherd task will check the stats in 2 second intervals and will then reenable when necessasry.

Note that we already are updating the global structures directly if the differential gets too high. Reducing the differential may get you what you want.

OK, i'll check if the patches from Chris work for us and then add Tested-by on that.

In our case, vmstat updates are very rare (CPU is dominated by DPDK).

Quoted 1 line · Marcelo Tosatti

In our case, vmstat updates are very rare (CPU is dominated by DPDK).

What is the OS doing on the cores that DPDK runs on? I mean we here can clean a processor of all activities and are able to run for a long time without any interruptions.

Why would you still let the OS do things on that processor? If activities by the OS are required then the existing NOHZ setup already minimizes latency to a short burst (and Chris Metcalf's work improves on that).

What exactly is the issue you are seeing and want to address? I think we have similar aims and as far as I know the current situation is already good enough for what you may need. You may just not be aware of how to configure this.

I doubt that doing inline updates will do much good compared to what we already have and what the dataplan mode can do.

Quoted 4 lines

What exactly is the issue you are seeing and want to address? I think we have similar aims and as far as I know the current situation is already good enough for what you may need. You may just not be aware of how to configure this.

I want to disable vmstat worker thread completly from an isolated CPU. Because it adds overhead to a latency target, target which the lower the better.

Quoted 2 lines

I doubt that doing inline updates will do much good compared to what we already have and what the dataplan mode can do.

Can the dataplan mode disable vmstat worker thread completly on a given CPU?

Quoted 5 lines · Marcelo Tosatti

Quoted 4 lines

What exactly is the issue you are seeing and want to address? I think we have similar aims and as far as I know the current situation is already good enough for what you may need. You may just not be aware of how to configure this.

Quoted 3 lines

I want to disable vmstat worker thread completly from an isolated CPU. Because it adds overhead to a latency target, target which the lower the better.

NOHZ already does that. I wanted to know what your problem is that you see. The latency issue has already been solved as far as I can tell . Please tell me why the existing solutions are not sufficient for you.

Quoted 5 lines

Quoted 2 lines

I doubt that doing inline updates will do much good compared to what we already have and what the dataplan mode can do.

Can the dataplan mode disable vmstat worker thread completly on a given CPU?

That already occurs when you call quiet_vmstat() and is used by the NOHZ logic. Configure that correctly and you should be fine.

Quoted 3 lines

NOHZ already does that. I wanted to know what your problem is that you see. The latency issue has already been solved as far as I can tell . Please tell me why the existing solutions are not sufficient for you.

We don't want vmstat_worker to execute on a given CPU, even if the local CPU updates vm-statistics.

Because:

vmstat_worker increases latency of the application
(i can measure it if you want on a given CPU,
how many ns's the following takes:

Why the existing solutions are not sufficient:

1) task-isolation patchset seems too heavy for our usecase (we do want IPIs, signals, etc).

So this seems a little heavy for our usecase.

Quoted 6 lines · Marcelo Tosatti

Quoted 3 lines

NOHZ already does that. I wanted to know what your problem is that you see. The latency issue has already been solved as far as I can tell . Please tell me why the existing solutions are not sufficient for you.

We don't want vmstat_worker to execute on a given CPU, even if the local CPU updates vm-statistics.

Instead of responding you repeat describing what you want.

Quoted 5 lines

Because:

vmstat_worker increases latency of the application
(i can measure it if you want on a given CPU,
how many ns's the following takes:

That still is no use case.

Just a measurement of vmstat_worker. Pointless.

If you move the latency from the vmstat worker into the code thats updating the counters then you will require increased use of atomics which will increase contention which in turn will significantly increase the overall latency.

Use-case: realtime application on an isolated core which for some reason updates vmstatistics.

Quoted 1 line

Just a measurement of vmstat_worker. Pointless.

Shouldnt the focus be on general scenarios rather than particular usecases, so that the solution covers a wider range of usecases?

The situation as i see is as follows:

Your point of view is: an "isolated CPU" with a set of applications cannot update vm statistics, otherwise they pay the vmstat_update cost:

     kworker/5:1-245   [005] ....1..   673.454295: workqueue_execute_start: work struct ffffa0cf6e493e20: function vmstat_update
     kworker/5:1-245   [005] ....1..   673.454305: workqueue_execute_end: work struct ffffa0cf6e493e20

Thats 10us for example.

So if want to customize a realtime setup whose code updates vmstatistic, you are dead. You have to avoid any systemcall which possibly updates vmstatistics (now and in the future kernel versions).

The point is that these vmstat updates are rare. From
http://www.7-cpu.com/cpu/Haswell.html:

RAM Latency = 36 cycles + 57 ns (3.4 GHz i7-4770) RAM Latency = 62 cycles + 100 ns (3.6 GHz E5-2699 dual)

Lets round to 100ns = 0.1us.

You need 100 vmstat updates (all misses to RAM, the worst possible case) to have equivalent amount of time of the batching version.

But thats not the point. The point is the 10us interruption to execution of the realtime app (which can either mean your current deadline requirements are not met, or that another application with lowest latency requirement can't be used).

So why are you against integrating this simple, isolated patch which does not affect how current logic works?

Quoted 2 lines

So why are you against integrating this simple, isolated patch which does not affect how current logic works?

Frankly the argument does not make sense. Vmstat updates occur very infrequently (probably even less than you IPIs and the other OS stuff that also causes additional latencies that you seem to be willing to tolerate).

And you can configure the interval of vmstat updates freely.... Set the vmstat_interval to 60 seconds instead of 2 for a try? Is that rare enough?

Something that crossed my mind was to add a new tunable to set the vmstat_interval for each CPU, this way we could essentially disable it to the CPUs where DPDK is running. What's the implications of doing this besides not getting up to date stats in /proc/vmstat (which I still have to confirm would be OK)? Can this break anything in the kernel for example?

Quoted 6 lines · On Fri, 19 May 2017 12:13:26 -0500 (CDT) Christoph Lameter <[email protected]> wrote:

Quoted 2 lines

So why are you against integrating this simple, isolated patch which does not affect how current logic works?

Frankly the argument does not make sense. Vmstat updates occur very infrequently (probably even less than you IPIs and the other OS stuff that also causes additional latencies that you seem to be willing to tolerate).

Infrequently is not good enough. It only has to happen once to cause a problem.

Also, IPIs take a few us, usually less. That's not a problem. In our testing we see the preemption caused by the kworker take 10us or even more. I've never seeing it take 3us. I'm not saying this is not true, I'm saying if this is causing a problem to us it will cause a problem to other people too.

No, we'd have to set it high enough to disable it and this will affect all CPUs.

Something that crossed my mind was to add a new tunable to set the vmstat_interval for each CPU, this way we could essentially disable it to the CPUs where DPDK is running. What's the implications of doing this besides not getting up to date stats in /proc/vmstat (which I still have to confirm would be OK)? Can this break anything in the kernel for example?

Argument? We're showing you the data that this is causing a latency problem for us.

Quoted 6 lines

Something that crossed my mind was to add a new tunable to set the vmstat_interval for each CPU, this way we could essentially disable it to the CPUs where DPDK is running. What's the implications of doing this besides not getting up to date stats in /proc/vmstat (which I still have to confirm would be OK)? Can this break anything in the kernel for example?

Well, you get incorrect statistics.

Quoted 2 lines · Marcelo Tosatti

Argument? We're showing you the data that this is causing a latency problem for us.

Sorry I am not sure where the data shows a latency problem. There are interrupts and scheduler ticks. But what does this have to do with vmstat?

Show me your dpdk code running and trace the tick on / off events as well as the vmstat invocations. Also show all system calls occurring on the cpu that runs dpdk. That is necessary to see what triggers vmstat and how the system reacts to the changes to the differentials.

Then please rerun the test by setting the vmstat_interval to 60.

Do another run with your modifications and show the difference.

Quoted 7 lines

Quoted 6 lines

Something that crossed my mind was to add a new tunable to set the vmstat_interval for each CPU, this way we could essentially disable it to the CPUs where DPDK is running. What's the implications of doing this besides not getting up to date stats in /proc/vmstat (which I still have to confirm would be OK)? Can this break anything in the kernel for example?

Quoted 1 line

Well, you get incorrect statistics.

The statistics are never completely accurate. You will get less accurate statistics but they will be correct. The differentials may not be reflected in the counts shown via /proc but there is a cap on how inaccurate those can becore.

Quoted 6 lines

interrupts and scheduler ticks. But what does this have to do with vmstat?

Show me your dpdk code running and trace the tick on / off events as well as the vmstat invocations. Also show all system calls occurring on the cpu that runs dpdk. That is necessary to see what triggers vmstat and how the system reacts to the changes to the differentials.

Sure, i can get that to you. The question remains: Are you arguing its not valid for a realtime application to use any system call which changes a vmstat counter?

Because if they are allowed, then its obvious something like this is needed.

Quoted 6 lines · Marcelo Tosatti

Quoted 6 lines

interrupts and scheduler ticks. But what does this have to do with vmstat?

Show me your dpdk code running and trace the tick on / off events as well as the vmstat invocations. Also show all system calls occurring on the cpu that runs dpdk. That is necessary to see what triggers vmstat and how the system reacts to the changes to the differentials.

Quoted 4 lines

Sure, i can get that to you. The question remains: Are you arguing its not valid for a realtime application to use any system call which changes a vmstat counter?

A true realtime app would be conscientious of its use of the OS services because the use of the services may cause additional latencies and also cause timers etc to fire later. A realtime app that is willing to use these services is therefore willing to tolerate larger latencies. A realtime app that is using OS service may cause the timer tick to be enabled which also causes additional latencies.

I have seen completely OS noise free processing for extended time period when not using OS services and using RDMA for I/O. This fits my use case well.

If there are really these high latencies because of kworker processing for vmstat then maybe we need a different mechanism there (bh? or other triggers) and maybe we are using far too many counters so that the processing becomes a heavy user of resources.

Quoted 2 lines

Because if they are allowed, then its obvious something like this is needed.

I am still wondering what benefit there is. Lets get clear on the test load and see if this actually makes sense.

Quoted 2 lines · Marcelo Tosatti

Use-case: realtime application on an isolated core which for some reason updates vmstatistics.

Ok that is already only happening every 2 seconds by default and that interval is configurable via the vmstat_interval proc setting.

Quoted 4 lines

Quoted 1 line

Just a measurement of vmstat_worker. Pointless.

Shouldnt the focus be on general scenarios rather than particular usecases, so that the solution covers a wider range of usecases?

Yes indeed and as far as I can tell the wider usecases are covered. Not sure that there is anything required here.

Quoted 9 lines

The situation as i see is as follows:

Your point of view is: an "isolated CPU" with a set of applications cannot update vm statistics, otherwise they pay the vmstat_update cost:

     kworker/5:1-245   [005] ....1..   673.454295: workqueue_execute_start: work struct ffffa0cf6e493e20: function vmstat_update
     kworker/5:1-245   [005] ....1..   673.454305: workqueue_execute_end: work struct ffffa0cf6e493e20

Thats 10us for example.

Well with a decent cpu that is 3 usec and it occurs infrequently on the order of once per multiple seconds.

Quoted 3 lines

So if want to customize a realtime setup whose code updates vmstatistic, you are dead. You have to avoid any systemcall which possibly updates vmstatistics (now and in the future kernel versions).

You are already dead because you allow IPIs and other kernel processing which creates far more overhead. Still fail to see the point.

Quoted 7 lines

The point is that these vmstat updates are rare. From
http://www.7-cpu.com/cpu/Haswell.html:

RAM Latency = 36 cycles + 57 ns (3.4 GHz i7-4770) RAM Latency = 62 cycles + 100 ns (3.6 GHz E5-2699 dual)

Lets round to 100ns = 0.1us.

That depends on the kernel functionality used.

Quoted 2 lines

You need 100 vmstat updates (all misses to RAM, the worst possible case) to have equivalent amount of time of the batching version.

The batching version occurs every couple of seconds if at all.

Quoted 5 lines

But thats not the point. The point is the 10us interruption to execution of the realtime app (which can either mean your current deadline requirements are not met, or that another application with lowest latency requirement can't be used).

Ok then you need to get rid of the IPIs and the other stuff that you have going on with the OS first I think.

Quoted 2 lines

So why are you against integrating this simple, isolated patch which does not affect how current logic works?

Frankly the argument does not make sense. Vmstat updates occur very infrequently (probably even less than you IPIs and the other OS stuff that also causes additional latencies that you seem to be willing to tolerate).

And you can configure the interval of vmstat updates freely.... Set the vmstat_interval to 60 seconds instead of 2 for a try? Is that rare enough?

Quoted 3 lines

And you can configure the interval of vmstat updates freely.... Set the vmstat_interval to 60 seconds instead of 2 for a try? Is that rare enough?

Not rare enough. Never is rare enough.

Quoted 5 lines · Marcelo Tosatti

Quoted 3 lines

And you can configure the interval of vmstat updates freely.... Set the vmstat_interval to 60 seconds instead of 2 for a try? Is that rare enough?

Not rare enough. Never is rare enough.

Ok what about the other stuff that must be going on if you allow OS activity like f.e. the tick, scheduler etc etc.