Skip to contentSkip to conversation

show isolated & nohz_full cpus in sysfs

PATCH Superseded [email protected] 3 participants 9 messages Mar 27, 2015 – Mar 28, 2015 Driver core

lore
Share & tools
lore
#0 lore raw

cover show isolated & nohz_full cpus in sysfs

Currently there is no good way to get the isolated and nohz_full CPUs at runtime, because the kernel may have changed the CPUs specified on the commandline (when specifying all CPUs as isolated, or CPUs that do not exist, ...)

This series adds two files to /sys/devices/system/cpu, where users can see the isolated and nohz_full CPUs in the system.

#1 lore raw

PATCH 1/2 show isolated cpus in sysfs

From: Rik van Riel <[email protected]>

After system bootup, there is no totally reliable way to see which CPUs are isolated, because the kernel may modify the CPUs specified on the isolcpus= kernel command line option.

Export the CPU list that actually got isolated in sysfs, specifically in the file /sys/devices/system/cpu/isolated

---
 drivers/base/cpu.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
drivers/base/cpu.c +12 -0
index f160ea44a86d..ea23ee7b545b 100644
@@ -265,6 +265,17 @@ static ssize_t print_cpus_offline(struct device *dev,
 }
 static DEVICE_ATTR(offline, 0444, print_cpus_offline, NULL);
 
+static ssize_t print_cpus_isolated(struct device *dev,
+				  struct device_attribute *attr, char *buf)
+{
+	int n = 0, len = PAGE_SIZE-2;
+
+	n = scnprintf(buf, len, "%*pbl\n", cpumask_pr_args(cpu_isolated_map));
+
+	return n;
+}
+static DEVICE_ATTR(isolated, 0444, print_cpus_isolated, NULL);
+
 static void cpu_device_release(struct device *dev)
 {
 	/*
@@ -431,6 +442,7 @@ static struct attribute *cpu_root_attrs[] = {
 	&cpu_attrs[2].attr.attr,
 	&dev_attr_kernel_max.attr,
 	&dev_attr_offline.attr,
+	&dev_attr_isolated.attr,
 #ifdef CONFIG_GENERIC_CPU_AUTOPROBE
 	&dev_attr_modalias.attr,
 #endif
Signature · 1 line
2.1.0
#2 lore raw

PATCH 2/2 show nohz_full cpus in sysfs

In reply to [email protected]

From: Rik van Riel <[email protected]>

Currently there is no way to query which CPUs are in nohz_full mode from userspace.

Export the CPU list running in nohz_full mode in sysfs, specifically in the file /sys/devices/system/cpu/nohz_full

---
 drivers/base/cpu.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
drivers/base/cpu.c +17 -0
index ea23ee7b545b..78720e706176 100644
@@ -16,6 +16,7 @@
 #include <linux/acpi.h>
 #include <linux/of.h>
 #include <linux/cpufeature.h>
+#include <linux/tick.h>
 
 #include "base.h"
 
@@ -276,6 +277,19 @@ static ssize_t print_cpus_isolated(struct device *dev,
 }
 static DEVICE_ATTR(isolated, 0444, print_cpus_isolated, NULL);
 
+#ifdef CONFIG_NO_HZ_FULL
+static ssize_t print_cpus_nohz_full(struct device *dev,
+				  struct device_attribute *attr, char *buf)
+{
+	int n = 0, len = PAGE_SIZE-2;
+
+	n = scnprintf(buf, len, "%*pbl\n", cpumask_pr_args(tick_nohz_full_mask));
+
+	return n;
+}
+static DEVICE_ATTR(nohz_full, 0444, print_cpus_nohz_full, NULL);
+#endif
+
 static void cpu_device_release(struct device *dev)
 {
 	/*
@@ -443,6 +457,9 @@ static struct attribute *cpu_root_attrs[] = {
 	&dev_attr_kernel_max.attr,
 	&dev_attr_offline.attr,
 	&dev_attr_isolated.attr,
+#ifdef CONFIG_NO_HZ_FULL
+	&dev_attr_nohz_full.attr,
+#endif
 #ifdef CONFIG_GENERIC_CPU_AUTOPROBE
 	&dev_attr_modalias.attr,
 #endif
Signature · 1 line
2.1.0
#3 lore raw

Re: [PATCH 2/2] show nohz_full cpus in sysfs

Quoted 4 lines · On Fri, 2015-03-27 at 17:50 -0400, [email protected] wrote:

From: Rik van Riel <[email protected]>

Currently there is no way to query which CPUs are in nohz_full mode from userspace.

Hm, they're both (as of your last set) invariant. Is this so an HPC app can automatically bind itself or something? You can't have more than one such app, or rather if you did, they'd need more than which CPUs are HPC capable, they'd need occupancy too, so the query mechanism seems kinda useless. Box driver has to allocate CPUs, and presumably knows the configuration of the box (those who don't become ex box drivers).

-Mike

#4 lore raw

Re: [PATCH 2/2] show nohz_full cpus in sysfs

Quoted 7 lines · Mike Galbraith
Quoted 4 lines · On Fri, 2015-03-27 at 17:50 -0400, [email protected] wrote:

From: Rik van Riel <[email protected]>

Currently there is no way to query which CPUs are in nohz_full mode from userspace.

Hm, they're both (as of your last set) invariant.

So are most of the others in /sys/device/system/cpu

That does not make it less useful to discover what the system setup turned out to be after boot.

Quoted 6 lines

Is this so an HPC app
can automatically bind itself or something? You can't have more than one such app, or rather if you did, they'd need more than which CPUs are HPC capable, they'd need occupancy too, so the query mechanism seems kinda useless. Box driver has to allocate CPUs, and presumably knows the configuration of the box (those who don't become ex box drivers).

Knowing what system you bought does not reduce the usefulness of /proc/cpuinfo.

The CPUs that actually end up isolated or nohz_full can differ from what was specified on the kernel commandline, and being able to see which CPUs ended up in the desired state seems useful.

It can be used by programs like irqbalance to avoid binding IRQs to isolated or nohz_full CPUs, by libvirt to know which CPUs do not get load balancing of SCHED_OTHER tasks, etc...

Signature · 1 line
All rights reversed
#5 lore raw

Re: [PATCH 2/2] show nohz_full cpus in sysfs

Quoted 3 lines · Rik van Riel

It can be used by programs like irqbalance to avoid binding IRQs to isolated or nohz_full CPUs, by libvirt to know which CPUs do not get load balancing of SCHED_OTHER tasks, etc...

Ok, other system packages learning to avoid these does make good sense.

-Mike

#6 lore raw

Re: [PATCH 2/2] show nohz_full cpus in sysfs

In reply to [email protected]

2015-03-27 22:50 GMT+01:00 <[email protected]>:

Quoted 4 lines

From: Rik van Riel <[email protected]>

Currently there is no way to query which CPUs are in nohz_full mode from userspace.

Well you can watch dmesg | grep NO_HZ
But surely sysfs is more convenient from an app.

I guess it's ok, as long as it's strictly Read Only. Here it seems to be the case. And it's not chmod'able, right?

#7 lore raw

Re: [PATCH 2/2] show nohz_full cpus in sysfs

Quoted 11 lines · Frederic Weisbecker

2015-03-27 22:50 GMT+01:00 <[email protected]>:

Quoted 4 lines

From: Rik van Riel <[email protected]>

Currently there is no way to query which CPUs are in nohz_full mode from userspace.

Well you can watch dmesg | grep NO_HZ
But surely sysfs is more convenient from an app.

I guess it's ok, as long as it's strictly Read Only. Here it seems to be the case. And it's not chmod'able, right?

I followed the other code for files in that directory.

Quick testing shows that the cpu info files in /sys/devices/system/cpu are chmoddable, but writing to them fails with -EIO because there is no function set up to handle writes. So yeah, read only :)

Signature · 1 line
All rights reversed
#8 lore raw

Re: [PATCH 2/2] show nohz_full cpus in sysfs

Quoted 13 lines · Rik van Riel
Quoted 11 lines · Frederic Weisbecker

2015-03-27 22:50 GMT+01:00 <[email protected]>:

Quoted 4 lines

From: Rik van Riel <[email protected]>

Currently there is no way to query which CPUs are in nohz_full mode from userspace.

Well you can watch dmesg | grep NO_HZ
But surely sysfs is more convenient from an app.

I guess it's ok, as long as it's strictly Read Only. Here it seems to be the case. And it's not chmod'able, right?

Quoted 6 lines

I followed the other code for files in that directory.

Quick testing shows that the cpu info files in /sys/devices/system/cpu are chmoddable, but writing to them fails with -EIO because there is no function set up to handle writes. So yeah, read only :)

Perfect :-)