Skip to contentSkip to conversation

hwmon: Add amd_energy driver to report energy counters

PATCH Superseded Naveen Krishna Chatradhi 2 participants 5 messages Apr 17, 2020 – Apr 23, 2020

Subsystems (2)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/2 hwmon: Add amd_energy driver to report energy counters

This patch adds hwmon based amd_energy driver support for family 17h processors from AMD.

The driver provides following interface to the userspace
1. Reports the per core consumption
* file: "energy%d_input", label: "Ecore%03d"
2. Reports per socket energy consumption
* file: "energy%d_input", label: "Esocket%d"
3. Reports scaled energy value in Joules.

---
Changes in V2:
Addressed comments from Guenter. Major changes include
1. Use chip_info instead of groups
2. Use standard file naming conventions
3. Report scaled values

 drivers/hwmon/Kconfig      |  10 ++
 drivers/hwmon/Makefile     |   1 +
 drivers/hwmon/amd_energy.c | 251 +++++++++++++++++++++++++++++++++++++
 3 files changed, 262 insertions(+)
 create mode 100644 drivers/hwmon/amd_energy.c
drivers/hwmon/Kconfig +10 -0
index 05a30832c6ba..d83f1403b429 100644
@@ -324,6 +324,16 @@ config SENSORS_FAM15H_POWER
 	  This driver can also be built as a module. If so, the module
 	  will be called fam15h_power.
 
+config SENSORS_AMD_ENERGY
+	tristate "AMD RAPL MSR based Energy driver"
+	depends on X86
+	help
+	  If you say yes here you get support for core and package energy
+	  sensors, based on RAPL MSR for AMD family 17h and above CPUs.
+
+	  This driver can also be built as a module. If so, the module
+	  will be called as amd_energy.
+
 config SENSORS_APPLESMC
 	tristate "Apple SMC (Motion sensor, light sensor, keyboard backlight)"
 	depends on INPUT && X86
drivers/hwmon/Makefile +1 -0
index b0b9c8e57176..318f89dc7133 100644
@@ -45,6 +45,7 @@ obj-$(CONFIG_SENSORS_ADT7411)	+= adt7411.o
 obj-$(CONFIG_SENSORS_ADT7462)	+= adt7462.o
 obj-$(CONFIG_SENSORS_ADT7470)	+= adt7470.o
 obj-$(CONFIG_SENSORS_ADT7475)	+= adt7475.o
+obj-$(CONFIG_SENSORS_AMD_ENERGY) += amd_energy.o
 obj-$(CONFIG_SENSORS_APPLESMC)	+= applesmc.o
 obj-$(CONFIG_SENSORS_ARM_SCMI)	+= scmi-hwmon.o
 obj-$(CONFIG_SENSORS_ARM_SCPI)	+= scpi-hwmon.o
drivers/hwmon/amd_energy.c new file +251 -0

251 lines not on this page · raw message

Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 2/2 hwmon: (amd_energy) Add documentation

Document amd_energy driver with all chips supported by it.

---
 Documentation/hwmon/amd_energy.rst | 83 ++++++++++++++++++++++++++++++
 Documentation/hwmon/index.rst      |  1 +
 2 files changed, 84 insertions(+)
 create mode 100644 Documentation/hwmon/amd_energy.rst
Documentation/hwmon/amd_energy.rst new file +83 -0
new file mode 100644
index 000000000000..3bd5855b4a6b
@@ -0,0 +1,83 @@
+Kernel driver amd_energy
+==========================
+
+Supported chips:
+
+* AMD Family 17h Processors
+
+  Prefix: 'amd_energy'
+
+  Addresses used:  RAPL MSRs
+
+  Datasheets:
+
+  - Processor Programming Reference (PPR) for AMD Family 17h Model 01h, Revision B1 Processors
+
+	https://developer.amd.com/wp-content/resources/55570-B1_PUB.zip
+
+  - Preliminary Processor Programming Reference (PPR) for AMD Family 17h Model 31h, Revision B0 Processors
+
+	https://developer.amd.com/wp-content/resources/56176_ppr_Family_17h_Model_71h_B0_pub_Rev_3.06.zip
+
+Author: Naveen Krishna Chatradhi <[email protected]>
+
+Description
+-----------
+
+The Energy driver exposes the energy counters that are
+reported via the Running Average Power Limit (RAPL)
+Model-specific Registers (MSRs) via the hardware monitor
+(HWMON) sysfs interface.
+
+1. Power, Energy and Time Units
+  * MSR_RAPL_POWER_UNIT/ C001_0299:
+    -> shared with all cores in the socket
+
+2. Energy consumed by each Core
+  * MSR_CORE_ENERGY_STATUS/ C001_029A:
+    -> 32-bitRO, Accumulator, core-level power reporting
+
+3. Energy consumed by Socket
+  * MSR_PACKAGE_ENERGY_STATUS/ C001_029B:
+    -> 32-bitRO, Accumulator, socket-level power reporting,
+    -> shared with all cores in socket
+
+These registers are updated every 1ms and cleared on
+reset of the system.
+
+Energy Caluclation
+------------------
+
+Energy information (in Joules) is based on the multiplier,
+1/2^ESU; where ESU is an unsigned integer read from
+MSR_RAPL_POWER_UNIT register. Default value is 10000b,
+indicating energy status unit is 15.3 micro-Joules increment.
+
+Reported values are scaled as per the formula
+
+scaled value = ((1/2^ESU) * (Raw value) * 1000000UL) in Joules
+
+Users calculate power for a given domain by calculating
+	dEnergy/dTime for that domain.
+
+Sysfs attributes
+----------------
+
+=============== ========  =====================================
+Attribute	Label	  Description
+===============	========  =====================================
+
+* For index N between [1] and [nr_cpus]
+
+===============	========  ======================================
+energy[N]_input EcoreX	  Core Energy   X = [0] to [nr_cpus - 1]
+			  Measured input core energy
+===============	========  ======================================
+
+* For N between [nr_cpus] and [nr_cpus + nr_socks]
+
+===============	========  ======================================
+energy[N]_input EsocketX  Socket Energy X = [0] to [nr_socks -1]
+			  Measured input socket energy
+=============== ========  ======================================
+
Documentation/hwmon/index.rst +1 -0
index 8ef62fd39787..fc4b89810e67 100644
@@ -39,6 +39,7 @@ Hardware Monitoring Kernel Drivers
    adt7470
    adt7475
    amc6821
+   amd_energy
    asb100
    asc7621
    aspeed-pwm-tacho
Signature · 1 line
2.17.1
maintainer
#2 lore raw

Re: [PATCH v2 2/2] hwmon: (amd_energy) Add documentation

Quoted 105 lines · Naveen Krishna Chatradhi

Document amd_energy driver with all chips supported by it.

---
 Documentation/hwmon/amd_energy.rst | 83 ++++++++++++++++++++++++++++++
 Documentation/hwmon/index.rst      |  1 +
 2 files changed, 84 insertions(+)
 create mode 100644 Documentation/hwmon/amd_energy.rst
Documentation/hwmon/amd_energy.rst new file +83 -0
new file mode 100644
index 000000000000..3bd5855b4a6b
@@ -0,0 +1,83 @@
+Kernel driver amd_energy
+==========================
+
+Supported chips:
+
+* AMD Family 17h Processors
+
+  Prefix: 'amd_energy'
+
+  Addresses used:  RAPL MSRs
+
+  Datasheets:
+
+  - Processor Programming Reference (PPR) for AMD Family 17h Model 01h, Revision B1 Processors
+
+	https://developer.amd.com/wp-content/resources/55570-B1_PUB.zip
+
+  - Preliminary Processor Programming Reference (PPR) for AMD Family 17h Model 31h, Revision B0 Processors
+
+	https://developer.amd.com/wp-content/resources/56176_ppr_Family_17h_Model_71h_B0_pub_Rev_3.06.zip
+
+Author: Naveen Krishna Chatradhi <[email protected]>
+
+Description
+-----------
+
+The Energy driver exposes the energy counters that are
+reported via the Running Average Power Limit (RAPL)
+Model-specific Registers (MSRs) via the hardware monitor
+(HWMON) sysfs interface.
+
+1. Power, Energy and Time Units
+  * MSR_RAPL_POWER_UNIT/ C001_0299:
+    -> shared with all cores in the socket
+
+2. Energy consumed by each Core
+  * MSR_CORE_ENERGY_STATUS/ C001_029A:
+    -> 32-bitRO, Accumulator, core-level power reporting
+
+3. Energy consumed by Socket
+  * MSR_PACKAGE_ENERGY_STATUS/ C001_029B:
+    -> 32-bitRO, Accumulator, socket-level power reporting,
+    -> shared with all cores in socket
+
+These registers are updated every 1ms and cleared on
+reset of the system.
+
+Energy Caluclation
+------------------
+
+Energy information (in Joules) is based on the multiplier,
+1/2^ESU; where ESU is an unsigned integer read from
+MSR_RAPL_POWER_UNIT register. Default value is 10000b,
+indicating energy status unit is 15.3 micro-Joules increment.
+
+Reported values are scaled as per the formula
+
+scaled value = ((1/2^ESU) * (Raw value) * 1000000UL) in Joules
+
+Users calculate power for a given domain by calculating
+	dEnergy/dTime for that domain.
+
+Sysfs attributes
+----------------
+
+=============== ========  =====================================
+Attribute	Label	  Description
+===============	========  =====================================
+
+* For index N between [1] and [nr_cpus]
+
+===============	========  ======================================
+energy[N]_input EcoreX	  Core Energy   X = [0] to [nr_cpus - 1]
+			  Measured input core energy
+===============	========  ======================================
+
+* For N between [nr_cpus] and [nr_cpus + nr_socks]
+
+===============	========  ======================================
+energy[N]_input EsocketX  Socket Energy X = [0] to [nr_socks -1]
+			  Measured input socket energy
+=============== ========  ======================================
+
Documentation/hwmon/index.rst +0 -0
index 8ef62fd39787..fc4b89810e67 100644

Documentation/hwmon/index.rst Hardware Monitoring Kernel Drivers · line 45

Quoted 7 lines
Documentation/hwmon/index.rst +1 -0
@@ -39,6 +39,7 @@ Hardware Monitoring Kernel Drivers
    adt7470
    adt7475
    amc6821
+   amd_energy
    asb100
    asc7621
    aspeed-pwm-tacho
groeck@server:~/src/linux-stable$ git am -s amd2
Applying: hwmon: (amd_energy) Add documentation
.git/rebase-apply/patch:95: new blank line at EOF.
+
warning: 1 line adds whitespace errors.

and:

/home/groeck/src/linux-stable/Documentation/hwmon/amd_energy.rst:33: WARNING: Enumerated list ends without a blank line; unexpected unindent. /home/groeck/src/linux-stable/Documentation/hwmon/amd_energy.rst:37: WARNING: Enumerated list ends without a blank line; unexpected unindent. /home/groeck/src/linux-stable/Documentation/hwmon/amd_energy.rst:41: WARNING: Enumerated list ends without a blank line; unexpected unindent.

Guenter

maintainer
#3 lore raw

Re: [PATCH v2 2/2] hwmon: (amd_energy) Add documentation

In reply to Naveen Krishna Chatradhi
Quoted 16 lines · Naveen Krishna Chatradhi

Document amd_energy driver with all chips supported by it.

---
 Documentation/hwmon/amd_energy.rst | 83 ++++++++++++++++++++++++++++++
 Documentation/hwmon/index.rst      |  1 +
 2 files changed, 84 insertions(+)
 create mode 100644 Documentation/hwmon/amd_energy.rst
Documentation/hwmon/amd_energy.rst new file +0 -0
new file mode 100644
index 000000000000..3bd5855b4a6b

Documentation/hwmon/amd_energy.rst line 1

Quoted 1 line
Documentation/hwmon/amd_energy.rst new file +1 -0
@@ -0,0 +1,83 @@
+Kernel driver amd_energy

WARNING: Missing or malformed SPDX-License-Identifier tag in line 1 #30: FILE: Documentation/hwmon/amd_energy.rst:1:
+Kernel driver amd_energy

maintainer
#4 lore raw
In reply to Naveen Krishna Chatradhi
Quoted 10 lines · Naveen Krishna Chatradhi

This patch adds hwmon based amd_energy driver support for family 17h processors from AMD.

The driver provides following interface to the userspace
1. Reports the per core consumption
* file: "energy%d_input", label: "Ecore%03d"
2. Reports per socket energy consumption
* file: "energy%d_input", label: "Esocket%d"
3. Reports scaled energy value in Joules.

Quoted 2 lines

Please run checkpatch --strict and fix the alignment errors it reports.

Quoted 55 lines

---
Changes in V2:
Addressed comments from Guenter. Major changes include
1. Use chip_info instead of groups
2. Use standard file naming conventions
3. Report scaled values

 drivers/hwmon/Kconfig      |  10 ++
 drivers/hwmon/Makefile     |   1 +
 drivers/hwmon/amd_energy.c | 251 +++++++++++++++++++++++++++++++++++++
 3 files changed, 262 insertions(+)
 create mode 100644 drivers/hwmon/amd_energy.c
drivers/hwmon/Kconfig +10 -0
index 05a30832c6ba..d83f1403b429 100644
@@ -324,6 +324,16 @@ config SENSORS_FAM15H_POWER
 	  This driver can also be built as a module. If so, the module
 	  will be called fam15h_power.
 
+config SENSORS_AMD_ENERGY
+	tristate "AMD RAPL MSR based Energy driver"
+	depends on X86
+	help
+	  If you say yes here you get support for core and package energy
+	  sensors, based on RAPL MSR for AMD family 17h and above CPUs.
+
+	  This driver can also be built as a module. If so, the module
+	  will be called as amd_energy.
+
 config SENSORS_APPLESMC
 	tristate "Apple SMC (Motion sensor, light sensor, keyboard backlight)"
 	depends on INPUT && X86
drivers/hwmon/Makefile +1 -0
index b0b9c8e57176..318f89dc7133 100644
@@ -45,6 +45,7 @@ obj-$(CONFIG_SENSORS_ADT7411)	+= adt7411.o
 obj-$(CONFIG_SENSORS_ADT7462)	+= adt7462.o
 obj-$(CONFIG_SENSORS_ADT7470)	+= adt7470.o
 obj-$(CONFIG_SENSORS_ADT7475)	+= adt7475.o
+obj-$(CONFIG_SENSORS_AMD_ENERGY) += amd_energy.o
 obj-$(CONFIG_SENSORS_APPLESMC)	+= applesmc.o
 obj-$(CONFIG_SENSORS_ARM_SCMI)	+= scmi-hwmon.o
 obj-$(CONFIG_SENSORS_ARM_SCPI)	+= scpi-hwmon.o
drivers/hwmon/amd_energy.c new file +3 -0
new file mode 100644
index 000000000000..7127f1f5f5ee
@@ -0,0 +1,251 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+/*

drivers/hwmon/amd_energy.c line 13

Quoted 10 lines
drivers/hwmon/amd_energy.c new file +10 -0
+ * Copyright (C) 2020 Advanced Micro Devices, Inc.
+ */
+
+#include <asm/cpu_device_id.h>
+
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+#include <linux/device.h>
+#include <linux/hwmon.h>
+#include <linux/hwmon-sysfs.h>

Leftover from v1 ?

drivers/hwmon/amd_energy.c line 20

Quoted 7 lines
drivers/hwmon/amd_energy.c new file +7 -0
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/processor.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <linux/sysfs.h>

Leftover from v1 ?

Quoted 62 lines
drivers/hwmon/amd_energy.c new file +62 -0
+#include <linux/types.h>
+
+#define DRVNAME			"amd_energy"
+
+#define ENERGY_PWR_UNIT_MSR	0xC0010299
+#define ENERGY_CORE_MSR		0xC001029A
+#define ENERGY_PKG_MSR		0xC001029B
+
+#define AMD_TIME_UNIT_MASK	0xF0000
+#define AMD_ENERGY_UNIT_MASK	0x01F00
+#define AMD_POWER_UNIT_MASK	0x0000F
+
+#define ENERGY_STATUS_MASK	0xffffffff
+
+#define AMD_FAM_17		0x17 /* ZP, SSP */
+
+/* Useful macros */
+#define AMD_CPU_FAM_ANY(_family, _model)	\
+{						\
+	.vendor		= X86_VENDOR_AMD,	\
+	.family		= _family,		\
+	.model		= _model,		\
+	.feature	= X86_FEATURE_ANY,	\
+}
+
+#define AMD_CPU_FAM(_model)			\
+	AMD_CPU_FAM_ANY(AMD_FAM_##_model, X86_MODEL_ANY)
+
+struct amd_energy_data {
+	struct hwmon_channel_info energy_info;
+	const struct hwmon_channel_info *info[2];
+	struct hwmon_chip_info chip;
+};
+
+static int nr_cpus, nr_socks;
+static uint64_t energy_units;
+
+static int amd_energy_read_labels(struct device *dev,
+			       enum hwmon_sensor_types type,
+			       u32 attr, int channel, const char **str)
+{
+	char *buf = devm_kcalloc(dev, 10, sizeof(char), GFP_KERNEL);
+
+	if (channel >= nr_cpus)
+		scnprintf(buf, 9, "Esocket%u", channel - nr_cpus);
+	else
+		scnprintf(buf, 9, "Ecore%03u", channel);
+
+	*str = buf;
+
+	return 0;
+}
+
+static int get_energy_units(void)
+{
+	uint64_t rapl_units;
+	int ret;
+
+	ret = rdmsrl_safe(ENERGY_PWR_UNIT_MSR, &rapl_units);
+	if (ret)
+		return -EAGAIN;
+

drivers/hwmon/amd_energy.c line 92

Quoted 10 lines
drivers/hwmon/amd_energy.c new file +10 -0
+	energy_units = (rapl_units & AMD_ENERGY_UNIT_MASK) >> 8;
+	return 0;
+}
+
+static int amd_energy_read(struct device *dev,
+			enum hwmon_sensor_types type,
+			u32 attr, int channel, long *val)
+{
+	uint64_t value;
+	int cpu = 0, ret;

There is no need to initialize 'cpu'.

Quoted 9 lines
drivers/hwmon/amd_energy.c new file +9 -0
+	u32 reg;
+
+	if (channel >= nr_cpus) {
+		reg = ENERGY_PKG_MSR;
+		cpu = cpumask_first_and(cpu_online_mask,
+				cpumask_of_node(channel - nr_cpus));
+	} else {
+		reg = ENERGY_CORE_MSR;
+		cpu = channel;

drivers/hwmon/amd_energy.c line 111

Quoted 10 lines
drivers/hwmon/amd_energy.c new file +10 -0
+	}
+
+	if (!cpu_online(cpu))
+		return -ENODEV;
+
+	ret = rdmsrl_safe_on_cpu(cpu, reg, &value);
+	if (ret)
+		return -EAGAIN;
+
+	if ((energy_units == 0) && get_energy_units())

Please no unnecessary ( ).

drivers/hwmon/amd_energy.c line 115

Quoted 4 lines
drivers/hwmon/amd_energy.c new file +4 -0
+		return -EAGAIN;
+
+	/* Energy consumed = (1/(2^ESU) * RAW * 1000000UL) Joules */
+	*val = (long)value * div64_ul(1000000UL, (1 << energy_units));

BIT ?

Quoted 18 lines
drivers/hwmon/amd_energy.c new file +18 -0
+
+	return 0;
+}
+
+static umode_t amd_energy_is_visible(const void *_data,
+			enum hwmon_sensor_types type,
+			u32 attr, int channel)
+{
+	return 0444;
+}
+
+static const struct hwmon_ops amd_energy_ops = {
+	.is_visible = amd_energy_is_visible,
+	.read = amd_energy_read,
+	.read_string = amd_energy_read_labels,
+};
+
+static int amd_create_sensor(struct device *dev,

drivers/hwmon/amd_energy.c line 143

Quoted 10 lines
drivers/hwmon/amd_energy.c new file +10 -0
+			struct amd_energy_data *data,
+			u8 type, u32 config)
+{
+	int i;
+	u32 *s_config;
+	struct hwmon_channel_info *info = &data->energy_info;
+
+	nr_socks = num_online_nodes();
+	nr_cpus = num_online_cpus();
+

This code makes me wonder what happens if some CPUs are taken offline/online dynamically. Have you tried that ?

Quoted 26 lines
drivers/hwmon/amd_energy.c new file +26 -0
+	s_config = devm_kcalloc(dev, nr_cpus + nr_socks,
+				sizeof(u32), GFP_KERNEL);
+	if (!s_config)
+		return -ENOMEM;
+
+	info->type = type;
+	info->config = s_config;
+
+	for (i = 0; i < nr_cpus + nr_socks; i++)
+		s_config[i] = config;
+
+	return 0;
+}
+
+static int amd_energy_probe(struct platform_device *pdev)
+{
+	struct device *hwmon_dev;
+	struct amd_energy_data *data;
+	struct device *dev = &pdev->dev;
+	int ret;
+
+	ret = get_energy_units();
+
+	data = devm_kzalloc(dev, sizeof(struct amd_energy_data), GFP_KERNEL);
+	if (!data)
+		return -ENOMEM;

drivers/hwmon/amd_energy.c line 179

Quoted 10 lines
drivers/hwmon/amd_energy.c new file +10 -0
+
+	data->chip.ops = &amd_energy_ops;
+	data->chip.info = data->info;
+
+	/* Populate per-core energy reporting */
+	data->info[0] = &data->energy_info;
+	amd_create_sensor(dev, data,  hwmon_energy,
+				HWMON_E_INPUT | HWMON_E_LABEL);
+
+	data->info[1] = NULL;

data was allocated with devm_kzalloc(). There is no need to clean that field again.

Quoted 73 lines
drivers/hwmon/amd_energy.c new file +72 -0
+
+	hwmon_dev = devm_hwmon_device_register_with_info(dev, DRVNAME,
+							 data,
+							 &data->chip,
+							 NULL);
+	return PTR_ERR_OR_ZERO(hwmon_dev);
+}
+
+static int amd_energy_remove(struct platform_device *pdev)
+{
+	return 0;
+}
+
+static const struct platform_device_id amd_energy_ids[] = {
+	{ .name = DRVNAME, },
+	{}
+};
+MODULE_DEVICE_TABLE(platform, amd_energy_ids);
+
+static struct platform_driver amd_energy_driver = {
+	.probe = amd_energy_probe,
+	.remove	= amd_energy_remove,
+	.id_table = amd_energy_ids,
+	.driver = {
+		.name = DRVNAME,
+	},
+};
+
+static struct platform_device *amd_energy_platdev;
+
+static const struct x86_cpu_id cpu_ids[] __initconst = {
+	AMD_CPU_FAM(17),
+	{}
+};
+MODULE_DEVICE_TABLE(x86cpu, cpu_ids);
+
+static int __init amd_energy_init(void)
+{
+	int ret;
+
+	if (!x86_match_cpu(cpu_ids))
+		return -ENODEV;
+
+	ret = platform_driver_register(&amd_energy_driver);
+	if (ret < 0)
+		return ret;
+
+	amd_energy_platdev = platform_device_alloc(DRVNAME, 0);
+	if (!amd_energy_platdev)
+		return -ENOMEM;
+
+	ret = platform_device_add(amd_energy_platdev);
+	if (ret) {
+		platform_device_unregister(amd_energy_platdev);
+		return ret;
+	}
+
+	return ret;
+}
+
+static void __exit amd_energy_exit(void)
+{
+	platform_device_unregister(amd_energy_platdev);
+	platform_driver_unregister(&amd_energy_driver);
+}
+
+module_init(amd_energy_init);
+module_exit(amd_energy_exit);
+
+MODULE_DESCRIPTION("Driver for AMD Energy reporting from RAPL MSR via HWMON interface");
+MODULE_AUTHOR("Naveen Krishna Chatradhi <[email protected]>");
+MODULE_LICENSE("GPL");