Skip to contentSkip to conversation

nvme: Add hardware monitoring support

PATCH Superseded Guenter Roeck 2 participants 3 messages Nov 1, 2019 NVMe

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 nvme: Add hardware monitoring support

nvme devices report temperature information in the controller information (for limits) and in the smart log. Currently, the only means to retrieve this information is the nvme command line interface, which requires super-user privileges.

At the same time, it would be desirable to be able to use NVMe temperature information for thermal control.

This patch adds support to read NVMe temperatures from the kernel using the hwmon API and adds temperature zones for NVMe drives. The thermal subsystem can use this information to set thermal policies, and userspace can access it using libsensors and/or the "sensors" command.

Example output from the "sensors" command:

nvme0-pci-0100

Adapter: PCI adapter
Composite:    +39.0°C  (high = +85.0°C, crit = +85.0°C)
Sensor 1:     +39.0°C
Sensor 2:     +41.0°C

---
v3: NVME -> NVMe
Call nvme_hwmon_init() only once, when the controller is first identified
Protect call to nvme_get_log() and reading the log with mutex Convert error return from nvme_get_log() to Linux error code in nvme_hwmon_get_smart_log()
Don't read smart log for reporting warning and critical limits Use get_unaligned_le16() instead of le16_to_cpup() to read the composite temperature
Use #ifdef CONFIG_NVME_HWMON instead of IS_ENABLED(CONFIG_NVME_HWMON) -EPROTO -> -EIO for generic NVMe level errors
Tab-align '=' in data structure initializations

v2: Use devm_kfree() to release memory in error path

Tested with the following NVMe drives:
Intel SSDPEKKW512G7 500GB
Samsung SSD 960 EVO 500GB
Samsung SSD 970 EVO 500GB
Samsung SSD 970 EVO 1TB

 drivers/nvme/host/Kconfig      |  10 ++
 drivers/nvme/host/Makefile     |   1 +
 drivers/nvme/host/core.c       |   6 ++
 drivers/nvme/host/nvme-hwmon.c | 181 +++++++++++++++++++++++++++++++++
 drivers/nvme/host/nvme.h       |   8 ++
 5 files changed, 206 insertions(+)
 create mode 100644 drivers/nvme/host/nvme-hwmon.c
drivers/nvme/host/Kconfig +10 -0
index 2b36f052bfb9..c6439638a419 100644
@@ -23,6 +23,16 @@ config NVME_MULTIPATH
 	   /dev/nvmeXnY device will show up for each NVMe namespaces,
 	   even if it is accessible through multiple controllers.
 
+config NVME_HWMON
+	bool "NVMe hardware monitoring"
+	depends on (NVME_CORE=y && HWMON=y) || (NVME_CORE=m && HWMON)
+	help
+	  This provides support for NVMe hardware monitoring. If enabled,
+	  a hardware monitoring device will be created for each NVMe drive
+	  in the system.
+
+	  If unsure, say N.
+
 config NVME_FABRICS
 	tristate
 
drivers/nvme/host/Makefile +1 -0
index 8a4b671c5f0c..03de4797a877 100644
@@ -14,6 +14,7 @@ nvme-core-$(CONFIG_TRACING)		+= trace.o
 nvme-core-$(CONFIG_NVME_MULTIPATH)	+= multipath.o
 nvme-core-$(CONFIG_NVM)			+= lightnvm.o
 nvme-core-$(CONFIG_FAULT_INJECTION_DEBUG_FS)	+= fault_inject.o
+nvme-core-$(CONFIG_NVME_HWMON)		+= nvme-hwmon.o
 
 nvme-y					+= pci.o
 
drivers/nvme/host/core.c +6 -0
index fa7ba09dca77..d039e392de36 100644
@@ -2796,6 +2796,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	ctrl->oncs = le16_to_cpu(id->oncs);
 	ctrl->mtfa = le16_to_cpu(id->mtfa);
 	ctrl->oaes = le32_to_cpu(id->oaes);
+	ctrl->wctemp = le16_to_cpu(id->wctemp);
+	ctrl->cctemp = le16_to_cpu(id->cctemp);
+
 	atomic_set(&ctrl->abort_limit, id->acl + 1);
 	ctrl->vwc = id->vwc;
 	if (id->mdts)
@@ -2895,6 +2898,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl)
 	if (ret < 0)
 		return ret;
 
+	if (!ctrl->identified)
+		nvme_hwmon_init(ctrl);
+
 	ctrl->identified = true;
 
 	return 0;
drivers/nvme/host/nvme-hwmon.c new file +181 -0

181 lines not on this page · raw message

drivers/nvme/host/nvme.h +8 -0
index 22e8401352c2..cb3b242a214e 100644
@@ -231,6 +231,8 @@ struct nvme_ctrl {
 	u16 kas;
 	u8 npss;
 	u8 apsta;
+	u16 wctemp;
+	u16 cctemp;
 	u32 oaes;
 	u32 aen_result;
 	u32 ctratt;
@@ -652,4 +654,10 @@ static inline struct nvme_ns *nvme_get_ns_from_dev(struct device *dev)
 	return dev_to_disk(dev)->private_data;
 }
 
+#ifdef CONFIG_NVME_HWMON
+void nvme_hwmon_init(struct nvme_ctrl *ctrl);
+#else
+static inline void nvme_hwmon_init(struct nvme_ctrl *ctrl) { }
+#endif
+
 #endif /* _NVME_H */
Signature · 1 line
2.17.1
#1 lore raw

2019年11月1日(金) 12:56 Guenter Roeck <[email protected]>:

drivers/nvme/host/nvme-hwmon.c line 155

Quoted 3 lines
drivers/nvme/host/nvme-hwmon.c new file +3 -0
+void nvme_hwmon_init(struct nvme_ctrl *ctrl)
+{
+       struct device *dev = ctrl->device;

Should we use 'ctrl->dev' instead of 'ctrl->device'?

The 'ctrl->device' is a pointer to char device and the '->of_node' member is NULL.

So if devm_hwmon_device_register_with_info() (i.e. __hwmon_device_register) is called with 'ctrl->device', it doesn't attempt to register a sensor to a DT thermal zone (i.e. hwmon_thermal_add_sensor() is not called at all).

This change was required, when I tried this nvme hwmon patch with the following DT thermal setup.

https://lore.kernel.org/linux-devicetree/[email protected]/

Quoted 11 lines
drivers/nvme/host/nvme-hwmon.c new file +11 -0
+       struct nvme_hwmon_data *data;
+       struct device *hwmon;
+       int err;
+
+       data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
+       if (!data)
+               return;
+
+       data->ctrl = ctrl;
+       mutex_init(&data->read_lock);
+

drivers/nvme/host/nvme-hwmon.c line 176

Quoted 10 lines
drivers/nvme/host/nvme-hwmon.c new file +10 -0
+       err = nvme_hwmon_get_smart_log(data);
+       if (err) {
+               dev_warn(dev, "Failed to read smart log (error %d)\n", err);
+               devm_kfree(dev, data);
+               return;
+       }
+
+       hwmon = devm_hwmon_device_register_with_info(dev, dev_name(dev), data,
+                                                    &nvme_hwmon_chip_info,
+                                                    NULL);

If the above change is applied, the second 'name' argument is changed from 'nvme0' to '0000:01:00.0' as a side effect. So we may want to change the second argument, too.

Quoted 5 lines
drivers/nvme/host/nvme-hwmon.c new file +5 -0
+       if (IS_ERR(hwmon)) {
+               dev_warn(dev, "Failed to instantiate hwmon device\n");
+               devm_kfree(dev, data);
+       }
+}
#2 lore raw
Quoted 7 lines · Akinobu Mita

2019年11月1日(金) 12:56 Guenter Roeck <[email protected]>:

drivers/nvme/host/nvme-hwmon.c line 155

Quoted 3 lines
drivers/nvme/host/nvme-hwmon.c new file +3 -0
+void nvme_hwmon_init(struct nvme_ctrl *ctrl)
+{
+       struct device *dev = ctrl->device;

Should we use 'ctrl->dev' instead of 'ctrl->device'?

Excellent point, and most definitely yes. I should have done that from the beginning.

Quoted 29 lines

The 'ctrl->device' is a pointer to char device and the '->of_node' member is NULL.

So if devm_hwmon_device_register_with_info() (i.e. __hwmon_device_register) is called with 'ctrl->device', it doesn't attempt to register a sensor to a DT thermal zone (i.e. hwmon_thermal_add_sensor() is not called at all).

This change was required, when I tried this nvme hwmon patch with the following DT thermal setup.

https://lore.kernel.org/linux-devicetree/[email protected]/

Quoted 17 lines
drivers/nvme/host/nvme-hwmon.c new file +17 -0
+       struct nvme_hwmon_data *data;
+       struct device *hwmon;
+       int err;
+
+       data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
+       if (!data)
+               return;
+
+       data->ctrl = ctrl;
+       mutex_init(&data->read_lock);
+
+       err = nvme_hwmon_get_smart_log(data);
+       if (err) {
+               dev_warn(dev, "Failed to read smart log (error %d)\n", err);
+               devm_kfree(dev, data);
+               return;
+       }
Quoted 9 lines

drivers/nvme/host/nvme-hwmon.c line 176

Quoted 4 lines
drivers/nvme/host/nvme-hwmon.c new file +4 -0
+
+       hwmon = devm_hwmon_device_register_with_info(dev, dev_name(dev), data,
+                                                    &nvme_hwmon_chip_info,
+                                                    NULL);

If the above change is applied, the second 'name' argument is changed from 'nvme0' to '0000:01:00.0' as a side effect. So we may want to change the second argument, too.

Yes. I'll just name it "nvme"; after all, that is sufficient and more consistent with other drivers. Currently, we get something like

	nvme0-pci-0100
	nvme1-pci-2500

if there are multiple drives, where the "0" and "1" are not really necessary.

Thanks!
Guenter