Skip to contentSkip to conversation

Add support for StorageD3Enable _DSD property

PATCH Superseded David E. Box 11 participants 40 messages Apr 28, 2020 – May 26, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover Add support for StorageD3Enable _DSD property

NVMe storage power management during suspend-to-idle, particularly on laptops, has been inconsistent with some devices working with D3 while others must rely on NVMe APST in order for power savings to be realized. Currently the default is to use APST unless quirked to do otherwise. However newer platforms, like Intel Comet Lake systems, may require NVMe drives to use D3 in order for the PCIe ports to be properly power managed. To make it easier for drivers to choose, these platforms may supply a special "StorageD3Enable" _DSD property under the root port that the device is attached to. If supplied, the driver must use D3 in order for the platform to realize the deepest power savings in suspend-to-idle.

The first patch adds the new _DSD GUID and fowards the property through the pci/acpi layer to the pci device.

The second patch adds support for the property to the nvme driver.

David E. Box (2):
pci: Add ACPI StorageD3Enable _DSD support drivers/nvme: Add support for ACPI StorageD3Enable property

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c |  7 ++++++
 drivers/pci/pci-acpi.c  | 47 +++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.c       |  6 ++++++
 drivers/pci/pci.h       |  4 ++++
 include/linux/pci.h     |  1 +
 6 files changed, 68 insertions(+)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/2 pci: Add ACPI StorageD3Enable _DSD support

NVMe storage power management during suspend-to-idle, particularly on laptops, has been inconsistent with some devices working with D3 while others must rely on NVMe APST in order for power savings to be realized. Currently the default is to use APST unless quirked to do otherwise. However newer platforms, like Intel Comet Lake systems, may require NVMe drives to use D3 in order for the PCIe ports to be properly power managed. To make it easier for drivers to choose, these platforms may supply a special "StorageD3Enable" _DSD property under the root port that the device is attached to. If supplied, the driver must use D3 in order for the platform to realize the deepest power savings in suspend-to-idle.

Adds support for the _DSD to the pci/acpi layer.

---
 drivers/acpi/property.c |  3 +++
 drivers/pci/pci-acpi.c  | 47 +++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.c       |  6 ++++++
 drivers/pci/pci.h       |  4 ++++
 include/linux/pci.h     |  1 +
 5 files changed, 61 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..f09375ab40e4 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* D3 Support for storage devivce: 5025030f-842f-4ab4-a561-99a5189762d0 */
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/pci/pci-acpi.c +47 -0

59 lines not on this page · raw message

drivers/pci/pci.c +6 -0
index 595fcf59843f..217d89f31e8b 100644
@@ -871,6 +871,11 @@ static inline bool platform_pci_bridge_d3(struct pci_dev *dev)
 	return pci_platform_pm ? pci_platform_pm->bridge_d3(dev) : false;
 }
 
+static inline bool platform_pci_storage_d3(struct pci_dev *dev)
+{
+	return pci_platform_pm->storage_d3(dev);
+}
+
 /**
  * pci_raw_set_power_state - Use PCI PM registers to set the power state of
  *			     given PCI device
@@ -2903,6 +2908,7 @@ void pci_pm_init(struct pci_dev *dev)
 	dev->d3_delay = PCI_PM_D3_WAIT;
 	dev->d3cold_delay = PCI_PM_D3COLD_WAIT;
 	dev->bridge_d3 = pci_bridge_d3_possible(dev);
+	dev->storage_d3 = platform_pci_storage_d3(dev);
 	dev->d3cold_allowed = true;
 
 	dev->d1_support = false;
drivers/pci/pci.h +4 -0
index 6d3f75867106..4dc4d05da810 100644
@@ -53,6 +53,9 @@ int pci_bus_error_reset(struct pci_dev *dev);
  *
  * @bridge_d3: Does the bridge allow entering into D3
  *
+ * @storage_d3: 'true' if the bridge contains a storage device that must use D3
+ *		to support platform s2idle
+ *
  * @is_manageable: returns 'true' if given device is power manageable by the
  *		   platform firmware
  *
@@ -77,6 +80,7 @@ int pci_bus_error_reset(struct pci_dev *dev);
  */
 struct pci_platform_pm_ops {
 	bool (*bridge_d3)(struct pci_dev *dev);
+	bool (*storage_d3)(struct pci_dev *dev);
 	bool (*is_manageable)(struct pci_dev *dev);
 	int (*set_state)(struct pci_dev *dev, pci_power_t state);
 	pci_power_t (*get_state)(struct pci_dev *dev);
include/linux/pci.h +1 -0
index 83ce1cdf5676..cbba1555020b 100644
@@ -346,6 +346,7 @@ struct pci_dev {
 	unsigned int	no_d1d2:1;	/* D1 and D2 are forbidden */
 	unsigned int	no_d3cold:1;	/* D3cold is forbidden */
 	unsigned int	bridge_d3:1;	/* Allow D3 for bridge */
+	unsigned int	storage_d3:1;	/* Storage dev must use D3 for s2idle */
 	unsigned int	d3cold_allowed:1;	/* D3cold is allowed by user */
 	unsigned int	mmio_always_on:1;	/* Disallow turning off io/mem
 						   decoding during BAR sizing */
Signature · 1 line
2.20.1
#2 lore raw

Re: [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support

Quoted 81 lines · CEST David E. Box

NVMe storage power management during suspend-to-idle, particularly on laptops, has been inconsistent with some devices working with D3 while others must rely on NVMe APST in order for power savings to be realized. Currently the default is to use APST unless quirked to do otherwise. However newer platforms, like Intel Comet Lake systems, may require NVMe drives to use D3 in order for the PCIe ports to be properly power managed. To make it easier for drivers to choose, these platforms may supply a special "StorageD3Enable" _DSD property under the root port that the device is attached to. If supplied, the driver must use D3 in order for the platform to realize the deepest power savings in suspend-to-idle.

Adds support for the _DSD to the pci/acpi layer.

---
 drivers/acpi/property.c |  3 +++
 drivers/pci/pci-acpi.c  | 47 +++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.c       |  6 ++++++
 drivers/pci/pci.h       |  4 ++++
 include/linux/pci.h     |  1 +
 5 files changed, 61 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..f09375ab40e4 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* D3 Support for storage devivce: 5025030f-842f-4ab4-a561-99a5189762d0 */
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/pci/pci-acpi.c +35 -0
index d21969fba6ab..5df249ebf022 100644
@@ -972,6 +972,52 @@ static bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	return val == 1;
 }
 
+static bool acpi_pci_storage_d3(struct pci_dev *dev)
+{
+	const struct fwnode_handle *fwnode;
+	struct acpi_device *adev;
+	struct pci_dev *root;
+	acpi_handle handle;
+	acpi_status status;
+	u8 val;
+
+	/*
+	 * Look for _DSD property specifying that the storage device on
+	 * the port must use D3 to support deep platform power savings during
+	 * suspend-to-idle
+	 */
+	root = pci_find_pcie_root_port(dev);
+	if (!root)
+		return false;
+
+	adev = ACPI_COMPANION(&root->dev);
+	if (root == dev) {
+		/*
+		 * It is possible that the ACPI companion is not yet bound
+		 * for the root port so look it up manually here.
+		 */
+		if (!adev && !pci_dev_is_added(root))
+			adev = acpi_pci_find_companion(&root->dev);
+	}
+
+	if (!adev)
+		return false;
+
+	status = acpi_get_handle(adev->handle, "PXSX", &handle);
+	if (ACPI_FAILURE(status))
+		return false;
+

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 1019

Quoted 10 lines
drivers/pci/pci-acpi.c +10 -0
+	adev = acpi_bus_get_acpi_device(handle);
+	if (!adev)
+		return false;
+
+	fwnode = acpi_fwnode_handle(adev);
+	if (!fwnode_property_read_u8(fwnode, "StorageD3Enable", &val))
+		return val == 1;
+
+	return false;
+}

Kind of orthogonal to what happens to the second patch in this series, I don't think that the PCI changes below are all needed.

IMO it would be sufficient to export the function above, maybe as pci_acpi_storage_d3(), to drivers, so that they can call it directly as desired.

Since _DSD return data are not allowed by the spec to change between subsequent invocations of it, the interested driver may call this function once at the device init time and quirk it accordingly if needed.

Cheers!

#3 lore raw

Re: [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support

Quoted 119 lines · Rafael J. Wysocki
Quoted 98 lines · CEST David E. Box

NVMe storage power management during suspend-to-idle, particularly on
laptops, has been inconsistent with some devices working with D3 while
others must rely on NVMe APST in order for power savings to be realized.
Currently the default is to use APST unless quirked to do otherwise.
However newer platforms, like Intel Comet Lake systems, may require NVMe
drives to use D3 in order for the PCIe ports to be properly power managed.
To make it easier for drivers to choose, these platforms may supply a
special "StorageD3Enable" _DSD property under the root port that the device
is attached to. If supplied, the driver must use D3 in order for the
platform to realize the deepest power savings in suspend-to-idle.

Adds support for the _DSD to the pci/acpi layer.

Link:
https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro

---
 drivers/acpi/property.c |  3 +++
 drivers/pci/pci-acpi.c  | 47
+++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.c       |  6 ++++++
 drivers/pci/pci.h       |  4 ++++
 include/linux/pci.h     |  1 +
 5 files changed, 61 insertions(+)
drivers/acpi/property.c +0 -0
index e601c4511a8b..f09375ab40e4 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-

ba72-9bf5a26ebe5d */

drivers/acpi/property.c +1 -0
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* D3 Support for storage devivce: 5025030f-842f-4ab4-a561-

99a5189762d0 */

drivers/acpi/property.c +2 -0
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-

1319f52a966b */

drivers/pci/pci-acpi.c +12 -0
index d21969fba6ab..5df249ebf022 100644
@@ -972,6 +972,52 @@ static bool acpi_pci_bridge_d3(struct pci_dev
 *dev)
 	return val == 1;
 }
 
+static bool acpi_pci_storage_d3(struct pci_dev *dev)
+{
+	const struct fwnode_handle *fwnode;
+	struct acpi_device *adev;
+	struct pci_dev *root;
+	acpi_handle handle;
+	acpi_status status;
+	u8 val;
+
+	/*
+	 * Look for _DSD property specifying that the storage device on
+	 * the port must use D3 to support deep platform power savings

during

drivers/pci/pci-acpi.c +10 -0
+	 * suspend-to-idle
+	 */
+	root = pci_find_pcie_root_port(dev);
+	if (!root)
+		return false;
+
+	adev = ACPI_COMPANION(&root->dev);
+	if (root == dev) {
+		/*
+		 * It is possible that the ACPI companion is not yet

bound

drivers/pci/pci-acpi.c +13 -0
+		 * for the root port so look it up manually here.
+		 */
+		if (!adev && !pci_dev_is_added(root))
+			adev = acpi_pci_find_companion(&root->dev);
+	}
+
+	if (!adev)
+		return false;
+
+	status = acpi_get_handle(adev->handle, "PXSX", &handle);
+	if (ACPI_FAILURE(status))
+		return false;
+

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 1019

Quoted 10 lines
drivers/pci/pci-acpi.c +10 -0
+	adev = acpi_bus_get_acpi_device(handle);
+	if (!adev)
+		return false;
+
+	fwnode = acpi_fwnode_handle(adev);
+	if (!fwnode_property_read_u8(fwnode, "StorageD3Enable", &val))
+		return val == 1;
+
+	return false;
+}

Kind of orthogonal to what happens to the second patch in this series, I don't
think that the PCI changes below are all needed.

IMO it would be sufficient to export the function above, maybe as pci_acpi_storage_d3(), to drivers, so that they can call it directly as
desired.

Quoted 4 lines

Since _DSD return data are not allowed by the spec to change between subsequent invocations of it, the interested driver may call this function
once at the device init time and quirk it accordingly if needed.

Yeah this is better. Will do. Thanks.

#4 lore raw

PATCH 2/2 drivers/nvme: Add support for ACPI StorageD3Enable property

In reply to David E. Box

NVMe storage power management during suspend-to-idle, particularly on laptops, has been inconsistent with some devices working with D3 while others must rely on NVMe APST in order for power savings to be realized. Currently the default is to use APST unless quirked to do otherwise. However newer platforms, like Intel Comet Lake systems, may require NVMe drives to use D3 in order for the PCIe ports to be properly power managed. To make it easier for drivers to choose, these platforms may supply a special "StorageD3Enable" _DSD property under the root port that the device is attached to. If supplied, the driver must use D3 in order for the platform to realize the deepest power savings in suspend-to-idle.

Add check of StorageD3Enable property during probe to use D3 as specified by platform firmware.

---
 drivers/nvme/host/pci.c | 7 +++++++
 1 file changed, 7 insertions(+)
drivers/nvme/host/pci.c +7 -0
index 4e79e412b276..4d67735975f6 100644
@@ -2777,6 +2777,13 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 
 	quirks |= check_vendor_combination_bug(pdev);
 
+	/*
+	 * Platform requires storage device to use D3 for kernel managed
+	 * suspend.
+	 */
+	if (pdev->storage_d3)
+		quirks |= NVME_QUIRK_SIMPLE_SUSPEND;
+
 	/*
 	 * Double check that our mempool alloc size will cover the biggest
 	 * command we support.
Signature · 1 line
2.20.1
#5 lore raw
In reply to David E. Box
Quoted 11 lines · David E. Box

NVMe storage power management during suspend-to-idle, particularly on laptops, has been inconsistent with some devices working with D3 while others must rely on NVMe APST in order for power savings to be realized. Currently the default is to use APST unless quirked to do otherwise. However newer platforms, like Intel Comet Lake systems, may require NVMe drives to use D3 in order for the PCIe ports to be properly power managed. To make it easier for drivers to choose, these platforms may supply a special "StorageD3Enable" _DSD property under the root port that the device is attached to. If supplied, the driver must use D3 in order for the platform to realize the deepest power savings in suspend-to-idle.

Quoted 4 lines

The first patch adds the new _DSD GUID and fowards the property through the pci/acpi layer to the pci device.

The second patch adds support for the property to the nvme driver.

I'm not sure who came up with the idea to put this into ACPI, but it belongs into NVMe. Please talk to the NVMe technical working group instead of trying to overrules them in an unrelated group that doesn't apply to all of PCIe.

#6 lore raw
Quoted 25 lines · Christoph Hellwig
Quoted 24 lines · David E. Box

NVMe storage power management during suspend-to-idle, particularly on
laptops, has been inconsistent with some devices working with D3 while
others must rely on NVMe APST in order for power savings to be realized.
Currently the default is to use APST unless quirked to do otherwise.
However newer platforms, like Intel Comet Lake systems, may require NVMe
drives to use D3 in order for the PCIe ports to be properly power managed.
To make it easier for drivers to choose, these platforms may supply a
special "StorageD3Enable" _DSD property under the root port that the device
is attached to. If supplied, the driver must use D3 in order for the
platform to realize the deepest power savings in suspend-to-idle.

The first patch adds the new _DSD GUID and fowards the property through the
pci/acpi layer to the pci device.

Quoted 7 lines
Quoted 1 line

The second patch adds support for the property to the nvme driver.

I'm not sure who came up with the idea to put this into ACPI, but it belongs into NVMe. Please talk to the NVMe technical working group instead of trying to overrules them in an unrelated group that doesn't
apply to all of PCIe.

Agreed that this is not ideal since it does not apply to all of PCIe. But as the property already exists on shipping systems, we need to be able to read it in the NVMe driver and the patch is consitent with the way properties under PCI ports are read.

David

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#7 lore raw
Quoted 10 lines · David E. Box
Quoted 5 lines

I'm not sure who came up with the idea to put this into ACPI, but it belongs into NVMe. Please talk to the NVMe technical working group instead of trying to overrules them in an unrelated group that doesn't
apply to all of PCIe.

Agreed that this is not ideal since it does not apply to all of PCIe. But as the property already exists on shipping systems, we need to be able to read it in the NVMe driver and the patch is consitent with the way properties under PCI ports are read.

The point is that it is not the BIOSes job do decide how Linux does power management. For example D3 has really horrible entry and exit latencies in many cases, and will lead to higher power usage.

#8 lore raw
Quoted 17 lines · Christoph Hellwig
Quoted 8 lines · David E. Box
Quoted 7 lines

I'm not sure who came up with the idea to put this into ACPI, but it
belongs into NVMe. Please talk to the NVMe technical working group
instead of trying to overrules them in an unrelated group that doesn't
apply to all of PCIe.

Quoted 7 lines

Agreed that this is not ideal since it does not apply to all of PCIe.
But as the property already exists on shipping systems, we need to be
able to read it in the NVMe driver and the patch is consitent with the
way properties under PCI ports are read.

Quoted 3 lines

The point is that it is not the BIOSes job do decide how Linux does power management. For example D3 has really horrible entry and exit latencies in many cases, and will lead to higher power usage.

The platform can know which pm policies will save the most power. But since the solution doesn't apply to all PCIe devices (despite BIOS specifying it that way) I'll withdraw this patch. Thanks.

David

#9 lore raw
Quoted 21 lines · David E. Box
Quoted 20 lines · Christoph Hellwig
Quoted 18 lines · David E. Box
Quoted 8 lines

I'm not sure who came up with the idea to put this into ACPI, but
it
belongs into NVMe. Please talk to the NVMe technical working group
instead of trying to overrules them in an unrelated group that doesn't
apply to all of PCIe.

Agreed that this is not ideal since it does not apply to all of PCIe.
But as the property already exists on shipping systems, we need to
be
able to read it in the NVMe driver and the patch is consitent with
the
way properties under PCI ports are read.

Quoted 8 lines
Quoted 4 lines

The point is that it is not the BIOSes job do decide how Linux does power management. For example D3 has really horrible entry and exit
latencies in many cases, and will lead to higher power usage.

The platform can know which pm policies will save the most power. But since the solution doesn't apply to all PCIe devices (despite BIOS specifying it that way) I'll withdraw this patch. Thanks.

Wait, why withdraw? In this case the platform is unfortunately preventing the standard driver from making a proper determination. So while I agree that it's not the BIOSes job, when the platform actively prevents proper operation due to some ill conceived non-standard platform property what is Linux left to do on these systems?

The *patch* is not trying to overrule NVME, and the best I can say is that the Intel Linux team was not in the loop when this was being decided between the platform BIOS implemenation and whomever thought they could just publish random ACPI properties that impacted NVME operation [1].

So now David is trying to get these platform unbroken because they are already shipping with this b0rkage.

[1]:
https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro

maintainer
#10 lore raw
Quoted 37 lines · Dan J
Quoted 21 lines · David E. Box
Quoted 20 lines · Christoph Hellwig
Quoted 18 lines · David E. Box
Quoted 8 lines

I'm not sure who came up with the idea to put this into ACPI, but
it
belongs into NVMe. Please talk to the NVMe technical working group
instead of trying to overrules them in an unrelated group that doesn't
apply to all of PCIe.

Agreed that this is not ideal since it does not apply to all of PCIe.
But as the property already exists on shipping systems, we need to
be
able to read it in the NVMe driver and the patch is consitent with
the
way properties under PCI ports are read.

Quoted 8 lines
Quoted 4 lines

The point is that it is not the BIOSes job do decide how Linux does power management. For example D3 has really horrible entry and exit
latencies in many cases, and will lead to higher power usage.

The platform can know which pm policies will save the most power. But since the solution doesn't apply to all PCIe devices (despite BIOS specifying it that way) I'll withdraw this patch. Thanks.

Wait, why withdraw? In this case the platform is unfortunately preventing the standard driver from making a proper determination. So while I agree that it's not the BIOSes job, when the platform actively prevents proper operation due to some ill conceived non-standard platform property what is Linux left to do on these systems?

Quoted 8 lines

The *patch* is not trying to overrule NVME, and the best I can say is that the Intel Linux team was not in the loop when this was being decided between the platform BIOS implemenation and whomever thought they could just publish random ACPI properties that impacted NVME operation [1].

So now David is trying to get these platform unbroken because they are already shipping with this b0rkage.

Rather than quirking all these cases, which I get the feeling there are many more than we've currently got in our quirk list, perhaps it'd be simpler to default to the simple suspend. AFAIK, the simple suspend works for all platforms, though it may not realize the best power savings and/or exit latency.

#11 lore raw
In reply to Williams, Dan J
Quoted 48 lines · Dan J
Quoted 28 lines · David E. Box
Quoted 21 lines · Christoph Hellwig
Quoted 10 lines · David E. Box
Quoted 9 lines

I'm not sure who came up with the idea to put this into ACPI, but
it
belongs into NVMe. Please talk to the NVMe technical working group
instead of trying to overrules them in an unrelated group that
doesn't
apply to all of PCIe.

Quoted 9 lines

Agreed that this is not ideal since it does not apply to all of PCIe.
But as the property already exists on shipping systems, we need to
be
able to read it in the NVMe driver and the patch is consitent with
the
way properties under PCI ports are read.

Quoted 5 lines

The point is that it is not the BIOSes job do decide how Linux does
power management. For example D3 has really horrible entry and exit
latencies in many cases, and will lead to higher power usage.

Quoted 4 lines

The platform can know which pm policies will save the most power. But
since the solution doesn't apply to all PCIe devices (despite BIOS specifying it that way) I'll withdraw this patch. Thanks.

Wait, why withdraw? In this case the platform is unfortunately preventing the standard driver from making a proper determination. So while I agree that it's not the BIOSes job, when the platform actively
prevents proper operation due to some ill conceived non-standard platform property what is Linux left to do on these systems?

The *patch* is not trying to overrule NVME, and the best I can say is that the Intel Linux team was not in the loop when this was being decided between the platform BIOS implemenation
and whomever thought
they could just publish random ACPI properties that impacted NVME operation [1].

Quoted 3 lines

So now David is trying to get these platform unbroken because they are
already shipping with this b0rkage.

Not drop completely. This patch copied the code used to read _DSD properties under PCI root ports. But I agree that such properties should apply to all devices on those ports and unfortuntely that's not the case here. BIOS got it wrong. My thought in dropping this patch is to rewrite it to read the property directly from the nvme driver. Not the way it's typically done either but it would avoid a global change in the pci core while allowing us to deal with the firmware we have.

David

#12 lore raw
Quoted 7 lines · David E. Box

Not drop completely. This patch copied the code used to read _DSD properties under PCI root ports. But I agree that such properties should apply to all devices on those ports and unfortuntely that's not the case here. BIOS got it wrong. My thought in dropping this patch is to rewrite it to read the property directly from the nvme driver. Not the way it's typically done either but it would avoid a global change in the pci core while allowing us to deal with the firmware we have.

I'd be happy to heave less of this crap in nvme actually. But I'm really pissed this shit got out in the wild. It wasn't clear from the mail that this is something already out there because the idiots coming up with it just went ahead with it. Please just update the commit logs and implementation to clearly mark it as a workaround for buggys systems, which just happen to at least be nice enough to tell us that they are buggy as f^$k.

#13 lore raw
Quoted 14 lines · On Fri, 2020-05-01 at 15:12 +0200, [email protected] wrote:
Quoted 4 lines · David E. Box

Not drop completely. This patch copied the code used to read _DSD properties under PCI root ports. But I agree that such properties should apply to all devices on those ports and unfortuntely that's not

Quoted 8 lines

the case here. BIOS got it wrong. My thought in dropping this patch is
to rewrite it to read the property directly from the nvme driver. Not
the way it's typically done either but it would avoid a global change
in the pci core while allowing us to deal with the firmware we have.

Quoted 8 lines

I'd be happy to heave less of this crap in nvme actually. But I'm really
pissed this shit got out in the wild. It wasn't clear from the mail that this is something already out there because the idiots coming up with it just went ahead with it. Please just update the commit logs and implementation to clearly mark it as a workaround for buggys systems, which just happen to at least be nice enough to tell us that they are buggy as f^$k.

Sure.

#14 lore raw
In reply to Williams, Dan J
Quoted 10 lines · Dan J
Quoted 3 lines

The platform can know which pm policies will save the most power. But since the solution doesn't apply to all PCIe devices (despite BIOS specifying it that way) I'll withdraw this patch. Thanks.

Wait, why withdraw? In this case the platform is unfortunately preventing the standard driver from making a proper determination. So while I agree that it's not the BIOSes job, when the platform actively prevents proper operation due to some ill conceived non-standard platform property what is Linux left to do on these systems?

Quoted 8 lines

The *patch* is not trying to overrule NVME, and the best I can say is that the Intel Linux team was not in the loop when this was being decided between the platform BIOS implemenation and whomever thought they could just publish random ACPI properties that impacted NVME operation [1].

So now David is trying to get these platform unbroken because they are already shipping with this b0rkage.

So can we please clearly mark this as a quirk and warn in the kernel log about a buggy BIOS?

#15 lore raw
In reply to Williams, Dan J
Quoted 8 lines · Dan J

The *patch* is not trying to overrule NVME, and the best I can say is that the Intel Linux team was not in the loop when this was being decided between the platform BIOS implemenation and whomever thought they could just publish random ACPI properties that impacted NVME operation [1].

So now David is trying to get these platform unbroken because they are already shipping with this b0rkage.

This is what we have WARN_TAINT() for though, right? It can suitably warn users when such breakage is detected in the platform.

_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme

#16 lore raw
Quoted 8 lines · David Woodhouse
Quoted 6 lines · Dan J

The *patch* is not trying to overrule NVME, and the best I can say is that the Intel Linux team was not in the loop when this was being decided between the platform BIOS implemenation and whomever thought they could just publish random ACPI properties that impacted NVME operation [1].

Quoted 6 lines
Quoted 2 lines

So now David is trying to get these platform unbroken because they are already shipping with this b0rkage.

This is what we have WARN_TAINT() for though, right? It can suitably warn users when such breakage is detected in the platform.

I see WARN_TAINT() as "BIOS implemented its specification wrong". This case is BIOS "implemented a mechanism in the wrong specification".

#17 lore raw

v2 cover nvme: Add support for ACPI StorageD3Enable property

In reply to David E. Box

This patch set implements a solution for a BIOS hack used on some currently shipping Intel systems to address issues with power management policy decisions concerning PCIe NVMe drives. Some newer Intel platforms, like some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power savings. This is particularly needed to support ATX power supply shutdown on desktop systems. In order to ensure this happens for root ports with storage devices, Microsoft apparently created this ACPI _DSD property as a way to override their driver policy. To my knowledge this property has not been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects Linux since the NVMe driver's default policy of using NVMe APST during suspend-to-idle would lead to higher power consumption for these platforms.

Patch 1 provides a symbol in the PCI/ACPI layer to read the property. Patch 2 uses the symbol in the NVMe driver to select D3 as a quirk if set.

Changes from V2:
- Export the pci_acpi_storage_d3 function for use by drivers as needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this change.
- Add info message that this is a platform quirk.

David E. Box (2):
PCI: Add ACPI StorageD3Enable _DSD support drivers/nvme: Add support for ACPI StorageD3Enable property

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c | 14 ++++++++++
 drivers/pci/pci-acpi.c  | 59 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h     |  2 ++
 4 files changed, 78 insertions(+)
Signature · 1 line
2.20.1
#18 lore raw

Re: [PATCH V2 0/2] nvme: Add support for ACPI StorageD3Enable property

Friendly reminder. Thanks.

David

Quoted 47 lines · David E. Box

This patch set implements a solution for a BIOS hack used on some currently
shipping Intel systems to address issues with power management policy decisions concerning PCIe NVMe drives. Some newer Intel platforms, like
some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power
savings. This is particularly needed to support ATX power supply shutdown
on desktop systems. In order to ensure this happens for root ports with
storage devices, Microsoft apparently created this ACPI _DSD property as a
way to override their driver policy. To my knowledge this property has not
been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects
Linux since the NVMe driver's default policy of using NVMe APST during
suspend-to-idle would lead to higher power consumption for these platforms.

Patch 1 provides a symbol in the PCI/ACPI layer to read the property. Patch 2 uses the symbol in the NVMe driver to select D3 as a quirk if set.

Changes from V2:
- Export the pci_acpi_storage_d3 function for use by drivers as needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this change.
- Add info message that this is a platform quirk.

David E. Box (2):
PCI: Add ACPI StorageD3Enable _DSD support drivers/nvme: Add support for ACPI StorageD3Enable property

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c | 14 ++++++++++
 drivers/pci/pci-acpi.c  | 59
+++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h     |  2 ++
 4 files changed, 78 insertions(+)
#19 lore raw

Re: [PATCH V2 0/2] nvme: Add support for ACPI StorageD3Enable property

Quoted 2 lines · David E. Box

Friendly reminder. Thanks.

Are you looking for this to be merged by ACPI with an NVMe ack, or merged by NVMe with an ACPI ack? It sometimes helps to be explicit to break the log jam.

Quoted 52 lines

David

Quoted 47 lines · David E. Box

This patch set implements a solution for a BIOS hack used on some currently
shipping Intel systems to address issues with power management policy decisions concerning PCIe NVMe drives. Some newer Intel platforms, like
some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power
savings. This is particularly needed to support ATX power supply shutdown
on desktop systems. In order to ensure this happens for root ports with
storage devices, Microsoft apparently created this ACPI _DSD property as a
way to override their driver policy. To my knowledge this property has not
been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects
Linux since the NVMe driver's default policy of using NVMe APST during
suspend-to-idle would lead to higher power consumption for these platforms.

Patch 1 provides a symbol in the PCI/ACPI layer to read the property. Patch 2 uses the symbol in the NVMe driver to select D3 as a quirk if set.

Changes from V2:
- Export the pci_acpi_storage_d3 function for use by drivers as needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this change.
- Add info message that this is a platform quirk.

David E. Box (2):
PCI: Add ACPI StorageD3Enable _DSD support drivers/nvme: Add support for ACPI StorageD3Enable property

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c | 14 ++++++++++
 drivers/pci/pci-acpi.c  | 59
+++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h     |  2 ++
 4 files changed, 78 insertions(+)
#20 lore raw

Re: [PATCH V2 0/2] nvme: Add support for ACPI StorageD3Enable property

Quoted 7 lines · Dan Williams
Quoted 1 line · David E. Box

Friendly reminder. Thanks.

Are you looking for this to be merged by ACPI with an NVMe ack, or merged by NVMe with an ACPI ack? It sometimes helps to be explicit to break the log jam.

Ah. NVMe merge with ACPI ack. Thanks.

#21 lore raw

PATCH v3 1/1 drivers/nvme: Add support for ACPI StorageD3Enable property

In reply to David E. Box

Cc: [email protected],
[email protected],
[email protected]

This patch implements a solution for a BIOS hack used on some currently shipping Intel systems to change driver power management policy for PCIe NVMe drives. Some newer Intel platforms, like some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power savings. This is particularly needed to support ATX power supply shutdown on desktop systems. In order to ensure this happens for root ports with storage devices, Microsoft apparently created this ACPI _DSD property as a way to influence their driver policy. To my knowledge this property has not been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects Linux since the NVMe driver's default policy of using NVMe APST during suspend-to-idle prevents the PCI root port from going to D3 and leads to higher power consumption for these platforms. The power consumption difference may be negligible on laptop systems, but many watts on desktop systems when the ATX power supply is blocked from powering down.

The patch creates a new nvme_acpi_storage_d3 function to check for the StorageD3Enable property during probe and enables D3 as a quirk if set. It also provides a 'noacpi' module parameter to allow skipping the quirk if needed.

Tested on:
PM961 NVMe SED Samsung 512GB
INTEL SSDPEKKF512G8

---

Changes from V3:
- Remove check for "not yet bound" ACPI companion device since this will not be a concern at driver probe time per Rafael.
- Move storage_d3 function out of PCI core and into NVMe driver since there's nothing the PCI core can do with this code as noted by Bjorn.

Changes from V2:
- Export the pci_acpi_storage_d3 function for use by drivers as needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this change.
- Add info message that this is a platform quirk.

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c | 55 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-99a5189762d0 */
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/nvme/host/pci.c +55 -0

79 lines not on this page · raw message

Signature · 1 line
2.20.1
#22 lore raw

PATCH v4 1/1 drivers/nvme: Add support for ACPI StorageD3Enable property

In reply to David E. Box

This patch implements a solution for a BIOS hack used on some currently shipping Intel systems to change driver power management policy for PCIe NVMe drives. Some newer Intel platforms, like some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power savings. This is particularly needed to support ATX power supply shutdown on desktop systems. In order to ensure this happens for root ports with storage devices, Microsoft apparently created this ACPI _DSD property as a way to influence their driver policy. To my knowledge this property has not been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects Linux since the NVMe driver's default policy of using NVMe APST during suspend-to-idle prevents the PCI root port from going to D3 and leads to higher power consumption for these platforms. The power consumption difference may be negligible on laptop systems, but many watts on desktop systems when the ATX power supply is blocked from powering down.

The patch creates a new nvme_acpi_storage_d3 function to check for the StorageD3Enable property during probe and enables D3 as a quirk if set. It also provides a 'noacpi' module parameter to allow skipping the quirk if needed.

Tested on:
PM961 NVMe SED Samsung 512GB
INTEL SSDPEKKF512G8

---
Changes from V3:
- Use pcie_find_root_port() instead of pci_find_pcie_root_port(), changed in 5.8.
- Remove "Cc:" emails that ended up at top of V3 commit message.
- Fix changelog numbering.

Changes from V2:
- Remove check for "not yet bound" ACPI companion device since this will not be a concern at driver probe time per Rafael.
- Move storage_d3 function out of PCI core and into NVMe driver since there's nothing the PCI core can do with this code as noted by Bjorn.

Changes from V1:
- Export the pci_acpi_storage_d3 function for use by drivers as needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this change.
- Add info message that this is a platform quirk.

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c | 55 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-99a5189762d0 */
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/nvme/host/pci.c +55 -0

79 lines not on this page · raw message

Signature · 1 line
2.20.1
#23 lore raw

Re: [PATCH v4] drivers/nvme: Add support for ACPI StorageD3Enable property

I love your patch! Yet something to improve:

Hi "David,

I love your patch! Yet something to improve:

[auto build test ERROR on pm/linux-next]
[also build test ERROR on block/for-next linus/master v5.8-rc3 next-20200702] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/David-E-Box/drivers-nvme-Add-support-for-ACPI-StorageD3Enable-property/20200703-065019
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm.git linux-next
config: x86_64-rhel-7.6 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-14) 9.3.0
reproduce (this is a W=1 build):
        # save the attached .config to linux build tree
        make W=1 ARCH=x86_64 

If you fix the issue, kindly add following tag as appropriate

All errors (new ones prefixed by >>, old ones prefixed by <<):

>> ERROR: modpost: "acpi_bus_get_acpi_device" [drivers/nvme/host/nvme.ko] undefined!
>> ERROR: modpost: "acpi_bus_put_acpi_device" [drivers/nvme/host/nvme.ko] undefined!

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#24 lore raw

Re: [PATCH v4] drivers/nvme: Add support for ACPI StorageD3Enable property

In reply to David E. Box
Quoted 114 lines · David E. Box

This patch implements a solution for a BIOS hack used on some currently shipping Intel systems to change driver power management policy for PCIe NVMe drives. Some newer Intel platforms, like some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power savings. This is particularly needed to support ATX power supply shutdown on desktop systems. In order to ensure this happens for root ports with storage devices, Microsoft apparently created this ACPI _DSD property as a way to influence their driver policy. To my knowledge this property has not been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects Linux since the NVMe driver's default policy of using NVMe APST during suspend-to-idle prevents the PCI root port from going to D3 and leads to higher power consumption for these platforms. The power consumption difference may be negligible on laptop systems, but many watts on desktop systems when the ATX power supply is blocked from powering down.

The patch creates a new nvme_acpi_storage_d3 function to check for the StorageD3Enable property during probe and enables D3 as a quirk if set. It also provides a 'noacpi' module parameter to allow skipping the quirk if needed.

Tested on:
PM961 NVMe SED Samsung 512GB
INTEL SSDPEKKF512G8

---
Changes from V3:
- Use pcie_find_root_port() instead of pci_find_pcie_root_port(), changed in 5.8.
- Remove "Cc:" emails that ended up at top of V3 commit message.
- Fix changelog numbering.

Changes from V2:
- Remove check for "not yet bound" ACPI companion device since this will not be a concern at driver probe time per Rafael.
- Move storage_d3 function out of PCI core and into NVMe driver since there's nothing the PCI core can do with this code as noted by Bjorn.

Changes from V1:
- Export the pci_acpi_storage_d3 function for use by drivers as needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this change.
- Add info message that this is a platform quirk.

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c | 55 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
        /* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
        GUID_INIT(0x6c501103, 0xc189, 0x4296,
                  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+       /* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-99a5189762d0 */
+       GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+                 0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/nvme/host/pci.c +23 -0
index e2bacd369a88..a3d3a82b0437 100644
@@ -4,6 +4,7 @@
  * Copyright (c) 2011-2014, Intel Corporation.
  */
 
+#include <linux/acpi.h>
 #include <linux/aer.h>
 #include <linux/async.h>
 #include <linux/blkdev.h>
@@ -94,6 +95,10 @@ static unsigned int poll_queues;
 module_param_cb(poll_queues, &io_queue_count_ops, &poll_queues, 0644);
 MODULE_PARM_DESC(poll_queues, "Number of queues to use for polled IO.");
 
+static bool noacpi;
+module_param(noacpi, bool, 0444);
+MODULE_PARM_DESC(noacpi, "disable acpi bios quirks");
+
 struct nvme_dev;
 struct nvme_queue;
 
@@ -2757,6 +2762,46 @@ static unsigned long check_vendor_combination_bug(struct pci_dev *pdev)
        return 0;
 }
 
+static bool nvme_acpi_storage_d3(struct pci_dev *dev)
+{
+       const struct fwnode_handle *fwnode;
+       struct acpi_device *adev;
+       struct pci_dev *root;
+       acpi_handle handle;
+       acpi_status status;
+       bool ret = false;
+       u8 val;
+
+       /*
+        * Look for _DSD property specifying that the storage device on
+        * the port must use D3 to support deep platform power savings during
+        * suspend-to-idle
+        */
+       root = pcie_find_root_port(dev);
+       if (!root)
+               return false;

drivers/nvme/host/pci.c static unsigned long check_vendor_combination_bug(struct pci_dev *pdev) · line 2792

Quoted 10 lines
drivers/nvme/host/pci.c +10 -0
+
+       adev = ACPI_COMPANION(&root->dev);
+       if (!adev)
+               return false;
+
+       status = acpi_get_handle(adev->handle, "PXSX", &handle);
+       if (ACPI_FAILURE(status))
+               return false;
+
+       adev = acpi_bus_get_acpi_device(handle);

This function needs to be exported to modules for nvme to be able to use it when modular.

drivers/nvme/host/pci.c static unsigned long check_vendor_combination_bug(struct pci_dev *pdev) · line 2800

Quoted 8 lines
drivers/nvme/host/pci.c +8 -0
+       if (!adev)
+               return false;
+
+       fwnode = acpi_fwnode_handle(adev);
+       if (!fwnode_property_read_u8(fwnode, "StorageD3Enable", &val))
+               ret = (val == 1);
+
+       acpi_bus_put_acpi_device(adev);

And same here.

Quoted 27 lines
drivers/nvme/host/pci.c +14 -1
+
+       return ret;
+}
+
 static void nvme_async_probe(void *data, async_cookie_t cookie)
 {
        struct nvme_dev *dev = data;
@@ -2806,6 +2851,16 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 
        quirks |= check_vendor_combination_bug(pdev);
 
+       if (!noacpi && nvme_acpi_storage_d3(pdev)) {
+               /*
+                * Some systems use a bios work around to ask for D3 on
+                * platforms that support kernel managed suspend.
+                */
+               dev_info(&pdev->dev,
+                        "platform quirk: setting simple suspend\n");
+               quirks |= NVME_QUIRK_SIMPLE_SUSPEND;
+       }
+
        /*
         * Double check that our mempool alloc size will cover the biggest
         * command we support.
--

2.20.1

#25 lore raw

Re: [PATCH v4] drivers/nvme: Add support for ACPI StorageD3Enable property

Quoted 153 lines · Rafael J. Wysocki
Quoted 151 lines · David E. Box

This patch implements a solution for a BIOS hack used on some currently
shipping Intel systems to change driver power management policy for PCIe
NVMe drives. Some newer Intel platforms, like some Comet Lake systems,
require that PCIe devices use D3 when doing suspend-to-idle in order to
allow the platform to realize maximum power savings. This is particularly
needed to support ATX power supply shutdown on desktop systems. In order to
ensure this happens for root ports with storage devices, Microsoft apparently created this ACPI _DSD property as a way to influence their
driver policy. To my knowledge this property has not been discussed with
the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects
Linux since the NVMe driver's default policy of using NVMe APST during
suspend-to-idle prevents the PCI root port from going to D3 and leads to
higher power consumption for these platforms. The power consumption difference may be negligible on laptop systems, but many watts on desktop
systems when the ATX power supply is blocked from powering down.

The patch creates a new nvme_acpi_storage_d3 function to check for the
StorageD3Enable property during probe and enables D3 as a quirk if set. It
also provides a 'noacpi' module parameter to allow skipping the quirk if
needed.

Tested on:
PM961 NVMe SED Samsung 512GB
INTEL SSDPEKKF512G8

Link:
https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro

---
Changes from V3:
- Use pcie_find_root_port() instead of
pci_find_pcie_root_port(),
changed in 5.8.
- Remove "Cc:" emails that ended up at top of V3 commit
message.
- Fix changelog numbering.

Changes from V2:
- Remove check for "not yet bound" ACPI companion device
since
this will not be a concern at driver probe time per
Rafael.
- Move storage_d3 function out of PCI core and into NVMe
driver
since there's nothing the PCI core can do with this code
as
noted by Bjorn.

Changes from V1:
- Export the pci_acpi_storage_d3 function for use by
drivers as
needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this
change.
- Add info message that this is a platform quirk.

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c | 55
+++++++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
drivers/acpi/property.c +0 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
        /* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-

ba72-9bf5a26ebe5d */

        GUID_INIT(0x6c501103, 0xc189, 0x4296,
                  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),

+ /* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-
99a5189762d0 */

drivers/acpi/property.c +2 -0
+       GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+                 0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-

1319f52a966b */

drivers/nvme/host/pci.c +5 -0
index e2bacd369a88..a3d3a82b0437 100644
@@ -4,6 +4,7 @@
  * Copyright (c) 2011-2014, Intel Corporation.
  */
 
+#include <linux/acpi.h>
 #include <linux/aer.h>
 #include <linux/async.h>
 #include <linux/blkdev.h>
@@ -94,6 +95,10 @@ static unsigned int poll_queues;
 module_param_cb(poll_queues, &io_queue_count_ops, &poll_queues,
 0644);
 MODULE_PARM_DESC(poll_queues, "Number of queues to use for polled
 IO.");
 
+static bool noacpi;
+module_param(noacpi, bool, 0444);
+MODULE_PARM_DESC(noacpi, "disable acpi bios quirks");
+
 struct nvme_dev;

struct nvme_queue;

drivers/nvme/host/pci.c +12 -0
@@ -2757,6 +2762,46 @@ static unsigned long
 check_vendor_combination_bug(struct pci_dev *pdev)
        return 0;
 }
 
+static bool nvme_acpi_storage_d3(struct pci_dev *dev)
+{
+       const struct fwnode_handle *fwnode;
+       struct acpi_device *adev;
+       struct pci_dev *root;
+       acpi_handle handle;
+       acpi_status status;
+       bool ret = false;
+       u8 val;
+
+       /*
+        * Look for _DSD property specifying that the storage

device on
+ * the port must use D3 to support deep platform power
savings during

drivers/nvme/host/pci.c +10 -0
+        * suspend-to-idle
+        */
+       root = pcie_find_root_port(dev);
+       if (!root)
+               return false;
+
+       adev = ACPI_COMPANION(&root->dev);
+       if (!adev)
+               return false;
+
Quoted 8 lines

drivers/nvme/host/pci.c static unsigned long check_vendor_combination_bug(struct pci_dev *pdev) · line 2792

Quoted 5 lines
drivers/nvme/host/pci.c +5 -0
+       status = acpi_get_handle(adev->handle, "PXSX", &handle);
+       if (ACPI_FAILURE(status))
+               return false;
+
+       adev = acpi_bus_get_acpi_device(handle);

This function needs to be exported to modules for nvme to be able to use it when modular.

Missed that they weren't exported when I moved from pci-acpi to nvme as it worked with my config. But looking again I see there's already an acpi_bus_get_device() that's exported and should work here. Unfortunately, it would be the only function that's not stubbed out so per Cristoph's later comment, this will need a CONFIG_ACPI wrapper.

David

#26 lore raw

Re: [PATCH v4] drivers/nvme: Add support for ACPI StorageD3Enable property

In reply to David E. Box
Quoted 124 lines · David E. Box

This patch implements a solution for a BIOS hack used on some currently shipping Intel systems to change driver power management policy for PCIe NVMe drives. Some newer Intel platforms, like some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power savings. This is particularly needed to support ATX power supply shutdown on desktop systems. In order to ensure this happens for root ports with storage devices, Microsoft apparently created this ACPI _DSD property as a way to influence their driver policy. To my knowledge this property has not been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects Linux since the NVMe driver's default policy of using NVMe APST during suspend-to-idle prevents the PCI root port from going to D3 and leads to higher power consumption for these platforms. The power consumption difference may be negligible on laptop systems, but many watts on desktop systems when the ATX power supply is blocked from powering down.

The patch creates a new nvme_acpi_storage_d3 function to check for the StorageD3Enable property during probe and enables D3 as a quirk if set. It also provides a 'noacpi' module parameter to allow skipping the quirk if needed.

Tested on:
PM961 NVMe SED Samsung 512GB
INTEL SSDPEKKF512G8

---
Changes from V3:

(quoted diff) +0 -0
 	- Use pcie_find_root_port() instead of pci_find_pcie_root_port(),
 	  changed in 5.8.
 	- Remove "Cc:" emails that ended up at top of V3 commit message.
 	- Fix changelog numbering.

Changes from V2:
- Remove check for "not yet bound" ACPI companion device since this will not be a concern at driver probe time per Rafael.
- Move storage_d3 function out of PCI core and into NVMe driver since there's nothing the PCI core can do with this code as noted by Bjorn.

Changes from V1:
- Export the pci_acpi_storage_d3 function for use by drivers as needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this change.
- Add info message that this is a platform quirk.

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c | 55 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-99a5189762d0 */
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/nvme/host/pci.c +34 -0
index e2bacd369a88..a3d3a82b0437 100644
@@ -4,6 +4,7 @@
  * Copyright (c) 2011-2014, Intel Corporation.
  */
 
+#include <linux/acpi.h>
 #include <linux/aer.h>
 #include <linux/async.h>
 #include <linux/blkdev.h>
@@ -94,6 +95,10 @@ static unsigned int poll_queues;
 module_param_cb(poll_queues, &io_queue_count_ops, &poll_queues, 0644);
 MODULE_PARM_DESC(poll_queues, "Number of queues to use for polled IO.");
 
+static bool noacpi;
+module_param(noacpi, bool, 0444);
+MODULE_PARM_DESC(noacpi, "disable acpi bios quirks");
+
 struct nvme_dev;
 struct nvme_queue;
 
@@ -2757,6 +2762,46 @@ static unsigned long check_vendor_combination_bug(struct pci_dev *pdev)
 	return 0;
 }
 
+static bool nvme_acpi_storage_d3(struct pci_dev *dev)
+{
+	const struct fwnode_handle *fwnode;
+	struct acpi_device *adev;
+	struct pci_dev *root;
+	acpi_handle handle;
+	acpi_status status;
+	bool ret = false;
+	u8 val;
+
+	/*
+	 * Look for _DSD property specifying that the storage device on
+	 * the port must use D3 to support deep platform power savings during
+	 * suspend-to-idle
+	 */
+	root = pcie_find_root_port(dev);
+	if (!root)
+		return false;
+
+	adev = ACPI_COMPANION(&root->dev);
+	if (!adev)
+		return false;
+
+	status = acpi_get_handle(adev->handle, "PXSX", &handle);
+	if (ACPI_FAILURE(status))
+		return false;
+
+	adev = acpi_bus_get_acpi_device(handle);
+	if (!adev)

drivers/nvme/host/pci.c static unsigned long check_vendor_combination_bug(struct pci_dev *pdev) · line 2803

Quoted 10 lines
drivers/nvme/host/pci.c +10 -0
+		return false;
+
+	fwnode = acpi_fwnode_handle(adev);
+	if (!fwnode_property_read_u8(fwnode, "StorageD3Enable", &val))
+		ret = (val == 1);
+
+	acpi_bus_put_acpi_device(adev);
+
+	return ret;
+}

Doesn't this need a CONFIG_ACPI of some sort? Or are all these calls properly stubbed out?

#27 lore raw

PATCH v5 1/1 drivers/nvme: Add support for ACPI StorageD3Enable property

In reply to David E. Box

This patch implements a solution for a BIOS hack used on some currently shipping Intel systems to change driver power management policy for PCIe NVMe drives. Some newer Intel platforms, like some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power savings. This is particularly needed to support ATX power supply shutdown on desktop systems. In order to ensure this happens for root ports with storage devices, Microsoft apparently created this ACPI _DSD property as a way to influence their driver policy. To my knowledge this property has not been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects Linux since the NVMe driver's default policy of using NVMe APST during suspend-to-idle prevents the PCI root port from going to D3 and leads to higher power consumption for these platforms. The power consumption difference may be negligible on laptop systems, but many watts on desktop systems when the ATX power supply is blocked from powering down.

The patch creates a new nvme_acpi_storage_d3 function to check for the StorageD3Enable property during probe and enables D3 as a quirk if set. It also provides a 'noacpi' module parameter to allow skipping the quirk if needed.

Tested on:
PM961 NVMe SED Samsung 512GB
INTEL SSDPEKKF512G8

---
Changes from V4:
- Add support for North complex PCI root ports.
- Use acpi_bus_get_device instead of acpi_bus_get_acpi_device. Also fixes compiler error on V4 reported by [email protected].
- Place CONFIG_ACPI around function since acpi_bus_get_device is not stubbed out.

Changes from V3:
- Use pcie_find_root_port() instead of pci_find_pcie_root_port(), changed in 5.8.
- Remove "Cc:" emails that ended up at top of V3 commit message.
- Fix changelog numbering.

Changes from V2:
- Remove check for "not yet bound" ACPI companion device since this will not be a concern at driver probe time per Rafael.
- Move storage_d3 function out of PCI core and into NVMe driver since there's nothing the PCI core can do with this code as noted by Bjorn.

Changes from V1:
- Export the pci_acpi_storage_d3 function for use by drivers as needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this change.
- Add info message that this is a platform quirk.

 drivers/acpi/property.c |  3 ++
 drivers/nvme/host/pci.c | 64 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 67 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-99a5189762d0 */
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/nvme/host/pci.c +64 -0

88 lines not on this page · raw message

Signature · 7 lines
2.20.1


_______________________________________________
Linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
maintainer Reviewed-by on v5 1/1
#28 lore raw

Re: [PATCH V5] drivers/nvme: Add support for ACPI StorageD3Enable property

Quoted 135 lines · David E. Box

This patch implements a solution for a BIOS hack used on some currently shipping Intel systems to change driver power management policy for PCIe NVMe drives. Some newer Intel platforms, like some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power savings. This is particularly needed to support ATX power supply shutdown on desktop systems. In order to ensure this happens for root ports with storage devices, Microsoft apparently created this ACPI _DSD property as a way to influence their driver policy. To my knowledge this property has not been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects Linux since the NVMe driver's default policy of using NVMe APST during suspend-to-idle prevents the PCI root port from going to D3 and leads to higher power consumption for these platforms. The power consumption difference may be negligible on laptop systems, but many watts on desktop systems when the ATX power supply is blocked from powering down.

The patch creates a new nvme_acpi_storage_d3 function to check for the StorageD3Enable property during probe and enables D3 as a quirk if set. It also provides a 'noacpi' module parameter to allow skipping the quirk if needed.

Tested on:
PM961 NVMe SED Samsung 512GB
INTEL SSDPEKKF512G8

---
Changes from V4:
- Add support for North complex PCI root ports.
- Use acpi_bus_get_device instead of acpi_bus_get_acpi_device. Also fixes compiler error on V4 reported by [email protected].
- Place CONFIG_ACPI around function since acpi_bus_get_device is not stubbed out.

Changes from V3:
- Use pcie_find_root_port() instead of pci_find_pcie_root_port(), changed in 5.8.
- Remove "Cc:" emails that ended up at top of V3 commit message.
- Fix changelog numbering.

Changes from V2:
- Remove check for "not yet bound" ACPI companion device since this will not be a concern at driver probe time per Rafael.
- Move storage_d3 function out of PCI core and into NVMe driver since there's nothing the PCI core can do with this code as noted by Bjorn.

Changes from V1:
- Export the pci_acpi_storage_d3 function for use by drivers as needed instead of modifying the pci header.
- Add missing put on acpi device handle.
- Add 'noacpi' module parameter to allow undoing this change.
- Add info message that this is a platform quirk.

 drivers/acpi/property.c |  3 ++
 drivers/nvme/host/pci.c | 64 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 67 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
        /* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
        GUID_INIT(0x6c501103, 0xc189, 0x4296,
                  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+       /* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-99a5189762d0 */
+       GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+                 0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/nvme/host/pci.c +36 -0
index b1d18f0633c7..7c0be363eb22 100644
@@ -4,6 +4,7 @@
  * Copyright (c) 2011-2014, Intel Corporation.
  */
 
+#include <linux/acpi.h>
 #include <linux/aer.h>
 #include <linux/async.h>
 #include <linux/blkdev.h>
@@ -94,6 +95,10 @@ static unsigned int poll_queues;
 module_param_cb(poll_queues, &io_queue_count_ops, &poll_queues, 0644);
 MODULE_PARM_DESC(poll_queues, "Number of queues to use for polled IO.");
 
+static bool noacpi;
+module_param(noacpi, bool, 0444);
+MODULE_PARM_DESC(noacpi, "disable acpi bios quirks");
+
 struct nvme_dev;
 struct nvme_queue;
 
@@ -2759,6 +2764,55 @@ static unsigned long check_vendor_combination_bug(struct pci_dev *pdev)
        return 0;
 }
 
+#ifdef CONFIG_ACPI
+static bool nvme_acpi_storage_d3(struct pci_dev *dev)
+{
+       const struct fwnode_handle *fwnode;
+       struct acpi_device *adev;
+       struct pci_dev *root;
+       acpi_handle handle;
+       acpi_status status;
+       u8 val;
+
+       /*
+        * Look for _DSD property specifying that the storage device on
+        * the port must use D3 to support deep platform power savings during
+        * suspend-to-idle
+        */
+       root = pcie_find_root_port(dev);
+       if (!root)
+               return false;
+
+       adev = ACPI_COMPANION(&root->dev);
+       if (!adev)
+               return false;
+
+       /*
+        * The property is defined in the PXSX device for South complex ports
+        * and in the PEGP device for North complex ports.
+        */
+       status = acpi_get_handle(adev->handle, "PXSX", &handle);
+       if (ACPI_FAILURE(status)) {
+               status = acpi_get_handle(adev->handle, "PEGP", &handle);
+               if (ACPI_FAILURE(status))

drivers/nvme/host/pci.c static unsigned long check_vendor_combination_bug(struct pci_dev *pdev) · line 2807

Quoted 10 lines
drivers/nvme/host/pci.c +10 -0
+                       return false;
+       }
+
+       if (acpi_bus_get_device(handle, &adev))
+               return false;
+
+       fwnode = acpi_fwnode_handle(adev);
+
+       return fwnode_property_read_u8(fwnode, "StorageD3Enable", &val) ?
+               false : val == 1;

I would write this as

if (fwnode_property_read_u8(acpi_fwnode_handle(adev),

                            "StorageD3Enable", &val))
        return false;

return val == 1;

to eliminate the redundant fwnode variable and untangle the last checks somewhat.

With that changed please feel free to add

to the patch.

Thanks!

Quoted 31 lines
drivers/nvme/host/pci.c +18 -1
+}
+#else
+static inline bool nvme_acpi_storage_d3(struct pci_dev *dev)
+{
+       return false;
+}
+#endif
+
 static void nvme_async_probe(void *data, async_cookie_t cookie)
 {
        struct nvme_dev *dev = data;
@@ -2808,6 +2862,16 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 
        quirks |= check_vendor_combination_bug(pdev);
 
+       if (!noacpi && nvme_acpi_storage_d3(pdev)) {
+               /*
+                * Some systems use a bios work around to ask for D3 on
+                * platforms that support kernel managed suspend.
+                */
+               dev_info(&pdev->dev,
+                        "platform quirk: setting simple suspend\n");
+               quirks |= NVME_QUIRK_SIMPLE_SUSPEND;
+       }
+
        /*
         * Double check that our mempool alloc size will cover the biggest
         * command we support.
--

2.20.1

maintainer Applied
#29 lore raw

Re: [PATCH V5] drivers/nvme: Add support for ACPI StorageD3Enable property

Quoted 6 lines · Rafael J. Wysocki

I would write this as

if (fwnode_property_read_u8(acpi_fwnode_handle(adev),

                            "StorageD3Enable", &val))
        return false;
Quoted 8 lines

return val == 1;

to eliminate the redundant fwnode variable and untangle the last checks somewhat.

With that changed please feel free to add

Applied to nvme-5.9 with the above fixup.

#30 lore raw

Re: [PATCH V5] drivers/nvme: Add support for ACPI StorageD3Enable property

In reply to David E. Box
Quoted 18 lines · David E. Box
drivers/nvme/host/pci.c +18 -0
+#ifdef CONFIG_ACPI
+static bool nvme_acpi_storage_d3(struct pci_dev *dev)
+{
+	const struct fwnode_handle *fwnode;
+	struct acpi_device *adev;
+	struct pci_dev *root;
+	acpi_handle handle;
+	acpi_status status;
+	u8 val;
+
+	/*
+	 * Look for _DSD property specifying that the storage device on
+	 * the port must use D3 to support deep platform power savings during
+	 * suspend-to-idle
+	 */
+	root = pcie_find_root_port(dev);
+	if (!root)
+		return false;

drivers/nvme/host/pci.c static unsigned long check_vendor_combination_bug(struct pci_dev *pdev) · line 2794

Quoted 10 lines
drivers/nvme/host/pci.c +10 -0
+
+	adev = ACPI_COMPANION(&root->dev);
+	if (!adev)
+		return false;
+
+	/*
+	 * The property is defined in the PXSX device for South complex ports
+	 * and in the PEGP device for North complex ports.
+	 */
+	status = acpi_get_handle(adev->handle, "PXSX", &handle);

So I'm curious why we need to directly look at the PXSX and PEGP devices instead of the ACPI_COMPANION node attached to the pci device?

I've looked around and I can't find any documentation that defines the the PXSX and PEGP device names.

I've dumped some ACPI from a system that uses the PXSX name and StorageD3Cold attribute:

    Scope (\_SB.PCI0.GP14)
    {
        Device (PXSX)
        {
            Name (_ADR, 0x0000000000000000)  // _ADR: Address
            Method (_STA, 0, NotSerialized)  // _STA: Status
            {
                Return (0x0F)
            }

            Name (_DSD, Package (0x02)  // _DSD: Device-Specific Data
            {
                ToUUID ("5025030f-842f-4ab4-a561-99a5189762d0"),
                Package (0x01)
                {
                    Package (0x02)
                    {
                        "StorageD3Enable",
                        One
                    }
                }
            })
        }
    }

It looks to me like it's just the firmware node for the NVMe device attached to the root port. Is that the correct assumption?

I'm wondering if we can simplify the look up logic to look at the ACPI_COMPANION of the pci device?

The reason I ask is that I'm working on enabling S0i3 on an AMD device. This device also defines the StorageD3Enable property, but it don't use the PXSX name:

    Scope (GPP6) {
        Device (NVME)
        {
            Name (_ADR, Zero)  // _ADR: Address

            Name (_DSD, Package (0x02)  // _DSD: Device-Specific Data
            {
                ToUUID ("5025030f-842f-4ab4-a561-99a5189762d0"),
                Package (0x01)
                {
                    Package (0x02)
                    {
                        "StorageD3Enable",
                        One
                    }
                }
            })
        }
    }

The Windows [documentation](https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro#d3-support) makes it sound like the _DSD should be defined on the PCI device.

I can send one of the following patches depending on the feedback:
1) Additionally check the pci device's ACPI_COMPANION for the _DSD.
2) Delete the PXSX and PEGP lookups and only look at the pci device's ACPI_COMPANION.

Quoted 4 lines
drivers/nvme/host/pci.c +4 -0
+	if (ACPI_FAILURE(status)) {
+		status = acpi_get_handle(adev->handle, "PEGP", &handle);
+		if (ACPI_FAILURE(status))
+			return false;

drivers/nvme/host/pci.c static unsigned long check_vendor_combination_bug(struct pci_dev *pdev) · line 2808

Quoted 10 lines
drivers/nvme/host/pci.c +10 -0
+	}
+
+	if (acpi_bus_get_device(handle, &adev))
+		return false;
+
+	fwnode = acpi_fwnode_handle(adev);
+
+	return fwnode_property_read_u8(fwnode, "StorageD3Enable", &val) ?
+		false : val == 1;
+}

Thanks,
Raul

p.s., Sorry for the second message, I somehow mangled the headers in the first message and dropped the Message-Id.

#31 lore raw

PATCH v2 1/2 PCI: Add ACPI StorageD3Enable _DSD support

In reply to David E. Box

StorageD3Enable is a boolean property that indicates that the platform wants to use D3 for PCIe storage drives during suspend-to-idle. It is a BIOS work around that is currently in use on shipping systems like some Intel Comet Lake platforms. It is meant to change default driver policy for suspend that may cause higher power consumption.

Add the DSD property for recognition by fwnode calls and provide an exported symbol for device drivers to use to read the property as needed.

---
 drivers/acpi/property.c |  3 +++
 drivers/pci/pci-acpi.c  | 59 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h     |  2 ++
 3 files changed, 64 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-99a5189762d0 */
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/pci/pci-acpi.c +59 -0

65 lines not on this page · raw message

include/linux/pci.h +2 -0
index 83ce1cdf5676..396fcb269a60 100644
@@ -2318,10 +2318,12 @@ struct irq_domain *pci_host_bridge_acpi_msi_domain(struct pci_bus *bus);
 void
 pci_msi_register_fwnode_provider(struct fwnode_handle *(*fn)(struct device *));
 bool pci_pr3_present(struct pci_dev *pdev);
+bool pci_acpi_storage_d3(struct pci_dev *dev);
 #else
 static inline struct irq_domain *
 pci_host_bridge_acpi_msi_domain(struct pci_bus *bus) { return NULL; }
 static inline bool pci_pr3_present(struct pci_dev *pdev) { return false; }
+static inline bool pci_acpi_storage_d3(struct pci_dev *dev) { return false; }
 #endif
 
 #ifdef CONFIG_EEH
Signature · 1 line
2.20.1
#32 lore raw

Re: [PATCH V2 1/2] PCI: Add ACPI StorageD3Enable _DSD support

Quoted 54 lines · David E. Box

StorageD3Enable is a boolean property that indicates that the platform wants to use D3 for PCIe storage drives during suspend-to-idle. It is a BIOS work around that is currently in use on shipping systems like some Intel Comet Lake platforms. It is meant to change default driver policy for suspend that may cause higher power consumption.

Add the DSD property for recognition by fwnode calls and provide an exported symbol for device drivers to use to read the property as needed.

---
 drivers/acpi/property.c |  3 +++
 drivers/pci/pci-acpi.c  | 59 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h     |  2 ++
 3 files changed, 64 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-99a5189762d0 */
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/pci/pci-acpi.c +15 -0
index d21969fba6ab..732df524e09c 100644
@@ -972,6 +972,65 @@ static bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	return val == 1;
 }
 
+/**
+ * pci_acpi_storage_d3 - whether root port requests D3 for idle suspend
+ * @pdev: PCI device to check
+ *
+ * Returns true if the ACPI companion device contains the "StorageD3Enable"
+ * _DSD property and the value is 1. This indicates that the root port is
+ * used by a storage device and the platform is requesting D3 for the
+ * device during suspend to idle in order to support platform pm.
+ */
+bool pci_acpi_storage_d3(struct pci_dev *dev)
+{
+	const struct fwnode_handle *fwnode;
+	struct acpi_device *adev;
+	struct pci_dev *root;
+	acpi_handle handle;

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 999

Quoted 10 lines
drivers/pci/pci-acpi.c +10 -0
+	acpi_status status;
+	bool ret = false;
+	u8 val;
+
+	/*
+	 * Look for _DSD property specifying that the storage device on
+	 * the port must use D3 to support deep platform power savings during
+	 * suspend-to-idle
+	 */
+	root = pci_find_pcie_root_port(dev);

I think this would need to be updated to apply to v5.8-rc1 after 6ae72bfa656e ("PCI: Unify pcie_find_root_port() and pci_find_pcie_root_port()").

https://git.kernel.org/linus/6ae72bfa656e

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 1010

Quoted 11 lines
drivers/pci/pci-acpi.c +11 -0
+	if (!root)
+		return false;
+
+	adev = ACPI_COMPANION(&root->dev);
+	if (!adev) {
+		/*
+		 * It is possible that the ACPI companion is not yet bound
+		 * for the root port so look it up manually here.
+		 */
+		if (!adev && !pci_dev_is_added(root))
+			adev = acpi_pci_find_companion(&root->dev);

I see that you copied this "ACPI companion not yet bound" thing from acpi_pci_bridge_d3(). But it's ugly.

Isn't there a way we can bind the ACPI companion during normal PCI enumeration so we don't need this exception case?

I really do not like the idea of putting this code in the PCI core because AFAICT the PCI core can do nothing with this information.

If we could make sure during enumeration that the root port always has an ACPI companion, this code could go to the nvme driver itself. And we could also clean up the ugliness in acpi_pci_bridge_d3().

Rafael, is that possible? I don't really know how the companion device gets set. Maybe this is could be done somewhere around pci_device_add()?

Quoted 51 lines
drivers/pci/pci-acpi.c +23 -0
+	}
+
+	if (!adev)
+		return false;
+
+	status = acpi_get_handle(adev->handle, "PXSX", &handle);
+	if (ACPI_FAILURE(status))
+		return false;
+
+	adev = acpi_bus_get_acpi_device(handle);
+	if (!adev)
+		return false;
+
+	fwnode = acpi_fwnode_handle(adev);
+	if (!fwnode_property_read_u8(fwnode, "StorageD3Enable", &val))
+		ret = (val == 1);
+
+	acpi_bus_put_acpi_device(adev);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(pci_acpi_storage_d3);
+
 static bool acpi_pci_power_manageable(struct pci_dev *dev)
 {
 	struct acpi_device *adev = ACPI_COMPANION(&dev->dev);
include/linux/pci.h +2 -0
index 83ce1cdf5676..396fcb269a60 100644
@@ -2318,10 +2318,12 @@ struct irq_domain *pci_host_bridge_acpi_msi_domain(struct pci_bus *bus);
 void
 pci_msi_register_fwnode_provider(struct fwnode_handle *(*fn)(struct device *));
 bool pci_pr3_present(struct pci_dev *pdev);
+bool pci_acpi_storage_d3(struct pci_dev *dev);
 #else
 static inline struct irq_domain *
 pci_host_bridge_acpi_msi_domain(struct pci_bus *bus) { return NULL; }
 static inline bool pci_pr3_present(struct pci_dev *pdev) { return false; }
+static inline bool pci_acpi_storage_d3(struct pci_dev *dev) { return false; }
 #endif
 
 #ifdef CONFIG_EEH
Signature · 7 lines
2.20.1


_______________________________________________
linux-nvme mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-nvme
#33 lore raw

Re: [PATCH V2 1/2] PCI: Add ACPI StorageD3Enable _DSD support

Quoted 94 lines · Bjorn Helgaas
Quoted 54 lines · David E. Box

StorageD3Enable is a boolean property that indicates that the platform wants to use D3 for PCIe storage drives during suspend-to-idle. It is a BIOS work around that is currently in use on shipping systems like some Intel Comet Lake platforms. It is meant to change default driver policy for suspend that may cause higher power consumption.

Add the DSD property for recognition by fwnode calls and provide an exported symbol for device drivers to use to read the property as needed.

---
 drivers/acpi/property.c |  3 +++
 drivers/pci/pci-acpi.c  | 59 +++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h     |  2 ++
 3 files changed, 64 insertions(+)
drivers/acpi/property.c +3 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
      /* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
      GUID_INIT(0x6c501103, 0xc189, 0x4296,
                0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+     /* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-99a5189762d0 */
+     GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+               0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
drivers/pci/pci-acpi.c +15 -0
index d21969fba6ab..732df524e09c 100644
@@ -972,6 +972,65 @@ static bool acpi_pci_bridge_d3(struct pci_dev *dev)
      return val == 1;
 }
 
+/**
+ * pci_acpi_storage_d3 - whether root port requests D3 for idle suspend
+ * @pdev: PCI device to check
+ *
+ * Returns true if the ACPI companion device contains the "StorageD3Enable"
+ * _DSD property and the value is 1. This indicates that the root port is
+ * used by a storage device and the platform is requesting D3 for the
+ * device during suspend to idle in order to support platform pm.
+ */
+bool pci_acpi_storage_d3(struct pci_dev *dev)
+{
+     const struct fwnode_handle *fwnode;
+     struct acpi_device *adev;
+     struct pci_dev *root;
+     acpi_handle handle;

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 999

Quoted 10 lines
drivers/pci/pci-acpi.c +10 -0
+     acpi_status status;
+     bool ret = false;
+     u8 val;
+
+     /*
+      * Look for _DSD property specifying that the storage device on
+      * the port must use D3 to support deep platform power savings during
+      * suspend-to-idle
+      */
+     root = pci_find_pcie_root_port(dev);

I think this would need to be updated to apply to v5.8-rc1 after 6ae72bfa656e ("PCI: Unify pcie_find_root_port() and pci_find_pcie_root_port()").

https://git.kernel.org/linus/6ae72bfa656e

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 1010

Quoted 11 lines
drivers/pci/pci-acpi.c +11 -0
+     if (!root)
+             return false;
+
+     adev = ACPI_COMPANION(&root->dev);
+     if (!adev) {
+             /*
+              * It is possible that the ACPI companion is not yet bound
+              * for the root port so look it up manually here.
+              */
+             if (!adev && !pci_dev_is_added(root))
+                     adev = acpi_pci_find_companion(&root->dev);

I see that you copied this "ACPI companion not yet bound" thing from acpi_pci_bridge_d3(). But it's ugly.

Isn't there a way we can bind the ACPI companion during normal PCI enumeration so we don't need this exception case?

I really do not like the idea of putting this code in the PCI core because AFAICT the PCI core can do nothing with this information.

Quoted 6 lines

If we could make sure during enumeration that the root port always has an ACPI companion, this code could go to the nvme driver itself. And we could also clean up the ugliness in acpi_pci_bridge_d3().

Rafael, is that possible? I don't really know how the companion device gets set.

That's a bit convoluted.

device_add() calls device_platform_notify(), before calling bus_add_device().

device_platform_notify() calls acpi_platform_notify() which invokes acpi_device_notify() that looks for the companion via type->find_companion() which for PCI points to acpi_pci_find_companion(). If found, the companion is attached to the dev structure as a physical_node, via acpi_bind_one().

So by the time bus_probe_device() runs, the companion should be there already - if it is there at all.

The parent ACPI companion should be present when the child is probing too, as per the above.

Quoted 1 line

Maybe this is could be done somewhere around pci_device_add()?

It is done in there.

It is not necessary to call acpi_pci_find_companion() from pci_acpi_storage_d3() as long as that function is required to be called by the target device's driver probe or later.

Ths acpi_pci_bridge_d3() case is different, though, AFAICS, because it is invoked in the pci_pm_init() path, via pci_bridge_d3_possible(), and that gets called from pci_device_add() *before* calling device_add().

Mika, is that why acpi_pci_find_companion() gets callled from acpi_pci_bridge_d3()?

#34 lore raw

Re: [PATCH V2 1/2] PCI: Add ACPI StorageD3Enable _DSD support

Quoted 11 lines · Rafael J. Wysocki

It is not necessary to call acpi_pci_find_companion() from pci_acpi_storage_d3() as long as that function is required to be called by the target device's driver probe or later.

Ths acpi_pci_bridge_d3() case is different, though, AFAICS, because it is invoked in the pci_pm_init() path, via pci_bridge_d3_possible(), and that gets called from pci_device_add() *before* calling device_add().

Mika, is that why acpi_pci_find_companion() gets callled from acpi_pci_bridge_d3()?

Yes, that's correct.

#35 lore raw

Re: [PATCH V2 1/2] PCI: Add ACPI StorageD3Enable _DSD support

In reply to Rafael J. Wysocki
Quoted 113 lines · Rafael J. Wysocki
Quoted 111 lines · Bjorn Helgaas
Quoted 80 lines · David E. Box

StorageD3Enable is a boolean property that indicates that the platform
wants to use D3 for PCIe storage drives during suspend-to-idle. It is a
BIOS work around that is currently in use on shipping systems like some
Intel Comet Lake platforms. It is meant to change default driver policy for
suspend that may cause higher power consumption.

Add the DSD property for recognition by fwnode calls and provide an
exported symbol for device drivers to use to read the property as needed.

Link:
https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro

---
 drivers/acpi/property.c |  3 +++
 drivers/pci/pci-acpi.c  | 59
+++++++++++++++++++++++++++++++++++++++++
 include/linux/pci.h     |  2 ++
 3 files changed, 64 insertions(+)
drivers/acpi/property.c +0 -0
index e601c4511a8b..c2e2ae774a19 100644
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
      /* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-

ba72-9bf5a26ebe5d */

      GUID_INIT(0x6c501103, 0xc189, 0x4296,
                0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),

+ /* Storage device needs D3 GUID: 5025030f-842f-4ab4-a561-
99a5189762d0 */

drivers/acpi/property.c +2 -0
+     GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+               0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-

1319f52a966b */

drivers/pci/pci-acpi.c +2 -0
index d21969fba6ab..732df524e09c 100644
@@ -972,6 +972,65 @@ static bool acpi_pci_bridge_d3(struct
 pci_dev *dev)
      return val == 1;
 }
 
+/**
+ * pci_acpi_storage_d3 - whether root port requests D3 for idle

suspend
+ * @pdev: PCI device to check
+ *
+ * Returns true if the ACPI companion device contains the "StorageD3Enable"
+ * _DSD property and the value is 1. This indicates that the root port is
+ * used by a storage device and the platform is requesting D3 for the
+ * device during suspend to idle in order to support platform pm.
+ */

drivers/pci/pci-acpi.c +12 -0
+bool pci_acpi_storage_d3(struct pci_dev *dev)
+{
+     const struct fwnode_handle *fwnode;
+     struct acpi_device *adev;
+     struct pci_dev *root;
+     acpi_handle handle;
+     acpi_status status;
+     bool ret = false;
+     u8 val;
+
+     /*
+      * Look for _DSD property specifying that the storage

device on
+ * the port must use D3 to support deep platform power
savings during

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 999

Quoted 3 lines
drivers/pci/pci-acpi.c +3 -0
+      * suspend-to-idle
+      */
+     root = pci_find_pcie_root_port(dev);

I think this would need to be updated to apply to v5.8-rc1 after 6ae72bfa656e ("PCI: Unify pcie_find_root_port() and pci_find_pcie_root_port()").

https://git.kernel.org/linus/6ae72bfa656e

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 1010

Quoted 12 lines
drivers/pci/pci-acpi.c +7 -0
+     if (!root)
+             return false;
+
+     adev = ACPI_COMPANION(&root->dev);
+     if (!adev) {
+             /*
+              * It is possible that the ACPI companion is not

yet bound

drivers/pci/pci-acpi.c +4 -0
+              * for the root port so look it up manually here.
+              */
+             if (!adev && !pci_dev_is_added(root))
+                     adev = acpi_pci_find_companion(&root->dev);

I see that you copied this "ACPI companion not yet bound" thing from
acpi_pci_bridge_d3(). But it's ugly.

Isn't there a way we can bind the ACPI companion during normal PCI enumeration so we don't need this exception case?

Quoted 6 lines
Quoted 6 lines

I really do not like the idea of putting this code in the PCI core because AFAICT the PCI core can do nothing with this information.

If we could make sure during enumeration that the root port always has
an ACPI companion, this code could go to the nvme driver itself.

This is fine with me if it's acceptable with the nvme maintainers.

Quoted 24 lines
Quoted 5 lines

And
we could also clean up the ugliness in acpi_pci_bridge_d3().

Rafael, is that possible? I don't really know how the companion device gets set.

That's a bit convoluted.

device_add() calls device_platform_notify(), before calling bus_add_device().

device_platform_notify() calls acpi_platform_notify() which invokes acpi_device_notify() that looks for the companion via type->find_companion() which for PCI points to acpi_pci_find_companion(). If found, the companion is attached to the
dev structure as a physical_node, via acpi_bind_one().

So by the time bus_probe_device() runs, the companion should be there already - if it is there at all.

The parent ACPI companion should be present when the child is probing too, as per the above.

Quoted 7 lines
Quoted 1 line

Maybe this is could be done somewhere around pci_device_add()?

It is done in there.

It is not necessary to call acpi_pci_find_companion() from pci_acpi_storage_d3() as long as that function is required to be called by the target device's driver probe or later.

Thanks for the clarification. Since this would get called at driver probe this bit of code is not needed.

David

#36 lore raw

Re: [PATCH V2 1/2] PCI: Add ACPI StorageD3Enable _DSD support

In reply to Rafael J. Wysocki
Quoted 85 lines · Rafael J. Wysocki
Quoted 61 lines · Bjorn Helgaas
Quoted 22 lines · David E. Box

StorageD3Enable is a boolean property that indicates that the platform wants to use D3 for PCIe storage drives during suspend-to-idle. It is a BIOS work around that is currently in use on shipping systems like some Intel Comet Lake platforms. It is meant to change default driver policy for suspend that may cause higher power consumption.

drivers/pci/pci-acpi.c +15 -0
+/**
+ * pci_acpi_storage_d3 - whether root port requests D3 for idle suspend
+ * @pdev: PCI device to check
+ *
+ * Returns true if the ACPI companion device contains the "StorageD3Enable"
+ * _DSD property and the value is 1. This indicates that the root port is
+ * used by a storage device and the platform is requesting D3 for the
+ * device during suspend to idle in order to support platform pm.
+ */
+bool pci_acpi_storage_d3(struct pci_dev *dev)
+{
+     const struct fwnode_handle *fwnode;
+     struct acpi_device *adev;
+     struct pci_dev *root;
+     acpi_handle handle;

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 999

Quoted 10 lines
drivers/pci/pci-acpi.c +10 -0
+     acpi_status status;
+     bool ret = false;
+     u8 val;
+
+     /*
+      * Look for _DSD property specifying that the storage device on
+      * the port must use D3 to support deep platform power savings during
+      * suspend-to-idle
+      */
+     root = pci_find_pcie_root_port(dev);

I think this would need to be updated to apply to v5.8-rc1 after 6ae72bfa656e ("PCI: Unify pcie_find_root_port() and pci_find_pcie_root_port()").

https://git.kernel.org/linus/6ae72bfa656e

drivers/pci/pci-acpi.c static bool acpi_pci_bridge_d3(struct pci_dev *dev) · line 1010

Quoted 11 lines
drivers/pci/pci-acpi.c +11 -0
+     if (!root)
+             return false;
+
+     adev = ACPI_COMPANION(&root->dev);
+     if (!adev) {
+             /*
+              * It is possible that the ACPI companion is not yet bound
+              * for the root port so look it up manually here.
+              */
+             if (!adev && !pci_dev_is_added(root))
+                     adev = acpi_pci_find_companion(&root->dev);

I see that you copied this "ACPI companion not yet bound" thing from acpi_pci_bridge_d3(). But it's ugly.

Isn't there a way we can bind the ACPI companion during normal PCI enumeration so we don't need this exception case?

I really do not like the idea of putting this code in the PCI core because AFAICT the PCI core can do nothing with this information.

Quoted 6 lines

If we could make sure during enumeration that the root port always has an ACPI companion, this code could go to the nvme driver itself. And we could also clean up the ugliness in acpi_pci_bridge_d3().

Rafael, is that possible? I don't really know how the companion device gets set.

That's a bit convoluted.

device_add() calls device_platform_notify(), before calling bus_add_device().

device_platform_notify() calls acpi_platform_notify() which invokes acpi_device_notify() that looks for the companion via type->find_companion() which for PCI points to acpi_pci_find_companion(). If found, the companion is attached to the dev structure as a physical_node, via acpi_bind_one().

So by the time bus_probe_device() runs, the companion should be there already - if it is there at all.

The parent ACPI companion should be present when the child is probing too, as per the above.

Quoted 7 lines
Quoted 1 line

Maybe this is could be done somewhere around pci_device_add()?

It is done in there.

It is not necessary to call acpi_pci_find_companion() from pci_acpi_storage_d3() as long as that function is required to be called by the target device's driver probe or later.

OK, great. IIUC, that means this function doesn't need to be in drivers/pci and it could be moved to the NVMe code.

Quoted 7 lines

Ths acpi_pci_bridge_d3() case is different, though, AFAICS, because it is invoked in the pci_pm_init() path, via pci_bridge_d3_possible(), and that gets called from pci_device_add() *before* calling device_add().

Mika, is that why acpi_pci_find_companion() gets called from acpi_pci_bridge_d3()?

Is pdev->bridge_d3 really needed before pci_device_add()? It would be really nice if there were a way to get rid of that manual lookup of the companion device in acpi_pci_bridge_d3().

Bjorn

#37 lore raw

Re: [PATCH V2 1/2] PCI: Add ACPI StorageD3Enable _DSD support

In reply to David E. Box
Quoted 2 lines · David E. Box

StorageD3Enable is a boolean property that indicates that the platform wants to use D3 for PCIe storage drives during suspend-to-idle.

Is this something that should apply to plug-in drives, or does this only apply to soldered-in things?

Quoted 3 lines

It is a
BIOS work around that is currently in use on shipping systems like some Intel Comet Lake platforms.

What is this BIOS work around? Is there a defect here that's being worked around? What's the defect?

Quoted 2 lines

It is meant to change default driver policy for
suspend that may cause higher power consumption.

I guess this means that by changing the driver policy from the default, we can save some power?

Quoted 4 lines

Add the DSD property for recognition by fwnode calls and provide an exported symbol for device drivers to use to read the property as needed.

There is surprisingly little information in this intro. The whole paragraph under "Modern Standby Power Management" is duplicated immediately below in "D3 Support". Maybe that's a copyediting error that displaced useful information.

It says "drivers should go to the deepest appropriate state" so "function drivers don't have to manage implementation details". No doubt "drivers" and "function drivers" is a meaningful distinction to Windows cognoscenti, but it's not to me.

It talks about "enabling D3" without specifying D3hot or D3cold.

It talks about "D3 support for storage devices." All PCI devices are required to support both D3hot and D3cold, so this must be talking about some other sort of support; I suppose maybe it's a hint about whether a driver should *use* D3hot (or D3cold, I can't tell).

It says nothing about where to look for the _DSD: on a Root Port or on the NVMe endpoint.

Bjorn

#38 lore raw

Re: [PATCH V2 1/2] PCI: Add ACPI StorageD3Enable _DSD support

Quoted 17 lines · Bjorn Helgaas
Quoted 3 lines · David E. Box

StorageD3Enable is a boolean property that indicates that the platform
wants to use D3 for PCIe storage drives during suspend-to-idle.

Is this something that should apply to plug-in drives, or does this only apply to soldered-in things?

Quoted 4 lines

It is a
BIOS work around that is currently in use on shipping systems like some
Intel Comet Lake platforms.

What is this BIOS work around? Is there a defect here that's being worked around? What's the defect?

Quoted 5 lines
Quoted 2 lines

It is meant to change default driver policy for
suspend that may cause higher power consumption.

I guess this means that by changing the driver policy from the default, we can save some power?

Yes. Maybe 'work around' was a poor choice of words. 'Getting around default driver policy' is the issue. There is no hardware defect. One of the uses of the suspend-to-idle flow is to support compliance with increasingly tighter energy regulations. One of the ways to do this on desktop systems is to power off the ATX power supply during s2idle and use the 5V standby rail for self refresh and other low power needs. But the platforms that support this can't shutdown the PS unless PCI ports are placed in D3. On Linux this won't happen with NVMe drives because the default driver policy is to use ASPM (NVMe APST) during s2idle. Windows has a related concern. So to 'get around' the driver choosing a policy that will result in higher power consumption, they implemented this _DSD to inform the OS of its preference for D3 on the PCI port.

David

#39 lore raw

PATCH v2 2/2 drivers/nvme: Add support for ACPI StorageD3Enable property

In reply to David E. Box

This patch implements a solution for a BIOS hack used on some currently shipping Intel systems to address issues with power management policy decisions concerning PCIe NVMe drives. Some newer Intel platforms, like some Comet Lake systems, require that PCIe devices use D3 when doing suspend-to-idle in order to allow the platform to realize maximum power savings. This is particularly needed to support ATX power supply shutdown on desktop systems. In order to ensure this happens for root ports with storage devices, Microsoft apparently created this ACPI _DSD property as a way to override their driver policy. To my knowledge this property has not been discussed with the NVME specification body.

Though the solution is not ideal, it addresses a problem that also affects Linux since the NVMe driver's default policy of using NVMe APST during suspend-to-idle would lead to higher power consumption for these platforms.

The patch uses the previously added pci_acpi_storage_d3 function to check for the StorageD3Enable property during probe and enables D3 as a quirk if set. It also provides a 'noacpi' module parameter to allow skipping the quirk if needed.

Tested on:
PM961 NVMe SED Samsung 512GB
INTEL SSDPEKKF512G8

---
 drivers/nvme/host/pci.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
drivers/nvme/host/pci.c +14 -0
index 4e79e412b276..427505c47e79 100644
@@ -78,6 +78,10 @@ static unsigned int poll_queues;
 module_param(poll_queues, uint, 0644);
 MODULE_PARM_DESC(poll_queues, "Number of queues to use for polled IO.");
 
+static bool noacpi;
+module_param(noacpi, bool, 0444);
+MODULE_PARM_DESC(noacpi, "disable all acpi bios quirks");
+
 struct nvme_dev;
 struct nvme_queue;
 
@@ -2777,6 +2781,16 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 
 	quirks |= check_vendor_combination_bug(pdev);
 
+	if (!noacpi && pci_acpi_storage_d3(pdev)) {
+		/*
+		 * Some systems use a bios work around to ask for D3 on
+		 * platforms that support kernel managed suspend.
+		 */
+		dev_info(&pdev->dev,
+			 "platform quirk: setting simple suspend\n");
+		quirks |= NVME_QUIRK_SIMPLE_SUSPEND;
+	}
+
 	/*
 	 * Double check that our mempool alloc size will cover the biggest
 	 * command we support.
Signature · 1 line
2.20.1