nvme/pci: default to simple suspend
Keith Busch
maintainerPATCH 1/1 nvme/pci: default to simple suspend
There is no complete set of attributes a driver can check to know if nvme power management is the correct thing to do in a runtime suspend situation. Every attempt so far to optimize idle power consumption and resume latency for a particular platform just leads to regressions elsewhere.
Default to the simple shutdown since it is the historically safest option, and provide a user parameter to override it if the user knows it's safe to use for their platform.
--- drivers/nvme/host/pci.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/nvme/host/pci.c +6 -1
@@ -103,6 +103,10 @@ static bool noacpi;module_param (noacpi, bool,0444 );MODULE_PARM_DESC (noacpi,"disable acpi bios quirks" ); +static bool pwr_mgmt; +module_param (pwr_mgmt, bool,0444 ); +MODULE_PARM_DESC (pwr_mgmt,"use nvme power management for runtime suspend" ); +struct nvme_dev ;struct nvme_queue ;
@@ -3094,7 +3098,8 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)dev_info (&pdev->dev,"platform quirk: setting simple suspend\n" ); quirks |=NVME_QUIRK_SIMPLE_SUSPEND ; - } + }else if (!pwr_mgmt) + quirks |=NVME_QUIRK_SIMPLE_SUSPEND ;/* * Double check that our mempool alloc size will cover the biggest
Signature · 1 line
2.25.4
Tested on Tegra194 platform with a Phison E12 NVMe drive.
Quoted 49 lines · Keith Busch
External email: Use caution opening links or attachments
There is no complete set of attributes a driver can check to know if nvme power management is the correct thing to do in a runtime suspend situation. Every attempt so far to optimize idle power consumption and resume latency for a particular platform just leads to regressions elsewhere.
Default to the simple shutdown since it is the historically safest option, and provide a user parameter to override it if the user knows it's safe to use for their platform.
Cc Rafael Wysocki <[email protected]>Cc Vidya Sagar <[email protected]>Signed-off-by Keith Busch <[email protected]>--- drivers/nvme/host/pci.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)drivers/nvme/host/pci.c +6
-2@@ -103,6 +103,10 @@ static bool noacpi;module_param (noacpi, bool,0444 );MODULE_PARM_DESC (noacpi,"disable acpi bios quirks" ); +static bool pwr_mgmt; +module_param (pwr_mgmt, bool,0444 ); +MODULE_PARM_DESC (pwr_mgmt,"use nvme power management for runtime suspend" ); +struct nvme_dev ;struct nvme_queue ;@@ -3094,7 +3098,8 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)dev_info (&pdev->dev,"platform quirk: setting simple suspend\n" ); quirks |=NVME_QUIRK_SIMPLE_SUSPEND ; - } + }else if (!pwr_mgmt) + quirks |=NVME_QUIRK_SIMPLE_SUSPEND ;/* * Double check that our mempool alloc size will cover the biggest -- 2.25.4
Christoph Hellwig
maintainerQuoted 9 lines · Keith Busch
There is no complete set of attributes a driver can check to know if nvme power management is the correct thing to do in a runtime suspend situation. Every attempt so far to optimize idle power consumption and resume latency for a particular platform just leads to regressions elsewhere.
Default to the simple shutdown since it is the historically safest option, and provide a user parameter to override it if the user knows it's safe to use for their platform.
Sigh. The platforms really should be asking for a explicit D3cold if they need one..
Keith Busch
maintainerQuoted 7 lines · Christoph Hellwig
Quoted 6 lines · Keith Busch
There is no complete set of attributes a driver can check to know if nvme power management is the correct thing to do in a runtime suspend situation. Every attempt so far to optimize idle power consumption and resume latency for a particular platform just leads to regressions elsewhere.
Quoted 6 lines
Quoted 3 lines
Default to the simple shutdown since it is the historically safest option, and provide a user parameter to override it if the user knows it's safe to use for their platform.
Sigh. The platforms really should be asking for a explicit D3cold if they need one..
It's too late now, but perhaps the new property should have been inverted since preparing for D3 was the previous default behavior; platforms could have instead explicitly asked for "no-D3" if they wanted it. That would have been easier for backward compatibility.
Christoph Hellwig
maintainerQuoted 12 lines · Keith Busch
Quoted 11 lines · Christoph Hellwig
Quoted 9 lines · Keith Busch
There is no complete set of attributes a driver can check to know if nvme power management is the correct thing to do in a runtime suspend situation. Every attempt so far to optimize idle power consumption and resume latency for a particular platform just leads to regressions elsewhere.
Default to the simple shutdown since it is the historically safest option, and provide a user parameter to override it if the user knows it's safe to use for their platform.
Quoted 7 lines
Quoted 2 lines
Sigh. The platforms really should be asking for a explicit D3cold if they need one..
It's too late now, but perhaps the new property should have been inverted since preparing for D3 was the previous default behavior; platforms could have instead explicitly asked for "no-D3" if they wanted it. That would have been easier for backward compatibility.
I'd really prefer to sort this out at the platform level. We can't work around broken platforms in nvme forever.
Keith Busch
maintainerQuoted 16 lines · Christoph Hellwig
Quoted 15 lines · Keith Busch
Quoted 13 lines · Christoph Hellwig
Quoted 9 lines · Keith Busch
There is no complete set of attributes a driver can check to know if nvme power management is the correct thing to do in a runtime suspend situation. Every attempt so far to optimize idle power consumption and resume latency for a particular platform just leads to regressions elsewhere.
Default to the simple shutdown since it is the historically safest option, and provide a user parameter to override it if the user knows it's safe to use for their platform.
Sigh. The platforms really should be asking for a explicit D3cold if they need one..
Quoted 7 lines
Quoted 4 lines
It's too late now, but perhaps the new property should have been inverted since preparing for D3 was the previous default behavior; platforms could have instead explicitly asked for "no-D3" if they wanted it. That would have been easier for backward compatibility.
I'd really prefer to sort this out at the platform level. We can't work around broken platforms in nvme forever.
I agree, but I'm not sure how to get everyone aligned.
How about this to resolve the regressions: if the platform doesn't provide StorageD3Enable property, can we just default to the simple shutdown method? We'd only use the nvme power management capabilities if the platform explicity says it doesn't want D3, making the default the same as the legacy behavior.
---
drivers/nvme/host/pci.c +4 -1
@@ -20,6 +20,7 @@#include <linux/mutex.h> #include <linux/once.h> #include <linux/pci.h> +#include <linux/property.h> #include <linux/suspend.h> #include <linux/t10-pi.h> #include <linux/types.h>
@@ -3086,7 +3087,9 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id) quirks |=check_vendor_combination_bug (pdev); -if (!noacpi &&acpi_storage_d3 (&pdev->dev)) { +if (!noacpi && (acpi_storage_d3 (&pdev->dev) || + !device_property_present (&pdev->dev, +"StorageD3Enable" ))) {/* * Some systems use a bios work around to ask for D3 on * platforms that support kernel managed suspend.
--
Christoph Hellwig
maintainerQuoted 10 lines · Keith Busch
Quoted 2 lines
I'd really prefer to sort this out at the platform level. We can't work around broken platforms in nvme forever.
I agree, but I'm not sure how to get everyone aligned.
How about this to resolve the regressions: if the platform doesn't provide StorageD3Enable property, can we just default to the simple shutdown method? We'd only use the nvme power management capabilities if the platform explicity says it doesn't want D3, making the default the same as the legacy behavior.
I don't think this will work, as most older platforms just won't have that attribute at all, and non-ACPI platforms most certainly won't have it. We'll need more quirks in the core PCI/PM code like we did for the AMD mobile platforms.
Keith Busch
maintainerQuoted 12 lines · Christoph Hellwig
Quoted 10 lines · Keith Busch
Quoted 2 lines
I'd really prefer to sort this out at the platform level. We can't work around broken platforms in nvme forever.
I agree, but I'm not sure how to get everyone aligned.
How about this to resolve the regressions: if the platform doesn't provide StorageD3Enable property, can we just default to the simple shutdown method? We'd only use the nvme power management capabilities if the platform explicity says it doesn't want D3, making the default the same as the legacy behavior.
Quoted 4 lines
I don't think this will work, as most older platforms just won't have that attribute at all, and non-ACPI platforms most certainly won't have it. We'll need more quirks in the core PCI/PM code like we did for the AMD mobile platforms.
old platform + old kernel -> simple suspend
old platform + new kernel -> nvme pm
This inevitably introduced regressions. Why should we need to quirk old platforms for a feature that didn't even exist when they were made? Shouldn't the quirk be the other way around?
Christoph Hellwig
maintainerQuoted 5 lines · Keith Busch
Quoted 4 lines
I don't think this will work, as most older platforms just won't have that attribute at all, and non-ACPI platforms most certainly won't have it. We'll need more quirks in the core PCI/PM code like we did for the AMD mobile platforms.
Quoted 7 lines
old platform + old kernel -> simple suspend
old platform + new kernel -> nvme pm
This inevitably introduced regressions. Why should we need to quirk old platforms for a feature that didn't even exist when they were made? Shouldn't the quirk be the other way around?
Because they are broken? If we do get a suspend request that does not explicitly asks for the device do be disabled we need to be able to rely on that. Where we is not just nvme, but any driver.
Hi Christoph,
Quoted 6 lines · Keith Busch
There is no complete set of attributes a driver can check to know if nvme power management is the correct thing to do in a runtime suspend situation. Every attempt so far to optimize idle power consumption and resume latency for a particular platform just leads to regressions elsewhere.
Quoted 8 lines
Default to the simple shutdown since it is the historically safest option, and provide a user parameter to override it if the user knows it's safe to use for their platform.
Cc Rafael Wysocki <[email protected]>Cc Vidya Sagar <[email protected]>Signed-off-by Keith Busch <[email protected]>
I was looking into the nvme simple shutdown path for the Qualcomm platforms and it looks to me that this patch from Keith is the way to go. I do agree with your resistance on adding the platform level support to the nvme driver, but there is no good way of working around it in the platform level.
PCI core only accepts the quirks for the host devices that could be passed onto the PCI device drivers like this one. In this case, this is not a quirk but actually an aggressive power saving feature (atleast on the Qcom platforms). Moreover, adding a flag to the PCI bus will make it applicable to all the child devices of the RC/bridge and that would be wrong.
In our case, the same power saving feature is not applicable to all PCI devices like WLAN for an example.
Then if we try to add the platform specific support in nvme driver using the of_machine_is_compatible() API, it will get piled up over the time. So just using the simple shutdown as the default suspend choice and using the module parameter to override it makes much sense to me.
Please let me know what you think.
Thanks,
Mani
Quoted 31 lines
--- drivers/nvme/host/pci.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)drivers/nvme/host/pci.c +6
-1@@ -103,6 +103,10 @@ static bool noacpi;module_param (noacpi, bool,0444 );MODULE_PARM_DESC (noacpi,"disable acpi bios quirks" ); +static bool pwr_mgmt; +module_param (pwr_mgmt, bool,0444 ); +MODULE_PARM_DESC (pwr_mgmt,"use nvme power management for runtime suspend" ); +struct nvme_dev ;struct nvme_queue ;@@ -3094,7 +3098,8 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)dev_info (&pdev->dev,"platform quirk: setting simple suspend\n" ); quirks |=NVME_QUIRK_SIMPLE_SUSPEND ; - } + }else if (!pwr_mgmt) + quirks |=NVME_QUIRK_SIMPLE_SUSPEND ;/* * Double check that our mempool alloc size will cover the biggest Signature · 1 line
2.25.4
Christoph Hellwig
maintainerQuoted 5 lines · Manivannan Sadhasivam
PCI core only accepts the quirks for the host devices that could be passed onto the PCI device drivers like this one. In this case, this is not a quirk but actually an aggressive power saving feature (atleast on the Qcom platforms). Moreover, adding a flag to the PCI bus will make it applicable to all the child devices of the RC/bridge and that would be wrong.
As you correctly state it is not a device quirk. It describes the power management applied by the platform. So we do need to communicate it through the core PM and/or PCI code. Please work with the relevant maintainers.
Quoted 2 lines
In our case, the same power saving feature is not applicable to all PCI devices like WLAN for an example.
This doesn't make sense. Your plaform can't know what device is connected to a given root port / slot. So you might have different policies per slot, but that has nothing to do with the Linux drivers for given devices.