nvme APST fixes
cover nvme APST fixes
This should fix all known regressions.
Andy Lutomirski (2):
nvme: Wait at least 6000ms before entering the deepest idle state
nvme: Quirk APST on Intel 600P/P3100 devices
drivers/nvme/host/core.c | 38 +++++++++++++++++++++++++++++++------- drivers/nvme/host/pci.c | 2 ++ 2 files changed, 33 insertions(+), 7 deletions(-)
Signature · 1 line
2.9.4
PATCH 1/2 nvme: Wait at least 6000ms before entering the deepest idle state
This should at least make vendors less nervous about Linux's APST policy. I'm not aware of any concrete bugs it would fix (although I was hoping it would fix the Samsung/Dell quirk).
--- drivers/nvme/host/core.c | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-)
drivers/nvme/host/core.c +31 -7
@@ -1325,13 +1325,7 @@ static void nvme_configure_apst(struct nvme_ctrl *ctrl)/* * APST (Autonomous Power State Transition) lets us program a * table of power state transitions that the controller will -* perform automatically. We configure it with a simple -* heuristic: we are willing to spend at most 2% of the time -* transitioning between power states. Therefore, when running -* in any given state, we will enter the next lower-power -* non-operational state after waiting 50 * (enlat + exlat) -* microseconds, as long as that state's total latency is under -* the requested maximum latency. +* perform automatically. * * We will not autonomously enter any non-operational state for * which the total latency exceeds ps_max_latency_us. Users
@@ -1405,9 +1399,39 @@ static void nvme_configure_apst(struct nvme_ctrl *ctrl)/* * This state is good. Use it as the APST idle * target for higher power states. +* +* Intel RSTe supposedly uses the following algorithm: +* 60ms delay to transition to the first +* non-operational state and 1000*exlat to each +* additional state. This is problematic. 60ms is +* too short if the first non-operational state has +* high latency, and 1000*exlat into a state is +* absurdly slow. (exlat=22ms seems typical for the +* deepest state. A delay of 22 seconds to enter that +* state means that it will almost never be entered at +* all, wasting power and, worse, turning otherwise +* easy-to-detect hardware/firmware bugs into sporadic +* problems. +* +* Linux is willing to spend at most 2% of the time +* transitioning between power states. Therefore, +* when running in any given state, we will enter the +* next lower-power non-operational state after +* waiting 50 * (enlat + exlat) microseconds, as long +* as that state's total latency is under the +* requested maximum latency. */ transition_ms = total_latency_us +19 ;do_div (transition_ms,20 ); + +/* +* Some vendors have expressed nervousness about +* entering the deepest state after less than six +* seconds. +*/ +if (state == ctrl->npss && transition_ms <6000 ) + transition_ms =6000 ; +if (transition_ms > (1 <<24 ) -1 ) transition_ms = (1 <<24 ) -1 ;
Signature · 1 line
2.9.4
Christoph Hellwig
maintainerRe: [PATCH 1/2] nvme: Wait at least 6000ms before entering the deepest idle state
Quoted 5 lines · Andy Lutomirski
This should at least make vendors less nervous about Linux's APST policy. I'm not aware of any concrete bugs it would fix (although I was hoping it would fix the Samsung/Dell quirk).
Cc stable at vger.kernel.org # v4.11
Just following a somewhat odd and arbitray policy from another driver that doesn't fix anything by itself certainly isn't stable material.
And to be honest I'm not sure what applying it will gain us at all.
Re: [PATCH 1/2] nvme: Wait at least 6000ms before entering the deepest idle state
Quoted 9 lines · Christoph Hellwig
Quoted 5 lines · Andy Lutomirski
This should at least make vendors less nervous about Linux's APST policy. I'm not aware of any concrete bugs it would fix (although I was hoping it would fix the Samsung/Dell quirk).
Cc stable at vger.kernel.org # v4.11Just following a somewhat odd and arbitray policy from another driver that doesn't fix anything by itself certainly isn't stable material.
Fair enough. The Cc: stable should certainly be dropped.
Quoted 2 lines
And to be honest I'm not sure what applying it will gain us at all.
I'd be fine with skipping this patch entirely at least until we find some evidence that it solves a problem instead.
Re: [PATCH 1/2] nvme: Wait at least 6000ms before entering the deepest idle state
Quoted 7 lines · Andy Lutomirski
Quoted 3 lines · Christoph Hellwig
Just following a somewhat odd and arbitray policy from another driver that doesn't fix anything by itself certainly isn't stable material.
I'd be fine with skipping this patch entirely at least until we find some evidence that it solves a problem instead.
It's certainly worth waiting for confirmation that it actually changes anything, but I did want to pipe up to say that "following a somewhat odd and arbitrary policy from another driver" is not wrong per se.
In fact, on the power management we pretty much had to do that, simply because that "odd and arbitrary policy" (Windows behavior) was the only thing that had ever been tested by anybody. Same goes for a lot of PCI subsystem behavior etc.
So it's not wrong per se to just emulate others' behaviors. But yes, we should have some reason for doing so, not just doing so blindly.
Linus
PATCH 2/2 nvme: Quirk APST on Intel 600P/P3100 devices
In reply to Andy LutomirskiThey have known firmware bugs. A fix is apparently in the works -- once fixed firmware is available, someone from Intel (Hi, Keith!) can adjust the quirk accordingly.
--- drivers/nvme/host/pci.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/nvme/host/pci.c +2 -0
@@ -2288,6 +2288,8 @@ static const struct pci_device_id nvme_id_table[] = { { PCI_VDEVICE(INTEL, 0x0a54), .driver_data =NVME_QUIRK_STRIPE_SIZE | NVME_QUIRK_DEALLOCATE_ZEROES, }, + { PCI_VDEVICE(INTEL, 0xf1a5), /* Intel 600P/P3100 */ + .driver_data = NVME_QUIRK_NO_DEEPEST_PS }, { PCI_VDEVICE(INTEL, 0x5845), /* Qemu emulated controller */ .driver_data = NVME_QUIRK_IDENTIFY_CNS, }, { PCI_DEVICE(0x1c58, 0x0003), /* HGST adapter */
Signature · 1 line
2.9.4
Re: [PATCH 2/2] nvme: Quirk APST on Intel 600P/P3100 devices
Thanks,
applied to nvme-4.12
Re: [PATCH 2/2] nvme: Quirk APST on Intel 600P/P3100 devices
In reply to Andy LutomirskiQuoted 3 lines · Andy Lutomirski
They have known firmware bugs. A fix is apparently in the works -- once fixed firmware is available, someone from Intel (Hi, Keith!) can adjust the quirk accordingly.
Here's the latest firmware with all the known fixes:
https://downloadcenter.intel.com/download/26491?v=t
I am not 100% sure if this fixes the issue you're quirking, but I think it falls under the issues the link initialization and ASPM comments in the release notes.
Re: [PATCH 2/2] nvme: Quirk APST on Intel 600P/P3100 devices
Quoted 5 lines · Keith Busch
Quoted 3 lines · Andy Lutomirski
They have known firmware bugs. A fix is apparently in the works -- once fixed firmware is available, someone from Intel (Hi, Keith!) can adjust the quirk accordingly.
Quoted 7 lines
Here's the latest firmware with all the known fixes:
https://downloadcenter.intel.com/download/26491?v=t
I am not 100% sure if this fixes the issue you're quirking, but I think it falls under the issues the link initialization and ASPM comments in the release notes.
Can you or someone at Intel supply a reliable way to distinguish a fixed firmware from an unfixed firmware? We have access to the "fr" string, but I don't know what to do with it.
PATCH 2/2 nvme: Quirk APST on Intel 600P/P3100 devices
Quoted 10 lines · Andy Lutomirski
Quoted 9 lines · Keith Busch
Quoted 3 lines · Andy Lutomirski
They have known firmware bugs. A fix is apparently in the works -- once fixed firmware is available, someone from Intel (Hi, Keith!) can adjust the quirk accordingly.
Here's the latest firmware with all the known fixes:
Quoted 7 lines
Quoted 3 lines
I am not 100% sure if this fixes the issue you're quirking, but I think it falls under the issues the link initialization and ASPM comments in the release notes.
Can you or someone at Intel supply a reliable way to distinguish a fixed firmware from an unfixed firmware? We have access to the "fr" string, but I don't know what to do with it.
FWIW, on a system with a 600P on the old firmware (109C) running 4.11.3, I went out to lunch and returned to find my system came down with a message something like
Quoted 1 line
controller is down; will reset: CSTS=<???>, PCI_STATUS=0xffff
I updated to the new firmware (121C) and saw it again a few hours later on the same 4.11.3 kernel. So I think it's prudent to quirk regardless of firmware revision.