Wait for device readiness after D3hot -> D0uninitialized transition
v2 cover Wait for device readiness after D3hot -> D0uninitialized transition
In pci_power_up(), wait for device to become ready after a soft reset (D3hot->D0uninitialized transition). We already wait 10 ms, but even after that, the device is permitted to respond to config requests with Request Retry Status (RRS) completion status. Use pci_dev_wait() to wait for successful completion.
Bjorn Helgaas (2):
PCI: Log device readiness timeouts as errors
PCI: Wait for device readiness after D3hot -> D0uninitialized
transition
drivers/pci/pci.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-)
Changes since v1:
- Log device readiness timeouts as errors, not warnings
- On timeout, set dev->current_state to D3cold and return -EIO
PATCH v2 1/2 PCI: Log device readiness timeouts as errors
pci_dev_wait() waits for a device to be Configuration-Ready after a reset, such as a Function-Level Reset (FLR), a soft reset during a D3hot-> D0uninitialized transition when No_Soft_Reset == 0), or a power-up sequence from D3cold->D0uninitialized.
If pci_dev_wait() returns success, the device is guaranteed to respond to configuration requests with Successful Completion status. If it times out, device is completely non-responsive.
Upgrade the log level from pci_warn() to pci_err() to reflect this failure state.
--- drivers/pci/pci.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/pci/pci.c +2 -2
@@ -1253,8 +1253,8 @@ static int pci_dev_wait(struct pci_dev *dev, char *reset_type, int timeout) }if (delay > timeout) { -pci_warn (dev,"not ready %dms after %s; giving up\n" , - delay -1 , reset_type); +pci_err (dev,"not ready %dms after %s; giving up\n" , + delay -1 , reset_type);return -ENOTTY ; }
Signature · 1 line
2.51.0
PATCH v2 2/2 PCI: Wait for device readiness after D3hot -> D0uninitialized transition
In reply to Bjorn HelgaasFrom: Bjorn Helgaas <[email protected]>
For a device that advertises No_Soft_Reset == 0, a transition from D3hot to D0uninitialized is a soft reset, and the resulting internal device state is undefined.
Per PCIe r7.0, sec 2.3.1, a transition from D3hot to D0uninitialized mandates a minimum 10 ms delay before accessing the device. Following this delay, the device is permitted to respond to initial configuration requests with a Request Retry Status (RRS) completion status if it needs more time to initialize.
Call pci_dev_wait() after pci_power_up() performs a D3hot->D0uninitialized transition to ensure the device is ready to accept config accesses, as is done after the similar transition in pci_pm_reset().
If the device is already ready, this is essentially a no-op except for one additional config read.
--- drivers/pci/pci.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-)
drivers/pci/pci.c +22 -2
@@ -1300,7 +1300,18 @@ int pci_power_up(struct pci_dev *dev)bool need_restore;pci_power_t state;u16 pmcsr; +int ret; +/* +* When setting power state to D0, platform_pci_set_power_state() +* ensures main power is on. If it puts the device in D0, it also +* completes any required delays after the transition; if it leaves +* the device in D1, D2, or D3hot, we use the PM Capability to +* transition to D0. +* +* In all cases, the device is either Configuration-Ready or +* inaccessible upon return. +*/ platform_pci_set_power_state (dev,PCI_D0 );if (!dev->pm_cap) {
@@ -1341,10 +1352,19 @@ int pci_power_up(struct pci_dev *dev)pci_write_config_word (dev, dev->pm_cap +PCI_PM_CTRL ,0 );/* Mandatory transition delays; see PCI PM 1.2. */ -if (state == PCI_D3hot) +if (state == PCI_D3hot) {pci_dev_d3_sleep (dev); -else if (state ==PCI_D2 ) +if (!(pmcsr &PCI_PM_CTRL_NO_SOFT_RESET )) { + ret =pci_dev_wait (dev,"power up D3hot->D0uninitialized" , +PCIE_RESET_READY_POLL_MS ); +if (ret) { + dev->current_state = PCI_D3cold; +return -EIO ; + } + } + }else if (state ==PCI_D2 ) {udelay (PCI_PM_D2_DELAY ); + } end: dev->current_state =PCI_D0 ;
Signature · 1 line
2.51.0
Re: [PATCH v2 2/2] PCI: Wait for device readiness after D3hot -> D0uninitialized transition
Quoted 49 lines · Bjorn Helgaas
From: Bjorn Helgaas <[email protected]>
For a device that advertises
No_Soft_Reset== 0, a transition from D3hot to D0uninitialized is a soft reset, and the resulting internal device state is undefined.Per PCIe r7.0, sec 2.3.1, a transition from D3hot to D0uninitialized mandates a minimum 10 ms delay before accessing the device. Following this delay, the device is permitted to respond to initial configuration requests with a Request Retry Status (RRS) completion status if it needs more time to initialize.
Call
pci_dev_wait()afterpci_power_up()performs aD3hot->D0uninitializedtransition to ensure the device is ready to accept config accesses, as is done after the similar transition inpci_pm_reset().If the device is already ready, this is essentially a no-op except for one additional config read.
Signed-off-by Bjorn Helgaas <[email protected]>--- drivers/pci/pci.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-)drivers/pci/pci.c +11
-0@@ -1300,7 +1300,18 @@ int pci_power_up(struct pci_dev *dev)bool need_restore;pci_power_t state;u16 pmcsr; +int ret; +/* +* When setting power state to D0, platform_pci_set_power_state() +* ensures main power is on. If it puts the device in D0, it also +* completes any required delays after the transition; if it leaves +* the device in D1, D2, or D3hot, we use the PM Capability to +* transition to D0. +* +* In all cases, the device is either Configuration-Ready or +* inaccessible upon return. +*/ platform_pci_set_power_state (dev,PCI_D0 );if (!dev->pm_cap) {
drivers/pci/pci.c int pci_power_up(struct pci_dev *dev) · line 1357
Quoted 8 lines
drivers/pci/pci.c +2
-2@@ -1341,10 +1352,19 @@ int pci_power_up(struct pci_dev *dev)pci_write_config_word (dev, dev->pm_cap +PCI_PM_CTRL ,0 );/* Mandatory transition delays; see PCI PM 1.2. */ -if (state == PCI_D3hot) +if (state == PCI_D3hot) {pci_dev_d3_sleep (dev); -else if (state ==PCI_D2 ) +if (!(pmcsr &PCI_PM_CTRL_NO_SOFT_RESET )) {
Hmm. The pmcsr value here is from before the pci_write_config_word(), so AFAICS it should be updated after the write.
This means an additional config space access and I wonder if it can be added to pci_dev_wait()?
Quoted 14 lines
drivers/pci/pci.c +9
-1+ ret =pci_dev_wait (dev,"power up D3hot->D0uninitialized" , +PCIE_RESET_READY_POLL_MS ); +if (ret) { + dev->current_state = PCI_D3cold; +return -EIO ; + } + } + }else if (state ==PCI_D2 ) {udelay (PCI_PM_D2_DELAY ); + } end: dev->current_state =PCI_D0 ; --
Re: [PATCH v2 2/2] PCI: Wait for device readiness after D3hot -> D0uninitialized transition
Quoted 5 lines · Rafael J. Wysocki
Quoted 4 lines · Bjorn Helgaas
+++
b/drivers/pci/pci.cdrivers/pci/pci.c +0
-0@@ -1341,10 +1352,19 @@ int pci_power_up(struct pci_dev *dev)pci_write_config_word (dev, dev->pm_cap +PCI_PM_CTRL ,0 );
Quoted 9 lines
drivers/pci/pci.c int pci_power_up(struct pci_dev *dev) · line 1357
Quoted 6 lines
drivers/pci/pci.c +2
-2/* Mandatory transition delays; see PCI PM 1.2. */ -if (state == PCI_D3hot) +if (state == PCI_D3hot) {pci_dev_d3_sleep (dev); -else if (state ==PCI_D2 ) +if (!(pmcsr &PCI_PM_CTRL_NO_SOFT_RESET )) {Hmm. The pmcsr value here is from before the
pci_write_config_word(), so AFAICS it should be updated after the write.
The No_Soft_Reset bit is RO (PCIe r7.0 sec 7.5.2.2), so its value should never change when writing zero to the Power Management Control/Status Register.
Thanks,
Lukas
Re: [PATCH v2 2/2] PCI: Wait for device readiness after D3hot -> D0uninitialized transition
Quoted 17 lines · Lukas Wunner
Quoted 5 lines · Rafael J. Wysocki
Quoted 4 lines · Bjorn Helgaas
+++
b/drivers/pci/pci.cdrivers/pci/pci.c +0
-0@@ -1341,10 +1352,19 @@ int pci_power_up(struct pci_dev *dev)pci_write_config_word (dev, dev->pm_cap +PCI_PM_CTRL ,0 );Quoted 9 lines
drivers/pci/pci.c int pci_power_up(struct pci_dev *dev) · line 1357
Quoted 6 lines
drivers/pci/pci.c +2
-2/* Mandatory transition delays; see PCI PM 1.2. */ -if (state == PCI_D3hot) +if (state == PCI_D3hot) {pci_dev_d3_sleep (dev); -else if (state ==PCI_D2 ) +if (!(pmcsr &PCI_PM_CTRL_NO_SOFT_RESET )) {Hmm. The pmcsr value here is from before the
pci_write_config_word(), so AFAICS it should be updated after the write.
Quoted 3 lines
The
No_Soft_Resetbit is RO (PCIe r7.0 sec 7.5.2.2), so its value should never change when writing zero to the Power Management Control/Status Register.
Oh right, I confused it with something else.
Quoted 16 lines · Bjorn Helgaas
In
pci_power_up(), wait for device to become ready after a soft reset (D3hot->D0uninitializedtransition). We already wait 10 ms, but even after that, the device is permitted to respond to config requests with Request Retry Status (RRS) completion status. Usepci_dev_wait()to wait for successful completion.Bjorn Helgaas (2):
PCI: Log device readiness timeouts as errors
PCI: Wait for device readiness after D3hot -> D0uninitialized
transitiondrivers/pci/pci.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-)
Quoted 5 lines
Changes since v1:
- Log device readiness timeouts as errors, not warnings
- On timeout, setdev->current_stateto D3cold and return-EIO
For the series:
Bjorn Helgaas
AppliedQuoted 15 lines · Bjorn Helgaas
In
pci_power_up(), wait for device to become ready after a soft reset (D3hot->D0uninitializedtransition). We already wait 10 ms, but even after that, the device is permitted to respond to config requests with Request Retry Status (RRS) completion status. Usepci_dev_wait()to wait for successful completion.Bjorn Helgaas (2):
PCI: Log device readiness timeouts as errors
PCI: Wait for device readiness after D3hot -> D0uninitialized
transitiondrivers/pci/pci.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-)
Quoted 5 lines
Changes since v1:
- Log device readiness timeouts as errors, not warnings
- On timeout, setdev->current_stateto D3cold and return-EIO
I applied these to pci/reset for v7.2.