Disabling ASPM L1 sub-states selectively from drivers?
I face the issue that a PCIe network chip misses RX packets if ASPM L1 sub-states are enabled. It seems that the RX FIFO is too small to buffer all incoming packets during ASPM exit latency.
So far pci_disable_link_state() only allows to disable L1 completely. Would it make sense to extend this function to allow disabling L1 sub-states selectively? Looking at pcie_config_aspm_link() this seems to be possible.
Bjorn Helgaas
maintainerHi Heiner,
Quoted 9 lines · Heiner Kallweit
I face the issue that a PCIe network chip misses RX packets if ASPM L1 sub-states are enabled. It seems that the RX FIFO is too small to buffer all incoming packets during ASPM exit latency.
So far
pci_disable_link_state()only allows to disable L1 completely. Would it make sense to extend this function to allow disabling L1 sub-states selectively? Looking atpcie_config_aspm_link()this seems to be possible.
We could certainly explore the option of selectively disabling L1 substates.
But before we do that, let's look at a couple things, because there are some Linux issues in that area, and it's possible we could make a generic fix that wouldn't require disabling the substates completely.
One problem is the ASPM L1.2 state depends on LTR information, and we don't support LTR correctly. There are a couple patches in -next to fix some problems, but we still don't handle cases where the BIOS doesn't program the LTR latencies and the LTR_L1.2_Threshold.
Can you open a report at bugzilla.kernel.org and attach the complete dmesg log (booted with "pci=earlydump") and the "sudo lspci -vvvxx" output?
Have you figured out which L1 substate specifically causes problems? If not, maybe we can use setpci to fiddle with things manually and narrow it down.
Bjorn
Quoted 19 lines · Bjorn Helgaas
Hi Heiner,
Quoted 9 lines · Heiner Kallweit
I face the issue that a PCIe network chip misses RX packets if ASPM L1 sub-states are enabled. It seems that the RX FIFO is too small to buffer all incoming packets during ASPM exit latency.
So far
pci_disable_link_state()only allows to disable L1 completely. Would it make sense to extend this function to allow disabling L1 sub-states selectively? Looking atpcie_config_aspm_link()this seems to be possible.We could certainly explore the option of selectively disabling L1 substates.
But before we do that, let's look at a couple things, because there are some Linux issues in that area, and it's possible we could make a generic fix that wouldn't require disabling the substates completely.
Quoted 9 lines
One problem is the ASPM L1.2 state depends on LTR information, and we don't support LTR correctly. There are a couple patches in -next to fix some problems, but we still don't handle cases where the BIOS doesn't program the LTR latencies and the LTR_L1.2_Threshold.
Can you open a report at bugzilla.kernel.org and attach the complete dmesg log (booted with "pci=earlydump") and the "sudo lspci -vvvxx" output?
I have no system where I can reproduce the issue, it was reported by a user: https://bugzilla.redhat.com/show_bug.cgi?id=1671958
In comments 61 and 65 you find the lspci -vv output for the network chip.
Quoted 4 lines
Have you figured out which L1 substate specifically causes problems? If not, maybe we can use setpci to fiddle with things manually and narrow it down.
So far all I can say is that with ASPM disabled completely the issue doesn't occur. I'd have to see how to disable L1.2 only with setpci and then let the user test.
Quoted 2 lines
Bjorn
Heiner