pci: Reenable the AMD IOMMU if it's mysteriously vanished over suspend
PATCH 1/1 pci: Reenable the AMD IOMMU if it's mysteriously vanished over suspend
AMD's reference BIOS code had a bug that could result in the firmware failing to reenable the iommu on resume. It transpires that this causes certain less than desirable behaviour when it comes to PCI accesses, to whit them ending up somewhere near Bristol when the more desirable outcome was Edinburgh. Sadness ensues, perhaps along with filesystem corruption. Let's make sure that it gets turned back on.
--- drivers/pci/quirks.c | 33 +++++++++++++++++++++++++++++++++ 1 files changed, 33 insertions(+), 0 deletions(-)
drivers/pci/quirks.c +33 -0
@@ -2695,6 +2695,39 @@ DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_RICOH, PCI_DEVICE_ID_RICOH_R5C832,#endif /*CONFIG_MMC_RICOH_MMC*/ +#ifdef CONFIG_AMD_IOMMU + +/* Some AMD BIOSes fail to reenable the iommu on resume */ + +static void amd_iommu_reenable (struct pci_dev *dev) +{ +u32 ioc_feature_control; +struct pci_dev *iommu; + + iommu =pci_get_device (PCI_VENDOR_ID_ATI ,0x5a23 ,NULL ); + +if (!iommu) +return ; + +/* Select Northbridge indirect register 0x75 and enable writing */ +pci_write_config_dword (dev,0x60 ,0x75 | (1 <<7 )); +pci_read_config_dword (dev,0x64 , &ioc_feature_control); + +/* Enable the iommu if it's vanished */ +if (!(ioc_feature_control &0x1 )) +pci_write_config_dword (dev,0x64 , ioc_feature_control |1 ); + +/* Disable writing again */ +pci_write_config_dword (dev,0x60 ,0x75 ); + +pci_dev_put (iommu); +} + +DECLARE_PCI_FIXUP_RESUME_EARLY (PCI_VENDOR_ID_ATI ,0x5a10 , amd_iommu_reenable); +DECLARE_PCI_FIXUP_RESUME_EARLY (PCI_VENDOR_ID_ATI ,0x5a12 , amd_iommu_reenable); +DECLARE_PCI_FIXUP_RESUME_EARLY (PCI_VENDOR_ID_ATI ,0x5a13 , amd_iommu_reenable); +#endif +static void pci_do_fixups (struct pci_dev *dev,struct pci_fixup *f,struct pci_fixup *end) {
Signature · 1 line
1.7.3
Quoted 6 lines · Matthew Garrett
AMD's reference BIOS code had a bug that could result in the firmware failing to reenable the iommu on resume. It transpires that this causes certain less than desirable behaviour when it comes to PCI accesses, to whit them ending up somewhere near Bristol when the more desirable outcome was Edinburgh. Sadness ensues, perhaps along with filesystem corruption. Let's make sure that it gets turned back on.
Of course, this isn't actually sufficient - nothing in the iommu code appears to reprogram the BAR, for instance. Joerg, how is this meant to work?
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 11 lines · Matthew Garrett
Quoted 6 lines · Matthew Garrett
AMD's reference BIOS code had a bug that could result in the firmware failing to reenable the iommu on resume. It transpires that this causes certain less than desirable behaviour when it comes to PCI accesses, to whit them ending up somewhere near Bristol when the more desirable outcome was Edinburgh. Sadness ensues, perhaps along with filesystem corruption. Let's make sure that it gets turned back on.
Of course, this isn't actually sufficient - nothing in the iommu code appears to reprogram the BAR, for instance. Joerg, how is this meant to work?
Enabling the IOMMU device is only part of what the BIOS is supposed to do to get the IOMMU ready for the OS. A handful of registers in the config space of the device and the IOMMU MMIO region must also be restored. Thats the simple part. The IOMMU device also spans two indirect register spaces to configure the IOMMU caches. I have no idea yet how this need to be configured. The plan is definitly to work-around a missing IOMMU device in the IOMMU driver. But I am still talking to some people to find out what exactly must be done, especially with the indirect register spaces.
Joerg
Quoted 9 lines · Joerg Roedel
Enabling the IOMMU device is only part of what the BIOS is supposed to do to get the IOMMU ready for the OS. A handful of registers in the config space of the device and the IOMMU MMIO region must also be restored. Thats the simple part. The IOMMU device also spans two indirect register spaces to configure the IOMMU caches. I have no idea yet how this need to be configured. The plan is definitly to work-around a missing IOMMU device in the IOMMU driver. But I am still talking to some people to find out what exactly must be done, especially with the indirect register spaces.
Chapter 8 of the register setup guide seems to have something on this - I'll try saving and restoring all of the setup it performs to see if it improves things.
Signature · 1 line
Matthew Garrett | [email protected]
Quoted 31 lines · Matthew Garrett
AMD's reference BIOS code had a bug that could result in the firmware failing to reenable the iommu on resume. It transpires that this causes certain less than desirable behaviour when it comes to PCI accesses, to whit them ending up somewhere near Bristol when the more desirable outcome was Edinburgh. Sadness ensues, perhaps along with filesystem corruption. Let's make sure that it gets turned back on.
Signed-off-by Matthew Garrett <[email protected]>--- drivers/pci/quirks.c | 33 +++++++++++++++++++++++++++++++++ 1 files changed, 33 insertions(+), 0 deletions(-)drivers/pci/quirks.c +11
-0@@ -2695,6 +2695,39 @@ DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_RICOH, PCI_DEVICE_ID_RICOH_R5C832,#endif /*CONFIG_MMC_RICOH_MMC*/ +#ifdef CONFIG_AMD_IOMMU + +/* Some AMD BIOSes fail to reenable the iommu on resume */ + +static void amd_iommu_reenable (struct pci_dev *dev) +{ +u32 ioc_feature_control; +struct pci_dev *iommu; + + iommu =pci_get_device (PCI_VENDOR_ID_ATI ,0x5a23 ,NULL ); +
drivers/pci/quirks.c DECLARE_PCI_FIXUP_RESUME_EARLY(PCI_VENDOR_ID_RICOH, PCI_DEVICE_ID_RICOH_R5C832, · line 2718
Quoted 10 lines
drivers/pci/quirks.c +10
-0+if (!iommu) +return ; + +/* Select Northbridge indirect register 0x75 and enable writing */ +pci_write_config_dword (dev,0x60 ,0x75 | (1 <<7 )); +pci_read_config_dword (dev,0x64 , &ioc_feature_control); + +/* Enable the iommu if it's vanished */ +if (!(ioc_feature_control &0x1 )) +pci_write_config_dword (dev,0x64 , ioc_feature_control |1 );
This sounds important and tricky enough that it'd be nice to have a note in dmesg about what we're doing.
Quoted 15 lines
drivers/pci/quirks.c +11
-0+/* Disable writing again */ +pci_write_config_dword (dev,0x60 ,0x75 ); + + pci_dev_put(iommu); +} + +DECLARE_PCI_FIXUP_RESUME_EARLY (PCI_VENDOR_ID_ATI ,0x5a10 , amd_iommu_reenable); +DECLARE_PCI_FIXUP_RESUME_EARLY (PCI_VENDOR_ID_ATI ,0x5a12 , amd_iommu_reenable); +DECLARE_PCI_FIXUP_RESUME_EARLY (PCI_VENDOR_ID_ATI ,0x5a13 , amd_iommu_reenable); +#endif +static void pci_do_fixups (struct pci_dev *dev,struct pci_fixup *f,struct pci_fixup *end) {