Skip to contentSkip to conversation

iommu/amd: Fix I/O page fault logging ratelimit test

PATCH In review Lennert Buytenhek 2 participants 3 messages Jul 19, 2021 – Jul 21, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iommu/amd: Fix I/O page fault logging ratelimit test

On an AMD system, I/O page faults are usually logged like this:

	drvname 0000:05:00.0: Event logged [IO_PAGE_FAULT domain=0x0000 address=0x0000000092050da0 flags=0x0020]

But sometimes they are logged like this instead, even for the exact same PCI device:

	AMD-Vi: Event logged [IO_PAGE_FAULT device=05:00.0 domain=0x0000 address=0x0000000092050de0 flags=0x0020]

This discrepancy appears to be caused by this code:

	if (dev_data && __ratelimit(&dev_data->rs)) {
		pci_err(pdev, "Event logged [IO_PAGE_FAULT domain=0x%04x address=0x%llx flags=0x%04x]\n",
			domain_id, address, flags);
	} else if (printk_ratelimit()) {
		pr_err("Event logged [IO_PAGE_FAULT device=%02x:%02x.%x domain=0x%04x address=0x%llx flags=0x%04x]\n",
			PCI_BUS_NUM(devid), PCI_SLOT(devid), PCI_FUNC(devid),
			domain_id, address, flags);
	}

If an I/O page fault occurs for a PCI device with associated iommu_dev_data, but for which the __ratelimit(&dev_data->rs) check fails, we'll give it a second chance with printk_ratelimit(), and if that check succeeds, we will log the fault anyway, but in a different format.

Change this to only check printk_ratelimit() if !dev_data, which seems to be what had been originally intended.

---
 drivers/iommu/amd/iommu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/amd/iommu.c +1 -1
index 811a49a95d04..7ae426b092f2 100644
@@ -483,7 +483,7 @@ static void amd_iommu_report_page_fault(u16 devid, u16 domain_id,
 	if (dev_data && __ratelimit(&dev_data->rs)) {
 		pci_err(pdev, "Event logged [IO_PAGE_FAULT domain=0x%04x address=0x%llx flags=0x%04x]\n",
 			domain_id, address, flags);
-	} else if (printk_ratelimit()) {
+	} else if (!dev_data && printk_ratelimit()) {
 		pr_err("Event logged [IO_PAGE_FAULT device=%02x:%02x.%x domain=0x%04x address=0x%llx flags=0x%04x]\n",
 			PCI_BUS_NUM(devid), PCI_SLOT(devid), PCI_FUNC(devid),
 			domain_id, address, flags);

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#1 lore raw

Hi Lennert,

drivers/iommu/amd/iommu.c static void amd_iommu_report_page_fault(u16 devid, u16 domain_id, · line 486

Quoted 12 lines · Lennert Buytenhek

On an AMD system, I/O page faults are usually logged like this:
....

drivers/iommu/amd/iommu.c +1 -1
index 811a49a95d04..7ae426b092f2 100644
@@ -483,7 +483,7 @@ static void amd_iommu_report_page_fault(u16 devid, u16 domain_id,
  	if (dev_data && __ratelimit(&dev_data->rs)) {
  		pci_err(pdev, "Event logged [IO_PAGE_FAULT domain=0x%04x address=0x%llx flags=0x%04x]\n",
  			domain_id, address, flags);
-	} else if (printk_ratelimit()) {
+	} else if (!dev_data && printk_ratelimit()) {

This seems a bit confusing. Also, according to the following comment in include/linux/printk.h:

/*
* Please don't use printk_ratelimit(), because it shares ratelimiting state
* with all other unrelated printk_ratelimit() callsites. Instead use
* printk_ratelimited() or plain old __ratelimit().
*/

We probably should move away from using printk_ratelimit() here. What about the following change instead?

drivers/iommu/amd/iommu.c +6 -5
index 811a49a95d04..8eb5d3519743 100644
@@ -480,11 +480,12 @@ static void amd_iommu_report_page_fault(u16 devid, u16 domain_id,
        if (pdev)
                dev_data = dev_iommu_priv_get(&pdev->dev);
 
-       if (dev_data && __ratelimit(&dev_data->rs)) {
-               pci_err(pdev, "Event logged [IO_PAGE_FAULT domain=0x%04x address=0x%llx flags=0x%04x]\n",
-                       domain_id, address, flags);
-       } else if (printk_ratelimit()) {
-               pr_err("Event logged [IO_PAGE_FAULT device=%02x:%02x.%x domain=0x%04x address=0x%llx flags=0x%04x]\n",
+       if (dev_data) {
+               if (__ratelimit(&dev_data->rs))
+                       pci_err(pdev, "Event logged [IO_PAGE_FAULT domain=0x%04x address=0x%llx flags=0x%04x]\n",
+                               domain_id, address, flags);
+       } else {
+               pr_err_ratelimited("Event logged [IO_PAGE_FAULT device=%02x:%02x.%x domain=0x%04x address=0x%llx flags=0x%04x]\n",
                        PCI_BUS_NUM(devid), PCI_SLOT(devid), PCI_FUNC(devid),
                        domain_id, address, flags);
        }

Note also that there might be other places in this file that would need similar modification as well.

Thanks,
Suravee

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#2 lore raw
Quoted 1 line · Suravee

Hi Lennert,

Hi Suravee,

Quoted 36 lines

drivers/iommu/amd/iommu.c static void amd_iommu_report_page_fault(u16 devid, u16 domain_id, · line 486

Quoted 12 lines

On an AMD system, I/O page faults are usually logged like this:
....

drivers/iommu/amd/iommu.c +1 -1
index 811a49a95d04..7ae426b092f2 100644
@@ -483,7 +483,7 @@ static void amd_iommu_report_page_fault(u16 devid, u16 domain_id,
  	if (dev_data && __ratelimit(&dev_data->rs)) {
  		pci_err(pdev, "Event logged [IO_PAGE_FAULT domain=0x%04x address=0x%llx flags=0x%04x]\n",
  			domain_id, address, flags);
-	} else if (printk_ratelimit()) {
+	} else if (!dev_data && printk_ratelimit()) {

This seems a bit confusing. Also, according to the following comment in include/linux/printk.h:

/*
* Please don't use printk_ratelimit(), because it shares ratelimiting state
* with all other unrelated printk_ratelimit() callsites. Instead use
* printk_ratelimited() or plain old __ratelimit().
*/

We probably should move away from using printk_ratelimit() here. What about the following change instead?

drivers/iommu/amd/iommu.c +0 -4
index 811a49a95d04..8eb5d3519743 100644
@@ -480,11 +480,12 @@ static void amd_iommu_report_page_fault(u16 devid, u16 domain_id,
        if (pdev)
                dev_data = dev_iommu_priv_get(&pdev->dev);
 
-       if (dev_data && __ratelimit(&dev_data->rs)) {
-               pci_err(pdev, "Event logged [IO_PAGE_FAULT domain=0x%04x address=0x%llx flags=0x%04x]\n",
-                       domain_id, address, flags);
-       } else if (printk_ratelimit()) {

drivers/iommu/amd/iommu.c static void amd_iommu_report_page_fault(u16 devid, u16 domain_id, · line 491

Quoted 10 lines
drivers/iommu/amd/iommu.c +6 -1
-               pr_err("Event logged [IO_PAGE_FAULT device=%02x:%02x.%x domain=0x%04x address=0x%llx flags=0x%04x]\n",
+       if (dev_data) {
+               if (__ratelimit(&dev_data->rs))
+                       pci_err(pdev, "Event logged [IO_PAGE_FAULT domain=0x%04x address=0x%llx flags=0x%04x]\n",
+                               domain_id, address, flags);
+       } else {
+               pr_err_ratelimited("Event logged [IO_PAGE_FAULT device=%02x:%02x.%x domain=0x%04x address=0x%llx flags=0x%04x]\n",
                        PCI_BUS_NUM(devid), PCI_SLOT(devid), PCI_FUNC(devid),
                        domain_id, address, flags);
        }

Looks good!

Quoted 2 lines

Note also that there might be other places in this file that would need similar modification as well.

Indeed, there are two more sites like these.

I've sent a new patch that incorporates your feedback. Thank you!

Cheers,
Lennert
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu