iommu/amd: Suppress IO_PAGE_FAULTs in kdump kernel
Joerg Roedel
maintainerPATCH 1/1 iommu/amd: Suppress IO_PAGE_FAULTs in kdump kernel
From: Joerg Roedel <[email protected]>
When booting into a kdump kernel, suppress IO_PAGE_FAULTs by default for all devices. But allow the faults again when a domain is assigned to a device.
--- drivers/iommu/amd_iommu.c | 3 ++- drivers/iommu/amd_iommu_init.c | 9 +++++++++ drivers/iommu/amd_iommu_types.h | 1 + 3 files changed, 12 insertions(+), 1 deletion(-)
drivers/iommu/amd_iommu.c +2 -1
@@ -2050,7 +2050,8 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats) flags |= tmp; } - flags &= ~(0xffffUL ); + + flags &= ~(DTE_FLAG_SA |0xffffULL ); flags |= domain->id; amd_iommu_dev_table[devid].data[1 ] = flags;
drivers/iommu/amd_iommu_init.c +9 -0
@@ -29,6 +29,7 @@#include <linux/export.h> #include <linux/iommu.h> #include <linux/kmemleak.h> +#include <linux/crash_dump.h> #include <asm/pci-direct.h> #include <asm/iommu.h> #include <asm/gart.h>
@@ -1898,6 +1899,14 @@ static void init_device_table_dma(void)for (devid =0 ; devid <= amd_iommu_last_bdf; ++devid) {set_dev_entry_bit (devid,DEV_ENTRY_VALID );set_dev_entry_bit (devid,DEV_ENTRY_TRANSLATION ); +/* +* In kdump kernels in-flight DMA from the old kernel might +* cause IO_PAGE_FAULTs. There are no reports that a kdump +* actually failed because of that, so just disable fault +* reporting in the hardware to get rid of the messages +*/ +if (is_kdump_kernel ()) +set_dev_entry_bit (devid,DEV_ENTRY_NO_PAGE_FAULT ); } }
drivers/iommu/amd_iommu_types.h +1 -0
@@ -322,6 +322,7 @@#define IOMMU_PTE_IW (1ULL << 62)#define DTE_FLAG_IOTLB (1ULL << 32) +#define DTE_FLAG_SA (1ULL << 34)#define DTE_FLAG_GV (1ULL << 55)#define DTE_FLAG_MASK (0x3ffULL << 32)#define DTE_GLX_SHIFT (56)
Signature · 1 line
2.7.4
Hi dear Joerg,
Quoted 5 lines · Joerg Roedel
From: Joerg Roedel <[email protected]>
When booting into a kdump kernel, suppress
IO_PAGE_FAULTsby default for all devices. But allow the faults again when a domain is assigned to a device.
I have two bugs at hand reported by customer, saying their system hang with amd iommu on. I remember I borrowed the system and found it hang very early so that no one knew what's happened. One time it printed several lines of boot message and I found it's amd iommu system, adding amd_iommu=off to make the system boot normally.
And with the kdump fix of amd iommu patchset applied, kdump kernel boots well. So maybe suppressing the fault message is not enough.
Thanks
Baoquan
Quoted 68 lines
Signed-off-by Joerg Roedel <[email protected]>--- drivers/iommu/amd_iommu.c | 3 ++- drivers/iommu/amd_iommu_init.c | 9 +++++++++ drivers/iommu/amd_iommu_types.h | 1 + 3 files changed, 12 insertions(+), 1 deletion(-)drivers/iommu/amd_iommu.c +2
-1@@ -2050,7 +2050,8 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats) flags |= tmp; } - flags &= ~(0xffffUL ); + + flags &= ~(DTE_FLAG_SA |0xffffULL ); flags |= domain->id; amd_iommu_dev_table[devid].data[1 ] = flags;drivers/iommu/amd_iommu_init.c +9
-0@@ -29,6 +29,7 @@#include <linux/export.h> #include <linux/iommu.h> #include <linux/kmemleak.h> +#include <linux/crash_dump.h> #include <asm/pci-direct.h> #include <asm/iommu.h> #include <asm/gart.h> @@ -1898,6 +1899,14 @@ static void init_device_table_dma(void)for (devid =0 ; devid <= amd_iommu_last_bdf; ++devid) {set_dev_entry_bit (devid,DEV_ENTRY_VALID );set_dev_entry_bit (devid,DEV_ENTRY_TRANSLATION ); +/* +* In kdump kernels in-flight DMA from the old kernel might +* cause IO_PAGE_FAULTs. There are no reports that a kdump +* actually failed because of that, so just disable fault +* reporting in the hardware to get rid of the messages +*/ +if (is_kdump_kernel ()) +set_dev_entry_bit (devid,DEV_ENTRY_NO_PAGE_FAULT ); } }drivers/iommu/amd_iommu_types.h +1
-0@@ -322,6 +322,7 @@#define IOMMU_PTE_IW (1ULL << 62)#define DTE_FLAG_IOTLB (1ULL << 32) +#define DTE_FLAG_SA (1ULL << 34)#define DTE_FLAG_GV (1ULL << 55)#define DTE_FLAG_MASK (0x3ffULL << 32)#define DTE_GLX_SHIFT (56)Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Hi Joerg,
Quoted 9 lines · Baoquan He
Hi dear Joerg,
Quoted 5 lines · Joerg Roedel
From: Joerg Roedel <[email protected]>
When booting into a kdump kernel, suppress
IO_PAGE_FAULTsby default for all devices. But allow the faults again when a domain is assigned to a device.
Quoted 8 lines
I have two bugs at hand reported by customer, saying their system hang with amd iommu on. I remember I borrowed the system and found it hang very early so that no one knew what's happened. One time it printed several lines of boot message and I found it's amd iommu system, adding amd_iommu=off to make the system boot normally.
And with the kdump fix of amd iommu patchset applied, kdump kernel boots well. So maybe suppressing the fault message is not enough.
Do you think whether it's necessary to continue my kdump fix of amd iommu patchset? Seems my last post was in Jan this year. I know you are very busy on fixing bugs and reviewing tons of patches. Without your guidance and reviewing, I absolutely can't make it. So I would like to hear your suggestions and idea.
I focused on kaslr issues recently, now most of them have been fixed. My boss discussed with me about the next plan. If you have other plan, I can sync it to our team about the status of upstream.
Thanks
Baoquan
Quoted 66 lines
Quoted 66 lines
--- drivers/iommu/amd_iommu.c | 3 ++- drivers/iommu/amd_iommu_init.c | 9 +++++++++ drivers/iommu/amd_iommu_types.h | 1 + 3 files changed, 12 insertions(+), 1 deletion(-)drivers/iommu/amd_iommu.c +2
-1@@ -2050,7 +2050,8 @@ static void set_dte_entry(u16 devid, struct protection_domain *domain, bool ats) flags |= tmp; } - flags &= ~(0xffffUL ); + + flags &= ~(DTE_FLAG_SA |0xffffULL ); flags |= domain->id; amd_iommu_dev_table[devid].data[1 ] = flags;drivers/iommu/amd_iommu_init.c +9
-0@@ -29,6 +29,7 @@#include <linux/export.h> #include <linux/iommu.h> #include <linux/kmemleak.h> +#include <linux/crash_dump.h> #include <asm/pci-direct.h> #include <asm/iommu.h> #include <asm/gart.h> @@ -1898,6 +1899,14 @@ static void init_device_table_dma(void)for (devid =0 ; devid <= amd_iommu_last_bdf; ++devid) {set_dev_entry_bit (devid,DEV_ENTRY_VALID );set_dev_entry_bit (devid,DEV_ENTRY_TRANSLATION ); +/* +* In kdump kernels in-flight DMA from the old kernel might +* cause IO_PAGE_FAULTs. There are no reports that a kdump +* actually failed because of that, so just disable fault +* reporting in the hardware to get rid of the messages +*/ +if (is_kdump_kernel ()) +set_dev_entry_bit (devid,DEV_ENTRY_NO_PAGE_FAULT ); } }drivers/iommu/amd_iommu_types.h +1
-0@@ -322,6 +322,7 @@#define IOMMU_PTE_IW (1ULL << 62)#define DTE_FLAG_IOTLB (1ULL << 32) +#define DTE_FLAG_SA (1ULL << 34)#define DTE_FLAG_GV (1ULL << 55)#define DTE_FLAG_MASK (0x3ffULL << 32)#define DTE_GLX_SHIFT (56)Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Hi Baoquan,
Quoted 9 lines · Baoquan He
Do you think whether it's necessary to continue my kdump fix of amd iommu patchset? Seems my last post was in Jan this year. I know you are very busy on fixing bugs and reviewing tons of patches. Without your guidance and reviewing, I absolutely can't make it. So I would like to hear your suggestions and idea.
I focused on kaslr issues recently, now most of them have been fixed. My boss discussed with me about the next plan. If you have other plan, I can sync it to our team about the status of upstream.
Sorry for my silence on the patches, I have not yet found the time to look deeply into them. I am still interested in them, so how about you do a rebase/repost after the next merge window and then I will take the time for a more in depth review and discussion?
Thanks,
Joerg
Quoted 9 lines · Joerg Roedel
Hi Baoquan,
Quoted 6 lines · Baoquan He
Do you think whether it's necessary to continue my kdump fix of amd iommu patchset? Seems my last post was in Jan this year. I know you are very busy on fixing bugs and reviewing tons of patches. Without your guidance and reviewing, I absolutely can't make it. So I would like to hear your suggestions and idea.
Quoted 8 lines
Quoted 3 lines
I focused on kaslr issues recently, now most of them have been fixed. My boss discussed with me about the next plan. If you have other plan, I can sync it to our team about the status of upstream.
Sorry for my silence on the patches, I have not yet found the time to look deeply into them. I am still interested in them, so how about you do a rebase/repost after the next merge window and then I will take the time for a more in depth review and discussion?
Thanks a lot for reply. Totally understood, so many patches need be reviewed, and so many different iommu types.
Sure, let me do it right away. I may need a little time to recall the details, go through the spec and code again. It won't take too long.
Thanks
Baoquan
Quoted 1 line
Hi Joerg,
Quoted 9 lines · Joerg Roedel
Hi Baoquan,
Quoted 6 lines · Baoquan He
Do you think whether it's necessary to continue my kdump fix of amd iommu patchset? Seems my last post was in Jan this year. I know you are very busy on fixing bugs and reviewing tons of patches. Without your guidance and reviewing, I absolutely can't make it. So I would like to hear your suggestions and idea.
Quoted 8 lines
Quoted 3 lines
I focused on kaslr issues recently, now most of them have been fixed. My boss discussed with me about the next plan. If you have other plan, I can sync it to our team about the status of upstream.
Sorry for my silence on the patches, I have not yet found the time to look deeply into them. I am still interested in them, so how about you do a rebase/repost after the next merge window and then I will take the time for a more in depth review and discussion?
I have rebased the amd iommu fix patches of kdump kernel on the latest upstream kernel, can I send them to you to have a look? Or just send to iommu and lkml mailing list?
Thanks
Baoquan
Hi Baoquan,
Quoted 3 lines · Baoquan He
I have rebased the amd iommu fix patches of kdump kernel on the latest upstream kernel, can I send them to you to have a look? Or just send to iommu and lkml mailing list?
Please send them to me and Cc iommu and lkml.
Thanks,
Joerg