iommu/amd: fix interrupt remapping for avic
Subsystems (2)
PATCH 1/1 iommu/amd: fix interrupt remapping for avic
Commit e52d58d54a32 ("iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE") accidentally removed an assumption that modify_irte_ga always set the valid bit and amd_iommu_activate_guest_mode relied on that.
Side effect of this is that on my machine, VFIO based VMs with AVIC enabled would eventually crash and show IOMMU errors like that:
AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0055 address=0xfffffffdf8000000 flags=0x0008]
--- drivers/iommu/amd/iommu.c | 1 + 1 file changed, 1 insertion(+)
drivers/iommu/amd/iommu.c +1 -0
@@ -3853,6 +3853,7 @@ int amd_iommu_activate_guest_mode(void *data) entry->hi.fields.ga_root_ptr = ir_data->ga_root_ptr; entry->hi.fields.vector = ir_data->ga_vector; entry->lo.fields_vapic.ga_tag = ir_data->ga_tag; + entry->lo.fields_remap.valid =1 ;return modify_irte_ga (ir_data->irq_2_irte.devid, ir_data->irq_2_irte.index, entry, ir_data);
Signature · 1 line
2.26.2
Suravee Suthikulpanit
reviewerMaxim,
Quoted 21 lines · Maxim Levitsky
Commit
e52d58d54a32("iommu/amd: Usecmpxchg_double()when updating 128-bit IRTE") accidentally removed an assumption thatmodify_irte_gaalways set the valid bit andamd_iommu_activate_guest_moderelied on that.Side effect of this is that on my machine, VFIO based VMs with AVIC enabled would eventually crash and show IOMMU errors like that:
AMD-Vi: Event logged [
IO_PAGE_FAULTdomain=0x0055 address=0xfffffffdf8000000 flags=0x0008]Fixes e52d58d54a321 ("iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE")Signed-off-by Maxim Levitsky <[email protected]>--- drivers/iommu/amd/iommu.c | 1 + 1 file changed, 1 insertion(+)drivers/iommu/amd/iommu.c +0
-0
drivers/iommu/amd/iommu.c int amd_iommu_activate_guest_mode(void *data) · line 3859
Quoted 7 lines
drivers/iommu/amd/iommu.c +1
-0@@ -3853,6 +3853,7 @@ int amd_iommu_activate_guest_mode(void *data) entry->hi.fields.ga_root_ptr = ir_data->ga_root_ptr; entry->hi.fields.vector = ir_data->ga_vector; entry->lo.fields_vapic.ga_tag = ir_data->ga_tag; + entry->lo.fields_remap.valid =1 ;return modify_irte_ga (ir_data->irq_2_irte.devid, ir_data->irq_2_irte.index, entry, ir_data);
Could you please try with the following patch instead?
drivers/iommu/amd/iommu.c +7 -2
@@ -3840,14 +3840,18 @@ int amd_iommu_activate_guest_mode(void *data) {struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry; +u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_vapic.valid; + entry->lo.val =0 ; entry->hi.val =0 ; + entry->lo.fields_vapic.valid = valid; entry->lo.fields_vapic.guest_mode =1 ; entry->lo.fields_vapic.ga_log_intr =1 ; entry->hi.fields.ga_root_ptr = ir_data->ga_root_ptr;
@@ -3864,12 +3868,14 @@ int amd_iommu_deactivate_guest_mode(void *data)struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry;struct irq_cfg *cfg = ir_data->cfg; -u64 valid = entry->lo.fields_remap.valid; +u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || !entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_remap.valid; + entry->lo.val =0 ; entry->hi.val =0 ; --
Thanks,
Suravee
Quoted 60 lines · Suravee Suthikulpanit
Maxim,
Quoted 21 lines · Maxim Levitsky
Commit
e52d58d54a32("iommu/amd: Usecmpxchg_double()when updating 128-bit IRTE") accidentally removed an assumption thatmodify_irte_gaalways set the valid bit andamd_iommu_activate_guest_moderelied on that.Side effect of this is that on my machine, VFIO based VMs with AVIC enabled would eventually crash and show IOMMU errors like that:
AMD-Vi: Event logged [
IO_PAGE_FAULTdomain=0x0055 address=0xfffffffdf8000000 flags=0x0008]Fixes e52d58d54a321 ("iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE")Signed-off-by Maxim Levitsky <[email protected]>--- drivers/iommu/amd/iommu.c | 1 + 1 file changed, 1 insertion(+)drivers/iommu/amd/iommu.c +0
-0drivers/iommu/amd/iommu.c int amd_iommu_activate_guest_mode(void *data) · line 3859
Quoted 7 lines
drivers/iommu/amd/iommu.c +1
-0@@ -3853,6 +3853,7 @@ int amd_iommu_activate_guest_mode(void *data) entry->hi.fields.ga_root_ptr = ir_data->ga_root_ptr; entry->hi.fields.vector = ir_data->ga_vector; entry->lo.fields_vapic.ga_tag = ir_data->ga_tag; + entry->lo.fields_remap.valid =1 ;return modify_irte_ga (ir_data->irq_2_irte.devid, ir_data->irq_2_irte.index, entry, ir_data);Could you please try with the following patch instead?
drivers/iommu/amd/iommu.c +4
-1@@ -3840,14 +3840,18 @@ int amd_iommu_activate_guest_mode(void *data) {struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry; +u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_vapic.valid; + entry->lo.val =0 ; entry->hi.val =0 ; + entry->lo.fields_vapic.valid = valid; entry->lo.fields_vapic.guest_mode =1 ; entry->lo.fields_vapic.ga_log_intr =1 ; entry->hi.fields.ga_root_ptr = ir_data->ga_root_ptr;@@ -3864,12 +3868,14 @@ int amd_iommu_deactivate_guest_mode(void *data)struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry;struct irq_cfg *cfg = ir_data->cfg; -u64 valid = entry->lo.fields_remap.valid;
drivers/iommu/amd/iommu.c int amd_iommu_deactivate_guest_mode(void *data) · line 3880
Quoted 10 lines
drivers/iommu/amd/iommu.c +3
-0+u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || !entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_remap.valid; + entry->lo.val =0 ; entry->hi.val =0 ;
I see. I based my approach on the fact that valid bit was
set always to true anyway before, plus that amd_iommu_activate_guest_mode should be really only called when someone activates a valid interrupt remapping entry, but IMHO the approach of preserving the valid bit is safer anyway.
It works on my system (I applied the patch manually, since either your or my email client, seems to mangle the patch)
Thanks,
Best regards,
Maxim Levitsky
Quoted 6 lines
drivers/iommu/amd/iommu.c +0
-1--Thanks,
Suravee
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Suravee Suthikulpanit
reviewerQuoted 76 lines · Maxim Levitsky
Quoted 60 lines · Suravee Suthikulpanit
Maxim,
Quoted 21 lines · Maxim Levitsky
Commit
e52d58d54a32("iommu/amd: Usecmpxchg_double()when updating 128-bit IRTE") accidentally removed an assumption thatmodify_irte_gaalways set the valid bit andamd_iommu_activate_guest_moderelied on that.Side effect of this is that on my machine, VFIO based VMs with AVIC enabled would eventually crash and show IOMMU errors like that:
AMD-Vi: Event logged [
IO_PAGE_FAULTdomain=0x0055 address=0xfffffffdf8000000 flags=0x0008]Fixes e52d58d54a321 ("iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE")Signed-off-by Maxim Levitsky <[email protected]>--- drivers/iommu/amd/iommu.c | 1 + 1 file changed, 1 insertion(+)drivers/iommu/amd/iommu.c +0
-0drivers/iommu/amd/iommu.c int amd_iommu_activate_guest_mode(void *data) · line 3859
Quoted 7 lines
drivers/iommu/amd/iommu.c +1
-0@@ -3853,6 +3853,7 @@ int amd_iommu_activate_guest_mode(void *data) entry->hi.fields.ga_root_ptr = ir_data->ga_root_ptr; entry->hi.fields.vector = ir_data->ga_vector; entry->lo.fields_vapic.ga_tag = ir_data->ga_tag; + entry->lo.fields_remap.valid =1 ;return modify_irte_ga (ir_data->irq_2_irte.devid, ir_data->irq_2_irte.index, entry, ir_data);Could you please try with the following patch instead?
drivers/iommu/amd/iommu.c +4
-1@@ -3840,14 +3840,18 @@ int amd_iommu_activate_guest_mode(void *data) {struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry; +u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_vapic.valid; + entry->lo.val =0 ; entry->hi.val =0 ; + entry->lo.fields_vapic.valid = valid; entry->lo.fields_vapic.guest_mode =1 ; entry->lo.fields_vapic.ga_log_intr =1 ; entry->hi.fields.ga_root_ptr = ir_data->ga_root_ptr;@@ -3864,12 +3868,14 @@ int amd_iommu_deactivate_guest_mode(void *data)struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry;struct irq_cfg *cfg = ir_data->cfg; -u64 valid = entry->lo.fields_remap.valid;drivers/iommu/amd/iommu.c int amd_iommu_deactivate_guest_mode(void *data) · line 3880
Quoted 10 lines
drivers/iommu/amd/iommu.c +3
-0+u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || !entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_remap.valid; + entry->lo.val =0 ; entry->hi.val =0 ;I see. I based my approach on the fact that valid bit was
set always to true anyway before, plus thatamd_iommu_activate_guest_modeshould be really only called when someone activates a valid interrupt remapping entry, but IMHO the approach of preserving the valid bit is safer anyway.
Quoted 8 lines
It works on my system (I applied the patch manually, since either your or my email client, seems to mangle the patch)
Thanks,
Best regards,
Maxim Levitsky
Sorry for the mangled patch. I'll submit the patch w/ your information. Thanks for your help reporting, debugging, and testing the patch.
Sincerely,
Suravee
Quoted 8 lines
Quoted 6 lines
drivers/iommu/amd/iommu.c +0
-1--Thanks,
Suravee
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Quoted 45 lines · Suravee Suthikulpanit
Quoted 44 lines · Maxim Levitsky
Quoted 28 lines · Suravee Suthikulpanit
Could you please try with the following patch instead?
drivers/iommu/amd/iommu.c +4
-1@@ -3840,14 +3840,18 @@ int amd_iommu_activate_guest_mode(void *data) {struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry; +u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_vapic.valid; + entry->lo.val =0 ; entry->hi.val =0 ; + entry->lo.fields_vapic.valid = valid; entry->lo.fields_vapic.guest_mode =1 ; entry->lo.fields_vapic.ga_log_intr =1 ; entry->hi.fields.ga_root_ptr = ir_data->ga_root_ptr;@@ -3864,12 +3868,14 @@ int amd_iommu_deactivate_guest_mode(void *data)struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry;struct irq_cfg *cfg = ir_data->cfg; -u64 valid = entry->lo.fields_remap.valid;drivers/iommu/amd/iommu.c int amd_iommu_deactivate_guest_mode(void *data) · line 3880
Quoted 10 lines
drivers/iommu/amd/iommu.c +3
-0+u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || !entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_remap.valid; + entry->lo.val =0 ; entry->hi.val =0 ;I see. I based my approach on the fact that valid bit was
set always to true anyway before, plus thatamd_iommu_activate_guest_modeshould be really only called when someone activates a valid interrupt remapping entry, but IMHO the approach of preserving the valid bit is safer anyway.
Quoted 7 lines
Quoted 3 lines
It works on my system (I applied the patch manually, since either your or my email client, seems to mangle the patch)
Sorry for the mangled patch. I'll submit the patch w/ your information. Thanks for your help reporting, debugging, and testing the patch.
I assume you're only doing the valid bit preservation in amd_iommu_activate_guest_mode() ? The null deref fix in amd_iommu_deactivate_guest_mode() was fixed elsewhere[0], or are you planning on merging both changes like the diff you attached?
Asking also because commit 26e495f341 ("iommu/amd: Restore IRTE.RemapEn bit after programming IRTE") was added in v5.4 and v5.8 stable trees but the v5.4 backport didn't include e52d58d54a321 ("iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE").
Joao
[0] https://lore.kernel.org/linux-iommu/[email protected]/
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Suravee Suthikulpanit
reviewerQuoted 50 lines · Joao Martins
Quoted 49 lines · Suravee Suthikulpanit
Quoted 47 lines · Maxim Levitsky
Quoted 28 lines · Suravee Suthikulpanit
Could you please try with the following patch instead?
drivers/iommu/amd/iommu.c +4
-1@@ -3840,14 +3840,18 @@ int amd_iommu_activate_guest_mode(void *data) {struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry; +u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_vapic.valid; + entry->lo.val =0 ; entry->hi.val =0 ; + entry->lo.fields_vapic.valid = valid; entry->lo.fields_vapic.guest_mode =1 ; entry->lo.fields_vapic.ga_log_intr =1 ; entry->hi.fields.ga_root_ptr = ir_data->ga_root_ptr;@@ -3864,12 +3868,14 @@ int amd_iommu_deactivate_guest_mode(void *data)struct amd_ir_data *ir_data = (struct amd_ir_data *)data;struct irte_ga *entry = (struct irte_ga *) ir_data->entry;struct irq_cfg *cfg = ir_data->cfg; -u64 valid = entry->lo.fields_remap.valid;drivers/iommu/amd/iommu.c int amd_iommu_deactivate_guest_mode(void *data) · line 3880
Quoted 10 lines
drivers/iommu/amd/iommu.c +3
-0+u64 valid;if (!AMD_IOMMU_GUEST_IR_VAPIC (amd_iommu_guest_ir) || !entry || !entry->lo.fields_vapic.guest_mode)return 0 ; + valid = entry->lo.fields_remap.valid; + entry->lo.val =0 ; entry->hi.val =0 ;I see. I based my approach on the fact that valid bit was
set always to true anyway before, plus thatamd_iommu_activate_guest_modeshould be really only called when someone activates a valid interrupt remapping entry, but IMHO the approach of preserving the valid bit is safer anyway.It works on my system (I applied the patch manually, since either your or my email client, seems to mangle the patch)
Quoted 6 lines
Quoted 3 lines
Sorry for the mangled patch. I'll submit the patch w/ your information. Thanks for your help reporting, debugging, and testing the patch.
I assume you're only doing the valid bit preservation in
amd_iommu_activate_guest_mode()? The null deref fix inamd_iommu_deactivate_guest_mode()was fixed elsewhere[0], or are you planning on merging both changes like the diff you attached?
I am planning to send a separate patch just for amd_iommu_activate_guest_mode().
Quoted 3 lines
Asking also because commit
26e495f341("iommu/amd: Restore IRTE.RemapEn bit after programming IRTE") was added in v5.4 and v5.8 stable trees but the v5.4 backport didn't includee52d58d54a321("iommu/amd: Usecmpxchg_double()when updating 128-bit IRTE").
We should probably backport the e52d58d54a321 along with the fixes in amd_iommu_activate_guest_mode() and amd_iommu_deactivate_guest_mode(). I'll work with the community to get these back-ported.
Thanks,
Suravee
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu