Skip to contentSkip to conversation

iommu/amd-vi: fix checking for Invalidate All support in amd_iommu_resume()

PATCH In review Roger Pau Monne 2 participants 4 messages Jun 13, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iommu/amd-vi: fix checking for Invalidate All support in amd_iommu_resume()

Do not rely on iommu local variable pointing to a valid amd_iommu element after the call to for_each_amd_iommu(). Instead check whether any IOMMU on the system doesn't support Invalidate All in order to perform the per-domain and per-device flushes.

Fixes 9c46139de889 ('amd iommu: Support INVALIDATE_IOMMU_ALL command.')

---
I don't have a way to test host suspend/resume, so the patch is only build tested.

---
 xen/drivers/passthrough/amd/iommu_init.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
xen/drivers/passthrough/amd/iommu_init.c +4 -1
index 9773ccfcb41f..4facff93d68b 100644
@@ -1580,6 +1580,7 @@ void cf_check amd_iommu_crash_shutdown(void)
 void cf_check amd_iommu_resume(void)
 {
     struct amd_iommu *iommu;
+    bool invalidate_all = true;
 
     for_each_amd_iommu ( iommu )
     {
@@ -1589,10 +1590,12 @@ void cf_check amd_iommu_resume(void)
         */
         disable_iommu(iommu);
         enable_iommu(iommu);
+        if ( !iommu->features.flds.ia_sup && invalidate_all )
+            invalidate_all = false;
     }
 
     /* flush all cache entries after iommu re-enabled */
-    if ( !iommu->features.flds.ia_sup )
+    if ( !invalidate_all )
     {
         invalidate_all_devices();
         invalidate_all_domain_pages();
Signature · 1 line
2.40.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 1 line · Roger Pau Monne

Do not rely on iommu local variable pointing to a valid amd_iommu

"Do not rely" sounds like we might, but we choose not to. But we may not rely on this, as the pointer simply isn't valid to deref past the loop. Hence perhaps better "We cannot rely ..." or even "The iommu local variable does not point to ..."?

Quoted 26 lines

element after the call to for_each_amd_iommu(). Instead check whether any IOMMU on the system doesn't support Invalidate All in order to perform the per-domain and per-device flushes.

Fixes 9c46139de889 ('amd iommu: Support INVALIDATE_IOMMU_ALL command.')

---
I don't have a way to test host suspend/resume, so the patch is only build tested.

---
 xen/drivers/passthrough/amd/iommu_init.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
xen/drivers/passthrough/amd/iommu_init.c +1 -0
index 9773ccfcb41f..4facff93d68b 100644
@@ -1580,6 +1580,7 @@ void cf_check amd_iommu_crash_shutdown(void)
 void cf_check amd_iommu_resume(void)
 {
     struct amd_iommu *iommu;
+    bool invalidate_all = true;
 
     for_each_amd_iommu ( iommu )
     {

xen/drivers/passthrough/amd/iommu_init.c void cf_check amd_iommu_resume(void) · line 1593

Quoted 4 lines
xen/drivers/passthrough/amd/iommu_init.c +1 -0
@@ -1589,10 +1590,12 @@ void cf_check amd_iommu_resume(void)
         */
         disable_iommu(iommu);
         enable_iommu(iommu);
+        if ( !iommu->features.flds.ia_sup && invalidate_all )

You don't really need the right hand part of the condition, do you?

Preferably with the adjustments (which I'd be happy to do while committing, as long as you agree)

Jan

Quoted 9 lines
xen/drivers/passthrough/amd/iommu_init.c +2 -1
+            invalidate_all = false;
     }
 
     /* flush all cache entries after iommu re-enabled */
-    if ( !iommu->features.flds.ia_sup )
+    if ( !invalidate_all )
     {
         invalidate_all_devices();
         invalidate_all_domain_pages();
#2 lore raw
Quoted 7 lines · Jan Beulich
Quoted 1 line · Roger Pau Monne

Do not rely on iommu local variable pointing to a valid amd_iommu

"Do not rely" sounds like we might, but we choose not to. But we may not rely on this, as the pointer simply isn't valid to deref past the loop. Hence perhaps better "We cannot rely ..." or even "The iommu local variable does not point to ..."?

"Xen cannot rely ..." doesn't modify the sentence too much and could likely be adjusted at commit if you agree?

Otherwise your last suggestion would also be OK by me.

Quoted 33 lines
Quoted 26 lines

element after the call to for_each_amd_iommu(). Instead check whether any IOMMU on the system doesn't support Invalidate All in order to perform the per-domain and per-device flushes.

Fixes 9c46139de889 ('amd iommu: Support INVALIDATE_IOMMU_ALL command.')

---
I don't have a way to test host suspend/resume, so the patch is only build tested.

---
 xen/drivers/passthrough/amd/iommu_init.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
xen/drivers/passthrough/amd/iommu_init.c +1 -0
index 9773ccfcb41f..4facff93d68b 100644
@@ -1580,6 +1580,7 @@ void cf_check amd_iommu_crash_shutdown(void)
 void cf_check amd_iommu_resume(void)
 {
     struct amd_iommu *iommu;
+    bool invalidate_all = true;
 
     for_each_amd_iommu ( iommu )
     {

xen/drivers/passthrough/amd/iommu_init.c void cf_check amd_iommu_resume(void) · line 1593

Quoted 4 lines
xen/drivers/passthrough/amd/iommu_init.c +1 -0
@@ -1589,10 +1590,12 @@ void cf_check amd_iommu_resume(void)
         */
         disable_iommu(iommu);
         enable_iommu(iommu);
+        if ( !iommu->features.flds.ia_sup && invalidate_all )

You don't really need the right hand part of the condition, do you?

Quoted 3 lines

Preferably with the adjustments (which I'd be happy to do while committing, as long as you agree)

Wanted to avoid repeatedly setting invalidate_all = false if all the IOMMUs on the system don't support Invalidate All.

I don't have a strong opinion, my first (local) version didn't have the right hand side of the condition, but then I realized that setting this for every IOMMU on the system could be wasteful.

Thanks, Roger.

#3 lore raw
Quoted 9 lines · Roger Pau Monné
Quoted 7 lines · Jan Beulich
Quoted 1 line · Roger Pau Monne

Do not rely on iommu local variable pointing to a valid amd_iommu

"Do not rely" sounds like we might, but we choose not to. But we may not rely on this, as the pointer simply isn't valid to deref past the loop. Hence perhaps better "We cannot rely ..." or even "The iommu local variable does not point to ..."?

Quoted 4 lines

"Xen cannot rely ..." doesn't modify the sentence too much and could likely be adjusted at commit if you agree?

Otherwise your last suggestion would also be OK by me.

I used that latter one, as being more concise.

Quoted 22 lines
Quoted 18 lines
Quoted 11 lines
xen/drivers/passthrough/amd/iommu_init.c +1 -0
@@ -1580,6 +1580,7 @@ void cf_check amd_iommu_crash_shutdown(void)
 void cf_check amd_iommu_resume(void)
 {
     struct amd_iommu *iommu;
+    bool invalidate_all = true;
 
     for_each_amd_iommu ( iommu )
     {

xen/drivers/passthrough/amd/iommu_init.c void cf_check amd_iommu_resume(void) · line 1593

Quoted 4 lines
xen/drivers/passthrough/amd/iommu_init.c +1 -0
@@ -1589,10 +1590,12 @@ void cf_check amd_iommu_resume(void)
         */
         disable_iommu(iommu);
         enable_iommu(iommu);
+        if ( !iommu->features.flds.ia_sup && invalidate_all )

You don't really need the right hand part of the condition, do you?

Quoted 3 lines

Preferably with the adjustments (which I'd be happy to do while committing, as long as you agree)

Quoted 6 lines

Wanted to avoid repeatedly setting invalidate_all = false if all the IOMMUs on the system don't support Invalidate All.

I don't have a strong opinion, my first (local) version didn't have the right hand side of the condition, but then I realized that setting this for every IOMMU on the system could be wasteful.

I've dropped that part: We don't care much about that tiny performance difference here (and it's unclear whether the extra check actually is a win or a loss), and imo the code is more clear with the simpler conditional.

Jan