Skip to contentSkip to conversation

hw/arm/smmuv3: Fix addr_mask for range-based invalidation

PATCH Superseded Zenghui Yu 2 participants 6 messages Dec 25, 2020 – Jan 29, 2021

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Zenghui Yu

author
#0 lore raw

PATCH 1/1 hw/arm/smmuv3: Fix addr_mask for range-based invalidation

When performing range-based IOTLB invalidation, we should decode the TG field into the corresponding translation granule size so that we can pass the correct invalidation range to backend. Set @granule to (tg * 2 + 10) to properly emulate the architecture.

Fixes d52915616c05 ("hw/arm/smmuv3: Get prepared for range invalidation")
---
 hw/arm/smmuv3.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +3 -1
index bbca0e9f20..65231c7d52 100644
@@ -801,7 +801,7 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
 {
     SMMUDevice *sdev = container_of(mr, SMMUDevice, iommu);
     IOMMUTLBEvent event;
-    uint8_t granule = tg;
+    uint8_t granule;
 
     if (!tg) {
         SMMUEventInfo event = {.inval_ste_allowed = true};
@@ -821,6 +821,8 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
             return;
         }
         granule = tt->granule_sz;
+    } else {
+        guanule = tg * 2 + 10;
     }
 
     event.type = IOMMU_NOTIFIER_UNMAP;
Signature · 1 line
2.19.1
author
#1 lore raw

hw/arm/smmuv3.c static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, · line 825

Quoted 6 lines · Zenghui Yu
hw/arm/smmuv3.c +2 -0
@@ -821,6 +821,8 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
              return;
          }
          granule = tt->granule_sz;
+    } else {
+        guanule = tg * 2 + 10;

I'm embarrassed about that.

s/guanule/granule/

Quoted 3 lines
hw/arm/smmuv3.c +0 -0
      }
 
      event.type = IOMMU_NOTIFIER_UNMAP;
#2 lore raw
In reply to Zenghui Yu

Hi Zenghui,

Quoted 7 lines · Zenghui Yu

When performing range-based IOTLB invalidation, we should decode the TG field into the corresponding translation granule size so that we can pass the correct invalidation range to backend. Set @granule to (tg * 2 + 10) to properly emulate the architecture.

Fixes d52915616c05 ("hw/arm/smmuv3: Get prepared for range invalidation")

Good catch! I tested with older guest kernels though. I wonder how I did not face the bug?

Quoted 18 lines
---
 hw/arm/smmuv3.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -1
index bbca0e9f20..65231c7d52 100644
@@ -801,7 +801,7 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
 {
     SMMUDevice *sdev = container_of(mr, SMMUDevice, iommu);
     IOMMUTLBEvent event;
-    uint8_t granule = tg;
+    uint8_t granule;
 
     if (!tg) {
         SMMUEventInfo event = {.inval_ste_allowed = true};

hw/arm/smmuv3.c static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, · line 825

Quoted 5 lines
hw/arm/smmuv3.c +2 -0
@@ -821,6 +821,8 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
             return;
         }
         granule = tt->granule_sz;
+    } else {
+        guanule = tg * 2 + 10;

maybe just init granule to this value above while fixing the typo.

Thanks

Eric

Quoted 4 lines
hw/arm/smmuv3.c +0 -0
     }
 
     event.type = IOMMU_NOTIFIER_UNMAP;
#3 lore raw

Hi Zenghui,

Quoted 8 lines · Auger Eric

Hi Zenghui,

Quoted 5 lines · Zenghui Yu

When performing range-based IOTLB invalidation, we should decode the TG field into the corresponding translation granule size so that we can pass the correct invalidation range to backend. Set @granule to (tg * 2 + 10) to properly emulate the architecture.

Quoted 5 lines
Quoted 2 lines
Fixes d52915616c05 ("hw/arm/smmuv3: Get prepared for range invalidation")

Good catch! I tested with older guest kernels though. I wonder how I did not face the bug?

Please ignore this wrong comment as this corresponds to recent kernels instead. Still puzzled anyway ;-)

Eric

Quoted 34 lines
Quoted 20 lines

---
 hw/arm/smmuv3.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -1
index bbca0e9f20..65231c7d52 100644
@@ -801,7 +801,7 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
 {
     SMMUDevice *sdev = container_of(mr, SMMUDevice, iommu);
     IOMMUTLBEvent event;
-    uint8_t granule = tg;
+    uint8_t granule;
 
     if (!tg) {
         SMMUEventInfo event = {.inval_ste_allowed = true};

hw/arm/smmuv3.c static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, · line 825

Quoted 5 lines
hw/arm/smmuv3.c +2 -0
@@ -821,6 +821,8 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
             return;
         }
         granule = tt->granule_sz;
+    } else {
+        guanule = tg * 2 + 10;

maybe just init granule to this value above while fixing the typo.

Thanks

Eric

Quoted 4 lines
hw/arm/smmuv3.c +0 -0
     }
 
     event.type = IOMMU_NOTIFIER_UNMAP;
author
#4 lore raw

Hi Eric,

Quoted 11 lines · Auger Eric

Hi Zenghui,

Quoted 8 lines · Auger Eric

Hi Zenghui,

Quoted 5 lines · Zenghui Yu

When performing range-based IOTLB invalidation, we should decode the TG field into the corresponding translation granule size so that we can pass the correct invalidation range to backend. Set @granule to (tg * 2 + 10) to properly emulate the architecture.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Fixes d52915616c05 ("hw/arm/smmuv3: Get prepared for range invalidation")

Good catch! I tested with older guest kernels though. I wonder how I did not face the bug?

Please ignore this wrong comment as this corresponds to recent kernels instead. Still puzzled anyway ;-)

I noticed this when looking through your nested SMMU series and I didn't have much clue about the impact on the real setups.

I guess we may receive some unexpected fault events with this bug. But I think we may miss it for some reasons:

- the stale TLB entries happen to be evicted due to heavy traffic
- some form of over-invalidation is performed by your implementation
- ...

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
---
  hw/arm/smmuv3.c | 4 +++-
  1 file changed, 3 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -1
index bbca0e9f20..65231c7d52 100644
@@ -801,7 +801,7 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
  {
      SMMUDevice *sdev = container_of(mr, SMMUDevice, iommu);
      IOMMUTLBEvent event;
-    uint8_t granule = tg;
+    uint8_t granule;
 
Quoted 9 lines
Quoted 9 lines

hw/arm/smmuv3.c static void smmuv3_notify_iova(IOMMUMemoryRegion *mr, · line 825

Quoted 8 lines
hw/arm/smmuv3.c +2 -0
      if (!tg) {
          SMMUEventInfo event = {.inval_ste_allowed = true};
@@ -821,6 +821,8 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
              return;
          }
          granule = tt->granule_sz;
+    } else {
+        guanule = tg * 2 + 10;

maybe just init granule to this value above while fixing the typo.

My intention is to initialize @granule to this value explicitly for the range-based invalidation case. But I'm okay with either way.

Thanks,
Zenghui

#5 lore raw

Hi Zenghui,

Quoted 28 lines · Zenghui Yu

Hi Eric,

Quoted 13 lines · Auger Eric

Hi Zenghui,

Quoted 10 lines · Auger Eric

Hi Zenghui,

Quoted 7 lines · Zenghui Yu

When performing range-based IOTLB invalidation, we should decode the TG field into the corresponding translation granule size so that we can pass
the correct invalidation range to backend. Set @granule to (tg * 2 +
10) to
properly emulate the architecture.

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Fixes d52915616c05 ("hw/arm/smmuv3: Get prepared for range

invalidation")

Good catch! I tested with older guest kernels though. I wonder how I did not face the bug?

Please ignore this wrong comment as this corresponds to recent kernels instead. Still puzzled anyway ;-)

I noticed this when looking through your nested SMMU series and I didn't have much clue about the impact on the real setups.

Quoted 6 lines

I guess we may receive some unexpected fault events with this bug. But I think we may miss it for some reasons:

 - the stale TLB entries happen to be evicted due to heavy traffic
 - some form of over-invalidation is performed by your implementation
 - ...

Yep I will further trace things. Anyway thank you for spotting it.

Quoted 27 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

---
  hw/arm/smmuv3.c | 4 +++-

  1 file changed, 3 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -1
index bbca0e9f20..65231c7d52 100644
@@ -801,7 +801,7 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion
 *mr,
   {
       SMMUDevice *sdev = container_of(mr, SMMUDevice, iommu);
       IOMMUTLBEvent event;
-    uint8_t granule = tg;
+    uint8_t granule;
         if (!tg) {
           SMMUEventInfo event = {.inval_ste_allowed = true};
Quoted 8 lines
Quoted 8 lines

hw/arm/smmuv3.c static void smmuv3_notify_iova(IOMMUMemoryRegion · line 826

Quoted 7 lines
hw/arm/smmuv3.c +2 -0
@@ -821,6 +821,8 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion
 *mr,
               return;
           }
           granule = tt->granule_sz;
+    } else {
+        guanule = tg * 2 + 10;

maybe just init granule to this value above while fixing the typo.

Quoted 2 lines

My intention is to initialize @granule to this value explicitly for the range-based invalidation case. But I'm okay with either way.

same for me ;-)

Eric

Quoted 5 lines

Thanks,
Zenghui