Skip to contentSkip to conversation

intel_iommu: Fix unexpected unmaps during global unmap

PATCH In review Peter Xu 6 participants 14 messages Jun 24, 2019 – Jul 4, 2019

lore
Share & tools
lore

Peter Xu

author
#0 lore raw

v2 cover intel_iommu: Fix unexpected unmaps during global unmap

v2:
- rename helper to get_naturally_aligned_size(), simplify the codes as suggested [Paolo]
- check against vtd page size when looping over for unmaps [Yan]
- add r-b for Eric

Please review, thanks.

Peter Xu (1):
intel_iommu: Fix unexpected unmaps during global unmap

Yan Zhao (1):
intel_iommu: Fix incorrect "end" for vtd_address_space_unmap

 hw/i386/intel_iommu.c | 71 ++++++++++++++++++++++++++-----------------
 1 file changed, 43 insertions(+), 28 deletions(-)
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH v2 1/2 intel_iommu: Fix incorrect "end" for vtd_address_space_unmap

From: Yan Zhao <[email protected]>

IOMMUNotifier is with inclusive ranges, so we should check against (VTD_ADDRESS_SIZE(s->aw_bits) - 1).

[peterx: split from another bigger patch]
---
 hw/i386/intel_iommu.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
hw/i386/intel_iommu.c +2 -2
index 44b1231157..719ce19ab3 100644
@@ -3379,12 +3379,12 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
      * VT-d spec), otherwise we need to consider overflow of 64 bits.
      */
 
-    if (end > VTD_ADDRESS_SIZE(s->aw_bits)) {
+    if (end > VTD_ADDRESS_SIZE(s->aw_bits) - 1) {
         /*
          * Don't need to unmap regions that is bigger than the whole
          * VT-d supported address space size
          */
-        end = VTD_ADDRESS_SIZE(s->aw_bits);
+        end = VTD_ADDRESS_SIZE(s->aw_bits) - 1;
     }
 
     assert(start <= end);
Signature · 1 line
2.21.0
Acked-by on v2 1/2
#2 lore raw

Re: [Qemu-devel] [PATCH v2 1/2] intel_iommu: Fix incorrect "end" for vtd_address_space_unmap

Quoted 22 lines · Peter Xu

From: Yan Zhao <[email protected]>

IOMMUNotifier is with inclusive ranges, so we should check against (VTD_ADDRESS_SIZE(s->aw_bits) - 1).

[peterx: split from another bigger patch]
---
  hw/i386/intel_iommu.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
hw/i386/intel_iommu.c +0 -1
index 44b1231157..719ce19ab3 100644
@@ -3379,12 +3379,12 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
       * VT-d spec), otherwise we need to consider overflow of 64 bits.
       */
 
-    if (end > VTD_ADDRESS_SIZE(s->aw_bits)) {

hw/i386/intel_iommu.c static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) · line 3390

Quoted 10 lines
hw/i386/intel_iommu.c +2 -1
+    if (end > VTD_ADDRESS_SIZE(s->aw_bits) - 1) {
          /*
           * Don't need to unmap regions that is bigger than the whole
           * VT-d supported address space size
           */
-        end = VTD_ADDRESS_SIZE(s->aw_bits);
+        end = VTD_ADDRESS_SIZE(s->aw_bits) - 1;
      }
 
      assert(start <= end);
author
#3 lore raw

PATCH v2 2/2 intel_iommu: Fix unexpected unmaps during global unmap

In reply to Peter Xu

This is an replacement work of Yan Zhao's patch:

https://www.mail-archive.com/[email protected]/msg625340.html

vtd_address_space_unmap() will do proper page mask alignment to make sure each IOTLB message will have correct masks for notification messages (2^N-1), but sometimes it can be expanded to even supercede the registered range. That could lead to unexpected UNMAP of already mapped regions in some other notifiers.

Instead of doing mindless expension of the start address and address mask, we split the range into smaller ones and guarantee that each small range will have correct masks (2^N-1) and at the same time we should also try our best to generate as less IOTLB messages as possible.

---
 hw/i386/intel_iommu.c | 67 ++++++++++++++++++++++++++-----------------
 1 file changed, 41 insertions(+), 26 deletions(-)
hw/i386/intel_iommu.c +41 -26
index 719ce19ab3..de86f53b4e 100644
@@ -3363,11 +3363,28 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
     return vtd_dev_as;
 }
 
+static uint64_t get_naturally_aligned_size(uint64_t start,
+                                           uint64_t size, int gaw)
+{
+    uint64_t max_mask = 1ULL << gaw;
+    uint64_t alignment = start ? start & -start : max_mask;
+
+    alignment = MIN(alignment, max_mask);
+    size = MIN(size, max_mask);
+
+    if (alignment <= size) {
+        /* Increase the alignment of start */
+        return alignment;
+    } else {
+        /* Find the largest page mask from size */
+        return 1ULL << (63 - clz64(size));
+    }
+}
+
 /* Unmap the whole range in the notifier's scope. */
 static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
 {
-    IOMMUTLBEntry entry;
-    hwaddr size;
+    hwaddr size, remain;
     hwaddr start = n->start;
     hwaddr end = n->end;
     IntelIOMMUState *s = as->iommu_state;
@@ -3388,39 +3405,37 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
     }
 
     assert(start <= end);
-    size = end - start;
+    size = remain = end - start + 1;
 
-    if (ctpop64(size) != 1) {
-        /*
-         * This size cannot format a correct mask. Let's enlarge it to
-         * suite the minimum available mask.
-         */
-        int n = 64 - clz64(size);
-        if (n > s->aw_bits) {
-            /* should not happen, but in case it happens, limit it */
-            n = s->aw_bits;
-        }
-        size = 1ULL << n;
+    while (remain >= VTD_PAGE_SIZE) {
+        IOMMUTLBEntry entry;
+        uint64_t mask = get_naturally_aligned_size(start, remain, s->aw_bits);
+
+        assert(mask);
+
+        entry.iova = start;
+        entry.addr_mask = mask - 1;
+        entry.target_as = &address_space_memory;
+        entry.perm = IOMMU_NONE;
+        /* This field is meaningless for unmap */
+        entry.translated_addr = 0;
+
+        memory_region_notify_one(n, &entry);
+
+        start += mask;
+        remain -= mask;
     }
 
-    entry.target_as = &address_space_memory;
-    /* Adjust iova for the size */
-    entry.iova = n->start & ~(size - 1);
-    /* This field is meaningless for unmap */
-    entry.translated_addr = 0;
-    entry.perm = IOMMU_NONE;
-    entry.addr_mask = size - 1;
+    assert(!remain);
 
     trace_vtd_as_unmap_whole(pci_bus_num(as->bus),
                              VTD_PCI_SLOT(as->devfn),
                              VTD_PCI_FUNC(as->devfn),
-                             entry.iova, size);
+                             n->start, size);
 
-    map.iova = entry.iova;
-    map.size = entry.addr_mask;
+    map.iova = n->start;
+    map.size = size;
     iova_tree_remove(as->iova_tree, &map);
-
-    memory_region_notify_one(n, &entry);
 }
 
 static void vtd_address_space_unmap_all(IntelIOMMUState *s)
Signature · 1 line
2.21.0
#4 lore raw

Hi Peter,

Quoted 28 lines · Peter Xu

This is an replacement work of Yan Zhao's patch:

https://www.mail-archive.com/[email protected]/msg625340.html

vtd_address_space_unmap() will do proper page mask alignment to make sure each IOTLB message will have correct masks for notification messages (2^N-1), but sometimes it can be expanded to even supercede the registered range. That could lead to unexpected UNMAP of already mapped regions in some other notifiers.

Instead of doing mindless expension of the start address and address mask, we split the range into smaller ones and guarantee that each small range will have correct masks (2^N-1) and at the same time we should also try our best to generate as less IOTLB messages as possible.

---
 hw/i386/intel_iommu.c | 67 ++++++++++++++++++++++++++-----------------
 1 file changed, 41 insertions(+), 26 deletions(-)
hw/i386/intel_iommu.c +0 -0
index 719ce19ab3..de86f53b4e 100644
@@ -3363,11 +3363,28 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
     return vtd_dev_as;

hw/i386/intel_iommu.c VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) · line 3373

Quoted 10 lines
hw/i386/intel_iommu.c +8 -0
 }
 
+static uint64_t get_naturally_aligned_size(uint64_t start,
+                                           uint64_t size, int gaw)
+{
+    uint64_t max_mask = 1ULL << gaw;
+    uint64_t alignment = start ? start & -start : max_mask;
+
+    alignment = MIN(alignment, max_mask);
+    size = MIN(size, max_mask);

this does not not prevent from invalidating beyond gaw if start != 0, right?

hw/i386/intel_iommu.c VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) · line 3376

Quoted 3 lines
hw/i386/intel_iommu.c +3 -0
+
+    if (alignment <= size) {
+        /* Increase the alignment of start */

I don't really get this comment

Quoted 24 lines
hw/i386/intel_iommu.c +8 -5
+        return alignment;
+    } else {
+        /* Find the largest page mask from size */
+        return 1ULL << (63 - clz64(size));
+    }> +}
+
 /* Unmap the whole range in the notifier's scope. */
 static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
 {
-    IOMMUTLBEntry entry;
-    hwaddr size;
+    hwaddr size, remain;
     hwaddr start = n->start;
     hwaddr end = n->end;
     IntelIOMMUState *s = as->iommu_state;
@@ -3388,39 +3405,37 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
     }
 
     assert(start <= end);
-    size = end - start;
+    size = remain = end - start + 1;
 
-    if (ctpop64(size) != 1) {
-        /*

hw/i386/intel_iommu.c static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) · line 3410

Quoted 10 lines
hw/i386/intel_iommu.c +1 -9
-         * This size cannot format a correct mask. Let's enlarge it to
-         * suite the minimum available mask.
-         */
-        int n = 64 - clz64(size);
-        if (n > s->aw_bits) {
-            /* should not happen, but in case it happens, limit it */
-            n = s->aw_bits;
-        }
-        size = 1ULL << n;
+    while (remain >= VTD_PAGE_SIZE) {

Can't we stop as soon as entry.iova exceeds gaw as well?

Quoted 34 lines
hw/i386/intel_iommu.c +18 -8
+        IOMMUTLBEntry entry;
+        uint64_t mask = get_naturally_aligned_size(start, remain, s->aw_bits);
+
+        assert(mask);
+
+        entry.iova = start;
+        entry.addr_mask = mask - 1;
+        entry.target_as = &address_space_memory;
+        entry.perm = IOMMU_NONE;
+        /* This field is meaningless for unmap */
+        entry.translated_addr = 0;
+
+        memory_region_notify_one(n, &entry);
+
+        start += mask;
+        remain -= mask;
     }
 
-    entry.target_as = &address_space_memory;
-    /* Adjust iova for the size */
-    entry.iova = n->start & ~(size - 1);
-    /* This field is meaningless for unmap */
-    entry.translated_addr = 0;
-    entry.perm = IOMMU_NONE;
-    entry.addr_mask = size - 1;
+    assert(!remain);
 
     trace_vtd_as_unmap_whole(pci_bus_num(as->bus),
                              VTD_PCI_SLOT(as->devfn),
                              VTD_PCI_FUNC(as->devfn),
-                             entry.iova, size);
+                             n->start, size);
 

hw/i386/intel_iommu.c static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) · line 3441

Quoted 10 lines
hw/i386/intel_iommu.c +2 -4
-    map.iova = entry.iova;
-    map.size = entry.addr_mask;
+    map.iova = n->start;
+    map.size = size;
     iova_tree_remove(as->iova_tree, &map);
-
-    memory_region_notify_one(n, &entry);
 }
 
 static void vtd_address_space_unmap_all(IntelIOMMUState *s)

Thanks

Eric

author
#5 lore raw
Quoted 33 lines · Auger Eric

Hi Peter,

Quoted 30 lines · Peter Xu

This is an replacement work of Yan Zhao's patch:

https://www.mail-archive.com/[email protected]/msg625340.html

vtd_address_space_unmap() will do proper page mask alignment to make sure each IOTLB message will have correct masks for notification messages (2^N-1), but sometimes it can be expanded to even supercede the registered range. That could lead to unexpected UNMAP of already mapped regions in some other notifiers.

Instead of doing mindless expension of the start address and address mask, we split the range into smaller ones and guarantee that each small range will have correct masks (2^N-1) and at the same time we should also try our best to generate as less IOTLB messages as possible.

---
 hw/i386/intel_iommu.c | 67 ++++++++++++++++++++++++++-----------------
 1 file changed, 41 insertions(+), 26 deletions(-)
hw/i386/intel_iommu.c +0 -0
index 719ce19ab3..de86f53b4e 100644
@@ -3363,11 +3363,28 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
     return vtd_dev_as;
 }
 
Quoted 9 lines

hw/i386/intel_iommu.c VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) · line 3373

Quoted 8 lines
hw/i386/intel_iommu.c +8 -0
+static uint64_t get_naturally_aligned_size(uint64_t start,
+                                           uint64_t size, int gaw)
+{
+    uint64_t max_mask = 1ULL << gaw;
+    uint64_t alignment = start ? start & -start : max_mask;
+
+    alignment = MIN(alignment, max_mask);
+    size = MIN(size, max_mask);

this does not not prevent from invalidating beyond gaw if start != 0, right?

Yes. But at the start of vtd_address_space_unmap(), we have:

    if (end > VTD_ADDRESS_SIZE(s->aw_bits) - 1) {
        /*
         * Don't need to unmap regions that is bigger than the whole
         * VT-d supported address space size
         */
        end = VTD_ADDRESS_SIZE(s->aw_bits) - 1;
    }

So we don't need to worry about (start+size) exceeding GAW?

[1]

Quoted 4 lines

hw/i386/intel_iommu.c VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) · line 3376

Quoted 3 lines
hw/i386/intel_iommu.c +3 -0
+
+    if (alignment <= size) {
+        /* Increase the alignment of start */

I don't really get this comment

This comment comes from Paolo, but I'll try to explain - it tries to mean that this "alignment" will be used as an increasement to "start" variable, so finally variable "start" will align with larger mask size.

Better comments welcomed... :)

Quoted 22 lines
Quoted 22 lines
hw/i386/intel_iommu.c +8 -3
+        return alignment;
+    } else {
+        /* Find the largest page mask from size */
+        return 1ULL << (63 - clz64(size));
+    }> +}
+
 /* Unmap the whole range in the notifier's scope. */
 static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
 {
-    IOMMUTLBEntry entry;
-    hwaddr size;
+    hwaddr size, remain;
     hwaddr start = n->start;
     hwaddr end = n->end;
     IntelIOMMUState *s = as->iommu_state;
@@ -3388,39 +3405,37 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
     }
 
     assert(start <= end);
-    size = end - start;
+    size = remain = end - start + 1;
 
Quoted 13 lines

hw/i386/intel_iommu.c static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) · line 3410

Quoted 12 lines
hw/i386/intel_iommu.c +1 -11
-    if (ctpop64(size) != 1) {
-        /*
-         * This size cannot format a correct mask. Let's enlarge it to
-         * suite the minimum available mask.
-         */
-        int n = 64 - clz64(size);
-        if (n > s->aw_bits) {
-            /* should not happen, but in case it happens, limit it */
-            n = s->aw_bits;
-        }
-        size = 1ULL << n;
+    while (remain >= VTD_PAGE_SIZE) {

Can't we stop as soon as entry.iova exceeds gaw as well?

As explained at [1], I think we've already checked it.

Thanks,

Signature · 1 line
Peter Xu
#6 lore raw
Quoted 54 lines · Peter Xu
Quoted 33 lines · Auger Eric

Hi Peter,

Quoted 30 lines · Peter Xu

This is an replacement work of Yan Zhao's patch:

https://www.mail-archive.com/[email protected]/msg625340.html

vtd_address_space_unmap() will do proper page mask alignment to make sure each IOTLB message will have correct masks for notification messages (2^N-1), but sometimes it can be expanded to even supercede the registered range. That could lead to unexpected UNMAP of already mapped regions in some other notifiers.

Instead of doing mindless expension of the start address and address mask, we split the range into smaller ones and guarantee that each small range will have correct masks (2^N-1) and at the same time we should also try our best to generate as less IOTLB messages as possible.

---
 hw/i386/intel_iommu.c | 67 ++++++++++++++++++++++++++-----------------
 1 file changed, 41 insertions(+), 26 deletions(-)
hw/i386/intel_iommu.c +0 -0
index 719ce19ab3..de86f53b4e 100644
@@ -3363,11 +3363,28 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
     return vtd_dev_as;
 }
 
Quoted 9 lines

hw/i386/intel_iommu.c VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) · line 3373

Quoted 8 lines
hw/i386/intel_iommu.c +8 -0
+static uint64_t get_naturally_aligned_size(uint64_t start,
+                                           uint64_t size, int gaw)
+{
+    uint64_t max_mask = 1ULL << gaw;
+    uint64_t alignment = start ? start & -start : max_mask;
+
+    alignment = MIN(alignment, max_mask);
+    size = MIN(size, max_mask);

this does not not prevent from invalidating beyond gaw if start != 0, right?

Yes. But at the start of vtd_address_space_unmap(), we have:

    if (end > VTD_ADDRESS_SIZE(s->aw_bits) - 1) {
        /*
         * Don't need to unmap regions that is bigger than the whole
         * VT-d supported address space size
         */
        end = VTD_ADDRESS_SIZE(s->aw_bits) - 1;
    }
Quoted 1 line

So we don't need to worry about (start+size) exceeding GAW?

Hum yes. Reviewed the previous patch with blinkers ...

Quoted 6 lines

[1]

Quoted 3 lines
Quoted 3 lines
hw/i386/intel_iommu.c +3 -0
+
+    if (alignment <= size) {
+        /* Increase the alignment of start */
Quoted 8 lines
Quoted 1 line

I don't really get this comment

This comment comes from Paolo, but I'll try to explain - it tries to mean that this "alignment" will be used as an increasement to "start" variable, so finally variable "start" will align with larger mask size.

Better comments welcomed... :)

smallest page mask from @start or gaw?

Quoted 37 lines
Quoted 23 lines
Quoted 23 lines
hw/i386/intel_iommu.c +8 -3
+        return alignment;
+    } else {
+        /* Find the largest page mask from size */
+        return 1ULL << (63 - clz64(size));
+    }> +}
+
 /* Unmap the whole range in the notifier's scope. */
 static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
 {
-    IOMMUTLBEntry entry;
-    hwaddr size;
+    hwaddr size, remain;
     hwaddr start = n->start;
     hwaddr end = n->end;
     IntelIOMMUState *s = as->iommu_state;
@@ -3388,39 +3405,37 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
     }
 
     assert(start <= end);
-    size = end - start;
+    size = remain = end - start + 1;
 
Quoted 13 lines

hw/i386/intel_iommu.c static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) · line 3410

Quoted 12 lines
hw/i386/intel_iommu.c +1 -11
-    if (ctpop64(size) != 1) {
-        /*
-         * This size cannot format a correct mask. Let's enlarge it to
-         * suite the minimum available mask.
-         */
-        int n = 64 - clz64(size);
-        if (n > s->aw_bits) {
-            /* should not happen, but in case it happens, limit it */
-            n = s->aw_bits;
-        }
-        size = 1ULL << n;
+    while (remain >= VTD_PAGE_SIZE) {

Can't we stop as soon as entry.iova exceeds gaw as well?

Quoted 1 line

As explained at [1], I think we've already checked it.

OK

Thanks

Eric

Quoted 3 lines

Thanks,

#7 lore raw
Quoted 10 lines · Auger Eric
Quoted 9 lines
Quoted 3 lines

hw/i386/intel_iommu.c VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn) · line 3376

Quoted 2 lines
hw/i386/intel_iommu.c +2 -0
+    if (alignment <= size) {
+        /* Increase the alignment of start */

I don't really get this comment

This comment comes from Paolo, but I'll try to explain - it tries to mean that this "alignment" will be used as an increasement to "start" variable, so finally variable "start" will align with larger mask size.

Better comments welcomed... :)

smallest page mask from @start or gaw?

What it means is that there will be "more 0 bits" at the beginning of start. Perhaps "On the next iteration start will be aligned to a bigger power of two"? I can do this when applying.

Paolo

#8 lore raw
In reply to Peter Xu
Quoted 110 lines · Peter Xu

This is an replacement work of Yan Zhao's patch:

https://www.mail-archive.com/[email protected]/msg625340.html

vtd_address_space_unmap() will do proper page mask alignment to make sure each IOTLB message will have correct masks for notification messages (2^N-1), but sometimes it can be expanded to even supercede the registered range. That could lead to unexpected UNMAP of already mapped regions in some other notifiers.

Instead of doing mindless expension of the start address and address mask, we split the range into smaller ones and guarantee that each small range will have correct masks (2^N-1) and at the same time we should also try our best to generate as less IOTLB messages as possible.

---
 hw/i386/intel_iommu.c | 67 ++++++++++++++++++++++++++-----------------
 1 file changed, 41 insertions(+), 26 deletions(-)
hw/i386/intel_iommu.c +39 -22
index 719ce19ab3..de86f53b4e 100644
@@ -3363,11 +3363,28 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
     return vtd_dev_as;
 }
 
+static uint64_t get_naturally_aligned_size(uint64_t start,
+                                           uint64_t size, int gaw)
+{
+    uint64_t max_mask = 1ULL << gaw;
+    uint64_t alignment = start ? start & -start : max_mask;
+
+    alignment = MIN(alignment, max_mask);
+    size = MIN(size, max_mask);
+
+    if (alignment <= size) {
+        /* Increase the alignment of start */
+        return alignment;
+    } else {
+        /* Find the largest page mask from size */
+        return 1ULL << (63 - clz64(size));
+    }
+}
+
 /* Unmap the whole range in the notifier's scope. */
 static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
 {
-    IOMMUTLBEntry entry;
-    hwaddr size;
+    hwaddr size, remain;
     hwaddr start = n->start;
     hwaddr end = n->end;
     IntelIOMMUState *s = as->iommu_state;
@@ -3388,39 +3405,37 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
     }
 
     assert(start <= end);
-    size = end - start;
+    size = remain = end - start + 1;
 
-    if (ctpop64(size) != 1) {
-        /*
-         * This size cannot format a correct mask. Let's enlarge it to
-         * suite the minimum available mask.
-         */
-        int n = 64 - clz64(size);
-        if (n > s->aw_bits) {
-            /* should not happen, but in case it happens, limit it */
-            n = s->aw_bits;
-        }
-        size = 1ULL << n;
+    while (remain >= VTD_PAGE_SIZE) {
+        IOMMUTLBEntry entry;
+        uint64_t mask = get_naturally_aligned_size(start, remain, s->aw_bits);
+
+        assert(mask);
+
+        entry.iova = start;
+        entry.addr_mask = mask - 1;
+        entry.target_as = &address_space_memory;
+        entry.perm = IOMMU_NONE;
+        /* This field is meaningless for unmap */
+        entry.translated_addr = 0;
+
+        memory_region_notify_one(n, &entry);
+
+        start += mask;
+        remain -= mask;
     }
 
-    entry.target_as = &address_space_memory;
-    /* Adjust iova for the size */
-    entry.iova = n->start & ~(size - 1);
-    /* This field is meaningless for unmap */
-    entry.translated_addr = 0;
-    entry.perm = IOMMU_NONE;
-    entry.addr_mask = size - 1;
+    assert(!remain);
 
     trace_vtd_as_unmap_whole(pci_bus_num(as->bus),
                              VTD_PCI_SLOT(as->devfn),
                              VTD_PCI_FUNC(as->devfn),
-                             entry.iova, size);
+                             n->start, size);
 

hw/i386/intel_iommu.c static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) · line 3441

Quoted 10 lines
hw/i386/intel_iommu.c +2 -4
-    map.iova = entry.iova;
-    map.size = entry.addr_mask;
+    map.iova = n->start;
+    map.size = size;
     iova_tree_remove(as->iova_tree, &map);
-
-    memory_region_notify_one(n, &entry);
 }
 
 static void vtd_address_space_unmap_all(IntelIOMMUState *s)

Looks good, ignore my previous message.

Paolo

Tested-by on v2 2/2
#9 lore raw
In reply to Peter Xu
Quoted 122 lines · Peter Xu

This is an replacement work of Yan Zhao's patch:

https://www.mail-archive.com/[email protected]/msg625340.html

vtd_address_space_unmap() will do proper page mask alignment to make sure each IOTLB message will have correct masks for notification messages (2^N-1), but sometimes it can be expanded to even supercede the registered range. That could lead to unexpected UNMAP of already mapped regions in some other notifiers.

Instead of doing mindless expension of the start address and address mask, we split the range into smaller ones and guarantee that each small range will have correct masks (2^N-1) and at the same time we should also try our best to generate as less IOTLB messages as possible.

---
 hw/i386/intel_iommu.c | 67 ++++++++++++++++++++++++++-----------------
 1 file changed, 41 insertions(+), 26 deletions(-)
hw/i386/intel_iommu.c +41 -26
index 719ce19ab3..de86f53b4e 100644
@@ -3363,11 +3363,28 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
     return vtd_dev_as;
 }
 
+static uint64_t get_naturally_aligned_size(uint64_t start,
+                                           uint64_t size, int gaw)
+{
+    uint64_t max_mask = 1ULL << gaw;
+    uint64_t alignment = start ? start & -start : max_mask;
+
+    alignment = MIN(alignment, max_mask);
+    size = MIN(size, max_mask);
+
+    if (alignment <= size) {
+        /* Increase the alignment of start */
+        return alignment;
+    } else {
+        /* Find the largest page mask from size */
+        return 1ULL << (63 - clz64(size));
+    }
+}
+
 /* Unmap the whole range in the notifier's scope. */
 static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
 {
-    IOMMUTLBEntry entry;
-    hwaddr size;
+    hwaddr size, remain;
     hwaddr start = n->start;
     hwaddr end = n->end;
     IntelIOMMUState *s = as->iommu_state;
@@ -3388,39 +3405,37 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
     }
 
     assert(start <= end);
-    size = end - start;
+    size = remain = end - start + 1;
 
-    if (ctpop64(size) != 1) {
-        /*
-         * This size cannot format a correct mask. Let's enlarge it to
-         * suite the minimum available mask.
-         */
-        int n = 64 - clz64(size);
-        if (n > s->aw_bits) {
-            /* should not happen, but in case it happens, limit it */
-            n = s->aw_bits;
-        }
-        size = 1ULL << n;
+    while (remain >= VTD_PAGE_SIZE) {
+        IOMMUTLBEntry entry;
+        uint64_t mask = get_naturally_aligned_size(start, remain, s->aw_bits);
+
+        assert(mask);
+
+        entry.iova = start;
+        entry.addr_mask = mask - 1;
+        entry.target_as = &address_space_memory;
+        entry.perm = IOMMU_NONE;
+        /* This field is meaningless for unmap */
+        entry.translated_addr = 0;
+
+        memory_region_notify_one(n, &entry);
+
+        start += mask;
+        remain -= mask;
     }
 
-    entry.target_as = &address_space_memory;
-    /* Adjust iova for the size */
-    entry.iova = n->start & ~(size - 1);
-    /* This field is meaningless for unmap */
-    entry.translated_addr = 0;
-    entry.perm = IOMMU_NONE;
-    entry.addr_mask = size - 1;
+    assert(!remain);
 
     trace_vtd_as_unmap_whole(pci_bus_num(as->bus),
                              VTD_PCI_SLOT(as->devfn),
                              VTD_PCI_FUNC(as->devfn),
-                             entry.iova, size);
+                             n->start, size);
 
-    map.iova = entry.iova;
-    map.size = entry.addr_mask;
+    map.iova = n->start;
+    map.size = size;
     iova_tree_remove(as->iova_tree, &map);
-
-    memory_region_notify_one(n, &entry);
 }
 
 static void vtd_address_space_unmap_all(IntelIOMMUState *s)
Signature · 1 line
2.21.0
Reviewed-by on v2 2/2
#10 lore raw
In reply to Peter Xu

Hi Peter,

Quoted 10 lines · Peter Xu

This is an replacement work of Yan Zhao's patch:

https://www.mail-archive.com/[email protected]/msg625340.html

vtd_address_space_unmap() will do proper page mask alignment to make sure each IOTLB message will have correct masks for notification messages (2^N-1), but sometimes it can be expanded to even supercede the registered range. That could lead to unexpected UNMAP of already mapped regions in some other notifiers.

Quoted 8 lines

Instead of doing mindless expension of the start address and address mask, we split the range into smaller ones and guarantee that each small range will have correct masks (2^N-1) and at the same time we should also try our best to generate as less IOTLB messages as possible.

Thanks

Eric

Quoted 102 lines
---
 hw/i386/intel_iommu.c | 67 ++++++++++++++++++++++++++-----------------
 1 file changed, 41 insertions(+), 26 deletions(-)
hw/i386/intel_iommu.c +41 -26
index 719ce19ab3..de86f53b4e 100644
@@ -3363,11 +3363,28 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
     return vtd_dev_as;
 }
 
+static uint64_t get_naturally_aligned_size(uint64_t start,
+                                           uint64_t size, int gaw)
+{
+    uint64_t max_mask = 1ULL << gaw;
+    uint64_t alignment = start ? start & -start : max_mask;
+
+    alignment = MIN(alignment, max_mask);
+    size = MIN(size, max_mask);
+
+    if (alignment <= size) {
+        /* Increase the alignment of start */
+        return alignment;
+    } else {
+        /* Find the largest page mask from size */
+        return 1ULL << (63 - clz64(size));
+    }
+}
+
 /* Unmap the whole range in the notifier's scope. */
 static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
 {
-    IOMMUTLBEntry entry;
-    hwaddr size;
+    hwaddr size, remain;
     hwaddr start = n->start;
     hwaddr end = n->end;
     IntelIOMMUState *s = as->iommu_state;
@@ -3388,39 +3405,37 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
     }
 
     assert(start <= end);
-    size = end - start;
+    size = remain = end - start + 1;
 
-    if (ctpop64(size) != 1) {
-        /*
-         * This size cannot format a correct mask. Let's enlarge it to
-         * suite the minimum available mask.
-         */
-        int n = 64 - clz64(size);
-        if (n > s->aw_bits) {
-            /* should not happen, but in case it happens, limit it */
-            n = s->aw_bits;
-        }
-        size = 1ULL << n;
+    while (remain >= VTD_PAGE_SIZE) {
+        IOMMUTLBEntry entry;
+        uint64_t mask = get_naturally_aligned_size(start, remain, s->aw_bits);
+
+        assert(mask);
+
+        entry.iova = start;
+        entry.addr_mask = mask - 1;
+        entry.target_as = &address_space_memory;
+        entry.perm = IOMMU_NONE;
+        /* This field is meaningless for unmap */
+        entry.translated_addr = 0;
+
+        memory_region_notify_one(n, &entry);
+
+        start += mask;
+        remain -= mask;
     }
 
-    entry.target_as = &address_space_memory;
-    /* Adjust iova for the size */
-    entry.iova = n->start & ~(size - 1);
-    /* This field is meaningless for unmap */
-    entry.translated_addr = 0;
-    entry.perm = IOMMU_NONE;
-    entry.addr_mask = size - 1;
+    assert(!remain);
 
     trace_vtd_as_unmap_whole(pci_bus_num(as->bus),
                              VTD_PCI_SLOT(as->devfn),
                              VTD_PCI_FUNC(as->devfn),
-                             entry.iova, size);
+                             n->start, size);
 
-    map.iova = entry.iova;
-    map.size = entry.addr_mask;
+    map.iova = n->start;
+    map.size = size;
     iova_tree_remove(as->iova_tree, &map);
-
-    memory_region_notify_one(n, &entry);
 }
 
 static void vtd_address_space_unmap_all(IntelIOMMUState *s)
#11 lore raw
In reply to Peter Xu
Quoted 8 lines · Peter Xu

This is an replacement work of Yan Zhao's patch:

https://www.mail-archive.com/[email protected]/msg625340.html

vtd_address_space_unmap() will do proper page mask alignment to make sure each IOTLB message will have correct masks for notification messages (2^N-1), but sometimes it can be expanded to even supercede the registered range.

I wonder under what circumstance that could we meet this?

Thanks

Quoted 112 lines

That could lead to unexpected UNMAP of already
mapped regions in some other notifiers.

Instead of doing mindless expension of the start address and address mask, we split the range into smaller ones and guarantee that each small range will have correct masks (2^N-1) and at the same time we should also try our best to generate as less IOTLB messages as possible.

---
  hw/i386/intel_iommu.c | 67 ++++++++++++++++++++++++++-----------------
  1 file changed, 41 insertions(+), 26 deletions(-)
hw/i386/intel_iommu.c +41 -26
index 719ce19ab3..de86f53b4e 100644
@@ -3363,11 +3363,28 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
      return vtd_dev_as;
  }
 
+static uint64_t get_naturally_aligned_size(uint64_t start,
+                                           uint64_t size, int gaw)
+{
+    uint64_t max_mask = 1ULL << gaw;
+    uint64_t alignment = start ? start & -start : max_mask;
+
+    alignment = MIN(alignment, max_mask);
+    size = MIN(size, max_mask);
+
+    if (alignment <= size) {
+        /* Increase the alignment of start */
+        return alignment;
+    } else {
+        /* Find the largest page mask from size */
+        return 1ULL << (63 - clz64(size));
+    }
+}
+
  /* Unmap the whole range in the notifier's scope. */
  static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
  {
-    IOMMUTLBEntry entry;
-    hwaddr size;
+    hwaddr size, remain;
      hwaddr start = n->start;
      hwaddr end = n->end;
      IntelIOMMUState *s = as->iommu_state;
@@ -3388,39 +3405,37 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n)
      }
 
      assert(start <= end);
-    size = end - start;
+    size = remain = end - start + 1;
 
-    if (ctpop64(size) != 1) {
-        /*
-         * This size cannot format a correct mask. Let's enlarge it to
-         * suite the minimum available mask.
-         */
-        int n = 64 - clz64(size);
-        if (n > s->aw_bits) {
-            /* should not happen, but in case it happens, limit it */
-            n = s->aw_bits;
-        }
-        size = 1ULL << n;
+    while (remain >= VTD_PAGE_SIZE) {
+        IOMMUTLBEntry entry;
+        uint64_t mask = get_naturally_aligned_size(start, remain, s->aw_bits);
+
+        assert(mask);
+
+        entry.iova = start;
+        entry.addr_mask = mask - 1;
+        entry.target_as = &address_space_memory;
+        entry.perm = IOMMU_NONE;
+        /* This field is meaningless for unmap */
+        entry.translated_addr = 0;
+
+        memory_region_notify_one(n, &entry);
+
+        start += mask;
+        remain -= mask;
      }
 
-    entry.target_as = &address_space_memory;
-    /* Adjust iova for the size */
-    entry.iova = n->start & ~(size - 1);
-    /* This field is meaningless for unmap */
-    entry.translated_addr = 0;
-    entry.perm = IOMMU_NONE;
-    entry.addr_mask = size - 1;
+    assert(!remain);
 
      trace_vtd_as_unmap_whole(pci_bus_num(as->bus),
                               VTD_PCI_SLOT(as->devfn),
                               VTD_PCI_FUNC(as->devfn),
-                             entry.iova, size);
+                             n->start, size);
 
-    map.iova = entry.iova;
-    map.size = entry.addr_mask;
+    map.iova = n->start;
+    map.size = size;
      iova_tree_remove(as->iova_tree, &map);
-
-    memory_region_notify_one(n, &entry);
  }
 
  static void vtd_address_space_unmap_all(IntelIOMMUState *s)
author
#12 lore raw
Quoted 6 lines · Jason Wang
Quoted 4 lines · Peter Xu

This is an replacement work of Yan Zhao's patch:

https://www.mail-archive.com/[email protected]/msg625340.html

Quoted 7 lines
Quoted 4 lines

vtd_address_space_unmap() will do proper page mask alignment to make sure each IOTLB message will have correct masks for notification messages (2^N-1), but sometimes it can be expanded to even supercede the registered range.

I wonder under what circumstance that could we meet this?

Sorry I forgot to reply-all just now...

I've asked a similar question, and Yan's answer is here:

https://www.mail-archive.com/[email protected]/msg625597.html

Regards,

Signature · 1 line
Peter Xu
Reviewed-by on the v2 series
#13 lore raw
In reply to Peter Xu
Quoted 7 lines · Peter Xu

v2:
- rename helper to get_naturally_aligned_size(), simplify the codes as suggested [Paolo]
- check against vtd page size when looping over for unmaps [Yan]
- add r-b for Eric

Please review, thanks.

Series:

Quoted 11 lines

Peter Xu (1):
intel_iommu: Fix unexpected unmaps during global unmap

Yan Zhao (1):
intel_iommu: Fix incorrect "end" for vtd_address_space_unmap

 hw/i386/intel_iommu.c | 71 ++++++++++++++++++++++++++-----------------
 1 file changed, 43 insertions(+), 28 deletions(-)
Signature · 1 line
2.21.0