Skip to contentSkip to conversation

Fixing kvm_iommu_put_pages()

PATCH David Woodhouse 2 participants 5 messages Mar 12, 2014 – Mar 17, 2014 4 messages not in the archive KVM

lore
Share & tools
lore
#0 lore raw

In kvm_iommu_put_pages() we use this bizarre and horrid API for iommu_unmap() where we call it to unmap PAGE_SIZE at a time, but if it actually finds a larger page than that in its page tables then it's allowed to unmap and return the larger size.

For some IOMMUs including the Intel one, breaking unmaps into tiny chunks like this is *really* pessimal.

It looks like the actual reason it's done this way is because we need to unpin the underlying physical pages when we're done with them, and the only way we can do that is by doing it a page at a time calling iommu_iova_to_phys() on each page.

I wonder if this could be better handled by passing a callback function in to the iommu_unmap() function, which can be called for each physical address range that's unmapped.

That way, we wouldn't need all the cache and IOTLB flushing between the individual page unmaps; we could still just do it the quick way.

It would look something like this...

virt/kvm/iommu.c +1 -28
index 0df7d4b..bf3f4f9 100644
@@ -278,41 +278,14 @@ static void kvm_iommu_put_pages(struct kvm *kvm,
 				gfn_t base_gfn, unsigned long npages)
 {
 	struct iommu_domain *domain;
-	gfn_t end_gfn, gfn;
-	pfn_t pfn;
-	u64 phys;
 
 	domain  = kvm->arch.iommu_domain;
-	end_gfn = base_gfn + npages;
-	gfn     = base_gfn;
 
 	/* check if iommu exists and in use */
 	if (!domain)
 		return;
 
-	while (gfn < end_gfn) {
-		unsigned long unmap_pages;
-		size_t size;
-
-		/* Get physical address */
-		phys = iommu_iova_to_phys(domain, gfn_to_gpa(gfn));
-
-		if (!phys) {
-			gfn++;
-			continue;
-		}
-
-		pfn  = phys >> PAGE_SHIFT;
-
-		/* Unmap address from IO address space */
-		size       = iommu_unmap(domain, gfn_to_gpa(gfn), PAGE_SIZE);
-		unmap_pages = 1ULL << get_order(size);
-
-		/* Unpin all pages we just unmapped to not leak any memory */
-		kvm_unpin_pages(kvm, pfn, unmap_pages);
-
-		gfn += unmap_pages;
-	}
+	iommu_unmap(domain, gfn_to_gpa(base_gfn), npages << PAGE_SHIFT, kvm_unpin_pages, kvm);
 }
 
 void kvm_iommu_unmap_pages(struct kvm *kvm, struct kvm_memory_slot *slot)
Signature · 2 lines
David Woodhouse                            Open Source Technology Centre
[email protected]                              Intel Corporation

A message that is not in the archive (sent off-list or not yet received): 1394585481.2270.179.camel-Fexsq3y4057IgHVZqg5X0TlWvGAXklZc@public.gmane.org

#2 lore raw
Quoted 60 lines · David Woodhouse

In kvm_iommu_put_pages() we use this bizarre and horrid API for iommu_unmap() where we call it to unmap PAGE_SIZE at a time, but if it actually finds a larger page than that in its page tables then it's allowed to unmap and return the larger size.

For some IOMMUs including the Intel one, breaking unmaps into tiny chunks like this is *really* pessimal.

It looks like the actual reason it's done this way is because we need to unpin the underlying physical pages when we're done with them, and the only way we can do that is by doing it a page at a time calling iommu_iova_to_phys() on each page.

I wonder if this could be better handled by passing a callback function in to the iommu_unmap() function, which can be called for each physical address range that's unmapped.

That way, we wouldn't need all the cache and IOTLB flushing between the individual page unmaps; we could still just do it the quick way.

It would look something like this...

virt/kvm/iommu.c +0 -22
index 0df7d4b..bf3f4f9 100644
@@ -278,41 +278,14 @@ static void kvm_iommu_put_pages(struct kvm *kvm,
 				gfn_t base_gfn, unsigned long npages)
 {
 	struct iommu_domain *domain;
-	gfn_t end_gfn, gfn;
-	pfn_t pfn;
-	u64 phys;
 
 	domain  = kvm->arch.iommu_domain;
-	end_gfn = base_gfn + npages;
-	gfn     = base_gfn;
 
 	/* check if iommu exists and in use */
 	if (!domain)
 		return;
 
-	while (gfn < end_gfn) {
-		unsigned long unmap_pages;
-		size_t size;
-
-		/* Get physical address */
-		phys = iommu_iova_to_phys(domain, gfn_to_gpa(gfn));
-
-		if (!phys) {
-			gfn++;
-			continue;
-		}
-
-		pfn  = phys >> PAGE_SHIFT;
-
-		/* Unmap address from IO address space */
-		size       = iommu_unmap(domain, gfn_to_gpa(gfn), PAGE_SIZE);
-		unmap_pages = 1ULL << get_order(size);

virt/kvm/iommu.c static void kvm_iommu_put_pages(struct kvm *kvm, · line 291

Quoted 10 lines
virt/kvm/iommu.c +1 -6
-
-		/* Unpin all pages we just unmapped to not leak any memory */
-		kvm_unpin_pages(kvm, pfn, unmap_pages);
-
-		gfn += unmap_pages;
-	}
+	iommu_unmap(domain, gfn_to_gpa(base_gfn), npages << PAGE_SHIFT, kvm_unpin_pages, kvm);
 }
 
 void kvm_iommu_unmap_pages(struct kvm *kvm, struct kvm_memory_slot *slot)

As mentioned on irc, I think this could probably be made to work and doing a callback would likely give the IOMMU driver the most flexibility versus some sort of interface for iommu_iova_to_phys() to return the mapping size. Also note that KVM device assignment is being phased out in favor of VFIO-based device assignment. vfio_unmap_unpin() has a similar loop where we attempt to unmap a PAGE_SIZE chunk and unpin based on the actual size unmapped. Thanks,

Alex

A message that is not in the archive (sent off-list or not yet received): 1395071061.8201.14.camel-85EaTFmN5p//[email protected]

#4 lore raw
Quoted 6 lines · Alex Williamson

As mentioned on irc, I think this could probably be made to work and doing a callback would likely give the IOMMU driver the most flexibility versus some sort of interface for iommu_iova_to_phys() to return the mapping size.

Is it acceptable to have an API which says that the physical address may *still* be accessible through the IOTLB until iommu_unmap() returns? Even *after* the callback has been called for that physical address?

We have the physical addresses to hand as we clear the PTEs, of course. But we haven't flushed the IOTLB yet. It would be painful to keep a *list* of pages and then call the callback after the IOTLB is flushed; we'd prefer to punt that cost to the caller, if the caller needs to worry about use-after-free problems.

If that's too much of a pain, then maybe augmenting iova_to_phys() might be the better option...?

Signature · 1 line
dwmw2

A message that is not in the archive (sent off-list or not yet received): 1395071417.4588.45.camel-W2I5cNIroUsVm/[email protected]

#6 lore raw
Quoted 12 lines · David Woodhouse
Quoted 6 lines · Alex Williamson

As mentioned on irc, I think this could probably be made to work and doing a callback would likely give the IOMMU driver the most flexibility versus some sort of interface for iommu_iova_to_phys() to return the mapping size.

Is it acceptable to have an API which says that the physical address may *still* be accessible through the IOTLB until iommu_unmap() returns? Even *after* the callback has been called for that physical address?

Quoted 8 lines

We have the physical addresses to hand as we clear the PTEs, of course. But we haven't flushed the IOTLB yet. It would be painful to keep a *list* of pages and then call the callback after the IOTLB is flushed; we'd prefer to punt that cost to the caller, if the caller needs to worry about use-after-free problems.

If that's too much of a pain, then maybe augmenting iova_to_phys() might be the better option...?

In order to maintain the trivial callback you suggest, the IOTLB would need to guarantee that the physical addresses are not accessible prior to the callback. Without such a guarantee, we can't unpin the pages and the callback wouldn't be able to do anything more than collect a list of pages to later unpin. For reducing IOTLB flushes maybe that still makes sense, but it does seem like it could easily get ugly. For instance, the callback would need the option to return -errno if it runs out of resources, so the IOMMU driver would need the ability to reinstate the last chunk.

It's starting to seem like augmenting iova_to_phys() might be the better path. It fixes the API and allows for a trivial (if sub-optimal) implementation like we have today. An optimal implementation would require looping through iova_to_phys() for the entire range to be unmapped, caching the translation and size of each, calling iommu_unmap() once for the entire range, then using the cached values to unpin physical memory. At least it's easier for the caller to bound it's resource consumption that way. Thanks,

Alex

A message that is not in the archive (sent off-list or not yet received): 1395075072.8201.58.camel-85EaTFmN5p//[email protected]

#8 lore raw
Quoted 10 lines · Alex Williamson

In order to maintain the trivial callback you suggest, the IOTLB would need to guarantee that the physical addresses are not accessible prior to the callback. Without such a guarantee, we can't unpin the pages and the callback wouldn't be able to do anything more than collect a list of pages to later unpin. For reducing IOTLB flushes maybe that still makes sense, but it does seem like it could easily get ugly. For instance, the callback would need the option to return -errno if it runs out of resources, so the IOMMU driver would need the ability to reinstate the last chunk.

Well, unless they're your own pages and you can (ab)use pg->freelist to make a list of them without having to actually do any allocations?

But yeah, that's the API that's on offer from the IOMMU side I think. You get to choose if it's acceptable to you or not. If not, we need to revert to the simpler iommu_iova_to_phys() option.

Signature · 1 line
dwmw2