Fixing kvm_iommu_put_pages()
In kvm_iommu_put_pages() we use this bizarre and horrid API for iommu_unmap() where we call it to unmap PAGE_SIZE at a time, but if it actually finds a larger page than that in its page tables then it's allowed to unmap and return the larger size.
For some IOMMUs including the Intel one, breaking unmaps into tiny chunks like this is *really* pessimal.
It looks like the actual reason it's done this way is because we need to unpin the underlying physical pages when we're done with them, and the only way we can do that is by doing it a page at a time calling iommu_iova_to_phys() on each page.
I wonder if this could be better handled by passing a callback function in to the iommu_unmap() function, which can be called for each physical address range that's unmapped.
That way, we wouldn't need all the cache and IOTLB flushing between the individual page unmaps; we could still just do it the quick way.
It would look something like this...
virt/kvm/iommu.c +1 -28
@@ -278,41 +278,14 @@ static void kvm_iommu_put_pages(struct kvm *kvm, gfn_t base_gfn, unsigned long npages) {struct iommu_domain *domain; -gfn_t end_gfn, gfn; -pfn_t pfn; -u64 phys; domain = kvm->arch.iommu_domain; - end_gfn = base_gfn + npages; - gfn = base_gfn;/* check if iommu exists and in use */ if (!domain)return ; -while (gfn < end_gfn) { -unsigned long unmap_pages; -size_t size; - -/* Get physical address */ - phys =iommu_iova_to_phys (domain,gfn_to_gpa (gfn)); - -if (!phys) { - gfn++; -continue ; - } - - pfn = phys >>PAGE_SHIFT ; - -/* Unmap address from IO address space */ - size =iommu_unmap (domain,gfn_to_gpa (gfn),PAGE_SIZE ); - unmap_pages =1ULL <<get_order (size); - -/* Unpin all pages we just unmapped to not leak any memory */ -kvm_unpin_pages (kvm, pfn, unmap_pages); - - gfn += unmap_pages; - } +iommu_unmap (domain,gfn_to_gpa (base_gfn), npages <<PAGE_SHIFT , kvm_unpin_pages, kvm); }void kvm_iommu_unmap_pages (struct kvm *kvm,struct kvm_memory_slot *slot)
Signature · 2 lines
David Woodhouse Open Source Technology Centre [email protected] Intel Corporation
A message that is not in the archive (sent off-list or not yet received): 1394585481.2270.179.camel-Fexsq3y4057IgHVZqg5X0TlWvGAXklZc@public.gmane.org
Quoted 60 lines · David Woodhouse
In
kvm_iommu_put_pages()we use this bizarre and horrid API foriommu_unmap()where we call it to unmapPAGE_SIZEat a time, but if it actually finds a larger page than that in its page tables then it's allowed to unmap and return the larger size.For some IOMMUs including the Intel one, breaking unmaps into tiny chunks like this is *really* pessimal.
It looks like the actual reason it's done this way is because we need to unpin the underlying physical pages when we're done with them, and the only way we can do that is by doing it a page at a time calling
iommu_iova_to_phys()on each page.I wonder if this could be better handled by passing a callback function in to the
iommu_unmap()function, which can be called for each physical address range that's unmapped.That way, we wouldn't need all the cache and IOTLB flushing between the individual page unmaps; we could still just do it the quick way.
It would look something like this...
virt/kvm/iommu.c +0
-22@@ -278,41 +278,14 @@ static void kvm_iommu_put_pages(struct kvm *kvm, gfn_t base_gfn, unsigned long npages) {struct iommu_domain *domain; -gfn_t end_gfn, gfn; -pfn_t pfn; -u64 phys; domain = kvm->arch.iommu_domain; - end_gfn = base_gfn + npages; - gfn = base_gfn;/* check if iommu exists and in use */ if (!domain)return ; -while (gfn < end_gfn) { -unsigned long unmap_pages; -size_t size; - -/* Get physical address */ - phys =iommu_iova_to_phys (domain,gfn_to_gpa (gfn)); - -if (!phys) { - gfn++; -continue ; - } - - pfn = phys >>PAGE_SHIFT ; - -/* Unmap address from IO address space */ - size =iommu_unmap (domain,gfn_to_gpa (gfn),PAGE_SIZE ); - unmap_pages =1ULL <<get_order (size);
virt/kvm/iommu.c static void kvm_iommu_put_pages(struct kvm *kvm, · line 291
Quoted 10 lines
virt/kvm/iommu.c +1
-6- -/* Unpin all pages we just unmapped to not leak any memory */ -kvm_unpin_pages (kvm, pfn, unmap_pages); - - gfn += unmap_pages; - } +iommu_unmap (domain,gfn_to_gpa (base_gfn), npages <<PAGE_SHIFT , kvm_unpin_pages, kvm); }void kvm_iommu_unmap_pages (struct kvm *kvm,struct kvm_memory_slot *slot)
As mentioned on irc, I think this could probably be made to work and doing a callback would likely give the IOMMU driver the most flexibility versus some sort of interface for iommu_iova_to_phys() to return the mapping size. Also note that KVM device assignment is being phased out in favor of VFIO-based device assignment. vfio_unmap_unpin() has a similar loop where we attempt to unmap a PAGE_SIZE chunk and unpin based on the actual size unmapped. Thanks,
Alex
A message that is not in the archive (sent off-list or not yet received): 1395071061.8201.14.camel-85EaTFmN5p//[email protected]
Quoted 6 lines · Alex Williamson
As mentioned on irc, I think this could probably be made to work and doing a callback would likely give the IOMMU driver the most flexibility versus some sort of interface for
iommu_iova_to_phys()to return the mapping size.
Is it acceptable to have an API which says that the physical address may *still* be accessible through the IOTLB until iommu_unmap() returns? Even *after* the callback has been called for that physical address?
We have the physical addresses to hand as we clear the PTEs, of course. But we haven't flushed the IOTLB yet. It would be painful to keep a *list* of pages and then call the callback after the IOTLB is flushed; we'd prefer to punt that cost to the caller, if the caller needs to worry about use-after-free problems.
If that's too much of a pain, then maybe augmenting iova_to_phys() might be the better option...?
Signature · 1 line
dwmw2
A message that is not in the archive (sent off-list or not yet received): 1395071417.4588.45.camel-W2I5cNIroUsVm/[email protected]
Quoted 12 lines · David Woodhouse
Quoted 6 lines · Alex Williamson
As mentioned on irc, I think this could probably be made to work and doing a callback would likely give the IOMMU driver the most flexibility versus some sort of interface for
iommu_iova_to_phys()to return the mapping size.Is it acceptable to have an API which says that the physical address may *still* be accessible through the IOTLB until
iommu_unmap()returns? Even *after* the callback has been called for that physical address?
Quoted 8 lines
We have the physical addresses to hand as we clear the PTEs, of course. But we haven't flushed the IOTLB yet. It would be painful to keep a *list* of pages and then call the callback after the IOTLB is flushed; we'd prefer to punt that cost to the caller, if the caller needs to worry about use-after-free problems.
If that's too much of a pain, then maybe augmenting
iova_to_phys()might be the better option...?
In order to maintain the trivial callback you suggest, the IOTLB would need to guarantee that the physical addresses are not accessible prior to the callback. Without such a guarantee, we can't unpin the pages and the callback wouldn't be able to do anything more than collect a list of pages to later unpin. For reducing IOTLB flushes maybe that still makes sense, but it does seem like it could easily get ugly. For instance, the callback would need the option to return -errno if it runs out of resources, so the IOMMU driver would need the ability to reinstate the last chunk.
It's starting to seem like augmenting iova_to_phys() might be the better path. It fixes the API and allows for a trivial (if sub-optimal) implementation like we have today. An optimal implementation would require looping through iova_to_phys() for the entire range to be unmapped, caching the translation and size of each, calling iommu_unmap() once for the entire range, then using the cached values to unpin physical memory. At least it's easier for the caller to bound it's resource consumption that way. Thanks,
Alex
A message that is not in the archive (sent off-list or not yet received): 1395075072.8201.58.camel-85EaTFmN5p//[email protected]
Quoted 10 lines · Alex Williamson
In order to maintain the trivial callback you suggest, the IOTLB would need to guarantee that the physical addresses are not accessible prior to the callback. Without such a guarantee, we can't unpin the pages and the callback wouldn't be able to do anything more than collect a list of pages to later unpin. For reducing IOTLB flushes maybe that still makes sense, but it does seem like it could easily get ugly. For instance, the callback would need the option to return -errno if it runs out of resources, so the IOMMU driver would need the ability to reinstate the last chunk.
Well, unless they're your own pages and you can (ab)use pg->freelist to make a list of them without having to actually do any allocations?
But yeah, that's the API that's on offer from the IOMMU side I think. You get to choose if it's acceptable to you or not. If not, we need to revert to the simpler iommu_iova_to_phys() option.
Signature · 1 line
dwmw2