iommufd: Allow mapping from KVM's guest_memfd
Subsystems (4)
RFC PATCH 1/1 iommufd: Allow mapping from KVM's guest_memfd
CoCo VMs get their private memory allocated from guest_memfd ("gmemfd") which is a KVM facility similar to memfd. The gmemfds does not allow mapping private memory to the userspace so the IOMMU_IOAS_MAP ioctl does not work.
Use the existing IOMMU_IOAS_MAP_FILE ioctl to allow mapping from fd + offset. Detect the gmemfd case in pfn_reader_user_pin().
For the new guest_memfd type, no additional reference is taken as pinning is guaranteed by the KVM guest_memfd library.
There is no KVM-GMEMFD->IOMMUFD direct notification mechanism as the assumption is that:
1) page stage change events will be handled by VMM which is going to call IOMMUFD to remap pages;
2) shrinking GMEMFD equals to VM memory unplug and VMM is going to handle it.
---
This is for Trusted IO == TEE-IO == PCIe TDISP, etc.
Previously posted here:
https://lore.kernel.org/r/[email protected]
The main comment was "what is the lifetime of those folios()" and GMEMFD + QEMU should take care of it.
And horrendous stuff like this is not really useful:
https://github.com/AMDESE/linux-kvm/commit/7d73fd2cccb8489b1
--- include/linux/kvm_host.h | 4 + drivers/iommu/iommufd/pages.c | 80 ++++++++++++++++++-- virt/kvm/guest_memfd.c | 36 +++++++++ 3 files changed, 113 insertions(+), 7 deletions(-)
include/linux/kvm_host.h +4 -0
@@ -2673,4 +2673,8 @@ unsigned long kvm_get_vm_memory_attributes(struct kvm *kvm, gfn_t gfn);int kvm_vm_ioctl_set_mem_attributes (struct kvm *kvm,struct kvm_memory_attributes2 *attrs); +bool kvm_is_gmemfd (struct file *file); +struct folio *kvm_gmemfd_get_pfn (struct file *file,unsigned long index, +unsigned long *pfn,int *max_order); +#endif
drivers/iommu/iommufd/pages.c +73 -7
128 lines not on this page · raw message
virt/kvm/guest_memfd.c +36 -0
@@ -1794,3 +1794,39 @@ void kvm_gmem_exit(void)rcu_barrier ();kmem_cache_destroy (kvm_gmem_inode_cachep); } + +bool kvm_is_gmemfd (struct file *file) +{ +if (!file) +return false; + +if (file->f_op != &kvm_gmem_fops) +return false; + +return true; +} +EXPORT_SYMBOL_GPL (kvm_is_gmemfd); + +struct folio *kvm_gmemfd_get_pfn (struct file *file,unsigned long index, +unsigned long *pfn,int *max_order) +{ +struct inode *inode =file_inode (file); +struct folio *folio; + +if (!inode || !kvm_is_gmemfd (file)) +return NULL ; + + folio =kvm_gmem_get_folio (inode, index); +if (!folio) +return NULL ; + + + *pfn =folio_pfn (folio) + (index & (folio_nr_pages (folio) -1 )); + *max_order =folio_order (folio); + +folio_put (folio); +folio_unlock (folio); + +return folio; +} +EXPORT_SYMBOL_GPL (kvm_gmemfd_get_pfn);
Signature · 1 line
2.52.0
Sean Christopherson
maintainerQuoted 9 lines · Alexey Kardashevskiy
For the new
guest_memfdtype, no additional reference is taken as pinning is guaranteed by the KVMguest_memfdlibrary.There is no KVM-GMEMFD->IOMMUFD direct notification mechanism as the assumption is that:
1) page stage change events will be handled by VMM which is going to call IOMMUFD to remap pages;
2) shrinking GMEMFD equals to VM memory unplug and VMM is going to handle it.
The VMM is outside of the kernel's effective TCB. Assuming the VMM will always do the right thing is a non-starter.
Quoted 11 lines · Sean Christopherson
Quoted 9 lines · Alexey Kardashevskiy
For the new
guest_memfdtype, no additional reference is taken as pinning is guaranteed by the KVMguest_memfdlibrary.There is no KVM-GMEMFD->IOMMUFD direct notification mechanism as the assumption is that:
1) page stage change events will be handled by VMM which is going to call IOMMUFD to remap pages;
2) shrinking GMEMFD equals to VM memory unplug and VMM is going to handle it.
Quoted 2 lines
The VMM is outside of the kernel's effective TCB. Assuming the VMM will always do the right thing is a non-starter.
Right.
But, say, for 1), VMM does not the right thing and skips on PSC - the AMD host will observe IOMMU fault events - noisy but harmless. I wonder if it is different for others though.
Truncating gmemfd is bad, is having gmemfd->iommufd notification going to be enough for a starter? Thanks,
Signature · 1 line
Alexey
Quoted 14 lines · Alexey Kardashevskiy
Quoted 11 lines · Sean Christopherson
Quoted 9 lines · Alexey Kardashevskiy
For the new
guest_memfdtype, no additional reference is taken as pinning is guaranteed by the KVMguest_memfdlibrary.There is no KVM-GMEMFD->IOMMUFD direct notification mechanism as the assumption is that:
1) page stage change events will be handled by VMM which is going to call IOMMUFD to remap pages;
2) shrinking GMEMFD equals to VM memory unplug and VMM is going to handle it.
Quoted 8 lines
Quoted 2 lines
The VMM is outside of the kernel's effective TCB. Assuming the VMM will always do the right thing is a non-starter.
Right.
But, say, for 1), VMM does not the right thing and skips on PSC - the AMD host will observe IOMMU fault events - noisy but harmless. I wonder if it is different for others though.
ARM is also supposed to be safe as GPT faults are contained, IIRC.
However, it is not like AMD in many important ways here. Critically ARM has a split guest physical space where the low addresses are all private and the upper addresses are all shared.
Thus on Linux the iommu should be programed with the shared pages mapped into the shared address range. It would be wasteful to program it with large amounts of IOPTEs that are already know to be private.
I think if you are fully doing in-place conversion then you could program the entire shared address range to point to the memory pool (eg with 1G huge pages) and rely entirely on the GPT to arbitrate access. I don't think that is implemented in Linux though?
While on AMD, IIRC, the iommu should be programed with both the shared and private pages in the respective GPA locations, but due to the RMP matching insanity you have to keep restructuring the IOPTEs to exactly match the RMP layout.
I have no idea what Intel needs.
Jason
Quoted 18 lines · Jason Gunthorpe
Quoted 17 lines · Alexey Kardashevskiy
Quoted 13 lines · Sean Christopherson
Quoted 9 lines · Alexey Kardashevskiy
For the new
guest_memfdtype, no additional reference is taken as pinning is guaranteed by the KVMguest_memfdlibrary.There is no KVM-GMEMFD->IOMMUFD direct notification mechanism as the assumption is that:
1) page stage change events will be handled by VMM which is going to call IOMMUFD to remap pages;
2) shrinking GMEMFD equals to VM memory unplug and VMM is going to handle it.The VMM is outside of the kernel's effective TCB. Assuming the VMM will always do the right thing is a non-starter.
Quoted 7 lines
Quoted 5 lines
Right.
But, say, for 1), VMM does not the right thing and skips on PSC - the AMD host will observe IOMMU fault events - noisy but harmless. I wonder if it is different for others though.
ARM is also supposed to be safe as GPT faults are contained, IIRC.
Intel TDX will cause host machine check and restart, which are not contained.
Quoted 4 lines
However, it is not like AMD in many important ways here. Critically ARM has a split guest physical space where the low addresses are all private and the upper addresses are all shared.
This is same as Intel TDX, the GPA shared bit are used by IOMMU to target shared/private. You can imagine for T=1, there are 2 IOPTs, or 1 IOPT with all private at lower address & all shared at higher address.
Quoted 4 lines
Thus on Linux the iommu should be programed with the shared pages mapped into the shared address range. It would be wasteful to program it with large amounts of IOPTEs that are already know to be private.
For Intel TDX, it is not just a waste, the redundant IOMMU mappings are dangerous.
Quoted 6 lines
I think if you are fully doing in-place conversion then you could program the entire shared address range to point to the memory pool (eg with 1G huge pages) and rely entirely on the GPT to arbitrate access. I don't think that is implemented in Linux though?
Quoted 6 lines
While on AMD, IIRC, the iommu should be programed with both the shared and private pages in the respective GPA locations, but due to the RMP matching insanity you have to keep restructuring the IOPTEs to exactly match the RMP layout.
I have no idea what Intel needs.
Secure part of IOPT (lower address) reuses KVM MMU (S-EPT) so needs no extra update but needs a global IOTLB flush. The Shared part of IOPT for T=1 needs update based on GPA.
Quoted 2 lines
Jason
Quoted 11 lines · Sean Christopherson
Quoted 9 lines · Alexey Kardashevskiy
For the new
guest_memfdtype, no additional reference is taken as pinning is guaranteed by the KVMguest_memfdlibrary.There is no KVM-GMEMFD->IOMMUFD direct notification mechanism as the assumption is that:
1) page stage change events will be handled by VMM which is going to call IOMMUFD to remap pages;
2) shrinking GMEMFD equals to VM memory unplug and VMM is going to handle it.
Quoted 2 lines
The VMM is outside of the kernel's effective TCB. Assuming the VMM will always do the right thing is a non-starter.
I think looking up the guest_memfd file from the userspace address (uptr) is a good start, and in order not to assume much of the userspace VMM, we could register the mapping with guest_memfd, so that when there's a conversion or truncation, guest_memfd will invalidate the registered mapping in addition to the rest of the mappings being invalidated.
At LPC (2025) [1][2], people pointed out that needing to force unmapping during page state changes (aka conversions) are a TDX-only issue. It seems like on SNP and ARM, the faults generated due to the host accessing guest private memory can be caught and handled, so it's not super terrible if there's no unmapping during conversions. Perhaps Alexey and Aneesh can explain more :)
Will said pKVM actually would rather not unmap from the IOMMU on conversions.
I didn't think of this before LPC but forcing unmapping during truncation (aka shrinking guest_memfd) is probably necessary for overall system stability and correctness, so notifying and having guest_memfd track where its pages were mapped in the IOMMU is necessary. Whether or not to unmap during conversions could be a arch-specific thing, but all architectures would want the memory unmapped if the memory is removed from guest_memfd ownership.
[1] Slides: https://lpc.events/event/19/contributions/2184/attachments/1752/3816/2025-12-12-lpc-coco-mc-optimizing-guest-memfd-conversions.pdf
[2] Notes: https://github.com/joergroedel/coco-microconference/blob/main/2025/optimizing_guest_memfd_shared_private_conversions.md
Quoted 13 lines · Ackerley Tng
Quoted 11 lines · Sean Christopherson
Quoted 9 lines · Alexey Kardashevskiy
For the new
guest_memfdtype, no additional reference is taken as pinning is guaranteed by the KVMguest_memfdlibrary.There is no KVM-GMEMFD->IOMMUFD direct notification mechanism as the assumption is that:
1) page stage change events will be handled by VMM which is going to call IOMMUFD to remap pages;
2) shrinking GMEMFD equals to VM memory unplug and VMM is going to handle it.
Quoted 5 lines
Quoted 2 lines
The VMM is outside of the kernel's effective TCB. Assuming the VMM will always do the right thing is a non-starter.
I think looking up the
guest_memfdfile from the userspace address (uptr) is a good start
Please no, if we need complicated things like notifiers then it is better to start directly with the struct file interface and get immediately into some guestmemfd API instead of trying to get their from a VMA. A VMA doesn't help in any way and just complicates things.
Quoted 7 lines
I didn't think of this before LPC but forcing unmapping during truncation (aka shrinking
guest_memfd) is probably necessary for overall system stability and correctness, so notifying and havingguest_memfdtrack where its pages were mapped in the IOMMU is necessary. Whether or not to unmap during conversions could be a arch-specific thing, but all architectures would want the memory unmapped if the memory is removed fromguest_memfdownership.
Things like truncate are a bit easier to handle, you do need a protective notifier, but if it detects truncate while an iommufd area still covers the truncated region it can just revoke the whole area. Userspace made a mistake and gets burned but the kernel is safe. We don't need something complicated kernel side to automatically handle removing just the slice of truncated guestmemfd, for example.
If guestmemfd is fully pinned and cannot free memory outside of truncate that may be good enough (though somehow I think that is not the case) - and I don't understand what issues Intel has with iommu access.
Jason
Sean Christopherson
maintainerQuoted 17 lines · Jason Gunthorpe
Quoted 16 lines · Ackerley Tng
Quoted 13 lines · Sean Christopherson
Quoted 9 lines · Alexey Kardashevskiy
For the new
guest_memfdtype, no additional reference is taken as pinning is guaranteed by the KVMguest_memfdlibrary.There is no KVM-GMEMFD->IOMMUFD direct notification mechanism as the assumption is that:
1) page stage change events will be handled by VMM which is going to call IOMMUFD to remap pages;
2) shrinking GMEMFD equals to VM memory unplug and VMM is going to handle it.The VMM is outside of the kernel's effective TCB. Assuming the VMM will always do the right thing is a non-starter.
Quoted 7 lines
Quoted 2 lines
I think looking up the
guest_memfdfile from the userspace address (uptr) is a good startPlease no, if we need complicated things like notifiers then it is better to start directly with the struct file interface and get immediately into some guestmemfd API instead of trying to get their from a VMA. A VMA doesn't help in any way and just complicates things.
+1000. Anything that _requires_ a VMA to do something with guest_memfd is broken by design.
Quoted 8 lines
Quoted 7 lines
I didn't think of this before LPC but forcing unmapping during truncation (aka shrinking
guest_memfd) is probably necessary for overall system stability and correctness, so notifying and havingguest_memfdtrack where its pages were mapped in the IOMMU is necessary. Whether or not to unmap during conversions could be a arch-specific thing, but all architectures would want the memory unmapped if the memory is removed fromguest_memfdownership.
Quoted 6 lines
Things like truncate are a bit easier to handle, you do need a protective notifier, but if it detects truncate while an iommufd area still covers the truncated region it can just revoke the whole area. Userspace made a mistake and gets burned but the kernel is safe. We don't need something complicated kernel side to automatically handle removing just the slice of truncated guestmemfd, for example.
Yeah, as long as the behavior is well-documented from time zero, we can probably get away with fairly draconian behavior.
Quoted 3 lines
If guestmemfd is fully pinned and cannot free memory outside of truncate that may be good enough (though somehow I think that is not the case)
With in-place conversion, PUNCH_HOLE and private=>shared conversions are the only two ways to partial "remove" memory from guest_memfd, so it may really be that simple.
Quoted 7 lines · Sean Christopherson
Quoted 3 lines
If guestmemfd is fully pinned and cannot free memory outside of truncate that may be good enough (though somehow I think that is not the case)
With in-place conversion,
PUNCH_HOLEand private=>shared conversions are the only two ways to partial "remove" memory fromguest_memfd, so it may really be that simple.
PUNCH_HOLE can be treated like truncate right?
I'm confused though - I thought in-place conversion ment that private<->shared re-used the existing memory allocation? Why does it "remove" memory?
Or perhaps more broadly, where is the shared memory kept/accessed in these guest memfd systems?
Jason
Sean Christopherson
maintainerQuoted 11 lines · Jason Gunthorpe
Quoted 7 lines · Sean Christopherson
Quoted 3 lines
If guestmemfd is fully pinned and cannot free memory outside of truncate that may be good enough (though somehow I think that is not the case)
With in-place conversion,
PUNCH_HOLEand private=>shared conversions are the only two ways to partial "remove" memory fromguest_memfd, so it may really be that simple.
PUNCH_HOLEcan be treated like truncate right?
Yep. Tomato, tomato. I called out PUNCH_HOLE because guest_memfd doesn't support a pure truncate, the size is immutable (ignoring that destroying the inode is kinda sorta a truncate).
Quoted 6 lines
I'm confused though - I thought in-place conversion ment that private<->shared re-used the existing memory allocation? Why does it "remove" memory?
Or perhaps more broadly, where is the shared memory kept/accessed in these guest memfd systems?
Oh, the physical memory doesn't change, but the IOMMU might care that memory is being converted from private<=>shared. AMD IOMMU probably doesn't? But unless Intel IOMMU reuses S-EPT from the VM itself, the IOMMU page tables will need to be updated.
FWIW, conceptually, we're basically treating private=>shared in particular as "free() + alloc()" that just so happens to guarantee the allocated page is the same.
Quoted 11 lines · Sean Christopherson
Quoted 6 lines
I'm confused though - I thought in-place conversion ment that private<->shared re-used the existing memory allocation? Why does it "remove" memory?
Or perhaps more broadly, where is the shared memory kept/accessed in these guest memfd systems?
Oh, the physical memory doesn't change, but the IOMMU might care that memory is being converted from private<=>shared. AMD IOMMU probably doesn't? But unless Intel IOMMU reuses S-EPT from the VM itself, the IOMMU page tables will need to be updated.
Okay, so then it is probably OK for AMD and ARM to just let shared/private happen and whatever userspace does or doesn't do is not important. The IOPTE will point at guaranteed allocated memory and any faults caused by imporerly putting private in a shared slot will be contained.
I have no idea what happens to Intel if the shared IOMMU points to a private page? The machine catches fire and daemons spawn from a fissure?
Or maybe we are lucky and it generates a nice contained fault like the other two so we don't need to build something elaborate and special to make up for horrible hardware? Pretty please?
Jason
Quoted 11 lines · Jason Gunthorpe
Quoted 10 lines · Sean Christopherson
Quoted 6 lines
I'm confused though - I thought in-place conversion ment that private<->shared re-used the existing memory allocation? Why does it "remove" memory?
Or perhaps more broadly, where is the shared memory kept/accessed in these guest memfd systems?
Oh, the physical memory doesn't change, but the IOMMU might care that memory is being converted from private<=>shared. AMD IOMMU probably doesn't? But unless Intel IOMMU reuses S-EPT from the VM itself, the IOMMU page tables will need to
Intel secure IOMMU does reuse S-EPT, but that doesn't mean IOMMU mapping stay still, at least IOTLB needs flush.
Quoted 11 lines
Quoted 1 line
be updated.
Okay, so then it is probably OK for AMD and ARM to just let shared/private happen and whatever userspace does or doesn't do is not important. The IOPTE will point at guaranteed allocated memory and any faults caused by imporerly putting private in a shared slot will be contained.
I have no idea what happens to Intel if the shared IOMMU points to a private page? The machine catches fire and daemons spawn from a fissure?
Will cause host machine check and host restart, same as host CPU accessing encrypted memory. Intel TDX has no lower level privilege protection table so the wrong accessing will actually impact the memory encryption engine.
Quoted 6 lines
Or maybe we are lucky and it generates a nice contained fault like the other two so we don't need to build something elaborate and special to make up for horrible hardware? Pretty please?
Jason
Quoted 4 lines · Xu Yilun
Will cause host machine check and host restart, same as host CPU accessing encrypted memory. Intel TDX has no lower level privilege protection table so the wrong accessing will actually impact the memory encryption engine.
Blah, of course it does.
So Intel needs a two step synchronization to wipe the IOPTEs before any shared private conversions and restore the right ones after.
AMD needs a nasty HW synchronization with RMP changes, but otherwise wants to map the entire physical space.
ARM doesn't care much, I think it could safely do either approach?
These are very different behaviors so I would expect that userspace needs to signal which of the two it wants.
It feels like we need a fairly complex dedicated synchronization logic in iommufd coupled to the shared/private machinery in guestmemfd
Not really sure how to implement the Intel version right now, it is sort of like a nasty version of SVA..
Jason
Quoted 11 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Will cause host machine check and host restart, same as host CPU accessing encrypted memory. Intel TDX has no lower level privilege protection table so the wrong accessing will actually impact the memory encryption engine.
Blah, of course it does.
So Intel needs a two step synchronization to wipe the IOPTEs before any shared private conversions and restore the right ones after.
Mainly about shared IOPTE (for both T=0 table & T=1 table): "unmap before conversion to private" & "map after conversion to shared"
I see there are already some consideration in QEMU to support in-place conversion + shared passthrough [*], using uptr, but seems that's exactly what you are objecting to.
[*]: https://lore.kernel.org/all/[email protected]/
For Intel, T=1 private IOPTE reuses S-EPT, this is the real CC business and the correctness is managed by KVM & firmware, no notification needed.
Further more, I think "unmap shared IOPTE before conversion to private" may be the only concern to ensure kernel safety, other steps could be fully left to userspace. Hope the downgrading from "remap" to "invalidate" simplifies the notification.
Quoted 16 lines
AMD needs a nasty HW synchronization with RMP changes, but otherwise wants to map the entire physical space.
ARM doesn't care much, I think it could safely do either approach?
These are very different behaviors so I would expect that userspace needs to signal which of the two it wants.
It feels like we need a fairly complex dedicated synchronization logic in iommufd coupled to the shared/private machinery in guestmemfd
Not really sure how to implement the Intel version right now, it is sort of like a nasty version of SVA..
Jason
Quoted 13 lines · Xu Yilun
Quoted 11 lines · Jason Gunthorpe
Quoted 4 lines · Xu Yilun
Will cause host machine check and host restart, same as host CPU accessing encrypted memory. Intel TDX has no lower level privilege protection table so the wrong accessing will actually impact the memory encryption engine.
Blah, of course it does.
So Intel needs a two step synchronization to wipe the IOPTEs before any shared private conversions and restore the right ones after.
Quoted 6 lines
Mainly about shared IOPTE (for both T=0 table & T=1 table): "unmap before conversion to private" & "map after conversion to shared"
I see there are already some consideration in QEMU to support in-place conversion + shared passthrough [*], using uptr, but seems that's exactly what you are objecting to
There is some ugly stuff in qemu trying to make this work with VFIO..
Quoted 4 lines
Further more, I think "unmap shared IOPTE before conversion to private" may be the only concern to ensure kernel safety, other steps could be fully left to userspace. Hope the downgrading from "remap" to "invalidate" simplifies the notification.
Maybe, but there is still the large issue of how to deal with fragmenting the mapping and breaking/re-consolidating huge pages, which is not trivial..
To really make this work well we may need iommufd to actively mirror the guestmemfd into IOPTEs and dynamically track changes.
I will think about it..
Jason