Skip to contentSkip to conversation

KVM: Restricted mapping of guest_memfd at the host and pKVM/arm64 support

RFC PATCH Superseded Fuad Tabba 4 participants 21 messages Aug 1, 2024 – Sep 3, 2024

Subsystems (7)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Fuad Tabba

author
#0 lore raw

v2 cover KVM: Restricted mapping of guest_memfd at the host and pKVM/arm64 support

This series adds restricted mmap() support to guest_memfd, as well as support for guest_memfd on pKVM/arm64. It is based on Linux 6.10.

Main changes since V1 [1]:

- Decoupled whether guest memory is mappable from KVM memory attributes (SeanC)

Mappability is now tracked in the guest_mem object, orthogonal to whether userspace wants the memory to be private or shared. Moreover, the memory attributes capability (i.e., KVM_CAP_MEMORY_ATTRIBUTES) is not enabled for pKVM, since for software-based hypervisors such as pKVM and Gunyah, userspace is informed of the state of the memory via hypervisor exits if needed.

Even if attributes are enabled, this patch series would still work (modulo bugs), without compromising guest memory nor crashing the system.

- Use page_mapped() instead of page_mapcount() to check if page is mapped (DavidH)

- Add a new capability, KVM_CAP_GUEST_MEMFD_MAPPABLE, to query whether guest private memory can be mapped (with aforementioned restrictions)

- Add a selftest to check whether memory is mappable when the capability is enabled, and not mappable otherwise. Also, test the effect of punching holes in mapped memory. (DavidH)

By design, guest_memfd cannot be mapped, read, or written by the host. In pKVM, memory shared between a protected guest and the host is shared in-place, unlike the other confidential computing solutions that guest_memfd was originally envisaged for (e.g, TDX). When initializing a guest, as well as when accessing memory shared by the guest with the host, it would be useful to support mapping that memory at the host to avoid copying its contents.

One of the benefits of guest_memfd is that it prevents a misbehaving host process from crashing the system when attempting to access (deliberately or accidentally) protected guest memory, since this memory isn't mapped to begin with. Without guest_memfd, the hypervisor would still prevent such accesses, but in certain cases the host kernel wouldn't be able to recover, causing the system to crash.

Support for mmap() in this patch series maintains the invariant that only memory shared with the host, either explicitly by the guest or implicitly before the guest has started running (in order to populate its memory) is allowed to have a valid mapping at the host. At no time should private (as viewed by the hypervisor) guest memory be mapped at the host.

This patch series is divided into two parts:

The first part is to the KVM core code. It adds opt-in support for mapping guest memory only as long as it is shared. For that, the host needs to know the mappability status of guest memory. Therefore, the series adds a structure to track whether memory is mappable. This new structure is associated with each guest_memfd object.

The second part of the series adds guest_memfd support for pKVM/arm64.

We don't enforce the invariant that only memory shared with the host can be mapped by the host userspace in file_operations::mmap(), but we enforce it in vm_operations_struct:fault(). On vm_operations_struct::fault(), we check whether the page is allowed to be mapped. If not, we deliver a SIGBUS to the current task, as discussed in the Linux MM Alignment Session on this topic [2].

Currently there's no support for huge pages, which is something we hope to add in the future, and seems to be a hot topic for the upcoming LPC 2024 [3].

Cheers,
/fuad

[1] https://lore.kernel.org/all/[email protected]/

[2] https://lore.kernel.org/all/[email protected]/

[3] https://lpc.events/event/18/sessions/183/#20240919

Fuad Tabba (10):
KVM: Introduce kvm_gmem_get_pfn_locked(), which retains the folio lock
KVM: Add restricted support for mapping guestmem by the host
KVM: Implement kvm_(read|/write)_guest_page for private memory slots
KVM: Add KVM capability to check if guest_memfd can be mapped by the
host
KVM: selftests: guest_memfd mmap() test when mapping is allowed
KVM: arm64: Skip VMA checks for slots without userspace address
KVM: arm64: Do not allow changes to private memory slots
KVM: arm64: Handle guest_memfd()-backed guest page faults
KVM: arm64: arm64 has private memory support when config is enabled
KVM: arm64: Enable private memory kconfig for arm64

 arch/arm64/include/asm/kvm_host.h             |   3 +
 arch/arm64/kvm/Kconfig                        |   1 +
 arch/arm64/kvm/mmu.c                          | 139 +++++++++-
 include/linux/kvm_host.h                      |  72 +++++
 include/uapi/linux/kvm.h                      |   3 +-
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../testing/selftests/kvm/guest_memfd_test.c  |  47 +++-
 virt/kvm/Kconfig                              |   4 +
 virt/kvm/guest_memfd.c                        | 129 ++++++++-
 virt/kvm/kvm_main.c                           | 253 ++++++++++++++++--
 10 files changed, 628 insertions(+), 24 deletions(-)

base-commit: 0c3836482481200ead7b416ca80c68a29cfdaabd

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
author
#1 lore raw

RFC PATCH v2 1/10 KVM: Introduce kvm_gmem_get_pfn_locked(), which retains the folio lock

Create a new variant of kvm_gmem_get_pfn(), which retains the folio lock if it returns successfully.

---
 include/linux/kvm_host.h | 11 +++++++++++
 virt/kvm/guest_memfd.c   | 19 ++++++++++++++++---
 2 files changed, 27 insertions(+), 3 deletions(-)
include/linux/kvm_host.h +11 -0
index 692c01e41a18..43a157f8171a 100644
@@ -2431,6 +2431,8 @@ static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn)
 #ifdef CONFIG_KVM_PRIVATE_MEM
 int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 		     gfn_t gfn, kvm_pfn_t *pfn, int *max_order);
+int kvm_gmem_get_pfn_locked(struct kvm *kvm, struct kvm_memory_slot *slot,
+			      gfn_t gfn, kvm_pfn_t *pfn, int *max_order);
 #else
 static inline int kvm_gmem_get_pfn(struct kvm *kvm,
 				   struct kvm_memory_slot *slot, gfn_t gfn,
@@ -2439,6 +2441,15 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm,
 	KVM_BUG_ON(1, kvm);
 	return -EIO;
 }
+
+static inline int kvm_gmem_get_pfn_locked(struct kvm *kvm,
+					  struct kvm_memory_slot *slot,
+					  gfn_t gfn, kvm_pfn_t *pfn,
+					  int *max_order)
+{
+	KVM_BUG_ON(1, kvm);
+	return -EIO;
+}
 #endif /* CONFIG_KVM_PRIVATE_MEM */
 
 #endif
virt/kvm/guest_memfd.c +16 -3
index 747fe251e445..f3f4334a9ccb 100644
@@ -482,8 +482,8 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot)
 	fput(file);
 }
 
-int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
-		     gfn_t gfn, kvm_pfn_t *pfn, int *max_order)
+int kvm_gmem_get_pfn_locked(struct kvm *kvm, struct kvm_memory_slot *slot,
+			    gfn_t gfn, kvm_pfn_t *pfn, int *max_order)
 {
 	pgoff_t index = gfn - slot->base_gfn + slot->gmem.pgoff;
 	struct kvm_gmem *gmem;
@@ -524,10 +524,23 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
 
 	r = 0;
 
-	folio_unlock(folio);
 out_fput:
 	fput(file);
 
 	return r;
 }
+EXPORT_SYMBOL_GPL(kvm_gmem_get_pfn_locked);
+
+int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot,
+		     gfn_t gfn, kvm_pfn_t *pfn, int *max_order)
+{
+	int r;
+
+	r = kvm_gmem_get_pfn_locked(kvm, slot, gfn, pfn, max_order);
+	if (r)
+		return r;
+
+	unlock_page(pfn_to_page(*pfn));
+	return 0;
+}
 EXPORT_SYMBOL_GPL(kvm_gmem_get_pfn);
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
author
#2 lore raw

RFC PATCH v2 2/10 KVM: Add restricted support for mapping guestmem by the host

In reply to Fuad Tabba

Add support for mmap() and fault() for guest_memfd in the host. The ability to fault in a guest page is contingent on that page being shared with the host. To track this, this patch adds a new xarray to each guest_memfd object, which tracks the mappability of guest frames.

The guest_memfd PRIVATE memory attribute is not used for two reasons. First because it reflects the userspace expectation for that memory location, and therefore can be toggled by userspace. The second is, although each guest_memfd file has a 1:1 binding with a KVM instance, the plan is to allow multiple files per inode, e.g. to allow intra-host migration to a new KVM instance, without destroying guest_memfd.

This new feature is gated with a new configuration option, CONFIG_KVM_PRIVATE_MEM_MAPPABLE.

---
 include/linux/kvm_host.h |  61 ++++++++++++++++++++
 virt/kvm/Kconfig         |   4 ++
 virt/kvm/guest_memfd.c   | 110 +++++++++++++++++++++++++++++++++++
 virt/kvm/kvm_main.c      | 122 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 297 insertions(+)
include/linux/kvm_host.h +61 -0
index 43a157f8171a..ab1344327e57 100644
@@ -2452,4 +2452,65 @@ static inline int kvm_gmem_get_pfn_locked(struct kvm *kvm,
 }
 #endif /* CONFIG_KVM_PRIVATE_MEM */
 
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+bool kvm_gmem_is_mappable(struct kvm *kvm, gfn_t gfn, gfn_t end);
+bool kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end);
+int kvm_gmem_set_mappable(struct kvm *kvm, gfn_t start, gfn_t end);
+int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end);
+int kvm_slot_gmem_toggle_mappable(struct kvm_memory_slot *slot, gfn_t start,
+				  gfn_t end, bool is_mappable);
+int kvm_slot_gmem_set_mappable(struct kvm_memory_slot *slot, gfn_t start,
+			       gfn_t end);
+int kvm_slot_gmem_clear_mappable(struct kvm_memory_slot *slot, gfn_t start,
+				 gfn_t end);
+bool kvm_slot_gmem_is_mappable(struct kvm_memory_slot *slot, gfn_t gfn);
+#else
+static inline bool kvm_gmem_is_mappable(struct kvm *kvm, gfn_t gfn, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return false;
+}
+static inline bool kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return false;
+}
+static inline int kvm_gmem_set_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start,
+					  gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline int kvm_slot_gmem_toggle_mappable(struct kvm_memory_slot *slot,
+						gfn_t start, gfn_t end,
+						bool is_mappable)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline int kvm_slot_gmem_set_mappable(struct kvm_memory_slot *slot,
+					     gfn_t start, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline int kvm_slot_gmem_clear_mappable(struct kvm_memory_slot *slot,
+					       gfn_t start, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline bool kvm_slot_gmem_is_mappable(struct kvm_memory_slot *slot,
+					     gfn_t gfn)
+{
+	WARN_ON_ONCE(1);
+	return false;
+}
+#endif /* CONFIG_KVM_PRIVATE_MEM_MAPPABLE */
+
 #endif
virt/kvm/Kconfig +4 -0
index 29b73eedfe74..a3970c5eca7b 100644
@@ -109,3 +109,7 @@ config KVM_GENERIC_PRIVATE_MEM
        select KVM_GENERIC_MEMORY_ATTRIBUTES
        select KVM_PRIVATE_MEM
        bool
+
+config KVM_PRIVATE_MEM_MAPPABLE
+       select KVM_PRIVATE_MEM
+       bool
virt/kvm/guest_memfd.c +110 -0

135 lines not on this page · raw message

virt/kvm/kvm_main.c +122 -0

128 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#3 lore raw

Re: [RFC PATCH v2 02/10] KVM: Add restricted support for mapping guestmem by the host

Quoted 9 lines · Fuad Tabba

Add support for mmap() and fault() for guest_memfd in the host. The ability to fault in a guest page is contingent on that page being shared with the host. To track this, this patch adds a new xarray to each guest_memfd object, which tracks the mappability of guest frames.

The guest_memfd PRIVATE memory attribute is not used for two reasons. First because it reflects the userspace expectation for that memory location, and therefore can be toggled by userspace.

Thank you for clarifying why the PRIVATE memory attribute cannot be used. I think that having the attributes with guest_memfd is a good idea.

Since faultability is a property of the memory contents, shouldn't faultability be stored on the inode rather than the file?

Quoted 4 lines

The second is, although each guest_memfd file has a 1:1 binding with a KVM instance, the plan is to allow multiple files per inode, e.g. to allow intra-host migration to a new KVM instance, without destroying guest_memfd.

I think you also alluded to the concept of inodes vs files above.

To store the xarray on the inode, we would probably have to make guest_memfd use its own mount so that we can use .evict_inode() from struct address_space_operations to clean up the inode neatly, perhaps the way it was done in guest_memfd v12 [1].

This RFC to enable intra-host migration [2] was built with the version of guest_memfd that used its own mount. IIUC, the gmem struct stored on the file is meant to be the binding between struct kvm and the memory contents [3], so the gmem struct won't be transferred and a new gmem struct will be created.

Quoted 17 lines

This new feature is gated with a new configuration option, CONFIG_KVM_PRIVATE_MEM_MAPPABLE.

---
 include/linux/kvm_host.h |  61 ++++++++++++++++++++
 virt/kvm/Kconfig         |   4 ++
 virt/kvm/guest_memfd.c   | 110 +++++++++++++++++++++++++++++++++++
 virt/kvm/kvm_main.c      | 122 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 297 insertions(+)
include/linux/kvm_host.h +0 -0
index 43a157f8171a..ab1344327e57 100644

include/linux/kvm_host.h static inline int kvm_gmem_get_pfn_locked(struct kvm *kvm, · line 2458

Quoted 7 lines
include/linux/kvm_host.h +4 -0
@@ -2452,4 +2452,65 @@ static inline int kvm_gmem_get_pfn_locked(struct kvm *kvm,
 }
 #endif /* CONFIG_KVM_PRIVATE_MEM */
 
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+bool kvm_gmem_is_mappable(struct kvm *kvm, gfn_t gfn, gfn_t end);
+bool kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end);
+int kvm_gmem_set_mappable(struct kvm *kvm, gfn_t start, gfn_t end);

How will kvm_gmem_is_mapped() and kvm_gmem_set_mappable() be used?

Quoted 97 lines
include/linux/kvm_host.h +57 -0
+int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end);
+int kvm_slot_gmem_toggle_mappable(struct kvm_memory_slot *slot, gfn_t start,
+				  gfn_t end, bool is_mappable);
+int kvm_slot_gmem_set_mappable(struct kvm_memory_slot *slot, gfn_t start,
+			       gfn_t end);
+int kvm_slot_gmem_clear_mappable(struct kvm_memory_slot *slot, gfn_t start,
+				 gfn_t end);
+bool kvm_slot_gmem_is_mappable(struct kvm_memory_slot *slot, gfn_t gfn);
+#else
+static inline bool kvm_gmem_is_mappable(struct kvm *kvm, gfn_t gfn, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return false;
+}
+static inline bool kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return false;
+}
+static inline int kvm_gmem_set_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start,
+					  gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline int kvm_slot_gmem_toggle_mappable(struct kvm_memory_slot *slot,
+						gfn_t start, gfn_t end,
+						bool is_mappable)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline int kvm_slot_gmem_set_mappable(struct kvm_memory_slot *slot,
+					     gfn_t start, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline int kvm_slot_gmem_clear_mappable(struct kvm_memory_slot *slot,
+					       gfn_t start, gfn_t end)
+{
+	WARN_ON_ONCE(1);
+	return -EINVAL;
+}
+static inline bool kvm_slot_gmem_is_mappable(struct kvm_memory_slot *slot,
+					     gfn_t gfn)
+{
+	WARN_ON_ONCE(1);
+	return false;
+}
+#endif /* CONFIG_KVM_PRIVATE_MEM_MAPPABLE */
+
 #endif
virt/kvm/Kconfig +4 -0
index 29b73eedfe74..a3970c5eca7b 100644
@@ -109,3 +109,7 @@ config KVM_GENERIC_PRIVATE_MEM
        select KVM_GENERIC_MEMORY_ATTRIBUTES
        select KVM_PRIVATE_MEM
        bool
+
+config KVM_PRIVATE_MEM_MAPPABLE
+       select KVM_PRIVATE_MEM
+       bool
virt/kvm/guest_memfd.c +8 -0
index f3f4334a9ccb..0a1f266a16f9 100644
@@ -11,6 +11,9 @@ struct kvm_gmem {
 	struct kvm *kvm;
 	struct xarray bindings;
 	struct list_head entry;
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+	struct xarray unmappable_gfns;
+#endif
 };
 
 static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
@@ -230,6 +233,11 @@ static int kvm_gmem_release(struct inode *inode, struct file *file)
 	mutex_unlock(&kvm->slots_lock);
 
 	xa_destroy(&gmem->bindings);
+
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+	xa_destroy(&gmem->unmappable_gfns);
+#endif
+
 	kfree(gmem);
 
 	kvm_put_kvm(kvm);

virt/kvm/guest_memfd.c static inline struct file *kvm_gmem_get_file(struct kvm_memory_slot *slot) · line 264

Quoted 9 lines
virt/kvm/guest_memfd.c +6 -0
@@ -248,7 +256,105 @@ static inline struct file *kvm_gmem_get_file(struct kvm_memory_slot *slot)
 	return get_file_active(&slot->gmem.file);
 }
 
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+int kvm_slot_gmem_toggle_mappable(struct kvm_memory_slot *slot, gfn_t start,
+				  gfn_t end, bool is_mappable)
+{
+	struct kvm_gmem *gmem = slot->gmem.file->private_data;
+	void *xval = is_mappable ? NULL : xa_mk_value(true);

IIUC storing stuff in the xarray takes memory, so if we want to save memory, we should minimize entries in the xarray. For pKVM, do you expect more mappable, or more unmappable gfns?

For TDX most of the memory will be private, so we expect fewer mappable gfns, and we'd prefer "entry in xarray == mappable".

Quoted 230 lines
virt/kvm/guest_memfd.c +96 -0
+	void *r;
+
+	r = xa_store_range(&gmem->unmappable_gfns, start, end - 1, xval, GFP_KERNEL);
+
+	return xa_err(r);
+}
+
+int kvm_slot_gmem_set_mappable(struct kvm_memory_slot *slot, gfn_t start, gfn_t end)
+{
+	return kvm_slot_gmem_toggle_mappable(slot, start, end, true);
+}
+
+int kvm_slot_gmem_clear_mappable(struct kvm_memory_slot *slot, gfn_t start, gfn_t end)
+{
+	return kvm_slot_gmem_toggle_mappable(slot, start, end, false);
+}
+
+bool kvm_slot_gmem_is_mappable(struct kvm_memory_slot *slot, gfn_t gfn)
+{
+	struct kvm_gmem *gmem = slot->gmem.file->private_data;
+	unsigned long _gfn = gfn;
+
+	return !xa_find(&gmem->unmappable_gfns, &_gfn, ULONG_MAX, XA_PRESENT);
+}
+
+static bool kvm_gmem_isfaultable(struct vm_fault *vmf)
+{
+	struct kvm_gmem *gmem = vmf->vma->vm_file->private_data;
+	struct inode *inode = file_inode(vmf->vma->vm_file);
+	pgoff_t pgoff = vmf->pgoff;
+	struct kvm_memory_slot *slot;
+	unsigned long index;
+	bool r = true;
+
+	filemap_invalidate_lock(inode->i_mapping);
+
+	xa_for_each_range(&gmem->bindings, index, slot, pgoff, pgoff) {
+		pgoff_t base_gfn = slot->base_gfn;
+		pgoff_t gfn_pgoff = slot->gmem.pgoff;
+		pgoff_t gfn = base_gfn + max(gfn_pgoff, pgoff) - gfn_pgoff;
+
+		if (!kvm_slot_gmem_is_mappable(slot, gfn)) {
+			r = false;
+			break;
+		}
+	}
+
+	filemap_invalidate_unlock(inode->i_mapping);
+
+	return r;
+}
+
+static vm_fault_t kvm_gmem_fault(struct vm_fault *vmf)
+{
+	struct folio *folio;
+
+	folio = kvm_gmem_get_folio(file_inode(vmf->vma->vm_file), vmf->pgoff);
+	if (!folio)
+		return VM_FAULT_SIGBUS;
+
+	if (!kvm_gmem_isfaultable(vmf)) {
+		folio_unlock(folio);
+		folio_put(folio);
+		return VM_FAULT_SIGBUS;
+	}
+
+	vmf->page = folio_file_page(folio, vmf->pgoff);
+	return VM_FAULT_LOCKED;
+}
+
+static const struct vm_operations_struct kvm_gmem_vm_ops = {
+	.fault = kvm_gmem_fault,
+};
+
+static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) !=
+	    (VM_SHARED | VM_MAYSHARE)) {
+		return -EINVAL;
+	}
+
+	file_accessed(file);
+	vm_flags_set(vma, VM_DONTDUMP);
+	vma->vm_ops = &kvm_gmem_vm_ops;
+
+	return 0;
+}
+#else
+#define kvm_gmem_mmap NULL
+#endif /* CONFIG_KVM_PRIVATE_MEM_MAPPABLE */
+
 static struct file_operations kvm_gmem_fops = {
+	.mmap		= kvm_gmem_mmap,
 	.open		= generic_file_open,
 	.release	= kvm_gmem_release,
 	.fallocate	= kvm_gmem_fallocate,
@@ -369,6 +475,10 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
 	xa_init(&gmem->bindings);
 	list_add(&gmem->entry, &inode->i_mapping->i_private_list);
 
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+	xa_init(&gmem->unmappable_gfns);
+#endif
+
 	fd_install(fd, file);
 	return fd;
 
virt/kvm/kvm_main.c +115 -0
index 1192942aef91..f4b4498d4de6 100644
@@ -3265,6 +3265,128 @@ static int next_segment(unsigned long len, int offset)
 		return len;
 }
 
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+static bool __kvm_gmem_is_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	struct kvm_memslot_iter iter;
+
+	lockdep_assert_held(&kvm->slots_lock);
+
+	kvm_for_each_memslot_in_gfn_range(&iter, kvm_memslots(kvm), start, end) {
+		struct kvm_memory_slot *memslot = iter.slot;
+		gfn_t gfn_start, gfn_end, i;
+
+		gfn_start = max(start, memslot->base_gfn);
+		gfn_end = min(end, memslot->base_gfn + memslot->npages);
+		if (WARN_ON_ONCE(gfn_start >= gfn_end))
+			continue;
+
+		for (i = gfn_start; i < gfn_end; i++) {
+			if (!kvm_slot_gmem_is_mappable(memslot, i))
+				return false;
+		}
+	}
+
+	return true;
+}
+
+bool kvm_gmem_is_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	bool r;
+
+	mutex_lock(&kvm->slots_lock);
+	r = __kvm_gmem_is_mappable(kvm, start, end);
+	mutex_unlock(&kvm->slots_lock);
+
+	return r;
+}
+
+static bool __kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	struct kvm_memslot_iter iter;
+
+	lockdep_assert_held(&kvm->slots_lock);
+
+	kvm_for_each_memslot_in_gfn_range(&iter, kvm_memslots(kvm), start, end) {
+		struct kvm_memory_slot *memslot = iter.slot;
+		gfn_t gfn_start, gfn_end, i;
+
+		gfn_start = max(start, memslot->base_gfn);
+		gfn_end = min(end, memslot->base_gfn + memslot->npages);
+		if (WARN_ON_ONCE(gfn_start >= gfn_end))
+			continue;
+
+		for (i = gfn_start; i < gfn_end; i++) {
+			struct page *page;
+			bool is_mapped;
+			kvm_pfn_t pfn;
+
+			if (WARN_ON_ONCE(kvm_gmem_get_pfn_locked(kvm, memslot, i, &pfn, NULL)))
+				continue;
+
+			page = pfn_to_page(pfn);
+			is_mapped = page_mapped(page) || page_maybe_dma_pinned(page);
+			unlock_page(page);
+			put_page(page);
+
+			if (is_mapped)
+				return true;
+		}
+	}
+
+	return false;
+}
+
+bool kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	bool r;
+
+	mutex_lock(&kvm->slots_lock);
+	r = __kvm_gmem_is_mapped(kvm, start, end);
+	mutex_unlock(&kvm->slots_lock);
+
+	return r;
+}
+
+static int kvm_gmem_toggle_mappable(struct kvm *kvm, gfn_t start, gfn_t end,
+				    bool is_mappable)
+{
+	struct kvm_memslot_iter iter;
+	int r = 0;
+
+	mutex_lock(&kvm->slots_lock);
+
+	kvm_for_each_memslot_in_gfn_range(&iter, kvm_memslots(kvm), start, end) {
+		struct kvm_memory_slot *memslot = iter.slot;
+		gfn_t gfn_start, gfn_end;
+
+		gfn_start = max(start, memslot->base_gfn);
+		gfn_end = min(end, memslot->base_gfn + memslot->npages);
+		if (WARN_ON_ONCE(start >= end))
+			continue;
+
+		r = kvm_slot_gmem_toggle_mappable(memslot, gfn_start, gfn_end, is_mappable);
+		if (WARN_ON_ONCE(r))
+			break;
+	}
+
+	mutex_unlock(&kvm->slots_lock);
+
+	return r;
+}
+
+int kvm_gmem_set_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	return kvm_gmem_toggle_mappable(kvm, start, end, true);
+}
+

virt/kvm/kvm_main.c static int next_segment(unsigned long len, int offset) · line 3392

Quoted 10 lines
virt/kvm/kvm_main.c +7 -0
+int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+	return kvm_gmem_toggle_mappable(kvm, start, end, false);
+}
+
+#endif /* CONFIG_KVM_PRIVATE_MEM_MAPPABLE */
+
 /* Copy @len bytes from guest memory at '(@gfn * PAGE_SIZE) + @offset' to @data */
 static int __kvm_read_guest_page(struct kvm_memory_slot *slot, gfn_t gfn,
 				 void *data, int offset, int len)

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/T/
[3] https://lore.kernel.org/all/[email protected]/

author
#4 lore raw

Re: [RFC PATCH v2 02/10] KVM: Add restricted support for mapping guestmem by the host

Hi Ackerley,

Quoted 13 lines · Ackerley Tng
Quoted 9 lines · Fuad Tabba

Add support for mmap() and fault() for guest_memfd in the host. The ability to fault in a guest page is contingent on that page being shared with the host. To track this, this patch adds a new xarray to each guest_memfd object, which tracks the mappability of guest frames.

The guest_memfd PRIVATE memory attribute is not used for two reasons. First because it reflects the userspace expectation for that memory location, and therefore can be toggled by userspace.

Quoted 6 lines

Thank you for clarifying why the PRIVATE memory attribute cannot be used. I think that having the attributes with guest_memfd is a good idea.

Since faultability is a property of the memory contents, shouldn't faultability be stored on the inode rather than the file?

I don't have a strong opinion about this, but by not mappable, this means not having a valid mapping in the host, not that the "mmap" call in and of itself is not allowed.

Quoted 13 lines
Quoted 5 lines

The second is, although each guest_memfd file has a 1:1 binding with a KVM instance, the plan is to allow multiple files per inode, e.g. to allow intra-host migration to a new KVM instance, without destroying guest_memfd.

I think you also alluded to the concept of inodes vs files above.

To store the xarray on the inode, we would probably have to make guest_memfd use its own mount so that we can use .evict_inode() from struct address_space_operations to clean up the inode neatly, perhaps the way it was done in guest_memfd v12 [1].

Quoted 5 lines

This RFC to enable intra-host migration [2] was built with the version of guest_memfd that used its own mount. IIUC, the gmem struct stored on the file is meant to be the binding between struct kvm and the memory contents [3], so the gmem struct won't be transferred and a new gmem struct will be created.

I thought that the gmem object itself is unique, and that even if we're transferring it from one inode to another it would still be there. If not, then you're right and we need to store this data in the inode.

Thanks for pointing me to the RFC. I'll have a look at it and use it as a reference when I respin this.

Quoted 20 lines
Quoted 20 lines

This new feature is gated with a new configuration option, CONFIG_KVM_PRIVATE_MEM_MAPPABLE.

---
 include/linux/kvm_host.h |  61 ++++++++++++++++++++
 virt/kvm/Kconfig         |   4 ++
 virt/kvm/guest_memfd.c   | 110 +++++++++++++++++++++++++++++++++++
 virt/kvm/kvm_main.c      | 122 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 297 insertions(+)
include/linux/kvm_host.h +0 -0
index 43a157f8171a..ab1344327e57 100644
@@ -2452,4 +2452,65 @@ static inline int kvm_gmem_get_pfn_locked(struct kvm *kvm,
 }
 #endif /* CONFIG_KVM_PRIVATE_MEM */
 
Quoted 6 lines

include/linux/kvm_host.h static inline int kvm_gmem_get_pfn_locked(struct kvm *kvm, · line 2458

Quoted 4 lines
include/linux/kvm_host.h +4 -0
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+bool kvm_gmem_is_mappable(struct kvm *kvm, gfn_t gfn, gfn_t end);
+bool kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end);
+int kvm_gmem_set_mappable(struct kvm *kvm, gfn_t start, gfn_t end);

How will kvm_gmem_is_mapped() and kvm_gmem_set_mappable() be used?

In this patch series I only use __ kvm_gmem_is_mapped(), you're right. In later patches (not posted, pkvm specific), I use kvm_gmem_is_mapped() in the processing of an unshare call from the guest.

Quoted 111 lines
Quoted 97 lines
include/linux/kvm_host.h +57 -0
+int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end);
+int kvm_slot_gmem_toggle_mappable(struct kvm_memory_slot *slot, gfn_t start,
+                               gfn_t end, bool is_mappable);
+int kvm_slot_gmem_set_mappable(struct kvm_memory_slot *slot, gfn_t start,
+                            gfn_t end);
+int kvm_slot_gmem_clear_mappable(struct kvm_memory_slot *slot, gfn_t start,
+                              gfn_t end);
+bool kvm_slot_gmem_is_mappable(struct kvm_memory_slot *slot, gfn_t gfn);
+#else
+static inline bool kvm_gmem_is_mappable(struct kvm *kvm, gfn_t gfn, gfn_t end)
+{
+     WARN_ON_ONCE(1);
+     return false;
+}
+static inline bool kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+     WARN_ON_ONCE(1);
+     return false;
+}
+static inline int kvm_gmem_set_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+     WARN_ON_ONCE(1);
+     return -EINVAL;
+}
+static inline int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start,
+                                       gfn_t end)
+{
+     WARN_ON_ONCE(1);
+     return -EINVAL;
+}
+static inline int kvm_slot_gmem_toggle_mappable(struct kvm_memory_slot *slot,
+                                             gfn_t start, gfn_t end,
+                                             bool is_mappable)
+{
+     WARN_ON_ONCE(1);
+     return -EINVAL;
+}
+static inline int kvm_slot_gmem_set_mappable(struct kvm_memory_slot *slot,
+                                          gfn_t start, gfn_t end)
+{
+     WARN_ON_ONCE(1);
+     return -EINVAL;
+}
+static inline int kvm_slot_gmem_clear_mappable(struct kvm_memory_slot *slot,
+                                            gfn_t start, gfn_t end)
+{
+     WARN_ON_ONCE(1);
+     return -EINVAL;
+}
+static inline bool kvm_slot_gmem_is_mappable(struct kvm_memory_slot *slot,
+                                          gfn_t gfn)
+{
+     WARN_ON_ONCE(1);
+     return false;
+}
+#endif /* CONFIG_KVM_PRIVATE_MEM_MAPPABLE */
+
 #endif
virt/kvm/Kconfig +4 -0
index 29b73eedfe74..a3970c5eca7b 100644
@@ -109,3 +109,7 @@ config KVM_GENERIC_PRIVATE_MEM
        select KVM_GENERIC_MEMORY_ATTRIBUTES
        select KVM_PRIVATE_MEM
        bool
+
+config KVM_PRIVATE_MEM_MAPPABLE
+       select KVM_PRIVATE_MEM
+       bool
virt/kvm/guest_memfd.c +8 -0
index f3f4334a9ccb..0a1f266a16f9 100644
@@ -11,6 +11,9 @@ struct kvm_gmem {
      struct kvm *kvm;
      struct xarray bindings;
      struct list_head entry;
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+     struct xarray unmappable_gfns;
+#endif
 };
 
 static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index)
@@ -230,6 +233,11 @@ static int kvm_gmem_release(struct inode *inode, struct file *file)
      mutex_unlock(&kvm->slots_lock);
 
      xa_destroy(&gmem->bindings);
+
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+     xa_destroy(&gmem->unmappable_gfns);
+#endif
+
      kfree(gmem);
 
      kvm_put_kvm(kvm);

virt/kvm/guest_memfd.c static inline struct file *kvm_gmem_get_file(struct kvm_memory_slot *slot) · line 264

Quoted 9 lines
virt/kvm/guest_memfd.c +6 -0
@@ -248,7 +256,105 @@ static inline struct file *kvm_gmem_get_file(struct kvm_memory_slot *slot)
      return get_file_active(&slot->gmem.file);
 }
 
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+int kvm_slot_gmem_toggle_mappable(struct kvm_memory_slot *slot, gfn_t start,
+                               gfn_t end, bool is_mappable)
+{
+     struct kvm_gmem *gmem = slot->gmem.file->private_data;
+     void *xval = is_mappable ? NULL : xa_mk_value(true);

IIUC storing stuff in the xarray takes memory, so if we want to save memory, we should minimize entries in the xarray. For pKVM, do you expect more mappable, or more unmappable gfns?

Quoted 2 lines

For TDX most of the memory will be private, so we expect fewer mappable gfns, and we'd prefer "entry in xarray == mappable".

This is similar to a discussion I've had earlier regarding the PRIVATE attribute [*].

I don't have a strong opinion. It's trivial to reverse the polarity. But to answer your question, most of the memory in pKVM would be private for protected guests, but shared for non-protected guests. So, it's not easy to know which would be the more common use case.

Cheers,
/fuad

[*] https://lore.kernel.org/lkml/[email protected]/T/#:~:text=consistent%20across%20implementations.-,Yeah%2C%20we%20discussed%20this%20in%20v12%5B*%5D.%20%20The%20default%20really%20doesn%27t%20matter%20for%20memory,-overheads%20or%20performances

Quoted 244 lines
Quoted 230 lines
virt/kvm/guest_memfd.c +96 -0
+     void *r;
+
+     r = xa_store_range(&gmem->unmappable_gfns, start, end - 1, xval, GFP_KERNEL);
+
+     return xa_err(r);
+}
+
+int kvm_slot_gmem_set_mappable(struct kvm_memory_slot *slot, gfn_t start, gfn_t end)
+{
+     return kvm_slot_gmem_toggle_mappable(slot, start, end, true);
+}
+
+int kvm_slot_gmem_clear_mappable(struct kvm_memory_slot *slot, gfn_t start, gfn_t end)
+{
+     return kvm_slot_gmem_toggle_mappable(slot, start, end, false);
+}
+
+bool kvm_slot_gmem_is_mappable(struct kvm_memory_slot *slot, gfn_t gfn)
+{
+     struct kvm_gmem *gmem = slot->gmem.file->private_data;
+     unsigned long _gfn = gfn;
+
+     return !xa_find(&gmem->unmappable_gfns, &_gfn, ULONG_MAX, XA_PRESENT);
+}
+
+static bool kvm_gmem_isfaultable(struct vm_fault *vmf)
+{
+     struct kvm_gmem *gmem = vmf->vma->vm_file->private_data;
+     struct inode *inode = file_inode(vmf->vma->vm_file);
+     pgoff_t pgoff = vmf->pgoff;
+     struct kvm_memory_slot *slot;
+     unsigned long index;
+     bool r = true;
+
+     filemap_invalidate_lock(inode->i_mapping);
+
+     xa_for_each_range(&gmem->bindings, index, slot, pgoff, pgoff) {
+             pgoff_t base_gfn = slot->base_gfn;
+             pgoff_t gfn_pgoff = slot->gmem.pgoff;
+             pgoff_t gfn = base_gfn + max(gfn_pgoff, pgoff) - gfn_pgoff;
+
+             if (!kvm_slot_gmem_is_mappable(slot, gfn)) {
+                     r = false;
+                     break;
+             }
+     }
+
+     filemap_invalidate_unlock(inode->i_mapping);
+
+     return r;
+}
+
+static vm_fault_t kvm_gmem_fault(struct vm_fault *vmf)
+{
+     struct folio *folio;
+
+     folio = kvm_gmem_get_folio(file_inode(vmf->vma->vm_file), vmf->pgoff);
+     if (!folio)
+             return VM_FAULT_SIGBUS;
+
+     if (!kvm_gmem_isfaultable(vmf)) {
+             folio_unlock(folio);
+             folio_put(folio);
+             return VM_FAULT_SIGBUS;
+     }
+
+     vmf->page = folio_file_page(folio, vmf->pgoff);
+     return VM_FAULT_LOCKED;
+}
+
+static const struct vm_operations_struct kvm_gmem_vm_ops = {
+     .fault = kvm_gmem_fault,
+};
+
+static int kvm_gmem_mmap(struct file *file, struct vm_area_struct *vma)
+{
+     if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) !=
+         (VM_SHARED | VM_MAYSHARE)) {
+             return -EINVAL;
+     }
+
+     file_accessed(file);
+     vm_flags_set(vma, VM_DONTDUMP);
+     vma->vm_ops = &kvm_gmem_vm_ops;
+
+     return 0;
+}
+#else
+#define kvm_gmem_mmap NULL
+#endif /* CONFIG_KVM_PRIVATE_MEM_MAPPABLE */
+
 static struct file_operations kvm_gmem_fops = {
+     .mmap           = kvm_gmem_mmap,
      .open           = generic_file_open,
      .release        = kvm_gmem_release,
      .fallocate      = kvm_gmem_fallocate,
@@ -369,6 +475,10 @@ static int __kvm_gmem_create(struct kvm *kvm, loff_t size, u64 flags)
      xa_init(&gmem->bindings);
      list_add(&gmem->entry, &inode->i_mapping->i_private_list);
 
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+     xa_init(&gmem->unmappable_gfns);
+#endif
+
      fd_install(fd, file);
      return fd;
 
virt/kvm/kvm_main.c +115 -0
index 1192942aef91..f4b4498d4de6 100644
@@ -3265,6 +3265,128 @@ static int next_segment(unsigned long len, int offset)
              return len;
 }
 
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+static bool __kvm_gmem_is_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+     struct kvm_memslot_iter iter;
+
+     lockdep_assert_held(&kvm->slots_lock);
+
+     kvm_for_each_memslot_in_gfn_range(&iter, kvm_memslots(kvm), start, end) {
+             struct kvm_memory_slot *memslot = iter.slot;
+             gfn_t gfn_start, gfn_end, i;
+
+             gfn_start = max(start, memslot->base_gfn);
+             gfn_end = min(end, memslot->base_gfn + memslot->npages);
+             if (WARN_ON_ONCE(gfn_start >= gfn_end))
+                     continue;
+
+             for (i = gfn_start; i < gfn_end; i++) {
+                     if (!kvm_slot_gmem_is_mappable(memslot, i))
+                             return false;
+             }
+     }
+
+     return true;
+}
+
+bool kvm_gmem_is_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+     bool r;
+
+     mutex_lock(&kvm->slots_lock);
+     r = __kvm_gmem_is_mappable(kvm, start, end);
+     mutex_unlock(&kvm->slots_lock);
+
+     return r;
+}
+
+static bool __kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+     struct kvm_memslot_iter iter;
+
+     lockdep_assert_held(&kvm->slots_lock);
+
+     kvm_for_each_memslot_in_gfn_range(&iter, kvm_memslots(kvm), start, end) {
+             struct kvm_memory_slot *memslot = iter.slot;
+             gfn_t gfn_start, gfn_end, i;
+
+             gfn_start = max(start, memslot->base_gfn);
+             gfn_end = min(end, memslot->base_gfn + memslot->npages);
+             if (WARN_ON_ONCE(gfn_start >= gfn_end))
+                     continue;
+
+             for (i = gfn_start; i < gfn_end; i++) {
+                     struct page *page;
+                     bool is_mapped;
+                     kvm_pfn_t pfn;
+
+                     if (WARN_ON_ONCE(kvm_gmem_get_pfn_locked(kvm, memslot, i, &pfn, NULL)))
+                             continue;
+
+                     page = pfn_to_page(pfn);
+                     is_mapped = page_mapped(page) || page_maybe_dma_pinned(page);
+                     unlock_page(page);
+                     put_page(page);
+
+                     if (is_mapped)
+                             return true;
+             }
+     }
+
+     return false;
+}
+
+bool kvm_gmem_is_mapped(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+     bool r;
+
+     mutex_lock(&kvm->slots_lock);
+     r = __kvm_gmem_is_mapped(kvm, start, end);
+     mutex_unlock(&kvm->slots_lock);
+
+     return r;
+}
+
+static int kvm_gmem_toggle_mappable(struct kvm *kvm, gfn_t start, gfn_t end,
+                                 bool is_mappable)
+{
+     struct kvm_memslot_iter iter;
+     int r = 0;
+
+     mutex_lock(&kvm->slots_lock);
+
+     kvm_for_each_memslot_in_gfn_range(&iter, kvm_memslots(kvm), start, end) {
+             struct kvm_memory_slot *memslot = iter.slot;
+             gfn_t gfn_start, gfn_end;
+
+             gfn_start = max(start, memslot->base_gfn);
+             gfn_end = min(end, memslot->base_gfn + memslot->npages);
+             if (WARN_ON_ONCE(start >= end))
+                     continue;
+
+             r = kvm_slot_gmem_toggle_mappable(memslot, gfn_start, gfn_end, is_mappable);
+             if (WARN_ON_ONCE(r))
+                     break;
+     }
+
+     mutex_unlock(&kvm->slots_lock);
+
+     return r;
+}
+
+int kvm_gmem_set_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+     return kvm_gmem_toggle_mappable(kvm, start, end, true);
+}
+

virt/kvm/kvm_main.c static int next_segment(unsigned long len, int offset) · line 3392

Quoted 10 lines
virt/kvm/kvm_main.c +7 -0
+int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
+{
+     return kvm_gmem_toggle_mappable(kvm, start, end, false);
+}
+
+#endif /* CONFIG_KVM_PRIVATE_MEM_MAPPABLE */
+
 /* Copy @len bytes from guest memory at '(@gfn * PAGE_SIZE) + @offset' to @data */
 static int __kvm_read_guest_page(struct kvm_memory_slot *slot, gfn_t gfn,
                               void *data, int offset, int len)

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/T/
[3] https://lore.kernel.org/all/[email protected]/

author
#5 lore raw

RFC PATCH v2 3/10 KVM: Implement kvm_(read|/write)_guest_page for private memory slots

In reply to Fuad Tabba

Make __kvm_read_guest_page/__kvm_write_guest_page capable of accessing guest memory if no userspace address is available. Moreover, check that the memory being accessed is shared with the host before attempting the access.

KVM at the host might need to access shared memory that is not mapped in the host userspace but is in fact shared with the host, e.g., when accounting for stolen time. This allows the access without relying on the slot's userspace_addr being set.

This does not circumvent protection, since the access is only attempted if the memory is mappable by the host, which implies shareability.

---
 virt/kvm/kvm_main.c | 127 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 111 insertions(+), 16 deletions(-)
virt/kvm/kvm_main.c +111 -16
index f4b4498d4de6..ec6255c7325e 100644
@@ -3385,20 +3385,108 @@ int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
 	return kvm_gmem_toggle_mappable(kvm, start, end, false);
 }
 
+static int __kvm_read_private_guest_page(struct kvm *kvm,
+					 struct kvm_memory_slot *slot,
+					 gfn_t gfn, void *data, int offset,
+					 int len)
+{
+	struct page *page;
+	u64 pfn;
+	int r = 0;
+
+	if (size_add(offset, len) > PAGE_SIZE)
+		return -E2BIG;
+
+	mutex_lock(&kvm->slots_lock);
+
+	if (!__kvm_gmem_is_mappable(kvm, gfn, gfn + 1)) {
+		r = -EPERM;
+		goto unlock;
+	}
+
+	r = kvm_gmem_get_pfn_locked(kvm, slot, gfn, &pfn, NULL);
+	if (r)
+		goto unlock;
+
+	page = pfn_to_page(pfn);
+	memcpy(data, page_address(page) + offset, len);
+	unlock_page(page);
+	kvm_release_pfn_clean(pfn);
+unlock:
+	mutex_unlock(&kvm->slots_lock);
+
+	return r;
+}
+
+static int __kvm_write_private_guest_page(struct kvm *kvm,
+					  struct kvm_memory_slot *slot,
+					  gfn_t gfn, const void *data,
+					  int offset, int len)
+{
+	struct page *page;
+	u64 pfn;
+	int r = 0;
+
+	if (size_add(offset, len) > PAGE_SIZE)
+		return -E2BIG;
+
+	mutex_lock(&kvm->slots_lock);
+
+	if (!__kvm_gmem_is_mappable(kvm, gfn, gfn + 1)) {
+		r = -EPERM;
+		goto unlock;
+	}
+
+	r = kvm_gmem_get_pfn_locked(kvm, slot, gfn, &pfn, NULL);
+	if (r)
+		goto unlock;
+
+	page = pfn_to_page(pfn);
+	memcpy(page_address(page) + offset, data, len);
+	unlock_page(page);
+	kvm_release_pfn_dirty(pfn);
+unlock:
+	mutex_unlock(&kvm->slots_lock);
+
+	return r;
+}
+#else
+static int __kvm_read_private_guest_page(struct kvm *kvm,
+					 struct kvm_memory_slot *slot,
+					 gfn_t gfn, void *data, int offset,
+					 int len)
+{
+	WARN_ON_ONCE(1);
+	return -EIO;
+}
+
+static int __kvm_write_private_guest_page(struct kvm *kvm,
+					  struct kvm_memory_slot *slot,
+					  gfn_t gfn, const void *data,
+					  int offset, int len)
+{
+	WARN_ON_ONCE(1);
+	return -EIO;
+}
 #endif /* CONFIG_KVM_PRIVATE_MEM_MAPPABLE */
 
 /* Copy @len bytes from guest memory at '(@gfn * PAGE_SIZE) + @offset' to @data */
-static int __kvm_read_guest_page(struct kvm_memory_slot *slot, gfn_t gfn,
-				 void *data, int offset, int len)
+
+static int __kvm_read_guest_page(struct kvm *kvm, struct kvm_memory_slot *slot,
+				 gfn_t gfn, void *data, int offset, int len)
 {
-	int r;
 	unsigned long addr;
 
+	if (IS_ENABLED(CONFIG_KVM_PRIVATE_MEM_MAPPABLE) &&
+	    kvm_slot_can_be_private(slot)) {
+		return __kvm_read_private_guest_page(kvm, slot, gfn, data,
+						     offset, len);
+	}
+
 	addr = gfn_to_hva_memslot_prot(slot, gfn, NULL);
 	if (kvm_is_error_hva(addr))
 		return -EFAULT;
-	r = __copy_from_user(data, (void __user *)addr + offset, len);
-	if (r)
+	if (__copy_from_user(data, (void __user *)addr + offset, len))
 		return -EFAULT;
 	return 0;
 }
@@ -3408,7 +3496,7 @@ int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset,
 {
 	struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn);
 
-	return __kvm_read_guest_page(slot, gfn, data, offset, len);
+	return __kvm_read_guest_page(kvm, slot, gfn, data, offset, len);
 }
 EXPORT_SYMBOL_GPL(kvm_read_guest_page);
 
@@ -3417,7 +3505,7 @@ int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data,
 {
 	struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
 
-	return __kvm_read_guest_page(slot, gfn, data, offset, len);
+	return __kvm_read_guest_page(vcpu->kvm, slot, gfn, data, offset, len);
 }
 EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_page);
 
@@ -3492,17 +3580,24 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic);
 /* Copy @len bytes from @data into guest memory at '(@gfn * PAGE_SIZE) + @offset' */
 static int __kvm_write_guest_page(struct kvm *kvm,
 				  struct kvm_memory_slot *memslot, gfn_t gfn,
-			          const void *data, int offset, int len)
+				  const void *data, int offset, int len)
 {
-	int r;
-	unsigned long addr;
+	if (IS_ENABLED(CONFIG_KVM_PRIVATE_MEM_MAPPABLE) &&
+	    kvm_slot_can_be_private(memslot)) {
+		int r = __kvm_write_private_guest_page(kvm, memslot, gfn, data,
+						       offset, len);
+
+		if (r)
+			return r;
+	} else {
+		unsigned long addr = gfn_to_hva_memslot(memslot, gfn);
+
+		if (kvm_is_error_hva(addr))
+			return -EFAULT;
+		if (__copy_to_user((void __user *)addr + offset, data, len))
+			return -EFAULT;
+	}
 
-	addr = gfn_to_hva_memslot(memslot, gfn);
-	if (kvm_is_error_hva(addr))
-		return -EFAULT;
-	r = __copy_to_user((void __user *)addr + offset, data, len);
-	if (r)
-		return -EFAULT;
 	mark_page_dirty_in_slot(kvm, memslot, gfn);
 	return 0;
 }
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#6 lore raw

Re: [RFC PATCH v2 03/10] KVM: Implement kvm_(read|/write)_guest_page for private memory slots

Quoted 9 lines · Fuad Tabba

Make __kvm_read_guest_page/__kvm_write_guest_page capable of accessing guest memory if no userspace address is available. Moreover, check that the memory being accessed is shared with the host before attempting the access.

KVM at the host might need to access shared memory that is not mapped in the host userspace but is in fact shared with the host, e.g., when accounting for stolen time. This allows the access without relying on the slot's userspace_addr being set.

Why? As evidenced by the amount of code below, special casing guest_memfd isn't trivial, and taking kvm->slots_lock is likely a complete non-starter. In the happy case, uaccess is about as fast as can be, and has no inherent scaling issues.

Quoted 14 lines

This does not circumvent protection, since the access is only attempted if the memory is mappable by the host, which implies shareability.

---
 virt/kvm/kvm_main.c | 127 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 111 insertions(+), 16 deletions(-)
virt/kvm/kvm_main.c +0 -0
index f4b4498d4de6..ec6255c7325e 100644

virt/kvm/kvm_main.c int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end) · line 3388

Quoted 4 lines
virt/kvm/kvm_main.c +1 -0
@@ -3385,20 +3385,108 @@ int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
 	return kvm_gmem_toggle_mappable(kvm, start, end, false);
 }
 
+static int __kvm_read_private_guest_page(struct kvm *kvm,

The changelog says this is for accessing memory that is shared, but this says "private".

Quoted 3 lines
virt/kvm/kvm_main.c +3 -0
+					 struct kvm_memory_slot *slot,
+					 gfn_t gfn, void *data, int offset,
+					 int len)
author
#7 lore raw

Re: [RFC PATCH v2 03/10] KVM: Implement kvm_(read|/write)_guest_page for private memory slots

Hi Sean,

Quoted 33 lines · Sean Christopherson
Quoted 9 lines · Fuad Tabba

Make __kvm_read_guest_page/__kvm_write_guest_page capable of accessing guest memory if no userspace address is available. Moreover, check that the memory being accessed is shared with the host before attempting the access.

KVM at the host might need to access shared memory that is not mapped in the host userspace but is in fact shared with the host, e.g., when accounting for stolen time. This allows the access without relying on the slot's userspace_addr being set.

Why? As evidenced by the amount of code below, special casing guest_memfd isn't trivial, and taking kvm->slots_lock is likely a complete non-starter. In the happy case, uaccess is about as fast as can be, and has no inherent scaling issues.

Quoted 17 lines

This does not circumvent protection, since the access is only attempted if the memory is mappable by the host, which implies shareability.

---
 virt/kvm/kvm_main.c | 127 ++++++++++++++++++++++++++++++++++++++------
 1 file changed, 111 insertions(+), 16 deletions(-)
virt/kvm/kvm_main.c +0 -0
index f4b4498d4de6..ec6255c7325e 100644
@@ -3385,20 +3385,108 @@ int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end)
      return kvm_gmem_toggle_mappable(kvm, start, end, false);
 }
 
Quoted 4 lines

virt/kvm/kvm_main.c int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end) · line 3388

Quoted 1 line
virt/kvm/kvm_main.c +1 -0
+static int __kvm_read_private_guest_page(struct kvm *kvm,

The changelog says this is for accessing memory that is shared, but this says "private".

This is bad naming on my part. Instead, I should call this function something like, read_guestmem_page (and similar for the write one). Thanks for pointing this out.

Cheers,
/fuad

Quoted 3 lines
Quoted 3 lines
virt/kvm/kvm_main.c +3 -0
+                                      struct kvm_memory_slot *slot,
+                                      gfn_t gfn, void *data, int offset,
+                                      int len)
author
#8 lore raw

RFC PATCH v2 4/10 KVM: Add KVM capability to check if guest_memfd can be mapped by the host

In reply to Fuad Tabba

Add the KVM capability KVM_CAP_GUEST_MEMFD_MAPPABLE, which is true if mapping guest memory is supported by the host.

---
 include/uapi/linux/kvm.h | 3 ++-
 virt/kvm/kvm_main.c      | 4 ++++
 2 files changed, 6 insertions(+), 1 deletion(-)
include/uapi/linux/kvm.h +2 -1
index d03842abae57..783d0c3f4cb1 100644
@@ -916,7 +916,8 @@ struct kvm_enable_cap {
 #define KVM_CAP_MEMORY_FAULT_INFO 232
 #define KVM_CAP_MEMORY_ATTRIBUTES 233
 #define KVM_CAP_GUEST_MEMFD 234
-#define KVM_CAP_VM_TYPES 235
+#define KVM_CAP_GUEST_MEMFD_MAPPABLE 235
+#define KVM_CAP_VM_TYPES 236
 
 struct kvm_irq_routing_irqchip {
 	__u32 irqchip;
virt/kvm/kvm_main.c +4 -0
index ec6255c7325e..485c39fc373c 100644
@@ -5077,6 +5077,10 @@ static int kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)
 #ifdef CONFIG_KVM_PRIVATE_MEM
 	case KVM_CAP_GUEST_MEMFD:
 		return !kvm || kvm_arch_has_private_mem(kvm);
+#endif
+#ifdef CONFIG_KVM_PRIVATE_MEM_MAPPABLE
+	case KVM_CAP_GUEST_MEMFD_MAPPABLE:
+		return !kvm || kvm_arch_has_private_mem(kvm);
 #endif
 	default:
 		break;
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#9 lore raw

Re: [RFC PATCH v2 04/10] KVM: Add KVM capability to check if guest_memfd can be mapped by the host

Quoted 10 lines · Fuad Tabba

Add the KVM capability KVM_CAP_GUEST_MEMFD_MAPPABLE, which is true if mapping guest memory is supported by the host.

---
 include/uapi/linux/kvm.h | 3 ++-
 virt/kvm/kvm_main.c      | 4 ++++
 2 files changed, 6 insertions(+), 1 deletion(-)

<snip>

Why do we need a cap for KVM_CAP_GUEST_MEMFD_MAPPABLE instead of just making guest_memfd mmap-able?

Is this to prevent breaking userspace, because a user might be relying on guest_memfd being not mmap-able?

author
#10 lore raw

Re: [RFC PATCH v2 04/10] KVM: Add KVM capability to check if guest_memfd can be mapped by the host

Hi Ackerley,

Quoted 12 lines · Ackerley Tng
Quoted 9 lines · Fuad Tabba

Add the KVM capability KVM_CAP_GUEST_MEMFD_MAPPABLE, which is true if mapping guest memory is supported by the host.

---
 include/uapi/linux/kvm.h | 3 ++-
 virt/kvm/kvm_main.c      | 4 ++++
 2 files changed, 6 insertions(+), 1 deletion(-)
Quoted 7 lines
Quoted 1 line

<snip>

Why do we need a cap for KVM_CAP_GUEST_MEMFD_MAPPABLE instead of just making guest_memfd mmap-able?

Is this to prevent breaking userspace, because a user might be relying on guest_memfd being not mmap-able?

To be able to check that the ability is there, since it is a new capability not available in Linux 6.9 not 6.10 (i.e., after guest_memfd()) was introduced.

Cheers,
/fuad

author
#11 lore raw

RFC PATCH v2 5/10 KVM: selftests: guest_memfd mmap() test when mapping is allowed

In reply to Fuad Tabba

Expand the guest_memfd selftests to include testing mapping guest memory if the capability is supported, and that still checks that memory is not mappable if the capability isn't supported.

Also, build the guest_memfd selftest for aarch64.

---
 tools/testing/selftests/kvm/Makefile          |  1 +
 .../testing/selftests/kvm/guest_memfd_test.c  | 47 ++++++++++++++++++-
 2 files changed, 46 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/Makefile +1 -0
index ac280dcba996..fb63f7e956d4 100644
@@ -166,6 +166,7 @@ TEST_GEN_PROGS_aarch64 += arch_timer
 TEST_GEN_PROGS_aarch64 += demand_paging_test
 TEST_GEN_PROGS_aarch64 += dirty_log_test
 TEST_GEN_PROGS_aarch64 += dirty_log_perf_test
+TEST_GEN_PROGS_aarch64 += guest_memfd_test
 TEST_GEN_PROGS_aarch64 += guest_print_test
 TEST_GEN_PROGS_aarch64 += get-reg-list
 TEST_GEN_PROGS_aarch64 += kvm_create_max_vcpus
tools/testing/selftests/kvm/guest_memfd_test.c +45 -2

64 lines not on this page · raw message

Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
author
#12 lore raw

RFC PATCH v2 6/10 KVM: arm64: Skip VMA checks for slots without userspace address

In reply to Fuad Tabba

Memory slots backed by guest memory might be created with no intention of being mapped by the host. These are recognized by not having a userspace address in the memory slot.

VMA checks are neither possible nor necessary for this kind of slot, so skip them.

---
 arch/arm64/kvm/mmu.c | 8 ++++++++
 1 file changed, 8 insertions(+)
arch/arm64/kvm/mmu.c +8 -0
index 8bcab0cc3fe9..e632e10ea395 100644
@@ -948,6 +948,10 @@ static void stage2_unmap_memslot(struct kvm *kvm,
 	phys_addr_t size = PAGE_SIZE * memslot->npages;
 	hva_t reg_end = hva + size;
 
+	/* Host will not map this private memory without a userspace address. */
+	if (kvm_slot_can_be_private(memslot) && !hva)
+		return;
+
 	/*
 	 * A memory region could potentially cover multiple VMAs, and any holes
 	 * between them, so iterate over all of them to find out if we should
@@ -1976,6 +1980,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
 	hva = new->userspace_addr;
 	reg_end = hva + (new->npages << PAGE_SHIFT);
 
+	/* Host will not map this private memory without a userspace address. */
+	if ((kvm_slot_can_be_private(new)) && !hva)
+		return 0;
+
 	mmap_read_lock(current->mm);
 	/*
 	 * A memory region could potentially cover multiple VMAs, and any holes
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
author
#13 lore raw

RFC PATCH v2 7/10 KVM: arm64: Do not allow changes to private memory slots

In reply to Fuad Tabba

Handling changes to private memory slots can be difficult, since it would probably require some cooperation from the hypervisor and/or the guest. Do not allow such changes for now.

---
 arch/arm64/kvm/mmu.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/arm64/kvm/mmu.c +4 -0
index e632e10ea395..b1fc636fb670 100644
@@ -1970,6 +1970,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
 			change != KVM_MR_FLAGS_ONLY)
 		return 0;
 
+	if ((change == KVM_MR_MOVE || change == KVM_MR_FLAGS_ONLY) &&
+	    ((kvm_slot_can_be_private(old)) || (kvm_slot_can_be_private(new))))
+		return -EPERM;
+
 	/*
 	 * Prevent userspace from creating a memory region outside of the IPA
 	 * space addressable by the KVM guest IPA space.
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
author
#14 lore raw

RFC PATCH v2 8/10 KVM: arm64: Handle guest_memfd()-backed guest page faults

In reply to Fuad Tabba

Add arm64 support for resolving guest page faults on guest_memfd() backed memslots. This support is not contingent on pKVM, or other confidential computing support, and works in both VHE and nVHE modes.

Without confidential computing, this support is useful for testing and debugging. In the future, it might also be useful should a user want to use guest_memfd() for all code, whether it's for a protected guest or not.

For now, the fault granule is restricted to PAGE_SIZE.

---
 arch/arm64/kvm/mmu.c | 127 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 125 insertions(+), 2 deletions(-)
arch/arm64/kvm/mmu.c +125 -2
index b1fc636fb670..e15167865cab 100644
@@ -1378,6 +1378,123 @@ static bool kvm_vma_mte_allowed(struct vm_area_struct *vma)
 	return vma->vm_flags & VM_MTE_ALLOWED;
 }
 
+static int guest_memfd_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
+			     struct kvm_memory_slot *memslot, bool fault_is_perm)
+{
+	struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache;
+	bool exec_fault = kvm_vcpu_trap_is_exec_fault(vcpu);
+	bool logging_active = memslot_is_logging(memslot);
+	struct kvm_pgtable *pgt = vcpu->arch.hw_mmu->pgt;
+	enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
+	bool write_fault = kvm_is_write_fault(vcpu);
+	struct mm_struct *mm = current->mm;
+	gfn_t gfn = gpa_to_gfn(fault_ipa);
+	struct kvm *kvm = vcpu->kvm;
+	unsigned long mmu_seq;
+	struct page *page;
+	kvm_pfn_t pfn;
+	int ret;
+
+	/* For now, guest_memfd() only supports PAGE_SIZE granules. */
+	if (WARN_ON_ONCE(fault_is_perm &&
+			 kvm_vcpu_trap_get_perm_fault_granule(vcpu) != PAGE_SIZE)) {
+		return -EFAULT;
+	}
+
+	VM_BUG_ON(write_fault && exec_fault);
+
+	if (fault_is_perm && !write_fault && !exec_fault) {
+		kvm_err("Unexpected L2 read permission error\n");
+		return -EFAULT;
+	}
+
+	/*
+	 * Permission faults just need to update the existing leaf entry,
+	 * and so normally don't require allocations from the memcache. The
+	 * only exception to this is when dirty logging is enabled at runtime
+	 * and a write fault needs to collapse a block entry into a table.
+	 */
+	if (!fault_is_perm || (logging_active && write_fault)) {
+		ret = kvm_mmu_topup_memory_cache(memcache,
+						 kvm_mmu_cache_min_pages(vcpu->arch.hw_mmu));
+		if (ret)
+			return ret;
+	}
+
+	/*
+	 * Read mmu_invalidate_seq so that KVM can detect if the results of
+	 * kvm_gmem_get_pfn_locked() become stale prior to acquiring
+	 * kvm->mmu_lock.
+	 */
+	mmu_seq = vcpu->kvm->mmu_invalidate_seq;
+
+	/* To pair with the smp_wmb() in kvm_mmu_invalidate_end(). */
+	smp_rmb();
+
+	ret = kvm_gmem_get_pfn_locked(kvm, memslot, gfn, &pfn, NULL);
+	if (ret)
+		return ret;
+
+	page = pfn_to_page(pfn);
+
+	if (!kvm_gmem_is_mappable(kvm, gfn, gfn + 1) &&
+	    (page_mapped(page) || page_maybe_dma_pinned(page))) {
+		return -EPERM;
+	}
+
+	/*
+	 * Once it's faulted in, a guest_memfd() page will stay in memory.
+	 * Therefore, count it as locked.
+	 */
+	if (!fault_is_perm) {
+		ret = account_locked_vm(mm, 1, true);
+		if (ret)
+			goto unlock_page;
+	}
+
+	read_lock(&kvm->mmu_lock);
+	if (mmu_invalidate_retry(kvm, mmu_seq))
+		goto unlock_mmu;
+
+	if (write_fault)
+		prot |= KVM_PGTABLE_PROT_W;
+
+	if (exec_fault)
+		prot |= KVM_PGTABLE_PROT_X;
+
+	if (cpus_have_final_cap(ARM64_HAS_CACHE_DIC))
+		prot |= KVM_PGTABLE_PROT_X;
+
+	/*
+	 * Under the premise of getting a FSC_PERM fault, we just need to relax
+	 * permissions.
+	 */
+	if (fault_is_perm)
+		ret = kvm_pgtable_stage2_relax_perms(pgt, fault_ipa, prot);
+	else
+		ret = kvm_pgtable_stage2_map(pgt, fault_ipa, PAGE_SIZE,
+					__pfn_to_phys(pfn), prot,
+					memcache,
+					KVM_PGTABLE_WALK_HANDLE_FAULT |
+					KVM_PGTABLE_WALK_SHARED);
+
+	/* Mark the page dirty only if the fault is handled successfully */
+	if (write_fault && !ret) {
+		kvm_set_pfn_dirty(pfn);
+		mark_page_dirty_in_slot(kvm, memslot, gfn);
+	}
+
+unlock_mmu:
+	read_unlock(&kvm->mmu_lock);
+
+	if (ret && !fault_is_perm)
+		account_locked_vm(mm, 1, false);
+unlock_page:
+	unlock_page(page);
+	put_page(page);
+	return ret != -EAGAIN ? ret : 0;
+}
+
 static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 			  struct kvm_memory_slot *memslot, unsigned long hva,
 			  bool fault_is_perm)
@@ -1748,8 +1865,14 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu)
 		goto out_unlock;
 	}
 
-	ret = user_mem_abort(vcpu, fault_ipa, memslot, hva,
-			     esr_fsc_is_permission_fault(esr));
+	if (kvm_slot_can_be_private(memslot)) {
+		ret = guest_memfd_abort(vcpu, fault_ipa, memslot,
+					esr_fsc_is_permission_fault(esr));
+	} else {
+		ret = user_mem_abort(vcpu, fault_ipa, memslot, hva,
+				     esr_fsc_is_permission_fault(esr));
+	}
+
 	if (ret == 0)
 		ret = 1;
 out:
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
author
#15 lore raw

RFC PATCH v2 9/10 KVM: arm64: arm64 has private memory support when config is enabled

In reply to Fuad Tabba

Implement kvm_arch_has_private_mem() in arm64, making it dependent on the configuration option.

---
 arch/arm64/include/asm/kvm_host.h | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/include/asm/kvm_host.h +3 -0
index 36b8e97bf49e..8f7d78ee9557 100644
@@ -1414,4 +1414,7 @@ bool kvm_arm_vcpu_stopped(struct kvm_vcpu *vcpu);
 		(pa + pi + pa3) == 1;					\
 	})
 
+#define kvm_arch_has_private_mem(kvm)					\
+	(IS_ENABLED(CONFIG_KVM_PRIVATE_MEM) && is_protected_kvm_enabled())
+
 #endif /* __ARM64_KVM_HOST_H__ */
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#16 lore raw

Re: [RFC PATCH v2 09/10] KVM: arm64: arm64 has private memory support when config is enabled

Hi Fuad,

Quoted 13 lines · Fuad Tabba

Implement kvm_arch_has_private_mem() in arm64, making it dependent on the configuration option.

---
 arch/arm64/include/asm/kvm_host.h | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/include/asm/kvm_host.h +0 -0
index 36b8e97bf49e..8f7d78ee9557 100644

arch/arm64/include/asm/kvm_host.h bool kvm_arm_vcpu_stopped(struct kvm_vcpu *vcpu); · line 1419

Quoted 6 lines
arch/arm64/include/asm/kvm_host.h +3 -0
@@ -1414,4 +1414,7 @@ bool kvm_arm_vcpu_stopped(struct kvm_vcpu *vcpu);
                (pa + pi + pa3) == 1;                                   \
        })
 
+#define kvm_arch_has_private_mem(kvm)                                  \
+       (IS_ENABLED(CONFIG_KVM_PRIVATE_MEM) && is_protected_kvm_enabled())
+

Would it make sense to have some ARM equivalent of KVM_X86_SW_PROTECTED_VM here? Both for easier testing of guest_memfd on ARM, as well as for future non-coco usecases.

Quoted 4 lines
arch/arm64/include/asm/kvm_host.h +0 -1
 #endif /* __ARM64_KVM_HOST_H__ */
--

2.46.0.rc1.232.g9752f9e123-goog

Best,
Patrick

author
#17 lore raw

Re: [RFC PATCH v2 09/10] KVM: arm64: arm64 has private memory support when config is enabled

Hi Patrick,

Quoted 20 lines · Patrick Roy

Hi Fuad,

Quoted 16 lines · Fuad Tabba

Implement kvm_arch_has_private_mem() in arm64, making it dependent on the configuration option.

---
 arch/arm64/include/asm/kvm_host.h | 3 +++
 1 file changed, 3 insertions(+)
arch/arm64/include/asm/kvm_host.h +0 -0
index 36b8e97bf49e..8f7d78ee9557 100644
@@ -1414,4 +1414,7 @@ bool kvm_arm_vcpu_stopped(struct kvm_vcpu *vcpu);
                (pa + pi + pa3) == 1;                                   \
        })
 
Quoted 7 lines

arch/arm64/include/asm/kvm_host.h bool kvm_arm_vcpu_stopped(struct kvm_vcpu *vcpu); · line 1419

Quoted 3 lines
arch/arm64/include/asm/kvm_host.h +3 -0
+#define kvm_arch_has_private_mem(kvm)                                  \
+       (IS_ENABLED(CONFIG_KVM_PRIVATE_MEM) && is_protected_kvm_enabled())
+

Would it make sense to have some ARM equivalent of KVM_X86_SW_PROTECTED_VM here? Both for easier testing of guest_memfd on ARM, as well as for future non-coco usecases.

I don't really have a strong opinion about this. I thought that if/when that were to happen, it would be trivial to modify this macro.

Cheers,
/fuad

Quoted 7 lines
Quoted 4 lines
arch/arm64/include/asm/kvm_host.h +0 -1
 #endif /* __ARM64_KVM_HOST_H__ */
--

2.46.0.rc1.232.g9752f9e123-goog

Best,
Patrick

author
#18 lore raw

RFC PATCH v2 10/10 KVM: arm64: Enable private memory kconfig for arm64

In reply to Fuad Tabba

Now that the infrastructure is in place for arm64 to support guest private memory, enable it in the arm64 kernel configuration.

---
 arch/arm64/kvm/Kconfig | 1 +
 1 file changed, 1 insertion(+)
arch/arm64/kvm/Kconfig +1 -0
index 58f09370d17e..8b166c697930 100644
@@ -37,6 +37,7 @@ menuconfig KVM
 	select HAVE_KVM_VCPU_RUN_PID_CHANGE
 	select SCHED_INFO
 	select GUEST_PERF_EVENTS if PERF_EVENTS
+	select KVM_PRIVATE_MEM_MAPPABLE
 	help
 	  Support hosting virtualized guest machines.
 
Signature · 1 line
2.46.0.rc1.232.g9752f9e123-goog
#19 lore raw
In reply to Fuad Tabba
Quoted 28 lines · Fuad Tabba

This series adds restricted mmap() support to guest_memfd, as well as support for guest_memfd on pKVM/arm64. It is based on Linux 6.10.

Main changes since V1 [1]:

- Decoupled whether guest memory is mappable from KVM memory attributes (SeanC)

Mappability is now tracked in the guest_mem object, orthogonal to whether userspace wants the memory to be private or shared. Moreover, the memory attributes capability (i.e., KVM_CAP_MEMORY_ATTRIBUTES) is not enabled for pKVM, since for software-based hypervisors such as pKVM and Gunyah, userspace is informed of the state of the memory via hypervisor exits if needed.

Even if attributes are enabled, this patch series would still work (modulo bugs), without compromising guest memory nor crashing the system.

- Use page_mapped() instead of page_mapcount() to check if page is mapped (DavidH)

- Add a new capability, KVM_CAP_GUEST_MEMFD_MAPPABLE, to query whether guest private memory can be mapped (with aforementioned restrictions)

Quoted 6 lines

- Add a selftest to check whether memory is mappable when the capability is enabled, and not mappable otherwise. Also, test the effect of punching holes in mapped memory. (DavidH)

By design, guest_memfd cannot be mapped, read, or written by the host. In pKVM, memory shared between a protected guest and the

I think we should use "cannot be faulted in" to be clear that guest_memfd can be mmaped but not faulted in.

Would it be better to have all the variables/config macros be something about faultability instead of mappability?

Quoted 81 lines

host is shared in-place, unlike the other confidential computing solutions that guest_memfd was originally envisaged for (e.g, TDX). When initializing a guest, as well as when accessing memory shared by the guest with the host, it would be useful to support mapping that memory at the host to avoid copying its contents.

One of the benefits of guest_memfd is that it prevents a misbehaving host process from crashing the system when attempting to access (deliberately or accidentally) protected guest memory, since this memory isn't mapped to begin with. Without guest_memfd, the hypervisor would still prevent such accesses, but in certain cases the host kernel wouldn't be able to recover, causing the system to crash.

Support for mmap() in this patch series maintains the invariant that only memory shared with the host, either explicitly by the guest or implicitly before the guest has started running (in order to populate its memory) is allowed to have a valid mapping at the host. At no time should private (as viewed by the hypervisor) guest memory be mapped at the host.

This patch series is divided into two parts:

The first part is to the KVM core code. It adds opt-in support for mapping guest memory only as long as it is shared. For that, the host needs to know the mappability status of guest memory. Therefore, the series adds a structure to track whether memory is mappable. This new structure is associated with each guest_memfd object.

The second part of the series adds guest_memfd support for pKVM/arm64.

We don't enforce the invariant that only memory shared with the host can be mapped by the host userspace in file_operations::mmap(), but we enforce it in vm_operations_struct:fault(). On vm_operations_struct::fault(), we check whether the page is allowed to be mapped. If not, we deliver a SIGBUS to the current task, as discussed in the Linux MM Alignment Session on this topic [2].

Currently there's no support for huge pages, which is something we hope to add in the future, and seems to be a hot topic for the upcoming LPC 2024 [3].

Cheers,
/fuad

[1] https://lore.kernel.org/all/[email protected]/

[2] https://lore.kernel.org/all/[email protected]/

[3] https://lpc.events/event/18/sessions/183/#20240919

Fuad Tabba (10):
KVM: Introduce kvm_gmem_get_pfn_locked(), which retains the folio lock
KVM: Add restricted support for mapping guestmem by the host
KVM: Implement kvm_(read|/write)_guest_page for private memory slots
KVM: Add KVM capability to check if guest_memfd can be mapped by the
host
KVM: selftests: guest_memfd mmap() test when mapping is allowed
KVM: arm64: Skip VMA checks for slots without userspace address
KVM: arm64: Do not allow changes to private memory slots
KVM: arm64: Handle guest_memfd()-backed guest page faults
KVM: arm64: arm64 has private memory support when config is enabled
KVM: arm64: Enable private memory kconfig for arm64

 arch/arm64/include/asm/kvm_host.h             |   3 +
 arch/arm64/kvm/Kconfig                        |   1 +
 arch/arm64/kvm/mmu.c                          | 139 +++++++++-
 include/linux/kvm_host.h                      |  72 +++++
 include/uapi/linux/kvm.h                      |   3 +-
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../testing/selftests/kvm/guest_memfd_test.c  |  47 +++-
 virt/kvm/Kconfig                              |   4 +
 virt/kvm/guest_memfd.c                        | 129 ++++++++-
 virt/kvm/kvm_main.c                           | 253 ++++++++++++++++--
 10 files changed, 628 insertions(+), 24 deletions(-)

base-commit: 0c3836482481200ead7b416ca80c68a29cfdaabd

author
#20 lore raw

Hi Ackerley,

Quoted 35 lines · Ackerley Tng
Quoted 32 lines · Fuad Tabba

This series adds restricted mmap() support to guest_memfd, as well as support for guest_memfd on pKVM/arm64. It is based on Linux 6.10.

Main changes since V1 [1]:

- Decoupled whether guest memory is mappable from KVM memory attributes (SeanC)

Mappability is now tracked in the guest_mem object, orthogonal to whether userspace wants the memory to be private or shared. Moreover, the memory attributes capability (i.e., KVM_CAP_MEMORY_ATTRIBUTES) is not enabled for pKVM, since for software-based hypervisors such as pKVM and Gunyah, userspace is informed of the state of the memory via hypervisor exits if needed.

Even if attributes are enabled, this patch series would still work (modulo bugs), without compromising guest memory nor crashing the system.

- Use page_mapped() instead of page_mapcount() to check if page is mapped (DavidH)

- Add a new capability, KVM_CAP_GUEST_MEMFD_MAPPABLE, to query whether guest private memory can be mapped (with aforementioned restrictions)

- Add a selftest to check whether memory is mappable when the capability is enabled, and not mappable otherwise. Also, test the effect of punching holes in mapped memory. (DavidH)

Quoted 8 lines
Quoted 2 lines

By design, guest_memfd cannot be mapped, read, or written by the host. In pKVM, memory shared between a protected guest and the

I think we should use "cannot be faulted in" to be clear that guest_memfd can be mmaped but not faulted in.

Would it be better to have all the variables/config macros be something about faultability instead of mappability?

With mappability, I mean having a valid mapping in the host. But like I said in the reply to the other patch, I don't have a strong opinion about this.

Cheers,
/fuad

Quoted 81 lines
Quoted 81 lines

host is shared in-place, unlike the other confidential computing solutions that guest_memfd was originally envisaged for (e.g, TDX). When initializing a guest, as well as when accessing memory shared by the guest with the host, it would be useful to support mapping that memory at the host to avoid copying its contents.

One of the benefits of guest_memfd is that it prevents a misbehaving host process from crashing the system when attempting to access (deliberately or accidentally) protected guest memory, since this memory isn't mapped to begin with. Without guest_memfd, the hypervisor would still prevent such accesses, but in certain cases the host kernel wouldn't be able to recover, causing the system to crash.

Support for mmap() in this patch series maintains the invariant that only memory shared with the host, either explicitly by the guest or implicitly before the guest has started running (in order to populate its memory) is allowed to have a valid mapping at the host. At no time should private (as viewed by the hypervisor) guest memory be mapped at the host.

This patch series is divided into two parts:

The first part is to the KVM core code. It adds opt-in support for mapping guest memory only as long as it is shared. For that, the host needs to know the mappability status of guest memory. Therefore, the series adds a structure to track whether memory is mappable. This new structure is associated with each guest_memfd object.

The second part of the series adds guest_memfd support for pKVM/arm64.

We don't enforce the invariant that only memory shared with the host can be mapped by the host userspace in file_operations::mmap(), but we enforce it in vm_operations_struct:fault(). On vm_operations_struct::fault(), we check whether the page is allowed to be mapped. If not, we deliver a SIGBUS to the current task, as discussed in the Linux MM Alignment Session on this topic [2].

Currently there's no support for huge pages, which is something we hope to add in the future, and seems to be a hot topic for the upcoming LPC 2024 [3].

Cheers,
/fuad

[1] https://lore.kernel.org/all/[email protected]/

[2] https://lore.kernel.org/all/[email protected]/

[3] https://lpc.events/event/18/sessions/183/#20240919

Fuad Tabba (10):
KVM: Introduce kvm_gmem_get_pfn_locked(), which retains the folio lock
KVM: Add restricted support for mapping guestmem by the host
KVM: Implement kvm_(read|/write)_guest_page for private memory slots
KVM: Add KVM capability to check if guest_memfd can be mapped by the
host
KVM: selftests: guest_memfd mmap() test when mapping is allowed
KVM: arm64: Skip VMA checks for slots without userspace address
KVM: arm64: Do not allow changes to private memory slots
KVM: arm64: Handle guest_memfd()-backed guest page faults
KVM: arm64: arm64 has private memory support when config is enabled
KVM: arm64: Enable private memory kconfig for arm64

 arch/arm64/include/asm/kvm_host.h             |   3 +
 arch/arm64/kvm/Kconfig                        |   1 +
 arch/arm64/kvm/mmu.c                          | 139 +++++++++-
 include/linux/kvm_host.h                      |  72 +++++
 include/uapi/linux/kvm.h                      |   3 +-
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../testing/selftests/kvm/guest_memfd_test.c  |  47 +++-
 virt/kvm/Kconfig                              |   4 +
 virt/kvm/guest_memfd.c                        | 129 ++++++++-
 virt/kvm/kvm_main.c                           | 253 ++++++++++++++++--
 10 files changed, 628 insertions(+), 24 deletions(-)

base-commit: 0c3836482481200ead7b416ca80c68a29cfdaabd