Skip to contentSkip to conversation

Add guest_memfd support for arm64

PATCH Applied Fuad Tabba 4 participants 8 messages Jul 12 – Aug 1

Subsystems (2)
lore
Share & tools
lore

Fuad Tabba

author reviewer
#0 lore raw

v3 cover Add guest_memfd support for arm64

Hi folks,

Changes since v2 [1]:
- Assert guest_memfd >= 0 in kvm__register_ram_guest_memfd() (Alexandru)
- Don't split line after = sign (Alexandru)
- Disable MTE when --guest-memfd is used (Alexandru)
- Carried Reviewed-bys

Changes since v1 [2]:
- Use kvm_userspace_memory_region2 only for guest_memfd slots (Suzuki)
- Check KVM_CAP_GUEST_MEMFD before KVM_CAP_GUEST_MEMFD_FLAGS (Alexandru)
- Close guest_memfd fd after unmapping at teardown (Alexandru)
- Carried Reviewed-bys

Due to popular demand, i.e., Alexandru asking me about it, I repolished my guest_memfd support series. The one I sent years ago was stale and over-engineered, so this is a fresh start instead.

This series adds support for backing guest RAM with guest_memfd on arm64, for non-protected VMs under both regular KVM and pKVM.

The first two patches are standalone fixes in the code this series builds on: an uninitialized return value in kvm__for_each_mem_bank(), and doubled newlines in die() messages. Patch 3 adds guest_memfd plumbing to kvm__register_mem(), using kvm_userspace_memory_region2 only for guest_memfd-backed slots.

A new --guest-memfd option creates guest RAM from a guest_memfd with mmap support (GUEST_MEMFD_FLAG_MMAP and GUEST_MEMFD_FLAG_INIT_SHARED, Linux 6.18+), maps it to provide the userspace mapping, and registers it with KVM_SET_USER_MEMORY_REGION2.

Protected VMs need in-place shared/private conversion of guest_memfd, which is still in progress in the kernel [3], so --protected rejects the option.

Based on kvmtool master (ca0ddafa941d).

Cheers,
/fuad

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/

Fuad Tabba (4):
Initialize the return value in kvm__for_each_mem_bank() Remove newline from end of die() aborts
Add support for registering guest_memfd-backed memory regions
arm64: Add --guest-memfd option to back guest RAM with guest_memfd

 arm64/include/kvm/kvm-config-arch.h |  5 ++-
 arm64/kvm.c                         | 44 +++++++++++++++++++++++----
 hw/cfi_flash.c                      |  2 +-
 include/kvm/kvm.h                   | 17 +++++++++--
 include/kvm/util.h                  |  1 +
 kvm.c                               | 42 ++++++++++++++++++--------
 util/util.c                         | 47 ++++++++++++++++++++++++++---
 7 files changed, 131 insertions(+), 27 deletions(-)
Signature · 1 line
2.39.5
author reviewer
#1 lore raw

PATCH v3 1/4 Initialize the return value in kvm__for_each_mem_bank()

If none of the bank types match, the function would return an uninitialized value.

---
 kvm.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
kvm.c +3 -2
index 7a52c6a..95ccfcb 100644
@@ -383,14 +383,15 @@ u64 host_to_guest_flat(struct kvm *kvm, void *ptr)
  * their type.
  *
  * If one call to @fun returns a non-zero value, stop iterating and return the
- * value. Otherwise, return zero.
+ * value. If none of the bank types match, return -ENODEV. Otherwise, return
+ * zero.
  */
 int kvm__for_each_mem_bank(struct kvm *kvm, enum kvm_mem_type type,
 			   int (*fun)(struct kvm *kvm, struct kvm_mem_bank *bank, void *data),
 			   void *data)
 {
-	int ret;
 	struct kvm_mem_bank *bank;
+	int ret = -ENODEV;
 
 	list_for_each_entry(bank, &kvm->mem_banks, list) {
 		if (type != KVM_MEM_TYPE_ALL && !(bank->type & type))
Signature · 1 line
2.39.5
author reviewer
#2 lore raw

PATCH v3 2/4 Remove newline from end of die() aborts

In reply to Fuad Tabba

die() already adds the newline at the end of the string.

---
 util/util.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
util/util.c +5 -5
index 6f41e2c..05d49c4 100644
@@ -128,14 +128,14 @@ void *mmap_hugetlbfs(struct kvm *kvm, const char *htlbfs_path, u64 size)
 	unsigned long blk_size;
 
 	if (statfs(htlbfs_path, &sfs) < 0)
-		die("Can't stat %s\n", htlbfs_path);
+		die("Can't stat %s", htlbfs_path);
 
 	if ((unsigned int)sfs.f_type != HUGETLBFS_MAGIC)
-		die("%s is not hugetlbfs!\n", htlbfs_path);
+		die("%s is not hugetlbfs!", htlbfs_path);
 
 	blk_size = (unsigned long)sfs.f_bsize;
 	if (sfs.f_bsize == 0 || blk_size > size) {
-		die("Can't use hugetlbfs pagesize %ld for mem size %lld\n",
+		die("Can't use hugetlbfs pagesize %ld for mem size %lld",
 			blk_size, (unsigned long long)size);
 	}
 
@@ -144,10 +144,10 @@ void *mmap_hugetlbfs(struct kvm *kvm, const char *htlbfs_path, u64 size)
 	snprintf(mpath, PATH_MAX, "%s/kvmtoolXXXXXX", htlbfs_path);
 	fd = mkstemp(mpath);
 	if (fd < 0)
-		die("Can't open %s for hugetlbfs map\n", mpath);
+		die("Can't open %s for hugetlbfs map", mpath);
 	unlink(mpath);
 	if (ftruncate(fd, size) < 0)
-		die("Can't ftruncate for mem mapping size %lld\n",
+		die("Can't ftruncate for mem mapping size %lld",
 			(unsigned long long)size);
 	addr = mmap(NULL, size, PROT_RW, MAP_PRIVATE, fd, 0);
 	close(fd);
Signature · 1 line
2.39.5
author reviewer
#3 lore raw

PATCH v3 3/4 Add support for registering guest_memfd-backed memory regions

In reply to Fuad Tabba

Extend kvm__register_mem() with a guest_memfd file descriptor and offset. A bank with a guest_memfd is registered with KVM_SET_USER_MEMORY_REGION2 and KVM_MEM_GUEST_MEMFD, binding the memslot to the file. This prepares for backing guest RAM with guest_memfd on arm64.

---
 hw/cfi_flash.c    |  2 +-
 include/kvm/kvm.h | 16 ++++++++++++++--
 kvm.c             | 36 ++++++++++++++++++++++++++----------
 3 files changed, 41 insertions(+), 13 deletions(-)
hw/cfi_flash.c +1 -1
index b68d9e0..4413955 100644
@@ -451,7 +451,7 @@ static int map_flash_memory(struct kvm *kvm, struct cfi_flash_device *sfdev)
 	int ret;
 
 	ret = kvm__register_mem(kvm, sfdev->base_addr, sfdev->size,
-				sfdev->flash_memory,
+				sfdev->flash_memory, -1, 0,
 				KVM_MEM_TYPE_RAM | KVM_MEM_TYPE_READONLY);
 	if (!ret)
 		sfdev->is_mapped = true;
include/kvm/kvm.h +14 -2
index eb23e2f..077aaee 100644
@@ -128,11 +128,23 @@ bool kvm__emulate_io(struct kvm_cpu *vcpu, u16 port, void *data, int direction,
 bool kvm__emulate_mmio(struct kvm_cpu *vcpu, u64 phys_addr, u8 *data, u32 len, u8 is_write);
 int kvm__destroy_mem(struct kvm *kvm, u64 guest_phys, u64 size, void *userspace_addr);
 int kvm__register_mem(struct kvm *kvm, u64 guest_phys, u64 size, void *userspace_addr,
+		      int guest_memfd, u64 guest_memfd_offset,
 		      enum kvm_mem_type type);
 static inline int kvm__register_ram(struct kvm *kvm, u64 guest_phys, u64 size,
 				    void *userspace_addr)
 {
 	return kvm__register_mem(kvm, guest_phys, size, userspace_addr,
+				 -1, 0, KVM_MEM_TYPE_RAM);
+}
+
+static inline int kvm__register_ram_guest_memfd(struct kvm *kvm, u64 guest_phys,
+						u64 size, void *userspace_addr,
+						int guest_memfd,
+						u64 guest_memfd_offset)
+{
+	BUG_ON(guest_memfd < 0);
+	return kvm__register_mem(kvm, guest_phys, size, userspace_addr,
+				 guest_memfd, guest_memfd_offset,
 				 KVM_MEM_TYPE_RAM);
 }
 
@@ -140,13 +152,13 @@ static inline int kvm__register_dev_mem(struct kvm *kvm, u64 guest_phys,
 					u64 size, void *userspace_addr)
 {
 	return kvm__register_mem(kvm, guest_phys, size, userspace_addr,
-				 KVM_MEM_TYPE_DEVICE);
+				 -1, 0, KVM_MEM_TYPE_DEVICE);
 }
 
 static inline int kvm__reserve_mem(struct kvm *kvm, u64 guest_phys, u64 size)
 {
 	return kvm__register_mem(kvm, guest_phys, size, NULL,
-				 KVM_MEM_TYPE_RESERVED);
+				 -1, 0, KVM_MEM_TYPE_RESERVED);
 }
 
 int __must_check kvm__register_iotrap(struct kvm *kvm, u64 phys_addr, u64 len,
kvm.c +26 -10
index 95ccfcb..947721b 100644
@@ -237,9 +237,9 @@ out:
 }
 
 int kvm__register_mem(struct kvm *kvm, u64 guest_phys, u64 size,
-		      void *userspace_addr, enum kvm_mem_type type)
+		      void *userspace_addr, int guest_memfd,
+		      u64 guest_memfd_offset, enum kvm_mem_type type)
 {
-	struct kvm_userspace_memory_region mem;
 	struct kvm_mem_bank *merged = NULL;
 	struct kvm_mem_bank *bank;
 	struct list_head *prev_entry;
@@ -320,15 +320,31 @@ int kvm__register_mem(struct kvm *kvm, u64 guest_phys, u64 size,
 		flags |= KVM_MEM_READONLY;
 
 	if (type != KVM_MEM_TYPE_RESERVED) {
-		mem = (struct kvm_userspace_memory_region) {
-			.slot			= slot,
-			.flags			= flags,
-			.guest_phys_addr	= guest_phys,
-			.memory_size		= size,
-			.userspace_addr		= (unsigned long)userspace_addr,
-		};
+		if (guest_memfd >= 0) {
+			struct kvm_userspace_memory_region2 mem2 = {
+				.slot			= slot,
+				.flags			= flags | KVM_MEM_GUEST_MEMFD,
+				.guest_phys_addr	= guest_phys,
+				.memory_size		= size,
+				.userspace_addr		= (unsigned long)userspace_addr,
+				.guest_memfd		= guest_memfd,
+				.guest_memfd_offset	= guest_memfd_offset,
+			};
 
-		ret = ioctl(kvm->vm_fd, KVM_SET_USER_MEMORY_REGION, &mem);
+			ret = ioctl(kvm->vm_fd, KVM_SET_USER_MEMORY_REGION2,
+				    &mem2);
+		} else {
+			struct kvm_userspace_memory_region mem = {
+				.slot			= slot,
+				.flags			= flags,
+				.guest_phys_addr	= guest_phys,
+				.memory_size		= size,
+				.userspace_addr		= (unsigned long)userspace_addr,
+			};
+
+			ret = ioctl(kvm->vm_fd, KVM_SET_USER_MEMORY_REGION,
+				    &mem);
+		}
 		if (ret < 0) {
 			ret = -errno;
 			goto out;
Signature · 1 line
2.39.5
author reviewer
#4 lore raw

PATCH v3 4/4 arm64: Add --guest-memfd option to back guest RAM with guest_memfd

In reply to Fuad Tabba

Back guest RAM with a guest_memfd created with GUEST_MEMFD_FLAG_MMAP and GUEST_MEMFD_FLAG_INIT_SHARED, mmapped MAP_SHARED to provide the userspace mapping. The guest_memfd file descriptor is closed at teardown, after the mapping is removed.

MTE is disabled when --guest-memfd is used: guest_memfd VMAs do not have VM_MTE_ALLOWED, so KVM rejects the memslot if MTE is enabled.

Protected VMs need in-place shared/private conversion of guest_memfd, which the kernel does not support yet, so reject the combination of --protected and --guest-memfd.

---
 arm64/include/kvm/kvm-config-arch.h |  5 +++-
 arm64/kvm.c                         | 44 +++++++++++++++++++++++++----
 include/kvm/kvm.h                   |  1 +
 include/kvm/util.h                  |  1 +
 kvm.c                               |  1 +
 util/util.c                         | 37 ++++++++++++++++++++++++
 6 files changed, 82 insertions(+), 7 deletions(-)
arm64/include/kvm/kvm-config-arch.h +4 -1
index c2702d5..d8a8ef7 100644
@@ -20,6 +20,7 @@ struct kvm_config_arch {
 	bool		no_pvtime;
 	bool		psci;
 	bool		protected;
+	bool		guest_memfd;
 };
 
 int irqchip_parser(const struct option *opt, const char *arg, int unset);
@@ -73,6 +74,8 @@ int sve_vl_parser(const struct option *opt, const char *arg, int unset);
 	OPT_BOOLEAN('\0', "e2h0", &(cfg)->e2h0,					\
 		    "Create guest without VHE support"),			\
 	OPT_BOOLEAN('\0', "protected", &(cfg)->protected,			\
-			"Create a protected VM when pKVM is enabled"),
+			"Create a protected VM when pKVM is enabled"),		\
+	OPT_BOOLEAN('\0', "guest-memfd", &(cfg)->guest_memfd,			\
+			"Use guest_memfd to back guest RAM"),
 
 #endif /* ARM_COMMON__KVM_CONFIG_ARCH_H */
arm64/kvm.c +38 -6

80 lines not on this page · raw message

include/kvm/kvm.h +1 -0
index 077aaee..a9376b6 100644
@@ -90,6 +90,7 @@ struct kvm {
 	u64			ram_size;	/* Guest memory size, in bytes */
 	void			*ram_start;
 	u64			ram_pagesize;
+	int			ram_guest_memfd; /* guest_memfd backing RAM, -1 if none */
 	struct mutex		mem_banks_lock;
 	struct list_head	mem_banks;
 
include/kvm/util.h +1 -0
index a6dba11..9e23431 100644
@@ -148,5 +148,6 @@ static inline int pow2_size(unsigned long x)
 struct kvm;
 void *mmap_hugetlbfs(struct kvm *kvm, const char *htlbfs_path, u64 size);
 void *mmap_anon_or_hugetlbfs(struct kvm *kvm, const char *hugetlbfs_path, u64 size);
+void *mmap_guest_memfd(struct kvm *kvm, u64 size);
 
 #endif /* KVM__UTIL_H */
kvm.c +1 -0
index 947721b..96583f9 100644
@@ -162,6 +162,7 @@ struct kvm *kvm__new(void)
 	mutex_init(&kvm->mem_banks_lock);
 	kvm->sys_fd = -1;
 	kvm->vm_fd = -1;
+	kvm->ram_guest_memfd = -1;
 
 #ifdef KVM_BRLOCK_DEBUG
 	kvm->brlock_sem = (pthread_rwlock_t) PTHREAD_RWLOCK_INITIALIZER;
util/util.c +37 -0
index 05d49c4..51a4591 100644
@@ -6,7 +6,9 @@
 #include "kvm/util.h"
 
 #include <kvm/kvm.h>
+#include <linux/kvm.h>
 #include <linux/magic.h>	/* For HUGETLBFS_MAGIC */
+#include <sys/ioctl.h>
 #include <sys/mman.h>
 #include <sys/stat.h>
 #include <sys/statfs.h>
@@ -169,3 +171,38 @@ void *mmap_anon_or_hugetlbfs(struct kvm *kvm, const char *hugetlbfs_path, u64 si
 		return mmap(NULL, size, PROT_RW, MAP_ANON_NORESERVE, -1, 0);
 	}
 }
+
+/*
+ * Map guest RAM backed by a guest_memfd. The file descriptor is kept open
+ * until teardown (kvm__arch_delete_ram), after the mapping is removed.
+ */
+void *mmap_guest_memfd(struct kvm *kvm, u64 size)
+{
+	u64 flags = GUEST_MEMFD_FLAG_MMAP | GUEST_MEMFD_FLAG_INIT_SHARED;
+	struct kvm_create_guest_memfd gmem = {
+		.size	= size,
+		.flags	= flags,
+	};
+	void *addr;
+	int ret;
+
+	if (!kvm__supports_extension(kvm, KVM_CAP_GUEST_MEMFD))
+		die("kernel does not support guest_memfd");
+
+	ret = ioctl(kvm->vm_fd, KVM_CHECK_EXTENSION, KVM_CAP_GUEST_MEMFD_FLAGS);
+	if (ret < 0 || ((u64)ret & flags) != flags)
+		die("VM type does not support guest_memfd with mmap");
+
+	ret = ioctl(kvm->vm_fd, KVM_CREATE_GUEST_MEMFD, &gmem);
+	if (ret < 0)
+		die_perror("KVM_CREATE_GUEST_MEMFD");
+
+	addr = mmap(NULL, size, PROT_RW, MAP_SHARED, ret, 0);
+	if (addr == MAP_FAILED)
+		die_perror("guest_memfd mmap");
+
+	kvm->ram_guest_memfd = ret;
+	kvm->ram_pagesize = getpagesize();
+
+	return addr;
+}
Signature · 1 line
2.39.5
Reviewed-by Tested-by on the v3 series
#5 lore raw
In reply to Fuad Tabba

Hi,

Quoted 32 lines · Fuad Tabba

Hi folks,

Changes since v2 [1]:
- Assert guest_memfd >= 0 in kvm__register_ram_guest_memfd() (Alexandru)
- Don't split line after = sign (Alexandru)
- Disable MTE when --guest-memfd is used (Alexandru)
- Carried Reviewed-bys

Changes since v1 [2]:
- Use kvm_userspace_memory_region2 only for guest_memfd slots (Suzuki)
- Check KVM_CAP_GUEST_MEMFD before KVM_CAP_GUEST_MEMFD_FLAGS (Alexandru)
- Close guest_memfd fd after unmapping at teardown (Alexandru)
- Carried Reviewed-bys

Due to popular demand, i.e., Alexandru asking me about it, I repolished my guest_memfd support series. The one I sent years ago was stale and over-engineered, so this is a fresh start instead.

This series adds support for backing guest RAM with guest_memfd on arm64, for non-protected VMs under both regular KVM and pKVM.

The first two patches are standalone fixes in the code this series builds on: an uninitialized return value in kvm__for_each_mem_bank(), and doubled newlines in die() messages. Patch 3 adds guest_memfd plumbing to kvm__register_mem(), using kvm_userspace_memory_region2 only for guest_memfd-backed slots.

A new --guest-memfd option creates guest RAM from a guest_memfd with mmap support (GUEST_MEMFD_FLAG_MMAP and GUEST_MEMFD_FLAG_INIT_SHARED, Linux 6.18+), maps it to provide the userspace mapping, and registers it with KVM_SET_USER_MEMORY_REGION2.

Quoted 5 lines

Protected VMs need in-place shared/private conversion of guest_memfd, which is still in progress in the kernel [3], so --protected rejects the option.

Based on kvmtool master (ca0ddafa941d).

Looks good to me:

Tested it on an Orion O6 board, by booting a VM with guest_memfd. And also used it as a base for testing the MTE + guest_memfd patch [1]:

[1] https://lore.kernel.org/kvmarm/[email protected]/

Thanks,
Alex

Quoted 27 lines

Cheers,
/fuad

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/

Fuad Tabba (4):
Initialize the return value in kvm__for_each_mem_bank() Remove newline from end of die() aborts
Add support for registering guest_memfd-backed memory regions
arm64: Add --guest-memfd option to back guest RAM with guest_memfd

 arm64/include/kvm/kvm-config-arch.h |  5 ++-
 arm64/kvm.c                         | 44 +++++++++++++++++++++++----
 hw/cfi_flash.c                      |  2 +-
 include/kvm/kvm.h                   | 17 +++++++++--
 include/kvm/util.h                  |  1 +
 kvm.c                               | 42 ++++++++++++++++++--------
 util/util.c                         | 47 ++++++++++++++++++++++++++---
 7 files changed, 131 insertions(+), 27 deletions(-)
Signature · 1 line
2.39.5
reviewer Reviewed-by on the v3 series
#6 lore raw
In reply to Fuad Tabba

Hi Fuad

Quoted 35 lines · Fuad Tabba

Hi folks,

Due to popular demand, i.e., Alexandru asking me about it, I repolished my guest_memfd support series. The one I sent years ago was stale and over-engineered, so this is a fresh start instead.

This series adds support for backing guest RAM with guest_memfd on arm64, for non-protected VMs under both regular KVM and pKVM.

The first two patches are standalone fixes in the code this series builds on: an uninitialized return value in kvm__for_each_mem_bank(), and doubled newlines in die() messages. Patch 3 adds guest_memfd plumbing to kvm__register_mem(), using kvm_userspace_memory_region2 only for guest_memfd-backed slots.

A new --guest-memfd option creates guest RAM from a guest_memfd with mmap support (GUEST_MEMFD_FLAG_MMAP and GUEST_MEMFD_FLAG_INIT_SHARED, Linux 6.18+), maps it to provide the userspace mapping, and registers it with KVM_SET_USER_MEMORY_REGION2.

Protected VMs need in-place shared/private conversion of guest_memfd, which is still in progress in the kernel [3], so --protected rejects the option.

Based on kvmtool master (ca0ddafa941d).

Cheers,
/fuad

[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/all/[email protected]/
[3] https://lore.kernel.org/all/[email protected]/

Quoted 5 lines

Fuad Tabba (4):
Initialize the return value in kvm__for_each_mem_bank() Remove newline from end of die() aborts
Add support for registering guest_memfd-backed memory regions
arm64: Add --guest-memfd option to back guest RAM with guest_memfd

Thanks for reworking this again ! For the series:

Applied
#7 lore raw
In reply to Fuad Tabba
Quoted 6 lines · Fuad Tabba

Changes since v2 [1]:
- Assert guest_memfd >= 0 in kvm__register_ram_guest_memfd() (Alexandru)
- Don't split line after = sign (Alexandru)
- Disable MTE when --guest-memfd is used (Alexandru)
- Carried Reviewed-bys

Quoted 7 lines

Changes since v1 [2]:
- Use kvm_userspace_memory_region2 only for guest_memfd slots (Suzuki)
- Check KVM_CAP_GUEST_MEMFD before KVM_CAP_GUEST_MEMFD_FLAGS (Alexandru)
- Close guest_memfd fd after unmapping at teardown (Alexandru)
- Carried Reviewed-bys

[...]

Applied to kvmtool (master), thanks!

[1/4] Initialize the return value in kvm__for_each_mem_bank()

      https://git.kernel.org/will/kvmtool/c/8e77acfac060
[2/4] Remove newline from end of die() aborts
      https://git.kernel.org/will/kvmtool/c/a6d971e56ee2
[3/4] Add support for registering guest_memfd-backed memory regions
      https://git.kernel.org/will/kvmtool/c/3f5bf00c2449
[4/4] arm64: Add --guest-memfd option to back guest RAM with guest_memfd
      https://git.kernel.org/will/kvmtool/c/0717297de3d5

Cheers,

Signature · 5 lines
Will

https://fixes.arm64.dev
https://next.arm64.dev
https://will.arm64.dev