Skip to contentSkip to conversation

Setting memory policy for restrictedmem file

RFC PATCH In review Ackerley Tng 3 participants 9 messages Apr 14, 2023

Subsystems (13)
lore
Share & tools
lore
#0 lore raw

cover Setting memory policy for restrictedmem file

Hello,

This patchset builds upon the memfd_restricted() system call that was discussed in the 'KVM: mm: fd-based approach for supporting KVM' patch series [1].

The tree can be found at:
https://github.com/googleprodkernel/linux-cc/tree/restrictedmem-set-memory-policy

In this patchset, a new syscall is introduced, which allows userspace to set the memory policy (e.g. NUMA bindings) for a restrictedmem file, to the granularity of offsets within the file.

The offset/length tuple is termed a file_range which is passed to the kernel via a pointer to get around the limit of 6 arguments for a syscall.

The following other approaches were also considered:

1. Pre-configuring a mount with a memory policy and providing that mount to memfd_restricted() as proposed at [2].
+ Pro: It allows choice of a specific backing mount with custom memory policy configurations
+ Con: Will need to create an entire new mount just to set memory policy for a restrictedmem file; files on the same mount cannot have different memory policies.

2. Passing memory policy to the memfd_restricted() syscall at creation time.
+ Pro: Only need to make a single syscall to create a file with a given memory policy
+ Con: At creation time, the kernel doesn’t know the size of the restrictedmem file. Given that memory policy is stored in the inode based on ranges (start, end), it is awkward for the kernel to store the memory policy and then add hooks to set the memory policy when allocation is done.

3. A more generic fbind(): it seems like this new functionality is really only needed for restrictedmem files, hence a separate, specific syscall was proposed to avoid complexities with handling conflicting policies that may be specified via other syscalls like mbind()

TODOs

+ Return -EINVAL if file_range is not within the size of the file and tests for this

Dependencies:

+ Chao’s work on UPM [3]

[1] https://lore.kernel.org/lkml/[email protected]/T/
[2] https://lore.kernel.org/lkml/[email protected]/T/
[3] https://github.com/chao-p/linux/commits/privmem-v11.5

---

Ackerley Tng (6):
mm: shmem: Refactor out shmem_shared_policy() function
mm: mempolicy: Refactor out mpol_init_from_nodemask
mm: mempolicy: Refactor out __mpol_set_shared_policy()
mm: mempolicy: Add and expose mpol_create
mm: restrictedmem: Add memfd_restricted_bind() syscall
selftests: mm: Add selftest for memfd_restricted_bind()

 arch/x86/entry/syscalls/syscall_32.tbl        |   1 +
 arch/x86/entry/syscalls/syscall_64.tbl        |   1 +
 include/linux/mempolicy.h                     |   4 +
 include/linux/shmem_fs.h                      |   7 +
 include/linux/syscalls.h                      |   5 +
 include/uapi/asm-generic/unistd.h             |   5 +-
 include/uapi/linux/mempolicy.h                |   7 +-
 kernel/sys_ni.c                               |   1 +
 mm/mempolicy.c                                | 100 ++++++++++---
 mm/restrictedmem.c                            |  75 ++++++++++
 mm/shmem.c                                    |  10 +-
 scripts/checksyscalls.sh                      |   1 +
 tools/testing/selftests/mm/.gitignore         |   1 +
 tools/testing/selftests/mm/Makefile           |   8 +
 .../selftests/mm/memfd_restricted_bind.c      | 139 ++++++++++++++++++
 .../mm/restrictedmem_testmod/Makefile         |  21 +++
 .../restrictedmem_testmod.c                   |  89 +++++++++++
 tools/testing/selftests/mm/run_vmtests.sh     |   6 +
 18 files changed, 454 insertions(+), 27 deletions(-)
 create mode 100644 tools/testing/selftests/mm/memfd_restricted_bind.c
 create mode 100644 tools/testing/selftests/mm/restrictedmem_testmod/Makefile
 create mode 100644 tools/testing/selftests/mm/restrictedmem_testmod/restrictedmem_testmod.c

--
2.40.0.634.g4ca3ef3211-goog

#1 lore raw

RFC PATCH 1/6 mm: shmem: Refactor out shmem_shared_policy() function

Refactor out shmem_shared_policy() to allow reading of a file's shared mempolicy

---
 include/linux/shmem_fs.h |  7 +++++++
 mm/shmem.c               | 10 ++++++----
 2 files changed, 13 insertions(+), 4 deletions(-)
include/linux/shmem_fs.h +7 -0
index d9e57485a686..bc1eeb4b4bd9 100644
@@ -134,6 +134,13 @@ static inline bool shmem_file(struct file *file)
 	return shmem_mapping(file->f_mapping);
 }
 
+static inline struct shared_policy *shmem_shared_policy(struct file *file)
+{
+	struct inode *inode = file_inode(file);
+
+	return &SHMEM_I(inode)->policy;
+}
+
 /*
  * If fallocate(FALLOC_FL_KEEP_SIZE) has been used, there may be pages
  * beyond i_size's notion of EOF, which fallocate has committed to reserving:
mm/shmem.c +6 -4
index b053cd1f12da..4f801f398454 100644
@@ -2248,20 +2248,22 @@ unsigned long shmem_get_unmapped_area(struct file *file,
 }
 
 #ifdef CONFIG_NUMA
+
 static int shmem_set_policy(struct vm_area_struct *vma, struct mempolicy *mpol)
 {
-	struct inode *inode = file_inode(vma->vm_file);
-	return mpol_set_shared_policy(&SHMEM_I(inode)->policy, vma, mpol);
+	struct shared_policy *info;
+
+	info = shmem_shared_policy(vma->vm_file);
+	return mpol_set_shared_policy(info, vma, mpol);
 }
 
 static struct mempolicy *shmem_get_policy(struct vm_area_struct *vma,
 					  unsigned long addr)
 {
-	struct inode *inode = file_inode(vma->vm_file);
 	pgoff_t index;
 
 	index = ((addr - vma->vm_start) >> PAGE_SHIFT) + vma->vm_pgoff;
-	return mpol_shared_policy_lookup(&SHMEM_I(inode)->policy, index);
+	return mpol_shared_policy_lookup(shmem_shared_policy(vma->vm_file), index);
 }
 #endif
 
Signature · 1 line
2.40.0.634.g4ca3ef3211-goog
#2 lore raw

RFC PATCH 2/6 mm: mempolicy: Refactor out mpol_init_from_nodemask

In reply to Ackerley Tng

Refactor out mpol_init_from_nodemask() to simplify logic in do_mbind().

mpol_init_from_nodemask() will be used to perform similar functionality in do_memfd_restricted_bind() in a later patch.

---
 mm/mempolicy.c | 32 +++++++++++++++++++++-----------
 1 file changed, 21 insertions(+), 11 deletions(-)
mm/mempolicy.c +21 -11
index a256a241fd1d..a2655b626731 100644
@@ -1254,6 +1254,25 @@ static struct page *new_page(struct page *page, unsigned long start)
 }
 #endif
 
+static long mpol_init_from_nodemask(struct mempolicy *mpol, const nodemask_t *nmask,
+				    bool always_unlock)
+{
+	long err;
+	NODEMASK_SCRATCH(scratch);
+
+	if (!scratch)
+		return -ENOMEM;
+
+	/* Cannot take lock before allocating in NODEMASK_SCRATCH */
+	mmap_write_lock(current->mm);
+	err = mpol_set_nodemask(mpol, nmask, scratch);
+	if (always_unlock || err)
+		mmap_write_unlock(current->mm);
+
+	NODEMASK_SCRATCH_FREE(scratch);
+	return err;
+}
+
 static long do_mbind(unsigned long start, unsigned long len,
 		     unsigned short mode, unsigned short mode_flags,
 		     nodemask_t *nmask, unsigned long flags)
@@ -1306,17 +1325,8 @@ static long do_mbind(unsigned long start, unsigned long len,
 
 		lru_cache_disable();
 	}
-	{
-		NODEMASK_SCRATCH(scratch);
-		if (scratch) {
-			mmap_write_lock(mm);
-			err = mpol_set_nodemask(new, nmask, scratch);
-			if (err)
-				mmap_write_unlock(mm);
-		} else
-			err = -ENOMEM;
-		NODEMASK_SCRATCH_FREE(scratch);
-	}
+
+	err = mpol_init_from_nodemask(new, nmask, false);
 	if (err)
 		goto mpol_out;
 
Signature · 1 line
2.40.0.634.g4ca3ef3211-goog
#3 lore raw

RFC PATCH 3/6 mm: mempolicy: Refactor out __mpol_set_shared_policy()

In reply to Ackerley Tng

Refactor out __mpol_set_shared_policy() to remove dependency on struct vm_area_struct, since only 2 parameters from struct vm_area_struct are used.

__mpol_set_shared_policy() will be used in a later patch by restrictedmem_set_shared_policy().

---
 include/linux/mempolicy.h |  2 ++
 mm/mempolicy.c            | 29 +++++++++++++++++++----------
 2 files changed, 21 insertions(+), 10 deletions(-)
include/linux/mempolicy.h +2 -0
index d232de7cdc56..9a2a2dd95432 100644
@@ -126,6 +126,8 @@ struct shared_policy {
 
 int vma_dup_policy(struct vm_area_struct *src, struct vm_area_struct *dst);
 void mpol_shared_policy_init(struct shared_policy *sp, struct mempolicy *mpol);
+int __mpol_set_shared_policy(struct shared_policy *info, struct mempolicy *mpol,
+			     unsigned long pgoff_start, unsigned long npages);
 int mpol_set_shared_policy(struct shared_policy *info,
 				struct vm_area_struct *vma,
 				struct mempolicy *new);
mm/mempolicy.c +19 -10
index a2655b626731..f3fa5494e4a8 100644
@@ -2817,30 +2817,39 @@ void mpol_shared_policy_init(struct shared_policy *sp, struct mempolicy *mpol)
 	}
 }
 
-int mpol_set_shared_policy(struct shared_policy *info,
-			struct vm_area_struct *vma, struct mempolicy *npol)
+int __mpol_set_shared_policy(struct shared_policy *info, struct mempolicy *mpol,
+			     unsigned long pgoff_start, unsigned long npages)
 {
 	int err;
 	struct sp_node *new = NULL;
-	unsigned long sz = vma_pages(vma);
+	unsigned long pgoff_end = pgoff_start + npages;
 
 	pr_debug("set_shared_policy %lx sz %lu %d %d %lx\n",
-		 vma->vm_pgoff,
-		 sz, npol ? npol->mode : -1,
-		 npol ? npol->flags : -1,
-		 npol ? nodes_addr(npol->nodes)[0] : NUMA_NO_NODE);
+		 pgoff_start, npages,
+		 mpol ? mpol->mode : -1,
+		 mpol ? mpol->flags : -1,
+		 mpol ? nodes_addr(mpol->nodes)[0] : NUMA_NO_NODE);
 
-	if (npol) {
-		new = sp_alloc(vma->vm_pgoff, vma->vm_pgoff + sz, npol);
+	if (mpol) {
+		new = sp_alloc(pgoff_start, pgoff_end, mpol);
 		if (!new)
 			return -ENOMEM;
 	}
-	err = shared_policy_replace(info, vma->vm_pgoff, vma->vm_pgoff+sz, new);
+
+	err = shared_policy_replace(info, pgoff_start, pgoff_end, new);
+
 	if (err && new)
 		sp_free(new);
+
 	return err;
 }
 
+int mpol_set_shared_policy(struct shared_policy *info,
+			struct vm_area_struct *vma, struct mempolicy *mpol)
+{
+	return __mpol_set_shared_policy(info, mpol, vma->vm_pgoff, vma_pages(vma));
+}
+
 /* Free a backing policy store on inode delete. */
 void mpol_free_shared_policy(struct shared_policy *p)
 {
Signature · 1 line
2.40.0.634.g4ca3ef3211-goog
#4 lore raw

RFC PATCH 4/6 mm: mempolicy: Add and expose mpol_create

In reply to Ackerley Tng

mpol_create builds a mempolicy based on mode, nmask and maxnode.

mpol_create is exposed for use in memfd_restricted_bind() in a later patch.

---
 include/linux/mempolicy.h |  2 ++
 mm/mempolicy.c            | 39 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 41 insertions(+)
include/linux/mempolicy.h +2 -0
index 9a2a2dd95432..15facd9de087 100644
@@ -125,6 +125,8 @@ struct shared_policy {
 };
 
 int vma_dup_policy(struct vm_area_struct *src, struct vm_area_struct *dst);
+struct mempolicy *mpol_create(
+	unsigned long mode, const unsigned long __user *nmask, unsigned long maxnode)
 void mpol_shared_policy_init(struct shared_policy *sp, struct mempolicy *mpol);
 int __mpol_set_shared_policy(struct shared_policy *info, struct mempolicy *mpol,
 			     unsigned long pgoff_start, unsigned long npages);
mm/mempolicy.c +39 -0
index f3fa5494e4a8..f4fe241c17ff 100644
@@ -3181,3 +3181,42 @@ void mpol_to_str(char *buffer, int maxlen, struct mempolicy *pol)
 		p += scnprintf(p, buffer + maxlen - p, ":%*pbl",
 			       nodemask_pr_args(&nodes));
 }
+
+/**
+ * mpol_create - build mempolicy based on mode, nmask and maxnode
+ * @mode:  policy mode, as in MPOL_MODE_FLAGS
+ * @nmask:  node mask from userspace
+ * @maxnode:  number of valid bits in nmask
+ *
+ * Will allocate a new struct mempolicy that has to be released with
+ * mpol_put. Will also take and release the write lock mmap_lock in current->mm.
+ */
+struct mempolicy *mpol_create(
+	unsigned long mode, const unsigned long __user *nmask, unsigned long maxnode)
+{
+	int err;
+	unsigned short mode_flags;
+	nodemask_t nodes;
+	int lmode = mode;
+	struct mempolicy *mpol;
+
+	err = sanitize_mpol_flags(&lmode, &mode_flags);
+	if (err)
+		return ERR_PTR(err);
+
+	err = get_nodes(&nodes, nmask, maxnode);
+	if (err)
+		return ERR_PTR(err);
+
+	mpol = mpol_new(mode, mode_flags, &nodes);
+	if (IS_ERR(mpol))
+		return mpol;
+
+	err = mpol_init_from_nodemask(mpol, &nodes, true);
+	if (err) {
+		mpol_put(mpol);
+		return ERR_PTR(err);
+	}
+
+	return mpol;
+}
Signature · 1 line
2.40.0.634.g4ca3ef3211-goog
#5 lore raw

RFC PATCH 5/6 mm: restrictedmem: Add memfd_restricted_bind() syscall

In reply to Ackerley Tng

memfd_restricted_bind() sets the NUMA memory policy, which consists of a policy mode and zero or more nodes, for an offset within a restrictedmem file with file descriptor fd and continuing for len bytes.

This is intended to be like mbind() but specially for restrictedmem files, which cannot be mmap()ed into userspace and hence has no memory addresses that can be used with mbind().

Unlike mbind(), memfd_restricted_bind() will override any existing memory policy if a new memory policy is defined for the same ranges.

For now, memfd_restricted_bind() does not perform migrations and no flags are supported.

This syscall is specialised just for restrictedmem files because this functionality is not required by other files.

---
 arch/x86/entry/syscalls/syscall_32.tbl |  1 +
 arch/x86/entry/syscalls/syscall_64.tbl |  1 +
 include/linux/mempolicy.h              |  2 +-
 include/linux/syscalls.h               |  5 ++
 include/uapi/asm-generic/unistd.h      |  5 +-
 include/uapi/linux/mempolicy.h         |  7 ++-
 kernel/sys_ni.c                        |  1 +
 mm/restrictedmem.c                     | 75 ++++++++++++++++++++++++++
 scripts/checksyscalls.sh               |  1 +
 9 files changed, 95 insertions(+), 3 deletions(-)
arch/x86/entry/syscalls/syscall_32.tbl +1 -0
index dc70ba90247e..c94e9ce46cc3 100644
@@ -456,3 +456,4 @@
 449	i386	futex_waitv		sys_futex_waitv
 450	i386	set_mempolicy_home_node		sys_set_mempolicy_home_node
 451	i386	memfd_restricted	sys_memfd_restricted
+452	i386	memfd_restricted_bind	sys_memfd_restricted_bind
arch/x86/entry/syscalls/syscall_64.tbl +1 -0
index 06516abc8318..6bd86b45d63a 100644
@@ -373,6 +373,7 @@
 449	common	futex_waitv		sys_futex_waitv
 450	common	set_mempolicy_home_node	sys_set_mempolicy_home_node
 451	common	memfd_restricted	sys_memfd_restricted
+452	common	memfd_restricted_bind	sys_memfd_restricted_bind
 
 #
 # Due to a historical design error, certain syscalls are numbered differently
include/linux/mempolicy.h +1 -1
index 15facd9de087..af62233df0c0 100644
@@ -126,7 +126,7 @@ struct shared_policy {
 
 int vma_dup_policy(struct vm_area_struct *src, struct vm_area_struct *dst);
 struct mempolicy *mpol_create(
-	unsigned long mode, const unsigned long __user *nmask, unsigned long maxnode)
+	unsigned long mode, const unsigned long __user *nmask, unsigned long maxnode);
 void mpol_shared_policy_init(struct shared_policy *sp, struct mempolicy *mpol);
 int __mpol_set_shared_policy(struct shared_policy *info, struct mempolicy *mpol,
 			     unsigned long pgoff_start, unsigned long npages);
include/linux/syscalls.h +5 -0
index 660be0bf89d5..852b202d3837 100644
@@ -1059,6 +1059,11 @@ asmlinkage long sys_set_mempolicy_home_node(unsigned long start, unsigned long l
 					    unsigned long home_node,
 					    unsigned long flags);
 asmlinkage long sys_memfd_restricted(unsigned int flags);
+asmlinkage long sys_memfd_restricted_bind(int fd, struct file_range __user *range,
+					  unsigned long mode,
+					  const unsigned long __user *nmask,
+					  unsigned long maxnode,
+					  unsigned int flags);
 
 /*
  * Architecture-specific system calls
include/uapi/asm-generic/unistd.h +4 -1
index e2ea7cd964f8..b5a1385bb4a7 100644
@@ -889,10 +889,13 @@ __SYSCALL(__NR_set_mempolicy_home_node, sys_set_mempolicy_home_node)
 #ifdef __ARCH_WANT_MEMFD_RESTRICTED
 #define __NR_memfd_restricted 451
 __SYSCALL(__NR_memfd_restricted, sys_memfd_restricted)
+
+#define __NR_memfd_restricted_bind 452
+__SYSCALL(__NR_memfd_restricted_bind, sys_memfd_restricted_bind)
 #endif
 
 #undef __NR_syscalls
-#define __NR_syscalls 452
+#define __NR_syscalls 453
 
 /*
  * 32 bit systems traditionally used different
include/uapi/linux/mempolicy.h +6 -1
index 046d0ccba4cd..979499abd253 100644
@@ -6,9 +6,9 @@
 #ifndef _UAPI_LINUX_MEMPOLICY_H
 #define _UAPI_LINUX_MEMPOLICY_H
 
+#include <asm-generic/posix_types.h>
 #include <linux/errno.h>
 
-
 /*
  * Both the MPOL_* mempolicy mode and the MPOL_F_* optional mode flags are
  * passed by the user to either set_mempolicy() or mbind() in an 'int' actual.
@@ -72,4 +72,9 @@ enum {
 #define RECLAIM_WRITE	(1<<1)	/* Writeout pages during reclaim */
 #define RECLAIM_UNMAP	(1<<2)	/* Unmap pages during reclaim */
 
+struct file_range {
+	__kernel_loff_t offset;
+	__kernel_size_t len;
+};
+
 #endif /* _UAPI_LINUX_MEMPOLICY_H */
kernel/sys_ni.c +1 -0
index 7c4a32cbd2e7..db24d3fe6dc5 100644
@@ -362,6 +362,7 @@ COND_SYSCALL(memfd_secret);
 
 /* memfd_restricted */
 COND_SYSCALL(memfd_restricted);
+COND_SYSCALL(memfd_restricted_bind);
 
 /*
  * Architecture specific weak syscall entries.
mm/restrictedmem.c +75 -0

82 lines not on this page · raw message

scripts/checksyscalls.sh +1 -0
index 3c4d2508226a..e253529cf1ec 100755
@@ -46,6 +46,7 @@ cat << EOF
 
 #ifndef __ARCH_WANT_MEMFD_RESTRICTED
 #define __IGNORE_memfd_restricted
+#define __IGNORE_memfd_restricted_bind
 #endif
 
 /* Missing flags argument */
Signature · 1 line
2.40.0.634.g4ca3ef3211-goog
#6 lore raw

RFC PATCH 6/6 selftests: mm: Add selftest for memfd_restricted_bind()

In reply to Ackerley Tng

This selftest uses memfd_restricted_bind() to set the mempolicy for a restrictedmem file, and then checks that pages were indeed allocated according to that policy.

Because restrictedmem pages are never mapped into userspace memory, the usual ways of checking which NUMA node the page was allocated on (e.g. /proc/pid/numa_maps) cannot be used.

This selftest adds a small kernel module that overloads the ioctl syscall on /proc/restrictedmem to request a restrictedmem page and get the node it was allocated on. The page is freed within the ioctl handler.

---
 tools/testing/selftests/mm/.gitignore         |   1 +
 tools/testing/selftests/mm/Makefile           |   8 +
 .../selftests/mm/memfd_restricted_bind.c      | 139 ++++++++++++++++++
 .../mm/restrictedmem_testmod/Makefile         |  21 +++
 .../restrictedmem_testmod.c                   |  89 +++++++++++
 tools/testing/selftests/mm/run_vmtests.sh     |   6 +
 6 files changed, 264 insertions(+)
 create mode 100644 tools/testing/selftests/mm/memfd_restricted_bind.c
 create mode 100644 tools/testing/selftests/mm/restrictedmem_testmod/Makefile
 create mode 100644 tools/testing/selftests/mm/restrictedmem_testmod/restrictedmem_testmod.c
tools/testing/selftests/mm/.gitignore +1 -0
index fb6e4233374d..10c5701b9645 100644
@@ -31,6 +31,7 @@ map_fixed_noreplace
 write_to_hugetlbfs
 hmm-tests
 memfd_restricted
+memfd_restricted_bind
 memfd_secret
 soft-dirty
 split_huge_page_test
tools/testing/selftests/mm/Makefile +8 -0
index 5ec338ea1fed..4a6cf922db45 100644
@@ -46,6 +46,8 @@ TEST_GEN_FILES += map_fixed_noreplace
 TEST_GEN_FILES += map_hugetlb
 TEST_GEN_FILES += map_populate
 TEST_GEN_FILES += memfd_restricted
+TEST_GEN_FILES += memfd_restricted_bind
+TEST_GEN_FILES += restrictedmem_testmod.ko
 TEST_GEN_FILES += memfd_secret
 TEST_GEN_FILES += migration
 TEST_GEN_FILES += mlock-random-test
@@ -171,6 +173,12 @@ $(OUTPUT)/ksm_tests: LDLIBS += -lnuma
 
 $(OUTPUT)/migration: LDLIBS += -lnuma
 
+$(OUTPUT)/memfd_restricted_bind: LDLIBS += -lnuma
+$(OUTPUT)/restrictedmem_testmod.ko: $(wildcard restrictedmem_testmod/Makefile restrictedmem_testmod/*.[ch])
+	$(call msg,MOD,,$@)
+	$(Q)$(MAKE) -C restrictedmem_testmod
+	$(Q)cp restrictedmem_testmod/restrictedmem_testmod.ko $@
+
 local_config.mk local_config.h: check_config.sh
 	/bin/sh ./check_config.sh $(CC)
 
tools/testing/selftests/mm/memfd_restricted_bind.c new file +139 -0

139 lines not on this page · raw message

tools/testing/selftests/mm/restrictedmem_testmod/Makefile new file +21 -0
new file mode 100644
index 000000000000..11b1d5d15e3c
@@ -0,0 +1,21 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+RESTRICTEDMEM_TESTMOD_DIR := $(realpath $(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
+KDIR ?= $(abspath $(RESTRICTEDMEM_TESTMOD_DIR)/../../../../..)
+
+ifeq ($(V),1)
+Q =
+else
+Q = @
+endif
+
+MODULES = restrictedmem_testmod.ko
+
+obj-m += restrictedmem_testmod.o
+CFLAGS_restrictedmem_testmod.o = -I$(src)
+
+all:
+	+$(Q)make -C $(KDIR) M=$(RESTRICTEDMEM_TESTMOD_DIR) modules
+
+clean:
+	+$(Q)make -C $(KDIR) M=$(RESTRICTEDMEM_TESTMOD_DIR) clean
tools/testing/selftests/mm/restrictedmem_testmod/restrictedmem_testmod.c new file +89 -0

89 lines not on this page · raw message

tools/testing/selftests/mm/run_vmtests.sh +6 -0
index 53de84e3ec2c..bdc853d6afe4 100644
@@ -40,6 +40,8 @@ separated by spaces:
 	test memadvise(2) MADV_POPULATE_{READ,WRITE} options
 - memfd_restricted_
 	test memfd_restricted(2)
+- memfd_restricted_bind
+	test memfd_restricted_bind(2)
 - memfd_secret
 	test memfd_secret(2)
 - process_mrelease
@@ -240,6 +242,10 @@ CATEGORY="madv_populate" run_test ./madv_populate
 
 CATEGORY="memfd_restricted" run_test ./memfd_restricted
 
+test_selected "memfd_restricted_bind" && insmod ./restrictedmem_testmod.ko && \
+  CATEGORY="memfd_restricted_bind" run_test ./memfd_restricted_bind && \
+  rmmod restrictedmem_testmod > /dev/null
+
 CATEGORY="memfd_secret" run_test ./memfd_secret
 
 # KSM MADV_MERGEABLE test with 10 identical pages
Signature · 1 line
2.40.0.634.g4ca3ef3211-goog
reviewer
#7 lore raw
In reply to Ackerley Tng
Quoted 19 lines · Ackerley Tng

Hello,

This patchset builds upon the memfd_restricted() system call that was discussed in the 'KVM: mm: fd-based approach for supporting KVM' patch series [1].

The tree can be found at:
https://github.com/googleprodkernel/linux-cc/tree/restrictedmem-set-memory-policy

In this patchset, a new syscall is introduced, which allows userspace to set the memory policy (e.g. NUMA bindings) for a restrictedmem file, to the granularity of offsets within the file.

The offset/length tuple is termed a file_range which is passed to the kernel via a pointer to get around the limit of 6 arguments for a syscall.

The following other approaches were also considered:

Quoted 7 lines

1. Pre-configuring a mount with a memory policy and providing that mount to memfd_restricted() as proposed at [2].
+ Pro: It allows choice of a specific backing mount with custom memory policy configurations
+ Con: Will need to create an entire new mount just to set memory policy for a restrictedmem file; files on the same mount cannot have different memory policies.

Could you expand on this some more please? How many restricted files/mounts do we expect? My understanding was that this would be essentially a backing store for guest memory so it would scale with the number of guests.

Quoted 9 lines

2. Passing memory policy to the memfd_restricted() syscall at creation time.
+ Pro: Only need to make a single syscall to create a file with a given memory policy
+ Con: At creation time, the kernel doesn’t know the size of the restrictedmem file. Given that memory policy is stored in the inode based on ranges (start, end), it is awkward for the kernel to store the memory policy and then add hooks to set the memory policy when allocation is done.

Quoted 5 lines

3. A more generic fbind(): it seems like this new functionality is really only needed for restrictedmem files, hence a separate, specific syscall was proposed to avoid complexities with handling conflicting policies that may be specified via other syscalls like mbind()

I do not think it is a good idea to make the syscall restrict mem specific. History shows that users are much more creative when it comes to usecases than us. I do understand that the nature of restricted memory is that it is not mapable but memory policies without a mapping are a reasonable concept in genereal. After all this just tells where the memory should be allocated from. Do we need to implement that for any other fs? No, you can safely return EINVAL for anything but memfd_restricted fd for now but you shouldn't limit usecases upfront.

Quoted 2 lines

TODOs

How do you query a policy for the specific fd? Are there any plans to add a syscall for that as well but you just wait for the direction for the set method?

Quoted 46 lines

+ Return -EINVAL if file_range is not within the size of the file and tests for this

Dependencies:

+ Chao’s work on UPM [3]

[1] https://lore.kernel.org/lkml/[email protected]/T/
[2] https://lore.kernel.org/lkml/[email protected]/T/
[3] https://github.com/chao-p/linux/commits/privmem-v11.5

---

Ackerley Tng (6):
mm: shmem: Refactor out shmem_shared_policy() function
mm: mempolicy: Refactor out mpol_init_from_nodemask
mm: mempolicy: Refactor out __mpol_set_shared_policy()
mm: mempolicy: Add and expose mpol_create
mm: restrictedmem: Add memfd_restricted_bind() syscall
selftests: mm: Add selftest for memfd_restricted_bind()

 arch/x86/entry/syscalls/syscall_32.tbl        |   1 +
 arch/x86/entry/syscalls/syscall_64.tbl        |   1 +
 include/linux/mempolicy.h                     |   4 +
 include/linux/shmem_fs.h                      |   7 +
 include/linux/syscalls.h                      |   5 +
 include/uapi/asm-generic/unistd.h             |   5 +-
 include/uapi/linux/mempolicy.h                |   7 +-
 kernel/sys_ni.c                               |   1 +
 mm/mempolicy.c                                | 100 ++++++++++---
 mm/restrictedmem.c                            |  75 ++++++++++
 mm/shmem.c                                    |  10 +-
 scripts/checksyscalls.sh                      |   1 +
 tools/testing/selftests/mm/.gitignore         |   1 +
 tools/testing/selftests/mm/Makefile           |   8 +
 .../selftests/mm/memfd_restricted_bind.c      | 139 ++++++++++++++++++
 .../mm/restrictedmem_testmod/Makefile         |  21 +++
 .../restrictedmem_testmod.c                   |  89 +++++++++++
 tools/testing/selftests/mm/run_vmtests.sh     |   6 +
 18 files changed, 454 insertions(+), 27 deletions(-)
 create mode 100644 tools/testing/selftests/mm/memfd_restricted_bind.c
 create mode 100644 tools/testing/selftests/mm/restrictedmem_testmod/Makefile
 create mode 100644 tools/testing/selftests/mm/restrictedmem_testmod/restrictedmem_testmod.c

--
2.40.0.634.g4ca3ef3211-goog

Signature · 2 lines
Michal Hocko
SUSE Labs
#8 lore raw
Quoted 9 lines · Michal Hocko
Quoted 5 lines · Ackerley Tng

3. A more generic fbind(): it seems like this new functionality is really only needed for restrictedmem files, hence a separate, specific syscall was proposed to avoid complexities with handling conflicting policies that may be specified via other syscalls like mbind()

I do not think it is a good idea to make the syscall restrict mem specific.

+1. IMO, any uAPI that isn't directly related to the fundamental properties of restricted memory, i.e. isn't truly unique to restrictedmem, should be added as generic fd-based uAPI.

Quoted 7 lines

History shows that users are much more creative when it comes to usecases than us. I do understand that the nature of restricted memory is that it is not mapable but memory policies without a mapping are a reasonable concept in genereal. After all this just tells where the memory should be allocated from. Do we need to implement that for any other fs? No, you can safely return EINVAL for anything but memfd_restricted fd for now but you shouldn't limit usecases upfront.

I would even go a step further and say that we should seriously reconsider the design/implemenation of memfd_restricted() if a generic fbind() needs explicit handling from the restricted memory code. One of the goals with memfd_restricted() is to rely on the underlying backing store to handle all of the "normal" behaviors.