mm: add new syscall set_mempolicy_home_node
Subsystems (20)
v4 cover mm: add new syscall set_mempolicy_home_node
Changes from v3:
* Fix build warning reported by kernel test robot
Changes from RFC v2:
* Rebase to latest kernel
* Update numa_memory_policy.rst
Aneesh Kumar K.V (3):
mm/mempolicy: use policy_node helper with MPOL_PREFERRED_MANY mm/mempolicy: add set_mempolicy_home_node syscall mm/mempolicy: wire up syscall set_mempolicy_home_node
.../admin-guide/mm/numa_memory_policy.rst | 14 ++++- arch/alpha/kernel/syscalls/syscall.tbl | 1 + arch/arm/tools/syscall.tbl | 1 + arch/arm64/include/asm/unistd.h | 2 +- arch/arm64/include/asm/unistd32.h | 2 + arch/ia64/kernel/syscalls/syscall.tbl | 1 + arch/m68k/kernel/syscalls/syscall.tbl | 1 + arch/microblaze/kernel/syscalls/syscall.tbl | 1 + arch/mips/kernel/syscalls/syscall_n32.tbl | 1 + arch/mips/kernel/syscalls/syscall_n64.tbl | 1 + arch/mips/kernel/syscalls/syscall_o32.tbl | 1 + arch/parisc/kernel/syscalls/syscall.tbl | 1 + arch/powerpc/kernel/syscalls/syscall.tbl | 1 + arch/s390/kernel/syscalls/syscall.tbl | 1 + arch/sh/kernel/syscalls/syscall.tbl | 1 + arch/sparc/kernel/syscalls/syscall.tbl | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/xtensa/kernel/syscalls/syscall.tbl | 1 + include/linux/mempolicy.h | 1 + include/linux/syscalls.h | 3 + include/uapi/asm-generic/unistd.h | 4 +- kernel/sys_ni.c | 1 + mm/mempolicy.c | 59 ++++++++++++++++++- 24 files changed, 97 insertions(+), 5 deletions(-)
Signature · 1 line
2.31.1
PATCH v4 1/3 mm/mempolicy: use policy_node helper with MPOL_PREFERRED_MANY
A followup patch will enable setting a home node with MPOL_PREFERRED_MANY memory policy. To facilitate that switch to using policy_node helper. There is no functional change in this patch.
--- mm/mempolicy.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
mm/mempolicy.c +3 -2
@@ -2061,7 +2061,7 @@ static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int order, preferred_gfp &= ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); page =__alloc_pages (preferred_gfp, order, nid, &pol->nodes);if (!page) - page =__alloc_pages (gfp, order,numa_node_id (),NULL ); + page =__alloc_pages (gfp, order, nid,NULL );return page; }
@@ -2102,6 +2102,7 @@ struct page *alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma, }if (pol->mode ==MPOL_PREFERRED_MANY ) { + node =policy_node (gfp, pol, node); page =alloc_pages_preferred_many (gfp, order, node, pol);mpol_cond_put (pol); goto out;
@@ -2186,7 +2187,7 @@ struct page *alloc_pages(gfp_t gfp, unsigned order) page =alloc_page_interleave (gfp, order,interleave_nodes (pol)); else if (pol->mode == MPOL_PREFERRED_MANY) page =alloc_pages_preferred_many (gfp, order, -numa_node_id (), pol); +policy_node (gfp, pol,numa_node_id ()), pol); else page = __alloc_pages(gfp, order, policy_node(gfp, pol, numa_node_id()),
Signature · 1 line
2.31.1
PATCH v4 2/3 mm/mempolicy: add set_mempolicy_home_node syscall
In reply to Aneesh Kumar K.VThis syscall can be used to set a home node for the MPOL_BIND and MPOL_PREFERRED_MANY memory policy. Users should use this syscall after setting up a memory policy for the specified range as shown below.
mbind(p, nr_pages * page_size, MPOL_BIND, new_nodes->maskp,
new_nodes->size + 1, 0); sys_set_mempolicy_home_node((unsigned long)p, nr_pages * page_size, home_node, 0);
The syscall allows specifying a home node/preferred node from which kernel will fulfill memory allocation requests first.
For address range with MPOL_BIND memory policy, if nodemask specifies more than one node, page allocations will come from the node in the nodemask with sufficient free memory that is closest to the home node/preferred node.
For MPOL_PREFERRED_MANY if the nodemask specifies more than one node, page allocation will come from the node in the nodemask with sufficient free memory that is closest to the home node/preferred node. If there is not enough memory in all the nodes specified in the nodemask, the allocation will be attempted from the closest numa node to the home node in the system.
This helps applications to hint at a memory allocation preference node and fallback to _only_ a set of nodes if the memory is not available on the preferred node. Fallback allocation is attempted from the node which is nearest to the preferred node.
This helps applications to have control on memory allocation numa nodes and avoids default fallback to slow memory NUMA nodes. For example a system with NUMA nodes 1,2 and 3 with DRAM memory and 10, 11 and 12 of slow memory
new_nodes = numa_bitmask_alloc(nr_nodes);
numa_bitmask_setbit(new_nodes, 1);
numa_bitmask_setbit(new_nodes, 2);
numa_bitmask_setbit(new_nodes, 3);
p = mmap(NULL, nr_pages * page_size, protflag, mapflag, -1, 0); mbind(p, nr_pages * page_size, MPOL_BIND, new_nodes->maskp, new_nodes->size + 1, 0);
sys_set_mempolicy_home_node(p, nr_pages * page_size, 2, 0);
This will allocate from nodes closer to node 2 and will make sure kernel will only allocate from nodes 1, 2 and3. Memory will not be allocated from slow memory nodes 10, 11 and 12
With MPOL_PREFERRED_MANY on the other hand will first try to allocate from the closest node to node 2 from the node list 1, 2 and 3. If those nodes don't have enough memory, kernel will allocate from slow memory node 10, 11 and 12 which ever is closer to node 2.
--- .../admin-guide/mm/numa_memory_policy.rst | 14 ++++- include/linux/mempolicy.h | 1 + mm/mempolicy.c | 54 +++++++++++++++++++ 3 files changed, 68 insertions(+), 1 deletion(-)
Documentation/admin-guide/mm/numa_memory_policy.rst +13 -1
@@ -408,7 +408,7 @@ follows: Memory Policy APIs ================== -Linux supports 3 system calls for controlling memory policy. These APIS +Linux supports 4 system calls for controlling memory policy. These APIS always affect only the calling task, the calling task's address space, or some shared object mapped into the calling task's address space.
@@ -460,6 +460,18 @@ requested via the 'flags' argument. See the mbind(2) man page for more details. +Set home node for a Range of Task's Address Spacec:: + + long sys_set_mempolicy_home_node(unsigned long start, unsigned long len, + unsigned long home_node, + unsigned long flags); + +sys_set_mempolicy_home_node set the home node for a VMA policy present in the +task's address range. The system call updates the home node only for the existing +mempolicy range. Other address ranges are ignored. A home node is the NUMA node +closest to which page allocation will come from. + + Memory Policy Command Line Interface ====================================
include/linux/mempolicy.h +1 -0
@@ -47,6 +47,7 @@ struct mempolicy {unsigned short mode;/* See MPOL_* above */ unsigned short flags;/* See set_mempolicy() MPOL_F_* above */ nodemask_t nodes;/* interleave/bind/perfer */ +int home_node;/* Home node to use for MPOL_BIND and MPOL_PREFERRED_MANY */ union {nodemask_t cpuset_mems_allowed;/* relative to these nodes */
mm/mempolicy.c +54 -0
78 lines not on this page · raw message
Signature · 1 line
2.31.1
Jonathan Corbet
maintainerRe: [PATCH v4 2/3] mm/mempolicy: add set_mempolicy_home_node syscall
Quoted 4 lines · Aneesh Kumar K.V
This syscall can be used to set a home node for the
MPOL_BINDandMPOL_PREFERRED_MANYmemory policy. Users should use this syscall after setting up a memory policy for the specified range as shown below.
So I noticed one little thing as I was looking at this...
[...]
Quoted 35 lines
mm/mempolicy.c +35
-0+SYSCALL_DEFINE4(set_mempolicy_home_node, unsigned long, start, unsigned long, len, + unsigned long, home_node, unsigned long, flags) +{ +struct mm_struct *mm = current->mm; +struct vm_area_struct *vma; +struct mempolicy *new; +unsigned long vmstart; +unsigned long vmend; +unsigned long end; +int err = -ENOENT ; + +if (start & ~PAGE_MASK ) +return -EINVAL ; + + len = (len +PAGE_SIZE -1 ) &PAGE_MASK ; + end = start + len; + +if (end < start) +return -EINVAL ; +if (end == start) +return 0 ; +mmap_write_lock (mm); + vma =find_vma (mm, start); +for (; vma && vma->vm_start < end; vma = vma->vm_next) { + + vmstart =max (start, vma->vm_start); + vmend =min (end, vma->vm_end); + new =mpol_dup (vma_policy (vma)); +if (IS_ERR (new)) { + err =PTR_ERR (new); +break ; + } +/* +* Only update home node if there is an existing vma policy +*/
mm/mempolicy.c static long kernel_mbind(unsigned long start, unsigned long len, · line 1525
Quoted 10 lines
mm/mempolicy.c +10
-0+if (!new) +continue ; + new->home_node = home_node; + err =mbind_range (mm, vmstart, vmend, new); +if (err) +break ; + } +mmap_write_unlock (mm); +return err; +}
You never do anything with flags; I believe you want to check and ensure that it's zero if there are no defined flags at this point?
Thanks,
jon
Re: [PATCH v4 2/3] mm/mempolicy: add set_mempolicy_home_node syscall
Quoted 51 lines · Jonathan Corbet
Quoted 4 lines · Aneesh Kumar K.V
This syscall can be used to set a home node for the
MPOL_BINDandMPOL_PREFERRED_MANYmemory policy. Users should use this syscall after setting up a memory policy for the specified range as shown below.So I noticed one little thing as I was looking at this...
[...]
Quoted 40 lines
mm/mempolicy.c +40
-0+SYSCALL_DEFINE4(set_mempolicy_home_node, unsigned long, start, unsigned long, len, + unsigned long, home_node, unsigned long, flags) +{ +struct mm_struct *mm = current->mm; +struct vm_area_struct *vma; +struct mempolicy *new; +unsigned long vmstart; +unsigned long vmend; +unsigned long end; +int err = -ENOENT ; + +if (start & ~PAGE_MASK ) +return -EINVAL ; + + len = (len +PAGE_SIZE -1 ) &PAGE_MASK ; + end = start + len; + +if (end < start) +return -EINVAL ; +if (end == start) +return 0 ; +mmap_write_lock (mm); + vma =find_vma (mm, start); +for (; vma && vma->vm_start < end; vma = vma->vm_next) { + + vmstart =max (start, vma->vm_start); + vmend =min (end, vma->vm_end); + new =mpol_dup (vma_policy (vma)); +if (IS_ERR (new)) { + err =PTR_ERR (new); +break ; + } +/* +* Only update home node if there is an existing vma policy +*/ +if (!new) +continue ; + new->home_node = home_node; + err =mbind_range (mm, vmstart, vmend, new); +if (err)
Quoted 10 lines
mm/mempolicy.c static long kernel_mbind(unsigned long start, unsigned long len, · line 1525
Quoted 5 lines
mm/mempolicy.c +5
-0+break ; + } +mmap_write_unlock (mm); +return err; +}You never do anything with flags; I believe you want to check and ensure that it's zero if there are no defined flags at this point?
Will update that in v5. I guess EINVAL is the right error for flags != 0?
-aneesh
PATCH v4 3/3 mm/mempolicy: wire up syscall set_mempolicy_home_node
In reply to Aneesh Kumar K.VCc: Ben Widawsky <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: Feng Tang <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: Andrea Arcangeli <[email protected]>
Cc: Mel Gorman <[email protected]>
Cc: Mike Kravetz <[email protected]>
Cc: Randy Dunlap <[email protected]>
Cc: Vlastimil Babka <[email protected]>
Cc: Andi Kleen <[email protected]>
Cc: Dan Williams <[email protected]>
Cc: Huang Ying <[email protected]>
Cc: [email protected]
Fix build failure
--- arch/alpha/kernel/syscalls/syscall.tbl | 1 + arch/arm/tools/syscall.tbl | 1 + arch/arm64/include/asm/unistd.h | 2 +- arch/arm64/include/asm/unistd32.h | 2 ++ arch/ia64/kernel/syscalls/syscall.tbl | 1 + arch/m68k/kernel/syscalls/syscall.tbl | 1 + arch/microblaze/kernel/syscalls/syscall.tbl | 1 + arch/mips/kernel/syscalls/syscall_n32.tbl | 1 + arch/mips/kernel/syscalls/syscall_n64.tbl | 1 + arch/mips/kernel/syscalls/syscall_o32.tbl | 1 + arch/parisc/kernel/syscalls/syscall.tbl | 1 + arch/powerpc/kernel/syscalls/syscall.tbl | 1 + arch/s390/kernel/syscalls/syscall.tbl | 1 + arch/sh/kernel/syscalls/syscall.tbl | 1 + arch/sparc/kernel/syscalls/syscall.tbl | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/xtensa/kernel/syscalls/syscall.tbl | 1 + include/linux/syscalls.h | 3 +++ include/uapi/asm-generic/unistd.h | 4 +++- kernel/sys_ni.c | 1 + 21 files changed, 26 insertions(+), 2 deletions(-)
arch/alpha/kernel/syscalls/syscall.tbl +1 -0
@@ -488,3 +488,4 @@ 556 common landlock_restrict_self sys_landlock_restrict_self # 557 reserved for memfd_secret 558 common process_mrelease sys_process_mrelease +559 common set_mempolicy_home_node sys_ni_syscall
arch/arm/tools/syscall.tbl +1 -0
@@ -462,3 +462,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node
arch/arm64/include/asm/unistd.h +1 -1
@@ -38,7 +38,7 @@#define __ARM_NR_compat_set_tls (__ARM_NR_COMPAT_BASE + 5)#define __ARM_NR_COMPAT_END (__ARM_NR_COMPAT_BASE + 0x800) -#define __NR_compat_syscalls 449 +#define __NR_compat_syscalls 450#endif #define __ARCH_WANT_SYS_CLONE
arch/arm64/include/asm/unistd32.h +2 -0
@@ -903,6 +903,8 @@ __SYSCALL(__NR_landlock_add_rule, sys_landlock_add_rule) __SYSCALL(__NR_landlock_restrict_self, sys_landlock_restrict_self)#define __NR_process_mrelease 448 __SYSCALL(__NR_process_mrelease, sys_process_mrelease) +#define __NR_set_mempolicy_home_node 449 +__SYSCALL (__NR_set_mempolicy_home_node, sys_set_mempolicy_home_node)/* * Please add new compat syscalls above this comment and update
arch/ia64/kernel/syscalls/syscall.tbl +1 -0
@@ -369,3 +369,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node
arch/m68k/kernel/syscalls/syscall.tbl +1 -0
@@ -448,3 +448,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node
arch/microblaze/kernel/syscalls/syscall.tbl +1 -0
@@ -454,3 +454,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node
arch/mips/kernel/syscalls/syscall_n32.tbl +1 -0
@@ -387,3 +387,4 @@ 446 n32 landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 n32 process_mrelease sys_process_mrelease +449 n32 set_mempolicy_home_node sys_set_mempolicy_home_node
arch/mips/kernel/syscalls/syscall_n64.tbl +1 -0
@@ -363,3 +363,4 @@ 446 n64 landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 n64 process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node
arch/mips/kernel/syscalls/syscall_o32.tbl +1 -0
@@ -436,3 +436,4 @@ 446 o32 landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 o32 process_mrelease sys_process_mrelease +449 o32 set_mempolicy_home_node sys_set_mempolicy_home_node
arch/parisc/kernel/syscalls/syscall.tbl +1 -0
@@ -446,3 +446,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node
arch/powerpc/kernel/syscalls/syscall.tbl +1 -0
@@ -528,3 +528,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease +449 nospu set_mempolicy_home_node sys_set_mempolicy_home_node
arch/s390/kernel/syscalls/syscall.tbl +1 -0
@@ -451,3 +451,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node sys_set_mempolicy_home_node
arch/sh/kernel/syscalls/syscall.tbl +1 -0
@@ -451,3 +451,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node
arch/sparc/kernel/syscalls/syscall.tbl +1 -0
@@ -494,3 +494,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node
arch/x86/entry/syscalls/syscall_32.tbl +1 -0
@@ -453,3 +453,4 @@ 446 i386 landlock_restrict_self sys_landlock_restrict_self 447 i386 memfd_secret sys_memfd_secret 448 i386 process_mrelease sys_process_mrelease +449 i386 set_mempolicy_home_node sys_set_mempolicy_home_node
arch/x86/entry/syscalls/syscall_64.tbl +1 -0
@@ -370,6 +370,7 @@ 446 common landlock_restrict_self sys_landlock_restrict_self 447 common memfd_secret sys_memfd_secret 448 common process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node # # Due to a historical design error, certain syscalls are numbered differently
arch/xtensa/kernel/syscalls/syscall.tbl +1 -0
@@ -419,3 +419,4 @@ 446 common landlock_restrict_self sys_landlock_restrict_self # 447 reserved for memfd_secret 448 common process_mrelease sys_process_mrelease +449 common set_mempolicy_home_node sys_set_mempolicy_home_node
include/linux/syscalls.h +3 -0
@@ -1052,6 +1052,9 @@ asmlinkage long sys_landlock_add_rule(int ruleset_fd, enum landlock_rule_type ru const void __user *rule_attr, __u32 flags);asmlinkage long sys_landlock_restrict_self (int ruleset_fd,__u32 flags);asmlinkage long sys_memfd_secret (unsigned int flags); +asmlinkage long sys_set_mempolicy_home_node (unsigned long start,unsigned long len, +unsigned long home_node, +unsigned long flags);/* * Architecture-specific system calls
include/uapi/asm-generic/unistd.h +3 -1
@@ -879,9 +879,11 @@ __SYSCALL(__NR_memfd_secret, sys_memfd_secret)#endif #define __NR_process_mrelease 448 __SYSCALL(__NR_process_mrelease, sys_process_mrelease) +#define __NR_set_mempolicy_home_node 449 +__SYSCALL(__NR_set_mempolicy_home_node, sys_set_mempolicy_home_node)#undef __NR_syscalls -#define __NR_syscalls 449 +#define __NR_syscalls 450/* * 32 bit systems traditionally used different
kernel/sys_ni.c +1 -0
@@ -296,6 +296,7 @@ COND_SYSCALL(get_mempolicy);COND_SYSCALL (set_mempolicy);COND_SYSCALL (migrate_pages);COND_SYSCALL (move_pages); +COND_SYSCALL (set_mempolicy_home_node);COND_SYSCALL (perf_event_open);COND_SYSCALL (accept4);
Signature · 1 line
2.31.1