mm: introduce process_mrelease system call
Subsystems (18)
PATCH v7 1/2 mm: introduce process_mrelease system call
In modern systems it's not unusual to have a system component monitoring memory conditions of the system and tasked with keeping system memory pressure under control. One way to accomplish that is to kill non-essential processes to free up memory for more important ones. Examples of this are Facebook's OOM killer daemon called oomd and Android's low memory killer daemon called lmkd.
For such system component it's important to be able to free memory quickly and efficiently. Unfortunately the time process takes to free up its memory after receiving a SIGKILL might vary based on the state of the process (uninterruptible sleep), size and OPP level of the core the process is running. A mechanism to free resources of the target process in a more predictable way would improve system's ability to control its memory pressure.
Introduce process_mrelease system call that releases memory of a dying process from the context of the caller. This way the memory is freed in a more controllable way with CPU affinity and priority of the caller. The workload of freeing the memory will also be charged to the caller. The operation is allowed only on a dying process.
After previous discussions [1, 2, 3] the decision was made [4] to introduce a dedicated system call to cover this use case.
The API is as follows,
int process_mrelease(int pidfd, unsigned int flags);
DESCRIPTION
The process_mrelease() system call is used to free the memory of an exiting process.
The pidfd selects the process referred to by the PID file descriptor.
(See pidfd_open(2) for further information)
The flags argument is reserved for future use; currently, this argument must be specified as 0.
RETURN VALUE
On success, process_mrelease() returns 0. On error, -1 is returned and errno is set to indicate the error.
ERRORS
EBADF pidfd is not a valid PID file descriptor.
EAGAIN Failed to release part of the address space.
EINTR The call was interrupted by a signal; see signal(7).
EINVAL flags is not 0.
EINVAL The memory of the task cannot be released because the
process is not exiting, the address space is shared with another live process or there is a core dump in progress.
ENOSYS This system call is not supported, for example, without
MMU support built into Linux.
ESRCH The target process does not exist (i.e., it has terminated
and been waited on).
[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/linux-api/[email protected]/
[3] https://lore.kernel.org/linux-api/[email protected]/
[4] https://lore.kernel.org/linux-api/[email protected]/
---
changes in v7:
- Fixed pidfd_open misspelling, per Andrew Morton
- Fixed wrong task pinning after find_lock_task_mm() issue, per Michal Hocko
- Moved MMF_OOM_SKIP check before task_will_free_mem(), per Michal Hocko
mm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)
mm/oom_kill.c +73 -0
@@ -28,6 +28,7 @@#include <linux/sched/task.h> #include <linux/sched/debug.h> #include <linux/swap.h> +#include <linux/syscalls.h> #include <linux/timex.h> #include <linux/jiffies.h> #include <linux/cpuset.h>
@@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void)out_of_memory (&oc);mutex_unlock (&oom_lock); } + +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) +{ +#ifdef CONFIG_MMU +struct mm_struct *mm =NULL ; +struct task_struct *task; +struct task_struct *p; +unsigned int f_flags; +struct pid *pid; +long ret =0 ; + +if (flags) +return -EINVAL ; + + pid =pidfd_get_pid (pidfd, &f_flags); +if (IS_ERR (pid)) +return PTR_ERR (pid); + + task =get_pid_task (pid,PIDTYPE_PID ); +if (!task) { + ret = -ESRCH ; + goto put_pid; + } + +/* +* If the task is dying and in the process of releasing its memory +* then get its mm. +*/ + p =find_lock_task_mm (task); +if (!p) { + ret = -ESRCH ; + goto put_pid; + } +if (task != p) { +get_task_struct (p); +put_task_struct (task); + task = p; + } + +/* If the work has been done already, just exit with success */ +if (test_bit (MMF_OOM_SKIP , &task->mm->flags)) + goto put_task; + +if (task_will_free_mem (task) && (task->flags &PF_KTHREAD ) ==0 ) { + mm = task->mm; +mmget (mm); + } +task_unlock (task); +if (!mm) { + ret = -EINVAL ; + goto put_task; + } + +if (mmap_read_lock_killable (mm)) { + ret = -EINTR ; + goto put_mm; + } +if (!__oom_reap_task_mm (mm)) + ret = -EAGAIN ; +mmap_read_unlock (mm); + +put_mm: +mmput (mm); +put_task: +put_task_struct (task); +put_pid: +put_pid (pid); +return ret; +#else +return -ENOSYS ; +#endif /* CONFIG_MMU */ +}
Signature · 1 line
2.32.0.554.ge1b32706d8-goog
PATCH v7 2/2 mm: wire up syscall process_mrelease
Split off from prev patch in the series that implements the syscall.
--- arch/alpha/kernel/syscalls/syscall.tbl | 2 ++ arch/arm/tools/syscall.tbl | 2 ++ arch/arm64/include/asm/unistd.h | 2 +- arch/arm64/include/asm/unistd32.h | 2 ++ arch/ia64/kernel/syscalls/syscall.tbl | 2 ++ arch/m68k/kernel/syscalls/syscall.tbl | 2 ++ arch/microblaze/kernel/syscalls/syscall.tbl | 2 ++ arch/mips/kernel/syscalls/syscall_n32.tbl | 2 ++ arch/mips/kernel/syscalls/syscall_n64.tbl | 2 ++ arch/mips/kernel/syscalls/syscall_o32.tbl | 2 ++ arch/parisc/kernel/syscalls/syscall.tbl | 2 ++ arch/powerpc/kernel/syscalls/syscall.tbl | 2 ++ arch/s390/kernel/syscalls/syscall.tbl | 2 ++ arch/sh/kernel/syscalls/syscall.tbl | 2 ++ arch/sparc/kernel/syscalls/syscall.tbl | 2 ++ arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 1 + arch/xtensa/kernel/syscalls/syscall.tbl | 2 ++ include/linux/syscalls.h | 1 + include/uapi/asm-generic/unistd.h | 4 +++- kernel/sys_ni.c | 1 + 21 files changed, 38 insertions(+), 2 deletions(-)
arch/alpha/kernel/syscalls/syscall.tbl +2 -0
@@ -486,3 +486,5 @@ 554 common landlock_create_ruleset sys_landlock_create_ruleset 555 common landlock_add_rule sys_landlock_add_rule 556 common landlock_restrict_self sys_landlock_restrict_self +# 557 reserved for memfd_secret +558 common process_mrelease sys_process_mrelease
arch/arm/tools/syscall.tbl +2 -0
@@ -460,3 +460,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease
arch/arm64/include/asm/unistd.h +1 -1
@@ -38,7 +38,7 @@#define __ARM_NR_compat_set_tls (__ARM_NR_COMPAT_BASE + 5)#define __ARM_NR_COMPAT_END (__ARM_NR_COMPAT_BASE + 0x800) -#define __NR_compat_syscalls 447 +#define __NR_compat_syscalls 449#endif #define __ARCH_WANT_SYS_CLONE
arch/arm64/include/asm/unistd32.h +2 -0
@@ -901,6 +901,8 @@ __SYSCALL(__NR_landlock_create_ruleset, sys_landlock_create_ruleset) __SYSCALL(__NR_landlock_add_rule, sys_landlock_add_rule)#define __NR_landlock_restrict_self 446 __SYSCALL(__NR_landlock_restrict_self, sys_landlock_restrict_self) +#define __NR_process_mrelease 448 +__SYSCALL (__NR_process_mrelease, sys_process_mrelease)/* * Please add new compat syscalls above this comment and update
arch/ia64/kernel/syscalls/syscall.tbl +2 -0
@@ -367,3 +367,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease
arch/m68k/kernel/syscalls/syscall.tbl +2 -0
@@ -446,3 +446,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease
arch/microblaze/kernel/syscalls/syscall.tbl +2 -0
@@ -452,3 +452,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease
arch/mips/kernel/syscalls/syscall_n32.tbl +2 -0
@@ -385,3 +385,5 @@ 444 n32 landlock_create_ruleset sys_landlock_create_ruleset 445 n32 landlock_add_rule sys_landlock_add_rule 446 n32 landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 n32 process_mrelease sys_process_mrelease
arch/mips/kernel/syscalls/syscall_n64.tbl +2 -0
@@ -361,3 +361,5 @@ 444 n64 landlock_create_ruleset sys_landlock_create_ruleset 445 n64 landlock_add_rule sys_landlock_add_rule 446 n64 landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 n64 process_mrelease sys_process_mrelease
arch/mips/kernel/syscalls/syscall_o32.tbl +2 -0
@@ -434,3 +434,5 @@ 444 o32 landlock_create_ruleset sys_landlock_create_ruleset 445 o32 landlock_add_rule sys_landlock_add_rule 446 o32 landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 o32 process_mrelease sys_process_mrelease
arch/parisc/kernel/syscalls/syscall.tbl +2 -0
@@ -444,3 +444,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease
arch/powerpc/kernel/syscalls/syscall.tbl +2 -0
@@ -526,3 +526,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease
arch/s390/kernel/syscalls/syscall.tbl +2 -0
@@ -449,3 +449,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease sys_process_mrelease
arch/sh/kernel/syscalls/syscall.tbl +2 -0
@@ -449,3 +449,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease
arch/sparc/kernel/syscalls/syscall.tbl +2 -0
@@ -492,3 +492,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease
arch/x86/entry/syscalls/syscall_32.tbl +1 -0
@@ -452,3 +452,4 @@ 445 i386 landlock_add_rule sys_landlock_add_rule 446 i386 landlock_restrict_self sys_landlock_restrict_self 447 i386 memfd_secret sys_memfd_secret +448 i386 process_mrelease sys_process_mrelease
arch/x86/entry/syscalls/syscall_64.tbl +1 -0
@@ -369,6 +369,7 @@ 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self 447 common memfd_secret sys_memfd_secret +448 common process_mrelease sys_process_mrelease # # Due to a historical design error, certain syscalls are numbered differently
arch/xtensa/kernel/syscalls/syscall.tbl +2 -0
@@ -417,3 +417,5 @@ 444 common landlock_create_ruleset sys_landlock_create_ruleset 445 common landlock_add_rule sys_landlock_add_rule 446 common landlock_restrict_self sys_landlock_restrict_self +# 447 reserved for memfd_secret +448 common process_mrelease sys_process_mrelease
include/linux/syscalls.h +1 -0
@@ -915,6 +915,7 @@ asmlinkage long sys_mincore(unsigned long start, size_t len,asmlinkage long sys_madvise (unsigned long start,size_t len,int behavior);asmlinkage long sys_process_madvise (int pidfd,const struct iovec __user *vec,size_t vlen,int behavior,unsigned int flags); +asmlinkage long sys_process_mrelease (int pidfd,unsigned int flags);asmlinkage long sys_remap_file_pages (unsigned long start,unsigned long size,unsigned long prot,unsigned long pgoff,unsigned long flags);
include/uapi/asm-generic/unistd.h +3 -1
@@ -877,9 +877,11 @@ __SYSCALL(__NR_landlock_restrict_self, sys_landlock_restrict_self)#define __NR_memfd_secret 447 __SYSCALL(__NR_memfd_secret, sys_memfd_secret)#endif +#define __NR_process_mrelease 448 +__SYSCALL(__NR_process_mrelease, sys_process_mrelease)#undef __NR_syscalls -#define __NR_syscalls 448 +#define __NR_syscalls 449/* * 32 bit systems traditionally used different
kernel/sys_ni.c +1 -0
@@ -289,6 +289,7 @@ COND_SYSCALL(munlockall);COND_SYSCALL (mincore);COND_SYSCALL (madvise);COND_SYSCALL (process_madvise); +COND_SYSCALL (process_mrelease);COND_SYSCALL (remap_file_pages);COND_SYSCALL (mbind);COND_SYSCALL_COMPAT (mbind);
Signature · 1 line
2.32.0.554.ge1b32706d8-goog
Quoted 117 lines · Suren Baghdasaryan
In modern systems it's not unusual to have a system component monitoring memory conditions of the system and tasked with keeping system memory pressure under control. One way to accomplish that is to kill non-essential processes to free up memory for more important ones. Examples of this are Facebook's OOM killer daemon called oomd and Android's low memory killer daemon called lmkd.
For such system component it's important to be able to free memory quickly and efficiently. Unfortunately the time process takes to free up its memory after receiving a SIGKILL might vary based on the state of the process (uninterruptible sleep), size and OPP level of the core the process is running. A mechanism to free resources of the target process in a more predictable way would improve system's ability to control its memory pressure.
Introduceprocess_mreleasesystem call that releases memory of a dying process from the context of the caller. This way the memory is freed in a more controllable way with CPU affinity and priority of the caller. The workload of freeing the memory will also be charged to the caller. The operation is allowed only on a dying process.After previous discussions [1, 2, 3] the decision was made [4] to introduce a dedicated system call to cover this use case.
The API is as follows,
int
process_mrelease(int pidfd, unsigned int flags);DESCRIPTION
Theprocess_mrelease()system call is used to free the memory of an exiting process.The pidfd selects the process referred to by the PID file descriptor.
(Seepidfd_open(2)for further information)The flags argument is reserved for future use; currently, this argument must be specified as 0.
RETURN VALUE
On success,process_mrelease()returns 0. On error, -1 is returned and errno is set to indicate the error.ERRORS
EBADF pidfd is not a valid PID file descriptor.EAGAIN Failed to release part of the address space.
EINTR The call was interrupted by a signal; see signal(7).
EINVAL flags is not 0.
EINVAL The memory of the task cannot be released because the
process is not exiting, the address space is shared with another live process or there is a core dump in progress.ENOSYS This system call is not supported, for example, without
MMU support built into Linux.ESRCH The target process does not exist (i.e., it has terminated
and been waited on).[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/linux-api/[email protected]/
[3] https://lore.kernel.org/linux-api/[email protected]/
[4] https://lore.kernel.org/linux-api/[email protected]/Signed-off-by Suren Baghdasaryan <[email protected]>---
changes in v7:
- Fixedpidfd_openmisspelling, per Andrew Morton
- Fixed wrong task pinning afterfind_lock_task_mm()issue, per Michal Hocko
- MovedMMF_OOM_SKIPcheck beforetask_will_free_mem(), per Michal Hockomm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)mm/oom_kill.c +26
-0@@ -28,6 +28,7 @@#include <linux/sched/task.h> #include <linux/sched/debug.h> #include <linux/swap.h> +#include <linux/syscalls.h> #include <linux/timex.h> #include <linux/jiffies.h> #include <linux/cpuset.h> @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void)out_of_memory (&oc);mutex_unlock (&oom_lock); } + +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) +{ +#ifdef CONFIG_MMU +struct mm_struct *mm =NULL ; +struct task_struct *task; +struct task_struct *p; +unsigned int f_flags; +struct pid *pid; +long ret =0 ; + +if (flags) +return -EINVAL ; + + pid =pidfd_get_pid (pidfd, &f_flags); +if (IS_ERR (pid)) +return PTR_ERR (pid); + + task =get_pid_task (pid,PIDTYPE_PID ); +if (!task) { + ret = -ESRCH ; + goto put_pid; + } + +/*
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1179
Quoted 10 lines
mm/oom_kill.c +10
-0+* If the task is dying and in the process of releasing its memory +* then get its mm. +*/ + p =find_lock_task_mm (task); +if (!p) { + ret = -ESRCH ; + goto put_pid; + } +if (task != p) { +get_task_struct (p);
Wouldn't we want to obtain the mm from p ? I thought that was the whole exercise of going via find_lock_task_mm().
Signature · 3 lines
Thanks, David / dhildenb
Quoted 125 lines · David Hildenbrand
Quoted 123 lines · Suren Baghdasaryan
In modern systems it's not unusual to have a system component monitoring memory conditions of the system and tasked with keeping system memory pressure under control. One way to accomplish that is to kill non-essential processes to free up memory for more important ones. Examples of this are Facebook's OOM killer daemon called oomd and Android's low memory killer daemon called lmkd.
For such system component it's important to be able to free memory quickly and efficiently. Unfortunately the time process takes to free up its memory after receiving a SIGKILL might vary based on the state of the process (uninterruptible sleep), size and OPP level of the core the process is running. A mechanism to free resources of the target process in a more predictable way would improve system's ability to control its memory pressure.
Introduceprocess_mreleasesystem call that releases memory of a dying process from the context of the caller. This way the memory is freed in a more controllable way with CPU affinity and priority of the caller. The workload of freeing the memory will also be charged to the caller. The operation is allowed only on a dying process.After previous discussions [1, 2, 3] the decision was made [4] to introduce a dedicated system call to cover this use case.
The API is as follows,
int
process_mrelease(int pidfd, unsigned int flags);DESCRIPTION
Theprocess_mrelease()system call is used to free the memory of an exiting process.The pidfd selects the process referred to by the PID file descriptor.
(Seepidfd_open(2)for further information)The flags argument is reserved for future use; currently, this argument must be specified as 0.
RETURN VALUE
On success,process_mrelease()returns 0. On error, -1 is returned and errno is set to indicate the error.ERRORS
EBADF pidfd is not a valid PID file descriptor.EAGAIN Failed to release part of the address space.
EINTR The call was interrupted by a signal; see signal(7).
EINVAL flags is not 0.
EINVAL The memory of the task cannot be released because the
process is not exiting, the address space is shared with another live process or there is a core dump in progress.ENOSYS This system call is not supported, for example, without
MMU support built into Linux.ESRCH The target process does not exist (i.e., it has terminated
and been waited on).[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/linux-api/[email protected]/
[3] https://lore.kernel.org/linux-api/[email protected]/
[4] https://lore.kernel.org/linux-api/[email protected]/Signed-off-by Suren Baghdasaryan <[email protected]>---
changes in v7:
- Fixedpidfd_openmisspelling, per Andrew Morton
- Fixed wrong task pinning afterfind_lock_task_mm()issue, per Michal Hocko
- MovedMMF_OOM_SKIPcheck beforetask_will_free_mem(), per Michal Hockomm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)mm/oom_kill.c +32
-0@@ -28,6 +28,7 @@#include <linux/sched/task.h> #include <linux/sched/debug.h> #include <linux/swap.h> +#include <linux/syscalls.h> #include <linux/timex.h> #include <linux/jiffies.h> #include <linux/cpuset.h> @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void)out_of_memory (&oc);mutex_unlock (&oom_lock); } + +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) +{ +#ifdef CONFIG_MMU +struct mm_struct *mm =NULL ; +struct task_struct *task; +struct task_struct *p; +unsigned int f_flags; +struct pid *pid; +long ret =0 ; + +if (flags) +return -EINVAL ; + + pid =pidfd_get_pid (pidfd, &f_flags); +if (IS_ERR (pid)) +return PTR_ERR (pid); + + task =get_pid_task (pid,PIDTYPE_PID ); +if (!task) { + ret = -ESRCH ; + goto put_pid; + } + +/* +* If the task is dying and in the process of releasing its memory +* then get its mm. +*/ + p =find_lock_task_mm (task); +if (!p) { + ret = -ESRCH ;
Quoted 8 lines
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1179
Quoted 4 lines
mm/oom_kill.c +4
-0+ goto put_pid; + } +if (task != p) { +get_task_struct (p);Wouldn't we want to obtain the mm from p ? I thought that was the whole exercise of going via
find_lock_task_mm().
Yes, that's what we do after checking task_will_free_mem(). task_will_free_mem() requires us to hold task_lock and find_lock_task_mm() achieves that ensuring that mm is still valid, but it might return a task other than the original one. That's why we do this dance with pinning the new task and unpinning the original one. The same dance is performed in __oom_kill_process(). I was contemplating adding a parameter to find_lock_task_mm() to request this unpin/pin be done within that function but then decided to keep it simple for now.
Did I address your question or did I misunderstand it?
Quoted 9 lines
--
Thanks,David / dhildenb
--
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
Quoted 135 lines · Suren Baghdasaryan
Quoted 125 lines · David Hildenbrand
Quoted 123 lines · Suren Baghdasaryan
In modern systems it's not unusual to have a system component monitoring memory conditions of the system and tasked with keeping system memory pressure under control. One way to accomplish that is to kill non-essential processes to free up memory for more important ones. Examples of this are Facebook's OOM killer daemon called oomd and Android's low memory killer daemon called lmkd.
For such system component it's important to be able to free memory quickly and efficiently. Unfortunately the time process takes to free up its memory after receiving a SIGKILL might vary based on the state of the process (uninterruptible sleep), size and OPP level of the core the process is running. A mechanism to free resources of the target process in a more predictable way would improve system's ability to control its memory pressure.
Introduceprocess_mreleasesystem call that releases memory of a dying process from the context of the caller. This way the memory is freed in a more controllable way with CPU affinity and priority of the caller. The workload of freeing the memory will also be charged to the caller. The operation is allowed only on a dying process.After previous discussions [1, 2, 3] the decision was made [4] to introduce a dedicated system call to cover this use case.
The API is as follows,
int
process_mrelease(int pidfd, unsigned int flags);DESCRIPTION
Theprocess_mrelease()system call is used to free the memory of an exiting process.The pidfd selects the process referred to by the PID file descriptor.
(Seepidfd_open(2)for further information)The flags argument is reserved for future use; currently, this argument must be specified as 0.
RETURN VALUE
On success,process_mrelease()returns 0. On error, -1 is returned and errno is set to indicate the error.ERRORS
EBADF pidfd is not a valid PID file descriptor.EAGAIN Failed to release part of the address space.
EINTR The call was interrupted by a signal; see signal(7).
EINVAL flags is not 0.
EINVAL The memory of the task cannot be released because the
process is not exiting, the address space is shared with another live process or there is a core dump in progress.ENOSYS This system call is not supported, for example, without
MMU support built into Linux.ESRCH The target process does not exist (i.e., it has terminated
and been waited on).[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/linux-api/[email protected]/
[3] https://lore.kernel.org/linux-api/[email protected]/
[4] https://lore.kernel.org/linux-api/[email protected]/Signed-off-by Suren Baghdasaryan <[email protected]>---
changes in v7:
- Fixedpidfd_openmisspelling, per Andrew Morton
- Fixed wrong task pinning afterfind_lock_task_mm()issue, per Michal Hocko
- MovedMMF_OOM_SKIPcheck beforetask_will_free_mem(), per Michal Hockomm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)mm/oom_kill.c +32
-0@@ -28,6 +28,7 @@#include <linux/sched/task.h> #include <linux/sched/debug.h> #include <linux/swap.h> +#include <linux/syscalls.h> #include <linux/timex.h> #include <linux/jiffies.h> #include <linux/cpuset.h> @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void)out_of_memory (&oc);mutex_unlock (&oom_lock); } + +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) +{ +#ifdef CONFIG_MMU +struct mm_struct *mm =NULL ; +struct task_struct *task; +struct task_struct *p; +unsigned int f_flags; +struct pid *pid; +long ret =0 ; + +if (flags) +return -EINVAL ; + + pid =pidfd_get_pid (pidfd, &f_flags); +if (IS_ERR (pid)) +return PTR_ERR (pid); + + task =get_pid_task (pid,PIDTYPE_PID ); +if (!task) { + ret = -ESRCH ; + goto put_pid; + } + +/* +* If the task is dying and in the process of releasing its memory +* then get its mm. +*/ + p =find_lock_task_mm (task); +if (!p) { + ret = -ESRCH ;Quoted 8 lines
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1179
Quoted 4 lines
mm/oom_kill.c +4
-0+ goto put_pid; + } +if (task != p) { +get_task_struct (p);Wouldn't we want to obtain the mm from p ? I thought that was the whole exercise of going via
find_lock_task_mm().
Quoted 10 lines
Yes, that's what we do after checking
task_will_free_mem().task_will_free_mem()requires us to holdtask_lockandfind_lock_task_mm()achieves that ensuring that mm is still valid, but it might return a task other than the original one. That's why we do this dance with pinning the new task and unpinning the original one. The same dance is performed in__oom_kill_process(). I was contemplating adding a parameter tofind_lock_task_mm()to request this unpin/pin be done within that function but then decided to keep it simple for now.
Did I address your question or did I misunderstand it?
Excuse my tired eyes, I missed the "task = p;"
Feel free to carry my ack along, even if there are minor changes.
Signature · 3 lines
Thanks, David / dhildenb
Quoted 136 lines · Suren Baghdasaryan
Quoted 125 lines · David Hildenbrand
Quoted 123 lines · Suren Baghdasaryan
In modern systems it's not unusual to have a system component monitoring memory conditions of the system and tasked with keeping system memory pressure under control. One way to accomplish that is to kill non-essential processes to free up memory for more important ones. Examples of this are Facebook's OOM killer daemon called oomd and Android's low memory killer daemon called lmkd.
For such system component it's important to be able to free memory quickly and efficiently. Unfortunately the time process takes to free up its memory after receiving a SIGKILL might vary based on the state of the process (uninterruptible sleep), size and OPP level of the core the process is running. A mechanism to free resources of the target process in a more predictable way would improve system's ability to control its memory pressure.
Introduceprocess_mreleasesystem call that releases memory of a dying process from the context of the caller. This way the memory is freed in a more controllable way with CPU affinity and priority of the caller. The workload of freeing the memory will also be charged to the caller. The operation is allowed only on a dying process.After previous discussions [1, 2, 3] the decision was made [4] to introduce a dedicated system call to cover this use case.
The API is as follows,
int
process_mrelease(int pidfd, unsigned int flags);DESCRIPTION
Theprocess_mrelease()system call is used to free the memory of an exiting process.The pidfd selects the process referred to by the PID file descriptor.
(Seepidfd_open(2)for further information)The flags argument is reserved for future use; currently, this argument must be specified as 0.
RETURN VALUE
On success,process_mrelease()returns 0. On error, -1 is returned and errno is set to indicate the error.ERRORS
EBADF pidfd is not a valid PID file descriptor.EAGAIN Failed to release part of the address space.
EINTR The call was interrupted by a signal; see signal(7).
EINVAL flags is not 0.
EINVAL The memory of the task cannot be released because the
process is not exiting, the address space is shared with another live process or there is a core dump in progress.ENOSYS This system call is not supported, for example, without
MMU support built into Linux.ESRCH The target process does not exist (i.e., it has terminated
and been waited on).[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/linux-api/[email protected]/
[3] https://lore.kernel.org/linux-api/[email protected]/
[4] https://lore.kernel.org/linux-api/[email protected]/Signed-off-by Suren Baghdasaryan <[email protected]>---
changes in v7:
- Fixedpidfd_openmisspelling, per Andrew Morton
- Fixed wrong task pinning afterfind_lock_task_mm()issue, per Michal Hocko
- MovedMMF_OOM_SKIPcheck beforetask_will_free_mem(), per Michal Hockomm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)mm/oom_kill.c +32
-0@@ -28,6 +28,7 @@#include <linux/sched/task.h> #include <linux/sched/debug.h> #include <linux/swap.h> +#include <linux/syscalls.h> #include <linux/timex.h> #include <linux/jiffies.h> #include <linux/cpuset.h> @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void)out_of_memory (&oc);mutex_unlock (&oom_lock); } + +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) +{ +#ifdef CONFIG_MMU +struct mm_struct *mm =NULL ; +struct task_struct *task; +struct task_struct *p; +unsigned int f_flags; +struct pid *pid; +long ret =0 ; + +if (flags) +return -EINVAL ; + + pid =pidfd_get_pid (pidfd, &f_flags); +if (IS_ERR (pid)) +return PTR_ERR (pid); + + task =get_pid_task (pid,PIDTYPE_PID ); +if (!task) { + ret = -ESRCH ; + goto put_pid; + } + +/* +* If the task is dying and in the process of releasing its memory +* then get its mm. +*/ + p =find_lock_task_mm (task); +if (!p) { + ret = -ESRCH ;Quoted 8 lines
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1179
Quoted 4 lines
mm/oom_kill.c +4
-0+ goto put_pid; + } +if (task != p) { +get_task_struct (p);Wouldn't we want to obtain the mm from p ? I thought that was the whole exercise of going via
find_lock_task_mm().
Quoted 11 lines
Yes, that's what we do after checking
task_will_free_mem().task_will_free_mem()requires us to holdtask_lockandfind_lock_task_mm()achieves that ensuring that mm is still valid, but it might return a task other than the original one. That's why we do this dance with pinning the new task and unpinning the original one. The same dance is performed in__oom_kill_process(). I was contemplating adding a parameter tofind_lock_task_mm()to request this unpin/pin be done within that function but then decided to keep it simple for now.
Did I address your question or did I misunderstand it?
One question I have is why mmget() and not mmgrab()? I see mmgrab() in oom_kill.c.
Quoted 150 lines · Shakeel Butt
Quoted 136 lines · Suren Baghdasaryan
Quoted 125 lines · David Hildenbrand
Quoted 123 lines · Suren Baghdasaryan
In modern systems it's not unusual to have a system component monitoring memory conditions of the system and tasked with keeping system memory pressure under control. One way to accomplish that is to kill non-essential processes to free up memory for more important ones. Examples of this are Facebook's OOM killer daemon called oomd and Android's low memory killer daemon called lmkd.
For such system component it's important to be able to free memory quickly and efficiently. Unfortunately the time process takes to free up its memory after receiving a SIGKILL might vary based on the state of the process (uninterruptible sleep), size and OPP level of the core the process is running. A mechanism to free resources of the target process in a more predictable way would improve system's ability to control its memory pressure.
Introduceprocess_mreleasesystem call that releases memory of a dying process from the context of the caller. This way the memory is freed in a more controllable way with CPU affinity and priority of the caller. The workload of freeing the memory will also be charged to the caller. The operation is allowed only on a dying process.After previous discussions [1, 2, 3] the decision was made [4] to introduce a dedicated system call to cover this use case.
The API is as follows,
int
process_mrelease(int pidfd, unsigned int flags);DESCRIPTION
Theprocess_mrelease()system call is used to free the memory of an exiting process.The pidfd selects the process referred to by the PID file descriptor.
(Seepidfd_open(2)for further information)The flags argument is reserved for future use; currently, this argument must be specified as 0.
RETURN VALUE
On success,process_mrelease()returns 0. On error, -1 is returned and errno is set to indicate the error.ERRORS
EBADF pidfd is not a valid PID file descriptor.EAGAIN Failed to release part of the address space.
EINTR The call was interrupted by a signal; see signal(7).
EINVAL flags is not 0.
EINVAL The memory of the task cannot be released because the
process is not exiting, the address space is shared with another live process or there is a core dump in progress.ENOSYS This system call is not supported, for example, without
MMU support built into Linux.ESRCH The target process does not exist (i.e., it has terminated
and been waited on).[1] https://lore.kernel.org/lkml/[email protected]/
[2] https://lore.kernel.org/linux-api/[email protected]/
[3] https://lore.kernel.org/linux-api/[email protected]/
[4] https://lore.kernel.org/linux-api/[email protected]/Signed-off-by Suren Baghdasaryan <[email protected]>---
changes in v7:
- Fixedpidfd_openmisspelling, per Andrew Morton
- Fixed wrong task pinning afterfind_lock_task_mm()issue, per Michal Hocko
- MovedMMF_OOM_SKIPcheck beforetask_will_free_mem(), per Michal Hockomm/oom_kill.c | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)mm/oom_kill.c +32
-0@@ -28,6 +28,7 @@#include <linux/sched/task.h> #include <linux/sched/debug.h> #include <linux/swap.h> +#include <linux/syscalls.h> #include <linux/timex.h> #include <linux/jiffies.h> #include <linux/cpuset.h> @@ -1141,3 +1142,75 @@ void pagefault_out_of_memory(void)out_of_memory (&oc);mutex_unlock (&oom_lock); } + +SYSCALL_DEFINE2(process_mrelease, int, pidfd, unsigned int, flags) +{ +#ifdef CONFIG_MMU +struct mm_struct *mm =NULL ; +struct task_struct *task; +struct task_struct *p; +unsigned int f_flags; +struct pid *pid; +long ret =0 ; + +if (flags) +return -EINVAL ; + + pid =pidfd_get_pid (pidfd, &f_flags); +if (IS_ERR (pid)) +return PTR_ERR (pid); + + task =get_pid_task (pid,PIDTYPE_PID ); +if (!task) { + ret = -ESRCH ; + goto put_pid; + } + +/* +* If the task is dying and in the process of releasing its memory +* then get its mm. +*/ + p =find_lock_task_mm (task); +if (!p) { + ret = -ESRCH ;Quoted 8 lines
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1179
Quoted 4 lines
mm/oom_kill.c +4
-0+ goto put_pid; + } +if (task != p) { +get_task_struct (p);Wouldn't we want to obtain the mm from p ? I thought that was the whole exercise of going via
find_lock_task_mm().Quoted 11 lines
Yes, that's what we do after checking
task_will_free_mem().task_will_free_mem()requires us to holdtask_lockandfind_lock_task_mm()achieves that ensuring that mm is still valid, but it might return a task other than the original one. That's why we do this dance with pinning the new task and unpinning the original one. The same dance is performed in__oom_kill_process(). I was contemplating adding a parameter tofind_lock_task_mm()to request this unpin/pin be done within that function but then decided to keep it simple for now.
Did I address your question or did I misunderstand it?
Quoted 2 lines
One question I have is why
mmget()and notmmgrab()? I seemmgrab()inoom_kill.c.
You are likely right here. The caller's context probably can't be considered a "real user" when reaping the mm. However, we take an mmap_lock shortly after, so not sure if in practice there is much difference.
Michal, WDYT?
Michal Hocko
maintainerOn Thu 05-08-21 11:37:06, Suren Baghdasaryan wrote:
[...]
Quoted 8 lines
Quoted 2 lines
One question I have is why
mmget()and notmmgrab()? I seemmgrab()inoom_kill.c.You are likely right here. The caller's context probably can't be considered a "real user" when reaping the mm. However, we take an
mmap_lockshortly after, so not sure if in practice there is much difference.
Michal, WDYT?
As explained in other response. mmget is to pin address space to not go away. You do not need that for this purpose. All you need is to pin mm to not go away. Address space can be unmapped concurrently.
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
maintainerOn Thu 05-08-21 10:08:58, Suren Baghdasaryan wrote:
[...]
Quoted 4 lines
mm/oom_kill.c +4
-0+/* +* If the task is dying and in the process of releasing its memory +* then get its mm. +*/
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1182
Quoted 10 lines
mm/oom_kill.c +10
-0+ p =find_lock_task_mm (task); +if (!p) { + ret = -ESRCH ; + goto put_pid; + } +if (task != p) { +get_task_struct (p); +put_task_struct (task); + task = p; + }
Why do you need to take a reference to the p here? You are under task_lock so this will not go away and you only need p to get your mm.
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1186
Quoted 4 lines
mm/oom_kill.c +4
-0+ +/* If the work has been done already, just exit with success */ +if (test_bit (MMF_OOM_SKIP , &task->mm->flags)) + goto put_task;
You want to release the task_lock
Review
Quoted 2 lines
(quoted diff) +2
-0+ + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) {
you want task_will_free_mem(p) and what is the point of the PF_KTHREAD check?
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1190
Quoted 2 lines
mm/oom_kill.c +2
-0+ mm = task->mm; +mmget (mm);
All you need is to make sure mm will not get released under your feet once task_lock is released so mmgrab is the right thing to do here. The address space can be torn down in parallel and that is OK and desirable.
I think you really want something like this:
if (flags)
return -EINVAL;
pid = pidfd_get_pid(fd, &f_flags);
if (IS_ERR(pid))
return PTR_ERR(pid);
task = get_pid_task(pid, PIDTYPE_PID);
if (!task) {
ret = -ESRCH;
goto put_pid;
}
/*
* Make sure to chose a thread which still has a reference to mm
* during the group exit
*/
p = find_lock_task_mm(task);
if (!p) {
ret = -ESRCH;
goto put_task;
}
mm = task->mm;
mmgrab(mm);
reap = true;
/* If the work has been done already, just exit with success */
if (test_bit(MMF_OOM_SKIP, &mm->flags)) {
reap = false;
} else if (!task_will_free_mem(p)) {
reap = false;
ret = -EINVAL;
}
task_unlock(p);
if (!reap)
goto dropmm;;
/* Do the work*/dropmm:
mmdrop(mm); put_task: put_task(task); put_pid: put_pid(pid); return ret;
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 1 line · Michal Hocko
[...]
Quoted 17 lines
I think you really want something like this:
if (flags) return -EINVAL; pid = pidfd_get_pid(fd, &f_flags); if (IS_ERR(pid)) return PTR_ERR(pid); task = get_pid_task(pid, PIDTYPE_PID); if (!task) { ret = -ESRCH; goto put_pid; } /* * Make sure to chose a thread which still has a reference to mm * during the group exit
Quoted 8 lines
*/
p = find_lock_task_mm(task);
if (!p) {
ret = -ESRCH;
goto put_task;
}
mm = task->mm;mm = p->mm;
Quoted 29 lines
mmgrab(mm); reap = true; /* If the work has been done already, just exit with success */ if (test_bit(MMF_OOM_SKIP, &mm->flags)) { reap = false; } else if (!task_will_free_mem(p)) { reap = false; ret = -EINVAL; } task_unlock(p); if (!reap) goto dropmm;; /* Do the work*/dropmm:
mmdrop(mm); put_task: put_task(task); put_pid: put_pid(pid); return ret;--
Michal Hocko
SUSE Labs
Michal Hocko
maintainerQuoted 27 lines · Shakeel Butt
Quoted 1 line · Michal Hocko
[...]
Quoted 24 lines
I think you really want something like this:
if (flags) return -EINVAL; pid = pidfd_get_pid(fd, &f_flags); if (IS_ERR(pid)) return PTR_ERR(pid); task = get_pid_task(pid, PIDTYPE_PID); if (!task) { ret = -ESRCH; goto put_pid; } /* * Make sure to chose a thread which still has a reference to mm * during the group exit */ p = find_lock_task_mm(task); if (!p) { ret = -ESRCH; goto put_task; }
Quoted 3 lines
Quoted 1 line
mm = task->mm;mm =
p->mm;
right. Thanks!
Signature · 2 lines
Michal Hocko SUSE Labs
Quoted 18 lines · Michal Hocko
On Thu 05-08-21 10:08:58, Suren Baghdasaryan wrote:
[...]Quoted 4 lines
mm/oom_kill.c +4
-0+/* +* If the task is dying and in the process of releasing its memory +* then get its mm. +*/ mm/oom_kill.c void pagefault_out_of_memory(void) · line 1182
Quoted 10 lines
mm/oom_kill.c +10
-0+ p =find_lock_task_mm (task); +if (!p) { + ret = -ESRCH ; + goto put_pid; + } +if (task != p) { +get_task_struct (p); +put_task_struct (task); + task = p; + }
Quoted 2 lines
Why do you need to take a reference to the p here? You are under
task_lockso this will not go away and you only need p to get your mm.
True.
Quoted 7 lines
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1186
Quoted 5 lines
mm/oom_kill.c +4
-0+ +/* If the work has been done already, just exit with success */ +if (test_bit (MMF_OOM_SKIP , &task->mm->flags)) + goto put_task;You want to release the
task_lock
Missed it again :(
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) {you want
task_will_free_mem(p)and what is the point of thePF_KTHREADcheck?
Yeah, looks like task_will_free_mem() covers that case already.
Quoted 50 lines
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1190
Quoted 3 lines
mm/oom_kill.c +2
-0+ mm = task->mm; +mmget (mm);All you need is to make sure mm will not get released under your feet once
task_lockis released so mmgrab is the right thing to do here. The address space can be torn down in parallel and that is OK and desirable.I think you really want something like this:
if (flags) return -EINVAL; pid = pidfd_get_pid(fd, &f_flags); if (IS_ERR(pid)) return PTR_ERR(pid); task = get_pid_task(pid, PIDTYPE_PID); if (!task) { ret = -ESRCH; goto put_pid; } /* * Make sure to chose a thread which still has a reference to mm * during the group exit */ p = find_lock_task_mm(task); if (!p) { ret = -ESRCH; goto put_task; } mm = task->mm; mmgrab(mm); reap = true; /* If the work has been done already, just exit with success */ if (test_bit(MMF_OOM_SKIP, &mm->flags)) { reap = false; } else if (!task_will_free_mem(p)) { reap = false; ret = -EINVAL; } task_unlock(p); if (!reap) goto dropmm;; /* Do the work*/
Quoted 9 lines
dropmm:
mmdrop(mm); put_task: put_task(task); put_pid: put_pid(pid); return ret;
This is indeed simpler to follow. I'll adopt your version. Thanks!
Quoted 3 lines
--
Michal Hocko
SUSE Labs
Quoted 101 lines · Suren Baghdasaryan
Quoted 18 lines · Michal Hocko
On Thu 05-08-21 10:08:58, Suren Baghdasaryan wrote:
[...]Quoted 4 lines
mm/oom_kill.c +4
-0+/* +* If the task is dying and in the process of releasing its memory +* then get its mm. +*/ mm/oom_kill.c void pagefault_out_of_memory(void) · line 1182
Quoted 10 lines
mm/oom_kill.c +10
-0+ p =find_lock_task_mm (task); +if (!p) { + ret = -ESRCH ; + goto put_pid; + } +if (task != p) { +get_task_struct (p); +put_task_struct (task); + task = p; + }Quoted 2 lines
Why do you need to take a reference to the p here? You are under
task_lockso this will not go away and you only need p to get your mm.True.
Quoted 7 lines
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1186
Quoted 5 lines
mm/oom_kill.c +4
-0+ +/* If the work has been done already, just exit with success */ +if (test_bit (MMF_OOM_SKIP , &task->mm->flags)) + goto put_task;You want to release the
task_lockMissed it again :(
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +2
-0+ + if (task_will_free_mem(task) && (task->flags & PF_KTHREAD) == 0) {you want
task_will_free_mem(p)and what is the point of thePF_KTHREADcheck?Yeah, looks like
task_will_free_mem()covers that case already.Quoted 57 lines
mm/oom_kill.c void pagefault_out_of_memory(void) · line 1190
Quoted 3 lines
mm/oom_kill.c +2
-0+ mm = task->mm; +mmget (mm);All you need is to make sure mm will not get released under your feet once
task_lockis released so mmgrab is the right thing to do here. The address space can be torn down in parallel and that is OK and desirable.I think you really want something like this:
if (flags) return -EINVAL; pid = pidfd_get_pid(fd, &f_flags); if (IS_ERR(pid)) return PTR_ERR(pid); task = get_pid_task(pid, PIDTYPE_PID); if (!task) { ret = -ESRCH; goto put_pid; } /* * Make sure to chose a thread which still has a reference to mm * during the group exit */ p = find_lock_task_mm(task); if (!p) { ret = -ESRCH; goto put_task; } mm = task->mm; mmgrab(mm); reap = true; /* If the work has been done already, just exit with success */ if (test_bit(MMF_OOM_SKIP, &mm->flags)) { reap = false; } else if (!task_will_free_mem(p)) { reap = false; ret = -EINVAL; } task_unlock(p); if (!reap) goto dropmm;; /* Do the work*/dropmm:
mmdrop(mm); put_task: put_task(task); put_pid: put_pid(pid);
Quoted 4 lines
Quoted 2 lines
return ret;This is indeed simpler to follow. I'll adopt your version. Thanks!
v8 is posted at https://lore.kernel.org/patchwork/patch/1473697/
Testing shows performance improvement from replacing mmget with mmgrab.
Quoted 4 lines
Quoted 4 lines
--
Michal Hocko
SUSE Labs