introduce memory hinting API for external process
Subsystems (23)
v8 cover introduce memory hinting API for external process
Now, we have MADV_PAGEOUT and MADV_COLD as madvise hinting API. With that, application could give hints to kernel what memory range are preferred to be reclaimed. However, in some platform(e.g., Android), the information required to make the hinting decision is not known to the app.
Instead, it is known to a centralized userspace daemon(e.g., ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement.
To solve the concern, this patch introduces new syscall - process_madvise(2). Bascially, it's same with madvise(2) syscall but it has some differences.
1. It needs pidfd of target process to provide the hint
2. It supports only MADV_{COLD|PAGEOUT} at this moment.
Other hints in madvise will be opened when there are explicit requests from community to prevent unexpected bugs we couldn't support.
3. Only privileged processes can do something for other process's address space.
For more detail of the new API, please see "mm: introduce external memory hinting API" description in this patchset.
* from v7 - http://lore.kernel.org/r/[email protected]
* dropping pid support from new syscall and fold releated patches into syscall patch
* dropping KSM patch by discussion - Oleksandr, I lost the discussion. Please resend the single patch against of the patchset if you resolves the discussion.
https://lore.kernel.org/linux-api/[email protected]/
* from v6 - https://lore.kernel.org/linux-api/[email protected]/
* fix comments and descriptions - Suren
* Add Reviewed-by - Suren
* fix build break reported by 0-day
* from v5 - https://lore.kernel.org/linux-mm/[email protected]/
* use null task and requestor's mm for io_madvise - Jann and Jens
* use right commit description for moving pidfd_get_pid - Christoph
* from v4 - https://lore.kernel.org/linux-mm/[email protected]/
* pass mm down to functions, not accessing task->mm - Jann
* clean up - Alexander
* add Reviewed-by - Alexander, SeongJae
* patch reordering
* from v3 - https://lore.kernel.org/linux-mm/[email protected]/
* verify task->mm aftere access_mm - Oleg
* split some patches for easy review - Alexander
* clean up fatal signal checking - Suren
* from v2 - https://lore.kernel.org/linux-mm/[email protected]/
* check signal callee and caller to bail out - Kirill Tkhai
* put more clarification for justification of new API
* from v1 - https://lore.kernel.org/linux-mm/[email protected]/
* fix syscall number - SeongJae
* use get_pid_task - Kirill Tkhai
* extend API to support pid as well as pidfd - Kirill Tkhai
Minchan Kim (4):
mm/madvise: pass task and mm to do_madvise
pid: move pidfd_get_pid() to pid.c
mm/madvise: introduce process_madvise() syscall: an external memory
hinting API
mm/madvise: check fatal signal pending of target process
arch/alpha/kernel/syscalls/syscall.tbl | 1 + arch/arm/tools/syscall.tbl | 1 + arch/arm64/include/asm/unistd.h | 2 +- arch/arm64/include/asm/unistd32.h | 2 + arch/ia64/kernel/syscalls/syscall.tbl | 1 + arch/m68k/kernel/syscalls/syscall.tbl | 1 + arch/microblaze/kernel/syscalls/syscall.tbl | 1 + arch/mips/kernel/syscalls/syscall_n32.tbl | 1 + arch/mips/kernel/syscalls/syscall_n64.tbl | 1 + arch/mips/kernel/syscalls/syscall_o32.tbl | 1 + arch/parisc/kernel/syscalls/syscall.tbl | 1 + arch/powerpc/kernel/syscalls/syscall.tbl | 1 + arch/s390/kernel/syscalls/syscall.tbl | 1 + arch/sh/kernel/syscalls/syscall.tbl | 1 + arch/sparc/kernel/syscalls/syscall.tbl | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 2 + arch/xtensa/kernel/syscalls/syscall.tbl | 1 + fs/io_uring.c | 2 +- include/linux/compat.h | 4 + include/linux/mm.h | 3 +- include/linux/pid.h | 1 + include/linux/syscalls.h | 2 + include/uapi/asm-generic/unistd.h | 4 +- kernel/exit.c | 17 -- kernel/pid.c | 17 ++ kernel/sys_ni.c | 2 + mm/madvise.c | 190 +++++++++++++++++--- 28 files changed, 217 insertions(+), 46 deletions(-)
Signature · 1 line
2.27.0.111.gc72c7da667-goog
PATCH v8 1/4 mm/madvise: pass task and mm to do_madvise
Patch series "introduce memory hinting API for external process", v8.
Now, we have MADV_PAGEOUT and MADV_COLD as madvise hinting API. With that, application could give hints to kernel what memory range are preferred to be reclaimed. However, in some platform(e.g., Android), the information required to make the hinting decision is not known to the app. Instead, it is known to a centralized userspace daemon(e.g., ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement.
To solve the concern, this patch introduces new syscall - process_madvise(2). Bascially, it's same with madvise(2) syscall but it has some differences.
1. It needs pidfd of target process to provide the hint
2. It supports only MADV_{COLD|PAGEOUT|MERGEABLE|UNMEREABLE} at this moment. Other hints in madvise will be opened when there are explicit requests from community to prevent unexpected bugs we couldn't support.
3. Only privileged processes can do something for other process's address space.
For more detail of the new API, please see "mm: introduce external memory hinting API" description in this patchset.
This patch (of 4):
In upcoming patches, do_madvise will be called from external process context so we shouldn't asssume "current" is always hinted process's task_struct.
Furthermore, we must not access mm_struct via task->mm, but obtain it via access_mm() once (in the following patch) and only use that pointer [1], so pass it to do_madvise() as well. Note the vma->vm_mm pointers are safe, so we can use them further down the call stack.
And let's pass *current* and current->mm as arguments of do_madvise so it shouldn't change existing behavior but prepare next patch to make review easy.
Note: io_madvise passes NULL as target_task argument of do_madvise because it couldn't know who is target.
[1] http://lore.kernel.org/r/CAG48ez27=pwm5m_N_988xT1huO7g7h6arTQL44zev6TD-h-7Tg@mail.gmail.com
--- fs/io_uring.c | 2 +- include/linux/mm.h | 3 ++- mm/madvise.c | 40 +++++++++++++++++++++++----------------- 3 files changed, 26 insertions(+), 19 deletions(-)
fs/io_uring.c +1 -1
@@ -3586,7 +3586,7 @@ static int io_madvise(struct io_kiocb *req, bool force_nonblock)if (force_nonblock)return -EAGAIN ; - ret =do_madvise (ma->addr, ma->len, ma->advice); + ret =do_madvise (NULL , current->mm, ma->addr, ma->len, ma->advice);if (ret <0 )req_set_fail_links (req);io_cqring_add_event (req, ret);
include/linux/mm.h +2 -1
@@ -2541,7 +2541,8 @@ extern int __do_munmap(struct mm_struct *, unsigned long, size_t, struct list_head *uf, bool downgrade);extern int do_munmap (struct mm_struct *,unsigned long ,size_t ,struct list_head *uf); -extern int do_madvise (unsigned long start,size_t len_in,int behavior); +extern int do_madvise (struct task_struct *target_task,struct mm_struct *mm, +unsigned long start,size_t len_in,int behavior); static inline unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
mm/madvise.c +23 -17
@@ -22,12 +22,14 @@#include <linux/file.h> #include <linux/blkdev.h> #include <linux/backing-dev.h> +#include <linux/compat.h> #include <linux/pagewalk.h> #include <linux/swap.h> #include <linux/swapops.h> #include <linux/shmem_fs.h> #include <linux/mmu_notifier.h> #include <linux/sched/mm.h> +#include <linux/uio.h> #include <asm/tlb.h>
@@ -255,6 +257,7 @@ static long madvise_willneed(struct vm_area_struct *vma,struct vm_area_struct **prev, unsigned long start, unsigned long end) { + struct mm_struct *mm = vma->vm_mm;struct file *file = vma->vm_file;loff_t offset;
@@ -289,12 +292,12 @@ static long madvise_willneed(struct vm_area_struct *vma,*/ *prev =NULL ;/* tell sys_madvise we drop mmap_lock */ get_file (file); -mmap_read_unlock (current->mm); +mmap_read_unlock (mm); offset = (loff_t)(start - vma->vm_start) + ((loff_t )vma->vm_pgoff <<PAGE_SHIFT );vfs_fadvise (file, offset, end - start,POSIX_FADV_WILLNEED );fput (file); -mmap_read_lock (current->mm); +mmap_read_lock (mm);return 0 ; }
@@ -683,7 +686,6 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,if (nr_swap) {if (current->mm == mm)sync_mm_rss (mm); -add_mm_counter (mm,MM_SWAPENTS , nr_swap); }arch_leave_lazy_mmu_mode ();
@@ -763,6 +765,8 @@ static long madvise_dontneed_free(struct vm_area_struct *vma, unsigned long start, unsigned long end, int behavior) { + struct mm_struct *mm = vma->vm_mm; + *prev = vma;if (!can_madv_lru_vma (vma))return -EINVAL ;
@@ -770,8 +774,8 @@ static long madvise_dontneed_free(struct vm_area_struct *vma,if (!userfaultfd_remove (vma, start, end)) { *prev =NULL ;/* mmap_lock has been dropped, prev is stale */ -mmap_read_lock (current->mm); - vma =find_vma (current->mm, start); +mmap_read_lock (mm); + vma =find_vma (mm, start);if (!vma)return -ENOMEM ;if (start < vma->vm_start) {
@@ -825,6 +829,7 @@ static long madvise_remove(struct vm_area_struct *vma,loff_t offset;int error;struct file *f; +struct mm_struct *mm = vma->vm_mm; *prev =NULL ;/* tell sys_madvise we drop mmap_lock */
@@ -852,13 +857,13 @@ static long madvise_remove(struct vm_area_struct *vma,get_file (f);if (userfaultfd_remove (vma, start, end)) {/* mmap_lock was not released by userfaultfd_remove() */ -mmap_read_unlock (current->mm); +mmap_read_unlock (mm); } error =vfs_fallocate (f,FALLOC_FL_PUNCH_HOLE |FALLOC_FL_KEEP_SIZE , offset, end - start);fput (f); -mmap_read_lock (current->mm); +mmap_read_lock (mm);return error; }
@@ -1051,7 +1056,8 @@ madvise_behavior_valid(int behavior)* -EBADF - map exists, but area maps something that isn't a file. * -EAGAIN - a kernel resource was temporarily unavailable. */ -int do_madvise (unsigned long start,size_t len_in,int behavior) +int do_madvise (struct task_struct *target_task,struct mm_struct *mm, +unsigned long start,size_t len_in,int behavior) {unsigned long end, tmp;struct vm_area_struct *vma, *prev;
@@ -1089,7 +1095,7 @@ int do_madvise(unsigned long start, size_t len_in, int behavior) write =madvise_need_mmap_write (behavior);if (write) { -if (mmap_write_lock_killable (current->mm)) +if (mmap_write_lock_killable (mm))return -EINTR ;/*
@@ -1104,12 +1110,12 @@ int do_madvise(unsigned long start, size_t len_in, int behavior)* but for now we have the mmget_still_valid() * model. */ -if (!mmget_still_valid (current->mm)) { -mmap_write_unlock (current->mm); +if (!mmget_still_valid (mm)) { +mmap_write_unlock (mm);return -EINTR ; } }else { -mmap_read_lock (current->mm); +mmap_read_lock (mm); }/*
@@ -1117,7 +1123,7 @@ int do_madvise(unsigned long start, size_t len_in, int behavior)* ranges, just ignore them, but return -ENOMEM at the end. * - different from the way of handling in mlock etc. */ - vma =find_vma_prev (current->mm, start, &prev); + vma =find_vma_prev (mm, start, &prev);if (vma && start > vma->vm_start) prev = vma;
@@ -1154,19 +1160,19 @@ int do_madvise(unsigned long start, size_t len_in, int behavior)if (prev) vma = prev->vm_next;else /* madvise_remove dropped mmap_lock */ - vma =find_vma (current->mm, start); + vma =find_vma (mm, start); } out:blk_finish_plug (&plug);if (write) -mmap_write_unlock (current->mm); +mmap_write_unlock (mm);else -mmap_read_unlock (current->mm); +mmap_read_unlock (mm);return error; } SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior) { -return do_madvise (start, len_in, behavior); +return do_madvise (current, current->mm, start, len_in, behavior); }
Signature · 1 line
2.27.0.111.gc72c7da667-goog
PATCH v8 2/4 pid: move pidfd_get_pid() to pid.c
In reply to Minchan Kimprocess_madvise syscall needs pidfd_get_pid function to translate pidfd to pid so this patch move the function to kernel/pid.c.
--- include/linux/pid.h | 1 + kernel/exit.c | 17 ----------------- kernel/pid.c | 17 +++++++++++++++++ 3 files changed, 18 insertions(+), 17 deletions(-)
include/linux/pid.h +1 -0
@@ -77,6 +77,7 @@ extern const struct file_operations pidfd_fops;struct file ;extern struct pid *pidfd_pid (const struct file *file); +struct pid *pidfd_get_pid (unsigned int fd);static inline struct pid *get_pid (struct pid *pid) {
kernel/exit.c +0 -17
@@ -1475,23 +1475,6 @@ static long do_wait(struct wait_opts *wo)return retval; } -static struct pid *pidfd_get_pid (unsigned int fd) -{ -struct fd f; -struct pid *pid; - - f =fdget (fd); -if (!f.file) -return ERR_PTR (-EBADF ); - - pid =pidfd_pid (f.file); -if (!IS_ERR (pid)) -get_pid (pid); - -fdput (f); -return pid; -} -static long kernel_waitid (int which,pid_t upid,struct waitid_info *infop,int options,struct rusage *ru) {
kernel/pid.c +17 -0
@@ -518,6 +518,23 @@ struct pid *find_ge_pid(int nr, struct pid_namespace *ns)return idr_get_next (&ns->idr, &nr); } +struct pid *pidfd_get_pid (unsigned int fd) +{ +struct fd f; +struct pid *pid; + + f =fdget (fd); +if (!f.file) +return ERR_PTR (-EBADF ); + + pid =pidfd_pid (f.file); +if (!IS_ERR (pid)) +get_pid (pid); + +fdput (f); +return pid; +} +/** * pidfd_create() - Create a new pid file descriptor. *
Signature · 1 line
2.27.0.111.gc72c7da667-goog
PATCH v8 3/4 mm/madvise: introduce process_madvise() syscall: an external memory hinting API
In reply to Minchan KimThere is usecase that System Management Software(SMS) want to give a memory hint like MADV_[COLD|PAGEEOUT] to other processes and in the case of Android, it is the ActivityManagerService.
The information required to make the reclaim decision is not known to the app. Instead, it is known to the centralized userspace daemon(ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement.
To solve the issue, this patch introduces a new syscall process_madvise(2). It uses pidfd of an external process to give the hint. It also supports vector address range because Android app has thousands of vmas due to zygote so it's totally waste of CPU and power if we should call the syscall one by one for each vma.(With testing 2000-vma syscall vs 1-vector syscall, it showed 15% performance improvement. I think it would be bigger in real practice because the testing ran very cache friendly environment).
Another potential use case for the vector range is to amortize the cost ofTLB shootdowns for multiple ranges when using MADV_DONTNEED; this could benefit users like TCP receive zerocopy and malloc implementations. In future, we could find more usecases for other advises so let's make it happens as API since we introduce a new syscall at this moment. With that, existing madvise(2) user could replace it with process_madvise(2) with their own pid if they want to have batch address ranges support feature.
ince it could affect other process's address range, only privileged process(PTRACE_MODE_ATTACH_FSCREDS) or something else(e.g., being the same UID) gives it the right to ptrace the process could use it successfully. The flag argument is reserved for future use if we need to extend the API.
I think supporting all hints madvise has/will supported/support to process_madvise is rather risky. Because we are not sure all hints make sense from external process and implementation for the hint may rely on the caller being in the current context so it could be error-prone. Thus, I just limited hints as MADV_[COLD|PAGEOUT] in this patch.
If someone want to add other hints, we could hear hear the usecase and review it for each hint. It's safer for maintenance rather than introducing a buggy syscall but hard to fix it later.
So finally, the API is as follows,
ssize_t process_madvise(int pidfd, const struct iovec *iovec,
unsigned long vlen, int advice, unsigned int flags);
DESCRIPTION
The process_madvise() system call is used to give advice or directions to the kernel about the address ranges from external process as well as local process. It provides the advice to address ranges of process described by iovec and vlen. The goal of such advice is to improve system or application performance.
The pidfd selects the process referred to by the PID file descriptor specified in pidfd. (See pidofd_open(2) for further information)
The pointer iovec points to an array of iovec structures, defined in <sys/uio.h> as:
struct iovec {
void *iov_base; /* starting address */
size_t iov_len; /* number of bytes to be advised */
};
The iovec describes address ranges beginning at address(iov_base) and with size length of bytes(iov_len).
The vlen represents the number of elements in iovec.
The advice is indicated in the advice argument, which is one of the following at this moment if the target process specified by pidfd is external.
MADV_COLD
MADV_PAGEOUT
Permission to provide a hint to external process is governed by a ptrace access mode PTRACE_MODE_ATTACH_FSCREDS check; see ptrace(2).
The process_madvise supports every advice madvise(2) has if target process is in same thread group with calling process so user could use process_madvise(2) to extend existing madvise(2) to support vector address ranges.
RETURN VALUE
On success, process_madvise() returns the number of bytes advised. This return value may be less than the total number of requested bytes, if an error occurred. The caller should check return value to determine whether a partial advice occurred.
FAQ:
Q.1 - Why does any external entity have better knowledge?
Quote from Sandeep
"For Android, every application (including the special SystemServer) are forked from Zygote. The reason of course is to share as many libraries and classes between the two as possible to benefit from the preloading during boot.
After applications start, (almost) all of the APIs end up calling into this SystemServer process over IPC (binder) and back to the application.
In a fully running system, the SystemServer monitors every single process periodically to calculate their PSS / RSS and also decides which process is "important" to the user for interactivity.
So, because of how these processes start _and_ the fact that the SystemServer is looping to monitor each process, it does tend to *know* which address range of the application is not used / useful.
Besides, we can never rely on applications to clean things up themselves. We've had the "hey app1, the system is low on memory, please trim your memory usage down" notifications for a long time[1]. They rely on applications honoring the broadcasts and very few do.
So, if we want to avoid the inevitable killing of the application and restarting it, some way to be able to tell the OS about unimportant memory in these applications will be useful.
- ssp
Q.2 - How to guarantee the race(i.e., object validation) between when giving a hint from an external process and get the hint from the target process?
process_madvise operates on the target process's address space as it exists at the instant that process_madvise is called. If the space target process can run between the time the process_madvise process inspects the target process address space and the time that process_madvise is actually called, process_madvise may operate on memory regions that the calling process does not expect. It's the responsibility of the process calling process_madvise to close this race condition. For example, the calling process can suspend the target process with ptrace, SIGSTOP, or the freezer cgroup so that it doesn't have an opportunity to change its own address space before process_madvise is called. Another option is to operate on memory regions that the caller knows a priori will be unchanged in the target process. Yet another option is to accept the race for certain process_madvise calls after reasoning that mistargeting will do no harm. The suggested API itself does not provide synchronization. It also apply other APIs like move_pages, process_vm_write.
The race isn't really a problem though. Why is it so wrong to require that callers do their own synchronization in some manner? Nobody objects to write(2) merely because it's possible for two processes to open the same file and clobber each other's writes --- instead, we tell people to use flock or something. Think about mmap. It never guarantees newly allocated address space is still valid when the user tries to access it because other threads could unmap the memory right before. That's where we need synchronization by using other API or design from userside. It shouldn't be part of API itself. If someone needs more fine-grained synchronization rather than process level, there were two ideas suggested - cookie[2] and anon-fd[3]. Both are applicable via using last reserved argument of the API but I don't think it's necessary right now since we have already ways to prevent the race so don't want to add additional complexity with more fine-grained optimization model.
To make the API extend, it reserved an unsigned long as last argument so we could support it in future if someone really needs it.
Q.3 - Why doesn't ptrace work?
Injecting an madvise in the target process using ptrace would not work for us because such injected madvise would have to be executed by the target process, which means that process would have to be runnable and that creates the risk of the abovementioned race and hinting a wrong VMA. Furthermore, we want to act the hint in caller's context, not the callee's, because the callee is usually limited in cpuset/cgroups or even freezed state so they can't act by themselves quick enough, which causes more thrashing/kill. It doesn't work if the target process are ptraced(e.g., strace, debugger, minidump) because a process can have at most one ptracer.
[1] https://developer.android.com/topic/performance/memory"
[2] process_getinfo for getting the cookie which is updated whenever vma of process address layout are changed - Daniel Colascione -
https://lore.kernel.org/lkml/[email protected]/T/#m7694416fd179b2066a2c62b5b139b14e3894e224
[3] anonymous fd which is used for the object(i.e., address range) validation - Michal Hocko -
https://lore.kernel.org/lkml/[email protected]/
--- arch/alpha/kernel/syscalls/syscall.tbl | 1 + arch/arm/tools/syscall.tbl | 1 + arch/arm64/include/asm/unistd.h | 2 +- arch/arm64/include/asm/unistd32.h | 2 + arch/ia64/kernel/syscalls/syscall.tbl | 1 + arch/m68k/kernel/syscalls/syscall.tbl | 1 + arch/microblaze/kernel/syscalls/syscall.tbl | 1 + arch/mips/kernel/syscalls/syscall_n32.tbl | 1 + arch/mips/kernel/syscalls/syscall_n64.tbl | 1 + arch/mips/kernel/syscalls/syscall_o32.tbl | 1 + arch/parisc/kernel/syscalls/syscall.tbl | 1 + arch/powerpc/kernel/syscalls/syscall.tbl | 1 + arch/s390/kernel/syscalls/syscall.tbl | 1 + arch/sh/kernel/syscalls/syscall.tbl | 1 + arch/sparc/kernel/syscalls/syscall.tbl | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 2 + arch/xtensa/kernel/syscalls/syscall.tbl | 1 + include/linux/compat.h | 4 + include/linux/syscalls.h | 2 + include/uapi/asm-generic/unistd.h | 4 +- kernel/sys_ni.c | 2 + mm/madvise.c | 121 ++++++++++++++++++++ 23 files changed, 152 insertions(+), 2 deletions(-)
arch/alpha/kernel/syscalls/syscall.tbl +1 -0
@@ -482,3 +482,4 @@ 550 common watch_mount sys_watch_mount 551 common watch_sb sys_watch_sb 552 common fsinfo sys_fsinfo +553 common process_madvise sys_process_madvise
arch/arm/tools/syscall.tbl +1 -0
@@ -456,3 +456,4 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise
arch/arm64/include/asm/unistd.h +1 -1
@@ -38,7 +38,7 @@#define __ARM_NR_compat_set_tls (__ARM_NR_COMPAT_BASE + 5)#define __ARM_NR_COMPAT_END (__ARM_NR_COMPAT_BASE + 0x800) -#define __NR_compat_syscalls 443 +#define __NR_compat_syscalls 444#endif #define __ARCH_WANT_SYS_CLONE
arch/arm64/include/asm/unistd32.h +2 -0
@@ -893,6 +893,8 @@ __SYSCALL(__NR_watch_mount, sys_watch_mount) __SYSCALL(__NR_watch_sb, sys_watch_sb)#define __NR_fsinfo 442 __SYSCALL(__NR_fsinfo, sys_fsinfo) +#define __NR_fsinfo 443 +__SYSCALL (__NR_process_madvise, compat_sys_process_madvise)/* * Please add new compat syscalls above this comment and update
arch/ia64/kernel/syscalls/syscall.tbl +1 -0
@@ -363,3 +363,4 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise
arch/m68k/kernel/syscalls/syscall.tbl +1 -0
@@ -442,3 +442,4 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise
arch/microblaze/kernel/syscalls/syscall.tbl +1 -0
@@ -448,3 +448,4 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise
arch/mips/kernel/syscalls/syscall_n32.tbl +1 -0
@@ -381,3 +381,4 @@ 440 n32 watch_mount sys_watch_mount 441 n32 watch_sb sys_watch_sb 442 n32 fsinfo sys_fsinfo +443 n32 process_madvise compat_sys_process_madvise
arch/mips/kernel/syscalls/syscall_n64.tbl +1 -0
@@ -357,3 +357,4 @@ 440 n64 watch_mount sys_watch_mount 441 n64 watch_sb sys_watch_sb 442 n64 fsinfo sys_fsinfo +443 n64 process_madvise sys_process_madvise
arch/mips/kernel/syscalls/syscall_o32.tbl +1 -0
@@ -430,3 +430,4 @@ 440 o32 watch_mount sys_watch_mount 441 o32 watch_sb sys_watch_sb 442 o32 fsinfo sys_fsinfo +443 o32 process_madvise sys_process_madvise compat_sys_process_madvise
arch/parisc/kernel/syscalls/syscall.tbl +1 -0
@@ -440,3 +440,4 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise compat_sys_process_madvise
arch/powerpc/kernel/syscalls/syscall.tbl +1 -0
@@ -487,3 +487,4 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise compat_sys_process_madvise
arch/s390/kernel/syscalls/syscall.tbl +1 -0
@@ -445,3 +445,4 @@ 440 common watch_mount sys_watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise compat_sys_process_madvise
arch/sh/kernel/syscalls/syscall.tbl +1 -0
@@ -445,3 +445,4 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise
arch/sparc/kernel/syscalls/syscall.tbl +1 -0
@@ -488,3 +488,4 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise compat_sys_process_madvise
arch/x86/entry/syscalls/syscall_32.tbl +1 -0
@@ -447,3 +447,4 @@ 440 i386 watch_mount sys_watch_mount 441 i386 watch_sb sys_watch_sb 442 i386 fsinfo sys_fsinfo +443 i386 process_madvise sys_process_madvise compat_sys_process_madvise
arch/x86/entry/syscalls/syscall_64.tbl +2 -0
@@ -364,6 +364,7 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 64 process_madvise sys_process_madvise # # x32-specific system call numbers start at 512 to avoid cache impact
@@ -407,3 +408,4 @@ 545 x32 execveat compat_sys_execveat 546 x32 preadv2 compat_sys_preadv64v2 547 x32 pwritev2 compat_sys_pwritev64v2 +548 x32 process_madvise compat_sys_process_madvise
arch/xtensa/kernel/syscalls/syscall.tbl +1 -0
@@ -413,3 +413,4 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 common process_madvise sys_process_madvise
include/linux/compat.h +4 -0
@@ -827,6 +827,10 @@ asmlinkage long compat_sys_pwritev64v2(unsigned long fd, unsigned long vlen, loff_t pos, rwf_t flags);#endif +asmlinkage ssize_t compat_sys_process_madvise(compat_int_t pidfd, +const struct compat_iovec __user *vec, +compat_ulong_t vlen,compat_int_t behavior, +compat_uint_t flags);/* * Deprecated system calls which are still defined in
include/linux/syscalls.h +2 -0
@@ -880,6 +880,8 @@ asmlinkage long sys_munlockall(void);asmlinkage long sys_mincore (unsigned long start,size_t len,unsigned char __user * vec);asmlinkage long sys_madvise (unsigned long start,size_t len,int behavior); +asmlinkage long sys_process_madvise (int pidfd,const struct iovec __user *vec, +unsigned long vlen,int behavior,unsigned int flags);asmlinkage long sys_remap_file_pages (unsigned long start,unsigned long size,unsigned long prot,unsigned long pgoff,unsigned long flags);
include/uapi/asm-generic/unistd.h +3 -1
@@ -865,9 +865,11 @@ __SYSCALL(__NR_watch_mount, sys_watch_mount) __SYSCALL(__NR_watch_sb, sys_watch_sb)#define __NR_fsinfo 442 __SYSCALL(__NR_fsinfo, sys_fsinfo) +#define __NR_fsinfo 443 +__SC_COMP(__NR_process_madvise, sys_process_madvise, compat_sys_process_madvise)#undef __NR_syscalls -#define __NR_syscalls 443 +#define __NR_syscalls 444/* * 32 bit systems traditionally used different
kernel/sys_ni.c +2 -0
@@ -287,6 +287,8 @@ COND_SYSCALL(mlockall);COND_SYSCALL (munlockall);COND_SYSCALL (mincore);COND_SYSCALL (madvise); +COND_SYSCALL (process_madvise); +COND_SYSCALL_COMPAT (process_madvise);COND_SYSCALL (remap_file_pages);COND_SYSCALL (mbind);COND_SYSCALL_COMPAT (mbind);
mm/madvise.c +121 -0
142 lines not on this page · raw message
Signature · 1 line
2.27.0.111.gc72c7da667-goog
PATCH v8 4/4 mm/madvise: check fatal signal pending of target process
In reply to Minchan KimBail out to prevent unnecessary CPU overhead if target process has pending fatal signal during (MADV_COLD|MADV_PAGEOUT) operation.
--- mm/madvise.c | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-)
mm/madvise.c +21 -8
@@ -39,6 +39,7 @@struct madvise_walk_private {struct mmu_gather *tlb;bool pageout; +struct task_struct *target_task; };/*
@@ -319,6 +320,10 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,if (fatal_signal_pending (current))return -EINTR ; +if (private->target_task && +fatal_signal_pending (private->target_task)) +return -EINTR ; +#ifdef CONFIG_TRANSPARENT_HUGEPAGE if (pmd_trans_huge (*pmd)) {pmd_t orig_pmd;
@@ -480,12 +485,14 @@ static const struct mm_walk_ops cold_walk_ops = { };static void madvise_cold_page_range (struct mmu_gather *tlb, +struct task_struct *task,struct vm_area_struct *vma,unsigned long addr,unsigned long end) {struct madvise_walk_private walk_private = { .pageout = false, .tlb = tlb, + .target_task = task, };tlb_start_vma (tlb, vma);
@@ -493,7 +500,8 @@ static void madvise_cold_page_range(struct mmu_gather *tlb,tlb_end_vma (tlb, vma); } -static long madvise_cold (struct vm_area_struct *vma, +static long madvise_cold (struct task_struct *task, +struct vm_area_struct *vma,struct vm_area_struct **prev,unsigned long start_addr,unsigned long end_addr) {
@@ -506,19 +514,21 @@ static long madvise_cold(struct vm_area_struct *vma,lru_add_drain ();tlb_gather_mmu (&tlb, mm, start_addr, end_addr); -madvise_cold_page_range (&tlb, vma, start_addr, end_addr); +madvise_cold_page_range (&tlb, task, vma, start_addr, end_addr);tlb_finish_mmu (&tlb, start_addr, end_addr);return 0 ; }static void madvise_pageout_page_range (struct mmu_gather *tlb, +struct task_struct *task,struct vm_area_struct *vma,unsigned long addr,unsigned long end) {struct madvise_walk_private walk_private = { .pageout = true, .tlb = tlb, + .target_task = task, };tlb_start_vma (tlb, vma);
@@ -542,7 +552,8 @@ static inline bool can_do_pageout(struct vm_area_struct *vma)inode_permission (file_inode (vma->vm_file),MAY_WRITE ) ==0 ; } -static long madvise_pageout (struct vm_area_struct *vma, +static long madvise_pageout (struct task_struct *task, +struct vm_area_struct *vma,struct vm_area_struct **prev,unsigned long start_addr,unsigned long end_addr) {
@@ -558,7 +569,7 @@ static long madvise_pageout(struct vm_area_struct *vma,lru_add_drain ();tlb_gather_mmu (&tlb, mm, start_addr, end_addr); -madvise_pageout_page_range (&tlb, vma, start_addr, end_addr); +madvise_pageout_page_range (&tlb, task, vma, start_addr, end_addr);tlb_finish_mmu (&tlb, start_addr, end_addr);return 0 ;
@@ -938,7 +949,8 @@ static int madvise_inject_error(int behavior,#endif static long -madvise_vma (struct vm_area_struct *vma,struct vm_area_struct **prev, +madvise_vma (struct task_struct *task,struct vm_area_struct *vma, +struct vm_area_struct **prev,unsigned long start,unsigned long end,int behavior) {switch (behavior) {
@@ -947,9 +959,9 @@ madvise_vma(struct vm_area_struct *vma, struct vm_area_struct **prev,case MADV_WILLNEED :return madvise_willneed (vma, prev, start, end);case MADV_COLD : -return madvise_cold (vma, prev, start, end); +return madvise_cold (task, vma, prev, start, end);case MADV_PAGEOUT : -return madvise_pageout (vma, prev, start, end); +return madvise_pageout (task, vma, prev, start, end);case MADV_FREE :case MADV_DONTNEED :return madvise_dontneed_free (vma, prev, start, end, behavior);
@@ -1166,7 +1178,8 @@ int do_madvise(struct task_struct *target_task, struct mm_struct *mm, tmp = end;/* Here vma->vm_start <= start < tmp <= (end|vma->vm_end). */ - error =madvise_vma (vma, &prev, start, tmp, behavior); + error =madvise_vma (target_task, vma, &prev, + start, tmp, behavior);if (error) goto out; start = tmp;
Signature · 1 line
2.27.0.111.gc72c7da667-goog
Quoted 18 lines · Minchan Kim
Now, we have
MADV_PAGEOUTandMADV_COLDas madvise hinting API. With that, application could give hints to kernel what memory range are preferred to be reclaimed. However, in someplatform(e.g., Android), the information required to make the hinting decision is not known to the app.
Instead, it is known to a centralized userspacedaemon(e.g., ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement.To solve the concern, this patch introduces new syscall -
process_madvise(2). Bascially, it's same with madvise(2) syscall but it has some differences.1. It needs pidfd of target process to provide the hint
2. It supports only MADV_{COLD|PAGEOUT} at this moment.
Other hints in madvise will be opened when there are explicit requests from community to prevent unexpected bugs we couldn't support.
3. Only privileged processes can do something for other process's address space.
Quoted 8 lines
For more detail of the new API, please see "mm: introduce external memory hinting API" description in this patchset.
* from v7 - http://lore.kernel.org/r/[email protected]
* dropping pid support from new syscall and fold releated patches into syscall patch
* dropping KSM patch by discussion - Oleksandr, I lost the discussion. Please resend the single patch against of the patchset if you resolves the discussion.
https://lore.kernel.org/linux-api/[email protected]/
Oleksandr, it seems you discussed something with Vlastimil but couldn't find conclustion yet and Since Jann put an a new note in the thread, I detach the patch from this patchset.
Please send the KSM patch based on this patchset if you belive there is no need to be actionable for comments.
Hello.
Quoted 28 lines · Minchan Kim
Quoted 18 lines · Minchan Kim
Now, we have
MADV_PAGEOUTandMADV_COLDas madvise hinting API. With that, application could give hints to kernel what memory range are preferred to be reclaimed. However, in someplatform(e.g., Android), the information required to make the hinting decision is not known to the app.
Instead, it is known to a centralized userspacedaemon(e.g., ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement.To solve the concern, this patch introduces new syscall -
process_madvise(2). Bascially, it's same with madvise(2) syscall but it has some differences.1. It needs pidfd of target process to provide the hint
2. It supports only MADV_{COLD|PAGEOUT} at this moment.
Other hints in madvise will be opened when there are explicit requests from community to prevent unexpected bugs we couldn't support.
3. Only privileged processes can do something for other process's address space.Quoted 8 lines
For more detail of the new API, please see "mm: introduce external memory hinting API" description in this patchset.
* from v7 - http://lore.kernel.org/r/[email protected]
* dropping pid support from new syscall and fold releated patches into syscall patch
* dropping KSM patch by discussion - Oleksandr, I lost the discussion. Please resend the single patch against of the patchset if you resolves the discussion.
https://lore.kernel.org/linux-api/[email protected]/
Quoted 6 lines
Oleksandr, it seems you discussed something with Vlastimil but couldn't find conclustion yet and Since Jann put an a new note in the thread, I detach the patch from this patchset.
Please send the KSM patch based on this patchset if you belive there is no need to be actionable for comments.
I don't think we came to some definite conclusion, but I'll try to implement Vlastimil's suggestion and then send it separately for evaluation.
Lets keep KSM bit out of your submission so I won't slow it down.
Signature · 3 lines
Best regards,
Oleksandr Natalenko (post-factum)
Principal Software Maintenance EngineerQuoted 18 lines · Minchan Kim
Now, we have
MADV_PAGEOUTandMADV_COLDas madvise hinting API. With that, application could give hints to kernel what memory range are preferred to be reclaimed. However, in someplatform(e.g., Android), the information required to make the hinting decision is not known to the app.
Instead, it is known to a centralized userspacedaemon(e.g., ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement.To solve the concern, this patch introduces new syscall -
process_madvise(2). Bascially, it's same with madvise(2) syscall but it has some differences.1. It needs pidfd of target process to provide the hint
2. It supports only MADV_{COLD|PAGEOUT} at this moment.
Other hints in madvise will be opened when there are explicit requests from community to prevent unexpected bugs we couldn't support.
3. Only privileged processes can do something for other process's address space.
Quoted 8 lines
For more detail of the new API, please see "mm: introduce external memory hinting API" description in this patchset.
* from v7 - http://lore.kernel.org/r/[email protected]
* dropping pid support from new syscall and fold releated patches into syscall patch
* dropping KSM patch by discussion - Oleksandr, I lost the discussion. Please resend the single patch against of the patchset if you resolves the discussion.
https://lore.kernel.org/linux-api/[email protected]/
What "next" tag this (v8) submission is based on please?
Quoted 69 lines
* from v6 - https://lore.kernel.org/linux-api/[email protected]/
* fix comments and descriptions - Suren
* Add Reviewed-by - Suren
* fix build break reported by 0-day* from v5 - https://lore.kernel.org/linux-mm/[email protected]/
* use null task and requestor's mm forio_madvise- Jann and Jens
* use right commit description for movingpidfd_get_pid- Christoph* from v4 - https://lore.kernel.org/linux-mm/[email protected]/
* pass mm down to functions, not accessingtask->mm- Jann
* clean up - Alexander
* add Reviewed-by - Alexander, SeongJae
* patch reordering* from v3 - https://lore.kernel.org/linux-mm/[email protected]/
* verifytask->mmaftereaccess_mm- Oleg
* split some patches for easy review - Alexander
* clean up fatal signal checking - Suren* from v2 - https://lore.kernel.org/linux-mm/[email protected]/
* check signal callee and caller to bail out - Kirill Tkhai
* put more clarification for justification of new API* from v1 - https://lore.kernel.org/linux-mm/[email protected]/
* fix syscall number - SeongJae
* useget_pid_task- Kirill Tkhai
* extend API to support pid as well as pidfd - Kirill TkhaiMinchan Kim (4):
mm/madvise: pass task and mm todo_madvise
pid: movepidfd_get_pid()topid.c
mm/madvise: introduceprocess_madvise()syscall: an external memory
hinting API
mm/madvise: check fatal signal pending of target processarch/alpha/kernel/syscalls/syscall.tbl | 1 + arch/arm/tools/syscall.tbl | 1 + arch/arm64/include/asm/unistd.h | 2 +- arch/arm64/include/asm/unistd32.h | 2 + arch/ia64/kernel/syscalls/syscall.tbl | 1 + arch/m68k/kernel/syscalls/syscall.tbl | 1 + arch/microblaze/kernel/syscalls/syscall.tbl | 1 + arch/mips/kernel/syscalls/syscall_n32.tbl | 1 + arch/mips/kernel/syscalls/syscall_n64.tbl | 1 + arch/mips/kernel/syscalls/syscall_o32.tbl | 1 + arch/parisc/kernel/syscalls/syscall.tbl | 1 + arch/powerpc/kernel/syscalls/syscall.tbl | 1 + arch/s390/kernel/syscalls/syscall.tbl | 1 + arch/sh/kernel/syscalls/syscall.tbl | 1 + arch/sparc/kernel/syscalls/syscall.tbl | 1 + arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 2 + arch/xtensa/kernel/syscalls/syscall.tbl | 1 + fs/io_uring.c | 2 +- include/linux/compat.h | 4 + include/linux/mm.h | 3 +- include/linux/pid.h | 1 + include/linux/syscalls.h | 2 + include/uapi/asm-generic/unistd.h | 4 +- kernel/exit.c | 17 -- kernel/pid.c | 17 ++ kernel/sys_ni.c | 2 + mm/madvise.c | 190 +++++++++++++++++--- 28 files changed, 217 insertions(+), 46 deletions(-)Signature · 1 line
2.27.0.111.gc72c7da667-goog
Signature · 3 lines
Best regards,
Oleksandr Natalenko (post-factum)
Principal Software Maintenance EngineerHi Oleksandr,
Quoted 22 lines · Oleksandr Natalenko
Quoted 21 lines · Minchan Kim
Now, we have
MADV_PAGEOUTandMADV_COLDas madvise hinting API. With that, application could give hints to kernel what memory range are preferred to be reclaimed. However, in someplatform(e.g., Android), the information required to make the hinting decision is not known to the app.
Instead, it is known to a centralized userspacedaemon(e.g., ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement.To solve the concern, this patch introduces new syscall -
process_madvise(2). Bascially, it's same with madvise(2) syscall but it has some differences.1. It needs pidfd of target process to provide the hint
2. It supports only MADV_{COLD|PAGEOUT} at this moment.
Other hints in madvise will be opened when there are explicit requests from community to prevent unexpected bugs we couldn't support.
3. Only privileged processes can do something for other process's address space.For more detail of the new API, please see "mm: introduce external memory hinting API" description in this patchset.
Quoted 7 lines
Quoted 5 lines
* from v7 - http://lore.kernel.org/r/[email protected]
* dropping pid support from new syscall and fold releated patches into syscall patch
* dropping KSM patch by discussion - Oleksandr, I lost the discussion. Please resend the single patch against of the patchset if you resolves the discussion.
https://lore.kernel.org/linux-api/[email protected]/What "next" tag this (v8) submission is based on please?
It's against on v5.8-rc1-mmots-2020-06-20-21-44 from mmotm - https://github.com/hnaz/linux-mm.git
Re: [PATCH v8 4/4] mm/madvise: check fatal signal pending of target process
In reply to Minchan KimQuoted 19 lines · Minchan Kim
Bail out to prevent unnecessary CPU overhead if target process has pending fatal signal during (MADV_COLD|MADV_PAGEOUT) operation.
Signed-off-by Minchan Kim <[email protected]>Reviewed-by Suren Baghdasaryan <[email protected]>Reviewed-by Vlastimil Babka <[email protected]>Cc Alexander Duyck <[email protected]>Cc Brian Geffon <[email protected]>Cc Christian Brauner <[email protected]>Cc Daniel Colascione <[email protected]>Cc Jann Horn <[email protected]>Cc Jens Axboe <[email protected]>Cc Joel Fernandes <[email protected]>Cc Johannes Weiner <[email protected]>Cc John Dias <[email protected]>Cc Kirill Tkhai <[email protected]>Cc Michal Hocko <[email protected]>Cc Oleksandr Natalenko <[email protected]>
Quoted 8 lines
Cc Sandeep Patil <[email protected]>Cc SeongJae Park <[email protected]>Cc SeongJae Park <[email protected]>Cc Shakeel Butt <[email protected]>Cc Sonny Rao <[email protected]>Cc Tim Murray <[email protected]>Cc Christian Brauner <[email protected]>Cc <[email protected]>
Re: [PATCH v8 3/4] mm/madvise: introduce process_madvise() syscall: an external memory hinting API
In reply to Minchan KimQuoted 90 lines · Minchan Kim
mm/madvise.c +60
-0@@ -17,6 +17,7 @@#include <linux/falloc.h> #include <linux/fadvise.h> #include <linux/sched.h> +#include <linux/sched/mm.h> #include <linux/ksm.h> #include <linux/fs.h> #include <linux/file.h> @@ -995,6 +996,18 @@ madvise_behavior_valid(int behavior) } } +static bool +process_madvise_behavior_valid (int behavior) +{ +switch (behavior) { +case MADV_COLD : +case MADV_PAGEOUT : +return true; +default : +return false; + } +} +/* * The madvise(2) system call. * @@ -1042,6 +1055,11 @@ madvise_behavior_valid(int behavior)* MADV_DONTDUMP - the application wants to prevent pages in the given range * from being included in its core dump. * MADV_DODUMP - cancel MADV_DONTDUMP: no longer exclude from core dump. +* MADV_COLD - the application is not expected to use this memory soon, +* deactivate pages in this range so that they can be reclaimed +* easily if memory pressure hanppens. +* MADV_PAGEOUT - the application is not expected to use this memory soon, +* page out the pages in this range immediately. * * return values: * zero - success @@ -1176,3 +1194,106 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior) {return do_madvise (current, current->mm, start, len_in, behavior); } + +static int process_madvise_vec (struct task_struct *target_task, +struct mm_struct *mm,struct iov_iter *iter,int behavior) +{ +struct iovec iovec; +int ret =0 ; + +while (iov_iter_count (iter)) { + iovec =iov_iter_iovec (iter); + ret =do_madvise (target_task, mm, (unsigned long )iovec.iov_base, + iovec.iov_len, behavior); +if (ret <0 ) +break ; +iov_iter_advance (iter, iovec.iov_len); + } + +return ret; +} + +static ssize_t do_process_madvise (int pidfd,struct iov_iter *iter, +int behavior,unsigned int flags) +{ +ssize_t ret; +struct pid *pid; +struct task_struct *task; +struct mm_struct *mm; +size_t total_len =iov_iter_count (iter); + +if (flags !=0 ) +return -EINVAL ; + + pid =pidfd_get_pid (pidfd); +if (IS_ERR (pid)) +return PTR_ERR (pid); + + task =get_pid_task (pid,PIDTYPE_PID ); +if (!task) { + ret = -ESRCH ; + goto put_pid; + } + + if (task->mm != current->mm &&
mm/madvise.c SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior) · line 1248
Quoted 10 lines
mm/madvise.c +10
-0+ !process_madvise_behavior_valid(behavior)) { + ret = -EINVAL ; + goto release_task; + } + + mm =mm_access (task,PTRACE_MODE_ATTACH_FSCREDS ); +if (IS_ERR_OR_NULL (mm)) { + ret =IS_ERR (mm) ?PTR_ERR (mm) : -ESRCH ; + goto release_task; + }
mm is always task->mm right? I'm wondering if it would be better to find the mm directly in process_madvise_vec() rather than passing it into the function. I'm not sure why we'd pass both task and mm here.
+
Quoted 4 lines
mm/madvise.c +4
-0+ ret =process_madvise_vec (task, mm, iter, behavior); +if (ret >=0 ) + ret = total_len -iov_iter_count (iter); +
mm/madvise.c SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior) · line 1263
Quoted 10 lines
mm/madvise.c +10
-0+mmput (mm); +release_task: +put_task_struct (task); +put_pid: +put_pid (pid); +return ret; +} + +SYSCALL_DEFINE5(process_madvise, int, pidfd, const struct iovec __user *, vec, + unsigned long, vlen, int, behavior, unsigned int, flags)
I love the idea of adding the flags parameter here and I can think of an immediate use case for MADV_HUGEPAGE, which is overloaded.
Today, MADV_HUGEPAGE controls enablement depending on system config and controls defrag behavior based on system config. It also cannot be opted out of without setting MADV_NOHUGEPAGE :)
I was thinking of a flag that users could use to trigger an immediate collapse in process context regardless of the system config.
So I'm a big advocate of this flags parameter and consider it an absolute must for the API.
Re: [PATCH v8 1/4] mm/madvise: pass task and mm to do_madvise
In reply to Minchan KimQuoted 68 lines · Minchan Kim
Patch series "introduce memory hinting API for external process", v8.
Now, we have
MADV_PAGEOUTandMADV_COLDas madvise hinting API. With that, application could give hints to kernel what memory range are preferred to be reclaimed. However, in someplatform(e.g., Android), the information required to make the hinting decision is not known to the app. Instead, it is known to a centralized userspacedaemon(e.g., ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement.To solve the concern, this patch introduces new syscall -
process_madvise(2). Bascially, it's same with madvise(2) syscall but it has some differences.1. It needs pidfd of target process to provide the hint
2. It supports only MADV_{COLD|PAGEOUT|MERGEABLE|UNMEREABLE} at this moment. Other hints in madvise will be opened when there are explicit requests from community to prevent unexpected bugs we couldn't support.
3. Only privileged processes can do something for other process's address space.
For more detail of the new API, please see "mm: introduce external memory hinting API" description in this patchset.
This patch (of 4):
In upcoming patches,
do_madvisewill be called from external process context so we shouldn't asssume "current" is always hinted process'stask_struct.Furthermore, we must not access
mm_structviatask->mm, but obtain it viaaccess_mm()once (in the following patch) and only use that pointer [1], so pass it todo_madvise()as well. Note thevma->vm_mmpointers are safe, so we can use them further down the call stack.And let's pass *current* and
current->mmas arguments ofdo_madviseso it shouldn't change existing behavior but prepare next patch to make review easy.Note:
io_madvisepasses NULL astarget_taskargument ofdo_madvisebecause it couldn't know who is target.[1] http://lore.kernel.org/r/CAG48ez27=pwm5m_N_988xT1huO7g7h6arTQL44zev6TD-h-7Tg@mail.gmail.com
[[email protected]: changelog tweak][[email protected]: use current->mm for io_uring][[email protected]: fix it for upstream changes][[email protected]: whoops][[email protected]: add missing includes]Signed-off-by Minchan Kim <[email protected]>Reviewed-by Suren Baghdasaryan <[email protected]>Reviewed-by Vlastimil Babka <[email protected]>Cc Jens Axboe <[email protected]>Cc Jann Horn <[email protected]>Cc Tim Murray <[email protected]>Cc Daniel Colascione <[email protected]>Cc Sandeep Patil <[email protected]>Cc Sonny Rao <[email protected]>Cc Brian Geffon <[email protected]>Cc Michal Hocko <[email protected]>Cc Johannes Weiner <[email protected]>Cc Shakeel Butt <[email protected]>Cc John Dias <[email protected]>Cc Joel Fernandes <[email protected]>
Quoted 8 lines
Cc Alexander Duyck <[email protected]>Cc SeongJae Park <[email protected]>Cc Christian Brauner <[email protected]>Cc Kirill Tkhai <[email protected]>Cc Oleksandr Natalenko <[email protected]>Cc SeongJae Park <[email protected]>Cc Christian Brauner <[email protected]>Cc <[email protected]>
Re: [PATCH v8 2/4] pid: move pidfd_get_pid() to pid.c
In reply to Minchan KimQuoted 20 lines · Minchan Kim
process_madvisesyscall needspidfd_get_pidfunction to translate pidfd to pid so this patch move the function tokernel/pid.c.Signed-off-by Minchan Kim <[email protected]>Reviewed-by Suren Baghdasaryan <[email protected]>Suggested-by Alexander Duyck <[email protected]>Reviewed-by Alexander Duyck <[email protected]>Reviewed-by Vlastimil Babka <[email protected]>Cc Jens Axboe <[email protected]>Cc Jann Horn <[email protected]>Cc Brian Geffon <[email protected]>Cc Daniel Colascione <[email protected]>Cc Joel Fernandes <[email protected]>Cc Johannes Weiner <[email protected]>Cc John Dias <[email protected]>Cc Kirill Tkhai <[email protected]>Cc Michal Hocko <[email protected]>Cc Oleksandr Natalenko <[email protected]>
Quoted 8 lines
Cc Sandeep Patil <[email protected]>Cc SeongJae Park <[email protected]>Cc SeongJae Park <[email protected]>Cc Shakeel Butt <[email protected]>Cc Sonny Rao <[email protected]>Cc Tim Murray <[email protected]>Cc Christian Brauner <[email protected]>Cc <[email protected]>
Re: [PATCH v8 3/4] mm/madvise: introduce process_madvise() syscall: an external memory hinting API
In reply to David RientjesQuoted 98 lines · David Rientjes
Quoted 96 lines · Minchan Kim
mm/madvise.c +67
-0@@ -17,6 +17,7 @@#include <linux/falloc.h> #include <linux/fadvise.h> #include <linux/sched.h> +#include <linux/sched/mm.h> #include <linux/ksm.h> #include <linux/fs.h> #include <linux/file.h> @@ -995,6 +996,18 @@ madvise_behavior_valid(int behavior) } } +static bool +process_madvise_behavior_valid (int behavior) +{ +switch (behavior) { +case MADV_COLD : +case MADV_PAGEOUT : +return true; +default : +return false; + } +} +/* * The madvise(2) system call. * @@ -1042,6 +1055,11 @@ madvise_behavior_valid(int behavior)* MADV_DONTDUMP - the application wants to prevent pages in the given range * from being included in its core dump. * MADV_DODUMP - cancel MADV_DONTDUMP: no longer exclude from core dump. +* MADV_COLD - the application is not expected to use this memory soon, +* deactivate pages in this range so that they can be reclaimed +* easily if memory pressure hanppens. +* MADV_PAGEOUT - the application is not expected to use this memory soon, +* page out the pages in this range immediately. * * return values: * zero - success @@ -1176,3 +1194,106 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior) {return do_madvise (current, current->mm, start, len_in, behavior); } + +static int process_madvise_vec (struct task_struct *target_task, +struct mm_struct *mm,struct iov_iter *iter,int behavior) +{ +struct iovec iovec; +int ret =0 ; + +while (iov_iter_count (iter)) { + iovec =iov_iter_iovec (iter); + ret =do_madvise (target_task, mm, (unsigned long )iovec.iov_base, + iovec.iov_len, behavior); +if (ret <0 ) +break ; +iov_iter_advance (iter, iovec.iov_len); + } + +return ret; +} + +static ssize_t do_process_madvise (int pidfd,struct iov_iter *iter, +int behavior,unsigned int flags) +{ +ssize_t ret; +struct pid *pid; +struct task_struct *task; +struct mm_struct *mm; +size_t total_len =iov_iter_count (iter); + +if (flags !=0 ) +return -EINVAL ; + + pid =pidfd_get_pid (pidfd); +if (IS_ERR (pid)) +return PTR_ERR (pid); + + task =get_pid_task (pid,PIDTYPE_PID ); +if (!task) { + ret = -ESRCH ; + goto put_pid; + } + +if (task->mm != current->mm && + !process_madvise_behavior_valid (behavior)) { + ret = -EINVAL ; + goto release_task; + } + + mm =mm_access (task,PTRACE_MODE_ATTACH_FSCREDS ); +if (IS_ERR_OR_NULL (mm)) {
Quoted 8 lines
mm/madvise.c SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior) · line 1248
Quoted 4 lines
mm/madvise.c +3
-0+ ret =IS_ERR (mm) ?PTR_ERR (mm) : -ESRCH ; + goto release_task; + }mm is always
task->mmright? I'm wondering if it would be better to find the mm directly inprocess_madvise_vec()rather than passing it into the function. I'm not sure why we'd pass both task and mm here.
That's because of hint Jann provided in the past version.
https://lore.kernel.org/linux-api/CAG48ez27=pwm5m_N_988xT1huO7g7h6arTQL44zev6TD-h-7Tg@mail.gmail.com/
Thanks for the review, David.
Arnd Bergmann
maintainerRe: [PATCH v8 3/4] mm/madvise: introduce process_madvise() syscall: an external memory hinting API
In reply to Minchan KimQuoted 4 lines · Minchan Kim
So finally, the API is as follows,
ssize_tprocess_madvise(int pidfd, const struct iovec *iovec,
unsigned long vlen, int advice, unsigned int flags);
I had not followed the discussion earlier and only now came across the syscall in linux-next, sorry for stirring things up this late.
Quoted 13 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -364,6 +364,7 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 64 process_madvise sys_process_madvise # # x32-specific system call numbers start at 512 to avoid cache impact
arch/x86/entry/syscalls/syscall_64.tbl line 411
Quoted 4 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -407,3 +408,4 @@ 545 x32 execveat compat_sys_execveat 546 x32 preadv2 compat_sys_preadv64v2 547 x32 pwritev2 compat_sys_pwritev64v2 +548 x32 process_madvise compat_sys_process_madvise
I think we should not add any new x32-specific syscalls. Instead I think the compat_sys_process_madvise/sys_process_madvise can be merged into one.
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 5 lines
arch/x86/entry/syscalls/syscall_64.tbl +5
-0+ mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS); + if (IS_ERR_OR_NULL(mm)) { + ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH; + goto release_task; + }
Minor point: Having to use IS_ERR_OR_NULL() tends to be fragile, and I would try to avoid that. Can mm_access() be changed to itself return PTR_ERR(-ESRCH) instead of NULL to improve its calling conventions? I see there are only three other callers.
Quoted 9 lines
arch/x86/entry/syscalls/syscall_64.tbl +9
-0+ ret = import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + } + return ret; +} + +#ifdef CONFIG_COMPAT
...
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 7 lines
arch/x86/entry/syscalls/syscall_64.tbl +7
-0+ + ret = compat_import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), + &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + }
Every syscall that passes an iovec seems to do this. If we make import_iovec() handle both cases directly, this syscall and a number of others can be simplified, and you avoid the x32 entry point I mentioned above
Something like (untested)
index dad8d0cfaaf7..0de4ddff24c1 100644
lib/iov_iter.c +0 -0
@@ -1683,8 +1683,13 @@ ssize_t import_iovec(int type, const struct
iovec __user * uvector,
{
ssize_t n;
struct iovec *p;- n = rw_copy_check_uvector(type, uvector, nr_segs, fast_segs,
- *iov, &p);
+
+ if (in_compat_syscall())
+ n = compat_rw_copy_check_uvector(type, uvector, nr_segs,
+ fast_segs, *iov, &p);
+ else
+ n = rw_copy_check_uvector(type, uvector, nr_segs,
+ fast_segs, *iov, &p);
if (n < 0) {
if (p != *iov)
kfree(p);
ArndRe: [PATCH v8 3/4] mm/madvise: introduce process_madvise() syscall: an external memory hinting API
Quoted 86 lines · Arnd Bergmann
Quoted 4 lines · Minchan Kim
So finally, the API is as follows,
ssize_tprocess_madvise(int pidfd, const struct iovec *iovec,
unsigned long vlen, int advice, unsigned int flags);I had not followed the discussion earlier and only now came across the syscall in linux-next, sorry for stirring things up this late.
Quoted 13 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -364,6 +364,7 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 64 process_madvise sys_process_madvise # # x32-specific system call numbers start at 512 to avoid cache impactarch/x86/entry/syscalls/syscall_64.tbl line 411
Quoted 4 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -407,3 +408,4 @@ 545 x32 execveat compat_sys_execveat 546 x32 preadv2 compat_sys_preadv64v2 547 x32 pwritev2 compat_sys_pwritev64v2 +548 x32 process_madvise compat_sys_process_madviseI think we should not add any new x32-specific syscalls. Instead I think the compat_sys_process_madvise/sys_process_madvise can be merged into one.
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 5 lines
arch/x86/entry/syscalls/syscall_64.tbl +5
-0+ mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS); + if (IS_ERR_OR_NULL(mm)) { + ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH; + goto release_task; + }Minor point: Having to use
IS_ERR_OR_NULL()tends to be fragile, and I would try to avoid that. Canmm_access()be changed to itself returnPTR_ERR(-ESRCH)instead of NULL to improve its calling conventions? I see there are only three other callers.Quoted 9 lines
arch/x86/entry/syscalls/syscall_64.tbl +9
-0+ ret = import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + } + return ret; +} + +#ifdef CONFIG_COMPAT...
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 7 lines
arch/x86/entry/syscalls/syscall_64.tbl +7
-0+ + ret = compat_import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), + &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + }Every syscall that passes an iovec seems to do this. If we make
import_iovec()handle both cases directly, this syscall and a number of others can be simplified, and you avoid the x32 entry point I mentioned aboveSomething like (untested)
index
dad8d0cfaaf7..0de4ddff24c1100644lib/iov_iter.c +7
-2@@ -1683,8 +1683,13 @@ ssize_t import_iovec(int type, const structiovec __user * uvector, { ssize_t n;struct iovec *p; - n =rw_copy_check_uvector (type, uvector, nr_segs, fast_segs, - *iov, &p); + +if (in_compat_syscall ()) + n =compat_rw_copy_check_uvector (type, uvector, nr_segs, + fast_segs, *iov, &p); +else + n =rw_copy_check_uvector (type, uvector, nr_segs, + fast_segs, *iov, &p);if (n <0 ) {if (p != *iov)
Quoted 1 line
kfree(p);
Doesn't work for the async case, where you want to be holding on to the allocated iovec. But in general I think it's a good helper for the sync case, which is by far the majority.
Signature · 1 line
Jens Axboe
Re: [PATCH v8 3/4] mm/madvise: introduce process_madvise() syscall: an external memory hinting API
Quoted 69 lines · Jens Axboe
Quoted 67 lines · Arnd Bergmann
Quoted 4 lines · Minchan Kim
So finally, the API is as follows,
ssize_tprocess_madvise(int pidfd, const struct iovec *iovec,
unsigned long vlen, int advice, unsigned int flags);I had not followed the discussion earlier and only now came across the syscall in linux-next, sorry for stirring things up this late.
Quoted 13 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -364,6 +364,7 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 64 process_madvise sys_process_madvise # # x32-specific system call numbers start at 512 to avoid cache impactarch/x86/entry/syscalls/syscall_64.tbl line 411
Quoted 4 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -407,3 +408,4 @@ 545 x32 execveat compat_sys_execveat 546 x32 preadv2 compat_sys_preadv64v2 547 x32 pwritev2 compat_sys_pwritev64v2 +548 x32 process_madvise compat_sys_process_madviseI think we should not add any new x32-specific syscalls. Instead I think the compat_sys_process_madvise/sys_process_madvise can be merged into one.
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 5 lines
arch/x86/entry/syscalls/syscall_64.tbl +5
-0+ mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS); + if (IS_ERR_OR_NULL(mm)) { + ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH; + goto release_task; + }Minor point: Having to use
IS_ERR_OR_NULL()tends to be fragile, and I would try to avoid that. Canmm_access()be changed to itself returnPTR_ERR(-ESRCH)instead of NULL to improve its calling conventions? I see there are only three other callers.Quoted 9 lines
arch/x86/entry/syscalls/syscall_64.tbl +9
-0+ ret = import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + } + return ret; +} + +#ifdef CONFIG_COMPAT...
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 7 lines
arch/x86/entry/syscalls/syscall_64.tbl +7
-0+ + ret = compat_import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), + &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + }Every syscall that passes an iovec seems to do this. If we make
import_iovec()handle both cases directly, this syscall and a number of others can be simplified, and you avoid the x32 entry point I mentioned aboveSomething like (untested)
Quoted 12 lines
lib/iov_iter.c ssize_t import_iovec(int type, const struct · line 1688
Quoted 12 lines
index
dad8d0cfaaf7..0de4ddff24c1100644lib/iov_iter.c +2
-2@@ -1683,8 +1683,13 @@ ssize_t import_iovec(int type, const structiovec __user * uvector, { ssize_t n;struct iovec *p; - n =rw_copy_check_uvector (type, uvector, nr_segs, fast_segs, - *iov, &p); + +if (in_compat_syscall ())
I suggested the exact same solutions roughly 1.5 weeks ago. :) Fun when I saw you mentioning this in BBB I knew exactly what you were referring too. :)
Christian
Re: [PATCH v8 3/4] mm/madvise: introduce process_madvise() syscall: an external memory hinting API
Quoted 70 lines · Christian Brauner
Quoted 69 lines · Jens Axboe
Quoted 67 lines · Arnd Bergmann
Quoted 4 lines · Minchan Kim
So finally, the API is as follows,
ssize_tprocess_madvise(int pidfd, const struct iovec *iovec,
unsigned long vlen, int advice, unsigned int flags);I had not followed the discussion earlier and only now came across the syscall in linux-next, sorry for stirring things up this late.
Quoted 13 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -364,6 +364,7 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 64 process_madvise sys_process_madvise # # x32-specific system call numbers start at 512 to avoid cache impactarch/x86/entry/syscalls/syscall_64.tbl line 411
Quoted 4 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -407,3 +408,4 @@ 545 x32 execveat compat_sys_execveat 546 x32 preadv2 compat_sys_preadv64v2 547 x32 pwritev2 compat_sys_pwritev64v2 +548 x32 process_madvise compat_sys_process_madviseI think we should not add any new x32-specific syscalls. Instead I think the compat_sys_process_madvise/sys_process_madvise can be merged into one.
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 5 lines
arch/x86/entry/syscalls/syscall_64.tbl +5
-0+ mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS); + if (IS_ERR_OR_NULL(mm)) { + ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH; + goto release_task; + }Minor point: Having to use
IS_ERR_OR_NULL()tends to be fragile, and I would try to avoid that. Canmm_access()be changed to itself returnPTR_ERR(-ESRCH)instead of NULL to improve its calling conventions? I see there are only three other callers.Quoted 9 lines
arch/x86/entry/syscalls/syscall_64.tbl +9
-0+ ret = import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + } + return ret; +} + +#ifdef CONFIG_COMPAT...
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 7 lines
arch/x86/entry/syscalls/syscall_64.tbl +7
-0+ + ret = compat_import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), + &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + }Every syscall that passes an iovec seems to do this. If we make
import_iovec()handle both cases directly, this syscall and a number of others can be simplified, and you avoid the x32 entry point I mentioned aboveSomething like (untested)
Quoted 17 lines
Quoted 4 lines
Quoted 4 lines
index
dad8d0cfaaf7..0de4ddff24c1100644lib/iov_iter.c +0
-0@@ -1683,8 +1683,13 @@ ssize_t import_iovec(int type, const structQuoted 8 lines
lib/iov_iter.c ssize_t import_iovec(int type, const struct · line 1688
Quoted 8 lines
lib/iov_iter.c +2
-2iovec __user * uvector, { ssize_t n;struct iovec *p; - n =rw_copy_check_uvector (type, uvector, nr_segs, fast_segs, - *iov, &p); + +if (in_compat_syscall ())I suggested the exact same solutions roughly 1.5 weeks ago. :) Fun when I saw you mentioning this in BBB I knew exactly what you were referring too. :)
https://lore.kernel.org/linux-man/20200816081227.ngw3l45c5uncesmr@wittgenstein/
Yes, Christian suggested the idea but mostly for only this new syscall. I don't have the time to revise the patchset yet but may have next week. I will follow Christian's suggestion.
Thanks.
Re: [PATCH v8 3/4] mm/madvise: introduce process_madvise() syscall: an external memory hinting API
In reply to Jens AxboeQuoted 84 lines · Jens Axboe
Quoted 83 lines · Arnd Bergmann
Quoted 4 lines · Minchan Kim
So finally, the API is as follows,
ssize_tprocess_madvise(int pidfd, const struct iovec *iovec,
unsigned long vlen, int advice, unsigned int flags);I had not followed the discussion earlier and only now came across the syscall in linux-next, sorry for stirring things up this late.
Quoted 13 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -364,6 +364,7 @@ 440 common watch_mount sys_watch_mount 441 common watch_sb sys_watch_sb 442 common fsinfo sys_fsinfo +443 64 process_madvise sys_process_madvise # # x32-specific system call numbers start at 512 to avoid cache impactarch/x86/entry/syscalls/syscall_64.tbl line 411
Quoted 4 lines
arch/x86/entry/syscalls/syscall_64.tbl +1
-0@@ -407,3 +408,4 @@ 545 x32 execveat compat_sys_execveat 546 x32 preadv2 compat_sys_preadv64v2 547 x32 pwritev2 compat_sys_pwritev64v2 +548 x32 process_madvise compat_sys_process_madviseI think we should not add any new x32-specific syscalls. Instead I think the compat_sys_process_madvise/sys_process_madvise can be merged into one.
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 5 lines
arch/x86/entry/syscalls/syscall_64.tbl +5
-0+ mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS); + if (IS_ERR_OR_NULL(mm)) { + ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH; + goto release_task; + }Minor point: Having to use
IS_ERR_OR_NULL()tends to be fragile, and I would try to avoid that. Canmm_access()be changed to itself returnPTR_ERR(-ESRCH)instead of NULL to improve its calling conventions? I see there are only three other callers.Quoted 9 lines
arch/x86/entry/syscalls/syscall_64.tbl +9
-0+ ret = import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + } + return ret; +} + +#ifdef CONFIG_COMPAT...
arch/x86/entry/syscalls/syscall_64.tbl
Quoted 7 lines
arch/x86/entry/syscalls/syscall_64.tbl +7
-0+ + ret = compat_import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), + &iov, &iter); + if (ret >= 0) { + ret = do_process_madvise(pidfd, &iter, behavior, flags); + kfree(iov); + }Every syscall that passes an iovec seems to do this. If we make
import_iovec()handle both cases directly, this syscall and a number of others can be simplified, and you avoid the x32 entry point I mentioned aboveSomething like (untested)
index
dad8d0cfaaf7..0de4ddff24c1100644lib/iov_iter.c +6
-2@@ -1683,8 +1683,13 @@ ssize_t import_iovec(int type, const structiovec __user * uvector, { ssize_t n;struct iovec *p; - n =rw_copy_check_uvector (type, uvector, nr_segs, fast_segs, - *iov, &p); + +if (in_compat_syscall ()) + n =compat_rw_copy_check_uvector (type, uvector, nr_segs, + fast_segs, *iov, &p); + else + n = rw_copy_check_uvector(type, uvector, nr_segs,
Quoted 8 lines
Quoted 4 lines
lib/iov_iter.c +1
-0+ fast_segs, *iov, &p);if (n <0 ) {if (p != *iov)kfree(p);Doesn't work for the async case, where you want to be holding on to the allocated iovec. But in general I think it's a good helper for the sync case, which is by far the majority.
Nevermind, I'm an idiot for reading this totally wrong.
Signature · 1 line
Jens Axboe
Arnd Bergmann
maintainerRe: [PATCH v8 3/4] mm/madvise: introduce process_madvise() syscall: an external memory hinting API
Quoted 24 lines · Jens Axboe
Quoted 13 lines · Jens Axboe
Quoted 13 lines
(quoted diff) +6
-2@@ -1683,8 +1683,13 @@ ssize_t import_iovec(int type, const struct iovec __user * uvector, { ssize_t n; struct iovec *p; - n = rw_copy_check_uvector(type, uvector, nr_segs, fast_segs, - *iov, &p); + + if (in_compat_syscall()) + n = compat_rw_copy_check_uvector(type, uvector, nr_segs, + fast_segs, *iov, &p); + else + n = rw_copy_check_uvector(type, uvector, nr_segs,Quoted 8 lines
Quoted 4 lines
(quoted diff) +1
-0+ fast_segs, *iov, &p); if (n < 0) { if (p != *iov)kfree(p);Doesn't work for the async case, where you want to be holding on to the allocated iovec. But in general I think it's a good helper for the sync case, which is by far the majority.
Quoted 2 lines
Nevermind, I'm an idiot for reading this totally wrong.
I think you are right about the need to pick the compat vs native behavior based on req->ctx->compat instead of in_compat_syscall() inside of io_import_iovec().
That one can probably call a lower-level version and when all other callers get changed to calling
ssize_t import_iovec(int type, const struct iovec __user * uvector,
unsigned nr_segs, unsigned fast_segs, struct iovec **iov, struct iov_iter *i)
{
return __import_iovec(type, uvector, nr_segs, fast_segs, iov,
i, in_compat_syscall());
}
Arnd
Re: [PATCH v8 3/4] mm/madvise: introduce process_madvise() syscall: an external memory hinting API
In reply to Arnd BergmannQuoted 3 lines · Arnd Bergmann
Every syscall that passes an iovec seems to do this. If we make
import_iovec()handle both cases directly, this syscall and a number of others can be simplified, and you avoid the x32 entry point I mentioned above
FYI, I do have a series that does this (even tested) and kills tons of compat syscalls by that. But by doing that I found the problem that compat syscalls issued by io_uring don't trigger in_compat_syscall(). I need to go back to fixing the io_uring vs in_compat_syscall() issue (probably for 5.9) and then submit the whole thing.