Skip to contentSkip to conversation

mm/ksm, proc: introduce remote madvise

RFC PATCH In review Oleksandr Natalenko 5 participants 15 messages May 16, 2019

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

cover mm/ksm, proc: introduce remote madvise

It all began with the fact that KSM works only on memory that is marked by madvise(). And the only way to get around that is to either:

* use LD_PRELOAD; or
* patch the kernel with something like UKSM or PKSM.

(i skip ptrace can of worms here intentionally)

To overcome this restriction, lets implement a per-process /proc knob, which allows calling madvise remotely. This can be used manually on a task in question or by some small userspace helper daemon that will do auto-KSM job for us.

Also, following the discussions from the previous submissions [2] and [3], make the interface more generic, so that it can be used for other madvise hints in the future. At this point, I'd like Android people to speak up, for instance, and clarify in which form they need page granularity or other things I've missed or have never heard about.

So, I think of three major consumers of this interface:

* hosts, that run containers, especially similar ones and especially in

    a trusted environment, sharing the same runtime like Node.js;

* heavy applications, that can be run in multiple instances, not limited to opensource ones like Firefox, but also those that cannot be modified since they are binary-only and, maybe, statically linked;

* Android environment that wants to do tricks with MADV_WILLNEED/DONTNEED or something similar.

On to the actual implementation. The per-process knob is named "madvise", and it is write-only. It accepts a madvise hint name to be executed. Currently, only KSM hints are implemented:

* to mark all the eligible VMAs as mergeable, use:

   # echo merge > /proc/<pid>/madvise

* to unmerge all the VMAs, use:

   # echo unmerge > /proc/<pid>/madvise

I've implemented address space level granularity instead of VMA/page granularity intentionally for simplicity. If the discussion goes in other directions, this can be re-implemented to act on a specific VMA (via map_files?) or page-wise.

Speaking of statistics, more numbers can be found in the very first submission, that is related to this one [1]. For my current setup with two Firefox instances I get 100 to 200 MiB saved for the second instance depending on the amount of tabs.

1 FF instance with 15 tabs:

   $ echo "$(cat /sys/kernel/mm/ksm/pages_sharing) * 4 / 1024" | bc
   410

2 FF instances, second one has 12 tabs (all the tabs are different):

   $ echo "$(cat /sys/kernel/mm/ksm/pages_sharing) * 4 / 1024" | bc
   592

At the very moment I do not have specific numbers for containerised workload, but those should be comparable in case the containers share similar/same runtime.

The history of this patchset:

* [2] was based on Timofey's submission [1], but it didn't use a dedicated kthread to walk through the list of tasks/VMAs. Instead, do_anonymous_page() was amended to implement fully automatic mode, but this approach was incorrect due to improper locking and not desired due to excessive complexity and being KSM-specific;
* [3] implemented KSM-specific madvise hints via sysfs, leaving traversing /proc to userspace if needed. The approach was not desired due to the fact that sysfs shouldn't implement any per-process API. Also, the interface was not generic enough to extend it for other users.

I drop all the "Reviewed-by" tags from previous submissions because of code changes and because the objective of this series is now somewhat different.

Please comment!

Thanks.

[1] https://lore.kernel.org/patchwork/patch/1012142/
[2] http://lkml.iu.edu/hypermail/linux/kernel/1905.1/02417.html
[3] http://lkml.iu.edu/hypermail/linux/kernel/1905.1/05076.html

Oleksandr Natalenko (5):
proc: introduce madvise placeholder
mm/ksm: introduce ksm_madvise_merge() helper
mm/ksm: introduce ksm_madvise_unmerge() helper
mm/ksm, proc: introduce remote merge
mm/ksm, proc: add remote madvise documentation

 Documentation/filesystems/proc.txt | 13 +++++
 fs/proc/base.c                     | 70 +++++++++++++++++++++++
 include/linux/ksm.h                |  4 ++
 mm/ksm.c                           | 92 +++++++++++++++++++-----------
 4 files changed, 145 insertions(+), 34 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

RFC PATCH 1/5 proc: introduce madvise placeholder

Add a write-only /proc/<pid>/madvise file to handle remote madvise operations.

For now, this is just a soother that does nothing.

---
 fs/proc/base.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
fs/proc/base.c +20 -0
index 6a803a0b75df..f69532d6b74f 100644
@@ -2957,6 +2957,25 @@ static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns,
 }
 #endif /* CONFIG_STACKLEAK_METRICS */
 
+static ssize_t madvise_write(struct file *file, const char __user *buf,
+		size_t count, loff_t *ppos)
+{
+	struct task_struct *task;
+
+	task = get_proc_task(file_inode(file));
+	if (!task)
+		return -ESRCH;
+
+	put_task_struct(task);
+
+	return count;
+}
+
+static const struct file_operations proc_madvise_operations = {
+	.write		= madvise_write,
+	.llseek		= noop_llseek,
+};
+
 /*
  * Thread groups
  */
@@ -3061,6 +3080,7 @@ static const struct pid_entry tgid_base_stuff[] = {
 #ifdef CONFIG_STACKLEAK_METRICS
 	ONE("stack_depth", S_IRUGO, proc_stack_depth),
 #endif
+	REG("madvise", S_IRUGO|S_IWUSR, proc_madvise_operations),
 };
 
 static int proc_tgid_base_readdir(struct file *file, struct dir_context *ctx)
Signature · 1 line
2.21.0
#2 lore raw

RFC PATCH 2/5 mm/ksm: introduce ksm_madvise_merge() helper

In reply to Oleksandr Natalenko

Move MADV_MERGEABLE part of ksm_madvise() into a dedicated helper since it will be further used for marking VMAs to be merged forcibly.

This does not bring any functional changes.

---
 include/linux/ksm.h |  2 ++
 mm/ksm.c            | 60 +++++++++++++++++++++++++++------------------
 2 files changed, 38 insertions(+), 24 deletions(-)
include/linux/ksm.h +2 -0
index e48b1e453ff5..e824b3141677 100644
@@ -19,6 +19,8 @@ struct stable_node;
 struct mem_cgroup;
 
 #ifdef CONFIG_KSM
+int ksm_madvise_merge(struct mm_struct *mm, struct vm_area_struct *vma,
+		unsigned long *vm_flags);
 int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
 		unsigned long end, int advice, unsigned long *vm_flags);
 int __ksm_enter(struct mm_struct *mm);
mm/ksm.c +36 -24

77 lines not on this page · raw message

Signature · 1 line
2.21.0
#3 lore raw

RFC PATCH 3/5 mm/ksm: introduce ksm_madvise_unmerge() helper

In reply to Oleksandr Natalenko

Move MADV_UNMERGEABLE part of ksm_madvise() into a dedicated helper since it will be further used for unmerging VMAs forcibly.

This does not bring any functional changes.

---
 include/linux/ksm.h |  2 ++
 mm/ksm.c            | 32 ++++++++++++++++++++++----------
 2 files changed, 24 insertions(+), 10 deletions(-)
include/linux/ksm.h +2 -0
index e824b3141677..a91a7cfc87a1 100644
@@ -21,6 +21,8 @@ struct mem_cgroup;
 #ifdef CONFIG_KSM
 int ksm_madvise_merge(struct mm_struct *mm, struct vm_area_struct *vma,
 		unsigned long *vm_flags);
+int ksm_madvise_unmerge(struct vm_area_struct *vma, unsigned long start,
+		unsigned long end, unsigned long *vm_flags);
 int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
 		unsigned long end, int advice, unsigned long *vm_flags);
 int __ksm_enter(struct mm_struct *mm);
mm/ksm.c +22 -10
index 1fdcf2fbd58d..e0357e25e09f 100644
@@ -2478,6 +2478,25 @@ int ksm_madvise_merge(struct mm_struct *mm, struct vm_area_struct *vma,
 	return 0;
 }
 
+int ksm_madvise_unmerge(struct vm_area_struct *vma, unsigned long start,
+		unsigned long end, unsigned long *vm_flags)
+{
+	int err;
+
+	if (!(*vm_flags & VM_MERGEABLE))
+		return 0;		/* just ignore the advice */
+
+	if (vma->anon_vma) {
+		err = unmerge_ksm_pages(vma, start, end);
+		if (err)
+			return err;
+	}
+
+	*vm_flags &= ~VM_MERGEABLE;
+
+	return 0;
+}
+
 int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
 		unsigned long end, int advice, unsigned long *vm_flags)
 {
@@ -2492,16 +2511,9 @@ int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
 		break;
 
 	case MADV_UNMERGEABLE:
-		if (!(*vm_flags & VM_MERGEABLE))
-			return 0;		/* just ignore the advice */
-
-		if (vma->anon_vma) {
-			err = unmerge_ksm_pages(vma, start, end);
-			if (err)
-				return err;
-		}
-
-		*vm_flags &= ~VM_MERGEABLE;
+		err = ksm_madvise_unmerge(vma, start, end, vm_flags);
+		if (err)
+			return err;
 		break;
 	}
 
Signature · 1 line
2.21.0
#4 lore raw

RFC PATCH 4/5 mm/ksm, proc: introduce remote merge

In reply to Oleksandr Natalenko

Use previously introduced remote madvise knob to mark task's anonymous memory as mergeable.

To force merging task's VMAs, "merge" hint is used:

   # echo merge > /proc/<pid>/madvise

Force unmerging is done similarly:

   # echo unmerge > /proc/<pid>/madvise

To achieve this, previously introduced ksm_madvise_*() helpers are used.

---
 fs/proc/base.c | 52 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 51 insertions(+), 1 deletion(-)
fs/proc/base.c +51 -1
index f69532d6b74f..6677580080ed 100644
@@ -94,6 +94,8 @@
 #include <linux/sched/debug.h>
 #include <linux/sched/stat.h>
 #include <linux/posix-timers.h>
+#include <linux/mman.h>
+#include <linux/ksm.h>
 #include <trace/events/oom.h>
 #include "internal.h"
 #include "fd.h"
@@ -2960,15 +2962,63 @@ static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns,
 static ssize_t madvise_write(struct file *file, const char __user *buf,
 		size_t count, loff_t *ppos)
 {
+	/* For now, only KSM hints are implemented */
+#ifdef CONFIG_KSM
+	char buffer[PROC_NUMBUF];
+	int behaviour;
 	struct task_struct *task;
+	struct mm_struct *mm;
+	int err = 0;
+	struct vm_area_struct *vma;
+
+	memset(buffer, 0, sizeof(buffer));
+	if (count > sizeof(buffer) - 1)
+		count = sizeof(buffer) - 1;
+	if (copy_from_user(buffer, buf, count))
+		return -EFAULT;
+
+	if (!memcmp("merge", buffer, min(sizeof("merge")-1, count)))
+		behaviour = MADV_MERGEABLE;
+	else if (!memcmp("unmerge", buffer, min(sizeof("unmerge")-1, count)))
+		behaviour = MADV_UNMERGEABLE;
+	else
+		return -EINVAL;
 
 	task = get_proc_task(file_inode(file));
 	if (!task)
 		return -ESRCH;
 
+	mm = get_task_mm(task);
+	if (!mm) {
+		err = -EINVAL;
+		goto out_put_task_struct;
+	}
+
+	down_write(&mm->mmap_sem);
+	switch (behaviour) {
+	case MADV_MERGEABLE:
+	case MADV_UNMERGEABLE:
+		vma = mm->mmap;
+		while (vma) {
+			if (behaviour == MADV_MERGEABLE)
+				ksm_madvise_merge(vma->vm_mm, vma, &vma->vm_flags);
+			else
+				ksm_madvise_unmerge(vma, vma->vm_start, vma->vm_end, &vma->vm_flags);
+			vma = vma->vm_next;
+		}
+		break;
+	}
+	up_write(&mm->mmap_sem);
+
+	mmput(mm);
+
+out_put_task_struct:
 	put_task_struct(task);
 
-	return count;
+	return err ? err : count;
+#else
+	return -EINVAL;
+#endif /* CONFIG_KSM */
 }
 
 static const struct file_operations proc_madvise_operations = {
Signature · 1 line
2.21.0
#5 lore raw

Re: [PATCH RFC 4/5] mm/ksm, proc: introduce remote merge

Quoted 5 lines · Oleksandr Natalenko

Use previously introduced remote madvise knob to mark task's anonymous memory as mergeable.

To force merging task's VMAs, "merge" hint is used:

Quoted 8 lines
   # echo merge > /proc/<pid>/madvise

Force unmerging is done similarly:

   # echo unmerge > /proc/<pid>/madvise

To achieve this, previously introduced ksm_madvise_*() helpers are used.

Why does this not require PTRACE_MODE_ATTACH_FSCREDS to the target process? Enabling KSM on another process is hazardous because it significantly increases the attack surface for side channels.

(Note that if you change this to require PTRACE_MODE_ATTACH_FSCREDS, you'll want to use mm_access() in the ->open handler and drop the mm in ->release. mm_access() from a ->write handler is not permitted.)

[...]

Quoted 10 lines
fs/proc/base.c +5 -0
@@ -2960,15 +2962,63 @@ static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns,
 static ssize_t madvise_write(struct file *file, const char __user *buf,
                size_t count, loff_t *ppos)
 {
+       /* For now, only KSM hints are implemented */
+#ifdef CONFIG_KSM
+       char buffer[PROC_NUMBUF];
+       int behaviour;
        struct task_struct *task;
+       struct mm_struct *mm;

fs/proc/base.c static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns, · line 2980

Quoted 10 lines
fs/proc/base.c +10 -0
+       int err = 0;
+       struct vm_area_struct *vma;
+
+       memset(buffer, 0, sizeof(buffer));
+       if (count > sizeof(buffer) - 1)
+               count = sizeof(buffer) - 1;
+       if (copy_from_user(buffer, buf, count))
+               return -EFAULT;
+
+       if (!memcmp("merge", buffer, min(sizeof("merge")-1, count)))

This means that you also match on something like "mergeblah". Just use strcmp().

Quoted 7 lines
fs/proc/base.c +5 -0
+               behaviour = MADV_MERGEABLE;
+       else if (!memcmp("unmerge", buffer, min(sizeof("unmerge")-1, count)))
+               behaviour = MADV_UNMERGEABLE;
+       else
+               return -EINVAL;
 
        task = get_proc_task(file_inode(file));

fs/proc/base.c static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns, · line 2997

Quoted 10 lines
fs/proc/base.c +7 -0
        if (!task)
                return -ESRCH;
 
+       mm = get_task_mm(task);
+       if (!mm) {
+               err = -EINVAL;
+               goto out_put_task_struct;
+       }
+
+       down_write(&mm->mmap_sem);

Should a check for mmget_still_valid(mm) be inserted here? See commit 04f5866e41fb70690e28397487d8bd8eea7d712a.

fs/proc/base.c static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns, · line 3000

Quoted 3 lines
fs/proc/base.c +3 -0
+       switch (behaviour) {
+       case MADV_MERGEABLE:
+       case MADV_UNMERGEABLE:

This switch isn't actually necessary at this point, right?

Quoted 23 lines
fs/proc/base.c +19 -1
+               vma = mm->mmap;
+               while (vma) {
+                       if (behaviour == MADV_MERGEABLE)
+                               ksm_madvise_merge(vma->vm_mm, vma, &vma->vm_flags);
+                       else
+                               ksm_madvise_unmerge(vma, vma->vm_start, vma->vm_end, &vma->vm_flags);
+                       vma = vma->vm_next;
+               }
+               break;
+       }
+       up_write(&mm->mmap_sem);
+
+       mmput(mm);
+
+out_put_task_struct:
        put_task_struct(task);
 
-       return count;
+       return err ? err : count;
+#else
+       return -EINVAL;
+#endif /* CONFIG_KSM */
 }
#6 lore raw

Re: [PATCH RFC 4/5] mm/ksm, proc: introduce remote merge

Hi.

Quoted 16 lines · Jann Horn
Quoted 5 lines · Oleksandr Natalenko

Use previously introduced remote madvise knob to mark task's anonymous memory as mergeable.

To force merging task's VMAs, "merge" hint is used:

Quoted 8 lines
   # echo merge > /proc/<pid>/madvise

Force unmerging is done similarly:

   # echo unmerge > /proc/<pid>/madvise

To achieve this, previously introduced ksm_madvise_*() helpers are used.

Quoted 7 lines

Why does this not require PTRACE_MODE_ATTACH_FSCREDS to the target process? Enabling KSM on another process is hazardous because it significantly increases the attack surface for side channels.

(Note that if you change this to require PTRACE_MODE_ATTACH_FSCREDS, you'll want to use mm_access() in the ->open handler and drop the mm in ->release. mm_access() from a ->write handler is not permitted.)

Sounds reasonable. So, something similar to what mem_open() & friends do now:

static int madvise_open(...)
...

	struct task_struct *task = get_proc_task(inode);
...
	if (task) {
		mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS);
		put_task_struct(task);
		if (!IS_ERR_OR_NULL(mm)) {
			mmgrab(mm);
			mmput(mm);

...

Then:

static ssize_t madvise_write(...)
...

	if (!mmget_not_zero(mm))
		goto out;

	down_write(&mm->mmap_sem);
	if (!mmget_still_valid(mm))
		goto skip_mm;
...
skip_mm:
	up_write(&mm->mmap_sem);

	mmput(mm);
out:
	return ...;

And, finally:

static int madvise_release(...)
...

		mmdrop(mm);

...

Right?

Quoted 15 lines

[...]

Quoted 14 lines
fs/proc/base.c +9 -0
@@ -2960,15 +2962,63 @@ static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns,
 static ssize_t madvise_write(struct file *file, const char __user *buf,
                size_t count, loff_t *ppos)
 {
+       /* For now, only KSM hints are implemented */
+#ifdef CONFIG_KSM
+       char buffer[PROC_NUMBUF];
+       int behaviour;
        struct task_struct *task;
+       struct mm_struct *mm;
+       int err = 0;
+       struct vm_area_struct *vma;
+
+       memset(buffer, 0, sizeof(buffer));
Quoted 8 lines

fs/proc/base.c static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns, · line 2980

Quoted 6 lines
fs/proc/base.c +6 -0
+       if (count > sizeof(buffer) - 1)
+               count = sizeof(buffer) - 1;
+       if (copy_from_user(buffer, buf, count))
+               return -EFAULT;
+
+       if (!memcmp("merge", buffer, min(sizeof("merge")-1, count)))

This means that you also match on something like "mergeblah". Just use strcmp().

I agree. Just to make it more interesting I must say that

/sys/kernel/mm/transparent_hugepage/enabled

uses memcmp in the very same way, and thus echoing "alwaysssss" or "madviseeee" works perfectly there, and it was like that from the very beginning, it seems. Should we fix it, or it became (zomg) a public API?

Quoted 18 lines
Quoted 7 lines
fs/proc/base.c +5 -0
+               behaviour = MADV_MERGEABLE;
+       else if (!memcmp("unmerge", buffer, min(sizeof("unmerge")-1, count)))
+               behaviour = MADV_UNMERGEABLE;
+       else
+               return -EINVAL;
 
        task = get_proc_task(file_inode(file));

fs/proc/base.c static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns, · line 2997

Quoted 10 lines
fs/proc/base.c +7 -0
        if (!task)
                return -ESRCH;
 
+       mm = get_task_mm(task);
+       if (!mm) {
+               err = -EINVAL;
+               goto out_put_task_struct;
+       }
+
+       down_write(&mm->mmap_sem);
Quoted 2 lines

Should a check for mmget_still_valid(mm) be inserted here? See commit 04f5866e41fb70690e28397487d8bd8eea7d712a.

Yeah, it seems so :/. Thanks for the pointer. I've put it into the madvise_write snippet above.

Quoted 5 lines

fs/proc/base.c static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns, · line 3000

Quoted 3 lines
fs/proc/base.c +3 -0
+       switch (behaviour) {
+       case MADV_MERGEABLE:
+       case MADV_UNMERGEABLE:

This switch isn't actually necessary at this point, right?

Yup, but it is there to highlight a possibility of adding other, non-KSM options. So, let it be, and I'll just re-arrange CONFIG_KSM ifdef instead.

Thank you.

Quoted 23 lines
Quoted 23 lines
fs/proc/base.c +19 -1
+               vma = mm->mmap;
+               while (vma) {
+                       if (behaviour == MADV_MERGEABLE)
+                               ksm_madvise_merge(vma->vm_mm, vma, &vma->vm_flags);
+                       else
+                               ksm_madvise_unmerge(vma, vma->vm_start, vma->vm_end, &vma->vm_flags);
+                       vma = vma->vm_next;
+               }
+               break;
+       }
+       up_write(&mm->mmap_sem);
+
+       mmput(mm);
+
+out_put_task_struct:
        put_task_struct(task);
 
-       return count;
+       return err ? err : count;
+#else
+       return -EINVAL;
+#endif /* CONFIG_KSM */
 }
Signature · 3 lines
  Best regards,
    Oleksandr Natalenko (post-factum)
    Senior Software Maintenance Engineer
#7 lore raw

Re: [PATCH RFC 4/5] mm/ksm, proc: introduce remote merge

Quoted 24 lines · Oleksandr Natalenko
Quoted 15 lines

[...]

Quoted 14 lines
fs/proc/base.c +9 -0
@@ -2960,15 +2962,63 @@ static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns,
 static ssize_t madvise_write(struct file *file, const char __user *buf,
                size_t count, loff_t *ppos)
 {
+       /* For now, only KSM hints are implemented */
+#ifdef CONFIG_KSM
+       char buffer[PROC_NUMBUF];
+       int behaviour;
        struct task_struct *task;
+       struct mm_struct *mm;
+       int err = 0;
+       struct vm_area_struct *vma;
+
+       memset(buffer, 0, sizeof(buffer));
Quoted 8 lines

fs/proc/base.c static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns, · line 2980

Quoted 6 lines
fs/proc/base.c +6 -0
+       if (count > sizeof(buffer) - 1)
+               count = sizeof(buffer) - 1;
+       if (copy_from_user(buffer, buf, count))
+               return -EFAULT;
+
+       if (!memcmp("merge", buffer, min(sizeof("merge")-1, count)))

This means that you also match on something like "mergeblah". Just use strcmp().

Quoted 7 lines

I agree. Just to make it more interesting I must say that

/sys/kernel/mm/transparent_hugepage/enabled

uses memcmp in the very same way, and thus echoing "alwaysssss" or "madviseeee" works perfectly there, and it was like that from the very beginning, it seems. Should we fix it, or it became (zomg) a public API?

Actually, maybe, the reason for using memcmp is to handle "echo"
properly: by default it puts a newline character at the end, so if we use just strcmp, echo should be called with -n, otherwise strcmp won't match the string.

Huh?

Quoted 1 line

[...]

Signature · 3 lines
  Best regards,
    Oleksandr Natalenko (post-factum)
    Senior Software Maintenance Engineer
#8 lore raw

Re: [PATCH RFC 4/5] mm/ksm, proc: introduce remote merge

Quoted 33 lines · Oleksandr Natalenko
Quoted 24 lines · Oleksandr Natalenko
Quoted 15 lines

[...]

Quoted 14 lines
fs/proc/base.c +9 -0
@@ -2960,15 +2962,63 @@ static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns,
 static ssize_t madvise_write(struct file *file, const char __user *buf,
                size_t count, loff_t *ppos)
 {
+       /* For now, only KSM hints are implemented */
+#ifdef CONFIG_KSM
+       char buffer[PROC_NUMBUF];
+       int behaviour;
        struct task_struct *task;
+       struct mm_struct *mm;
+       int err = 0;
+       struct vm_area_struct *vma;
+
+       memset(buffer, 0, sizeof(buffer));
Quoted 8 lines

fs/proc/base.c static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns, · line 2980

Quoted 6 lines
fs/proc/base.c +6 -0
+       if (count > sizeof(buffer) - 1)
+               count = sizeof(buffer) - 1;
+       if (copy_from_user(buffer, buf, count))
+               return -EFAULT;
+
+       if (!memcmp("merge", buffer, min(sizeof("merge")-1, count)))

This means that you also match on something like "mergeblah". Just use strcmp().

Quoted 7 lines

I agree. Just to make it more interesting I must say that

/sys/kernel/mm/transparent_hugepage/enabled

uses memcmp in the very same way, and thus echoing "alwaysssss" or "madviseeee" works perfectly there, and it was like that from the very beginning, it seems. Should we fix it, or it became (zomg) a public API?

Quoted 6 lines

Actually, maybe, the reason for using memcmp is to handle "echo"
properly: by default it puts a newline character at the end, so if we use just strcmp, echo should be called with -n, otherwise strcmp won't match the string.

Huh?

Ah, yes, other code like e.g. proc_setgroups_write() uses strncmp() and then has an extra check to make sure everything trailing is whitespace.

#9 lore raw

Re: [PATCH RFC 4/5] mm/ksm, proc: introduce remote merge

In reply to Oleksandr Natalenko
Quoted 58 lines · Oleksandr Natalenko
Quoted 16 lines · Jann Horn
Quoted 5 lines · Oleksandr Natalenko

Use previously introduced remote madvise knob to mark task's anonymous memory as mergeable.

To force merging task's VMAs, "merge" hint is used:

Quoted 8 lines
   # echo merge > /proc/<pid>/madvise

Force unmerging is done similarly:

   # echo unmerge > /proc/<pid>/madvise

To achieve this, previously introduced ksm_madvise_*() helpers are used.

Quoted 7 lines

Why does this not require PTRACE_MODE_ATTACH_FSCREDS to the target process? Enabling KSM on another process is hazardous because it significantly increases the attack surface for side channels.

(Note that if you change this to require PTRACE_MODE_ATTACH_FSCREDS, you'll want to use mm_access() in the ->open handler and drop the mm in ->release. mm_access() from a ->write handler is not permitted.)

Sounds reasonable. So, something similar to what mem_open() & friends do now:

static int madvise_open(...)
...

        struct task_struct *task = get_proc_task(inode);
...
        if (task) {
                mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS);
                put_task_struct(task);
                if (!IS_ERR_OR_NULL(mm)) {
                        mmgrab(mm);
                        mmput(mm);

...

Then:

static ssize_t madvise_write(...)
...

        if (!mmget_not_zero(mm))
                goto out;

        down_write(&mm->mmap_sem);
        if (!mmget_still_valid(mm))
                goto skip_mm;
...
skip_mm:
        up_write(&mm->mmap_sem);

        mmput(mm);
out:
        return ...;
Quoted 8 lines

And, finally:

static int madvise_release(...)
...

                mmdrop(mm);

...

Right?

Yeah, that looks reasonable.

#10 lore raw

Re: [PATCH RFC 4/5] mm/ksm, proc: introduce remote merge

In reply to Jann Horn
Quoted 2 lines · Jann Horn

On Thu, May 16, 2019 at 11:43 AM Oleksandr Natalenko <[email protected]> wrote:

[ ... ]

Quoted 6 lines

fs/proc/base.c static int proc_stack_depth(struct seq_file *m, struct pid_namespace *ns, · line 2997

Quoted 3 lines
fs/proc/base.c +3 -0
+       }
+
+       down_write(&mm->mmap_sem);

Should a check for mmget_still_valid(mm) be inserted here? See commit 04f5866e41fb70690e28397487d8bd8eea7d712a.

Yes - I'd say this is required here.

Thanks,

Signature · 1 line
Aaron Tomlin
#11 lore raw

RFC PATCH 5/5 mm/ksm, proc: add remote madvise documentation

In reply to Oleksandr Natalenko

Document respective /proc/<pid>/madvise knob.

---
 Documentation/filesystems/proc.txt | 13 +++++++++++++
 1 file changed, 13 insertions(+)
Documentation/filesystems/proc.txt +13 -0
index 66cad5c86171..17106e435bba 100644
@@ -45,6 +45,7 @@ Table of Contents
   3.9   /proc/<pid>/map_files - Information about memory mapped files
   3.10  /proc/<pid>/timerslack_ns - Task timerslack value
   3.11	/proc/<pid>/patch_state - Livepatch patch operation state
+  3.12  /proc/<pid>/madvise - Remote madvise
 
   4	Configuring procfs
   4.1	Mount options
@@ -1948,6 +1949,18 @@ patched.  If the patch is being enabled, then the task has already been
 patched.  If the patch is being disabled, then the task hasn't been
 unpatched yet.
 
+3.12    /proc/<pid>/madvise - Remote madvise
+--------------------------------------------
+This write-only file allows executing madvise operation for another task.
+
+If CONFIG_KSM is enabled, the following actions are available:
+
+  * marking task's memory as mergeable:
+    # echo merge > /proc/<pid>/madvise
+
+  * unmerging all the task's memory:
+    # echo unmerge > /proc/<pid>/madvise
+
 
 ------------------------------------------------------------------------------
 Configuring procfs
Signature · 1 line
2.21.0
#12 lore raw
In reply to Oleksandr Natalenko

On Thu 16-05-19 11:42:29, Oleksandr Natalenko wrote:
[...]

Quoted 7 lines

* to mark all the eligible VMAs as mergeable, use:

   # echo merge > /proc/<pid>/madvise

* to unmerge all the VMAs, use:

   # echo unmerge > /proc/<pid>/madvise

Please do not open a new thread until a previous one reaches some conclusion. I have outlined some ways to go forward in
http://lkml.kernel.org/r/[email protected].
I haven't heard any feedback on that, yet you open a 3rd way in a different thread. This will not help to move on with the discussion.

Please follow up on that thread.

Signature · 2 lines
Michal Hocko
SUSE Labs
#13 lore raw

Hi.

Quoted 10 lines · Michal Hocko

On Thu 16-05-19 11:42:29, Oleksandr Natalenko wrote:
[...]

Quoted 7 lines

* to mark all the eligible VMAs as mergeable, use:

   # echo merge > /proc/<pid>/madvise

* to unmerge all the VMAs, use:

   # echo unmerge > /proc/<pid>/madvise
Quoted 7 lines

Please do not open a new thread until a previous one reaches some conclusion. I have outlined some ways to go forward in
http://lkml.kernel.org/r/[email protected].
I haven't heard any feedback on that, yet you open a 3rd way in a different thread. This will not help to move on with the discussion.

Please follow up on that thread.

Sure, I will follow the thread once and if there are responses. Consider this one to be an intermediate summary of current suggestions and also an indication that it is better to have the code early for public eyes.

Thank you.

Quoted 3 lines
Signature · 2 lines
Michal Hocko
SUSE Labs
Signature · 3 lines
  Best regards,
    Oleksandr Natalenko (post-factum)
    Senior Software Maintenance Engineer
#14 lore raw
In reply to Oleksandr Natalenko
Quoted 7 lines · Oleksandr Natalenko

* to mark all the eligible VMAs as mergeable, use:

   # echo merge > /proc/<pid>/madvise

* to unmerge all the VMAs, use:

   # echo unmerge > /proc/<pid>/madvise

Please make a real system call (or abuse prctl(2) passing target's pid).

Your example automerge daemon could just call it and not bother with /proc.