Skip to contentSkip to conversation

mm/ksm: add option to automerge VMAs

RFC PATCH In review Oleksandr Natalenko 5 participants 21 messages May 14, 2019 – May 16, 2019

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v2 cover mm/ksm: add option to automerge VMAs

By default, KSM works only on memory that is marked by madvise(). And the only way to get around that is to either:

* use LD_PRELOAD; or
* patch the kernel with something like UKSM or PKSM.

Instead, lets implement a sysfs knob, which allows marking VMAs as mergeable. This can be used manually on some task in question or by some small userspace helper daemon.

The knob is named "force_madvise", and it is write-only. It accepts a PID to act on. To mark the VMAs as mergeable, use:

   # echo PID > /sys/kernel/mm/ksm/force_madvise

To unmerge all the VMAs, use the same approach, prepending the PID with the "minus" sign:

   # echo -PID > /sys/kernel/mm/ksm/force_madvise

This patchset is based on earlier Timofey's submission [1], but it doesn't use dedicated kthread to walk through the list of tasks/VMAs. Instead, it is up to userspace to traverse all the tasks in /proc if needed.

The previous suggestion [2] was based on amending do_anonymous_page() handler to implement fully automatic mode, but this approach was incorrect due to improper locking and not desired due to excessive complexity.

The current approach just implements minimal interface and leaves the decision on how and when to act to userspace.

Thanks.

[1] https://lore.kernel.org/patchwork/patch/1012142/
[2] http://lkml.iu.edu/hypermail/linux/kernel/1905.1/02417.html

Oleksandr Natalenko (4):
mm/ksm: introduce ksm_enter() helper mm/ksm: introduce ksm_leave() helper mm/ksm: introduce force_madvise knob mm/ksm: add force merging/unmerging documentation

 Documentation/admin-guide/mm/ksm.rst |  11 ++
 mm/ksm.c                             | 160 +++++++++++++++++++++------
 2 files changed, 137 insertions(+), 34 deletions(-)
Signature · 1 line
2.21.0
#1 lore raw

RFC PATCH v2 1/4 mm/ksm: introduce ksm_enter() helper

Move MADV_MERGEABLE part of ksm_madvise() into a dedicated helper since it will be further used for marking VMAs to be merged forcibly.

This does not bring any functional changes.

---
 mm/ksm.c | 60 +++++++++++++++++++++++++++++++++-----------------------
 1 file changed, 36 insertions(+), 24 deletions(-)
mm/ksm.c +36 -24
index fc64874dc6f4..02fdbee394cc 100644
@@ -2442,41 +2442,53 @@ static int ksm_scan_thread(void *nothing)
 	return 0;
 }
 
-int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
-		unsigned long end, int advice, unsigned long *vm_flags)
+static int ksm_enter(struct mm_struct *mm, struct vm_area_struct *vma,
+		unsigned long *vm_flags)
 {
-	struct mm_struct *mm = vma->vm_mm;
 	int err;
 
-	switch (advice) {
-	case MADV_MERGEABLE:
-		/*
-		 * Be somewhat over-protective for now!
-		 */
-		if (*vm_flags & (VM_MERGEABLE | VM_SHARED  | VM_MAYSHARE   |
-				 VM_PFNMAP    | VM_IO      | VM_DONTEXPAND |
-				 VM_HUGETLB | VM_MIXEDMAP))
-			return 0;		/* just ignore the advice */
+	/*
+	 * Be somewhat over-protective for now!
+	 */
+	if (*vm_flags & (VM_MERGEABLE | VM_SHARED  | VM_MAYSHARE   |
+			 VM_PFNMAP    | VM_IO      | VM_DONTEXPAND |
+			 VM_HUGETLB | VM_MIXEDMAP))
+		return 0;		/* just ignore the advice */
 
-		if (vma_is_dax(vma))
-			return 0;
+	if (vma_is_dax(vma))
+		return 0;
 
 #ifdef VM_SAO
-		if (*vm_flags & VM_SAO)
-			return 0;
+	if (*vm_flags & VM_SAO)
+		return 0;
 #endif
 #ifdef VM_SPARC_ADI
-		if (*vm_flags & VM_SPARC_ADI)
-			return 0;
+	if (*vm_flags & VM_SPARC_ADI)
+		return 0;
 #endif
 
-		if (!test_bit(MMF_VM_MERGEABLE, &mm->flags)) {
-			err = __ksm_enter(mm);
-			if (err)
-				return err;
-		}
+	if (!test_bit(MMF_VM_MERGEABLE, &mm->flags)) {
+		err = __ksm_enter(mm);
+		if (err)
+			return err;
+	}
+
+	*vm_flags |= VM_MERGEABLE;
+
+	return 0;
+}
 
-		*vm_flags |= VM_MERGEABLE;
+int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
+		unsigned long end, int advice, unsigned long *vm_flags)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	int err;
+
+	switch (advice) {
+	case MADV_MERGEABLE:
+		err = ksm_enter(mm, vma, vm_flags);
+		if (err)
+			return err;
 		break;
 
 	case MADV_UNMERGEABLE:
Signature · 1 line
2.21.0
#2 lore raw

RFC PATCH v2 2/4 mm/ksm: introduce ksm_leave() helper

In reply to Oleksandr Natalenko

Move MADV_UNMERGEABLE part of ksm_madvise() into a dedicated helper since it will be further used for unmerging VMAs forcibly.

This does not bring any functional changes.

---
 mm/ksm.c | 32 ++++++++++++++++++++++----------
 1 file changed, 22 insertions(+), 10 deletions(-)
mm/ksm.c +22 -10
index 02fdbee394cc..e9f3901168bb 100644
@@ -2478,6 +2478,25 @@ static int ksm_enter(struct mm_struct *mm, struct vm_area_struct *vma,
 	return 0;
 }
 
+static int ksm_leave(struct vm_area_struct *vma, unsigned long start,
+		unsigned long end, unsigned long *vm_flags)
+{
+	int err;
+
+	if (!(*vm_flags & VM_MERGEABLE))
+		return 0;		/* just ignore the advice */
+
+	if (vma->anon_vma) {
+		err = unmerge_ksm_pages(vma, start, end);
+		if (err)
+			return err;
+	}
+
+	*vm_flags &= ~VM_MERGEABLE;
+
+	return 0;
+}
+
 int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
 		unsigned long end, int advice, unsigned long *vm_flags)
 {
@@ -2492,16 +2511,9 @@ int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
 		break;
 
 	case MADV_UNMERGEABLE:
-		if (!(*vm_flags & VM_MERGEABLE))
-			return 0;		/* just ignore the advice */
-
-		if (vma->anon_vma) {
-			err = unmerge_ksm_pages(vma, start, end);
-			if (err)
-				return err;
-		}
-
-		*vm_flags &= ~VM_MERGEABLE;
+		err = ksm_leave(vma, start, end, vm_flags);
+		if (err)
+			return err;
 		break;
 	}
 
Signature · 1 line
2.21.0
#3 lore raw

RFC PATCH v2 3/4 mm/ksm: introduce force_madvise knob

In reply to Oleksandr Natalenko

Present a new sysfs knob to mark task's anonymous memory as mergeable.

To force merging task's VMAs, its PID is echoed in a write-only file:

   # echo PID > /sys/kernel/mm/ksm/force_madvise

Force unmerging is done similarly, but with "minus" sign:

   # echo -PID > /sys/kernel/mm/ksm/force_madvise

"0" or "-0" can be used to control the current task.

To achieve this, previously introduced ksm_enter()/ksm_leave() helpers are used in the "store" handler.

---
 mm/ksm.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)
mm/ksm.c +68 -0
index e9f3901168bb..22c59fb03d3a 100644
@@ -2879,10 +2879,77 @@ static void wait_while_offlining(void)
 
 #define KSM_ATTR_RO(_name) \
 	static struct kobj_attribute _name##_attr = __ATTR_RO(_name)
+#define KSM_ATTR_WO(_name) \
+	static struct kobj_attribute _name##_attr = __ATTR_WO(_name)
 #define KSM_ATTR(_name) \
 	static struct kobj_attribute _name##_attr = \
 		__ATTR(_name, 0644, _name##_show, _name##_store)
 
+static ssize_t force_madvise_store(struct kobject *kobj,
+				     struct kobj_attribute *attr,
+				     const char *buf, size_t count)
+{
+	int err;
+	pid_t pid;
+	bool merge = true;
+	struct task_struct *tsk;
+	struct mm_struct *mm;
+	struct vm_area_struct *vma;
+
+	err = kstrtoint(buf, 10, &pid);
+	if (err)
+		return -EINVAL;
+
+	if (pid < 0) {
+		pid = abs(pid);
+		merge = false;
+	}
+
+	if (!pid && *buf == '-')
+		merge = false;
+
+	rcu_read_lock();
+	if (pid) {
+		tsk = find_task_by_vpid(pid);
+		if (!tsk) {
+			err = -ESRCH;
+			rcu_read_unlock();
+			goto out;
+		}
+	} else {
+		tsk = current;
+	}
+
+	tsk = tsk->group_leader;
+
+	get_task_struct(tsk);
+	rcu_read_unlock();
+
+	mm = get_task_mm(tsk);
+	if (!mm) {
+		err = -EINVAL;
+		goto out_put_task_struct;
+	}
+	down_write(&mm->mmap_sem);
+	vma = mm->mmap;
+	while (vma) {
+		if (merge)
+			ksm_enter(vma->vm_mm, vma, &vma->vm_flags);
+		else
+			ksm_leave(vma, vma->vm_start, vma->vm_end, &vma->vm_flags);
+		vma = vma->vm_next;
+	}
+	up_write(&mm->mmap_sem);
+	mmput(mm);
+
+out_put_task_struct:
+	put_task_struct(tsk);
+
+out:
+	return err ? err : count;
+}
+KSM_ATTR_WO(force_madvise);
+
 static ssize_t sleep_millisecs_show(struct kobject *kobj,
 				    struct kobj_attribute *attr, char *buf)
 {
@@ -3185,6 +3252,7 @@ static ssize_t full_scans_show(struct kobject *kobj,
 KSM_ATTR_RO(full_scans);
 
 static struct attribute *ksm_attrs[] = {
+	&force_madvise_attr.attr,
 	&sleep_millisecs_attr.attr,
 	&pages_to_scan_attr.attr,
 	&run_attr.attr,
Signature · 1 line
2.21.0
Reviewed-by on v2 3/4
#4 lore raw

Re: [PATCH RFC v2 3/4] mm/ksm: introduce force_madvise knob

Quoted 103 lines · Oleksandr Natalenko

Present a new sysfs knob to mark task's anonymous memory as mergeable.

To force merging task's VMAs, its PID is echoed in a write-only file:

   # echo PID > /sys/kernel/mm/ksm/force_madvise

Force unmerging is done similarly, but with "minus" sign:

   # echo -PID > /sys/kernel/mm/ksm/force_madvise

"0" or "-0" can be used to control the current task.

To achieve this, previously introduced ksm_enter()/ksm_leave() helpers are used in the "store" handler.

---
 mm/ksm.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)
mm/ksm.c +67 -0
index e9f3901168bb..22c59fb03d3a 100644
@@ -2879,10 +2879,77 @@ static void wait_while_offlining(void)
 
 #define KSM_ATTR_RO(_name) \
 	static struct kobj_attribute _name##_attr = __ATTR_RO(_name)
+#define KSM_ATTR_WO(_name) \
+	static struct kobj_attribute _name##_attr = __ATTR_WO(_name)
 #define KSM_ATTR(_name) \
 	static struct kobj_attribute _name##_attr = \
 		__ATTR(_name, 0644, _name##_show, _name##_store)
 
+static ssize_t force_madvise_store(struct kobject *kobj,
+				     struct kobj_attribute *attr,
+				     const char *buf, size_t count)
+{
+	int err;
+	pid_t pid;
+	bool merge = true;
+	struct task_struct *tsk;
+	struct mm_struct *mm;
+	struct vm_area_struct *vma;
+
+	err = kstrtoint(buf, 10, &pid);
+	if (err)
+		return -EINVAL;
+
+	if (pid < 0) {
+		pid = abs(pid);
+		merge = false;
+	}
+
+	if (!pid && *buf == '-')
+		merge = false;
+
+	rcu_read_lock();
+	if (pid) {
+		tsk = find_task_by_vpid(pid);
+		if (!tsk) {
+			err = -ESRCH;
+			rcu_read_unlock();
+			goto out;
+		}
+	} else {
+		tsk = current;
+	}
+
+	tsk = tsk->group_leader;
+
+	get_task_struct(tsk);
+	rcu_read_unlock();
+
+	mm = get_task_mm(tsk);
+	if (!mm) {
+		err = -EINVAL;
+		goto out_put_task_struct;
+	}
+	down_write(&mm->mmap_sem);
+	vma = mm->mmap;
+	while (vma) {
+		if (merge)
+			ksm_enter(vma->vm_mm, vma, &vma->vm_flags);
+		else
+			ksm_leave(vma, vma->vm_start, vma->vm_end, &vma->vm_flags);
+		vma = vma->vm_next;
+	}
+	up_write(&mm->mmap_sem);
+	mmput(mm);
+
+out_put_task_struct:
+	put_task_struct(tsk);
+
+out:
+	return err ? err : count;
+}
+KSM_ATTR_WO(force_madvise);
+
 static ssize_t sleep_millisecs_show(struct kobject *kobj,
 				    struct kobj_attribute *attr, char *buf)
 {

mm/ksm.c static ssize_t full_scans_show(struct kobject *kobj, · line 3258

Quoted 7 lines
mm/ksm.c +1 -0
@@ -3185,6 +3252,7 @@ static ssize_t full_scans_show(struct kobject *kobj,
 KSM_ATTR_RO(full_scans);
 
 static struct attribute *ksm_attrs[] = {
+	&force_madvise_attr.attr,
 	&sleep_millisecs_attr.attr,
 	&pages_to_scan_attr.attr,
 	&run_attr.attr,

Looks fine to me.

Signature · 1 line
Aaron Tomlin
Reviewed-by
#5 lore raw

Re: [PATCH RFC v2 3/4] mm/ksm: introduce force_madvise knob

LGTM

вт, 14 мая 2019 г. в 16:22, Aaron Tomlin <[email protected]>:

Quoted 119 lines · Oleksandr Natalenko
Quoted 103 lines

Present a new sysfs knob to mark task's anonymous memory as mergeable.

To force merging task's VMAs, its PID is echoed in a write-only file:

   # echo PID > /sys/kernel/mm/ksm/force_madvise

Force unmerging is done similarly, but with "minus" sign:

   # echo -PID > /sys/kernel/mm/ksm/force_madvise

"0" or "-0" can be used to control the current task.

To achieve this, previously introduced ksm_enter()/ksm_leave() helpers are used in the "store" handler.

---
 mm/ksm.c | 68 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)
mm/ksm.c +67 -0
index e9f3901168bb..22c59fb03d3a 100644
@@ -2879,10 +2879,77 @@ static void wait_while_offlining(void)
 
 #define KSM_ATTR_RO(_name) \
      static struct kobj_attribute _name##_attr = __ATTR_RO(_name)
+#define KSM_ATTR_WO(_name) \
+     static struct kobj_attribute _name##_attr = __ATTR_WO(_name)
 #define KSM_ATTR(_name) \
      static struct kobj_attribute _name##_attr = \
              __ATTR(_name, 0644, _name##_show, _name##_store)
 
+static ssize_t force_madvise_store(struct kobject *kobj,
+                                  struct kobj_attribute *attr,
+                                  const char *buf, size_t count)
+{
+     int err;
+     pid_t pid;
+     bool merge = true;
+     struct task_struct *tsk;
+     struct mm_struct *mm;
+     struct vm_area_struct *vma;
+
+     err = kstrtoint(buf, 10, &pid);
+     if (err)
+             return -EINVAL;
+
+     if (pid < 0) {
+             pid = abs(pid);
+             merge = false;
+     }
+
+     if (!pid && *buf == '-')
+             merge = false;
+
+     rcu_read_lock();
+     if (pid) {
+             tsk = find_task_by_vpid(pid);
+             if (!tsk) {
+                     err = -ESRCH;
+                     rcu_read_unlock();
+                     goto out;
+             }
+     } else {
+             tsk = current;
+     }
+
+     tsk = tsk->group_leader;
+
+     get_task_struct(tsk);
+     rcu_read_unlock();
+
+     mm = get_task_mm(tsk);
+     if (!mm) {
+             err = -EINVAL;
+             goto out_put_task_struct;
+     }
+     down_write(&mm->mmap_sem);
+     vma = mm->mmap;
+     while (vma) {
+             if (merge)
+                     ksm_enter(vma->vm_mm, vma, &vma->vm_flags);
+             else
+                     ksm_leave(vma, vma->vm_start, vma->vm_end, &vma->vm_flags);
+             vma = vma->vm_next;
+     }
+     up_write(&mm->mmap_sem);
+     mmput(mm);
+
+out_put_task_struct:
+     put_task_struct(tsk);
+
+out:
+     return err ? err : count;
+}
+KSM_ATTR_WO(force_madvise);
+
 static ssize_t sleep_millisecs_show(struct kobject *kobj,
                                  struct kobj_attribute *attr, char *buf)
 {

mm/ksm.c static ssize_t full_scans_show(struct kobject *kobj, · line 3258

Quoted 7 lines
mm/ksm.c +1 -0
@@ -3185,6 +3252,7 @@ static ssize_t full_scans_show(struct kobject *kobj,
 KSM_ATTR_RO(full_scans);
 
 static struct attribute *ksm_attrs[] = {
+     &force_madvise_attr.attr,
      &sleep_millisecs_attr.attr,
      &pages_to_scan_attr.attr,
      &run_attr.attr,

Looks fine to me.

--
Aaron Tomlin

Signature · 2 lines
Have a nice day,
Timofey.
#6 lore raw

RFC PATCH v2 4/4 mm/ksm: add force merging/unmerging documentation

In reply to Oleksandr Natalenko

Document respective sysfs knob.

---
 Documentation/admin-guide/mm/ksm.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)
Documentation/admin-guide/mm/ksm.rst +11 -0
index 9303786632d1..4302b92910ec 100644
@@ -78,6 +78,17 @@ KSM daemon sysfs interface
 The KSM daemon is controlled by sysfs files in ``/sys/kernel/mm/ksm/``,
 readable by all but writable only by root:
 
+force_madvise
+        write-only control to force merging/unmerging for specific
+        task.
+
+        To mark the VMAs as mergeable, use:
+        ``echo PID > /sys/kernel/mm/ksm/force_madvise``
+
+        To unmerge all the VMAs, use:
+        ``echo -PID > /sys/kernel/mm/ksm/force_madvise``
+        (note the prepending "minus")
+
 pages_to_scan
         how many pages to scan before ksmd goes to sleep
         e.g. ``echo 100 > /sys/kernel/mm/ksm/pages_to_scan``.
Signature · 1 line
2.21.0
Reviewed-by on v2 4/4
#7 lore raw

Re: [PATCH RFC v2 4/4] mm/ksm: add force merging/unmerging documentation

LGTM for whole series

вт, 14 мая 2019 г. в 16:17, Oleksandr Natalenko <[email protected]>:

Quoted 17 lines

Document respective sysfs knob.

---
 Documentation/admin-guide/mm/ksm.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)
Documentation/admin-guide/mm/ksm.rst +1 -0
index 9303786632d1..4302b92910ec 100644
@@ -78,6 +78,17 @@ KSM daemon sysfs interface
 The KSM daemon is controlled by sysfs files in ``/sys/kernel/mm/ksm/``,
 readable by all but writable only by root:
 
+force_madvise

Documentation/admin-guide/mm/ksm.rst KSM daemon sysfs interface · line 91

Quoted 10 lines
Documentation/admin-guide/mm/ksm.rst +10 -0
+        write-only control to force merging/unmerging for specific
+        task.
+
+        To mark the VMAs as mergeable, use:
+        ``echo PID > /sys/kernel/mm/ksm/force_madvise``
+
+        To unmerge all the VMAs, use:
+        ``echo -PID > /sys/kernel/mm/ksm/force_madvise``
+        (note the prepending "minus")
+

In patch 3/4 you have special case with PID 0,
may be that also must be documented here?

Quoted 6 lines
Documentation/admin-guide/mm/ksm.rst +0 -1
 pages_to_scan
         how many pages to scan before ksmd goes to sleep
         e.g. ``echo 100 > /sys/kernel/mm/ksm/pages_to_scan``.
--

2.21.0

--
Have a nice day,
Timofey.

#8 lore raw

Re: [PATCH RFC v2 4/4] mm/ksm: add force merging/unmerging documentation

Hi.

Quoted 21 lines · Timofey Titovets

LGTM for whole series

вт, 14 мая 2019 г. в 16:17, Oleksandr Natalenko <[email protected]>:

Quoted 16 lines

Document respective sysfs knob.

---
 Documentation/admin-guide/mm/ksm.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)
Documentation/admin-guide/mm/ksm.rst +0 -0
index 9303786632d1..4302b92910ec 100644
@@ -78,6 +78,17 @@ KSM daemon sysfs interface
 The KSM daemon is controlled by sysfs files in ``/sys/kernel/mm/ksm/``,
 readable by all but writable only by root:
 
Quoted 13 lines

Documentation/admin-guide/mm/ksm.rst KSM daemon sysfs interface · line 91

Quoted 11 lines
Documentation/admin-guide/mm/ksm.rst +11 -0
+force_madvise
+        write-only control to force merging/unmerging for specific
+        task.
+
+        To mark the VMAs as mergeable, use:
+        ``echo PID > /sys/kernel/mm/ksm/force_madvise``
+
+        To unmerge all the VMAs, use:
+        ``echo -PID > /sys/kernel/mm/ksm/force_madvise``
+        (note the prepending "minus")
+

In patch 3/4 you have special case with PID 0,
may be that also must be documented here?

Thanks for the review. Yes, this is a valid point, I'll document it too.

Quoted 12 lines
Quoted 7 lines
Documentation/admin-guide/mm/ksm.rst +0 -1
 pages_to_scan
         how many pages to scan before ksmd goes to sleep
         e.g. ``echo 100 > /sys/kernel/mm/ksm/pages_to_scan``.
--

2.21.0

--
Have a nice day,
Timofey.

Signature · 3 lines
  Best regards,
    Oleksandr Natalenko (post-factum)
    Senior Software Maintenance Engineer
#9 lore raw
In reply to Oleksandr Natalenko

[This is adding a new user visible interface so you should be CCing linux-api mailing list. Also CC Hugh for KSM in general. Done now]

Quoted 24 lines · Oleksandr Natalenko

By default, KSM works only on memory that is marked by madvise(). And the only way to get around that is to either:

* use LD_PRELOAD; or
* patch the kernel with something like UKSM or PKSM.

Instead, lets implement a sysfs knob, which allows marking VMAs as mergeable. This can be used manually on some task in question or by some small userspace helper daemon.

The knob is named "force_madvise", and it is write-only. It accepts a PID to act on. To mark the VMAs as mergeable, use:

   # echo PID > /sys/kernel/mm/ksm/force_madvise

To unmerge all the VMAs, use the same approach, prepending the PID with the "minus" sign:

   # echo -PID > /sys/kernel/mm/ksm/force_madvise

This patchset is based on earlier Timofey's submission [1], but it doesn't use dedicated kthread to walk through the list of tasks/VMAs. Instead, it is up to userspace to traverse all the tasks in /proc if needed.

Quoted 7 lines

The previous suggestion [2] was based on amending do_anonymous_page() handler to implement fully automatic mode, but this approach was incorrect due to improper locking and not desired due to excessive complexity.

The current approach just implements minimal interface and leaves the decision on how and when to act to userspace.

Please make sure to describe a usecase that warrants adding a new interface we have to maintain for ever.

Quoted 19 lines

Thanks.

[1] https://lore.kernel.org/patchwork/patch/1012142/
[2] http://lkml.iu.edu/hypermail/linux/kernel/1905.1/02417.html

Oleksandr Natalenko (4):
mm/ksm: introduce ksm_enter() helper mm/ksm: introduce ksm_leave() helper mm/ksm: introduce force_madvise knob mm/ksm: add force merging/unmerging documentation

 Documentation/admin-guide/mm/ksm.rst |  11 ++
 mm/ksm.c                             | 160 +++++++++++++++++++++------
 2 files changed, 137 insertions(+), 34 deletions(-)
Signature · 1 line
2.21.0
Signature · 2 lines
Michal Hocko
SUSE Labs
#10 lore raw

[Forgot Hugh]

Quoted 62 lines · Michal Hocko

[This is adding a new user visible interface so you should be CCing linux-api mailing list. Also CC Hugh for KSM in general. Done now]

Quoted 24 lines · Oleksandr Natalenko

By default, KSM works only on memory that is marked by madvise(). And the only way to get around that is to either:

* use LD_PRELOAD; or
* patch the kernel with something like UKSM or PKSM.

Instead, lets implement a sysfs knob, which allows marking VMAs as mergeable. This can be used manually on some task in question or by some small userspace helper daemon.

The knob is named "force_madvise", and it is write-only. It accepts a PID to act on. To mark the VMAs as mergeable, use:

   # echo PID > /sys/kernel/mm/ksm/force_madvise

To unmerge all the VMAs, use the same approach, prepending the PID with the "minus" sign:

   # echo -PID > /sys/kernel/mm/ksm/force_madvise

This patchset is based on earlier Timofey's submission [1], but it doesn't use dedicated kthread to walk through the list of tasks/VMAs. Instead, it is up to userspace to traverse all the tasks in /proc if needed.

Quoted 7 lines

The previous suggestion [2] was based on amending do_anonymous_page() handler to implement fully automatic mode, but this approach was incorrect due to improper locking and not desired due to excessive complexity.

The current approach just implements minimal interface and leaves the decision on how and when to act to userspace.

Please make sure to describe a usecase that warrants adding a new interface we have to maintain for ever.

Quoted 19 lines

Thanks.

[1] https://lore.kernel.org/patchwork/patch/1012142/
[2] http://lkml.iu.edu/hypermail/linux/kernel/1905.1/02417.html

Oleksandr Natalenko (4):
mm/ksm: introduce ksm_enter() helper mm/ksm: introduce ksm_leave() helper mm/ksm: introduce force_madvise knob mm/ksm: add force merging/unmerging documentation

 Documentation/admin-guide/mm/ksm.rst |  11 ++
 mm/ksm.c                             | 160 +++++++++++++++++++++------
 2 files changed, 137 insertions(+), 34 deletions(-)
Signature · 1 line
2.21.0
Signature · 2 lines
Michal Hocko
SUSE Labs
Signature · 2 lines
Michal Hocko
SUSE Labs
#11 lore raw

Hi.

Quoted 5 lines · Michal Hocko

[Forgot Hugh]

Quoted 2 lines · Michal Hocko

[This is adding a new user visible interface so you should be CCing linux-api mailing list. Also CC Hugh for KSM in general. Done now]

Right, thanks for taking care of this.

Quoted 30 lines
Quoted 30 lines · Oleksandr Natalenko
Quoted 29 lines

By default, KSM works only on memory that is marked by madvise(). And the only way to get around that is to either:

* use LD_PRELOAD; or
* patch the kernel with something like UKSM or PKSM.

Instead, lets implement a sysfs knob, which allows marking VMAs as mergeable. This can be used manually on some task in question or by some small userspace helper daemon.

The knob is named "force_madvise", and it is write-only. It accepts a PID to act on. To mark the VMAs as mergeable, use:

   # echo PID > /sys/kernel/mm/ksm/force_madvise

To unmerge all the VMAs, use the same approach, prepending the PID with the "minus" sign:

   # echo -PID > /sys/kernel/mm/ksm/force_madvise

This patchset is based on earlier Timofey's submission [1], but it doesn't use dedicated kthread to walk through the list of tasks/VMAs. Instead, it is up to userspace to traverse all the tasks in /proc if needed.

The previous suggestion [2] was based on amending do_anonymous_page() handler to implement fully automatic mode, but this approach was incorrect due to improper locking and not desired due to excessive complexity.

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

The current approach just implements minimal interface and leaves the decision on how and when to act to userspace.

Please make sure to describe a usecase that warrants adding a new interface we have to maintain for ever.

I think of two major consumers of this interface:

1) hosts, that run containers, especially similar ones and especially in a trusted environment;

2) heavy applications, that can be run in multiple instances, not limited to opensource ones like Firefox, but also those that cannot be modified.

I'll add this justification to the cover letter once I send another iteration of this submission if necessary.

Thank you.

Quoted 28 lines
Quoted 24 lines
Quoted 20 lines

Thanks.

[1] https://lore.kernel.org/patchwork/patch/1012142/
[2] http://lkml.iu.edu/hypermail/linux/kernel/1905.1/02417.html

Oleksandr Natalenko (4):
mm/ksm: introduce ksm_enter() helper mm/ksm: introduce ksm_leave() helper mm/ksm: introduce force_madvise knob mm/ksm: add force merging/unmerging documentation

 Documentation/admin-guide/mm/ksm.rst |  11 ++
 mm/ksm.c                             | 160 +++++++++++++++++++++------
 2 files changed, 137 insertions(+), 34 deletions(-)
Signature · 1 line
2.21.0
Signature · 2 lines
Michal Hocko
SUSE Labs
Signature · 2 lines
Michal Hocko
SUSE Labs
Signature · 3 lines
  Best regards,
    Oleksandr Natalenko (post-factum)
    Senior Software Maintenance Engineer
#12 lore raw

On Wed 15-05-19 08:25:23, Oleksandr Natalenko wrote:
[...]

Quoted 11 lines
Quoted 2 lines
Quoted 2 lines

Please make sure to describe a usecase that warrants adding a new interface we have to maintain for ever.

I think of two major consumers of this interface:

1) hosts, that run containers, especially similar ones and especially in a trusted environment;

2) heavy applications, that can be run in multiple instances, not limited to opensource ones like Firefox, but also those that cannot be modified.

This is way too generic. Please provide something more specific. Ideally with numbers. Why those usecases cannot use an existing interfaces. Remember you are trying to add a new user interface which we will have to maintain for ever.

I will try to comment on the interface itself later. But I have to say that I am not impressed. Abusing sysfs for per process features is quite gross to be honest.

Signature · 2 lines
Michal Hocko
SUSE Labs
#13 lore raw

Hi.

Quoted 10 lines · Michal Hocko

On Wed 15-05-19 08:25:23, Oleksandr Natalenko wrote:
[...]

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

Please make sure to describe a usecase that warrants adding a new interface we have to maintain for ever.

I think of two major consumers of this interface:

1) hosts, that run containers, especially similar ones and especially in a trusted environment;

Quoted 8 lines
Quoted 3 lines

2) heavy applications, that can be run in multiple instances, not limited to opensource ones like Firefox, but also those that cannot be modified.

This is way too generic. Please provide something more specific. Ideally with numbers. Why those usecases cannot use an existing interfaces. Remember you are trying to add a new user interface which we will have to maintain for ever.

For my current setup with 2 Firefox instances I get 100 to 200 MiB saved for the second instance depending on the amount of tabs.

1 FF instance with 15 tabs:

$ echo "$(cat /sys/kernel/mm/ksm/pages_sharing) * 4 / 1024" | bc
410

2 FF instances, second one has 12 tabs (all the tabs are different):

$ echo "$(cat /sys/kernel/mm/ksm/pages_sharing) * 4 / 1024" | bc
592

At the very moment I do not have specific numbers for containerised workload, but those should be similar in case the containers share similar/same runtime (like multiple Node.js containers etc).

Answering your question regarding using existing interfaces, since there's only one, madvise(2), this requires modifying all the applications one wants to de-duplicate. In case of containers with arbitrary content or in case of binary-only apps this is pretty hard if not impossible to do properly.

Quoted 3 lines

I will try to comment on the interface itself later. But I have to say that I am not impressed. Abusing sysfs for per process features is quite gross to be honest.

Sure, please do.

Thanks for your time and inputs.

Signature · 3 lines
  Best regards,
    Oleksandr Natalenko (post-factum)
    Senior Software Maintenance Engineer
#14 lore raw

On Wed 15-05-19 09:37:23, Oleksandr Natalenko wrote:
[...]

Quoted 7 lines
Quoted 4 lines

This is way too generic. Please provide something more specific. Ideally with numbers. Why those usecases cannot use an existing interfaces. Remember you are trying to add a new user interface which we will have to maintain for ever.

For my current setup with 2 Firefox instances I get 100 to 200 MiB saved for the second instance depending on the amount of tabs.

What does prevent Firefox (an opensource project) to be updated to use the explicit merging?

[...]

Quoted 5 lines

Answering your question regarding using existing interfaces, since there's only one, madvise(2), this requires modifying all the applications one wants to de-duplicate. In case of containers with arbitrary content or in case of binary-only apps this is pretty hard if not impossible to do properly.

OK, this makes more sense. Please note that there are other people who would like to see certain madvise operations to be done on a remote process - e.g. to allow external memory management (Android would like to control memory aging so something like MADV_DONTNEED without loosing content and more probably) and potentially other madvise operations. Or maybe we need a completely new interface other than madvise.

In general, having a more generic API that would cover more usecases is definitely much more preferable than one ad-hoc API that handles a very specific usecase. So please try to think about a more generic

Signature · 2 lines
Michal Hocko
SUSE Labs
#15 lore raw
Quoted 5 lines · Michal Hocko
Quoted 2 lines

For my current setup with 2 Firefox instances I get 100 to 200 MiB saved for the second instance depending on the amount of tabs.

What does prevent Firefox (an opensource project) to be updated to use the explicit merging?

This was rather an example of a big project. Other big projects may be closed source, of course.

And yes, with regard to FF specifically I think nothing prevents it from being modified appropriately.

Quoted 6 lines
Quoted 5 lines

Answering your question regarding using existing interfaces, since there's only one, madvise(2), this requires modifying all the applications one wants to de-duplicate. In case of containers with arbitrary content or in case of binary-only apps this is pretty hard if not impossible to do properly.

Quoted 6 lines

OK, this makes more sense. Please note that there are other people who would like to see certain madvise operations to be done on a remote process - e.g. to allow external memory management (Android would like to control memory aging so something like MADV_DONTNEED without loosing content and more probably) and potentially other madvise operations. Or maybe we need a completely new interface other than madvise.

I didn't know about those intentions. Could you please point me to a relevant discussion so that I can check the details?

Quoted 3 lines

In general, having a more generic API that would cover more usecases is definitely much more preferable than one ad-hoc API that handles a very specific usecase. So please try to think about a more generic

Yup, I see now. Since you are aware of ongoing intentions, please do Cc those people then and/or let me know about previous discussions please. That way thinking of how a new API should be implemented (be it a sysfs file or something else) should be easier and more visible.

Thanks.

Signature · 3 lines
  Best regards,
    Oleksandr Natalenko (post-factum)
    Senior Software Maintenance Engineer
#16 lore raw
Quoted 9 lines · Oleksandr Natalenko
Quoted 5 lines · Michal Hocko
Quoted 2 lines

For my current setup with 2 Firefox instances I get 100 to 200 MiB saved for the second instance depending on the amount of tabs.

What does prevent Firefox (an opensource project) to be updated to use the explicit merging?

This was rather an example of a big project. Other big projects may be closed source, of course.

Again, specific examples are usually considered a much better justification than "something might use the feature".

[...]

Quoted 9 lines
Quoted 6 lines

OK, this makes more sense. Please note that there are other people who would like to see certain madvise operations to be done on a remote process - e.g. to allow external memory management (Android would like to control memory aging so something like MADV_DONTNEED without loosing content and more probably) and potentially other madvise operations. Or maybe we need a completely new interface other than madvise.

I didn't know about those intentions. Could you please point me to a relevant discussion so that I can check the details?

I am sorry I do not have any specific links to patches under discussion. We have discussed that topic at LSFMM this year
(https://lwn.net/Articles/787217/) and Google guys should be sending something soon.

Signature · 2 lines
Michal Hocko
SUSE Labs
#17 lore raw
In reply to Michal Hocko

[Cc Suren and Minchan - the email thread starts here [email protected]]

On Wed 15-05-19 08:53:11, Michal Hocko wrote:
[...]

Quoted 3 lines

I will try to comment on the interface itself later. But I have to say that I am not impressed. Abusing sysfs for per process features is quite gross to be honest.

I have already commented on this in other email. I consider sysfs an unsuitable interface for per-process API. Not to mention this particular one is very KSM specific while the question about setting different hints on memory of a remote process is a more generic question. As already mentioned there are usecases where people would like to say that a certain memory is cold from outside of the process context (e.g. monitor application). So essentially a form of a user space memory management. And this usecase sounds a bit similar to me and having a common api sounds more sensible to me.

One thing we were discussing at LSFMM this year was a way to either provide madvise_remote(pid, addr, length, advice) or a fadvise alternative over /proc/<pid>/map_vmas/<range> file descriptors (essentially resembling the existing map_files api) to achieve such a functionality. This is still a very rough idea but the api would sound much more generic to me and it would allow much wider range of usecases.

But maybe I am completely wrong and this is just opens a can of worms that we do not want.

Signature · 2 lines
Michal Hocko
SUSE Labs
#18 lore raw
Quoted 10 lines · Michal Hocko

[Cc Suren and Minchan - the email thread starts here [email protected]]

On Wed 15-05-19 08:53:11, Michal Hocko wrote:
[...]

Quoted 3 lines

I will try to comment on the interface itself later. But I have to say that I am not impressed. Abusing sysfs for per process features is quite gross to be honest.

I have already commented on this in other email. I consider sysfs an unsuitable interface for per-process API.

Wait, what? A new sysfs file/directory per process? That's crazy, no one must have benchmarked it :)

And I agree, sysfs is not for that, please don't abuse it. Proc is for process apis.

thanks,

greg k-h

#19 lore raw
Quoted 9 lines · Greg KH
Quoted 8 lines · Michal Hocko

[Cc Suren and Minchan - the email thread starts here [email protected]]

On Wed 15-05-19 08:53:11, Michal Hocko wrote:
[...]

Quoted 3 lines

I will try to comment on the interface itself later. But I have to say that I am not impressed. Abusing sysfs for per process features is quite gross to be honest.

Quoted 5 lines
Quoted 2 lines

I have already commented on this in other email. I consider sysfs an unsuitable interface for per-process API.

Wait, what? A new sysfs file/directory per process? That's crazy, no one must have benchmarked it :)

Just to clarify, that was not a per process file but rather per process API. Essentially echo $PID > $SYSFS_SPECIAL_FILE

Signature · 2 lines
Michal Hocko
SUSE Labs
#20 lore raw
Quoted 10 lines · Michal Hocko
Quoted 9 lines · Greg KH
Quoted 8 lines · Michal Hocko

[Cc Suren and Minchan - the email thread starts here [email protected]]

On Wed 15-05-19 08:53:11, Michal Hocko wrote:
[...]

Quoted 3 lines

I will try to comment on the interface itself later. But I have to say that I am not impressed. Abusing sysfs for per process features is quite gross to be honest.

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

I have already commented on this in other email. I consider sysfs an unsuitable interface for per-process API.

Wait, what? A new sysfs file/directory per process? That's crazy, no one must have benchmarked it :)

Just to clarify, that was not a per process file but rather per process API. Essentially echo $PID > $SYSFS_SPECIAL_FILE

Ick, no, that's not ok either. sysfs files are not a replacement for syscalls :)

thanks,

greg k-h