mm: process/cgroup ksm support

This adds a new prctl to API to enable and disable KSM on a per process basis instead of only at the VMA basis (with madvise).

1) Introduce new MMF_VM_MERGE_ANY flag

This introduces the new flag MMF_VM_MERGE_ANY flag. When this flag is set, kernel samepage merging (ksm) gets enabled for all vma's of a process.

2) add flag to __ksm_enter

This change adds the flag parameter to __ksm_enter. This allows to distinguish if ksm was called by prctl or madvise.

3) add flag to __ksm_exit call

This adds the flag parameter to the __ksm_exit() call. This allows to distinguish if this call is for an prctl or madvise invocation.

4) invoke madvise for all vmas in scan_get_next_rmap_item

If the new flag MMF_VM_MERGE_ANY has been set for a process, iterate over all the vmas and enable ksm if possible. For the vmas that can be ksm enabled this is only done once.

5) support disabling of ksm for a process

This adds the ability to disable ksm for a process if ksm has been enabled for the process.

6) add new prctl option to get and set ksm for a process

This adds two new options to the prctl system call
- enable ksm for all vmas of a process (if the vmas support it).
- query if ksm has been enabled for a process.

Signed-off-by Stefan Roesch <[email protected]>

kernel/sys.c +1 -0

@@ -2659,6 +2660,34 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
 	case PR_SET_VMA:
 		error = prctl_set_vma(arg2, arg3, arg4, arg5);
 		break;
+#ifdef CONFIG_KSM

kernel/sys.c +17 -0

+			else if (!test_bit(MMF_VM_MERGE_ANY, &me->mm->flags))
+				error = __ksm_enter(me->mm, MMF_VM_MERGE_ANY);
+			mmap_write_unlock(me->mm);
+		} else {
+			__ksm_exit(me->mm, MMF_VM_MERGE_ANY);
+		}
+		break;
+	case PR_GET_MEMORY_MERGE:
+		if (!capable(CAP_SYS_RESOURCE))
+			return -EPERM;
+
+		if (arg2 || arg3 || arg4 || arg5)
+			return -EINVAL;
+
+		error = !!test_bit(MMF_VM_MERGE_ANY, &me->mm->flags);
+		break;
+#endif
 	default:
 		error = -EINVAL;
 		break;

mm/ksm.c +12 -0

index 56808e3bfd19..23d6944f78ad 100644
@@ -1063,6 +1063,7 @@ static int unmerge_and_remove_all_rmap_items(void)
 
 			mm_slot_free(mm_slot_cache, mm_slot);
 			clear_bit(MMF_VM_MERGEABLE, &mm->flags);
+			clear_bit(MMF_VM_MERGE_ANY, &mm->flags);
 			mmdrop(mm);
 		} else
 			spin_unlock(&ksm_mmlist_lock);

@@ -2329,6 +2330,17 @@ static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot,
 	return rmap_item;
 }
 
+static bool vma_ksm_mergeable(struct vm_area_struct *vma)
+{
+	if (vma->vm_flags & VM_MERGEABLE)
+		return true;
+
+	if (test_bit(MMF_VM_MERGE_ANY, &vma->vm_mm->flags))
+		return true;
+
+	return false;
+}
+
 static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
 {
 	struct mm_struct *mm;

mm/ksm.c +8 -1

+		}
 		if (ksm_scan.address < vma->vm_start)
 			ksm_scan.address = vma->vm_start;
 		if (!vma->anon_vma)

@@ -2491,6 +2515,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
 
 		mm_slot_free(mm_slot_cache, mm_slot);
 		clear_bit(MMF_VM_MERGEABLE, &mm->flags);
+		clear_bit(MMF_VM_MERGE_ANY, &mm->flags);
 		mmap_read_unlock(mm);
 		mmdrop(mm);
 	} else {

@@ -2664,12 +2690,39 @@ int __ksm_enter(struct mm_struct *mm)
 	return 0;
 }
 
-void __ksm_exit(struct mm_struct *mm)
+static void unmerge_vmas(struct mm_struct *mm)
+{
+	struct vm_area_struct *vma;
+	struct vma_iterator vmi;
+
+	vma_iter_init(&vmi, mm, 0);

mm/ksm.c +5 -0

+		}
+	}
+	mmap_read_unlock(mm);
+}
+

Quoted 30 lines · Stefan Roesch

This adds a new prctl to API to enable and disable KSM on a per process basis instead of only at the VMA basis (with madvise).

1) Introduce new MMF_VM_MERGE_ANY flag

This introduces the new flag MMF_VM_MERGE_ANY flag. When this flag is set, kernel samepage merging (ksm) gets enabled for all vma's of a process.

2) add flag to __ksm_enter

This change adds the flag parameter to __ksm_enter. This allows to distinguish if ksm was called by prctl or madvise.

3) add flag to __ksm_exit call

This adds the flag parameter to the __ksm_exit() call. This allows to distinguish if this call is for an prctl or madvise invocation.

4) invoke madvise for all vmas in scan_get_next_rmap_item

If the new flag MMF_VM_MERGE_ANY has been set for a process, iterate over all the vmas and enable ksm if possible. For the vmas that can be ksm enabled this is only done once.

5) support disabling of ksm for a process

This adds the ability to disable ksm for a process if ksm has been enabled for the process.

Quoted 7 lines

6) add new prctl option to get and set ksm for a process

This adds two new options to the prctl system call
- enable ksm for all vmas of a process (if the vmas support it).
- query if ksm has been enabled for a process.

Signed-off-by Stefan Roesch <[email protected]>

Hey Stefan, thanks for merging the patches into one. I found it much easier to review.

Overall this looks straight-forward to me. A few comments below:

Quoted 5 lines

kernel/sys.c +1 -0

@@ -2659,6 +2660,34 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
 	case PR_SET_VMA:
 		error = prctl_set_vma(arg2, arg3, arg4, arg5);
 		break;
+#ifdef CONFIG_KSM

kernel/sys.c SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, · line 2673

Quoted 10 lines

kernel/sys.c +10 -0

+	case PR_SET_MEMORY_MERGE:
+		if (!capable(CAP_SYS_RESOURCE))
+			return -EPERM;
+
+		if (arg2) {
+			if (mmap_write_lock_killable(me->mm))
+				return -EINTR;
+
+			if (test_bit(MMF_VM_MERGEABLE, &me->mm->flags))
+				error = -EINVAL;

So if the workload has already madvised specific VMAs the process-enablement will fail. Why is that? Shouldn't it be possible to override a local decision from an outside context that has more perspective on both sharing opportunities and security aspects?

If there is a good reason for it, the -EINVAL should be addressed in the manpage. And maybe add a comment here as well.

Quoted 51 lines

kernel/sys.c +17 -0

+			else if (!test_bit(MMF_VM_MERGE_ANY, &me->mm->flags))
+				error = __ksm_enter(me->mm, MMF_VM_MERGE_ANY);
+			mmap_write_unlock(me->mm);
+		} else {
+			__ksm_exit(me->mm, MMF_VM_MERGE_ANY);
+		}
+		break;
+	case PR_GET_MEMORY_MERGE:
+		if (!capable(CAP_SYS_RESOURCE))
+			return -EPERM;
+
+		if (arg2 || arg3 || arg4 || arg5)
+			return -EINVAL;
+
+		error = !!test_bit(MMF_VM_MERGE_ANY, &me->mm->flags);
+		break;
+#endif
 	default:
 		error = -EINVAL;
 		break;

mm/ksm.c +12 -0

index 56808e3bfd19..23d6944f78ad 100644
@@ -1063,6 +1063,7 @@ static int unmerge_and_remove_all_rmap_items(void)
 
 			mm_slot_free(mm_slot_cache, mm_slot);
 			clear_bit(MMF_VM_MERGEABLE, &mm->flags);
+			clear_bit(MMF_VM_MERGE_ANY, &mm->flags);
 			mmdrop(mm);
 		} else
 			spin_unlock(&ksm_mmlist_lock);

@@ -2329,6 +2330,17 @@ static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot,
 	return rmap_item;
 }
 
+static bool vma_ksm_mergeable(struct vm_area_struct *vma)
+{
+	if (vma->vm_flags & VM_MERGEABLE)
+		return true;
+
+	if (test_bit(MMF_VM_MERGE_ANY, &vma->vm_mm->flags))
+		return true;
+
+	return false;
+}
+
 static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
 {
 	struct mm_struct *mm;

mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2422

Quoted 7 lines

mm/ksm.c +2 -1

@@ -2405,8 +2417,20 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
 		goto no_vmas;
 
 	for_each_vma(vmi, vma) {
-		if (!(vma->vm_flags & VM_MERGEABLE))
+		if (!vma_ksm_mergeable(vma))
 			continue;
+		if (!(vma->vm_flags & VM_MERGEABLE)) {

IMO, the helper obscures the interaction between the vma flag and the per-process flag here. How about:

		if (!(vma->vm_flags & VM_MERGEABLE)) {
			if (!test_bit(MMF_VM_MERGE_ANY, &vma->vm_mm->flags))
				continue;

			/*
			 * With per-process merging enabled, have the MM scan
			 * enroll any existing and new VMAs on the fly.
			 *
			ksm_madvise();
		}

mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2432

Quoted 10 lines

mm/ksm.c +10 -0

+			unsigned long flags = vma->vm_flags;
+
+			/* madvise failed, use next vma */
+			if (ksm_madvise(vma, vma->vm_start, vma->vm_end, MADV_MERGEABLE, &flags))
+				continue;
+			/* vma, not supported as being mergeable */
+			if (!(flags & VM_MERGEABLE))
+				continue;
+
+			vm_flags_set(vma, VM_MERGEABLE);

I don't understand the local flags. Can't it pass &vma->vm_flags to ksm_madvise()? It'll set VM_MERGEABLE on success. And you know it wasn't set before because the whole thing is inside the !set branch. The return value doesn't seem super useful, it's only the flag setting that matters:

			ksm_madvise(vma, vma->vm_start, vma->vm_end, MADV_MERGEABLE, &vma->vm_flags);
			/* madvise can fail, and will skip special vmas (pfnmaps and such) */
			if (!(vma->vm_flags & VM_MERGEABLE))
				continue;

Quoted 24 lines

mm/ksm.c +8 -1

+		}
 		if (ksm_scan.address < vma->vm_start)
 			ksm_scan.address = vma->vm_start;
 		if (!vma->anon_vma)

@@ -2491,6 +2515,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
 
 		mm_slot_free(mm_slot_cache, mm_slot);
 		clear_bit(MMF_VM_MERGEABLE, &mm->flags);
+		clear_bit(MMF_VM_MERGE_ANY, &mm->flags);
 		mmap_read_unlock(mm);
 		mmdrop(mm);
 	} else {

@@ -2664,12 +2690,39 @@ int __ksm_enter(struct mm_struct *mm)
 	return 0;
 }
 
-void __ksm_exit(struct mm_struct *mm)
+static void unmerge_vmas(struct mm_struct *mm)
+{
+	struct vm_area_struct *vma;
+	struct vma_iterator vmi;
+
+	vma_iter_init(&vmi, mm, 0);

mm/ksm.c int __ksm_enter(struct mm_struct *mm) · line 2708

Quoted 10 lines

mm/ksm.c +10 -0

+
+	mmap_read_lock(mm);
+	for_each_vma(vmi, vma) {
+		if (vma->vm_flags & VM_MERGEABLE) {
+			unsigned long flags = vma->vm_flags;
+
+			if (ksm_madvise(vma, vma->vm_start, vma->vm_end, MADV_UNMERGEABLE, &flags))
+				continue;
+
+			vm_flags_clear(vma, VM_MERGEABLE);

ksm_madvise() tests and clears VM_MERGEABLE, so AFAICS

	for_each_vma(vmi, vma)
		ksm_madvise();

should do it...

Quoted 5 lines

mm/ksm.c +5 -0

+		}
+	}
+	mmap_read_unlock(mm);
+}
+

mm/ksm.c int __ksm_enter(struct mm_struct *mm) · line 2723

Quoted 10 lines

mm/ksm.c +5 -0

+void __ksm_exit(struct mm_struct *mm, int flag)
 {
 	struct ksm_mm_slot *mm_slot;
 	struct mm_slot *slot;
 	int easy_to_free = 0;
 
+	if (!(current->flags & PF_EXITING) && flag == MMF_VM_MERGE_ANY &&
+		test_bit(MMF_VM_MERGE_ANY, &mm->flags)) {
+		clear_bit(MMF_VM_MERGE_ANY, &mm->flags);
+		unmerge_vmas(mm);

...and then it's short enough to just open-code it here and drop the unmerge_vmas() helper.

Quoted 41 lines · Johannes Weiner

mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2422

Quoted 8 lines · Stefan Roesch

mm/ksm.c +2 -1

@@ -2405,8 +2417,20 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
 		goto no_vmas;
 
 	for_each_vma(vmi, vma) {
-		if (!(vma->vm_flags & VM_MERGEABLE))
+		if (!vma_ksm_mergeable(vma))
 			continue;
+		if (!(vma->vm_flags & VM_MERGEABLE)) {

IMO, the helper obscures the interaction between the vma flag and the per-process flag here. How about:

		if (!(vma->vm_flags & VM_MERGEABLE)) {
			if (!test_bit(MMF_VM_MERGE_ANY, &vma->vm_mm->flags))
				continue;

			/*
			 * With per-process merging enabled, have the MM scan
			 * enroll any existing and new VMAs on the fly.
			 *
			ksm_madvise();
		}

mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2432

Quoted 10 lines

mm/ksm.c +10 -0

+			unsigned long flags = vma->vm_flags;
+
+			/* madvise failed, use next vma */
+			if (ksm_madvise(vma, vma->vm_start, vma->vm_end, MADV_MERGEABLE, &flags))
+				continue;
+			/* vma, not supported as being mergeable */
+			if (!(flags & VM_MERGEABLE))
+				continue;
+
+			vm_flags_set(vma, VM_MERGEABLE);

I don't understand the local flags. Can't it pass &vma->vm_flags to ksm_madvise()? It'll set VM_MERGEABLE on success. And you know it wasn't set before because the whole thing is inside the !set branch. The return value doesn't seem super useful, it's only the flag setting that matters:

Quoted 5 lines

			ksm_madvise(vma, vma->vm_start, vma->vm_end, MADV_MERGEABLE, &vma->vm_flags);
			/* madvise can fail, and will skip special vmas (pfnmaps and such) */
			if (!(vma->vm_flags & VM_MERGEABLE))
				continue;

vm_flags is defined as const. I cannot pass it directly inside the function, this is the reason, I'm using a local variable for it.

Quoted 42 lines · Stefan Roesch

Quoted 41 lines · Johannes Weiner

mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2422

Quoted 8 lines · Stefan Roesch

mm/ksm.c +2 -1

@@ -2405,8 +2417,20 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)
 		goto no_vmas;
 
 	for_each_vma(vmi, vma) {
-		if (!(vma->vm_flags & VM_MERGEABLE))
+		if (!vma_ksm_mergeable(vma))
 			continue;
+		if (!(vma->vm_flags & VM_MERGEABLE)) {

IMO, the helper obscures the interaction between the vma flag and the per-process flag here. How about:

		if (!(vma->vm_flags & VM_MERGEABLE)) {
			if (!test_bit(MMF_VM_MERGE_ANY, &vma->vm_mm->flags))
				continue;

			/*
			 * With per-process merging enabled, have the MM scan
			 * enroll any existing and new VMAs on the fly.
			 *
			ksm_madvise();
		}

mm/ksm.c static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) · line 2432

Quoted 10 lines

mm/ksm.c +10 -0

+			unsigned long flags = vma->vm_flags;
+
+			/* madvise failed, use next vma */
+			if (ksm_madvise(vma, vma->vm_start, vma->vm_end, MADV_MERGEABLE, &flags))
+				continue;
+			/* vma, not supported as being mergeable */
+			if (!(flags & VM_MERGEABLE))
+				continue;
+
+			vm_flags_set(vma, VM_MERGEABLE);

I don't understand the local flags. Can't it pass &vma->vm_flags to ksm_madvise()? It'll set VM_MERGEABLE on success. And you know it wasn't set before because the whole thing is inside the !set branch. The return value doesn't seem super useful, it's only the flag setting that matters:

Quoted 8 lines

Quoted 5 lines

			ksm_madvise(vma, vma->vm_start, vma->vm_end, MADV_MERGEABLE, &vma->vm_flags);
			/* madvise can fail, and will skip special vmas (pfnmaps and such) */
			if (!(vma->vm_flags & VM_MERGEABLE))
				continue;

vm_flags is defined as const. I cannot pass it directly inside the function, this is the reason, I'm using a local variable for it.

Oops, good catch.

However, while looking at the flag helpers, I'm also realizing that modifications requires the mmap_sem in write mode, which this code doesn't. This function might potentially scan the entire process address space, so you can't just change the lock mode, either.

Staring more at this, do you actually need to set VM_MERGEABLE on the individual vmas? There are only a few places that check VM_MERGEABLE, and AFAICS they can all just check for MMF_VM_MERGE_ANY also.

You'd need to factor out the vma compatibility checks from ksm_madvise(), and skip over special vmas during the mm scan. But those tests are all stable under the read lock, so that's fine.

The other thing ksm_madvise() does is ksm_enter() - but that's obviously not needed from inside the loop over ksm_enter'd mms. :)

This adds three new tests to the selftests for KSM. These tests use the new prctl API's to enable and disable KSM.

...

tools/testing/selftests/mm/Makefile +0 -0

index d90cdc06aa59..507cb22bdebd 100644

Quoted 10 lines · On Thu, 23 Feb 2023 20:40:00 -0800 Stefan Roesch <[email protected]> wrote:

This adds three new tests to the selftests for KSM. These tests use the new prctl API's to enable and disable KSM.

...

tools/testing/selftests/mm/Makefile +0 -0

index d90cdc06aa59..507cb22bdebd 100644

tools/testing/selftests/mm/Makefile MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p · line 36

Quoted 9 lines

tools/testing/selftests/mm/Makefile +2 -1

@@ -29,7 +29,8 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p
 # LDLIBS.
 MAKEFLAGS += --no-builtin-rules
 
-CFLAGS = -Wall -I $(top_srcdir) -I $(top_srcdir)/usr/include $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
+CFLAGS = -Wall -I $(top_srcdir)/tools/include/uapi
+CFLAGS += -I $(top_srcdir) -I $(top_srcdir)/usr/include $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
 LDLIBS = -lrt -lpthread
 TEST_GEN_FILES = cow
 TEST_GEN_FILES += compaction_test

This change runs afoul of the recently merged 8eb3751c73bec ("selftests: vm: Fix incorrect kernel headers search path").

I did this:

tools/testing/selftests/mm/Makefile renamed from tools/testing/selftests/mm/Makefile~selftests-mm-add-new-selftests-for-ksm +1 -1

@@ -29,7 +29,7 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p
 # LDLIBS.
 MAKEFLAGS += --no-builtin-rules
 
-CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
+CFLAGS = -Wall -I $(top_srcdir) -I $(top_srcdir)/tools/include/uapi $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
 LDLIBS = -lrt -lpthread
 TEST_GEN_FILES = cow
 TEST_GEN_FILES += compaction_test

_

But I expect it's a bit wrong. Please check?

Quoted 40 lines · Andrew Morton

Quoted 10 lines · On Thu, 23 Feb 2023 20:40:00 -0800 Stefan Roesch <[email protected]> wrote:

This adds three new tests to the selftests for KSM. These tests use the new prctl API's to enable and disable KSM.

...

tools/testing/selftests/mm/Makefile +0 -0

index d90cdc06aa59..507cb22bdebd 100644

tools/testing/selftests/mm/Makefile MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p · line 36

Quoted 9 lines

tools/testing/selftests/mm/Makefile +2 -1

@@ -29,7 +29,8 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p
  # LDLIBS.
  MAKEFLAGS += --no-builtin-rules
 
-CFLAGS = -Wall -I $(top_srcdir) -I $(top_srcdir)/usr/include $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
+CFLAGS = -Wall -I $(top_srcdir)/tools/include/uapi
+CFLAGS += -I $(top_srcdir) -I $(top_srcdir)/usr/include $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
  LDLIBS = -lrt -lpthread
  TEST_GEN_FILES = cow
  TEST_GEN_FILES += compaction_test

This change runs afoul of the recently merged 8eb3751c73bec ("selftests: vm: Fix incorrect kernel headers search path").

I did this:

tools/testing/selftests/mm/Makefile renamed from tools/testing/selftests/mm/Makefile~selftests-mm-add-new-selftests-for-ksm +1 -1

@@ -29,7 +29,7 @@ MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/' -e 's/ppc64.*/p
  # LDLIBS.
  MAKEFLAGS += --no-builtin-rules
 
-CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
+CFLAGS = -Wall -I $(top_srcdir) -I $(top_srcdir)/tools/include/uapi $(EXTRA_CFLAGS) $(KHDR_INCLUDES)
  LDLIBS = -lrt -lpthread
  TEST_GEN_FILES = cow
  TEST_GEN_FILES += compaction_test

_

Quoted 1 line

But I expect it's a bit wrong. Please check?

This change looks good.

So far KSM can only be enabled by calling madvise for memory regions. To be able to use KSM for more workloads, KSM needs to have the ability to be enabled / disabled at the process / cgroup level.

Quoted 3 lines · On Thu, 23 Feb 2023 20:39:57 -0800 Stefan Roesch <[email protected]> wrote:

So far KSM can only be enabled by calling madvise for memory regions. To be able to use KSM for more workloads, KSM needs to have the ability to be enabled / disabled at the process / cgroup level.

I'll toss this in for integration and testing, but I'd like to see reviewer input before proceeding further.

Please plan on adding suitable user-facing documentation? Presumably a patch for the prctl manpage?

Quoted 3 lines · On Thu, 23 Feb 2023 20:39:57 -0800 Stefan Roesch <[email protected]> wrote:

So far KSM can only be enabled by calling madvise for memory regions. To be able to use KSM for more workloads, KSM needs to have the ability to be enabled / disabled at the process / cgroup level.

I'll toss this in for integration and testing, but I'd like to see reviewer input before proceeding further.

Please plan on adding suitable user-facing documentation? Presumably a patch for the prctl manpage?

So far KSM can only be enabled by calling madvise for memory regions. To be able to use KSM for more workloads, KSM needs to have the ability to be enabled / disabled at the process / cgroup level.

Use case 1:
The madvise call is not available in the programming language. An example for this are programs with forked workloads using a garbage collected language without pointers. In such a language madvise cannot be made available.

In addition the addresses of objects get moved around as they are garbage collected. KSM sharing needs to be enabled "from the outside" for these type of workloads.

Use case 2:
The same interpreter can also be used for workloads where KSM brings no benefit or even has overhead. We'd like to be able to enable KSM on a workload by workload basis.

Use case 3:
With the madvise call sharing opportunities are only enabled for the current
process: it is a workload-local decision. A considerable number of sharing opportuniites may exist across multiple workloads or jobs. Only a higler level entity like a job scheduler or container can know for certain if its running one or more instances of a job. That job scheduler however doesn't have the necessary internal worklaod knowledge to make targeted madvise calls.

Security concerns:
In previous discussions security concerns have been brought up. The problem is that an individual workload does not have the knowledge about what else is running on a machine. Therefore it has to be very conservative in what memory areas can be shared or not. However, if the system is dedicated to running multiple jobs within the same security domain, its the job scheduler that has the knowledge that sharing can be safely enabled and is even desirable.

For some reason gmail thought it would be a good ideas to move this into the SPAM folder, so I only saw the recent replies just now.

I'm going to have a look at this soonish.

One point that popped up in the past and that I raised on the last RFC: we should think about letting processes *opt out/disable* KSM on their own. Either completely, or for selected VMAs.

Reasoning is, that if you have an application that really doesn't want some memory regions to be applicable to KSM (memory de-duplication attacks? Knowing that KSM on some regions will be counter-productive)

For example, remembering if MADV_UNMERGEABLE was called and not only clearing the VMA flag. So even if KSM would be force-enabled by some tooling after the process started, such regions would not get considered for KSM.

It would a bit like how we handle THP.

Quoted 26 lines

So far KSM can only be enabled by calling madvise for memory regions. To be able to use KSM for more workloads, KSM needs to have the ability to be enabled / disabled at the process / cgroup level.

Use case 1:
The madvise call is not available in the programming language. An example for this are programs with forked workloads using a garbage collected language without pointers. In such a language madvise cannot be made available.

In addition the addresses of objects get moved around as they are garbage collected. KSM sharing needs to be enabled "from the outside" for these type of workloads.

Use case 2:
The same interpreter can also be used for workloads where KSM brings no benefit or even has overhead. We'd like to be able to enable KSM on a workload by workload basis.

Use case 3:
With the madvise call sharing opportunities are only enabled for the current
process: it is a workload-local decision. A considerable number of sharing opportuniites may exist across multiple workloads or jobs. Only a higler level entity like a job scheduler or container can know for certain if its running one or more instances of a job. That job scheduler however doesn't have the necessary internal worklaod knowledge to make targeted madvise calls.

Quoted 7 lines

Security concerns:
In previous discussions security concerns have been brought up. The problem is that an individual workload does not have the knowledge about what else is running on a machine. Therefore it has to be very conservative in what memory areas can be shared or not. However, if the system is dedicated to running multiple jobs within the same security domain, its the job scheduler that has the knowledge that sharing can be safely enabled and is even desirable.

Note that there are some papers about why limiting memory deduplciation attacks to single security domains is not sufficient. Especially, the remote deduplication attacks fall into that category IIRC.

Quoted 13 lines

One point that popped up in the past and that I raised on the last RFC: we should think about letting processes *opt out/disable* KSM on their own. Either completely, or for selected VMAs.

Reasoning is, that if you have an application that really doesn't want some memory regions to be applicable to KSM (memory de-duplication attacks? Knowing that KSM on some regions will be counter-productive)

For example, remembering if MADV_UNMERGEABLE was called and not only clearing the VMA flag. So even if KSM would be force-enabled by some tooling after the process started, such regions would not get considered for KSM.

Quoted 1 line

It would a bit like how we handle THP.

I'm not sure the THP comparison is apt. THP is truly a local optimization that depends on the workload's access patterns. The environment isn't a true factor. It makes some sense that if there is a global policy to generally use THP the workload be able to opt out based on known sparse access patterns. At least until THP allocation strategy inside the kernel becomes smarter!

Merging opportunities and security questions are trickier. The application might know which data is sensitive, but it doesn't know whether its environment is safe or subject do memory attacks, so it cannot make that decision purely from inside.

There is a conceivable usecase where multiple instances of the same job are running inside a safe shared security domain and using the same sensitive data.

There is a conceivable usecase where the system and the workload collaborate to merge insensitive data across security domains.

I'm honestly not sure which usecase is more likely. My gut feeling is the first one, simply because of broader concerns of multiple security domains sharing kernel instances or physical hardware.

Quoted 35 lines · Stefan Roesch

Quoted 33 lines

So far KSM can only be enabled by calling madvise for memory regions. To be able to use KSM for more workloads, KSM needs to have the ability to be enabled / disabled at the process / cgroup level.

Use case 1:
The madvise call is not available in the programming language. An example for this are programs with forked workloads using a garbage collected language without pointers. In such a language madvise cannot be made available.

In addition the addresses of objects get moved around as they are garbage collected. KSM sharing needs to be enabled "from the outside" for these type of workloads.

Use case 2:
The same interpreter can also be used for workloads where KSM brings no benefit or even has overhead. We'd like to be able to enable KSM on a workload by workload basis.

Use case 3:
With the madvise call sharing opportunities are only enabled for the current
process: it is a workload-local decision. A considerable number of sharing opportuniites may exist across multiple workloads or jobs. Only a higler level entity like a job scheduler or container can know for certain if its running one or more instances of a job. That job scheduler however doesn't have the necessary internal worklaod knowledge to make targeted madvise calls.

Security concerns:
In previous discussions security concerns have been brought up. The problem is that an individual workload does not have the knowledge about what else is running on a machine. Therefore it has to be very conservative in what memory areas can be shared or not. However, if the system is dedicated to running multiple jobs within the same security domain, its the job scheduler that has the knowledge that sharing can be safely enabled and is even desirable.

Quoted 3 lines

Note that there are some papers about why limiting memory deduplciation attacks to single security domains is not sufficient. Especially, the remote deduplication attacks fall into that category IIRC.

I think it would be good to elaborate on that and include any caveats in the documentation.

Ultimately, the bar isn't whether there are attack vectors on a subset of possible usecases, but whether there are usecases where this can be used safely, which is obviously true.