mm/ksm: Fix incorrect accounting of KSM counters during fork.

Currently, the KSM-related counters in mm_struct, such as ksm_merging_pages, ksm_rmap_items, and ksm_zero_pages, are inherited by the child process during fork. This results in inconsistent accounting.

When a process uses KSM, identical pages are merged and an rmap item is created for each merged page. The ksm_merging_pages and ksm_rmap_items counters are updated accordingly. However, after a fork, these counters are copied to the child while the corresponding rmap items are not. As a result, when the child later triggers an unmerge, there are no rmap items present in the child, so the counters remain stale, leading to incorrect accounting.

A similar issue exists with ksm_zero_pages, which maintains both a global counter and a per-process counter. During fork, the per-process counter is inherited by the child, but the global counter is not incremented. Since the child also references zero pages, the global counter should be updated as well. Otherwise, during zero-page unmerge, both the global and per-process counters are decremented, causing the global counter to become inconsistent.

To fix this, ksm_merging_pages and ksm_rmap_items are reset to 0 during fork, and the global ksm_zero_pages counter is updated with the per-process ksm_zero_pages value inherited by the child. This ensures that KSM statistics remain accurate and reflect the activity of each process correctly.

Fixes 7609385337a4 ("ksm: count ksm merging pages for each process")

Fixes cb4df4cae4f2 ("ksm: count allocated ksm rmap_items for each process")

Fixes e2942062e01d ("ksm: count all zero pages placed by KSM")

cc [email protected] # v6.6

On Mon, 15 Sep 2025 20:33:04 +0530 Donet Tom <[email protected]> wrote:

Quoted 1 line

Fixes 7609385337a4 ("ksm: count ksm merging pages for each process")

Linux-v5.19

Quoted 1 line

Fixes cb4df4cae4f2 ("ksm: count allocated ksm rmap_items for each process")

Linux-v6.1

Quoted 1 line

Fixes e2942062e01d ("ksm: count all zero pages placed by KSM")

Linux-v6.10

Quoted 1 line

cc [email protected] # v6.6

So how was Linux-v6.6 arrived at?

(Cc Joe. Should checkpatch say something about this)?

Quoted 1 line · Andrew Morton

On Mon, 15 Sep 2025 20:33:04 +0530 Donet Tom <[email protected]> wrote:

[]

Quoted 12 lines

Quoted 1 line

Fixes 7609385337a4 ("ksm: count ksm merging pages for each process")

Linux-v5.19

Quoted 1 line

Fixes cb4df4cae4f2 ("ksm: count allocated ksm rmap_items for each process")

Linux-v6.1

Quoted 1 line

Fixes e2942062e01d ("ksm: count all zero pages placed by KSM")

Linux-v6.10

Quoted 3 lines

Quoted 1 line

cc [email protected] # v6.6

So how was Linux-v6.6 arrived at?

[]

Quoted 1 line

(Cc Joe. Should checkpatch say something about this)?

Probably not. Parsing variants of versions seems, umm, difficult.

Quoted 21 lines · On Mon, 15 Sep 2025 20:33:04 +0530 Donet Tom <[email protected]> wrote:

Currently, the KSM-related counters in mm_struct, such as ksm_merging_pages, ksm_rmap_items, and ksm_zero_pages, are inherited by the child process during fork. This results in inconsistent accounting.

When a process uses KSM, identical pages are merged and an rmap item is created for each merged page. The ksm_merging_pages and ksm_rmap_items counters are updated accordingly. However, after a fork, these counters are copied to the child while the corresponding rmap items are not. As a result, when the child later triggers an unmerge, there are no rmap items present in the child, so the counters remain stale, leading to incorrect accounting.

A similar issue exists with ksm_zero_pages, which maintains both a global counter and a per-process counter. During fork, the per-process counter is inherited by the child, but the global counter is not incremented. Since the child also references zero pages, the global counter should be updated as well. Otherwise, during zero-page unmerge, both the global and per-process counters are decremented, causing the global counter to become inconsistent.

Quoted 7 lines

To fix this, ksm_merging_pages and ksm_rmap_items are reset to 0 during fork, and the global ksm_zero_pages counter is updated with the per-process ksm_zero_pages value inherited by the child. This ensures that KSM statistics remain accurate and reflect the activity of each process correctly.

Fixes 7609385337a4 ("ksm: count ksm merging pages for each process")

Linux-v5.19

Quoted 1 line

Fixes cb4df4cae4f2 ("ksm: count allocated ksm rmap_items for each process")

Linux-v6.1

Quoted 1 line

Fixes e2942062e01d ("ksm: count all zero pages placed by KSM")

Linux-v6.10

Quoted 1 line

cc [email protected] # v6.6

So how was Linux-v6.6 arrived at?

I think the most important use for Fixes: is to tell the -stable maintainers which kernel version(s) we believe should receive the patch. So listing multiple Fixes: targets just causes confusion.

Quoted 39 lines · Andrew Morton

Quoted 21 lines · On Mon, 15 Sep 2025 20:33:04 +0530 Donet Tom <[email protected]> wrote:

Currently, the KSM-related counters in mm_struct, such as ksm_merging_pages, ksm_rmap_items, and ksm_zero_pages, are inherited by the child process during fork. This results in inconsistent accounting.

When a process uses KSM, identical pages are merged and an rmap item is created for each merged page. The ksm_merging_pages and ksm_rmap_items counters are updated accordingly. However, after a fork, these counters are copied to the child while the corresponding rmap items are not. As a result, when the child later triggers an unmerge, there are no rmap items present in the child, so the counters remain stale, leading to incorrect accounting.

A similar issue exists with ksm_zero_pages, which maintains both a global counter and a per-process counter. During fork, the per-process counter is inherited by the child, but the global counter is not incremented. Since the child also references zero pages, the global counter should be updated as well. Otherwise, during zero-page unmerge, both the global and per-process counters are decremented, causing the global counter to become inconsistent.

Quoted 7 lines

To fix this, ksm_merging_pages and ksm_rmap_items are reset to 0 during fork, and the global ksm_zero_pages counter is updated with the per-process ksm_zero_pages value inherited by the child. This ensures that KSM statistics remain accurate and reflect the activity of each process correctly.

Fixes 7609385337a4 ("ksm: count ksm merging pages for each process")

Linux-v5.19

Quoted 1 line

Fixes cb4df4cae4f2 ("ksm: count allocated ksm rmap_items for each process")

Linux-v6.1

Quoted 1 line

Fixes e2942062e01d ("ksm: count all zero pages placed by KSM")

Quoted 5 lines

Linux-v6.10

Quoted 1 line

cc [email protected] # v6.6

So how was Linux-v6.6 arrived at?

e2942062e01d is in v6.6, not in v6.10 - I suspect that this is why the "# v6.6" part was added.

Quoted 4 lines

I think the most important use for Fixes: is to tell the -stable maintainers which kernel version(s) we believe should receive the patch. So listing multiple Fixes: targets just causes confusion.

Right - there's no way of communicating if all the commits listed in multiple Fixes tags should exist in the tree, or any one of them, for the new fix to be applicable.

Quoted 8 lines · On Tue, 16 Sep 2025 00:33:09 -0400 Sasha Levin <[email protected]> wrote:

Quoted 3 lines · Andrew Morton

I think the most important use for Fixes: is to tell the -stable maintainers which kernel version(s) we believe should receive the patch. So listing multiple Fixes: targets just causes confusion.

Right - there's no way of communicating if all the commits listed in multiple Fixes tags should exist in the tree, or any one of them, for the new fix to be applicable.

So what should we do in this situation?

Quoted 39 lines · Andrew Morton

Quoted 21 lines · On Mon, 15 Sep 2025 20:33:04 +0530 Donet Tom <[email protected]> wrote:

Currently, the KSM-related counters in mm_struct, such as ksm_merging_pages, ksm_rmap_items, and ksm_zero_pages, are inherited by the child process during fork. This results in inconsistent accounting.

When a process uses KSM, identical pages are merged and an rmap item is created for each merged page. The ksm_merging_pages and ksm_rmap_items counters are updated accordingly. However, after a fork, these counters are copied to the child while the corresponding rmap items are not. As a result, when the child later triggers an unmerge, there are no rmap items present in the child, so the counters remain stale, leading to incorrect accounting.

A similar issue exists with ksm_zero_pages, which maintains both a global counter and a per-process counter. During fork, the per-process counter is inherited by the child, but the global counter is not incremented. Since the child also references zero pages, the global counter should be updated as well. Otherwise, during zero-page unmerge, both the global and per-process counters are decremented, causing the global counter to become inconsistent.

Quoted 7 lines

To fix this, ksm_merging_pages and ksm_rmap_items are reset to 0 during fork, and the global ksm_zero_pages counter is updated with the per-process ksm_zero_pages value inherited by the child. This ensures that KSM statistics remain accurate and reflect the activity of each process correctly.

Fixes 7609385337a4 ("ksm: count ksm merging pages for each process")

Linux-v5.19

Quoted 1 line

Fixes cb4df4cae4f2 ("ksm: count allocated ksm rmap_items for each process")

Linux-v6.1

Quoted 1 line

Fixes e2942062e01d ("ksm: count all zero pages placed by KSM")

Quoted 5 lines

Linux-v6.10

Quoted 1 line

cc [email protected] # v6.6

So how was Linux-v6.6 arrived at?

e2942062e01d is in v6.6, not in v6.10 - I suspect that this is why the "# v6.6" part was added.

Quoted 4 lines

I think the most important use for Fixes: is to tell the -stable maintainers which kernel version(s) we believe should receive the patch.  So listing multiple Fixes: targets just causes confusion.

Right - there's no way of communicating if all the commits listed in multiple Fixes tags should exist in the tree, or any one of them, for the new fix to be applicable.

Currently, the KSM-related counters in mm_struct, such as ksm_merging_pages, ksm_rmap_items, and ksm_zero_pages, are inherited by the child process during fork. This results in inconsistent accounting.

When a process uses KSM, identical pages are merged and an rmap item is created for each merged page. The ksm_merging_pages and ksm_rmap_items counters are updated accordingly. However, after a fork, these counters are copied to the child while the corresponding rmap items are not. As a result, when the child later triggers an unmerge, there are no rmap items present in the child, so the counters remain stale, leading to incorrect accounting.

A similar issue exists with ksm_zero_pages, which maintains both a global counter and a per-process counter. During fork, the per-process counter is inherited by the child, but the global counter is not incremented. Since the child also references zero pages, the global counter should be updated as well. Otherwise, during zero-page unmerge, both the global and per-process counters are decremented, causing the global counter to become inconsistent.

To fix this, ksm_merging_pages and ksm_rmap_items are reset to 0 during fork, and the global ksm_zero_pages counter is updated with the per-process ksm_zero_pages value inherited by the child. This ensures that KSM statistics remain accurate and reflect the activity of each process correctly.

Fixes 7609385337a4 ("ksm: count ksm merging pages for each process")

Fixes cb4df4cae4f2 ("ksm: count allocated ksm rmap_items for each process")

Fixes e2942062e01d ("ksm: count all zero pages placed by KSM")

cc [email protected] # v6.6

Signed-off-by Donet Tom <[email protected]>

---
  include/linux/ksm.h | 8 +++++++-
  1 file changed, 7 insertions(+), 1 deletion(-)

include/linux/ksm.h +0 -0

index 22e67ca7cba3..067538fc4d58 100644
@@ -56,8 +56,14 @@ static inline long mm_ksm_zero_pages(struct mm_struct *mm)
  static inline void ksm_fork(struct mm_struct *mm, struct mm_struct *oldmm)

Currently, the KSM-related counters in mm_struct, such as ksm_merging_pages, ksm_rmap_items, and ksm_zero_pages, are inherited by the child process during fork. This results in inconsistent accounting.

When a process uses KSM, identical pages are merged and an rmap item is created for each merged page. The ksm_merging_pages and ksm_rmap_items counters are updated accordingly. However, after a fork, these counters are copied to the child while the corresponding rmap items are not. As a result, when the child later triggers an unmerge, there are no rmap items present in the child, so the counters remain stale, leading to incorrect accounting.

A similar issue exists with ksm_zero_pages, which maintains both a global counter and a per-process counter. During fork, the per-process counter is inherited by the child, but the global counter is not incremented. Since the child also references zero pages, the global counter should be updated as well. Otherwise, during zero-page unmerge, both the global and per-process counters are decremented, causing the global counter to become inconsistent.

To fix this, ksm_merging_pages and ksm_rmap_items are reset to 0 during fork, and the global ksm_zero_pages counter is updated with the per-process ksm_zero_pages value inherited by the child. This ensures that KSM statistics remain accurate and reflect the activity of each process correctly.

Fixes 7609385337a4 ("ksm: count ksm merging pages for each process")

Fixes cb4df4cae4f2 ("ksm: count allocated ksm rmap_items for each process")

Fixes e2942062e01d ("ksm: count all zero pages placed by KSM")

cc [email protected] # v6.6

Signed-off-by Donet Tom <[email protected]>

---
  include/linux/ksm.h | 8 +++++++-
  1 file changed, 7 insertions(+), 1 deletion(-)

include/linux/ksm.h +7 -1

index 22e67ca7cba3..067538fc4d58 100644
@@ -56,8 +56,14 @@ static inline long mm_ksm_zero_pages(struct mm_struct *mm)
  static inline void ksm_fork(struct mm_struct *mm, struct mm_struct *oldmm)
  {
  	/* Adding mm to ksm is best effort on fork. */
-	if (mm_flags_test(MMF_VM_MERGEABLE, oldmm))
+	if (mm_flags_test(MMF_VM_MERGEABLE, oldmm)) {
+		long nr_ksm_zero_pages = atomic_long_read(&mm->ksm_zero_pages);
+
+		mm->ksm_merging_pages = 0;
+		mm->ksm_rmap_items = 0;
+		atomic_long_add(nr_ksm_zero_pages, &ksm_zero_pages);
  		__ksm_enter(mm);
+	}
  }
 
  static inline int ksm_execve(struct mm_struct *mm)

Added a new selftest to verify whether the ksm_merging_pages counter in mm_struct is not inherited by a child process after fork. This helps ensure correctness of KSM accounting across process creation.

Signed-off-by Donet Tom <[email protected]>

---
  .../selftests/mm/ksm_functional_tests.c       | 42 ++++++++++++++++++-
  1 file changed, 41 insertions(+), 1 deletion(-)

tools/testing/selftests/mm/ksm_functional_tests.c +10 -0

index 712f43c87736..645cefba2126 100644
@@ -602,6 +602,45 @@ static void test_prot_none(void)
  	munmap(map, size);
  }
 
+static void test_fork_ksm_merging_page_count(void)
+{
+	const unsigned int size = 2 * MiB;
+	char *map;
+	pid_t child_pid;
+	int status;
+
+	ksft_print_msg("[RUN] %s\n", __func__);
+
+	map = mmap_and_merge_range(0xcf, size, PROT_READ | PROT_WRITE, KSM_MERGE_MADVISE);

tools/testing/selftests/mm/ksm_functional_tests.c +5 -0

+	}
+
+	if (waitpid(child_pid, &status, 0) < 0) {
+		ksft_test_result_fail("waitpid() failed\n");
+		return;

Quoted 27 lines · Donet Tom

Added a new selftest to verify whether the ksm_merging_pages counter in mm_struct is not inherited by a child process after fork. This helps ensure correctness of KSM accounting across process creation.

Signed-off-by Donet Tom <[email protected]>

---
  .../selftests/mm/ksm_functional_tests.c       | 42 ++++++++++++++++++-

  1 file changed, 41 insertions(+), 1 deletion(-)

tools/testing/selftests/mm/ksm_functional_tests.c +9 -0

index 712f43c87736..645cefba2126 100644
@@ -602,6 +602,45 @@ static void test_prot_none(void)
       munmap(map, size);
   }
   +static void test_fork_ksm_merging_page_count(void)
+{
+    const unsigned int size = 2 * MiB;
+    char *map;
+    pid_t child_pid;
+    int status;
+
+    ksft_print_msg("[RUN] %s\n", __func__);
+
+    map = mmap_and_merge_range(0xcf, size, PROT_READ | PROT_WRITE, 
 KSM_MERGE_MADVISE);

tools/testing/selftests/mm/ksm_functional_tests.c static void test_prot_none(void) · line 624

Quoted 10 lines

tools/testing/selftests/mm/ksm_functional_tests.c +10 -0

+    if (map == MAP_FAILED)
+        return;
+
+    child_pid = fork();
+    if (!child_pid) {
+        init_global_file_handles();
+        exit(ksm_get_self_merging_pages());
+    } else if (child_pid < 0) {
+        ksft_test_result_fail("fork() failed\n");
+        return;

Probably
    goto out;

in all these fail cases

Quoted 7 lines

tools/testing/selftests/mm/ksm_functional_tests.c +7 -0

+    }
+
+    if (waitpid(child_pid, &status, 0) < 0) {
+        ksft_test_result_fail("waitpid() failed\n");
+        return;
+    }
+

tools/testing/selftests/mm/ksm_functional_tests.c static void test_prot_none(void) · line 639

Quoted 10 lines

tools/testing/selftests/mm/ksm_functional_tests.c +8 -0

+    status = WEXITSTATUS(status);
+    if (status) {
+        ksft_test_result_fail("ksm_merging_page in child: %d\n", 
 status);
+        return;
+    }
+
+    ksft_test_result_pass("ksm_merging_pages is not inherited after 
 fork\n");
+

And have

    out:

here

With that:

Acked-by David Hildenbrand <[email protected]>

Added a selftest to verify the behavior of the global KSM zero-page counter during fork. When a process forks, the per-process zero-page counter is inherited by the child, and the global counter should be updated with this inherited value. This test ensures that the global counter is correctly updated after fork.

Signed-off-by Donet Tom <[email protected]>

---
  .../selftests/mm/ksm_functional_tests.c       | 74 ++++++++++++++++++-
  1 file changed, 73 insertions(+), 1 deletion(-)

tools/testing/selftests/mm/ksm_functional_tests.c +26 -0

index 645cefba2126..f23597ac8066 100644
@@ -602,6 +602,77 @@ static void test_prot_none(void)
  	munmap(map, size);
  }
 
+long ksm_get_global_ksm_zero_pages(void)
+{
+	int global_ksm_zero_pages_fd;
+	char buf[10];
+	ssize_t ret;
+
+	global_ksm_zero_pages_fd = open("/sys/kernel/mm/ksm/ksm_zero_pages",
+								O_RDONLY);
+	if (global_ksm_zero_pages_fd < 0)
+		return -errno;
+
+	ret = pread(global_ksm_zero_pages_fd, buf, sizeof(buf) - 1, 0);
+	close(global_ksm_zero_pages_fd);
+	if (ret <= 0)
+		return -errno;
+	buf[ret] = 0;
+
+	return strtol(buf, NULL, 10);
+}
+
+static void test_fork_global_ksm_zero_pages_count(void)
+{
+	const unsigned int size = 2 * MiB;
+	char *map;
+	pid_t child_pid;
+	int status;

tools/testing/selftests/mm/ksm_functional_tests.c +11 -0

+	}
+	if (waitpid(child_pid, &status, 0) < 0) {
+		ksft_test_result_fail("waitpid() failed\n");
+		return;
+	}
+	status = WEXITSTATUS(status);
+	if (status < 0) {
+		ksft_test_result_fail("KSM unmerging failed in child\n");
+		return;
+	}
+

Quoted 45 lines · Donet Tom

Added a selftest to verify the behavior of the global KSM zero-page counter during fork. When a process forks, the per-process zero-page counter is inherited by the child, and the global counter should be updated with this inherited value. This test ensures that the global counter is correctly updated after fork.

Signed-off-by Donet Tom <[email protected]>

---
  .../selftests/mm/ksm_functional_tests.c       | 74 ++++++++++++++++++-

  1 file changed, 73 insertions(+), 1 deletion(-)

tools/testing/selftests/mm/ksm_functional_tests.c +25 -0

index 645cefba2126..f23597ac8066 100644
@@ -602,6 +602,77 @@ static void test_prot_none(void)
       munmap(map, size);
   }
   +long ksm_get_global_ksm_zero_pages(void)
+{
+    int global_ksm_zero_pages_fd;
+    char buf[10];
+    ssize_t ret;
+
+    global_ksm_zero_pages_fd = 
 open("/sys/kernel/mm/ksm/ksm_zero_pages",
+                                O_RDONLY);
+    if (global_ksm_zero_pages_fd < 0)
+        return -errno;
+
+    ret = pread(global_ksm_zero_pages_fd, buf, sizeof(buf) - 1, 0);
+    close(global_ksm_zero_pages_fd);
+    if (ret <= 0)
+        return -errno;
+    buf[ret] = 0;
+
+    return strtol(buf, NULL, 10);
+}
+
+static void test_fork_global_ksm_zero_pages_count(void)
+{
+    const unsigned int size = 2 * MiB;
+    char *map;
+    pid_t child_pid;
+    int status;

tools/testing/selftests/mm/ksm_functional_tests.c static void test_prot_none(void) · line 640

Quoted 10 lines

tools/testing/selftests/mm/ksm_functional_tests.c +10 -0

+    long g_zpages_before = 0, g_zpages_after = 0;
+
+    ksft_print_msg("[RUN] %s\n", __func__);
+
+    /* Unmerge all pages before test */
+    if (ksm_stop() < 0) {
+        ksft_test_result_fail("KSM unmerging failed\n");
+        return;
+    }
+    /* Get the global zero page count before test */

That only works when "use_zero_pages" is enabled, no?

tools/testing/selftests/mm/ksm_functional_tests.c static void test_prot_none(void) · line 652

Quoted 13 lines

tools/testing/selftests/mm/ksm_functional_tests.c +12 -0

+    g_zpages_before = ksm_get_global_ksm_zero_pages();
+    /* Let KSM deduplicate zero pages. */
+    map = mmap_and_merge_range(0x00, size, PROT_READ | PROT_WRITE, 
 KSM_MERGE_MADVISE);
+    if (map == MAP_FAILED)
+        return;
+
+    child_pid = fork();
+    if (!child_pid) {
+        exit(ksm_stop());
+    } else if (child_pid < 0) {
+        ksft_test_result_fail("fork() failed\n");
+        return;

Cleanup missing as for patch #2.

Quoted 20 lines

tools/testing/selftests/mm/ksm_functional_tests.c +19 -0

+    }
+    if (waitpid(child_pid, &status, 0) < 0) {
+        ksft_test_result_fail("waitpid() failed\n");
+        return;
+    }
+    status = WEXITSTATUS(status);
+    if (status < 0) {
+        ksft_test_result_fail("KSM unmerging failed in child\n");
+        return;
+    }
+
+    /* Verify global zero-page count remains unchanged */
+    g_zpages_after = ksm_get_global_ksm_zero_pages();
+    if (g_zpages_before != g_zpages_after) {
+        ksft_test_result_fail("Incorrect global ksm zero page count 
 after fork\n");
+        return;
+    }
+
+    ksft_test_result_pass("Global ksm zero page count is correct 

Quoted 3 lines

after fork\n");
+    ksm_stop();

What stops KSM from merging pages in other processes by accident concurrently and giving false failures?

Likely you would want to stop ksm before the fork.