Skip to contentSkip to conversation

mm: vmpressure: scale sampling window with machine size

PATCH Superseded deepakraog 4 participants 11 messages Jul 15 – Jul 22

Subsystems (7)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

deepakraog

author
#0 lore raw

PATCH 1/4 mm: vmpressure: scale sampling window with machine size

The vmpressure window size was fixed at 512 pages (SWAP_CLUSTER_MAX * 16). Scale it at boot from total memory and CPU count using the same fls-based formula as vmstat per-zone thresholds, clamped between 128 and 2048 pages.

---
 mm/vmpressure.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)
mm/vmpressure.c +22 -2
index f053554e5..8925d4ad3 100644
@@ -33,9 +33,29 @@
  * SWAP_CLUSTER_MAX, it makes sense to use it for the window size as well.
  *
  * TODO: Make the window size depend on machine size, as we do for vmstat
- * thresholds. Currently we set it to 512 pages (2MB for 4KB pages).
+ * thresholds. Scales with CPU count and total memory at boot.
  */
-static const unsigned long vmpressure_win = SWAP_CLUSTER_MAX * 16;
+static unsigned long vmpressure_win;
+
+static unsigned long __init vmpressure_window_size(void)
+{
+	unsigned long win;
+	int mem;
+
+	mem = totalram_pages() >> (27 - PAGE_SHIFT);
+	win = SWAP_CLUSTER_MAX * (unsigned long)fls(num_online_cpus()) *
+	      (1 + fls(mem));
+	win = max(win, SWAP_CLUSTER_MAX * 4UL);
+	win = min(win, SWAP_CLUSTER_MAX * 64UL);
+	return win;
+}
+
+static int __init vmpressure_win_init(void)
+{
+	vmpressure_win = vmpressure_window_size();
+	return 0;
+}
+core_initcall(vmpressure_win_init);
 
 /*
  * These thresholds are used when we account memory pressure through
Signature · 1 line
Deepak Rao Gaikwad
author
#1 lore raw

PATCH 2/4 selftests/proc: expand smaps field coverage and require TMPFS

Gate the test on CONFIG_TMPFS in the selftest config file. Assert Private_Clean, Private_Dirty, Swap, and SwapPss fields are present in /proc/$PID/smaps output for the mapped executable page.

---
 tools/testing/selftests/proc/config        | 1 +
 tools/testing/selftests/proc/proc-pid-vm.c | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)
tools/testing/selftests/proc/config +1 -0
index 68fbd2b35..7dfc3d381 100644
@@ -1 +1,2 @@
 CONFIG_PROC_FS=y
+CONFIG_TMPFS=y
tools/testing/selftests/proc/proc-pid-vm.c +4 -1
index 4e6a3e53f..59535bb3a 100644
@@ -20,7 +20,6 @@
  * Test /proc/$PID/smaps_rollup
  * Test /proc/$PID/statm
  *
- * FIXME require CONFIG_TMPFS which can be disabled
  * FIXME test other values from "smaps"
  * FIXME support other archs
  */
@@ -409,6 +408,10 @@ int main(void)
 			"AnonHugePages:         0 kB\n",
 			"Shared_Hugetlb:        0 kB\n",
 			"Private_Hugetlb:       0 kB\n",
+			"Private_Clean:         4 kB\n",
+			"Private_Dirty:         0 kB\n",
+			"Swap:                  0 kB\n",
+			"SwapPss:               0 kB\n",
 			"Locked:                0 kB\n",
 		};
 		int i;
Signature · 1 line
Deepak Rao Gaikwad
author
#2 lore raw

PATCH 3/4 mm: vrealloc: grow vm_area mappings in place

In reply to deepakraog

When vrealloc() needs more pages than are currently mapped, allocate and map additional pages at the end of the existing vm_area instead of always allocating a new area and memcpy()'ing. Fall back to the existing realloc path when in-place grow is not possible.

---
 mm/vmalloc.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)
mm/vmalloc.c +49 -1
index 1afca3568..46cf10c62 100644
@@ -4426,8 +4426,56 @@ void *vrealloc_node_align_noprof(const void *p, size_t size, unsigned long align
 		return (void *)p;
 	}
 
+	{
+		unsigned int new_nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT;
+		unsigned int old_nr_pages = vm->nr_pages;
+
+		if (new_nr_pages > old_nr_pages && !vm_area_page_order(vm) &&
+		    !(vm->flags & (VM_FLUSH_RESET_PERMS | VM_USERMAP)) &&
+		    gfp_has_io_fs(flags)) {
+			unsigned int extra = new_nr_pages - old_nr_pages;
+			unsigned long addr = (unsigned long)kasan_reset_tag(p);
+			struct vmap_node *vn = addr_to_node(addr);
+			struct page **pages;
+			int ret;
+
+			pages = kvmalloc_array(extra, sizeof(struct page *), gfp);
+			if (!pages)
+				goto need_realloc;
+
+			if (vm_area_alloc_pages(vmalloc_gfp_adjust(gfp, 0), nid,
+						0, extra, pages) != extra) {
+				kvfree(pages);
+				goto need_realloc;
+			}
+
+			spin_lock(&vn->busy.lock);
+			ret = vmap_pages_range(addr + ((unsigned long)old_nr_pages
+						       << PAGE_SHIFT),
+					       addr + ((unsigned long)new_nr_pages
+						       << PAGE_SHIFT),
+					       PAGE_KERNEL, pages, PAGE_SHIFT);
+			spin_unlock(&vn->busy.lock);
+			if (ret < 0) {
+				free_pages_bulk(pages, extra);
+				kvfree(pages);
+				goto need_realloc;
+			}
+
+			memcpy(&vm->pages[old_nr_pages], pages,
+			       extra * sizeof(struct page *));
+			kvfree(pages);
+			vm->nr_pages = new_nr_pages;
+			vm->requested_size = size;
+			kasan_vrealloc(p, old_size, size);
+			if (want_init_on_alloc(flags))
+				memset((void *)p + old_size, 0, size - old_size);
+			return (void *)p;
+		}
+	}
+
 need_realloc:
-	/* TODO: Grow the vm_area, i.e. allocate and map additional pages. */
+	/* Fall back to a fresh vm_area when in-place grow is not possible. */
 	n = __vmalloc_node_noprof(size, align, flags, nid, __builtin_return_address(0));
 
 	if (!n)
Signature · 1 line
Deepak Rao Gaikwad
author
#3 lore raw

PATCH 4/4 ceph: release unused cap reservation on readdir error

In reply to deepakraog

ceph_readdir_prepopulate() can exit with an error after partially consuming the request's cap reservation. Return unused reserved caps to the MDS client pool on the error path so they are not held until request teardown.

---
 fs/ceph/inode.c | 2 ++
 1 file changed, 2 insertions(+)
fs/ceph/inode.c +2 -0
index 61d7c0b81..2d6a80054 100644
@@ -2188,6 +2188,8 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req,
 		dput(dn);
 	}
 out:
+	if (err)
+		ceph_unreserve_caps(req->r_mdsc, &req->r_caps_reservation);
 	if (err == 0 && skipped == 0) {
 		set_bit(CEPH_MDS_R_DID_PREPOPULATE, &req->r_req_flags);
 		req->r_readdir_cache_idx = cache_ctl.index;
Signature · 1 line
Deepak Rao Gaikwad
#4 lore raw

Re: [PATCH 4/4] ceph: release unused cap reservation on readdir error

Quoted 6 lines · deepakraog

ceph_readdir_prepopulate() can exit with an error after partially consuming the request's cap reservation. Return unused reserved caps to
the MDS client pool on the error path so they are not held until request
teardown.

The ceph-devel list has received only 4/4 patch. Is it some patchset? If so, then why we haven't received the rest patches in series?

Do you have any issue related to the fix? How have you encountered the issue? Could you please explain more about the issue?

Thanks,
Slava.

Quoted 22 lines

---
 fs/ceph/inode.c | 2 ++

 1 file changed, 2 insertions(+)
fs/ceph/inode.c +0 -0
index 61d7c0b81..2d6a80054 100644
@@ -2188,6 +2188,8 @@ int ceph_readdir_prepopulate(struct
 ceph_mds_request *req,
  		dput(dn);
  	}

 out:

fs/ceph/inode.c +2 -0
+	if (err)
+		ceph_unreserve_caps(req->r_mdsc, &req-
Quoted 1 line

r_caps_reservation);

  if (err == 0 && skipped == 0) { set_bit(CEPH_MDS_R_DID_PREPOPULATE, &req-

Quoted 1 line

r_req_flags);

  req->r_readdir_cache_idx = cache_ctl.index;

author
#5 lore raw

Re: [PATCH 4/4] fix: release unused ceph cap reservation on readdir error

Hi Slava,

Sorry for the confusing [PATCH 4/4] subject — that was my mistake, not an intentional ceph patch series.

What happened: I batched four unrelated commits into one git send-email range. format-patch numbered them 1/4 through 4/4, but get_maintainer.pl routed each patch only to its subsystem list. ceph-devel received only patch 4.

This is a standalone one-patch fix for ceph. I have resent it as [PATCH 1/1] to ceph-devel in a separate thread.

The bug: in ceph_readdir_prepopulate(), ceph_fill_inode() consumes caps from req->r_caps_reservation. On error paths, unused caps were not returned to the MDS client pool (there is a FIXME: "release caps/leases if error occurs"). On success, ceph_unreserve_caps() is called in mds_client.c; on error it was skipped. The fix calls ceph_unreserve_caps() at out: when err != 0.

How I found it: code review while reading fs/ceph/inode.c — not a live Ceph cluster repro. Compiled fs/ceph/inode.o locally.

Thanks,
Deepak Rao Gaikwad

author
#6 lore raw

Re: [PATCH 4/4] fix: release unused ceph cap reservation on readdir error

Hi Slava,

A quick clarification on my testing note from the previous reply — I worded it poorly.

Testing: I compile-smoke-tested this change via ./scripts/vm-test.sh quick in a Lima Ubuntu 24.04 arm64 VM before sending (checkpatch on the commit plus building fs/ceph/inode.o along with the other touched objects in that script). The run passed on 2026-07-15T18:25:58Z (2026-07-15 23:55:58 IST).

I do not have a Ceph cluster/MicroCeph environment to runtime-reproduce the readdir error path or observe cap unreservation behavior under failure — that remains static analysis from code review.

No further patch changes from this; the [PATCH 1/1] submission stands.

Thanks,
Deepak Rao Gaikwad

reviewer
#7 lore raw
In reply to deepakraog
Quoted 3 lines · deepakraog

The vmpressure window size was fixed at 512 pages (SWAP_CLUSTER_MAX * 16). Scale it at boot from total memory and CPU count using the same fls-based formula as vmstat per-zone thresholds, clamped between 128 and 2048 pages.

This is not giving us any rationale why this is needed or desirable.

Signature · 2 lines
Michal Hocko
SUSE Labs
#8 lore raw
Quoted 6 lines · Michal Hocko
Quoted 3 lines · deepakraog

The vmpressure window size was fixed at 512 pages (SWAP_CLUSTER_MAX * 16). Scale it at boot from total memory and CPU count using the same fls-based formula as vmstat per-zone thresholds, clamped between 128 and 2048 pages.

This is not giving us any rationale why this is needed or desirable.

Agreed, the series are all somewhat corrupted here (seemingly some patches missing + not sent to the same list of people, unrelated stuff in the same series, etc.).

I replied to the v2 of this ([0]), in general this is essentially the same thing submitted by a previous person ([1]), and Deepak acked that ([2]) and agreed that this isn't worth pursuing.

Cheers, Lorenzo

[0]:https://lore.kernel.org/linux-mm/aljYxLusTzZkTfnH@lucifer/ [1]:https://lore.kernel.org/all/[email protected]/ [2]:https://lore.kernel.org/all/[email protected]/

Quoted 4 lines

--
Michal Hocko
SUSE Labs

author
#9 lore raw

PATCH 1/1 fix: release unused ceph cap reservation on readdir error

Return unused reserved caps to the MDS client pool when ceph_readdir_prepopulate() exits with an error.

Corrected standalone submission: this patch was mistakenly sent earlier as [PATCH 4/4] in an unrelated multi-subsystem series. It is ceph-only and resubmitted here as [PATCH 1/1].

---
 fs/ceph/inode.c | 2 ++
 1 file changed, 2 insertions(+)
fs/ceph/inode.c +2 -0
index 61d7c0b81..2d6a80054 100644
@@ -2188,6 +2188,8 @@ int ceph_readdir_prepopulate(struct ceph_mds_request *req,
 		dput(dn);
 	}
 out:
+	if (err)
+		ceph_unreserve_caps(req->r_mdsc, &req->r_caps_reservation);
 	if (err == 0 && skipped == 0) {
 		set_bit(CEPH_MDS_R_DID_PREPOPULATE, &req->r_req_flags);
 		req->r_readdir_cache_idx = cache_ctl.index;
Signature · 1 line
Deepak Rao Gaikwad
#10 lore raw

Re: [PATCH] fix: release unused ceph cap reservation on readdir error

Quoted 6 lines · deepakroag

Return unused reserved caps to the MDS client pool when ceph_readdir_prepopulate() exits with an error.

Corrected standalone submission: this patch was mistakenly sent earlier as [PATCH 4/4] in an unrelated multi-subsystem series.  It is ceph-only and resubmitted here as [PATCH 1/1].

Frankly speaking, it will be better to explain the issue detection and analysis here.

Quoted 21 lines

Link:
https://lore.kernel.org/ceph-devel/[email protected]/

---
 fs/ceph/inode.c | 2 ++

 1 file changed, 2 insertions(+)
fs/ceph/inode.c +0 -0
index 61d7c0b81..2d6a80054 100644
@@ -2188,6 +2188,8 @@ int ceph_readdir_prepopulate(struct
 ceph_mds_request *req,
  		dput(dn);
  	}

 out:

fs/ceph/inode.c +2 -0
+	if (err)
+		ceph_unreserve_caps(req->r_mdsc, &req-
Quoted 1 line

r_caps_reservation);

Quoted 4 lines

  if (err == 0 && skipped == 0) { set_bit(CEPH_MDS_R_DID_PREPOPULATE, &req-

Quoted 1 line

r_req_flags);

  req->r_readdir_cache_idx = cache_ctl.index;

OK. So, we need to do this because handle_reply() [1] execute this after ceph_readdir_prepopulate():

  if (err == 0) {
      ...
      ceph_unreserve_caps(mdsc, &req->r_caps_reservation);
  }

However, we are missing another execution branch [2]:

  if (test_bit(CEPH_MDS_R_ABORTED, &req->r_req_flags))
      return readdir_prepopulate_inodes_only(req, session);

I believe that we need to improve the patch.

Thanks,
Slava.

[1]
https://elixir.bootlin.com/linux/v7.2-rc3/source/fs/ceph/mds_client.c#L4186
[2]
https://elixir.bootlin.com/linux/v7.2-rc3/source/fs/ceph/inode.c#L2017