Skip to contentSkip to conversation

feat: scale vmpressure window with machine size

PATCH NAK deepakroag 2 participants 5 messages Jul 16 1 message not in the archive

Subsystems (6)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

deepakroag

author
#1 lore raw

PATCH v2 1/3 feat: scale vmpressure window with machine size

Replace the fixed 512-page vmpressure window with a boot-time value derived from total memory and CPU count, clamped between 128 and 2048 pages.

---
 mm/vmpressure.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)
mm/vmpressure.c +22 -2
index f053554e5..8925d4ad3 100644
@@ -33,9 +33,29 @@
  * SWAP_CLUSTER_MAX, it makes sense to use it for the window size as well.
  *
  * TODO: Make the window size depend on machine size, as we do for vmstat
- * thresholds. Currently we set it to 512 pages (2MB for 4KB pages).
+ * thresholds. Scales with CPU count and total memory at boot.
  */
-static const unsigned long vmpressure_win = SWAP_CLUSTER_MAX * 16;
+static unsigned long vmpressure_win;
+
+static unsigned long __init vmpressure_window_size(void)
+{
+	unsigned long win;
+	int mem;
+
+	mem = totalram_pages() >> (27 - PAGE_SHIFT);
+	win = SWAP_CLUSTER_MAX * (unsigned long)fls(num_online_cpus()) *
+	      (1 + fls(mem));
+	win = max(win, SWAP_CLUSTER_MAX * 4UL);
+	win = min(win, SWAP_CLUSTER_MAX * 64UL);
+	return win;
+}
+
+static int __init vmpressure_win_init(void)
+{
+	vmpressure_win = vmpressure_window_size();
+	return 0;
+}
+core_initcall(vmpressure_win_init);
 
 /*
  * These thresholds are used when we account memory pressure through
Signature · 1 line
Deepak Rao Gaikwad
#2 lore raw

NAK.

I'm not sure what you're doing here but this looks suspiciously like you're taking somebody else's (previously rejected) work and putting your name to it?

And you're sending a v2 without having sent a v1? What?

You're also getting basic process wrong here. We don't prefix with 'feat'. You only cc'd key people on 1/3 patches.

There doesn't appear to be a cover letter. You are sending random vmalloc changes along with this too?

Quoted 2 lines · deepakroag

Replace the fixed 512-page vmpressure window with a boot-time value derived from total memory and CPU count, clamped between 128 and 2048 pages.

So you're just making stuff up here? Clamp between 128 and 2048 because why not I guess? And no mention about the rest of the logic?

Quoted 3 lines

---

There was already a patch for this in [0] which looks suspiciously close to what you're doing here (modulo clamping without using a clamp macro), unacknowledged.

I hope you understand plagiarism is completely unacceptable?

In any case it was rejected then for various reasons, you re-sending the same idea without crediting that changes nothing.

Cheers, Lorenzo

[0]:https://lore.kernel.org/all/[email protected]/

Quoted 42 lines
 mm/vmpressure.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)
mm/vmpressure.c +22 -3
index f053554e5..8925d4ad3 100644
@@ -33,9 +33,29 @@
  * SWAP_CLUSTER_MAX, it makes sense to use it for the window size as well.
  *
  * TODO: Make the window size depend on machine size, as we do for vmstat
- * thresholds. Currently we set it to 512 pages (2MB for 4KB pages).
+ * thresholds. Scales with CPU count and total memory at boot.
  */
-static const unsigned long vmpressure_win = SWAP_CLUSTER_MAX * 16;
+static unsigned long vmpressure_win;
+
+static unsigned long __init vmpressure_window_size(void)
+{
+	unsigned long win;
+	int mem;
+
+	mem = totalram_pages() >> (27 - PAGE_SHIFT);
+	win = SWAP_CLUSTER_MAX * (unsigned long)fls(num_online_cpus()) *
+	      (1 + fls(mem));
+	win = max(win, SWAP_CLUSTER_MAX * 4UL);
+	win = min(win, SWAP_CLUSTER_MAX * 64UL);
+	return win;
+}
+
+static int __init vmpressure_win_init(void)
+{
+	vmpressure_win = vmpressure_window_size();
+	return 0;
+}
+core_initcall(vmpressure_win_init);
 
 /*
  * These thresholds are used when we account memory pressure through
--

Deepak Rao Gaikwad

author
#3 lore raw

Hi Lorenzo,

Thank you for the review. I withdraw the entire [PATCH v2 0/3] series.

You are right that the vmpressure change closely follows Benjamin Lee McQueen's earlier rejected thread without proper attribution or an RFC. That was a serious mistake on my part, and I am sorry for the confusion and for wasting your time.

I will not resubmit the vmpressure or vrealloc changes. I also understand that bundling unrelated mm, selftest, and vmalloc work in one series, using non-kernel subject prefixes, and sending v2 without a proper v1 review cycle were process errors on my side.

Going forward I will stick to small, original fixes, search lore before proposing mm changes, and follow the normal RFC/review process for anything touching vmpressure or vmalloc.

Thank you,
Deepak Rao Gaikwad

author
#4 lore raw

PATCH v2 2/3 test: expand proc smaps coverage and require TMPFS

In reply to a message not in the archive

Add CONFIG_TMPFS to the selftest config. Assert Private_Clean, Private_Dirty, Swap, and SwapPss fields in /proc/$PID/smaps output.

---
 tools/testing/selftests/proc/config        | 1 +
 tools/testing/selftests/proc/proc-pid-vm.c | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)
tools/testing/selftests/proc/config +1 -0
index 68fbd2b35..7dfc3d381 100644
@@ -1 +1,2 @@
 CONFIG_PROC_FS=y
+CONFIG_TMPFS=y
tools/testing/selftests/proc/proc-pid-vm.c +4 -1
index 4e6a3e53f..59535bb3a 100644
@@ -20,7 +20,6 @@
  * Test /proc/$PID/smaps_rollup
  * Test /proc/$PID/statm
  *
- * FIXME require CONFIG_TMPFS which can be disabled
  * FIXME test other values from "smaps"
  * FIXME support other archs
  */
@@ -409,6 +408,10 @@ int main(void)
 			"AnonHugePages:         0 kB\n",
 			"Shared_Hugetlb:        0 kB\n",
 			"Private_Hugetlb:       0 kB\n",
+			"Private_Clean:         4 kB\n",
+			"Private_Dirty:         0 kB\n",
+			"Swap:                  0 kB\n",
+			"SwapPss:               0 kB\n",
 			"Locked:                0 kB\n",
 		};
 		int i;
Signature · 1 line
Deepak Rao Gaikwad
author
#5 lore raw

PATCH v2 3/3 feat: grow vrealloc vm_area mappings in place

In reply to a message not in the archive

Allocate and map additional pages at the end of an existing vm_area when vrealloc() grows past mapped pages, falling back to the memcpy path otherwise.

Use flags (not gfp) in the in-place grow path for kvmalloc_array and vm_area_alloc_pages.

---
 mm/vmalloc.c | 50 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 49 insertions(+), 1 deletion(-)
mm/vmalloc.c +49 -1
index 1afca3568..1eff71240 100644
@@ -4426,8 +4426,56 @@ void *vrealloc_node_align_noprof(const void *p, size_t size, unsigned long align
 		return (void *)p;
 	}
 
+	{
+		unsigned int new_nr_pages = PAGE_ALIGN(size) >> PAGE_SHIFT;
+		unsigned int old_nr_pages = vm->nr_pages;
+
+		if (new_nr_pages > old_nr_pages && !vm_area_page_order(vm) &&
+		    !(vm->flags & (VM_FLUSH_RESET_PERMS | VM_USERMAP)) &&
+		    gfp_has_io_fs(flags)) {
+			unsigned int extra = new_nr_pages - old_nr_pages;
+			unsigned long addr = (unsigned long)kasan_reset_tag(p);
+			struct vmap_node *vn = addr_to_node(addr);
+			struct page **pages;
+			int ret;
+
+			pages = kvmalloc_array(extra, sizeof(struct page *), flags);
+			if (!pages)
+				goto need_realloc;
+
+			if (vm_area_alloc_pages(vmalloc_gfp_adjust(flags, 0), nid,
+						0, extra, pages) != extra) {
+				kvfree(pages);
+				goto need_realloc;
+			}
+
+			spin_lock(&vn->busy.lock);
+			ret = vmap_pages_range(addr + ((unsigned long)old_nr_pages
+						       << PAGE_SHIFT),
+					       addr + ((unsigned long)new_nr_pages
+						       << PAGE_SHIFT),
+					       PAGE_KERNEL, pages, PAGE_SHIFT);
+			spin_unlock(&vn->busy.lock);
+			if (ret < 0) {
+				free_pages_bulk(pages, extra);
+				kvfree(pages);
+				goto need_realloc;
+			}
+
+			memcpy(&vm->pages[old_nr_pages], pages,
+			       extra * sizeof(struct page *));
+			kvfree(pages);
+			vm->nr_pages = new_nr_pages;
+			vm->requested_size = size;
+			kasan_vrealloc(p, old_size, size);
+			if (want_init_on_alloc(flags))
+				memset((void *)p + old_size, 0, size - old_size);
+			return (void *)p;
+		}
+	}
+
 need_realloc:
-	/* TODO: Grow the vm_area, i.e. allocate and map additional pages. */
+	/* Fall back to a fresh vm_area when in-place grow is not possible. */
 	n = __vmalloc_node_noprof(size, align, flags, nid, __builtin_return_address(0));
 
 	if (!n)
Signature · 1 line
Deepak Rao Gaikwad