ea928e9e18: stress-ng.mremap.ops_per_sec 36.2% regression
Subsystems (3)
kernel test robot
build botHello,
kernel test robot noticed a 36.2% regression of stress-ng.mremap.ops_per_sec on:
commit: ea928e9e18da682e9a5bc40aa862bff7ce5ae42e ("mm/vmpressure: move v1 userspace eventfd code into memcontrol-v1.c") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: stress-ng
version: stress-ng-x86_64-29ce10a2c-1_20260712
with following parameters:
nr_threads: 100% testtime: 60s test: mremap cpufreq_governor: performance
config: x86_64-rhel-9.4 (CONFIG_MEMCG=y and CONFIG_MEMCG_V1 is not set)
compiler: gcc-14
test machine: 256 threads 4 sockets INTEL(R) XEON(R) PLATINUM 8592+ (Emerald Rapids) with 256G memory
(please refer to attached dmesg/kmsg for entire log/backtrace)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags | Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-lkp/[email protected]
2026-08-11 09:13:23 sysctl -w kernel.sched_autogroup_enabled=0 kernel.sched_autogroup_enabled = 0 2026-08-11 09:13:23 stress-ng --timeout 60 --times --verify --metrics --no-rand-seed --mremap 256
stress-ng: info: [10886] setting to a 1 min run per stressor
stress-ng: info: [10886] dispatching hogs: 256 mremap
stress-ng: info: [11033] mremap: using 1MB per stressor instance (total 256MB of 243.06GB available memory)
stress-ng: metrc: [10886] stressor bogo ops real time usr time sys time bogo ops/s bogo ops/s CPU used per RSS Max
stress-ng: metrc: [10886] (secs) (secs) (secs) (real time) (usr+sys time) instance (%) (KB)
stress-ng: metrc: [10886] mremap 2212529 59.96 209.34 15014.92 36901.22 145.33 99.19 6684
stress-ng: metrc: [10886] miscellaneous metrics:
stress-ng: metrc: [10886] mremap 151372.07 nanosecs per mremap call (harmonic mean of 256 instances)
stress-ng: info: [10886] for a 60.01s run time:
stress-ng: info: [10886] 15363.58s available CPU time
stress-ng: info: [10886] 209.34s user time ( 1.36%)
stress-ng: info: [10886] 15015.19s system time ( 97.73%)
stress-ng: info: [10886] 15224.53s total time ( 99.09%)
stress-ng: info: [10886] load average: 163.54 47.28 16.30
stress-ng: info: [10886] skipped: 0
stress-ng: info: [10886] passed: 256: mremap (256)
stress-ng: info: [10886] failed: 0
stress-ng: info: [10886] metrics untrustworthy: 0
stress-ng: info: [10886] successful run completed in 1 min
The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20260813/[email protected]
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Shakeel Butt
maintainerQuoted 19 lines · kernel test robot
Hello,
kernel test robot noticed a 36.2% regression of stress-ng.mremap.ops_per_sec on:
commit:
ea928e9e18da682e9a5bc40aa862bff7ce5ae42e("mm/vmpressure: move v1 userspace eventfd code intomemcontrol-v1.c") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git masterin testcase: stress-ng
version: stress-ng-x86_64-29ce10a2c-1_20260712
with following parameters:nr_threads: 100% testtime: 60s test: mremap cpufreq_governor: performance
Quoted 6 lines
config: x86_64-rhel-9.4 (
CONFIG_MEMCG=yandCONFIG_MEMCG_V1is not set)
compiler: gcc-14
test machine: 256 threads 4 sockets INTEL(R) XEON(R) PLATINUM 8592+ (Emerald Rapids) with 256G memory(please refer to attached dmesg/kmsg for entire log/backtrace)
Hi there,
Can you please test the following patch and see if it fixes the regression?
From 84c0b05b3bc5cf73ee66ead75aafb1ad684462c3 Mon Sep 17 00:00:00 2001
From: Shakeel Butt <[email protected]>
Date: Thu, 13 Aug 2026 09:38:28 -0700
Subject: [PATCH] memcg: keep vmstats_percpu off the memory_events[] cacheline
--- include/linux/memcontrol.h | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
include/linux/memcontrol.h +6 -4
@@ -246,8 +246,13 @@ struct mem_cgroup {/* handle for "memory.swap.events" */ struct cgroup_file swap_events_file; -/* memory.stat */ +/* Read-mostly. */ struct memcg_vmstats *vmstats; +struct memcg_vmstats_percpu __percpu *vmstats_percpu; +int kmemcg_id; + +/* Write-hot from here on; do not let it share with the above. */ +CACHELINE_PADDING (_pad_);/* memory.events */ atomic_long_t memory_events[MEMCG_NR_MEMORY_EVENTS ];
@@ -266,9 +271,6 @@ struct mem_cgroup {#if BITS_PER_LONG <64 seqlock_t socket_pressure_seqlock;#endif -int kmemcg_id; - -struct memcg_vmstats_percpu __percpu *vmstats_percpu;#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list;
Signature · 1 line
2.53.0-Meta
Quoted 62 lines · Shakeel Butt
Quoted 19 lines · kernel test robot
Hello,
kernel test robot noticed a 36.2% regression of stress-ng.mremap.ops_per_sec on:
commit:
ea928e9e18da682e9a5bc40aa862bff7ce5ae42e("mm/vmpressure: move v1 userspace eventfd code intomemcontrol-v1.c") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git masterin testcase: stress-ng
version: stress-ng-x86_64-29ce10a2c-1_20260712
with following parameters:nr_threads: 100% testtime: 60s test: mremap cpufreq_governor: performanceQuoted 6 lines
config: x86_64-rhel-9.4 (
CONFIG_MEMCG=yandCONFIG_MEMCG_V1is not set)
compiler: gcc-14
test machine: 256 threads 4 sockets INTEL(R) XEON(R) PLATINUM 8592+ (Emerald Rapids) with 256G memory(please refer to attached dmesg/kmsg for entire log/backtrace)
Hi there,
Can you please test the following patch and see if it fixes the regression?
From
84c0b05b3bc5cf73ee66ead75aafb1ad684462c3Mon Sep 17 00:00:00 2001
From: Shakeel Butt <[email protected]>
Date: Thu, 13 Aug 2026 09:38:28 -0700
Subject: [PATCH] memcg: keepvmstats_percpuoff the memory_events[] cachelineSigned-off-by Shakeel Butt <[email protected]>--- include/linux/memcontrol.h | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)include/linux/memcontrol.h +6
-1@@ -246,8 +246,13 @@ struct mem_cgroup {/* handle for "memory.swap.events" */ struct cgroup_file swap_events_file; -/* memory.stat */ +/* Read-mostly. */ struct memcg_vmstats *vmstats; +struct memcg_vmstats_percpu __percpu *vmstats_percpu; +int kmemcg_id; + +/* Write-hot from here on; do not let it share with the above. */ +CACHELINE_PADDING (_pad_);/* memory.events */ atomic_long_t memory_events[MEMCG_NR_MEMORY_EVENTS ];
include/linux/memcontrol.h struct mem_cgroup { · line 276
Quoted 9 lines
include/linux/memcontrol.h +0
-3@@ -266,9 +271,6 @@ struct mem_cgroup {#if BITS_PER_LONG <64 seqlock_t socket_pressure_seqlock;#endif -int kmemcg_id; - -struct memcg_vmstats_percpu __percpu *vmstats_percpu;#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list;
I was currently testing this diff, not sure which one would be better.
include/linux/memcontrol.h +7 -2
@@ -268,10 +268,15 @@ struct mem_cgroup {#endif int kmemcg_id; -struct memcg_vmstats_percpu __percpu *vmstats_percpu; -#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; +#endif + +/* Keep the hot per-CPU stats pointer away from memory event counters. */ +struct memcg_vmstats_percpu __percpu *vmstats_percpu +____cacheline_aligned_in_smp ; + +#ifdef CONFIG_CGROUP_WRITEBACK struct wb_domain cgwb_domain;struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT ];#endif
Shakeel Butt
maintainerQuoted 72 lines · Usama Arif
Quoted 69 lines · Shakeel Butt
Quoted 19 lines · kernel test robot
Hello,
kernel test robot noticed a 36.2% regression of stress-ng.mremap.ops_per_sec on:
commit:
ea928e9e18da682e9a5bc40aa862bff7ce5ae42e("mm/vmpressure: move v1 userspace eventfd code intomemcontrol-v1.c") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git masterin testcase: stress-ng
version: stress-ng-x86_64-29ce10a2c-1_20260712
with following parameters:nr_threads: 100% testtime: 60s test: mremap cpufreq_governor: performanceQuoted 6 lines
config: x86_64-rhel-9.4 (
CONFIG_MEMCG=yandCONFIG_MEMCG_V1is not set)
compiler: gcc-14
test machine: 256 threads 4 sockets INTEL(R) XEON(R) PLATINUM 8592+ (Emerald Rapids) with 256G memory(please refer to attached dmesg/kmsg for entire log/backtrace)
Hi there,
Can you please test the following patch and see if it fixes the regression?
From
84c0b05b3bc5cf73ee66ead75aafb1ad684462c3Mon Sep 17 00:00:00 2001
From: Shakeel Butt <[email protected]>
Date: Thu, 13 Aug 2026 09:38:28 -0700
Subject: [PATCH] memcg: keepvmstats_percpuoff the memory_events[] cachelineSigned-off-by Shakeel Butt <[email protected]>--- include/linux/memcontrol.h | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)include/linux/memcontrol.h +6
-4@@ -246,8 +246,13 @@ struct mem_cgroup {/* handle for "memory.swap.events" */ struct cgroup_file swap_events_file; -/* memory.stat */ +/* Read-mostly. */ struct memcg_vmstats *vmstats; +struct memcg_vmstats_percpu __percpu *vmstats_percpu; +int kmemcg_id; + +/* Write-hot from here on; do not let it share with the above. */ +CACHELINE_PADDING (_pad_);/* memory.events */ atomic_long_t memory_events[MEMCG_NR_MEMORY_EVENTS ];@@ -266,9 +271,6 @@ struct mem_cgroup {#if BITS_PER_LONG <64 seqlock_t socket_pressure_seqlock;#endif -int kmemcg_id; - -struct memcg_vmstats_percpu __percpu *vmstats_percpu;
Quoted 5 lines
include/linux/memcontrol.h struct mem_cgroup { · line 272
Quoted 2 lines
include/linux/memcontrol.h +0
-0#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list;I was currently testing this diff, not sure which one would be better.
I was just checking if false sharing of vmstats_percpu is the cause. If your patch does not increase the struct size, we can go with that as a backportable fix. I am planning to rearrange fields of struct mem_cgroup more drastically and have it more stable as future work as we continuously see these regressions keep popping up.
Quoted 24 lines
include/linux/memcontrol.h +7
-2@@ -268,10 +268,15 @@ struct mem_cgroup {#endif int kmemcg_id; -struct memcg_vmstats_percpu __percpu *vmstats_percpu; -#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; +#endif + +/* Keep the hot per-CPU stats pointer away from memory event counters. */ +struct memcg_vmstats_percpu __percpu *vmstats_percpu +____cacheline_aligned_in_smp ; + +#ifdef CONFIG_CGROUP_WRITEBACK struct wb_domain cgwb_domain;struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT ];#endif
Quoted 79 lines · Shakeel Butt
Quoted 72 lines · Usama Arif
Quoted 69 lines · Shakeel Butt
Quoted 19 lines · kernel test robot
Hello,
kernel test robot noticed a 36.2% regression of stress-ng.mremap.ops_per_sec on:
commit:
ea928e9e18da682e9a5bc40aa862bff7ce5ae42e("mm/vmpressure: move v1 userspace eventfd code intomemcontrol-v1.c") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git masterin testcase: stress-ng
version: stress-ng-x86_64-29ce10a2c-1_20260712
with following parameters:nr_threads: 100% testtime: 60s test: mremap cpufreq_governor: performanceQuoted 6 lines
config: x86_64-rhel-9.4 (
CONFIG_MEMCG=yandCONFIG_MEMCG_V1is not set)
compiler: gcc-14
test machine: 256 threads 4 sockets INTEL(R) XEON(R) PLATINUM 8592+ (Emerald Rapids) with 256G memory(please refer to attached dmesg/kmsg for entire log/backtrace)
Hi there,
Can you please test the following patch and see if it fixes the regression?
From
84c0b05b3bc5cf73ee66ead75aafb1ad684462c3Mon Sep 17 00:00:00 2001
From: Shakeel Butt <[email protected]>
Date: Thu, 13 Aug 2026 09:38:28 -0700
Subject: [PATCH] memcg: keepvmstats_percpuoff the memory_events[] cachelineSigned-off-by Shakeel Butt <[email protected]>--- include/linux/memcontrol.h | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)include/linux/memcontrol.h +6
-4@@ -246,8 +246,13 @@ struct mem_cgroup {/* handle for "memory.swap.events" */ struct cgroup_file swap_events_file; -/* memory.stat */ +/* Read-mostly. */ struct memcg_vmstats *vmstats; +struct memcg_vmstats_percpu __percpu *vmstats_percpu; +int kmemcg_id; + +/* Write-hot from here on; do not let it share with the above. */ +CACHELINE_PADDING (_pad_);/* memory.events */ atomic_long_t memory_events[MEMCG_NR_MEMORY_EVENTS ];@@ -266,9 +271,6 @@ struct mem_cgroup {#if BITS_PER_LONG <64 seqlock_t socket_pressure_seqlock;#endif -int kmemcg_id; - -struct memcg_vmstats_percpu __percpu *vmstats_percpu;Quoted 5 lines
include/linux/memcontrol.h struct mem_cgroup { · line 276
Quoted 2 lines
include/linux/memcontrol.h +0
-0#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list;I was currently testing this diff, not sure which one would be better.
Quoted 6 lines
I was just checking if false sharing of
vmstats_percpuis the cause. If your patch does not increase the struct size, we can go with that as a backportable fix. I am planning to rearrange fields of structmem_cgroupmore drastically and have it more stable as future work as we continuously see these regressions keep popping up.
Yes this makes sense. I did not expect such a big change in a benchmark with my patch, although I feel like the microbenchmark is probably not that realistic.
I think another issue is that its a 4 socket system.
I only have access to a single socket system, and I see a 4.38% regression.
Do you know if there a way for kernel test robot to test the below patch on its host?
From b862e84e7bd6a54b1546b7f21a6cf991253def18 Mon Sep 17 00:00:00 2001
From: Usama Arif <[email protected]>
Date: Thu, 13 Aug 2026 11:42:05 -0700
Subject: [PATCH] mm/memcontrol: avoid false sharing between vmstats and events
Moving v1 userspace eventfd handling into memcontrol-v1.c shrank struct vmpressure from 112 to 24 bytes when CONFIG_MEMCG_V1 is disabled. This moved memory_events_local[MEMCG_SWAP_FAIL] and the hot vmstats_percpu pointer onto the same cacheline.
The stress-ng mremap stressor exercises MADV_PAGEOUT with swap disabled, generating about 20 million MEMCG_SWAP_FAIL updates per 60-second run on a 176-CPU test system. Those writes bounce the line while memcg statistics paths load vmstats_percpu.
Move cgwb_list into the existing alignment gap and cacheline-align vmstats_percpu. This separates the pointer from the event counters without increasing the size of struct mem_cgroup in the tested configuration.
The blamed commit reduced median mremap throughput by 4.38% on the test system with one socket. The patched kernel brings the performance to within 0.5% of the parent which is within the observed boot-to-boot spread (up to 1.2%).
--- include/linux/memcontrol.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
include/linux/memcontrol.h +7 -2
@@ -268,10 +268,15 @@ struct mem_cgroup {#endif int kmemcg_id; -struct memcg_vmstats_percpu __percpu *vmstats_percpu; -#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; +#endif + +/* Keep the hot per-CPU stats pointer away from memory event counters. */ +struct memcg_vmstats_percpu __percpu *vmstats_percpu +____cacheline_aligned_in_smp ; + +#ifdef CONFIG_CGROUP_WRITEBACK struct wb_domain cgwb_domain;struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT ];#endif
Signature · 1 line
2.53.0-Meta
Quoted 24 lines
Quoted 24 lines
include/linux/memcontrol.h +7
-2@@ -268,10 +268,15 @@ struct mem_cgroup {#endif int kmemcg_id; -struct memcg_vmstats_percpu __percpu *vmstats_percpu; -#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; +#endif + +/* Keep the hot per-CPU stats pointer away from memory event counters. */ +struct memcg_vmstats_percpu __percpu *vmstats_percpu +____cacheline_aligned_in_smp ; + +#ifdef CONFIG_CGROUP_WRITEBACK struct wb_domain cgwb_domain;struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT ];#endif
Shakeel Butt
maintainerQuoted 94 lines · Usama Arif
Quoted 79 lines · Shakeel Butt
Quoted 72 lines · Usama Arif
Quoted 69 lines · Shakeel Butt
Quoted 19 lines · kernel test robot
Hello,
kernel test robot noticed a 36.2% regression of stress-ng.mremap.ops_per_sec on:
commit:
ea928e9e18da682e9a5bc40aa862bff7ce5ae42e("mm/vmpressure: move v1 userspace eventfd code intomemcontrol-v1.c") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git masterin testcase: stress-ng
version: stress-ng-x86_64-29ce10a2c-1_20260712
with following parameters:nr_threads: 100% testtime: 60s test: mremap cpufreq_governor: performanceQuoted 6 lines
config: x86_64-rhel-9.4 (
CONFIG_MEMCG=yandCONFIG_MEMCG_V1is not set)
compiler: gcc-14
test machine: 256 threads 4 sockets INTEL(R) XEON(R) PLATINUM 8592+ (Emerald Rapids) with 256G memory(please refer to attached dmesg/kmsg for entire log/backtrace)
Hi there,
Can you please test the following patch and see if it fixes the regression?
From
84c0b05b3bc5cf73ee66ead75aafb1ad684462c3Mon Sep 17 00:00:00 2001
From: Shakeel Butt <[email protected]>
Date: Thu, 13 Aug 2026 09:38:28 -0700
Subject: [PATCH] memcg: keepvmstats_percpuoff the memory_events[] cachelineSigned-off-by Shakeel Butt <[email protected]>--- include/linux/memcontrol.h | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)include/linux/memcontrol.h +6
-4@@ -246,8 +246,13 @@ struct mem_cgroup {/* handle for "memory.swap.events" */ struct cgroup_file swap_events_file; -/* memory.stat */ +/* Read-mostly. */ struct memcg_vmstats *vmstats; +struct memcg_vmstats_percpu __percpu *vmstats_percpu; +int kmemcg_id; + +/* Write-hot from here on; do not let it share with the above. */ +CACHELINE_PADDING (_pad_);/* memory.events */ atomic_long_t memory_events[MEMCG_NR_MEMORY_EVENTS ];@@ -266,9 +271,6 @@ struct mem_cgroup {#if BITS_PER_LONG <64 seqlock_t socket_pressure_seqlock;#endif -int kmemcg_id; - -struct memcg_vmstats_percpu __percpu *vmstats_percpu;Quoted 5 lines
include/linux/memcontrol.h struct mem_cgroup { · line 272
Quoted 2 lines
include/linux/memcontrol.h +0
-0#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list;I was currently testing this diff, not sure which one would be better.
Quoted 6 lines
I was just checking if false sharing of
vmstats_percpuis the cause. If your patch does not increase the struct size, we can go with that as a backportable fix. I am planning to rearrange fields of structmem_cgroupmore drastically and have it more stable as future work as we continuously see these regressions keep popping up.Yes this makes sense. I did not expect such a big change in a benchmark with my patch, although I feel like the microbenchmark is probably not that realistic.
Quoted 5 lines
I think another issue is that its a 4 socket system.
I only have access to a single socket system, and I see a 4.38% regression.Do you know if there a way for kernel test robot to test the below patch on its host?
+Oliver
I used to request Oliver to test a patch for performance regressions reported by intel test robot.
Quoted 87 lines
From
b862e84e7bd6a54b1546b7f21a6cf991253def18Mon Sep 17 00:00:00 2001
From: Usama Arif <[email protected]>
Date: Thu, 13 Aug 2026 11:42:05 -0700
Subject: [PATCH] mm/memcontrol: avoid false sharing between vmstats and eventsMoving v1 userspace eventfd handling into
memcontrol-v1.cshrank struct vmpressure from 112 to 24 bytes whenCONFIG_MEMCG_V1is disabled. This moved memory_events_local[MEMCG_SWAP_FAIL] and the hotvmstats_percpupointer onto the same cacheline.The stress-ng mremap stressor exercises
MADV_PAGEOUTwith swap disabled, generating about 20 millionMEMCG_SWAP_FAILupdates per 60-second run on a 176-CPU test system. Those writes bounce the line while memcg statistics paths loadvmstats_percpu.Move
cgwb_listinto the existing alignment gap and cacheline-alignvmstats_percpu. This separates the pointer from the event counters without increasing the size of structmem_cgroupin the tested configuration.The blamed commit reduced median mremap throughput by 4.38% on the test system with one socket. The patched kernel brings the performance to within 0.5% of the parent which is within the observed boot-to-boot spread (up to 1.2%).
Fixes ea928e9e18da ("mm/vmpressure: move v1 userspace eventfd code into memcontrol-v1.c")Reported-by kernel test robot <[email protected]>Signed-off-by Usama Arif <[email protected]>--- include/linux/memcontrol.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)include/linux/memcontrol.h +7
-2@@ -268,10 +268,15 @@ struct mem_cgroup {#endif int kmemcg_id; -struct memcg_vmstats_percpu __percpu *vmstats_percpu; -#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; +#endif + +/* Keep the hot per-CPU stats pointer away from memory event counters. */ +struct memcg_vmstats_percpu __percpu *vmstats_percpu +____cacheline_aligned_in_smp ; + +#ifdef CONFIG_CGROUP_WRITEBACK struct wb_domain cgwb_domain;struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT ];#endif Signature · 1 line
2.53.0-MetaQuoted 24 lines
Quoted 24 lines
include/linux/memcontrol.h +7
-2@@ -268,10 +268,15 @@ struct mem_cgroup {#endif int kmemcg_id; -struct memcg_vmstats_percpu __percpu *vmstats_percpu; -#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; +#endif + +/* Keep the hot per-CPU stats pointer away from memory event counters. */ +struct memcg_vmstats_percpu __percpu *vmstats_percpu +____cacheline_aligned_in_smp ; + +#ifdef CONFIG_CGROUP_WRITEBACK struct wb_domain cgwb_domain;struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT ];#endif
kernel test robot
build botTested the patch on the same test machine. Here are the test results from stress-ng mremap benchmark (60s, 256 instances):
On Fri, Aug 14, 2026 at 11:32:09AM +0100, Usama Arif wrote:
>
>
> On 13/08/2026 18:29, Shakeel Butt wrote:
> > On Thu, Aug 13, 2026 at 06:14:50PM +0100, Usama Arif wrote:
> >>
> >>
> >> On 13/08/2026 18:01, Shakeel Butt wrote:
> >>> On Thu, Aug 13, 2026 at 09:16:22PM +0800, kernel test robot wrote:
> >>>>
> >>>>
> >>>> Hello,
> >>>>
> >>>> kernel test robot noticed a 36.2% regression of stress-ng.mremap.ops_per_sec on:
> >>>>
> >>>> commit: ea928e9e18da682e9a5bc40aa862bff7ce5ae42e ("mm/vmpressure: move v1 userspace eventfd code into memcontrol-v1.c") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
> >>>>
> >>>> in testcase: stress-ng
> >>>> version: stress-ng-x86_64-29ce10a2c-1_20260712
> >>>> with following parameters:
> >>>>
> >>>> nr_threads: 100%
> >>>> testtime: 60s
> >>>> test: mremap
> >>>> cpufreq_governor: performance
> >>>>
> >>>>
> >>>>
> >>>> config: x86_64-rhel-9.4 (CONFIG_MEMCG=y and CONFIG_MEMCG_V1 is not set)
> >>>> compiler: gcc-14
> >>>> test machine: 256 threads 4 sockets INTEL(R) XEON(R) PLATINUM 8592+ (Emerald Rapids) with 256G memory
> >>>>
> >>>> (please refer to attached dmesg/kmsg for entire log/backtrace)
> >>>>
> >>>
> >>> Hi there,
> >>>
> >>> Can you please test the following patch and see if it fixes the regression?
> >>>
> >>>
> >>> From 84c0b05b3bc5cf73ee66ead75aafb1ad684462c3 Mon Sep 17 00:00:00 2001
> >>> From: Shakeel Butt <[email protected]>
> >>> Date: Thu, 13 Aug 2026 09:38:28 -0700
> >>> Subject: [PATCH] memcg: keep vmstats_percpu off the memory_events[] cacheline
> >>>
> >>> Signed-off-by: Shakeel Butt <[email protected]>
> >>> ---
> >>> include/linux/memcontrol.h | 10 ++++++----
> >>> 1 file changed, 6 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> >>> index e78bc98ab229..e25d5b9a1db8 100644
> >>> --- a/include/linux/memcontrol.h
> >>> +++ b/include/linux/memcontrol.h
> >>> @@ -246,8 +246,13 @@ struct mem_cgroup {
> >>> /* handle for "memory.swap.events" */
> >>> struct cgroup_file swap_events_file;
> >>>
> >>> - /* memory.stat */
> >>> + /* Read-mostly. */
> >>> struct memcg_vmstats *vmstats;
> >>> + struct memcg_vmstats_percpu __percpu *vmstats_percpu;
> >>> + int kmemcg_id;
> >>> +
> >>> + /* Write-hot from here on; do not let it share with the above. */
> >>> + CACHELINE_PADDING(_pad_);
> >>>
> >>> /* memory.events */
> >>> atomic_long_t memory_events[MEMCG_NR_MEMORY_EVENTS];
> >>> @@ -266,9 +271,6 @@ struct mem_cgroup {
> >>> #if BITS_PER_LONG < 64
> >>> seqlock_t socket_pressure_seqlock;
> >>> #endif
> >>> - int kmemcg_id;
> >>> -
> >>> - struct memcg_vmstats_percpu __percpu *vmstats_percpu;
> >>>
> >>> #ifdef CONFIG_CGROUP_WRITEBACK
> >>> struct list_head cgwb_list;
> >>
> >>
> >> I was currently testing this diff, not sure which one would be better.
> >
> > I was just checking if false sharing of vmstats_percpu is the cause. If your
> > patch does not increase the struct size, we can go with that as a backportable
> > fix. I am planning to rearrange fields of struct mem_cgroup more drastically and
> > have it more stable as future work as we continuously see these regressions keep
> > popping up.
> >
>
>
> Yes this makes sense. I did not expect such a big change in a benchmark
> with my patch, although I feel like the microbenchmark is probably not
> that realistic.
>
> I think another issue is that its a 4 socket system.
> I only have access to a single socket system, and I see a 4.38% regression.
>
> Do you know if there a way for kernel test robot to test the below patch
> on its host?
>
>
> >From b862e84e7bd6a54b1546b7f21a6cf991253def18 Mon Sep 17 00:00:00 2001
> From: Usama Arif <[email protected]>
> Date: Thu, 13 Aug 2026 11:42:05 -0700
> Subject: [PATCH] mm/memcontrol: avoid false sharing between vmstats and events
>
> Moving v1 userspace eventfd handling into memcontrol-v1.c shrank
> struct vmpressure from 112 to 24 bytes when CONFIG_MEMCG_V1 is disabled.
> This moved memory_events_local[MEMCG_SWAP_FAIL] and the hot
> vmstats_percpu pointer onto the same cacheline.
>
> The stress-ng mremap stressor exercises MADV_PAGEOUT with swap
> disabled, generating about 20 million MEMCG_SWAP_FAIL updates per
> 60-second run on a 176-CPU test system. Those writes bounce the line
> while memcg statistics paths load vmstats_percpu.
>
> Move cgwb_list into the existing alignment gap and cacheline-align
> vmstats_percpu. This separates the pointer from the event counters
> without increasing the size of struct mem_cgroup in the tested
> configuration.
>
> The blamed commit reduced median mremap throughput by 4.38% on the
> test system with one socket. The patched kernel brings the performance
> to within 0.5% of the parent which is within the observed boot-to-boot
> spread (up to 1.2%).
>
> Fixes: ea928e9e18da ("mm/vmpressure: move v1 userspace eventfd code into memcontrol-v1.c")
> Reported-by: kernel test robot <[email protected]>
> Closes: https://lore.kernel.org/oe-lkp/[email protected]
> Signed-off-by: Usama Arif <[email protected]>
> ---
> include/linux/memcontrol.h | 9 +++++++--
> 1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index e78bc98ab229b..215e2e87f42b2 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -268,10 +268,15 @@ struct mem_cgroup {
> #endif
> int kmemcg_id;
>
> - struct memcg_vmstats_percpu __percpu *vmstats_percpu;
> -
> #ifdef CONFIG_CGROUP_WRITEBACK
> struct list_head cgwb_list;
> +#endif
> +
> + /* Keep the hot per-CPU stats pointer away from memory event counters. */
> + struct memcg_vmstats_percpu __percpu *vmstats_percpu
> + ____cacheline_aligned_in_smp;
> +
> +#ifdef CONFIG_CGROUP_WRITEBACK
> struct wb_domain cgwb_domain;
> struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT];
> #endif
> --
> 2.53.0-Meta
>
>Tested the patch on the same test machine. Here are the test results from stress-ng mremap benchmark (60s, 256 instances):
Commit Avg.ops_per_sec
Base a33b5c91 57311.85
Reregression ea928e9e 36542.09
Fix 582a7676 57089.22
The performance regression introduced by ea928e9e is resolved.
>
> >>
> >> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> >> index e78bc98ab229b..215e2e87f42b2 100644
> >> --- a/include/linux/memcontrol.h
> >> +++ b/include/linux/memcontrol.h
> >> @@ -268,10 +268,15 @@ struct mem_cgroup {
> >> #endif
> >> int kmemcg_id;
> >>
> >> - struct memcg_vmstats_percpu __percpu *vmstats_percpu;
> >> -
> >> #ifdef CONFIG_CGROUP_WRITEBACK
> >> struct list_head cgwb_list;
> >> +#endif
> >> +
> >> + /* Keep the hot per-CPU stats pointer away from memory event counters. */
> >> + struct memcg_vmstats_percpu __percpu *vmstats_percpu
> >> + ____cacheline_aligned_in_smp;
> >> +
> >> +#ifdef CONFIG_CGROUP_WRITEBACK
> >> struct wb_domain cgwb_domain;
> >> struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT];
> >> #endif
> >>
> Quoted 165 lines · kernel test robot
Quoted 160 lines · Usama Arif
Quoted 79 lines · Shakeel Butt
Quoted 72 lines · Usama Arif
Quoted 69 lines · Shakeel Butt
Quoted 19 lines · kernel test robot
Hello,
kernel test robot noticed a 36.2% regression of stress-ng.mremap.ops_per_sec on:
commit:
ea928e9e18da682e9a5bc40aa862bff7ce5ae42e("mm/vmpressure: move v1 userspace eventfd code intomemcontrol-v1.c") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git masterin testcase: stress-ng
version: stress-ng-x86_64-29ce10a2c-1_20260712
with following parameters:nr_threads: 100% testtime: 60s test: mremap cpufreq_governor: performanceQuoted 6 lines
config: x86_64-rhel-9.4 (
CONFIG_MEMCG=yandCONFIG_MEMCG_V1is not set)
compiler: gcc-14
test machine: 256 threads 4 sockets INTEL(R) XEON(R) PLATINUM 8592+ (Emerald Rapids) with 256G memory(please refer to attached dmesg/kmsg for entire log/backtrace)
Hi there,
Can you please test the following patch and see if it fixes the regression?
From
84c0b05b3bc5cf73ee66ead75aafb1ad684462c3Mon Sep 17 00:00:00 2001
From: Shakeel Butt <[email protected]>
Date: Thu, 13 Aug 2026 09:38:28 -0700
Subject: [PATCH] memcg: keepvmstats_percpuoff the memory_events[] cachelineSigned-off-by Shakeel Butt <[email protected]>--- include/linux/memcontrol.h | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)include/linux/memcontrol.h +6
-4@@ -246,8 +246,13 @@ struct mem_cgroup {/* handle for "memory.swap.events" */ struct cgroup_file swap_events_file; -/* memory.stat */ +/* Read-mostly. */ struct memcg_vmstats *vmstats; +struct memcg_vmstats_percpu __percpu *vmstats_percpu; +int kmemcg_id; + +/* Write-hot from here on; do not let it share with the above. */ +CACHELINE_PADDING (_pad_);/* memory.events */ atomic_long_t memory_events[MEMCG_NR_MEMORY_EVENTS ];@@ -266,9 +271,6 @@ struct mem_cgroup {#if BITS_PER_LONG <64 seqlock_t socket_pressure_seqlock;#endif -int kmemcg_id; - -struct memcg_vmstats_percpu __percpu *vmstats_percpu;Quoted 5 lines
include/linux/memcontrol.h struct mem_cgroup { · line 272
Quoted 2 lines
include/linux/memcontrol.h +0
-0#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list;I was currently testing this diff, not sure which one would be better.
Quoted 6 lines
I was just checking if false sharing of
vmstats_percpuis the cause. If your patch does not increase the struct size, we can go with that as a backportable fix. I am planning to rearrange fields of structmem_cgroupmore drastically and have it more stable as future work as we continuously see these regressions keep popping up.Yes this makes sense. I did not expect such a big change in a benchmark with my patch, although I feel like the microbenchmark is probably not that realistic.
I think another issue is that its a 4 socket system.
I only have access to a single socket system, and I see a 4.38% regression.Do you know if there a way for kernel test robot to test the below patch on its host?
Quoted 1 line
From
b862e84e7bd6a54b1546b7f21a6cf991253def18Mon Sep 17 00:00:00 2001From: Usama Arif <[email protected]>
Date: Thu, 13 Aug 2026 11:42:05 -0700
Subject: [PATCH] mm/memcontrol: avoid false sharing between vmstats and eventsMoving v1 userspace eventfd handling into
memcontrol-v1.cshrank struct vmpressure from 112 to 24 bytes whenCONFIG_MEMCG_V1is disabled. This moved memory_events_local[MEMCG_SWAP_FAIL] and the hotvmstats_percpupointer onto the same cacheline.The stress-ng mremap stressor exercises
MADV_PAGEOUTwith swap disabled, generating about 20 millionMEMCG_SWAP_FAILupdates per 60-second run on a 176-CPU test system. Those writes bounce the line while memcg statistics paths loadvmstats_percpu.Move
cgwb_listinto the existing alignment gap and cacheline-alignvmstats_percpu. This separates the pointer from the event counters without increasing the size of structmem_cgroupin the tested configuration.The blamed commit reduced median mremap throughput by 4.38% on the test system with one socket. The patched kernel brings the performance to within 0.5% of the parent which is within the observed boot-to-boot spread (up to 1.2%).
Fixes ea928e9e18da ("mm/vmpressure: move v1 userspace eventfd code into memcontrol-v1.c")Reported-by kernel test robot <[email protected]>Signed-off-by Usama Arif <[email protected]>--- include/linux/memcontrol.h | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)include/linux/memcontrol.h +7
-2@@ -268,10 +268,15 @@ struct mem_cgroup {#endif int kmemcg_id; -struct memcg_vmstats_percpu __percpu *vmstats_percpu; -#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; +#endif + +/* Keep the hot per-CPU stats pointer away from memory event counters. */ +struct memcg_vmstats_percpu __percpu *vmstats_percpu +____cacheline_aligned_in_smp ; + +#ifdef CONFIG_CGROUP_WRITEBACK struct wb_domain cgwb_domain;struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT ];#endif Signature · 1 line
2.53.0-MetaTested the patch on the same test machine. Here are the test results from stress-ng mremap benchmark (60s, 256 instances):
Quoted 6 lines
Commit Avg.ops_per_sec
Basea33b5c9157311.85
Reregressionea928e9e36542.09
Fix582a767657089.22The performance regression introduced by
ea928e9eis resolved.
Thanks a lot.
Usame, can you please send a formal patch and you can add:
Quoted 27 lines
Quoted 27 lines
Quoted 26 lines
Quoted 26 lines
include/linux/memcontrol.h +7
-2@@ -268,10 +268,15 @@ struct mem_cgroup {#endif int kmemcg_id; -struct memcg_vmstats_percpu __percpu *vmstats_percpu; -#ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; +#endif + +/* Keep the hot per-CPU stats pointer away from memory event counters. */ +struct memcg_vmstats_percpu __percpu *vmstats_percpu +____cacheline_aligned_in_smp ; + +#ifdef CONFIG_CGROUP_WRITEBACK struct wb_domain cgwb_domain;struct memcg_cgwb_frn cgwb_frn[MEMCG_CGWB_FRN_CNT ];#endif