mm: writeback: ratelimit stat flush from mem_cgroup_wb_stats
Subsystems (3)
PATCH 1/1 mm: writeback: ratelimit stat flush from mem_cgroup_wb_stats
One of our workloads (Postgres 14) has regressed when migrated from 5.10 to 6.1 upstream kernel. The regression can be reproduced by sysbench's oltp_write_only benchmark. It seems like the always on rstat flush in mem_cgroup_wb_stats() is causing the regression. So, rate limit that specific rstat flush. One potential consequence would be the dirty throttling might be decided on stale memcg stats. However from our benchmarks and production traffic we have not observed any change in the dirty throttling behavior of the application.
--- mm/memcontrol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/memcontrol.c +1 -1
@@ -4776,7 +4776,7 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages,struct mem_cgroup *memcg =mem_cgroup_from_css (wb->memcg_css);struct mem_cgroup *parent; -mem_cgroup_flush_stats (memcg); +mem_cgroup_flush_stats_ratelimited (memcg); *pdirty =memcg_page_state (memcg,NR_FILE_DIRTY ); *pwriteback =memcg_page_state (memcg,NR_WRITEBACK );
Signature · 1 line
2.43.0.429.g432eaa2c6b-goog
Quoted 10 lines · Shakeel Butt
One of our workloads (Postgres 14) has regressed when migrated from 5.10 to 6.1 upstream kernel. The regression can be reproduced by sysbench's
oltp_write_onlybenchmark. It seems like the always on rstat flush inmem_cgroup_wb_stats()is causing the regression. So, rate limit that specific rstat flush. One potential consequence would be the dirty throttling might be decided on stale memcg stats. However from our benchmarks and production traffic we have not observed any change in the dirty throttling behavior of the application.Signed-off-by Shakeel Butt <[email protected]>
Quoted 10 lines · Shakeel Butt
One of our workloads (Postgres 14) has regressed when migrated from 5.10 to 6.1 upstream kernel. The regression can be reproduced by sysbench's
oltp_write_onlybenchmark. It seems like the always on rstat flush inmem_cgroup_wb_stats()is causing the regression. So, rate limit that specific rstat flush. One potential consequence would be the dirty throttling might be decided on stale memcg stats. However from our benchmarks and production traffic we have not observed any change in the dirty throttling behavior of the application.Signed-off-by Shakeel Butt <[email protected]>
Thanks!
Michal Koutný
maintainerHello.
Quoted 5 lines · Shakeel Butt
One of our workloads (Postgres 14) has regressed when migrated from 5.10 to 6.1 upstream kernel. The regression can be reproduced by sysbench's
oltp_write_onlybenchmark.
It seems like the always on rstat flush inmem_cgroup_wb_stats()is causing the regression.
Is the affected benchmark running in a non-root cgroup?
I'm asking whether this would warrant a
that introduced the global flush (in v6.1) but it was removed later in
7d7ef0a4686a ("mm: memcg: restore subtree stats flushing")
(so v6.8 could be possibly unaffected).
Thanks,
Michal
Quoted 10 lines · Michal Koutný
Hello.
Quoted 5 lines · Shakeel Butt
One of our workloads (Postgres 14) has regressed when migrated from 5.10 to 6.1 upstream kernel. The regression can be reproduced by sysbench's
oltp_write_onlybenchmark.
It seems like the always on rstat flush inmem_cgroup_wb_stats()is causing the regression.
Quoted 8 lines
Is the affected benchmark running in a non-root cgroup?
I'm asking whether this would warrant a
Fixes fd25a9e0e23b ("memcg: unify memcg stat flushing")that introduced the global flush (in v6.1) but it was removed later in
7d7ef0a4686a("mm: memcg: restore subtree stats flushing")
(so v6.8 could be possibly unaffected).
Yes, the benchmark and the workload were running in non-root cgroups.
Regarding the Fixes, please note that the regression was still there with 7d7ef0a4686a ("mm: memcg: restore subtree stats flushing"), so I would say that our first conversion to rstat infra would most probably have the issue as well which was 2d146aa3aa84 ("mm: memcontrol: switch to rstat").
So, the following fixes tag makes sense to me: