memcg: cleanup racy sum avoidance code
Subsystems (3)
PATCH 1/1 memcg: cleanup racy sum avoidance code
We used to have per-cpu memcg and lruvec stats and the readers have to traverse and sum the stats from each cpu. This summing was racy and may expose transient negative values. So, an explicit check was added to avoid such scenarios. Now these stats are moved to rstat infrastructure and are no more per-cpu, so we can remove the fixup for transient negative values.
--- include/linux/memcontrol.h | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)
include/linux/memcontrol.h +2 -13
@@ -991,30 +991,19 @@ static inline void mod_memcg_state(struct mem_cgroup *memcg,static inline unsigned long memcg_page_state (struct mem_cgroup *memcg,int idx) { -long x =READ_ONCE (memcg->vmstats.state[idx]); -#ifdef CONFIG_SMP -if (x <0 ) - x =0 ; -#endif -return x; +return READ_ONCE (memcg->vmstats.state[idx]); }static inline unsigned long lruvec_page_state (struct lruvec *lruvec,enum node_stat_item idx) {struct mem_cgroup_per_node *pn; -long x;if (mem_cgroup_disabled ())return node_page_state (lruvec_pgdat (lruvec), idx); pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x =READ_ONCE (pn->lruvec_stats.state[idx]); -#ifdef CONFIG_SMP -if (x <0 ) - x =0 ; -#endif -return x; +return READ_ONCE (pn->lruvec_stats.state[idx]); } static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec,
Signature · 1 line
2.32.0.432.gabb21c7263-goog
Quoted 8 lines · Shakeel Butt
We used to have per-cpu memcg and lruvec stats and the readers have to traverse and sum the stats from each cpu. This summing was racy and may expose transient negative values. So, an explicit check was added to avoid such scenarios. Now these stats are moved to rstat infrastructure and are no more per-cpu, so we can remove the fixup for transient negative values.
Signed-off-by Shakeel Butt <[email protected]>
Thanks!
Quoted 40 lines · Shakeel Butt
We used to have per-cpu memcg and lruvec stats and the readers have to traverse and sum the stats from each cpu. This summing was racy and may expose transient negative values. So, an explicit check was added to avoid such scenarios. Now these stats are moved to rstat infrastructure and are no more per-cpu, so we can remove the fixup for transient negative values.
Signed-off-by Shakeel Butt <[email protected]>--- include/linux/memcontrol.h | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)include/linux/memcontrol.h +1
-7@@ -991,30 +991,19 @@ static inline void mod_memcg_state(struct mem_cgroup *memcg,static inline unsigned long memcg_page_state (struct mem_cgroup *memcg,int idx) { -long x =READ_ONCE (memcg->vmstats.state[idx]); -#ifdef CONFIG_SMP -if (x <0 ) - x =0 ; -#endif -return x; +return READ_ONCE (memcg->vmstats.state[idx]); }static inline unsigned long lruvec_page_state (struct lruvec *lruvec,enum node_stat_item idx) {struct mem_cgroup_per_node *pn; -long x;if (mem_cgroup_disabled ())return node_page_state (lruvec_pgdat (lruvec), idx); pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec);
include/linux/memcontrol.h static inline void mod_memcg_state(struct mem_cgroup *memcg, · line 1009
Quoted 10 lines
include/linux/memcontrol.h +1
-6- x =READ_ONCE (pn->lruvec_stats.state[idx]); -#ifdef CONFIG_SMP -if (x <0 ) - x =0 ; -#endif -return x; +return READ_ONCE (pn->lruvec_stats.state[idx]); } static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec,
Signature · 3 lines
Thanks, David / dhildenb
Andrew Morton
maintainerQuoted 6 lines · On Tue, 27 Jul 2021 18:22:43 -0700 Shakeel Butt…
We used to have per-cpu memcg and lruvec stats and the readers have to traverse and sum the stats from each cpu. This summing was racy and may expose transient negative values. So, an explicit check was added to avoid such scenarios. Now these stats are moved to rstat infrastructure and are no more per-cpu, so we can remove the fixup for transient negative values.
We can't do anything about the same code in lruvec_page_state_local()?
Quoted 11 lines · Andrew Morton
Quoted 6 lines · On Tue, 27 Jul 2021 18:22:43 -0700 Shakeel Butt…
We used to have per-cpu memcg and lruvec stats and the readers have to traverse and sum the stats from each cpu. This summing was racy and may expose transient negative values. So, an explicit check was added to avoid such scenarios. Now these stats are moved to rstat infrastructure and are no more per-cpu, so we can remove the fixup for transient negative values.
We can't do anything about the same code in
lruvec_page_state_local()?
lruvec_page_state_local() is used by cgroup v1's memory.numa_stat for cgroup local stats (not hierarchical) and are still per-cpu. To make it non-per-cpu, we have to add 'long state_local[NR_VM_NODE_STAT_ITEMS]' in 'struct lruvec_stats' and do aggregation in rstat flushing. So, paying the cpu traversal cost with more memory usage. I am not sure if it is worth it.
Quoted 8 lines · Shakeel Butt
We used to have per-cpu memcg and lruvec stats and the readers have to traverse and sum the stats from each cpu. This summing was racy and may expose transient negative values. So, an explicit check was added to avoid such scenarios. Now these stats are moved to rstat infrastructure and are no more per-cpu, so we can remove the fixup for transient negative values.
Signed-off-by Shakeel Butt <[email protected]>
Quoted 43 lines
--- include/linux/memcontrol.h | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)include/linux/memcontrol.h +2
-13@@ -991,30 +991,19 @@ static inline void mod_memcg_state(struct mem_cgroup *memcg,static inline unsigned long memcg_page_state (struct mem_cgroup *memcg,int idx) { -long x =READ_ONCE (memcg->vmstats.state[idx]); -#ifdef CONFIG_SMP -if (x <0 ) - x =0 ; -#endif -return x; +return READ_ONCE (memcg->vmstats.state[idx]); }static inline unsigned long lruvec_page_state (struct lruvec *lruvec,enum node_stat_item idx) {struct mem_cgroup_per_node *pn; -long x;if (mem_cgroup_disabled ())return node_page_state (lruvec_pgdat (lruvec), idx); pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x =READ_ONCE (pn->lruvec_stats.state[idx]); -#ifdef CONFIG_SMP -if (x <0 ) - x =0 ; -#endif -return x; +return READ_ONCE (pn->lruvec_stats.state[idx]); } static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec,Signature · 1 line
2.32.0.432.gabb21c7263-goog
Signature · 2 lines
Michal Hocko SUSE Labs