bypass root memcg charges if no memcgs are possible
Subsystems (4)
v2 cover bypass root memcg charges if no memcgs are possible
Hi,
Here is my recent changes in my patchset to bypass charges from the root cgroup if no other memcgs are present in the system. At the time, the main complaint came from Michal Hocko, correctly pointing out that if hierarchy == 0, we can't bypass the root memcg forever and at some point, we need to transfer charges. This is now done in this patch.
DISCLAIMER:
I haven't yet got access to a big box again. I am sending this with outdated numbers in the interest of having this in the open earlier. However, the main idea stands, and I believe the numbers are still generally valid (albeit of course, it would be better to have them updated. Independent evaluations always welcome)
* v2
- Fixed some LRU bugs
- Only keep bypassing if we have root-level hierarchy.
Glauber Costa (5):
memcg: make nocpu_base available for non hotplug
memcg: provide root figures from system totals
memcg: make it suck faster
memcg: do not call page_cgroup_init at system_boot
memcg: do not walk all the way to the root for memcg
include/linux/memcontrol.h | 72 ++++++++++--- include/linux/page_cgroup.h | 28 ++--- init/main.c | 2 - mm/memcontrol.c | 243 ++++++++++++++++++++++++++++++++++++++++---- mm/page_cgroup.c | 150 ++++++++++++++------------- 5 files changed, 382 insertions(+), 113 deletions(-)
Signature · 1 line
1.8.1.2
PATCH v2 1/5 memcg: make nocpu_base available for non hotplug
We are using nocpu_base to accumulate charges on the main counters during cpu hotplug. I have a similar need, which is transferring charges to the root cgroup when lazily enabling memcg. Because system wide information is not kept per-cpu, it is hard to distribute it. This field works well for this. So we need to make it available for all usages, not only hotplug cases.
--- mm/memcontrol.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
mm/memcontrol.c +4 -4
@@ -921,11 +921,11 @@ static long mem_cgroup_read_stat(struct mem_cgroup *memcg,get_online_cpus ();for_each_online_cpu (cpu) val +=per_cpu (memcg->stat->count[idx], cpu); -#ifdef CONFIG_HOTPLUG_CPU +spin_lock (&memcg->pcp_counter_lock); val += memcg->nocpu_base.count[idx];spin_unlock (&memcg->pcp_counter_lock); -#endif +put_online_cpus ();return val; }
@@ -945,11 +945,11 @@ static unsigned long mem_cgroup_read_events(struct mem_cgroup *memcg,for_each_online_cpu (cpu) val +=per_cpu (memcg->stat->events[idx], cpu); -#ifdef CONFIG_HOTPLUG_CPU +spin_lock (&memcg->pcp_counter_lock); val += memcg->nocpu_base.events[idx];spin_unlock (&memcg->pcp_counter_lock); -#endif +return val; }
Signature · 1 line
1.8.1.2
Re: [PATCH v2 1/5] memcg: make nocpu_base available for non hotplug
Quoted 7 lines · Glauber Costa
We are using
nocpu_baseto accumulate charges on the main counters during cpu hotplug. I have a similar need, which is transferring charges to the root cgroup when lazily enabling memcg. Because system wide information is not kept per-cpu, it is hard to distribute it. This field works well for this. So we need to make it available for all usages, not only hotplug cases.
Quoted 5 lines
Signed-off-by Glauber Costa <[email protected]>Cc Michal Hocko <[email protected]>Cc Kamezawa Hiroyuki <[email protected]>Cc Johannes Weiner <[email protected]>Cc Tejun Heo <[email protected]>
Hmm..comments on nocpu_base definition will be updated in later patch ?
Quoted 37 lines
--- mm/memcontrol.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)mm/memcontrol.c +4
-4@@ -921,11 +921,11 @@ static long mem_cgroup_read_stat(struct mem_cgroup *memcg,get_online_cpus ();for_each_online_cpu (cpu) val +=per_cpu (memcg->stat->count[idx], cpu); -#ifdef CONFIG_HOTPLUG_CPU +spin_lock (&memcg->pcp_counter_lock); val += memcg->nocpu_base.count[idx];spin_unlock (&memcg->pcp_counter_lock); -#endif +put_online_cpus ();return val; }@@ -945,11 +945,11 @@ static unsigned long mem_cgroup_read_events(struct mem_cgroup *memcg,for_each_online_cpu (cpu) val +=per_cpu (memcg->stat->events[idx], cpu); -#ifdef CONFIG_HOTPLUG_CPU +spin_lock (&memcg->pcp_counter_lock); val += memcg->nocpu_base.events[idx];spin_unlock (&memcg->pcp_counter_lock); -#endif +return val; }
Re: [PATCH v2 1/5] memcg: make nocpu_base available for non hotplug
In reply to Glauber CostaQuoted 6 lines · Glauber Costa
We are using
nocpu_baseto accumulate charges on the main counters during cpu hotplug. I have a similar need, which is transferring charges to the root cgroup when lazily enabling memcg. Because system wide information is not kept per-cpu, it is hard to distribute it. This field works well for this. So we need to make it available for all usages, not only hotplug cases.
Could you also rename it to something else while you are at it? nocpu_base sounds outdated. What about overflow_base or something like that.
I am also wondering why do wee need pcp_counter_lock there. Doesn't get_online_cpus prevent from hotplug so mem_cgroup_drain_pcp_counter doesn't get called? I am sorry for this stupid question but I am lost in the hotplug callbacks...
Other than that I don't mind pulling nocpu_base outside the hotplug code and reusing it for something else. So you can add my
but I would be happier with a better name of course ;)
Quoted 48 lines
Signed-off-by Glauber Costa <[email protected]>Cc Michal Hocko <[email protected]>Cc Kamezawa Hiroyuki <[email protected]>Cc Johannes Weiner <[email protected]>Cc Tejun Heo <[email protected]>--- mm/memcontrol.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)mm/memcontrol.c +4
-4@@ -921,11 +921,11 @@ static long mem_cgroup_read_stat(struct mem_cgroup *memcg,get_online_cpus ();for_each_online_cpu (cpu) val +=per_cpu (memcg->stat->count[idx], cpu); -#ifdef CONFIG_HOTPLUG_CPU +spin_lock (&memcg->pcp_counter_lock); val += memcg->nocpu_base.count[idx];spin_unlock (&memcg->pcp_counter_lock); -#endif +put_online_cpus ();return val; }@@ -945,11 +945,11 @@ static unsigned long mem_cgroup_read_events(struct mem_cgroup *memcg,for_each_online_cpu (cpu) val +=per_cpu (memcg->stat->events[idx], cpu); -#ifdef CONFIG_HOTPLUG_CPU +spin_lock (&memcg->pcp_counter_lock); val += memcg->nocpu_base.events[idx];spin_unlock (&memcg->pcp_counter_lock); -#endif +return val; }Signature · 6 lines
1.8.1.2 -- To unsubscribe from this list: send the line "unsubscribe cgroups" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v2 2/5 memcg: provide root figures from system totals
In reply to Glauber CostaFor the root memcg, there is no need to rely on the res_counters if hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.
Limit and soft limit can't be set for the root cgroup, so they are left at RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.
--- mm/memcontrol.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)
mm/memcontrol.c +64 -0
@@ -4996,6 +4996,56 @@ static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap)return val <<PAGE_SHIFT ; } +static u64 memcg_read_root_rss (void ) +{ +struct task_struct *p; + +u64 rss =0 ; +read_lock (&tasklist_lock); +for_each_process (p) { +if (!p->mm) +continue ; +task_lock (p); + rss +=get_mm_rss (p->mm); +task_unlock (p); + } +read_unlock (&tasklist_lock); +return rss; +} + +static u64 mem_cgroup_read_root (enum res_type type,int name) +{ +if (name ==RES_LIMIT ) +return RESOURCE_MAX ; +if (name ==RES_SOFT_LIMIT ) +return RESOURCE_MAX ; +if (name ==RES_FAILCNT ) +return 0 ; +if (name ==RES_MAX_USAGE ) +return 0 ; + +if (WARN_ON_ONCE (name !=RES_USAGE )) +return 0 ; + +switch (type) { +case _MEM: +return (memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ; +case _MEMSWAP: { +struct sysinfo i; +si_swapinfo (&i); + +return ((memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ) + + i.totalswap - i.freeswap; + } +case _KMEM: +return 0 ; +default : +BUG (); + }; +} +static ssize_t mem_cgroup_read (struct cgroup *cont,struct cftype *cft,struct file *file,char __user *buf,size_t nbytes,loff_t *ppos)
@@ -5012,6 +5062,19 @@ static ssize_t mem_cgroup_read(struct cgroup *cont, struct cftype *cft,if (!do_swap_account && type == _MEMSWAP)return -EOPNOTSUPP ; +/* +* If we have root-level hierarchy, we can be certain that the charges +* in root are always global. We can then bypass the root cgroup +* entirely in this case, hopefuly leading to less contention in the +* root res_counters. The charges presented after reading it will +* always be the global charges. +*/ +if (mem_cgroup_disabled () || + (mem_cgroup_is_root (memcg) && memcg->use_hierarchy)) { + val =mem_cgroup_read_root (type, name); + goto root_bypass; + } +switch (type) {case _MEM:if (name ==RES_USAGE )
@@ -5032,6 +5095,7 @@ static ssize_t mem_cgroup_read(struct cgroup *cont, struct cftype *cft,BUG (); } +root_bypass: len =scnprintf (str,sizeof (str),"%llu\n" , (unsigned long long )val);return simple_read_from_buffer (buf, nbytes, ppos, str, len); }
Signature · 7 lines
1.8.1.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 11 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.
Quoted 6 lines
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <[email protected]>
I think this patch's calculation is wrong.
Quoted 18 lines
--- mm/memcontrol.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)mm/memcontrol.c +6
-0@@ -4996,6 +4996,56 @@ static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap)return val <<PAGE_SHIFT ; } +static u64 memcg_read_root_rss (void ) +{ +struct task_struct *p; + +u64 rss =0 ; +read_lock (&tasklist_lock);
mm/memcontrol.c static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) · line 5014
Quoted 10 lines
mm/memcontrol.c +10
-0+for_each_process (p) { +if (!p->mm) +continue ; +task_lock (p); + rss +=get_mm_rss (p->mm); +task_unlock (p); + } +read_unlock (&tasklist_lock); +return rss; +}
I think you can use rcu_read_lock() instead of tasklist_lock. Isn't it enough to use NR_ANON_LRU rather than this ?
Quoted 16 lines
mm/memcontrol.c +16
-0+ +static u64 mem_cgroup_read_root (enum res_type type,int name) +{ +if (name ==RES_LIMIT ) +return RESOURCE_MAX ; +if (name ==RES_SOFT_LIMIT ) +return RESOURCE_MAX ; +if (name ==RES_FAILCNT ) +return 0 ; +if (name ==RES_MAX_USAGE ) +return 0 ; + +if (WARN_ON_ONCE (name !=RES_USAGE )) +return 0 ; + +switch (type) {
mm/memcontrol.c static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) · line 5040
Quoted 10 lines
mm/memcontrol.c +10
-0+case _MEM: +return (memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ; +case _MEMSWAP: { +struct sysinfo i; +si_swapinfo (&i); + +return ((memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ) + + i.totalswap - i.freeswap;
How swapcache is handled ? ...and How kmem works with this calc ?
Thanks,
-Kame
Quoted 33 lines
mm/memcontrol.c +21
-0+ } +case _KMEM: +return 0 ; +default : +BUG (); + }; +} +static ssize_t mem_cgroup_read (struct cgroup *cont,struct cftype *cft,struct file *file,char __user *buf,size_t nbytes,loff_t *ppos)@@ -5012,6 +5062,19 @@ static ssize_t mem_cgroup_read(struct cgroup *cont, struct cftype *cft,if (!do_swap_account && type == _MEMSWAP)return -EOPNOTSUPP ; +/* +* If we have root-level hierarchy, we can be certain that the charges +* in root are always global. We can then bypass the root cgroup +* entirely in this case, hopefuly leading to less contention in the +* root res_counters. The charges presented after reading it will +* always be the global charges. +*/ +if (mem_cgroup_disabled () || + (mem_cgroup_is_root (memcg) && memcg->use_hierarchy)) { + val =mem_cgroup_read_root (type, name); + goto root_bypass; + } +switch (type) {case _MEM:if (name ==RES_USAGE )
mm/memcontrol.c static ssize_t mem_cgroup_read(struct cgroup *cont, struct cftype *cft, · line 5101
Quoted 7 lines
mm/memcontrol.c +1
-0@@ -5032,6 +5095,7 @@ static ssize_t mem_cgroup_read(struct cgroup *cont, struct cftype *cft,BUG (); } +root_bypass: len =scnprintf (str,sizeof (str),"%llu\n" , (unsigned long long )val);return simple_read_from_buffer (buf, nbytes, ppos, str, len); }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 12 lines · Kamezawa Hiroyuki
Quoted 11 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.
Quoted 9 lines
Quoted 6 lines
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>I think this patch's calculation is wrong.
where exactly ?
Quoted 23 lines
Quoted 23 lines
--- mm/memcontrol.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)mm/memcontrol.c +11
-0@@ -4996,6 +4996,56 @@ static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap)return val <<PAGE_SHIFT ; } +static u64 memcg_read_root_rss (void ) +{ +struct task_struct *p; + +u64 rss =0 ; +read_lock (&tasklist_lock); +for_each_process (p) { +if (!p->mm) +continue ; +task_lock (p); + rss +=get_mm_rss (p->mm);
Quoted 8 lines
mm/memcontrol.c static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) · line 5014
Quoted 5 lines
mm/memcontrol.c +5
-0+task_unlock (p); + } +read_unlock (&tasklist_lock); +return rss; +}I think you can use
rcu_read_lock()instead oftasklist_lock. Isn't it enough to useNR_ANON_LRUrather than this ?
Is it really just ANON_LRU ? get_mm_rss also include filepages, which are not in this list.
Maybe if we sum up *all* LRUs we would get the right result ?
About the tasklist lock, if I get values from the LRUs, maybe. Otherwise it is still necessary, no ?
Quoted 21 lines
Quoted 21 lines
mm/memcontrol.c +20
-0+ +static u64 mem_cgroup_read_root (enum res_type type,int name) +{ +if (name ==RES_LIMIT ) +return RESOURCE_MAX ; +if (name ==RES_SOFT_LIMIT ) +return RESOURCE_MAX ; +if (name ==RES_FAILCNT ) +return 0 ; +if (name ==RES_MAX_USAGE ) +return 0 ; + +if (WARN_ON_ONCE (name !=RES_USAGE )) +return 0 ; + +switch (type) { +case _MEM: +return (memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ; +case _MEMSWAP: {
Quoted 9 lines
mm/memcontrol.c static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) · line 5040
Quoted 6 lines
mm/memcontrol.c +6
-0+struct sysinfo i; +si_swapinfo (&i); + +return ((memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ) + + i.totalswap - i.freeswap;How swapcache is handled ? ...and How kmem works with this calc ?
I am ignoring kmem, because we don't account kmem for the root cgroup anyway.
Setting the limit is invalid, and we don't account until the limit is set. Then it will be 0, always.
For swapcache, I am hoping that totalswap - freeswap will cover everything swap related. If you think I am wrong, please enlighten me.
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 12 lines · Glauber Costa
Quoted 11 lines · Kamezawa Hiroyuki
mm/memcontrol.c static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) · line 5040
Quoted 7 lines · Glauber Costa
mm/memcontrol.c +7
-0+case _MEMSWAP: { +struct sysinfo i; +si_swapinfo (&i); + +return ((memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ) + + i.totalswap - i.freeswap;How swapcache is handled ? ...and How kmem works with this calc ?
Quoted 9 lines
I am ignoring kmem, because we don't account kmem for the root cgroup anyway.
Setting the limit is invalid, and we don't account until the limit is set. Then it will be 0, always.
For swapcache, I am hoping that totalswap - freeswap will cover everything swap related. If you think I am wrong, please enlighten me.
i.totalswap - i.freeswap = # of used swap entries.
SwapCache can be rss and used swap entry at the same time.
Thanks,
-Kame
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 19 lines · Kamezawa Hiroyuki
Quoted 18 lines · Glauber Costa
Quoted 11 lines · Kamezawa Hiroyuki
mm/memcontrol.c static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) · line 5040
Quoted 7 lines · Glauber Costa
mm/memcontrol.c +7
-0+case _MEMSWAP: { +struct sysinfo i; +si_swapinfo (&i); + +return ((memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ) + + i.totalswap - i.freeswap;How swapcache is handled ? ...and How kmem works with this calc ?
I am ignoring kmem, because we don't account kmem for the root cgroup anyway.
Setting the limit is invalid, and we don't account until the limit is set. Then it will be 0, always.
Quoted 8 lines
Quoted 3 lines
For swapcache, I am hoping that totalswap - freeswap will cover everything swap related. If you think I am wrong, please enlighten me.
i.totalswap - i.freeswap = # of used swap entries.
SwapCache can be rss and used swap entry at the same time.
Well, yes, but the rss entries would be accounted for in get_mm_rss(), won't they ?
What am I missing ?
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 26 lines · Glauber Costa
Quoted 25 lines · Kamezawa Hiroyuki
Quoted 12 lines · Glauber Costa
Quoted 11 lines · Kamezawa Hiroyuki
mm/memcontrol.c static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) · line 5040
Quoted 7 lines · Glauber Costa
mm/memcontrol.c +7
-0+case _MEMSWAP: { +struct sysinfo i; +si_swapinfo (&i); + +return ((memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ) + + i.totalswap - i.freeswap;How swapcache is handled ? ...and How kmem works with this calc ?
Quoted 9 lines
I am ignoring kmem, because we don't account kmem for the root cgroup anyway.
Setting the limit is invalid, and we don't account until the limit is set. Then it will be 0, always.
For swapcache, I am hoping that totalswap - freeswap will cover everything swap related. If you think I am wrong, please enlighten me.
i.totalswap - i.freeswap = # of used swap entries.
Quoted 7 lines
Quoted 2 lines
SwapCache can be rss and used swap entry at the same time.
Well, yes, but the rss entries would be accounted for in
get_mm_rss(), won't they ?What am I missing ?
I think the correct caluculation is
Sum of all RSS + All file caches + (i.total_swap - i.freeswap - # of mapped SwapCache)
In the patch, mapped SwapCache is counted as both of rss and swap.
BTW, how about
Sum of all LRU + (i.total_swap - i.freeswap - # of all SwapCache)
?
Thanks,
-Kame
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 36 lines · Kamezawa Hiroyuki
Quoted 26 lines · Glauber Costa
Quoted 25 lines · Kamezawa Hiroyuki
Quoted 12 lines · Glauber Costa
Quoted 11 lines · Kamezawa Hiroyuki
mm/memcontrol.c static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) · line 5040
Quoted 7 lines · Glauber Costa
+ case
_MEMSWAP: {mm/memcontrol.c +6
-0+struct sysinfo i; +si_swapinfo (&i); + +return ((memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ) + + i.totalswap - i.freeswap;How swapcache is handled ? ...and How kmem works with this calc ?
Quoted 9 lines
I am ignoring kmem, because we don't account kmem for the root cgroup anyway.
Setting the limit is invalid, and we don't account until the limit is set. Then it will be 0, always.
For swapcache, I am hoping that totalswap - freeswap will cover everything swap related. If you think I am wrong, please enlighten me.
i.totalswap - i.freeswap = # of used swap entries.
Quoted 7 lines
Quoted 2 lines
SwapCache can be rss and used swap entry at the same time.
Well, yes, but the rss entries would be accounted for in
get_mm_rss(), won't they ?What am I missing ?
Quoted 7 lines
I think the correct caluculation is
Sum of all RSS + All file caches + (i.total_swap - i.freeswap - # of mapped SwapCache)
In the patch, mapped SwapCache is counted as both of rss and swap.
After a quick look, swapcache is counted as file pages and meanwhile use a swap entry at the same time(__add_to{delete_from}_swap_cache()). Even though, I think we still do not need to exclude swapcache out, because it indeed uses two copy of resource: one is swap entry, one is cache, so the usage should count both of them in.
What I think it matters is that swapcache may be counted as both file pages and rss(if it's a process's anonymous page), which we need to subtract # of swapcache to avoid double-counting. But it isn't always
so: a shmem/tmpfs page may use swapcache and be counted as file pages but not a rss, then we can not subtract swapcache... Is there anything I lost?
Quoted 16 lines
BTW, how about
Sum of all LRU + (i.total_swap - i.freeswap - # of all SwapCache)
?Thanks,
-Kame
Signature · 2 lines
Thanks, Sha
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 52 lines · Sha Zhengju
Quoted 36 lines · Kamezawa Hiroyuki
Quoted 26 lines · Glauber Costa
Quoted 25 lines · Kamezawa Hiroyuki
Quoted 12 lines · Glauber Costa
Quoted 11 lines · Kamezawa Hiroyuki
Review
Quoted 7 lines · Glauber Costa
+ case
_MEMSWAP: {(quoted diff) +6
-0+ struct sysinfo i; + si_swapinfo(&i); + + return ((memcg_read_root_rss() + + atomic_long_read(&vm_stat[NR_FILE_PAGES])) << PAGE_SHIFT) + + i.totalswap - i.freeswap;How swapcache is handled ? ...and How kmem works with this calc ?
Quoted 9 lines
I am ignoring kmem, because we don't account kmem for the root cgroup anyway.
Setting the limit is invalid, and we don't account until the limit is set. Then it will be 0, always.
For swapcache, I am hoping that totalswap - freeswap will cover everything swap related. If you think I am wrong, please enlighten me.
i.totalswap - i.freeswap = # of used swap entries.
Quoted 7 lines
Quoted 2 lines
SwapCache can be rss and used swap entry at the same time.
Well, yes, but the rss entries would be accounted for in
get_mm_rss(), won't they ?What am I missing ?
Quoted 7 lines
I think the correct caluculation is
Sum of all RSS + All file caches + (i.total_swap - i.freeswap - # of mapped SwapCache)
In the patch, mapped SwapCache is counted as both of rss and swap.
After a quick look, swapcache is counted as file pages and meanwhile use a swap entry at the same time(__add_to{delete_from}_swap_cache()). Even though, I think we still do not need to exclude swapcache out, because it indeed uses two copy of resource: one is swap entry, one is cache, so the usage should count both of them in.
Quoted 7 lines
What I think it matters is that swapcache may be counted as both file pages and rss(if it's a process's anonymous page), which we need to subtract # of swapcache to avoid double-counting. But it isn't always
so: a shmem/tmpfs page may use swapcache and be counted as file pages but not a rss, then we can not subtract swapcache... Is there anything I lost?
Please don't think difficult. All pages for user/caches are counted in LRU. All swap-entry usage can be cauht by total_swap_pages - nr_swap_pages. We just need to subtract number of swap-cache which is double counted as swap-entry and a page in LRU.
NR_ACTIVE_ANON + NR_INACTIVE_ANON + NR_ACTIVE_FILE + NR_INACTIVE_FILE
+ NR_UNEVICTABLE + total_swap_pages - nr_swap_pages - NR_SWAP_CACHE
is the number we whant for memsw.usage_in_bytes.
Thanks,
-Kame
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 73 lines · Kamezawa Hiroyuki
Quoted 63 lines · Sha Zhengju
Quoted 45 lines · Kamezawa Hiroyuki
Quoted 33 lines · Glauber Costa
Quoted 31 lines · Kamezawa Hiroyuki
Quoted 17 lines · Glauber Costa
Quoted 15 lines · Kamezawa Hiroyuki
Review
Quoted 9 lines · Glauber Costa
+ case
_MEMSWAP: {(quoted diff) +7
-0+ struct sysinfo i; + si_swapinfo(&i); + + return ((memcg_read_root_rss() + + atomic_long_read(&vm_stat[NR_FILE_PAGES])) << PAGE_SHIFT) + + i.totalswap - i.freeswap;How swapcache is handled ? ...and How kmem works with this calc ?
Quoted 9 lines
I am ignoring kmem, because we don't account kmem for the root cgroup anyway.
Setting the limit is invalid, and we don't account until the limit is set. Then it will be 0, always.
For swapcache, I am hoping that totalswap - freeswap will cover everything swap related. If you think I am wrong, please enlighten me.
i.totalswap - i.freeswap = # of used swap entries.
Quoted 7 lines
Quoted 2 lines
SwapCache can be rss and used swap entry at the same time.
Well, yes, but the rss entries would be accounted for in
get_mm_rss(), won't they ?What am I missing ?
Quoted 8 lines
I think the correct caluculation is
Sum of all RSS + All file caches + (i.total_swap - i.freeswap - # of
mapped SwapCache)In the patch, mapped SwapCache is counted as both of rss and swap.
After a quick look, swapcache is counted as file pages and meanwhile use a swap entry at the same time(__add_to{delete_from}_swap_cache()). Even though, I think we still do not need to exclude swapcache out, because it indeed uses two copy of resource: one is swap entry, one is cache, so the usage should count both of them in.
Quoted 7 lines
What I think it matters is that swapcache may be counted as both file pages and rss(if it's a process's anonymous page), which we need to subtract # of swapcache to avoid double-counting. But it isn't always
so: a shmem/tmpfs page may use swapcache and be counted as file pages but not a rss, then we can not subtract swapcache... Is there anything I lost?
Quoted 8 lines
Please don't think difficult. All pages for user/caches are counted in LRU. All swap-entry usage can be cauht by
total_swap_pages-nr_swap_pages. We just need to subtract number of swap-cache which is double counted as swap-entry and a page in LRU.
NR_ACTIVE_ANON+NR_INACTIVE_ANON+NR_ACTIVE_FILE+NR_INACTIVE_FILE
+NR_UNEVICTABLE+total_swap_pages-nr_swap_pages-NR_SWAP_CACHE
Using LRU numbers is more suitable. But forgive me, I still doubt whether we should subtract NR_SWAP_CACHE out because it uses both a swap entry and a page cache and it isn't a real double counting.
Thanks,
Sha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 80 lines · Sha Zhengju
Quoted 78 lines · Kamezawa Hiroyuki
Quoted 63 lines · Sha Zhengju
Quoted 45 lines · Kamezawa Hiroyuki
Quoted 33 lines · Glauber Costa
Quoted 31 lines · Kamezawa Hiroyuki
Quoted 17 lines · Glauber Costa
Quoted 15 lines · Kamezawa Hiroyuki
Review
Quoted 9 lines · Glauber Costa
+ case
_MEMSWAP: {(quoted diff) +7
-0+ struct sysinfo i; + si_swapinfo(&i); + + return ((memcg_read_root_rss() + + atomic_long_read(&vm_stat[NR_FILE_PAGES])) << PAGE_SHIFT) + + i.totalswap - i.freeswap;How swapcache is handled ? ...and How kmem works with this calc ?
Quoted 9 lines
I am ignoring kmem, because we don't account kmem for the root cgroup anyway.
Setting the limit is invalid, and we don't account until the limit is set. Then it will be 0, always.
For swapcache, I am hoping that totalswap - freeswap will cover everything swap related. If you think I am wrong, please enlighten me.
i.totalswap - i.freeswap = # of used swap entries.
Quoted 7 lines
Quoted 2 lines
SwapCache can be rss and used swap entry at the same time.
Well, yes, but the rss entries would be accounted for in
get_mm_rss(), won't they ?What am I missing ?
Quoted 8 lines
I think the correct caluculation is
Sum of all RSS + All file caches + (i.total_swap - i.freeswap - # of
mapped SwapCache)In the patch, mapped SwapCache is counted as both of rss and swap.
After a quick look, swapcache is counted as file pages and meanwhile use a swap entry at the same time(__add_to{delete_from}_swap_cache()). Even though, I think we still do not need to exclude swapcache out, because it indeed uses two copy of resource: one is swap entry, one is cache, so the usage should count both of them in.
Quoted 7 lines
What I think it matters is that swapcache may be counted as both file pages and rss(if it's a process's anonymous page), which we need to subtract # of swapcache to avoid double-counting. But it isn't always
so: a shmem/tmpfs page may use swapcache and be counted as file pages but not a rss, then we can not subtract swapcache... Is there anything I lost?Please don't think difficult. All pages for user/caches are counted in LRU. All swap-entry usage can be cauht by
total_swap_pages-nr_swap_pages. We just need to subtract number of swap-cache which is double counted as swap-entry and a page in LRU.
Quoted 8 lines
Quoted 3 lines
NR_ACTIVE_ANON+NR_INACTIVE_ANON+NR_ACTIVE_FILE+NR_INACTIVE_FILE
+NR_UNEVICTABLE+total_swap_pages-nr_swap_pages-NR_SWAP_CACHEUsing LRU numbers is more suitable. But forgive me, I still doubt whether we should subtract
NR_SWAP_CACHEout because it uses both a swap entry and a page cache and it isn't a real double counting.
Used swap entry can be reclaimed if there are SwapCache on memory.
Thanks,
ーKame
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
In reply to Glauber CostaQuoted 56 lines · Glauber Costa
Quoted 12 lines · Kamezawa Hiroyuki
Quoted 11 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.Quoted 9 lines
Quoted 6 lines
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>I think this patch's calculation is wrong.
where exactly ?
Quoted 23 lines
Quoted 23 lines
--- mm/memcontrol.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)mm/memcontrol.c +11
-0@@ -4996,6 +4996,56 @@ static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap)return val <<PAGE_SHIFT ; } +static u64 memcg_read_root_rss (void ) +{ +struct task_struct *p; + +u64 rss =0 ; +read_lock (&tasklist_lock); +for_each_process (p) { +if (!p->mm) +continue ; +task_lock (p); + rss +=get_mm_rss (p->mm);Quoted 8 lines
mm/memcontrol.c static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) · line 5014
Quoted 5 lines
mm/memcontrol.c +5
-0+task_unlock (p); + } +read_unlock (&tasklist_lock); +return rss; +}I think you can use
rcu_read_lock()instead oftasklist_lock. Isn't it enough to useNR_ANON_LRUrather than this ?
Quoted 2 lines
Is it really just
ANON_LRU?get_mm_rssalso include filepages, which are not in this list.
And mlocked ones counted as Unevictable
Quoted 3 lines
Maybe if we sum up *all* LRUs we would get the right result ?
_MEM...i.e. ...usage_in_bytes is the sum of all LRUs.
Quoted 2 lines
About the tasklist lock, if I get values from the LRUs, maybe. Otherwise it is still necessary, no ?
tasklist is RCU list and we don't need locking at reading values, I think.
Thanks,
-Kame
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
In reply to Glauber CostaQuoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.
I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if nr_cgroups == 1 and move relevant global counters to the root at the moment when a first group is created?
The patch aims at reducing an overhead when there there are no other groups, right?
Quoted 106 lines
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>--- mm/memcontrol.c | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+)mm/memcontrol.c +64
-0@@ -4996,6 +4996,56 @@ static inline u64 mem_cgroup_usage(struct mem_cgroup *memcg, bool swap)return val <<PAGE_SHIFT ; } +static u64 memcg_read_root_rss (void ) +{ +struct task_struct *p; + +u64 rss =0 ; +read_lock (&tasklist_lock); +for_each_process (p) { +if (!p->mm) +continue ; +task_lock (p); + rss +=get_mm_rss (p->mm); +task_unlock (p); + } +read_unlock (&tasklist_lock); +return rss; +} + +static u64 mem_cgroup_read_root (enum res_type type,int name) +{ +if (name ==RES_LIMIT ) +return RESOURCE_MAX ; +if (name ==RES_SOFT_LIMIT ) +return RESOURCE_MAX ; +if (name ==RES_FAILCNT ) +return 0 ; +if (name ==RES_MAX_USAGE ) +return 0 ; + +if (WARN_ON_ONCE (name !=RES_USAGE )) +return 0 ; + +switch (type) { +case _MEM: +return (memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ; +case _MEMSWAP: { +struct sysinfo i; +si_swapinfo (&i); + +return ((memcg_read_root_rss () + +atomic_long_read (&vm_stat[NR_FILE_PAGES ])) <<PAGE_SHIFT ) + + i.totalswap - i.freeswap; + } +case _KMEM: +return 0 ; +default : +BUG (); + }; +} +static ssize_t mem_cgroup_read (struct cgroup *cont,struct cftype *cft,struct file *file,char __user *buf,size_t nbytes,loff_t *ppos)@@ -5012,6 +5062,19 @@ static ssize_t mem_cgroup_read(struct cgroup *cont, struct cftype *cft,if (!do_swap_account && type == _MEMSWAP)return -EOPNOTSUPP ; +/* +* If we have root-level hierarchy, we can be certain that the charges +* in root are always global. We can then bypass the root cgroup +* entirely in this case, hopefuly leading to less contention in the +* root res_counters. The charges presented after reading it will +* always be the global charges. +*/ +if (mem_cgroup_disabled () || + (mem_cgroup_is_root (memcg) && memcg->use_hierarchy)) { + val =mem_cgroup_read_root (type, name); + goto root_bypass; + } +switch (type) {case _MEM:if (name ==RES_USAGE )@@ -5032,6 +5095,7 @@ static ssize_t mem_cgroup_read(struct cgroup *cont, struct cftype *cft,BUG (); } +root_bypass: len =scnprintf (str,sizeof (str),"%llu\n" , (unsigned long long )val);return simple_read_from_buffer (buf, nbytes, ppos, str, len); }Signature · 6 lines
1.8.1.2 -- To unsubscribe from this list: send the line "unsubscribe cgroups" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.
Quoted 4 lines
I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?
OK, it seems that the very next patch does what I was looking for. So why all the churn in this patch?
Why do you want to make root even more special?
[...]
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 13 lines · Michal Hocko
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.
Quoted 8 lines
Quoted 4 lines
I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?OK, it seems that the very next patch does what I was looking for. So why all the churn in this patch?
Why do you want to make root even more special?
Because I am operating under the assumption that we want to handle that transparently and keep things working. If you tell me: "Hey, reading memory.usage_in_bytes from root should return 0!", then I can get rid of that. The fact that I keep bypassing when hierarchy is present, it is more of a reuse of the infrastructure since it's there anyway.
Also, I would like the root memcg to be usable, albeit cheap, for projects like memory pressure notifications.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 19 lines · Glauber Costa
Quoted 18 lines · Michal Hocko
Quoted 16 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?
Quoted 8 lines
Quoted 3 lines
OK, it seems that the very next patch does what I was looking for. So why all the churn in this patch?
Why do you want to make root even more special?Because I am operating under the assumption that we want to handle that transparently and keep things working. If you tell me: "Hey, reading memory.usage_in_bytes from root should return 0!", then I can get rid of that.
If you simply switch to accounting for root then you do not have to care about this, don't you?
Quoted 5 lines
The fact that I keep bypassing when hierarchy is present, it is more of a reuse of the infrastructure since it's there anyway.
Also, I would like the root memcg to be usable, albeit cheap, for projects like memory pressure notifications.
root memcg without any childre, right?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 24 lines · Michal Hocko
Quoted 23 lines · Glauber Costa
Quoted 21 lines · Michal Hocko
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.Quoted 4 lines
I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?OK, it seems that the very next patch does what I was looking for. So why all the churn in this patch?
Why do you want to make root even more special?
Quoted 8 lines
Quoted 4 lines
Because I am operating under the assumption that we want to handle that transparently and keep things working. If you tell me: "Hey, reading memory.usage_in_bytes from root should return 0!", then I can get rid of that.
If you simply switch to accounting for root then you do not have to care about this, don't you?
Of course not, but the whole point here is *not* accounting root. So if we are entirely skipping root account, it, I personally believe we need to replace it with something else so we can keep things working as much as we can.
It doesn't need to be perfect, though: There is no way we can have max_usage without something like a res_counter that locks memory charges. I believe we can live without that. But as for the basic statistics and numbers, I believe they should keep working.
Quoted 8 lines
Quoted 5 lines
The fact that I keep bypassing when hierarchy is present, it is more of a reuse of the infrastructure since it's there anyway.
Also, I would like the root memcg to be usable, albeit cheap, for projects like memory pressure notifications.
root memcg without any childre, right?
yes, of course.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 30 lines · Glauber Costa
Quoted 29 lines · Michal Hocko
Quoted 27 lines · Glauber Costa
Quoted 13 lines · Michal Hocko
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.Quoted 8 lines
Quoted 4 lines
I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?OK, it seems that the very next patch does what I was looking for. So why all the churn in this patch?
Why do you want to make root even more special?Because I am operating under the assumption that we want to handle that transparently and keep things working. If you tell me: "Hey, reading memory.usage_in_bytes from root should return 0!", then I can get rid of that.
Quoted 4 lines
Quoted 3 lines
If you simply switch to accounting for root then you do not have to care about this, don't you?
Of course not, but the whole point here is *not* accounting root.
I thought the objective was to not account root if there are no children. I would see the "not account root at all" as another step. And we are skipping charging it already (do not call mem_cgroup_do_charge) for root.
Quoted 8 lines
So if we are entirely skipping root account, it, I personally believe we need to replace it with something else so we can keep things working as much as we can.
It doesn't need to be perfect, though: There is no way we can have
max_usagewithout something like ares_counterthat locks memory charges. I believe we can live without that. But as for the basic statistics and numbers, I believe they should keep working.
[...]
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 31 lines · Michal Hocko
Quoted 30 lines · Glauber Costa
Quoted 29 lines · Michal Hocko
Quoted 27 lines · Glauber Costa
Quoted 13 lines · Michal Hocko
Quoted 12 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.Quoted 8 lines
Quoted 4 lines
I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?OK, it seems that the very next patch does what I was looking for. So why all the churn in this patch?
Why do you want to make root even more special?Because I am operating under the assumption that we want to handle that transparently and keep things working. If you tell me: "Hey, reading memory.usage_in_bytes from root should return 0!", then I can get rid of that.
Quoted 7 lines
Quoted 4 lines
Quoted 3 lines
If you simply switch to accounting for root then you do not have to care about this, don't you?
Of course not, but the whole point here is *not* accounting root.
I thought the objective was to not account root if there are no children.
It is the goal, yes. As I said: I want the root-only case to keep providing userspace with meaningful statistics, therefore the bypass. But since the machinery is in place, it is trivial to keep bypassing for use_hierarchy = 1 at the root level. If you believe it would be simpler, I could refrain from doing it.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 35 lines · Glauber Costa
Quoted 34 lines · Michal Hocko
Quoted 33 lines · Glauber Costa
Quoted 32 lines · Michal Hocko
Quoted 19 lines · Glauber Costa
Quoted 18 lines · Michal Hocko
Quoted 16 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?Quoted 8 lines
Quoted 3 lines
OK, it seems that the very next patch does what I was looking for. So why all the churn in this patch?
Why do you want to make root even more special?Because I am operating under the assumption that we want to handle that transparently and keep things working. If you tell me: "Hey, reading memory.usage_in_bytes from root should return 0!", then I can get rid of that.
If you simply switch to accounting for root then you do not have to care about this, don't you?
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Of course not, but the whole point here is *not* accounting root.
I thought the objective was to not account root if there are no children.
It is the goal, yes. As I said: I want the root-only case to keep providing userspace with meaningful statistics,
Sure, statistics need to stay at the place. I am not objecting on that.
Quoted 1 line
therefore the bypass.
I am just arguing about bypassing root even when there are children and use_hierarchy == 1 because it adds more code to maintain.
Quoted 3 lines
But since the machinery is in place, it is trivial to keep bypassing for
use_hierarchy= 1 at the root level. If you believe it would be simpler, I could refrain from doing it.
I am all for "the simple the better" and add more optimizations on top. We have a real issue now and we should eliminate it. My original plan was to look at the bottlenecks and eliminate them one after another in smaller steps. But all the work I have on the plate is preempting me from looking into that...
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 55 lines · Michal Hocko
Quoted 35 lines · Glauber Costa
Quoted 34 lines · Michal Hocko
Quoted 33 lines · Glauber Costa
Quoted 32 lines · Michal Hocko
Quoted 19 lines · Glauber Costa
Quoted 18 lines · Michal Hocko
Quoted 16 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?Quoted 8 lines
Quoted 3 lines
OK, it seems that the very next patch does what I was looking for. So why all the churn in this patch?
Why do you want to make root even more special?Because I am operating under the assumption that we want to handle that transparently and keep things working. If you tell me: "Hey, reading memory.usage_in_bytes from root should return 0!", then I can get rid of that.
If you simply switch to accounting for root then you do not have to care about this, don't you?
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Of course not, but the whole point here is *not* accounting root.
I thought the objective was to not account root if there are no children.
It is the goal, yes. As I said: I want the root-only case to keep providing userspace with meaningful statistics,
Sure, statistics need to stay at the place. I am not objecting on that.
Quoted 1 line
therefore the bypass.
I am just arguing about bypassing root even when there are children and
use_hierarchy== 1 because it adds more code to maintain.Quoted 3 lines
But since the machinery is in place, it is trivial to keep bypassing for
use_hierarchy= 1 at the root level. If you believe it would be simpler, I could refrain from doing it.
Quoted 6 lines
I am all for "the simple the better" and add more optimizations on top. We have a real issue now and we should eliminate it. My original plan was to look at the bottlenecks and eliminate them one after another in smaller steps. But all the work I have on the plate is preempting me from looking into that...
Been there, done that =)
I have no objections removing the special case for use_hierarchy == 1.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
Quoted 62 lines · Glauber Costa
Quoted 55 lines · Michal Hocko
Quoted 35 lines · Glauber Costa
Quoted 34 lines · Michal Hocko
Quoted 33 lines · Glauber Costa
Quoted 32 lines · Michal Hocko
Quoted 19 lines · Glauber Costa
Quoted 18 lines · Michal Hocko
Quoted 16 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?Quoted 8 lines
Quoted 3 lines
OK, it seems that the very next patch does what I was looking for. So why all the churn in this patch?
Why do you want to make root even more special?Because I am operating under the assumption that we want to handle that transparently and keep things working. If you tell me: "Hey, reading memory.usage_in_bytes from root should return 0!", then I can get rid of that.
If you simply switch to accounting for root then you do not have to care about this, don't you?
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Of course not, but the whole point here is *not* accounting root.
I thought the objective was to not account root if there are no children.
It is the goal, yes. As I said: I want the root-only case to keep providing userspace with meaningful statistics,
Sure, statistics need to stay at the place. I am not objecting on that.
Quoted 1 line
therefore the bypass.
I am just arguing about bypassing root even when there are children and
use_hierarchy== 1 because it adds more code to maintain.Quoted 3 lines
But since the machinery is in place, it is trivial to keep bypassing for
use_hierarchy= 1 at the root level. If you believe it would be simpler, I could refrain from doing it.Quoted 6 lines
I am all for "the simple the better" and add more optimizations on top. We have a real issue now and we should eliminate it. My original plan was to look at the bottlenecks and eliminate them one after another in smaller steps. But all the work I have on the plate is preempting me from looking into that...
Quoted 4 lines
Been there, done that =)
I have no objections removing the special case for
use_hierarchy== 1.
I agree.
Thanks,
-Kame
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
In reply to Glauber CostaOn Wed, Mar 20, 2013 at 12:08:17PM +0400, Glauber Costa wrote:
[...]
Quoted 9 lines
Quoted 8 lines
Quoted 5 lines
The fact that I keep bypassing when hierarchy is present, it is more of a reuse of the infrastructure since it's there anyway.
Also, I would like the root memcg to be usable, albeit cheap, for projects like memory pressure notifications.
root memcg without any childre, right?
yes, of course.
Just want to raise a voice of support for this one. Thanks to Glauber's efforts, we might not need another memory pressure interface for CONFIG_MEMCG=n case, since we might have CONFIG_MEMCG=y that will be super cheap when used with just a root memcg w/o children, and still usable for mem pressure. So this particular scenario is actually in demand[1].
Thanks!
Anton
[1] http://lkml.indiana.edu/hypermail/linux/kernel/1302.2/03173.html
Re: [PATCH v2 2/5] memcg: provide root figures from system totals
In reply to Michal HockoQuoted 12 lines · Michal Hocko
Quoted 10 lines · Glauber Costa
For the root memcg, there is no need to rely on the
res_countersif hierarchy is enabled The sum of all mem cgroups plus the tasks in root itself, is necessarily the amount of memory used for the whole system. Since those figures are already kept somewhere anyway, we can just return them here, without too much hassle.Limit and soft limit can't be set for the root cgroup, so they are left at
RESOURCE_MAX. Failcnt is left at 0, because its actual meaning is how many times we failed allocations due to the limit being hit. We will fail allocations in the root cgroup, but the limit will never the reason.
Quoted 7 lines
I do not like this very much to be honest. It just adds more hackery... Why cannot we simply not account if
nr_cgroups== 1 and move relevant global counters to the root at the moment when a first group is created?
The patch aims at reducing an overhead when there there are no other groups, right?
You've already noted yourself that this is done in a later patch.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v2 3/5 memcg: make it suck faster
In reply to Glauber CostaIt is an accepted fact that memcg sucks. But can it suck faster? Or in a more fair statement, can it at least stop draining everyone's performance when it is not in use?
This experimental and slightly crude patch demonstrates that we can do that by using static branches to patch it out until the first memcg comes to life. There are edges to be trimmed, and I appreciate comments for direction. In particular, the events in the root are not fired, but I believe this can be done without further problems by calling a specialized event check from mem_cgroup_newpage_charge().
My goal was to have enough numbers to demonstrate the performance gain that can come from it. I tested it in a 24-way 2-socket Intel box, 24 Gb mem. I used Mel Gorman's pft test, that he used to demonstrate this problem back in the Kernel Summit. There are three kernels:
nomemcg : memcg compile disabled.
base : memcg enabled, patch not applied.
bypassed : memcg enabled, with patch applied.
base bypassed
User 109.12 105.64
System 1646.84 1597.98
Elapsed 229.56 215.76
nomemcg bypassed
User 104.35 105.64
System 1578.19 1597.98
Elapsed 212.33 215.76
So as one can see, the difference between base and nomemcg in terms of both system time and elapsed time is quite drastic, and consistent with the figures shown by Mel Gorman in the Kernel summit. This is a ~ 7 % drop in performance, just by having memcg enabled. memcg functions appear heavily in the profiles, even if all tasks lives in the root memcg.
With bypassed kernel, we drop this down to 1.5 %, which starts to fall in the acceptable range. More investigation is needed to see if we can claim that last percent back, but I believe at last part of it should be.
--- include/linux/memcontrol.h | 72 ++++++++++++++++---- mm/memcontrol.c | 166 +++++++++++++++++++++++++++++++++++++++++---- mm/page_cgroup.c | 4 +- 3 files changed, 216 insertions(+), 26 deletions(-)
include/linux/memcontrol.h +60 -12
@@ -42,6 +42,26 @@ struct mem_cgroup_reclaim_cookie { };#ifdef CONFIG_MEMCG +extern struct static_key memcg_in_use_key; + +static inline bool mem_cgroup_subsys_disabled (void ) +{ +return !!mem_cgroup_subsys.disabled; +} + +static inline bool mem_cgroup_disabled (void ) +{ +/* +* Will always be false if subsys is disabled, because we have no one +* to bump it up. So the test suffices and we don't have to test the +* subsystem as well +*/ +if (!static_key_false (&memcg_in_use_key)) +return true; +return false; +} + +/* * All "charge" functions with gfp_mask should use GFP_KERNEL or * (gfp_mask & GFP_RECLAIM_MASK). In current implementatin, memcg doesn't
@@ -53,8 +73,18 @@ struct mem_cgroup_reclaim_cookie {* (Of course, if memcg does memory allocation in future, GFP_KERNEL is sane.) */ -extern int mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm, +extern int __mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask); + +static inline int +mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask) +{ +if (mem_cgroup_disabled ()) +return 0 ; +return __mem_cgroup_newpage_charge (page, mm, gfp_mask); +} +/* for swap handling */ extern int mem_cgroup_try_charge_swapin (struct mm_struct *mm,struct page *page,gfp_t mask,struct mem_cgroup **memcgp);
@@ -62,8 +92,17 @@ extern void mem_cgroup_commit_charge_swapin(struct page *page, struct mem_cgroup *memcg);extern void mem_cgroup_cancel_charge_swapin (struct mem_cgroup *memcg); -extern int mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, -gfp_t gfp_mask); + +extern int __mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask); +static inline int +mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask) +{ +if (mem_cgroup_disabled ()) +return 0 ; + +return __mem_cgroup_cache_charge (page, mm, gfp_mask); +}struct lruvec *mem_cgroup_zone_lruvec (struct zone *,struct mem_cgroup *);struct lruvec *mem_cgroup_page_lruvec (struct page *,struct zone *);
@@ -72,8 +111,24 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *, struct zone *);extern void mem_cgroup_uncharge_start (void );extern void mem_cgroup_uncharge_end (void ); -extern void mem_cgroup_uncharge_page (struct page *page); -extern void mem_cgroup_uncharge_cache_page (struct page *page); +extern void __mem_cgroup_uncharge_page (struct page *page); +extern void __mem_cgroup_uncharge_cache_page (struct page *page); + +static inline void mem_cgroup_uncharge_page (struct page *page) +{ +if (mem_cgroup_disabled ()) +return ; + +__mem_cgroup_uncharge_page (page); +} + +static inline void mem_cgroup_uncharge_cache_page (struct page *page) +{ +if (mem_cgroup_disabled ()) +return ; + +__mem_cgroup_uncharge_cache_page (page); +}bool __mem_cgroup_same_or_subtree (const struct mem_cgroup *root_memcg,struct mem_cgroup *memcg);
@@ -128,13 +183,6 @@ extern void mem_cgroup_replace_page_cache(struct page *oldpage,extern int do_swap_account;#endif -static inline bool mem_cgroup_disabled (void ) -{ -if (mem_cgroup_subsys.disabled) -return true; -return false; -} -void __mem_cgroup_begin_update_page_stat (struct page *page,bool *locked,unsigned long *flags);
mm/memcontrol.c +154 -12
254 lines not on this page · raw message
mm/page_cgroup.c +2 -2
@@ -68,7 +68,7 @@ void __init page_cgroup_init_flatmem(void)int nid, fail; -if (mem_cgroup_disabled ()) +if (mem_cgroup_subsys_disabled ())return ;for_each_online_node (nid) {
@@ -271,7 +271,7 @@ void __init page_cgroup_init(void)unsigned long pfn;int nid; -if (mem_cgroup_disabled ()) +if (mem_cgroup_subsys_disabled ())return ;for_each_node_state (nid,N_MEMORY ) {
Signature · 1 line
1.8.1.2
Re: [PATCH v2 3/5] memcg: make it suck faster
Quoted 37 lines · Glauber Costa
It is an accepted fact that memcg sucks. But can it suck faster? Or in a more fair statement, can it at least stop draining everyone's performance when it is not in use?
This experimental and slightly crude patch demonstrates that we can do that by using static branches to patch it out until the first memcg comes to life. There are edges to be trimmed, and I appreciate comments for direction. In particular, the events in the root are not fired, but I believe this can be done without further problems by calling a specialized event check from
mem_cgroup_newpage_charge().My goal was to have enough numbers to demonstrate the performance gain that can come from it. I tested it in a 24-way 2-socket Intel box, 24 Gb mem. I used Mel Gorman's pft test, that he used to demonstrate this problem back in the Kernel Summit. There are three kernels:
nomemcg : memcg compile disabled.
base : memcg enabled, patch not applied.
bypassed : memcg enabled, with patch applied.base bypassed
User 109.12 105.64
System 1646.84 1597.98
Elapsed 229.56 215.76nomemcg bypassed
User 104.35 105.64
System 1578.19 1597.98
Elapsed 212.33 215.76So as one can see, the difference between base and nomemcg in terms of both system time and elapsed time is quite drastic, and consistent with the figures shown by Mel Gorman in the Kernel summit. This is a ~ 7 % drop in performance, just by having memcg enabled. memcg functions appear heavily in the profiles, even if all tasks lives in the root memcg.
Quoted 5 lines
With bypassed kernel, we drop this down to 1.5 %, which starts to fall in the acceptable range. More investigation is needed to see if we can claim that last percent back, but I believe at last part of it should be.
seems nice.
Quoted 6 lines
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>
After quick look, it seems most parts are good. But I have a concern.
At memcg enablement, you move the numbers from vm_stat[] to res_counters.
Why you need it ? It's not explained.
And if it's necessary, uncharge will leak because page_cgroup is not marked as PCG_USED, pc->mem_cgroup == NULL. So, res.usage will not be decreased.
Could you fix it if you need to move numbers to res_counter ?
Thanks,
-Kame
Quoted 171 lines
--- include/linux/memcontrol.h | 72 ++++++++++++++++---- mm/memcontrol.c | 166 +++++++++++++++++++++++++++++++++++++++++---- mm/page_cgroup.c | 4 +- 3 files changed, 216 insertions(+), 26 deletions(-)include/linux/memcontrol.h +60
-12@@ -42,6 +42,26 @@ struct mem_cgroup_reclaim_cookie { };#ifdef CONFIG_MEMCG +extern struct static_key memcg_in_use_key; + +static inline bool mem_cgroup_subsys_disabled (void ) +{ +return !!mem_cgroup_subsys.disabled; +} + +static inline bool mem_cgroup_disabled (void ) +{ +/* +* Will always be false if subsys is disabled, because we have no one +* to bump it up. So the test suffices and we don't have to test the +* subsystem as well +*/ +if (!static_key_false (&memcg_in_use_key)) +return true; +return false; +} + +/* * All "charge" functions with gfp_mask should use GFP_KERNEL or * (gfp_mask & GFP_RECLAIM_MASK). In current implementatin, memcg doesn't @@ -53,8 +73,18 @@ struct mem_cgroup_reclaim_cookie {* (Of course, if memcg does memory allocation in future, GFP_KERNEL is sane.) */ -extern int mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm, +extern int __mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask); + +static inline int +mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask) +{ +if (mem_cgroup_disabled ()) +return 0 ; +return __mem_cgroup_newpage_charge (page, mm, gfp_mask); +} +/* for swap handling */ extern int mem_cgroup_try_charge_swapin (struct mm_struct *mm,struct page *page,gfp_t mask,struct mem_cgroup **memcgp);@@ -62,8 +92,17 @@ extern void mem_cgroup_commit_charge_swapin(struct page *page, struct mem_cgroup *memcg);extern void mem_cgroup_cancel_charge_swapin (struct mem_cgroup *memcg); -extern int mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, -gfp_t gfp_mask); + +extern int __mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask); +static inline int +mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask) +{ +if (mem_cgroup_disabled ()) +return 0 ; + +return __mem_cgroup_cache_charge (page, mm, gfp_mask); +}struct lruvec *mem_cgroup_zone_lruvec (struct zone *,struct mem_cgroup *);struct lruvec *mem_cgroup_page_lruvec (struct page *,struct zone *);@@ -72,8 +111,24 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *, struct zone *);extern void mem_cgroup_uncharge_start (void );extern void mem_cgroup_uncharge_end (void ); -extern void mem_cgroup_uncharge_page (struct page *page); -extern void mem_cgroup_uncharge_cache_page (struct page *page); +extern void __mem_cgroup_uncharge_page (struct page *page); +extern void __mem_cgroup_uncharge_cache_page (struct page *page); + +static inline void mem_cgroup_uncharge_page (struct page *page) +{ +if (mem_cgroup_disabled ()) +return ; + +__mem_cgroup_uncharge_page (page); +} + +static inline void mem_cgroup_uncharge_cache_page (struct page *page) +{ +if (mem_cgroup_disabled ()) +return ; + +__mem_cgroup_uncharge_cache_page (page); +}bool __mem_cgroup_same_or_subtree (const struct mem_cgroup *root_memcg,struct mem_cgroup *memcg);@@ -128,13 +183,6 @@ extern void mem_cgroup_replace_page_cache(struct page *oldpage,extern int do_swap_account;#endif -static inline bool mem_cgroup_disabled (void ) -{ -if (mem_cgroup_subsys.disabled) -return true; -return false; -} -void __mem_cgroup_begin_update_page_stat (struct page *page,bool *locked,unsigned long *flags);mm/memcontrol.c +14
-1@@ -575,6 +575,9 @@ static inline bool mem_cgroup_is_root(struct mem_cgroup *memcg)return (memcg == root_mem_cgroup); } +static bool memcg_charges_allowed = false; +struct static_key memcg_in_use_key; +/* Writing them here to avoid exposing memcg's inner layout */ #if defined(CONFIG_INET ) && defined(CONFIG_MEMCG_KMEM )@@ -710,6 +713,7 @@ static void disarm_static_keys(struct mem_cgroup *memcg) {disarm_sock_keys (memcg);disarm_kmem_keys (memcg); +static_key_slow_dec (&memcg_in_use_key); }static void drain_all_stock_async (struct mem_cgroup *memcg);@@ -1109,6 +1113,9 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p)if (unlikely (!p))return NULL ; +if (mem_cgroup_disabled ()) +return root_mem_cgroup; +return mem_cgroup_from_css (task_subsys_state (p, mem_cgroup_subsys_id)); }@@ -1157,9 +1164,12 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root,struct mem_cgroup *memcg =NULL ;int id =0 ; -if (mem_cgroup_disabled ()) +if (mem_cgroup_subsys_disabled ())return NULL ; +if (mem_cgroup_disabled ()) +return root_mem_cgroup; +if (!root) root = root_mem_cgroup;@@ -1335,6 +1345,20 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) memcg = pc->mem_cgroup;/* +* Because we lazily enable memcg only after first child group is +* created, we can have memcg == 0. Because page cgroup is created with +* GFP_ZERO, and after charging, all page cgroups will have a non-zero
mm/memcontrol.c struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) · line 1360
Quoted 10 lines
mm/memcontrol.c +10
-0+* cgroup attached (even if root), we can be sure that this is a +* used-but-not-accounted page. (due to lazyness). We could get around +* that by scanning all pages on cgroup init is too expensive. We can +* ultimately pay, but prefer to just to defer the update until we get +* here. We could take the opportunity to set PageCgroupUsed, but it +* won't be that important for the root cgroup. +*/ +if (!memcg &&PageLRU (page)) + pc->mem_cgroup = memcg = root_mem_cgroup; +
Hmm, more problems with memcg==NULL may happen ;)
Quoted 231 lines
mm/memcontrol.c +130
-11+/* * Surreptitiously switch any uncharged offlist page to root: * an uncharged page off lru does nothing to secure * its former mem_cgroup from sudden removal. @@ -3845,11 +3869,18 @@ static int mem_cgroup_charge_common(struct page *page, struct mm_struct *mm,return 0 ; } -int mem_cgroup_newpage_charge (struct page *page, +int __mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask) { -if (mem_cgroup_disabled ()) +/* +* The branch is actually very likely before the first memcg comes in. +* But since the code is patched out, we'll never reach it. It is only +* reachable when the code is patched in, and in that case it is +* unlikely. It will only happen during initial charges move. +*/ +if (unlikely (!memcg_charges_allowed))return 0 ; +VM_BUG_ON (page_mapped (page));VM_BUG_ON (page->mapping && !PageAnon (page));VM_BUG_ON (!mm);@@ -3962,15 +3993,13 @@ void mem_cgroup_commit_charge_swapin(struct page *page, MEM_CGROUP_CHARGE_TYPE_ANON); } -int mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, -gfp_t gfp_mask) +int __mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask) {struct mem_cgroup *memcg =NULL ;enum charge_type type =MEM_CGROUP_CHARGE_TYPE_CACHE ;int ret; -if (mem_cgroup_disabled ()) -return 0 ;if (PageCompound (page))return 0 ;@@ -4050,9 +4079,6 @@ __mem_cgroup_uncharge_common(struct page *page, enum charge_type ctype,struct page_cgroup *pc;bool anon; -if (mem_cgroup_disabled ()) -return NULL ; -VM_BUG_ON (PageSwapCache (page));if (PageTransHuge (page)) {@@ -4144,7 +4170,7 @@ unlock_out:return NULL ; } -void mem_cgroup_uncharge_page (struct page *page) +void __mem_cgroup_uncharge_page (struct page *page) {/* early check. */ if (page_mapped (page))@@ -4155,7 +4181,7 @@ void mem_cgroup_uncharge_page(struct page *page)__mem_cgroup_uncharge_common (page,MEM_CGROUP_CHARGE_TYPE_ANON , false); } -void mem_cgroup_uncharge_cache_page (struct page *page) +void __mem_cgroup_uncharge_cache_page (struct page *page) {VM_BUG_ON (page_mapped (page));VM_BUG_ON (page->mapping);@@ -4220,6 +4246,9 @@ mem_cgroup_uncharge_swapcache(struct page *page, swp_entry_t ent, bool swapout)struct mem_cgroup *memcg;int ctype =MEM_CGROUP_CHARGE_TYPE_SWAPOUT ; +if (mem_cgroup_disabled ()) +return ; +if (!swapout)/* this was a swap cache but the swap is unused ! */ ctype =MEM_CGROUP_CHARGE_TYPE_DROP ;@@ -6364,6 +6393,59 @@ free_out:return ERR_PTR (error); } +static void memcg_update_root_statistics (void ) +{ +int cpu; +u64 pgin, pgout, faults, mjfaults; + + pgin = pgout = faults = mjfaults =0 ; +for_each_online_cpu (cpu) { +struct vm_event_state *ev = &per_cpu (vm_event_states, cpu); +struct mem_cgroup_stat_cpu *memcg_stat; + + memcg_stat =per_cpu_ptr (root_mem_cgroup->stat, cpu); + + memcg_stat->events[MEM_CGROUP_EVENTS_PGPGIN ] = + ev->event[PGPGIN ]; + memcg_stat->events[MEM_CGROUP_EVENTS_PGPGOUT ] = + ev->event[PGPGOUT ]; + memcg_stat->events[MEM_CGROUP_EVENTS_PGFAULT ] = + ev->event[PGFAULT ]; + memcg_stat->events[MEM_CGROUP_EVENTS_PGMAJFAULT ] = + ev->event[PGMAJFAULT ]; + + memcg_stat->nr_page_events = ev->event[PGPGIN ] + + ev->event[PGPGOUT ]; + } + + root_mem_cgroup->nocpu_base.count[MEM_CGROUP_STAT_RSS ] = +memcg_read_root_rss (); + root_mem_cgroup->nocpu_base.count[MEM_CGROUP_STAT_CACHE ] = +atomic_long_read (&vm_stat[NR_FILE_PAGES ]); + root_mem_cgroup->nocpu_base.count[MEM_CGROUP_STAT_FILE_MAPPED ] = +atomic_long_read (&vm_stat[NR_FILE_MAPPED ]); +} + +static void memcg_update_root_lru (void ) +{ +struct zone *zone; +struct lruvec *lruvec; +struct mem_cgroup_per_zone *mz; +enum lru_list lru; + +for_each_populated_zone (zone) { +spin_lock_irq (&zone->lru_lock); + lruvec = &zone->lruvec; + mz =mem_cgroup_zoneinfo (root_mem_cgroup, +zone_to_nid (zone),zone_idx (zone)); + +for (lru =LRU_BASE ; lru <NR_LRU_LISTS ; lru++) + mz->lru_size[lru] = +zone_page_state (zone,NR_LRU_BASE + lru); +spin_unlock_irq (&zone->lru_lock); + } +} +static int mem_cgroup_css_online (struct cgroup *cont) {@@ -6407,6 +6489,66 @@ mem_cgroup_css_online(struct cgroup *cont) } error =memcg_init_kmem (memcg, &mem_cgroup_subsys); + +if (!error) { +static_key_slow_inc (&memcg_in_use_key); +/* +* The strategy to avoid races here is to let the charges just +* be globally made until we lock the res counter. Since we are +* copying charges from global statistics, it doesn't really +* matter when we do it, as long as we are consistent. So even +* after the code is patched in, they will continue being +* globally charged due to memcg_charges_allowed being set to +* false. +* +* Once we hold the res counter lock, though, we can already +* safely flip it: We will go through with the charging to the +* root memcg, but won't be able to actually charge it: we have +* the lock. +* +* This works because the mm stats are only updated after the +* memcg charging suceeds. If we block the charge by holding +* the res_counter lock, no other charges will happen in the +* system until we release it. +* +* manipulation always safe because the write side is always +* under the memcg_mutex. +*/ +if (!memcg_charges_allowed) { +struct zone *zone; + +get_online_cpus (); +spin_lock (&root_mem_cgroup->res.lock); + + memcg_charges_allowed = true; + + root_mem_cgroup->res.usage = +mem_cgroup_read_root (RES_USAGE , _MEM); + root_mem_cgroup->memsw.usage = +mem_cgroup_read_root (RES_USAGE , _MEMSWAP); +/* +* The max usage figure is not entirely accurate. The +* memory may have been higher in the past. But since +* we don't track that globally, this is the best we +* can do. +*/ + root_mem_cgroup->res.max_usage = + root_mem_cgroup->res.usage; + root_mem_cgroup->memsw.max_usage = + root_mem_cgroup->memsw.usage; + +memcg_update_root_statistics (); +memcg_update_root_lru (); +/* +* We are now 100 % consistent and all charges are +* transfered. New charges should reach the +* res_counter directly. +*/ +spin_unlock (&root_mem_cgroup->res.lock); +put_online_cpus (); + } + } +mutex_unlock (&memcg_create_mutex);if (error) {/* mm/page_cgroup.c +2
-2@@ -68,7 +68,7 @@ void __init page_cgroup_init_flatmem(void)int nid, fail; -if (mem_cgroup_disabled ()) +if (mem_cgroup_subsys_disabled ())return ;for_each_online_node (nid) {@@ -271,7 +271,7 @@ void __init page_cgroup_init(void)unsigned long pfn;int nid; -if (mem_cgroup_disabled ()) +if (mem_cgroup_subsys_disabled ())return ;for_each_node_state (nid,N_MEMORY ) {
Re: [PATCH v2 3/5] memcg: make it suck faster
Quoted 9 lines
Quoted 8 lines
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <[email protected]>
Quoted 4 lines
After quick look, it seems most parts are good. But I have a concern.
At memcg enablement, you move the numbers from vm_stat[] to
res_counters.
Not only to res_counters. Mostly to mem_cgroup_stat_cpu, but I do move to res_counters as well.
Quoted 1 line
Why you need it ? It's not explained.
Because at this point, the bypass will no longer be in effect and we need accurate figures in root cgroup about what happened so far.
If we always have root-level hierarchy, then the bypass could go on forever. But if we have not, we'll need to rely on whatever was in there.
Quoted 3 lines
And if it's necessary, uncharge will leak because
page_cgroupis not marked asPCG_USED,pc->mem_cgroup== NULL. So, res.usage will not be decreased.
The same problem happen when deriving an mz from a page. Since pc->mem_cgroup will be NULL. I am interpreting that as "root mem cgroup".
Maybe even better would be to scan page cgroup writing a magic. Then if we see that magic we are sure it is an uninitialized pc.
Quoted 2 lines
Could you fix it if you need to move numbers to
res_counter?
At least for the pages in LRUs, I can scan them all, and update their page information. I am just wondering if this isn't a *very* expensive operation. Fine that we do it once, but still, is potentially scanning *all* pages in the system.
So I've basically decided it is better to interpret pc->mem_cgroup = NULL as this uninitialized state. (and can change to a magic)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 3/5] memcg: make it suck faster
Quoted 24 lines · Glauber Costa
Quoted 9 lines
Quoted 8 lines
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>Quoted 4 lines
After quick look, it seems most parts are good. But I have a concern.
At memcg enablement, you move the numbers from vm_stat[] to
res_counters.Not only to
res_counters. Mostly tomem_cgroup_stat_cpu, but I do move tores_countersas well.Quoted 1 line
Why you need it ? It's not explained.
Because at this point, the bypass will no longer be in effect and we need accurate figures in root cgroup about what happened so far.
If we always have root-level hierarchy, then the bypass could go on forever. But if we have not, we'll need to rely on whatever was in there.
Quoted 7 lines
Quoted 3 lines
And if it's necessary, uncharge will leak because
page_cgroupis not marked asPCG_USED,pc->mem_cgroup== NULL. So, res.usage will not be decreased.The same problem happen when deriving an mz from a page. Since
pc->mem_cgroupwill be NULL. I am interpreting that as "root mem cgroup".
yes.
Quoted 6 lines
Maybe even better would be to scan page cgroup writing a magic. Then if we see that magic we are sure it is an uninitialized pc.
Quoted 2 lines
Could you fix it if you need to move numbers to
res_counter?
Quoted 8 lines
At least for the pages in LRUs, I can scan them all, and update their page information. I am just wondering if this isn't a *very* expensive operation. Fine that we do it once, but still, is potentially scanning *all* pages in the system.
So I've basically decided it is better to interpret
pc->mem_cgroup= NULL as this uninitialized state. (and can change to a magic)
I think it can work.
Thanks,
-Kame
Re: [PATCH v2 3/5] memcg: make it suck faster
In reply to Glauber CostaQuoted 320 lines · Glauber Costa
It is an accepted fact that memcg sucks. But can it suck faster? Or in a more fair statement, can it at least stop draining everyone's performance when it is not in use?
This experimental and slightly crude patch demonstrates that we can do that by using static branches to patch it out until the first memcg comes to life. There are edges to be trimmed, and I appreciate comments for direction. In particular, the events in the root are not fired, but I believe this can be done without further problems by calling a specialized event check from
mem_cgroup_newpage_charge().My goal was to have enough numbers to demonstrate the performance gain that can come from it. I tested it in a 24-way 2-socket Intel box, 24 Gb mem. I used Mel Gorman's pft test, that he used to demonstrate this problem back in the Kernel Summit. There are three kernels:
nomemcg : memcg compile disabled.
base : memcg enabled, patch not applied.
bypassed : memcg enabled, with patch applied.base bypassed
User 109.12 105.64
System 1646.84 1597.98
Elapsed 229.56 215.76nomemcg bypassed
User 104.35 105.64
System 1578.19 1597.98
Elapsed 212.33 215.76So as one can see, the difference between base and nomemcg in terms of both system time and elapsed time is quite drastic, and consistent with the figures shown by Mel Gorman in the Kernel summit. This is a ~ 7 % drop in performance, just by having memcg enabled. memcg functions appear heavily in the profiles, even if all tasks lives in the root memcg.
With bypassed kernel, we drop this down to 1.5 %, which starts to fall in the acceptable range. More investigation is needed to see if we can claim that last percent back, but I believe at last part of it should be.
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>--- include/linux/memcontrol.h | 72 ++++++++++++++++---- mm/memcontrol.c | 166 +++++++++++++++++++++++++++++++++++++++++---- mm/page_cgroup.c | 4 +- 3 files changed, 216 insertions(+), 26 deletions(-)include/linux/memcontrol.h +60
-12@@ -42,6 +42,26 @@ struct mem_cgroup_reclaim_cookie { };#ifdef CONFIG_MEMCG +extern struct static_key memcg_in_use_key; + +static inline bool mem_cgroup_subsys_disabled (void ) +{ +return !!mem_cgroup_subsys.disabled; +} + +static inline bool mem_cgroup_disabled (void ) +{ +/* +* Will always be false if subsys is disabled, because we have no one +* to bump it up. So the test suffices and we don't have to test the +* subsystem as well +*/ +if (!static_key_false (&memcg_in_use_key)) +return true; +return false; +} + +/* * All "charge" functions with gfp_mask should use GFP_KERNEL or * (gfp_mask & GFP_RECLAIM_MASK). In current implementatin, memcg doesn't @@ -53,8 +73,18 @@ struct mem_cgroup_reclaim_cookie {* (Of course, if memcg does memory allocation in future, GFP_KERNEL is sane.) */ -extern int mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm, +extern int __mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask); + +static inline int +mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask) +{ +if (mem_cgroup_disabled ()) +return 0 ; +return __mem_cgroup_newpage_charge (page, mm, gfp_mask); +} +/* for swap handling */ extern int mem_cgroup_try_charge_swapin (struct mm_struct *mm,struct page *page,gfp_t mask,struct mem_cgroup **memcgp);@@ -62,8 +92,17 @@ extern void mem_cgroup_commit_charge_swapin(struct page *page, struct mem_cgroup *memcg);extern void mem_cgroup_cancel_charge_swapin (struct mem_cgroup *memcg); -extern int mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, -gfp_t gfp_mask); + +extern int __mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask); +static inline int +mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask) +{ +if (mem_cgroup_disabled ()) +return 0 ; + +return __mem_cgroup_cache_charge (page, mm, gfp_mask); +}struct lruvec *mem_cgroup_zone_lruvec (struct zone *,struct mem_cgroup *);struct lruvec *mem_cgroup_page_lruvec (struct page *,struct zone *);@@ -72,8 +111,24 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *, struct zone *);extern void mem_cgroup_uncharge_start (void );extern void mem_cgroup_uncharge_end (void ); -extern void mem_cgroup_uncharge_page (struct page *page); -extern void mem_cgroup_uncharge_cache_page (struct page *page); +extern void __mem_cgroup_uncharge_page (struct page *page); +extern void __mem_cgroup_uncharge_cache_page (struct page *page); + +static inline void mem_cgroup_uncharge_page (struct page *page) +{ +if (mem_cgroup_disabled ()) +return ; + +__mem_cgroup_uncharge_page (page); +} + +static inline void mem_cgroup_uncharge_cache_page (struct page *page) +{ +if (mem_cgroup_disabled ()) +return ; + +__mem_cgroup_uncharge_cache_page (page); +}bool __mem_cgroup_same_or_subtree (const struct mem_cgroup *root_memcg,struct mem_cgroup *memcg);@@ -128,13 +183,6 @@ extern void mem_cgroup_replace_page_cache(struct page *oldpage,extern int do_swap_account;#endif -static inline bool mem_cgroup_disabled (void ) -{ -if (mem_cgroup_subsys.disabled) -return true; -return false; -} -void __mem_cgroup_begin_update_page_stat (struct page *page,bool *locked,unsigned long *flags);mm/memcontrol.c +47
-12@@ -575,6 +575,9 @@ static inline bool mem_cgroup_is_root(struct mem_cgroup *memcg)return (memcg == root_mem_cgroup); } +static bool memcg_charges_allowed = false; +struct static_key memcg_in_use_key; +/* Writing them here to avoid exposing memcg's inner layout */ #if defined(CONFIG_INET ) && defined(CONFIG_MEMCG_KMEM )@@ -710,6 +713,7 @@ static void disarm_static_keys(struct mem_cgroup *memcg) {disarm_sock_keys (memcg);disarm_kmem_keys (memcg); +static_key_slow_dec (&memcg_in_use_key); }static void drain_all_stock_async (struct mem_cgroup *memcg);@@ -1109,6 +1113,9 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p)if (unlikely (!p))return NULL ; +if (mem_cgroup_disabled ()) +return root_mem_cgroup; +return mem_cgroup_from_css (task_subsys_state (p, mem_cgroup_subsys_id)); }@@ -1157,9 +1164,12 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root,struct mem_cgroup *memcg =NULL ;int id =0 ; -if (mem_cgroup_disabled ()) +if (mem_cgroup_subsys_disabled ())return NULL ; +if (mem_cgroup_disabled ()) +return root_mem_cgroup; +if (!root) root = root_mem_cgroup;@@ -1335,6 +1345,20 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) memcg = pc->mem_cgroup;/* +* Because we lazily enable memcg only after first child group is +* created, we can have memcg == 0. Because page cgroup is created with +* GFP_ZERO, and after charging, all page cgroups will have a non-zero +* cgroup attached (even if root), we can be sure that this is a +* used-but-not-accounted page. (due to lazyness). We could get around +* that by scanning all pages on cgroup init is too expensive. We can +* ultimately pay, but prefer to just to defer the update until we get +* here. We could take the opportunity to set PageCgroupUsed, but it +* won't be that important for the root cgroup. +*/ +if (!memcg &&PageLRU (page)) + pc->mem_cgroup = memcg = root_mem_cgroup; + +/* * Surreptitiously switch any uncharged offlist page to root: * an uncharged page off lru does nothing to secure * its former mem_cgroup from sudden removal. @@ -3845,11 +3869,18 @@ static int mem_cgroup_charge_common(struct page *page, struct mm_struct *mm,return 0 ; } -int mem_cgroup_newpage_charge (struct page *page, +int __mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask) { -if (mem_cgroup_disabled ()) +/* +* The branch is actually very likely before the first memcg comes in. +* But since the code is patched out, we'll never reach it. It is only +* reachable when the code is patched in, and in that case it is +* unlikely. It will only happen during initial charges move. +*/ +if (unlikely (!memcg_charges_allowed))return 0 ; +VM_BUG_ON (page_mapped (page));VM_BUG_ON (page->mapping && !PageAnon (page));VM_BUG_ON (!mm);@@ -3962,15 +3993,13 @@ void mem_cgroup_commit_charge_swapin(struct page *page, MEM_CGROUP_CHARGE_TYPE_ANON); } -int mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, -gfp_t gfp_mask) +int __mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask) {struct mem_cgroup *memcg =NULL ;enum charge_type type =MEM_CGROUP_CHARGE_TYPE_CACHE ;int ret; -if (mem_cgroup_disabled ()) -return 0 ;if (PageCompound (page))return 0 ;@@ -4050,9 +4079,6 @@ __mem_cgroup_uncharge_common(struct page *page, enum charge_type ctype,struct page_cgroup *pc;bool anon; -if (mem_cgroup_disabled ()) -return NULL ; -VM_BUG_ON (PageSwapCache (page));if (PageTransHuge (page)) {@@ -4144,7 +4170,7 @@ unlock_out:return NULL ; } -void mem_cgroup_uncharge_page (struct page *page) +void __mem_cgroup_uncharge_page (struct page *page) {/* early check. */ if (page_mapped (page))@@ -4155,7 +4181,7 @@ void mem_cgroup_uncharge_page(struct page *page)__mem_cgroup_uncharge_common (page,MEM_CGROUP_CHARGE_TYPE_ANON , false); } -void mem_cgroup_uncharge_cache_page (struct page *page) +void __mem_cgroup_uncharge_cache_page (struct page *page) {VM_BUG_ON (page_mapped (page));VM_BUG_ON (page->mapping);@@ -4220,6 +4246,9 @@ mem_cgroup_uncharge_swapcache(struct page *page, swp_entry_t ent, bool swapout)struct mem_cgroup *memcg;int ctype =MEM_CGROUP_CHARGE_TYPE_SWAPOUT ; +if (mem_cgroup_disabled ()) +return ; +if (!swapout)/* this was a swap cache but the swap is unused ! */ ctype =MEM_CGROUP_CHARGE_TYPE_DROP ;@@ -6364,6 +6393,59 @@ free_out:return ERR_PTR (error); } +static void memcg_update_root_statistics (void ) +{ +int cpu; +u64 pgin, pgout, faults, mjfaults; + + pgin = pgout = faults = mjfaults =0 ;
mm/memcontrol.c free_out: · line 6411
Quoted 10 lines
mm/memcontrol.c +10
-0+for_each_online_cpu (cpu) { +struct vm_event_state *ev = &per_cpu (vm_event_states, cpu); +struct mem_cgroup_stat_cpu *memcg_stat; + + memcg_stat =per_cpu_ptr (root_mem_cgroup->stat, cpu); + + memcg_stat->events[MEM_CGROUP_EVENTS_PGPGIN ] = + ev->event[PGPGIN ]; + memcg_stat->events[MEM_CGROUP_EVENTS_PGPGOUT ] = + ev->event[PGPGOUT ];
ev->event[PGPGIN/PGPGOUT] is counted in block layer(submit_bio()) and represents the exactly number of pagein/pageout, but memcg PGPGIN/PGPGOUT events only count it as an event and ignore the page size. So here we can't straightforward take the ev->events for use.
mm/memcontrol.c free_out: · line 6418
Quoted 7 lines
mm/memcontrol.c +7
-0+ memcg_stat->events[MEM_CGROUP_EVENTS_PGFAULT ] = + ev->event[PGFAULT ]; + memcg_stat->events[MEM_CGROUP_EVENTS_PGMAJFAULT ] = + ev->event[PGMAJFAULT ]; + + memcg_stat->nr_page_events = ev->event[PGPGIN ] + + ev->event[PGPGOUT ];
There's no valid memcg->nr_page_events until now, so the threshold notifier, but some people may use it even only root memcg exists. Moreover, using PGPGIN + PGPGOUT(exactly number of pagein + pageout) as nr_page_events is also inaccurate IMHO.
Quoted 125 lines
mm/memcontrol.c +90
-0+ } + + root_mem_cgroup->nocpu_base.count[MEM_CGROUP_STAT_RSS ] = +memcg_read_root_rss (); + root_mem_cgroup->nocpu_base.count[MEM_CGROUP_STAT_CACHE ] = +atomic_long_read (&vm_stat[NR_FILE_PAGES ]); + root_mem_cgroup->nocpu_base.count[MEM_CGROUP_STAT_FILE_MAPPED ] = +atomic_long_read (&vm_stat[NR_FILE_MAPPED ]); +} + +static void memcg_update_root_lru (void ) +{ +struct zone *zone; +struct lruvec *lruvec; +struct mem_cgroup_per_zone *mz; +enum lru_list lru; + +for_each_populated_zone (zone) { +spin_lock_irq (&zone->lru_lock); + lruvec = &zone->lruvec; + mz =mem_cgroup_zoneinfo (root_mem_cgroup, +zone_to_nid (zone),zone_idx (zone)); + +for (lru =LRU_BASE ; lru <NR_LRU_LISTS ; lru++) + mz->lru_size[lru] = +zone_page_state (zone,NR_LRU_BASE + lru); +spin_unlock_irq (&zone->lru_lock); + } +} +static int mem_cgroup_css_online (struct cgroup *cont) {@@ -6407,6 +6489,66 @@ mem_cgroup_css_online(struct cgroup *cont) } error =memcg_init_kmem (memcg, &mem_cgroup_subsys); + +if (!error) { +static_key_slow_inc (&memcg_in_use_key); +/* +* The strategy to avoid races here is to let the charges just +* be globally made until we lock the res counter. Since we are +* copying charges from global statistics, it doesn't really +* matter when we do it, as long as we are consistent. So even +* after the code is patched in, they will continue being +* globally charged due to memcg_charges_allowed being set to +* false. +* +* Once we hold the res counter lock, though, we can already +* safely flip it: We will go through with the charging to the +* root memcg, but won't be able to actually charge it: we have +* the lock. +* +* This works because the mm stats are only updated after the +* memcg charging suceeds. If we block the charge by holding +* the res_counter lock, no other charges will happen in the +* system until we release it. +* +* manipulation always safe because the write side is always +* under the memcg_mutex. +*/ +if (!memcg_charges_allowed) { +struct zone *zone; + +get_online_cpus (); +spin_lock (&root_mem_cgroup->res.lock); + + memcg_charges_allowed = true; + + root_mem_cgroup->res.usage = +mem_cgroup_read_root (RES_USAGE , _MEM); + root_mem_cgroup->memsw.usage = +mem_cgroup_read_root (RES_USAGE , _MEMSWAP); +/* +* The max usage figure is not entirely accurate. The +* memory may have been higher in the past. But since +* we don't track that globally, this is the best we +* can do. +*/ + root_mem_cgroup->res.max_usage = + root_mem_cgroup->res.usage; + root_mem_cgroup->memsw.max_usage = + root_mem_cgroup->memsw.usage; + +memcg_update_root_statistics (); +memcg_update_root_lru (); +/* +* We are now 100 % consistent and all charges are +* transfered. New charges should reach the +* res_counter directly. +*/ +spin_unlock (&root_mem_cgroup->res.lock); +put_online_cpus (); + } + } +mutex_unlock (&memcg_create_mutex);if (error) {/* mm/page_cgroup.c +2
-3@@ -68,7 +68,7 @@ void __init page_cgroup_init_flatmem(void)int nid, fail; -if (mem_cgroup_disabled ()) +if (mem_cgroup_subsys_disabled ())return ;for_each_online_node (nid) {@@ -271,7 +271,7 @@ void __init page_cgroup_init(void)unsigned long pfn;int nid; -if (mem_cgroup_disabled ()) +if (mem_cgroup_subsys_disabled ())return ;for_each_node_state (nid,N_MEMORY ) { --1.8.1.2
Signature · 2 lines
Thanks, Sha
Re: [PATCH v2 3/5] memcg: make it suck faster
Quoted 12 lines · Sha Zhengju
Quoted 12 lines
+static void
memcg_update_root_statistics(void)Quoted 11 lines
mm/memcontrol.c +11
-0+{ +int cpu; +u64 pgin, pgout, faults, mjfaults; + + pgin = pgout = faults = mjfaults =0 ; +for_each_online_cpu (cpu) { +struct vm_event_state *ev = &per_cpu (vm_event_states, cpu); +struct mem_cgroup_stat_cpu *memcg_stat; + + memcg_stat =per_cpu_ptr (root_mem_cgroup->stat, cpu); +
Quoted 9 lines
Quoted 4 lines
mm/memcontrol.c free_out: · line 6411
Quoted 4 lines
mm/memcontrol.c +4
-0+ memcg_stat->events[MEM_CGROUP_EVENTS_PGPGIN ] = + ev->event[PGPGIN ]; + memcg_stat->events[MEM_CGROUP_EVENTS_PGPGOUT ] = + ev->event[PGPGOUT ];ev->event[PGPGIN/PGPGOUT] is counted in block layer(submit_bio()) and represents the exactly number of pagein/pageout, but memcg PGPGIN/PGPGOUT events only count it as an event and ignore the page size. So here we can't straightforward take the
ev->eventsfor use.
You are right about that. Although I can't think of a straightforward way to handle this. Well, except for the obvious of adding another global statistic.
Quoted 7 lines
Quoted 7 lines
mm/memcontrol.c free_out: · line 6418
Quoted 7 lines
mm/memcontrol.c +7
-0+ memcg_stat->events[MEM_CGROUP_EVENTS_PGFAULT ] = + ev->event[PGFAULT ]; + memcg_stat->events[MEM_CGROUP_EVENTS_PGMAJFAULT ] = + ev->event[PGMAJFAULT ]; + + memcg_stat->nr_page_events = ev->event[PGPGIN ] + + ev->event[PGPGOUT ];
Quoted 5 lines
There's no valid
memcg->nr_page_eventsuntil now, so the threshold notifier, but some people may use it even only root memcg exists. Moreover, using PGPGIN + PGPGOUT(exactly number of pagein + pageout) asnr_page_eventsis also inaccurate IMHO.
Humm, I believe I can zero out this. Looking at the code again, this is not imported to userspace. It is just used to activate the thresholds and the delta of nr_page_events is a lot more important than nr_page_events.
Re: [PATCH v2 3/5] memcg: make it suck faster
In reply to Glauber CostaQuoted 25 lines · Glauber Costa
It is an accepted fact that memcg sucks. But can it suck faster? Or in a more fair statement, can it at least stop draining everyone's performance when it is not in use?
This experimental and slightly crude patch demonstrates that we can do that by using static branches to patch it out until the first memcg comes to life. There are edges to be trimmed, and I appreciate comments for direction. In particular, the events in the root are not fired, but I believe this can be done without further problems by calling a specialized event check from
mem_cgroup_newpage_charge().My goal was to have enough numbers to demonstrate the performance gain that can come from it. I tested it in a 24-way 2-socket Intel box, 24 Gb mem. I used Mel Gorman's pft test, that he used to demonstrate this problem back in the Kernel Summit. There are three kernels:
nomemcg : memcg compile disabled.
base : memcg enabled, patch not applied.
bypassed : memcg enabled, with patch applied.base bypassed
User 109.12 105.64
System 1646.84 1597.98
Elapsed 229.56 215.76
Quoted 4 lines
nomemcg bypassed
User 104.35 105.64
System 1578.19 1597.98
Elapsed 212.33 215.76
Do you have profiles for where we spend the time?
Quoted 36 lines
So as one can see, the difference between base and nomemcg in terms of both system time and elapsed time is quite drastic, and consistent with the figures shown by Mel Gorman in the Kernel summit. This is a ~ 7 % drop in performance, just by having memcg enabled. memcg functions appear heavily in the profiles, even if all tasks lives in the root memcg.
With bypassed kernel, we drop this down to 1.5 %, which starts to fall in the acceptable range. More investigation is needed to see if we can claim that last percent back, but I believe at last part of it should be.
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>--- include/linux/memcontrol.h | 72 ++++++++++++++++---- mm/memcontrol.c | 166 +++++++++++++++++++++++++++++++++++++++++---- mm/page_cgroup.c | 4 +- 3 files changed, 216 insertions(+), 26 deletions(-)include/linux/memcontrol.h +4
-0@@ -42,6 +42,26 @@ struct mem_cgroup_reclaim_cookie { };#ifdef CONFIG_MEMCG +extern struct static_key memcg_in_use_key; + +static inline bool mem_cgroup_subsys_disabled (void ) +{
include/linux/memcontrol.h struct mem_cgroup_reclaim_cookie { · line 58
Quoted 10 lines
include/linux/memcontrol.h +10
-0+return !!mem_cgroup_subsys.disabled; +} + +static inline bool mem_cgroup_disabled (void ) +{ +/* +* Will always be false if subsys is disabled, because we have no one +* to bump it up. So the test suffices and we don't have to test the +* subsystem as well +*/
but static_key_false adds an atomic read here which is more costly so I am not sure you are optimizing much.
Quoted 36 lines
include/linux/memcontrol.h +18
-3+if (!static_key_false (&memcg_in_use_key)) +return true; +return false; +} + +/* * All "charge" functions with gfp_mask should use GFP_KERNEL or * (gfp_mask & GFP_RECLAIM_MASK). In current implementatin, memcg doesn't @@ -53,8 +73,18 @@ struct mem_cgroup_reclaim_cookie {* (Of course, if memcg does memory allocation in future, GFP_KERNEL is sane.) */ -extern int mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm, +extern int __mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask); + +static inline int +mem_cgroup_newpage_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask) +{ +if (mem_cgroup_disabled ()) +return 0 ; +return __mem_cgroup_newpage_charge (page, mm, gfp_mask); +} +/* for swap handling */ extern int mem_cgroup_try_charge_swapin (struct mm_struct *mm,struct page *page,gfp_t mask,struct mem_cgroup **memcgp);@@ -62,8 +92,17 @@ extern void mem_cgroup_commit_charge_swapin(struct page *page, struct mem_cgroup *memcg);extern void mem_cgroup_cancel_charge_swapin (struct mem_cgroup *memcg); -extern int mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, -gfp_t gfp_mask); +
include/linux/memcontrol.h extern void mem_cgroup_commit_charge_swapin(struct page *page, · line 105
Quoted 10 lines
include/linux/memcontrol.h +10
-0+extern int __mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm, +gfp_t gfp_mask); +static inline int +mem_cgroup_cache_charge (struct page *page,struct mm_struct *mm,gfp_t gfp_mask) +{ +if (mem_cgroup_disabled ()) +return 0 ; + +return __mem_cgroup_cache_charge (page, mm, gfp_mask); +}
Are there any reasons to not get down to __mem_cgroup_try_charge? We will not be perfect, all right, because some wrappers already do some work but we should at least cover most of them.
I am also thinking whether this stab at charging path is not just an overkill. Wouldn't it suffice to do something like:
mm/memcontrol.c +1 -1
@@ -2707,7 +2707,7 @@ static int __mem_cgroup_try_charge(struct mm_struct *mm,* thread group leader migrates. It's possible that mm is not * set, if so charge the root memcg (happens for pagecache usage). */ -if (!*ptr && !mm) +if (!*ptr && (!mm || !static_key_false (&memcg_in_use_key))) *ptr = root_mem_cgroup; again:if (*ptr) {/* css should be a valid one */
We should get rid of the biggest overhead, no?
Quoted 2 lines
include/linux/memcontrol.h +0
-0struct lruvec *mem_cgroup_zone_lruvec (struct zone *,struct mem_cgroup *);struct lruvec *mem_cgroup_page_lruvec (struct page *,struct zone *);
[...]
Quoted 4 lines
mm/memcontrol.c +0
-0
[...]
mm/memcontrol.c struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) · line 1345
Quoted 2 lines
mm/memcontrol.c +0
-0@@ -1335,6 +1345,20 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) memcg = pc->mem_cgroup;
I would expect that you want to prevent lookup as well if there are no other groups.
mm/memcontrol.c struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) · line 1358
Quoted 13 lines
mm/memcontrol.c +12
-0/* +* Because we lazily enable memcg only after first child group is +* created, we can have memcg == 0. Because page cgroup is created with +* GFP_ZERO, and after charging, all page cgroups will have a non-zero +* cgroup attached (even if root), we can be sure that this is a +* used-but-not-accounted page. (due to lazyness). We could get around +* that by scanning all pages on cgroup init is too expensive. We can +* ultimately pay, but prefer to just to defer the update until we get +* here. We could take the opportunity to set PageCgroupUsed, but it +* won't be that important for the root cgroup. +*/ +if (!memcg &&PageLRU (page)) + pc->mem_cgroup = memcg = root_mem_cgroup;
Why not return page_cgroup_zoneinfo(root_mem_cgroup, page); This would require messing up with __mem_cgroup_uncharge_common but that doesn't sound incredibly crazy (to the local standard of course ;)).
Quoted 5 lines
mm/memcontrol.c +2
-0+ +/* * Surreptitiously switch any uncharged offlist page to root: * an uncharged page off lru does nothing to secure * its former mem_cgroup from sudden removal.
[...]
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v2 3/5] memcg: make it suck faster
Sorry all for taking a lot of time to reply to this. I've been really busy.
Quoted 26 lines · Michal Hocko
Quoted 25 lines · Glauber Costa
It is an accepted fact that memcg sucks. But can it suck faster? Or in a more fair statement, can it at least stop draining everyone's performance when it is not in use?
This experimental and slightly crude patch demonstrates that we can do that by using static branches to patch it out until the first memcg comes to life. There are edges to be trimmed, and I appreciate comments for direction. In particular, the events in the root are not fired, but I believe this can be done without further problems by calling a specialized event check from
mem_cgroup_newpage_charge().My goal was to have enough numbers to demonstrate the performance gain that can come from it. I tested it in a 24-way 2-socket Intel box, 24 Gb mem. I used Mel Gorman's pft test, that he used to demonstrate this problem back in the Kernel Summit. There are three kernels:
nomemcg : memcg compile disabled.
base : memcg enabled, patch not applied.
bypassed : memcg enabled, with patch applied.base bypassed
User 109.12 105.64
System 1646.84 1597.98
Elapsed 229.56 215.76
Quoted 7 lines
Quoted 4 lines
nomemcg bypassed
User 104.35 105.64
System 1578.19 1597.98
Elapsed 212.33 215.76Do you have profiles for where we spend the time?
I don't *have* in the sense that I never saved them, but it is easy to grab. I've just run Mel's pft test with perf top -a in parallel, and that was mostly the charge and uncharge functions being run.
Quoted 10 lines
Quoted 10 lines
#ifdef
CONFIG_MEMCG(quoted diff) +9
-0+extern struct static_key memcg_in_use_key; + +static inline bool mem_cgroup_subsys_disabled(void) +{ + return !!mem_cgroup_subsys.disabled; +} + +static inline bool mem_cgroup_disabled(void) +{
Quoted 9 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ /* + * Will always be false if subsys is disabled, because we have no one + * to bump it up. So the test suffices and we don't have to test the + * subsystem as well + */but
static_key_falseadds an atomic read here which is more costly so I am not sure you are optimizing much.
No it doesn't. You're missing the point of static branches: The code is *patched out* until it is not used. So it adds a predictable deterministic jump instruction to the false statement, and that's it (hence their previous name 'jump label').
Quoted 16 lines
Review
Quoted 11 lines
(quoted diff) +11
-0+ +extern int __mem_cgroup_cache_charge(struct page *page, struct mm_struct *mm, + gfp_t gfp_mask); +static inline int +mem_cgroup_cache_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask) +{ + if (mem_cgroup_disabled()) + return 0; + + return __mem_cgroup_cache_charge(page, mm, gfp_mask); +}Are there any reasons to not get down to
__mem_cgroup_try_charge? We will not be perfect, all right, because some wrappers already do some work but we should at least cover most of them.
Quoted 2 lines
I am also thinking whether this stab at charging path is not just an overkill. Wouldn't it suffice to do something like:
I don't know. I could test. I just see no reason for that. Being able to patch out code in the caller level means we'll not incur even a function call. That's a generally accepted good thing to do in hot paths. Specially given the fact that the memcg overhead seems not to be concentrated in one single place, but as Christoph Lameter defined, "death by a thousand cuts", I'd much rather not even pay the function calls if I can avoid. If I introducing great complexity for that, fine, I could trade off. But honestly, the patch gets bigger but that's it.
Quoted 8 lines
Quoted 2 lines
struct lruvec *mem_cgroup_zone_lruvec(struct zone *, struct
mem_cgroup*); struct lruvec *mem_cgroup_page_lruvec(struct page *, struct zone *);[...]
Quoted 4 lines
mm/memcontrol.c +0
-0[...]
Quoted 6 lines
mm/memcontrol.c struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) · line 1345
Quoted 2 lines
mm/memcontrol.c +0
-0@@ -1335,6 +1345,20 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) memcg = pc->mem_cgroup;I would expect that you want to prevent lookup as well if there are no other groups.
well, that function in specific seems to be mostly called during reclaim, where I wasn't terribly concerned about optimizations, unlike the steady state functions.
Quoted 9 lines
Quoted 9 lines
mm/memcontrol.c +8
-0/* +* Because we lazily enable memcg only after first child group is +* created, we can have memcg == 0. Because page cgroup is created with +* GFP_ZERO, and after charging, all page cgroups will have a non-zero +* cgroup attached (even if root), we can be sure that this is a +* used-but-not-accounted page. (due to lazyness). We could get around +* that by scanning all pages on cgroup init is too expensive. We can +* ultimately pay, but prefer to just to defer the update until we get +* here. We could take the opportunity to set PageCgroupUsed, but it
Quoted 9 lines
mm/memcontrol.c struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) · line 1358
Quoted 4 lines
mm/memcontrol.c +4
-0+* won't be that important for the root cgroup. +*/ +if (!memcg &&PageLRU (page)) + pc->mem_cgroup = memcg = root_mem_cgroup;Why not return
page_cgroup_zoneinfo(root_mem_cgroup, page); This would require messing up with__mem_cgroup_uncharge_commonbut that doesn't sound incredibly crazy (to the local standard of course ;)).
Could you clarify?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 3/5] memcg: make it suck faster
Quoted 34 lines · Glauber Costa
Sorry all for taking a lot of time to reply to this. I've been really busy.
Quoted 31 lines · Michal Hocko
Quoted 29 lines · Glauber Costa
It is an accepted fact that memcg sucks. But can it suck faster? Or in a more fair statement, can it at least stop draining everyone's performance when it is not in use?
This experimental and slightly crude patch demonstrates that we can do that by using static branches to patch it out until the first memcg comes to life. There are edges to be trimmed, and I appreciate comments for direction. In particular, the events in the root are not fired, but I believe this can be done without further problems by calling a specialized event check from
mem_cgroup_newpage_charge().My goal was to have enough numbers to demonstrate the performance gain that can come from it. I tested it in a 24-way 2-socket Intel box, 24 Gb mem. I used Mel Gorman's pft test, that he used to demonstrate this problem back in the Kernel Summit. There are three kernels:
nomemcg : memcg compile disabled.
base : memcg enabled, patch not applied.
bypassed : memcg enabled, with patch applied.base bypassed
User 109.12 105.64
System 1646.84 1597.98
Elapsed 229.56 215.76nomemcg bypassed
User 104.35 105.64
System 1578.19 1597.98
Elapsed 212.33 215.76
Quoted 6 lines
Quoted 2 lines
Do you have profiles for where we spend the time?
I don't *have* in the sense that I never saved them, but it is easy to grab. I've just run Mel's pft test with perf top -a in parallel, and that was mostly the charge and uncharge functions being run.
Would be nice to have this information to know which parts need optimization with a higher priority. I do not think we will make it ~0% cost in a single run.
Quoted 20 lines
Quoted 10 lines
Quoted 10 lines
#ifdef
CONFIG_MEMCG(quoted diff) +9
-0+extern struct static_key memcg_in_use_key; + +static inline bool mem_cgroup_subsys_disabled(void) +{ + return !!mem_cgroup_subsys.disabled; +} + +static inline bool mem_cgroup_disabled(void) +{Quoted 9 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+ /* + * Will always be false if subsys is disabled, because we have no one + * to bump it up. So the test suffices and we don't have to test the + * subsystem as well + */but
static_key_falseadds an atomic read here which is more costly so I am not sure you are optimizing much.
Quoted 2 lines
No it doesn't. You're missing the point of static branches: The code is *patched out* until it is not used.
OK, I should have been more specific. It adds an atomic if static branches are disabled.
Quoted 19 lines
So it adds a predictable deterministic jump instruction to the false statement, and that's it (hence their previous name 'jump label').
Quoted 16 lines
Review
Quoted 11 lines
(quoted diff) +11
-0+ +extern int __mem_cgroup_cache_charge(struct page *page, struct mm_struct *mm, + gfp_t gfp_mask); +static inline int +mem_cgroup_cache_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask) +{ + if (mem_cgroup_disabled()) + return 0; + + return __mem_cgroup_cache_charge(page, mm, gfp_mask); +}Are there any reasons to not get down to
__mem_cgroup_try_charge? We will not be perfect, all right, because some wrappers already do some work but we should at least cover most of them.
Quoted 6 lines
Quoted 2 lines
I am also thinking whether this stab at charging path is not just an overkill. Wouldn't it suffice to do something like:
I don't know. I could test. I just see no reason for that. Being able to patch out code in the caller level means we'll not incur even a function call. That's a generally accepted good thing to do in hot paths.
Agreed. I just think that the charging path is rather complicated and changing it incrementally is a lower risk. Maybe we just find out that the biggest overhead can be reduced by a simpler approach.
Quoted 5 lines
Specially given the fact that the memcg overhead seems not to be concentrated in one single place, but as Christoph Lameter defined, "death by a thousand cuts", I'd much rather not even pay the function calls if I can avoid. If I introducing great complexity for that, fine, I could trade off. But honestly, the patch gets bigger but that's it.
And more code and all the charging paths are already quite complicated. I do not want to add more, if possible.
Quoted 14 lines
Quoted 8 lines
Quoted 2 lines
struct lruvec *mem_cgroup_zone_lruvec(struct zone *, struct
mem_cgroup*); struct lruvec *mem_cgroup_page_lruvec(struct page *, struct zone *);[...]
Quoted 4 lines
mm/memcontrol.c +0
-0[...]
Quoted 6 lines
mm/memcontrol.c struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) · line 1345
Quoted 2 lines
mm/memcontrol.c +0
-0@@ -1335,6 +1345,20 @@ struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) memcg = pc->mem_cgroup;I would expect that you want to prevent lookup as well if there are no other groups.
Quoted 3 lines
well, that function in specific seems to be mostly called during reclaim, where I wasn't terribly concerned about optimizations, unlike the steady state functions.
Not only from reclaim. Also when a page is uncharged. Anyway we can get the optimization almost for free here ;)
Quoted 19 lines
Quoted 9 lines
Quoted 9 lines
mm/memcontrol.c +8
-0/* +* Because we lazily enable memcg only after first child group is +* created, we can have memcg == 0. Because page cgroup is created with +* GFP_ZERO, and after charging, all page cgroups will have a non-zero +* cgroup attached (even if root), we can be sure that this is a +* used-but-not-accounted page. (due to lazyness). We could get around +* that by scanning all pages on cgroup init is too expensive. We can +* ultimately pay, but prefer to just to defer the update until we get +* here. We could take the opportunity to set PageCgroupUsed, but it Quoted 9 lines
mm/memcontrol.c struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct zone *zone) · line 1358
Quoted 4 lines
mm/memcontrol.c +4
-0+* won't be that important for the root cgroup. +*/ +if (!memcg &&PageLRU (page)) + pc->mem_cgroup = memcg = root_mem_cgroup;Why not return
page_cgroup_zoneinfo(root_mem_cgroup, page); This would require messing up with__mem_cgroup_uncharge_commonbut that doesn't sound incredibly crazy (to the local standard of course ;)).
Quoted 1 line
Could you clarify?
You can save some cycles by returning lruvec from here directly and do not get through:
if (!PageLRU(page) && !PageCgroupUsed(pc) && memcg != root_mem_cgroup) pc->mem_cgroup = memcg = root_mem_cgroup; mz = page_cgroup_zoneinfo(memcg, page); lruvec = &mz->lruvec;
again. Just a nano optimization in that path
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v2 4/5 memcg: do not call page_cgroup_init at system_boot
In reply to Glauber CostaIf we are not using memcg, there is no reason why we should allocate this structure, that will be a memory waste at best. We can do better at least in the sparsemem case, and allocate it when the first cgroup is requested. It should now not panic on failure, and we have to handle this right.
flatmem case is a bit more complicated, so that one is left out for the moment.
--- include/linux/page_cgroup.h | 28 +++++---- init/main.c | 2 - mm/memcontrol.c | 3 +- mm/page_cgroup.c | 150 ++++++++++++++++++++++++-------------------- 4 files changed, 99 insertions(+), 84 deletions(-)
include/linux/page_cgroup.h +16 -12
@@ -14,6 +14,7 @@ enum {#ifdef CONFIG_MEMCG #include <linux/bit_spinlock.h> +#include <linux/mmzone.h> /* * Page Cgroup can be considered as an extended mem_map.
@@ -27,19 +28,17 @@ struct page_cgroup {struct mem_cgroup *mem_cgroup; }; -void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat); - -#ifdef CONFIG_SPARSEMEM -static inline void __init page_cgroup_init_flatmem (void ) +static inline size_t page_cgroup_table_size (int nid) { -} -extern void __init page_cgroup_init (void ); +#ifdef CONFIG_SPARSEMEM +return sizeof (struct page_cgroup ) *PAGES_PER_SECTION ;#else -void __init page_cgroup_init_flatmem (void ); -static inline void __init page_cgroup_init (void ) -{ -} +return sizeof (struct page_cgroup ) *NODE_DATA (nid)->node_spanned_pages;#endif +} +void pgdat_page_cgroup_init (struct pglist_data *pgdat); + +extern int page_cgroup_init (void );struct page_cgroup *lookup_page_cgroup (struct page *page);struct page *lookup_cgroup_page (struct page_cgroup *pc);
@@ -85,7 +84,7 @@ static inline void unlock_page_cgroup(struct page_cgroup *pc)#else /* CONFIG_MEMCG */ struct page_cgroup ; -static inline void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat) +static inline void pgdat_page_cgroup_init (struct pglist_data *pgdat) { }
@@ -94,7 +93,12 @@ static inline struct page_cgroup *lookup_page_cgroup(struct page *page)return NULL ; } -static inline void page_cgroup_init (void ) +static inline int page_cgroup_init (void ) +{ +return 0 ; +} + +static inline void page_cgroup_destroy (void ) { }
init/main.c +0 -2
@@ -457,7 +457,6 @@ static void __init mm_init(void)* page_cgroup requires contiguous pages, * bigger than MAX_ORDER unless SPARSEMEM. */ -page_cgroup_init_flatmem ();mem_init ();kmem_cache_init ();percpu_init_late ();
@@ -592,7 +591,6 @@ asmlinkage void __init start_kernel(void) initrd_start = 0; }#endif -page_cgroup_init ();debug_objects_mem_init ();kmemleak_init ();setup_per_cpu_pageset ();
mm/memcontrol.c +2 -1
@@ -6377,7 +6377,8 @@ mem_cgroup_css_alloc(struct cgroup *cont)res_counter_init (&memcg->res,NULL );res_counter_init (&memcg->memsw,NULL );res_counter_init (&memcg->kmem,NULL ); - } + }else if (page_cgroup_init ()) + goto free_out; memcg->last_scanned_node =MAX_NUMNODES ;INIT_LIST_HEAD (&memcg->oom_notify);
mm/page_cgroup.c +81 -69
230 lines not on this page · raw message
Signature · 1 line
1.8.1.2
Re: [PATCH v2 4/5] memcg: do not call page_cgroup_init at system_boot
Quoted 15 lines · Glauber Costa
If we are not using memcg, there is no reason why we should allocate this structure, that will be a memory waste at best. We can do better at least in the sparsemem case, and allocate it when the first cgroup is requested. It should now not panic on failure, and we have to handle this right.
flatmem case is a bit more complicated, so that one is left out for the moment.
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>
Quoted 6 lines
--- include/linux/page_cgroup.h | 28 +++++---- init/main.c | 2 - mm/memcontrol.c | 3 +- mm/page_cgroup.c | 150 ++++++++++++++++++++++++-------------------- 4 files changed, 99 insertions(+), 84 deletions(-)
This patch seems a complicated mixture of clean-up and what-you-really-want.
Quoted 109 lines
include/linux/page_cgroup.h +16
-12@@ -14,6 +14,7 @@ enum {#ifdef CONFIG_MEMCG #include <linux/bit_spinlock.h> +#include <linux/mmzone.h> /* * Page Cgroup can be considered as an extended mem_map. @@ -27,19 +28,17 @@ struct page_cgroup {struct mem_cgroup *mem_cgroup; }; -void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat); - -#ifdef CONFIG_SPARSEMEM -static inline void __init page_cgroup_init_flatmem (void ) +static inline size_t page_cgroup_table_size (int nid) { -} -extern void __init page_cgroup_init (void ); +#ifdef CONFIG_SPARSEMEM +return sizeof (struct page_cgroup ) *PAGES_PER_SECTION ;#else -void __init page_cgroup_init_flatmem (void ); -static inline void __init page_cgroup_init (void ) -{ -} +return sizeof (struct page_cgroup ) *NODE_DATA (nid)->node_spanned_pages;#endif +} +void pgdat_page_cgroup_init (struct pglist_data *pgdat); + +extern int page_cgroup_init (void );struct page_cgroup *lookup_page_cgroup (struct page *page);struct page *lookup_cgroup_page (struct page_cgroup *pc);@@ -85,7 +84,7 @@ static inline void unlock_page_cgroup(struct page_cgroup *pc)#else /* CONFIG_MEMCG */ struct page_cgroup ; -static inline void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat) +static inline void pgdat_page_cgroup_init (struct pglist_data *pgdat) { }@@ -94,7 +93,12 @@ static inline struct page_cgroup *lookup_page_cgroup(struct page *page)return NULL ; } -static inline void page_cgroup_init (void ) +static inline int page_cgroup_init (void ) +{ +return 0 ; +} + +static inline void page_cgroup_destroy (void ) { }init/main.c +0
-2@@ -457,7 +457,6 @@ static void __init mm_init(void)* page_cgroup requires contiguous pages, * bigger than MAX_ORDER unless SPARSEMEM. */ -page_cgroup_init_flatmem ();mem_init ();kmem_cache_init ();percpu_init_late ();@@ -592,7 +591,6 @@ asmlinkage void __init start_kernel(void) initrd_start = 0; }#endif -page_cgroup_init ();debug_objects_mem_init ();kmemleak_init ();setup_per_cpu_pageset ();mm/memcontrol.c +2
-1@@ -6377,7 +6377,8 @@ mem_cgroup_css_alloc(struct cgroup *cont)res_counter_init (&memcg->res,NULL );res_counter_init (&memcg->memsw,NULL );res_counter_init (&memcg->kmem,NULL ); - } + }else if (page_cgroup_init ()) + goto free_out; memcg->last_scanned_node =MAX_NUMNODES ;INIT_LIST_HEAD (&memcg->oom_notify);mm/page_cgroup.c +1
-1@@ -12,11 +12,50 @@#include <linux/kmemleak.h> static unsigned long total_usage; +static unsigned long page_cgroup_initialized; -#if !defined(CONFIG_SPARSEMEM )
mm/page_cgroup.c line 26
Quoted 10 lines
mm/page_cgroup.c +10
-0+static void *alloc_page_cgroup (size_t size,int nid) +{ +gfp_t flags =GFP_KERNEL | __GFP_ZERO | __GFP_NOWARN; +void *addr =NULL ; + + addr =alloc_pages_exact_nid (nid, size, flags); +if (addr) { +kmemleak_alloc (addr, size,1 , flags); +return addr; + }
As far as I remember, this function was written for SPARSEMEM.
How big this "size" will be with FLATMEM/DISCONTIGMEM ? if 16GB, 16 * 1024 * 1024 * 1024 / 4096 * 16 = 64MB.
What happens if order > MAX_ORDER is passed to alloc_pages()...no warning ?
How about using vmalloc always if not SPARSEMEM ?
Quoted 10 lines
mm/page_cgroup.c +7
-0+ +if (node_state (nid,N_HIGH_MEMORY )) + addr =vzalloc_node (size, nid); +else + addr =vzalloc (size); +return addr; +}
mm/page_cgroup.c line 43
Quoted 10 lines
mm/page_cgroup.c +8
-1-void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat) +static void free_page_cgroup (void *addr) +{ +if (is_vmalloc_addr (addr)) { +vfree (addr); + }else { +struct page *page =virt_to_page (addr); +int nid =page_to_nid (page); +BUG_ON (PageReserved (page));
This BUG_ON() can be removed.
mm/page_cgroup.c line 54
Quoted 11 lines
mm/page_cgroup.c +11
-0+free_pages_exact (addr,page_cgroup_table_size (nid)); + } +} + +static void page_cgroup_msg (void ) +{ +printk (KERN_INFO "allocated %ld bytes of page_cgroup\n" , total_usage); +printk (KERN_INFO "please try 'cgroup_disable=memory' option if you " +"don't want memory cgroups.\nAlternatively, consider " +"deferring your memory cgroups creation.\n" ); +}
I think this warning can be removed because it's not boot option problem after this patch. I guess the boot option can be obsolete....
Quoted 194 lines
mm/page_cgroup.c +4
-0+ +#if !defined(CONFIG_SPARSEMEM ) + +void pgdat_page_cgroup_init (struct pglist_data *pgdat) {pgdat->node_page_cgroup = NULL; }mm/page_cgroup.c +40
-67@@ -42,20 +81,16 @@ struct page_cgroup *lookup_page_cgroup(struct page *page)return base + offset; } -static int __init alloc_node_page_cgroup (int nid) +static int alloc_node_page_cgroup (int nid) {struct page_cgroup *base;unsigned long table_size; -unsigned long nr_pages; - nr_pages =NODE_DATA (nid)->node_spanned_pages; -if (!nr_pages) + table_size =page_cgroup_table_size (nid); +if (!table_size)return 0 ; - table_size =sizeof (struct page_cgroup ) * nr_pages; - - base =__alloc_bootmem_node_nopanic (NODE_DATA (nid), - table_size,PAGE_SIZE ,__pa (MAX_DMA_ADDRESS )); + base =alloc_page_cgroup (table_size, nid);if (!base)return -ENOMEM ;NODE_DATA (nid)->node_page_cgroup = base;@@ -63,27 +98,29 @@ static int __init alloc_node_page_cgroup(int nid)return 0 ; } -void __init page_cgroup_init_flatmem (void ) +int page_cgroup_init (void ) { +int nid, fail, tmpnid; -int nid, fail; - -if (mem_cgroup_subsys_disabled ()) -return ; +/* only initialize it once */ +if (test_and_set_bit (0 , &page_cgroup_initialized)) +return 0 ;for_each_online_node (nid) { fail =alloc_node_page_cgroup (nid);if (fail) goto fail; } -printk (KERN_INFO "allocated %ld bytes of page_cgroup\n" , total_usage); -printk (KERN_INFO "please try 'cgroup_disable=memory' option if you" -" don't want memory cgroups\n" ); -return ; +page_cgroup_msg (); +return 0 ; fail: -printk (KERN_CRIT "allocation of page_cgroup failed.\n" ); -printk (KERN_CRIT "please try 'cgroup_disable=memory' boot option\n" ); -panic ("Out of memory" ); +for_each_online_node (tmpnid) { +if (tmpnid >= nid) +break ; +free_page_cgroup (NODE_DATA (tmpnid)->node_page_cgroup); + } + +return -ENOMEM ; }#else /* CONFIG_FLAT_NODE_MEM_MAP */ @@ -105,26 +142,7 @@ struct page_cgroup *lookup_page_cgroup(struct page *page)return section->page_cgroup + pfn; } -static void *__meminit alloc_page_cgroup (size_t size,int nid) -{ -gfp_t flags =GFP_KERNEL | __GFP_ZERO | __GFP_NOWARN; -void *addr =NULL ; - - addr =alloc_pages_exact_nid (nid, size, flags); -if (addr) { -kmemleak_alloc (addr, size,1 , flags); -return addr; - } - -if (node_state (nid,N_HIGH_MEMORY )) - addr =vzalloc_node (size, nid); -else - addr =vzalloc (size); - -return addr; -} - -static int __meminit init_section_page_cgroup (unsigned long pfn,int nid) +static int init_section_page_cgroup (unsigned long pfn,int nid) {struct mem_section *section;struct page_cgroup *base;@@ -135,7 +153,7 @@ static int __meminit init_section_page_cgroup(unsigned long pfn, int nid)if (section->page_cgroup)return 0 ; - table_size =sizeof (struct page_cgroup ) *PAGES_PER_SECTION ; + table_size =page_cgroup_table_size (nid); base =alloc_page_cgroup (table_size, nid);/* @@ -159,20 +177,6 @@ static int __meminit init_section_page_cgroup(unsigned long pfn, int nid) total_usage += table_size;return 0 ; } -#ifdef CONFIG_MEMORY_HOTPLUG -static void free_page_cgroup (void *addr) -{ -if (is_vmalloc_addr (addr)) { -vfree (addr); - }else { -struct page *page =virt_to_page (addr); -size_t table_size = -sizeof (struct page_cgroup ) *PAGES_PER_SECTION ; - -BUG_ON (PageReserved (page)); -free_pages_exact (addr, table_size); - } -}void __free_page_cgroup (unsigned long pfn) {@@ -187,6 +191,7 @@ void __free_page_cgroup(unsigned long pfn) ms->page_cgroup =NULL ; } +#ifdef CONFIG_MEMORY_HOTPLUG int __meminit online_page_cgroup (unsigned long start_pfn,unsigned long nr_pages,int nid)@@ -266,16 +271,16 @@ static int __meminit page_cgroup_callback(struct notifier_block *self,#endif -void __init page_cgroup_init (void ) +int page_cgroup_init (void ) {unsigned long pfn; -int nid; +unsigned long start_pfn, end_pfn; +int nid, tmpnid; -if (mem_cgroup_subsys_disabled ()) -return ; +if (test_and_set_bit (0 , &page_cgroup_initialized)) +return 0 ;for_each_node_state (nid,N_MEMORY ) { -unsigned long start_pfn, end_pfn; start_pfn =node_start_pfn (nid); end_pfn =node_end_pfn (nid);@@ -303,16 +308,23 @@ void __init page_cgroup_init(void) } }hotplug_memory_notifier (page_cgroup_callback,0 ); -printk (KERN_INFO "allocated %ld bytes of page_cgroup\n" , total_usage); -printk (KERN_INFO "please try 'cgroup_disable=memory' option if you " -"don't want memory cgroups\n" ); -return ; +page_cgroup_msg (); +return 0 ; oom: -printk (KERN_CRIT "try 'cgroup_disable=memory' boot option\n" ); -panic ("Out of memory" ); +for_each_node_state (tmpnid,N_MEMORY ) { +if (tmpnid >= nid) +break ; + + start_pfn =node_start_pfn (tmpnid); + end_pfn =node_end_pfn (tmpnid); + +for (pfn = start_pfn; pfn < end_pfn; pfn +=PAGES_PER_SECTION ) +__free_page_cgroup (pfn); + } +return -ENOMEM ; } -void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat) +void pgdat_page_cgroup_init (struct pglist_data *pgdat) {return ; }
Re: [PATCH v2 4/5] memcg: do not call page_cgroup_init at system_boot
Quoted 23 lines · Kamezawa Hiroyuki
Quoted 15 lines · Glauber Costa
If we are not using memcg, there is no reason why we should allocate this structure, that will be a memory waste at best. We can do better at least in the sparsemem case, and allocate it when the first cgroup is requested. It should now not panic on failure, and we have to handle this right.
flatmem case is a bit more complicated, so that one is left out for the moment.
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>Quoted 6 lines
--- include/linux/page_cgroup.h | 28 +++++---- init/main.c | 2 - mm/memcontrol.c | 3 +- mm/page_cgroup.c | 150 ++++++++++++++++++++++++-------------------- 4 files changed, 99 insertions(+), 84 deletions(-)
Quoted 2 lines
This patch seems a complicated mixture of clean-up and what-you-really-want.
I swear it is all what-I-really-want, any cleanups are non-intentional!
Quoted 14 lines
mm/page_cgroup.c line 26
Quoted 11 lines
mm/page_cgroup.c +10
-1-#if !defined(CONFIG_SPARSEMEM ) +static void *alloc_page_cgroup (size_t size,int nid) +{ +gfp_t flags =GFP_KERNEL | __GFP_ZERO | __GFP_NOWARN; +void *addr =NULL ; + + addr =alloc_pages_exact_nid (nid, size, flags); +if (addr) { +kmemleak_alloc (addr, size,1 , flags); +return addr; + }As far as I remember, this function was written for SPARSEMEM.
Quoted 6 lines
How big this "size" will be with FLATMEM/DISCONTIGMEM ? if 16GB, 16 * 1024 * 1024 * 1024 / 4096 * 16 = 64MB.
What happens if order >
MAX_ORDERis passed to alloc_pages()...no warning ?How about using vmalloc always if not SPARSEMEM ?
I don't oppose.
Quoted 13 lines
mm/page_cgroup.c line 43
Quoted 10 lines
mm/page_cgroup.c +8
-1-void __meminit pgdat_page_cgroup_init (struct pglist_data *pgdat) +static void free_page_cgroup (void *addr) +{ +if (is_vmalloc_addr (addr)) { +vfree (addr); + }else { +struct page *page =virt_to_page (addr); +int nid =page_to_nid (page); +BUG_ON (PageReserved (page));This
BUG_ON()can be removed.
You are right, although it is still a bug =)
Quoted 6 lines
Quoted 6 lines
mm/page_cgroup.c +6
-0+free_pages_exact (addr,page_cgroup_table_size (nid)); + } +} + +static void page_cgroup_msg (void ) +{
Quoted 9 lines
mm/page_cgroup.c line 54
Quoted 5 lines
mm/page_cgroup.c +5
-0+printk (KERN_INFO "allocated %ld bytes of page_cgroup\n" , total_usage); +printk (KERN_INFO "please try 'cgroup_disable=memory' option if you " +"don't want memory cgroups.\nAlternatively, consider " +"deferring your memory cgroups creation.\n" ); +}I think this warning can be removed because it's not boot option problem after this patch. I guess the boot option can be obsolete....
I think it is extremely useful, at least during the next couple of releases. A lot of distributions will create memcgs for no apparent reasons way before they are used (if used at all), as a placeholder only.
This can at least tell them that there is a way to stop paying a memory penalty (together with the actual memory footprint)
Re: [PATCH v2 4/5] memcg: do not call page_cgroup_init at system_boot
In reply to Glauber CostaQuoted 5 lines · Glauber Costa
If we are not using memcg, there is no reason why we should allocate this structure, that will be a memory waste at best. We can do better at least in the sparsemem case, and allocate it when the first cgroup is requested. It should now not panic on failure, and we have to handle this right.
lookup_page_cgroup needs a special handling as well. Callers are not prepared to get NULL and the current code would even explode with !CONFIG_DEBUG_VM.
Anyway, agreed with what Kame said. This is really hard to read. Would it be possible to split it up somehow - sorry for not being more helpful here...
Quoted 16 lines
flatmem case is a bit more complicated, so that one is left out for the moment.
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <[email protected]>--- include/linux/page_cgroup.h | 28 +++++---- init/main.c | 2 - mm/memcontrol.c | 3 +- mm/page_cgroup.c | 150 ++++++++++++++++++++++++-------------------- 4 files changed, 99 insertions(+), 84 deletions(-)
[...]
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v2 5/5 memcg: do not walk all the way to the root for memcg
In reply to Glauber CostaSince the root is special anyway, and we always get its figures from global counters anyway, there is no make all cgroups its descendants, wrt res_counters. The sad effect of doing that is that we need to lock the root for all allocations, since it is a common ancestor of everybody.
Not having the root as a common ancestor should lead to better scalability for not-uncommon case of tasks in the cgroup being node-bound to different nodes in NUMA systems.
--- mm/memcontrol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/memcontrol.c +1 -1
@@ -6464,7 +6464,7 @@ mem_cgroup_css_online(struct cgroup *cont) memcg->oom_kill_disable = parent->oom_kill_disable; memcg->swappiness =mem_cgroup_swappiness (parent); -if (parent->use_hierarchy) { +if (parent && !mem_cgroup_is_root (parent) && parent->use_hierarchy) {res_counter_init (&memcg->res, &parent->res);res_counter_init (&memcg->memsw, &parent->memsw);res_counter_init (&memcg->kmem, &parent->kmem);
Signature · 1 line
1.8.1.2
Re: [PATCH v2 5/5] memcg: do not walk all the way to the root for memcg
Quoted 10 lines · Glauber Costa
Since the root is special anyway, and we always get its figures from global counters anyway, there is no make all cgroups its descendants, wrt
res_counters. The sad effect of doing that is that we need to lock the root for all allocations, since it is a common ancestor of everybody.Not having the root as a common ancestor should lead to better scalability for not-uncommon case of tasks in the cgroup being node-bound to different nodes in NUMA systems.
Quoted 6 lines
Signed-off-by Glauber Costa <[email protected]>CC Michal Hocko <[email protected]>CC Kamezawa Hiroyuki <[email protected]>CC Johannes Weiner <[email protected]>CC Mel Gorman <[email protected]>CC Andrew Morton <akpm-de/[email protected]>
Quoted 18 lines
--- mm/memcontrol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/memcontrol.c +1
-1@@ -6464,7 +6464,7 @@ mem_cgroup_css_online(struct cgroup *cont) memcg->oom_kill_disable = parent->oom_kill_disable; memcg->swappiness =mem_cgroup_swappiness (parent); -if (parent->use_hierarchy) { +if (parent && !mem_cgroup_is_root (parent) && parent->use_hierarchy) {res_counter_init (&memcg->res, &parent->res);res_counter_init (&memcg->memsw, &parent->memsw);res_counter_init (&memcg->kmem, &parent->kmem);