Skip to contentSkip to conversation

mm, memcg: don't protect pages if memcg is not populated

PATCH In review Yafang Shao 2 participants 4 messages Nov 23, 2019 – Nov 25, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm, memcg: don't protect pages if memcg is not populated

memory.{min, low} keeps protecting pages in a memcg even if there're no process running in this memcg. That makes nonsense, because if there're no processes running in this memcg there may be something wrong happens and these protected pages are not useful now.

---
 mm/memcontrol.c | 3 +++
 1 file changed, 3 insertions(+)
mm/memcontrol.c +3 -0
index 1c4c08b..c96c282 100644
@@ -6376,6 +6376,9 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root,
 	if (memcg == root)
 		return MEMCG_PROT_NONE;
 
+	if (!cgroup_is_populated(memcg->css.cgroup))
+		return MEMCG_PROT_NONE;
+
 	usage = page_counter_read(&memcg->memory);
 	if (!usage)
 		return MEMCG_PROT_NONE;
Signature · 1 line
1.8.3.1
maintainer
#1 lore raw
Quoted 4 lines · Yafang Shao

memory.{min, low} keeps protecting pages in a memcg even if there're no process running in this memcg. That makes nonsense, because if there're no processes running in this memcg there may be something wrong happens and these protected pages are not useful now.

Well, the min resp. low limits protect charges rather than tasks. And thre are cases when there is no direct relation between the charged memory and a specific task - e.g. long living kernel objects or page cache. So there is nothing really unusual to have charges in an absence of any tasks. I do not really see any reason why the protection shouldn't apply to them. Such a be behavior would be quite inconsistent btw. Just imagine a case when there is _a_ task but that one doesn't really have any charges in the memcg. Why should we respect reclaim protection in that case?

Btw. offlined memcgs already have no min/low watermarks protect.

Do you have any specific usecase that suffers from this behavior?

Quoted 21 lines
---
 mm/memcontrol.c | 3 +++
 1 file changed, 3 insertions(+)
mm/memcontrol.c +3 -0
index 1c4c08b..c96c282 100644
@@ -6376,6 +6376,9 @@ enum mem_cgroup_protection mem_cgroup_protected(struct mem_cgroup *root,
 	if (memcg == root)
 		return MEMCG_PROT_NONE;
 
+	if (!cgroup_is_populated(memcg->css.cgroup))
+		return MEMCG_PROT_NONE;
+
 	usage = page_counter_read(&memcg->memory);
 	if (!usage)
 		return MEMCG_PROT_NONE;
Signature · 1 line
1.8.3.1
Signature · 2 lines
Michal Hocko
SUSE Labs
#2 lore raw
Quoted 7 lines · Michal Hocko
Quoted 4 lines · Yafang Shao

memory.{min, low} keeps protecting pages in a memcg even if there're no process running in this memcg. That makes nonsense, because if there're no processes running in this memcg there may be something wrong happens and these protected pages are not useful now.

Quoted 10 lines

Well, the min resp. low limits protect charges rather than tasks. And thre are cases when there is no direct relation between the charged memory and a specific task - e.g. long living kernel objects or page cache. So there is nothing really unusual to have charges in an absence of any tasks. I do not really see any reason why the protection shouldn't apply to them. Such a be behavior would be quite inconsistent btw. Just imagine a case when there is _a_ task but that one doesn't really have any charges in the memcg. Why should we respect reclaim protection in that case?

Per my understanding, the memory.{min, low} is used to protect the workload. I don't know in which cases we protect the 'content' only.

If one memcg is used to protect the content only, e.g. some file caches, then these file caches will be shared by processes in different memcgs, that may cause some unexpected issues, for example, how to handle the writeback throttle ?

I don't know how to use memory.{min, low} to protect long living kernel objects neither without any processes.

I would appreciate if you could explain my questions in detail.

Quoted 4 lines

Btw. offlined memcgs already have no min/low watermarks protect.

Do you have any specific usecase that suffers from this behavior?

For example, if oom happens, the processes in this memcg may be killed by OOM killer, and if memory.oom.group is set, all processes in it will be killed. So these protected pages are useless. In that cases, we should clear memory.{min, low} protection.

Thanks
Yafang

maintainer
#3 lore raw
Quoted 19 lines · Yafang Shao
Quoted 7 lines · Michal Hocko
Quoted 4 lines · Yafang Shao

memory.{min, low} keeps protecting pages in a memcg even if there're no process running in this memcg. That makes nonsense, because if there're no processes running in this memcg there may be something wrong happens and these protected pages are not useful now.

Quoted 10 lines

Well, the min resp. low limits protect charges rather than tasks. And thre are cases when there is no direct relation between the charged memory and a specific task - e.g. long living kernel objects or page cache. So there is nothing really unusual to have charges in an absence of any tasks. I do not really see any reason why the protection shouldn't apply to them. Such a be behavior would be quite inconsistent btw. Just imagine a case when there is _a_ task but that one doesn't really have any charges in the memcg. Why should we respect reclaim protection in that case?

Quoted 2 lines

Per my understanding, the memory.{min, low} is used to protect the workload. I don't know in which cases we protect the 'content' only.

I didn't say anything about the content. You are right that those limits are protecting a working set. I do not think it would be hard to imagine a workload which has a page cache to be the working set and want to protect it while tasks are only very short lived and so "no tasks" state wouldn't be all that unusual.

Quoted 4 lines

If one memcg is used to protect the content only, e.g. some file caches, then these file caches will be shared by processes in different memcgs, that may cause some unexpected issues, for example, how to handle the writeback throttle ?

Yes, sharing might lead to surprising behavior but I do not not understand how this is really related to the proposed patch.

Quoted 4 lines

I don't know how to use memory.{min, low} to protect long living kernel objects neither without any processes.

I would appreciate if you could explain my questions in detail.

See the example above.

Quoted 9 lines
Quoted 4 lines

Btw. offlined memcgs already have no min/low watermarks protect.

Do you have any specific usecase that suffers from this behavior?

For example, if oom happens, the processes in this memcg may be killed by OOM killer, and if memory.oom.group is set, all processes in it will be killed. So these protected pages are useless. In that cases, we should clear memory.{min, low} protection.

I do agree that the reclaim protection might lead to an unexpected OOM handling. Especially when charged objects are not directly bound to the process life time. But this patch is not going to help here in general. One has to be really careful when configuring the protection. It is a hard guarantee after all.

Signature · 2 lines
Michal Hocko
SUSE Labs