Skip to contentSkip to conversation

nommu: add anonymous page memcg accounting

PATCH Superseded Steven J. Magnani 3 participants 8 messages Oct 19, 2010 – Oct 21, 2010

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 nommu: add anonymous page memcg accounting

Add the necessary calls to track VM anonymous page usage (only).

V2 changes:
* Added update of memory cgroup documentation
* Clarify use of 'file' to distinguish anonymous mappings

---

Documentation/cgroups/memory.txt +25 -1
diff -uprN a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt
@@ -34,6 +34,7 @@ Current Status: linux-2.6.34-mmotm(devel
 
 Features:
  - accounting anonymous pages, file caches, swap caches usage and limiting them.
+   NOTE: On NOMMU systems, only anonymous pages are accounted.
  - private LRU and reclaim routine. (system's global LRU and private LRU
    work independently from each other)
  - optionally, memory+swap usage can be accounted and limited.
@@ -640,7 +641,30 @@ At reading, current status of OOM is sho
 	under_oom	 0 or 1 (if 1, the memory cgroup is under OOM, tasks may
 				 be stopped.)
 
-11. TODO
+11. NOMMU Support
+
+Systems without a Memory Management Unit do not support virtual memory,
+swapping, page faults, or migration, and are therefore limited to operating
+entirely within the system's RAM. On such systems, maintaining an ability to
+allocate sufficiently large blocks of contiguous memory is usually a challenge.
+This makes the overhead involved in memory cgroup support more of a concern,
+particularly when the memory page size is small.
+
+Typically, embedded systems are comparatively simple and deterministic, and are
+required to remain stable over long periods. Invocation of the OOM-killer, were
+it to occur in an uncontrolled manner, would likely destabilize such systems.
+
+Even a well-designed system may be presented with external stimuli that could
+lead to OOM conditions. One example is a system that is required to check a
+user-supplied removable FAT filesystem. As there is no way to bound the size
+or coherence of the user's filesystem, the memory required to run dosfsck on
+it may exceed the system's capacity. Running dosfsck in a memory cgroup
+can preserve system stability even in the face of excessive memory demands.
+
+At the present time, only anonymous pages are included in NOMMU memory cgroup
+accounting.
+
+12. TODO
 
 1. Add support for accounting huge pages (as a separate controller)
 2. Make per-cgroup scanner reclaim not-shared pages first
mm/nommu.c +31 -5
diff -uprN a/mm/nommu.c b/mm/nommu.c
@@ -524,8 +524,10 @@ static void delete_nommu_region(struct v
 /*
  * free a contiguous series of pages
  */
-static void free_page_series(unsigned long from, unsigned long to)
+static void free_page_series(unsigned long from, unsigned long to,
+			     const struct file *file)
 {
+	mem_cgroup_uncharge_start();
 	for (; from < to; from += PAGE_SIZE) {
 		struct page *page = virt_to_page(from);
 
@@ -534,8 +536,13 @@ static void free_page_series(unsigned lo
 		if (page_count(page) != 1)
 			kdebug("free page %p: refcount not one: %d",
 			       page, page_count(page));
+		/* Only anonymous pages are charged, currently */
+		if (!file)
+			mem_cgroup_uncharge_page(page);
+
 		put_page(page);
 	}
+	mem_cgroup_uncharge_end();
 }
 
 /*
@@ -563,7 +570,8 @@ static void __put_nommu_region(struct vm
 		 * from ramfs/tmpfs mustn't be released here */
 		if (region->vm_flags & VM_MAPPED_COPY) {
 			kdebug("free series");
-			free_page_series(region->vm_start, region->vm_top);
+			free_page_series(region->vm_start, region->vm_top,
+					 region->vm_file);
 		}
 		kmem_cache_free(vm_region_jar, region);
 	} else {
@@ -1117,9 +1125,27 @@ static int do_mmap_private(struct vm_are
 		set_page_refcounted(&pages[point]);
 
 	base = page_address(pages);
-	region->vm_flags = vma->vm_flags |= VM_MAPPED_COPY;
+
 	region->vm_start = (unsigned long) base;
 	region->vm_end   = region->vm_start + rlen;
+
+	/* Only anonymous pages are charged, currently */
+	if (!vma->vm_file) {
+		for (point = 0; point < total; point++) {
+			int charge_failed =
+				mem_cgroup_newpage_charge(&pages[point],
+							  current->mm,
+							  GFP_KERNEL);
+			if (charge_failed) {
+				free_page_series(region->vm_start,
+						 region->vm_end, NULL);
+				region->vm_start = region->vm_end = 0;
+				goto enomem;
+			}
+		}
+	}
+
+	region->vm_flags = vma->vm_flags |= VM_MAPPED_COPY;
 	region->vm_top   = region->vm_start + (total << PAGE_SHIFT);
 
 	vma->vm_start = region->vm_start;
@@ -1150,7 +1176,7 @@ static int do_mmap_private(struct vm_are
 	return 0;
 
 error_free:
-	free_page_series(region->vm_start, region->vm_end);
+	free_page_series(region->vm_start, region->vm_end, vma->vm_file);
 	region->vm_start = vma->vm_start = 0;
 	region->vm_end   = vma->vm_end = 0;
 	region->vm_top   = 0;
@@ -1555,7 +1581,7 @@ static int shrink_vma(struct mm_struct *
 	add_nommu_region(region);
 	up_write(&nommu_region_sem);
 
-	free_page_series(from, to);
+	free_page_series(from, to, vma->vm_file);
 	return 0;
 }
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#1 lore raw
Quoted 36 lines · Steven J. Magnani

Add the necessary calls to track VM anonymous page usage (only).

V2 changes:
* Added update of memory cgroup documentation
* Clarify use of 'file' to distinguish anonymous mappings

---
diff -uprN a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt

Documentation/cgroups/memory.txt +13 -1
@@ -34,6 +34,7 @@ Current Status: linux-2.6.34-mmotm(devel
 
 Features:
  - accounting anonymous pages, file caches, swap caches usage and limiting them.
+   NOTE: On NOMMU systems, only anonymous pages are accounted.
  - private LRU and reclaim routine. (system's global LRU and private LRU
    work independently from each other)
  - optionally, memory+swap usage can be accounted and limited.
@@ -640,7 +641,30 @@ At reading, current status of OOM is sho
 	under_oom	 0 or 1 (if 1, the memory cgroup is under OOM, tasks may
 				 be stopped.)
 
-11. TODO
+11. NOMMU Support
+
+Systems without a Memory Management Unit do not support virtual memory,
+swapping, page faults, or migration, and are therefore limited to operating
+entirely within the system's RAM. On such systems, maintaining an ability to
+allocate sufficiently large blocks of contiguous memory is usually a challenge.
+This makes the overhead involved in memory cgroup support more of a concern,
+particularly when the memory page size is small.
+
+Typically, embedded systems are comparatively simple and deterministic, and are
+required to remain stable over long periods. Invocation of the OOM-killer, were
+it to occur in an uncontrolled manner, would likely destabilize such systems.

Documentation/cgroups/memory.txt At reading, current status of OOM is sho · line 665

Quoted 10 lines
Documentation/cgroups/memory.txt +10 -0
+
+Even a well-designed system may be presented with external stimuli that could
+lead to OOM conditions. One example is a system that is required to check a
+user-supplied removable FAT filesystem. As there is no way to bound the size
+or coherence of the user's filesystem, the memory required to run dosfsck on
+it may exceed the system's capacity. Running dosfsck in a memory cgroup
+can preserve system stability even in the face of excessive memory demands.
+
+At the present time, only anonymous pages are included in NOMMU memory cgroup
+accounting.

What is the reason for tracking just anonymous memory?

Quoted 49 lines
Documentation/cgroups/memory.txt +2 -0
+
+12. TODO
 
 1. Add support for accounting huge pages (as a separate controller)
 2. Make per-cgroup scanner reclaim not-shared pages first

diff -uprN a/mm/nommu.c b/mm/nommu.c

mm/nommu.c +10 -3
@@ -524,8 +524,10 @@ static void delete_nommu_region(struct v
 /*
  * free a contiguous series of pages
  */
-static void free_page_series(unsigned long from, unsigned long to)
+static void free_page_series(unsigned long from, unsigned long to,
+			     const struct file *file)
 {
+	mem_cgroup_uncharge_start();
 	for (; from < to; from += PAGE_SIZE) {
 		struct page *page = virt_to_page(from);
 
@@ -534,8 +536,13 @@ static void free_page_series(unsigned lo
 		if (page_count(page) != 1)
 			kdebug("free page %p: refcount not one: %d",
 			       page, page_count(page));
+		/* Only anonymous pages are charged, currently */
+		if (!file)
+			mem_cgroup_uncharge_page(page);
+
 		put_page(page);
 	}
+	mem_cgroup_uncharge_end();
 }
 
 /*
@@ -563,7 +570,8 @@ static void __put_nommu_region(struct vm
 		 * from ramfs/tmpfs mustn't be released here */
 		if (region->vm_flags & VM_MAPPED_COPY) {
 			kdebug("free series");
-			free_page_series(region->vm_start, region->vm_top);
+			free_page_series(region->vm_start, region->vm_top,
+					 region->vm_file);
 		}
 		kmem_cache_free(vm_region_jar, region);
 	} else {
@@ -1117,9 +1125,27 @@ static int do_mmap_private(struct vm_are
 		set_page_refcounted(&pages[point]);
 
 	base = page_address(pages);
-	region->vm_flags = vma->vm_flags |= VM_MAPPED_COPY;

mm/nommu.c static int do_mmap_private(struct vm_are · line 1137

Quoted 10 lines
mm/nommu.c +8 -0
+
 	region->vm_start = (unsigned long) base;
 	region->vm_end   = region->vm_start + rlen;
+
+	/* Only anonymous pages are charged, currently */
+	if (!vma->vm_file) {
+		for (point = 0; point < total; point++) {
+			int charge_failed =
+				mem_cgroup_newpage_charge(&pages[point],
+							  current->mm,

Is current->mm same as vma->vm_mm? I think vma->vm_mm is cleaner.

Quoted 39 lines
mm/nommu.c +13 -3
+							  GFP_KERNEL);
+			if (charge_failed) {
+				free_page_series(region->vm_start,
+						 region->vm_end, NULL);
+				region->vm_start = region->vm_end = 0;
+				goto enomem;
+			}
+		}
+	}
+
+	region->vm_flags = vma->vm_flags |= VM_MAPPED_COPY;
 	region->vm_top   = region->vm_start + (total << PAGE_SHIFT);
 
 	vma->vm_start = region->vm_start;
@@ -1150,7 +1176,7 @@ static int do_mmap_private(struct vm_are
 	return 0;
 
 error_free:
-	free_page_series(region->vm_start, region->vm_end);
+	free_page_series(region->vm_start, region->vm_end, vma->vm_file);
 	region->vm_start = vma->vm_start = 0;
 	region->vm_end   = vma->vm_end = 0;
 	region->vm_top   = 0;
@@ -1555,7 +1581,7 @@ static int shrink_vma(struct mm_struct *
 	add_nommu_region(region);
 	up_write(&nommu_region_sem);
 
-	free_page_series(from, to);
+	free_page_series(from, to, vma->vm_file);
 	return 0;
 }
 
 
--

To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 8 lines
	Three Cheers,
	Balbir

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw
Quoted 27 lines · Balbir Singh
Quoted 25 lines · Steven J. Magnani

Add the necessary calls to track VM anonymous page usage (only).

V2 changes:
* Added update of memory cgroup documentation
* Clarify use of 'file' to distinguish anonymous mappings

---
diff -uprN a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt

Documentation/cgroups/memory.txt +2 -1
@@ -34,6 +34,7 @@ Current Status: linux-2.6.34-mmotm(devel
 
 Features:
  - accounting anonymous pages, file caches, swap caches usage and limiting them.
+   NOTE: On NOMMU systems, only anonymous pages are accounted.
  - private LRU and reclaim routine. (system's global LRU and private LRU
    work independently from each other)
  - optionally, memory+swap usage can be accounted and limited.
@@ -640,7 +641,30 @@ At reading, current status of OOM is sho
 	under_oom	 0 or 1 (if 1, the memory cgroup is under OOM, tasks may
 				 be stopped.)
 
-11. TODO
+11. NOMMU Support

<snip>

Quoted 5 lines

Documentation/cgroups/memory.txt At reading, current status of OOM is sho · line 665

Quoted 3 lines
Documentation/cgroups/memory.txt +3 -0
+
+At the present time, only anonymous pages are included in NOMMU memory cgroup
+accounting.

What is the reason for tracking just anonymous memory?

Tracking more than that is beyond my current scope, and perhaps of limited benefit under an assumption that NOMMU systems don't usually work with large files. The limitations of the implementation are documented, so hopefully anyone who needs more functionality will know that they need to implement it.

Quoted 3 lines
Quoted 3 lines

diff -uprN a/mm/nommu.c b/mm/nommu.c

mm/nommu.c +0 -0

<snip>

Quoted 17 lines
Quoted 5 lines
mm/nommu.c +0 -1
@@ -1117,9 +1125,27 @@ static int do_mmap_private(struct vm_are
 		set_page_refcounted(&pages[point]);
 
 	base = page_address(pages);
-	region->vm_flags = vma->vm_flags |= VM_MAPPED_COPY;

mm/nommu.c static int do_mmap_private(struct vm_are · line 1137

Quoted 10 lines
mm/nommu.c +8 -0
+
 	region->vm_start = (unsigned long) base;
 	region->vm_end   = region->vm_start + rlen;
+
+	/* Only anonymous pages are charged, currently */
+	if (!vma->vm_file) {
+		for (point = 0; point < total; point++) {
+			int charge_failed =
+				mem_cgroup_newpage_charge(&pages[point],
+							  current->mm,

Is current->mm same as vma->vm_mm? I think vma->vm_mm is cleaner.

I agree, but at the time this code runs, vma->vm_mm is NULL except for an executable file mapping - which is not the case for the anonymous pages we are trying to track. I will look into modifying do_mmap_pgoff() to set vm_mm before invoking do_mmap_private(); if that can be done without side effects, I'll change the code here as you suggest.

Thanks for the quick review.
------------------------------------------------------------------------
Steven J. Magnani "I claim this network for MARS!
www.digidescorp.com Earthling, return my space modulator!"

#include <standard.disclaimer>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#3 lore raw
Quoted 35 lines · On Tue, 19 Oct 2010 13:24:17 -0500 "Steven J. Magnani" <[email protected]>…
Quoted 27 lines · Balbir Singh
Quoted 25 lines · Steven J. Magnani

Add the necessary calls to track VM anonymous page usage (only).

V2 changes:
* Added update of memory cgroup documentation
* Clarify use of 'file' to distinguish anonymous mappings

---
diff -uprN a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt

Documentation/cgroups/memory.txt +2 -1
@@ -34,6 +34,7 @@ Current Status: linux-2.6.34-mmotm(devel
 
 Features:
  - accounting anonymous pages, file caches, swap caches usage and limiting them.
+   NOTE: On NOMMU systems, only anonymous pages are accounted.
  - private LRU and reclaim routine. (system's global LRU and private LRU
    work independently from each other)
  - optionally, memory+swap usage can be accounted and limited.
@@ -640,7 +641,30 @@ At reading, current status of OOM is sho
 	under_oom	 0 or 1 (if 1, the memory cgroup is under OOM, tasks may
 				 be stopped.)
 
-11. TODO
+11. NOMMU Support

<snip>

Quoted 5 lines

Documentation/cgroups/memory.txt At reading, current status of OOM is sho · line 665

Quoted 3 lines
Documentation/cgroups/memory.txt +3 -0
+
+At the present time, only anonymous pages are included in NOMMU memory cgroup
+accounting.

What is the reason for tracking just anonymous memory?

Quoted 6 lines

Tracking more than that is beyond my current scope, and perhaps of limited benefit under an assumption that NOMMU systems don't usually work with large files. The limitations of the implementation are documented, so hopefully anyone who needs more functionality will know that they need to implement it.

What happens at reaching limit ? memory can be reclaimed ?

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#4 lore raw
Quoted 11 lines · KAMEZAWA Hiroyuki
Quoted 8 lines · On Tue, 19 Oct 2010 13:24:17 -0500 "Steven J. Magnani" <[email protected]>…
Quoted 6 lines · Balbir Singh

Documentation/cgroups/memory.txt At reading, current status of OOM is sho · line 665

Quoted 3 lines · Steven J. Magnani
Documentation/cgroups/memory.txt +3 -0
+
+At the present time, only anonymous pages are included in NOMMU memory cgroup
+accounting.

What is the reason for tracking just anonymous memory?

Quoted 8 lines
Quoted 6 lines

Tracking more than that is beyond my current scope, and perhaps of limited benefit under an assumption that NOMMU systems don't usually work with large files. The limitations of the implementation are documented, so hopefully anyone who needs more functionality will know that they need to implement it.

What happens at reaching limit ? memory can be reclaimed ?

I'm not quite sure what you're asking. In my usage, the OOM-killer gets invoked and the 'runaway' dosfsck process gets terminated; at that point all its memory is freed.

Regards,
------------------------------------------------------------------------
Steven J. Magnani "I claim this network for MARS!
www.digidescorp.com Earthling, return my space modulator!"

#include <standard.disclaimer>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#5 lore raw
Quoted 19 lines · On Wed, 20 Oct 2010 07:49:17 -0500 "Steven J. Magnani" <[email protected]>…
Quoted 18 lines · KAMEZAWA Hiroyuki
Quoted 14 lines · On Tue, 19 Oct 2010 13:24:17 -0500 "Steven J. Magnani" <[email protected]>…
Quoted 6 lines · Balbir Singh

Documentation/cgroups/memory.txt At reading, current status of OOM is sho · line 665

Quoted 3 lines · Steven J. Magnani
Documentation/cgroups/memory.txt +3 -0
+
+At the present time, only anonymous pages are included in NOMMU memory cgroup
+accounting.

What is the reason for tracking just anonymous memory?

Tracking more than that is beyond my current scope, and perhaps of limited benefit under an assumption that NOMMU systems don't usually work with large files. The limitations of the implementation are documented, so hopefully anyone who needs more functionality will know that they need to implement it.

Quoted 6 lines
Quoted 1 line

What happens at reaching limit ? memory can be reclaimed ?

I'm not quite sure what you're asking. In my usage, the OOM-killer gets invoked and the 'runaway' dosfsck process gets terminated; at that point all its memory is freed.

Hmm. then, most of memcg codes are of no use for NOMMU.
I myself can't maintain NOMMU kernel. So, please test every -rc1 when this patch merged. OK ?

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#6 lore raw
Quoted 2 lines · KAMEZAWA Hiroyuki

I myself can't maintain NOMMU kernel. So, please test every -rc1 when this patch merged. OK ?

It's reasonable to ask that features be tested every so often, and since the memory cgroup code seems to be changing relatively frequently it probably needs exercising more often. I can't commit to this, though - I don't know how much longer I'll be working with NOMMU kernels, and they are notoriously fragile. Any memory access bug that would cause an oops (or SEGV) in a "normal" kernel can cause subtle and almost impossible to debug behavior on a NOMMU system. For this reason I always try to work with "stable" kernels (recent threads debating the term notwithstanding)...so it would be a pretty far jump to a -rc1 kernel.

Is this a showstopper (in which case David's patch to make the CGROUP_MEM_RES_CTLR Kconfig option depend on MMU should be implemented), or should I post V3 of the patch that has Balbir's suggested change?

Steve

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#7 lore raw
Quoted 11 lines · On Wed, 20 Oct 2010 21:35:16 -0500 "Steve Magnani" <[email protected]>…
Quoted 2 lines · KAMEZAWA Hiroyuki

I myself can't maintain NOMMU kernel. So, please test every -rc1 when this patch merged. OK ?

It's reasonable to ask that features be tested every so often, and since the memory cgroup code seems to be changing relatively frequently it probably needs exercising more often. I can't commit to this, though - I don't know how much longer I'll be working with NOMMU kernels, and they are notoriously fragile. Any memory access bug that would cause an oops (or SEGV) in a "normal" kernel can cause subtle and almost impossible to debug behavior on a NOMMU system. For this reason I always try to work with "stable" kernels (recent threads debating the term notwithstanding)...so it would be a pretty far jump to a -rc1 kernel.

Quoted 3 lines

Is this a showstopper (in which case David's patch to make the CGROUP_MEM_RES_CTLR Kconfig option depend on MMU should be implemented), or should I post V3 of the patch that has Balbir's suggested change?

I asked just because I'm curious and maintainance is always a concern. But it's not showstopper.

BTW, my request is updating Documentaion as:

1. Please clarify "When reaching limit, OOM-Kill is inoved."
2. Add TODO List.

    For example)For NOMMU guys, you don't need page_cgroup->lru because no page reclaim happens.
    I think you can delete page_cgroup->lru and save much memmory.

I'll ack if v3 seems O.K. But I'm sorry it's in merge-window and people will be busy for a while. please be patient.

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>