Skip to contentSkip to conversation

Zoned VM counters V5

PATCH In review Christoph Lameter 3 participants 25 messages Jun 21, 2006 – Jun 24, 2006 3 messages not in the archive

Subsystems (19)
lore
Share & tools
lore
#0 lore raw

cover Zoned VM counters V5

reliable whereas event counters do not need to be.

Zone based VM statistics are necessary to be able to determine what the state of memory in one zone is. In a NUMA system this can be helpful for local reclaim and other memory optimizations that may be able to shift VM load in order to get more balanced memory use.

It is also useful to know how the computing load affects the memory allocations on various zones. This patchset allows the retrieval of that data from userspace.

The patchset introduces a framework for counters that is a cross between the existing page_stats --which are simply global counters split per cpu-- and the approach of deferred incremental updates implemented for nr_pagecache.

Small per cpu 8 bit counters are added to struct zone. If the counter exceeds certain thresholds then the counters are accumulated in an array of atomic_long in the zone and in a global array that sums up all zone values. The small 8 bit counters are next to the per cpu page pointers and so they will be in high in the cpu cache when pages are allocated and freed.

Access to VM counter information for a zone and for the whole machine is then possible by simply indexing an array (Thanks to Nick Piggin for pointing out that approach). The access to the total number of pages of various types does no longer require the summing up of all per cpu counters.

Benefits of this patchset right now:

- Ability for UP and SMP configuration to determine how memory is balanced between the DMA, NORMAL and HIGHMEM zones.

- loops over all processors are avoided in writeback and reclaim paths. We can avoid caching the writeback information because the needed information is directly accessible.

- Special handling for nr_pagecache removed.

- zone_reclaim_interval vanishes since VM stats can now determine when it is worth to do local reclaim.

- Fast inline per node page state determination.

- Accurate counters in /sys/devices/system/node/node*/meminfo. Current counters are counting simply which processor allocated a page somewhere and guestimate based on that. So the counters were not useful to show the actual distribution of page use on a specific zone.

- The swap_prefetch patch requires per node statistics in order to figure out when processors of a node can prefetch. This patch provides some of the needed numbers.

- Detailed VM counters available in more /proc and /sys status files.

References to earlier discussions:
V1 http://marc.theaimsgroup.com/?l=linux-kernel&m=113511649910826&w=2
V2 http://marc.theaimsgroup.com/?l=linux-kernel&m=114980851924230&w=2
V3 http://marc.theaimsgroup.com/?l=linux-kernel&m=115014697910351&w=2
V4 http://marc.theaimsgroup.com/?l=linux-kernel&m=115024767318740&w=2

Performance tests with AIM7 did not show any regressions. Seems to be a tad faster even. Tested on ia64/NUMA. Builds fine on i386, SMP / UP. Includes fixes for s390/arm/uml arch code.

Changelog

V1->V2:
- Cleanup code, resequence and base patches on 2.6.17-rc6-mm1
- Reduce interrupt holdoffs
- Add zone reclaim interval removal patch

V2->V3:
- Against temp tree by Andrew. (2.6.17-rc6-mm2 - old patches) Temp patch at http://www.zip.com.au/~akpm/linux/patches/stuff/cl.bz2
- Incorporate additional fixes for arch code.
- Create vmstat.c/h from pieces of page_alloc.c.
- Do the swap prefetch support patches the right way.
- Reorganize patchset so that the tree compiles after each patch (However, swap prefetch/reiser4 patches are separate. So if a swap prefetch patch follows then two patches must be applied for the kernel to compile again).
- Do various prescribed tests. Make sure that there is no remaining reference to page state in some arch code.
- Optimize the node_page_state function so that it can be used inline.

V3->V4:
- nr_pagecache definition was not cleaned up in V3.
- Fix nfs issues with NR_UNSTABLE where the page reference was not valid and with NR_DIRTY.
- Update swap_prefetch patches after feedback from Colin.
- Rename NR_STAT_ITEMS to NR_VM_ZONE_STAT_ITEMS.
- IA64: Make CONFIG_DMA_IS_NORMAL depend on SGI_SN2. Others may be added in the future.
- Fix order issues with vmstat
- Limit crossposting

V4->V5:
- Drop special patches for swap prefetch and reiser4
- Rediff against 2.6.17-mm1.
- Rename NR_UNSTABLE -> NR_UNSTABLE_NFS
- Rename NR_DIRTY -> NR_FILE_DIRTY
- Rename NR_MAPPED -> NR_FILE_MAPPED
- Rename NR_PAGECACHE -> NR_FILE_PAGES
- Rename NR_ANON -> NR_ANON_PAGES
- Update strings displayed in /proc files but leave established strings as is.

The patchset consists of 14 patches that are following this one

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#1 lore raw

PATCH 1/14 Create vmstat.c/.h from page_alloc.c/.h

Move counter code from page_alloc.c/page-flags.h to vmstat.c/h.

Create vmstat.c/vmstat.h by separating the counter code and the proc functions.

Move the vm_stat_text array before zoneinfo_show.

mm/Makefile +1 -1
Index: linux-2.6.17-mm1/mm/Makefile
===================================================================
@@ -10,7 +10,7 @@ mmu-$(CONFIG_MMU)	:= fremap.o highmem.o 
 obj-y			:= bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \
 			   page_alloc.o page-writeback.o pdflush.o \
 			   readahead.o swap.o truncate.o vmscan.o \
-			   prio_tree.o util.o mmzone.o $(mmu-y)
+			   prio_tree.o util.o mmzone.o vmstat.o $(mmu-y)
 
 obj-$(CONFIG_SWAP)	+= page_io.o swap_state.o swapfile.o thrash.o
 obj-$(CONFIG_HUGETLBFS)	+= hugetlb.o
include/linux/mm.h +1 -1
Index: linux-2.6.17-mm1/include/linux/mm.h
===================================================================
@@ -4,6 +4,7 @@
 #include <linux/sched.h>
 #include <linux/errno.h>
 #include <linux/capability.h>
+#include <linux/vmstat.h>
 
 #ifdef __KERNEL__
 
@@ -37,7 +38,6 @@ extern int sysctl_legacy_va_layout;
 #include <asm/page.h>
 #include <asm/pgtable.h>
 #include <asm/processor.h>
-#include <asm/atomic.h>
 
 #define nth_page(page,n) pfn_to_page(page_to_pfn((page)) + (n))
 
include/linux/vmstat.h new file +135 -0

135 lines not on this page · raw message

mm/vmstat.c new file +417 -0

417 lines not on this page · raw message

mm/page_alloc.c +0 -407

419 lines not on this page · raw message

include/linux/page-flags.h +1 -131

144 lines not on this page · raw message

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#2 lore raw

PATCH 2/14 Basic ZVC (zoned vm counter) implementation

In reply to Christoph Lameter

Per zone counter infrastructure

The counters that we currently have for the VM are split per processor. The processor however has not much to do with the zone these pages belong to. We cannot tell f.e. how many ZONE_DMA pages are dirty.

So we are blind to potentially inbalances in the usage of memory in various zones. F.e. in a NUMA system we cannot tell how many pages are dirty on a particular node. If we knew then we could put measures into the VM to balance the use of memory between different zones and different nodes in a NUMA system. For example it would be possible to limit the dirty pages per node so that fast local memory is kept available even if a process is dirtying huge amounts of pages.

Another example is zone reclaim. We do not know how many unmapped pages exist per zone. So we just have to try to reclaim. If it is not working then we pause and try again later. It would be better if we knew when it makes sense to reclaim unmapped pages from a zone. This patchset allows the determination of the number of unmapped pages per zone. We can remove the zone reclaim interval with the counters introduced here.

Futhermore the ability to have various usage statistics available will allow the development of new NUMA balancing algorithms that may be able to improve the decision making in the scheduler of when to move a process to another node and hopefully will also enable automatic page migration through a user space program that can analyse the memory load distribution and then rebalance memory use in order to increase performance.

The counter framework here implements differential counters for each processor in struct zone. The differential counters are consolidated when a threshold is exceeded (like done in the current implementation for nr_pageache), when slab reaping occurs or when a consolidation function is called.

Consolidation uses atomic operations and accumulates counters per zone in the zone structure and also globally in the vm_stat array. VM functions can access the counts by simply indexing a global or zone specific array.

The arrangement of counters in an array also simplifies processing when output has to be generated for /proc/*.

Counters can be updated by calling inc/dec_zone_page_state or _inc/dec_zone_page_state analogous to *_page_state. The second group of functions can be called if it is known that interrupts are disabled.

Special optimized increment and decrement functions are provided. These can avoid certain checks and use increment or decrement instructions that an architecture may provide.

We also add a new CONFIG_DMA_IS_NORMAL that signifies that an architecture can do DMA to all memory and therefore ZONE_NORMAL will not be populated. This is only currently set for IA64 SGI SN2 and currently only affects node_page_state(). In the best case node_page_state can be reduced to retrieving a single counter for the one zone on the node.

include/linux/mmzone.h +9 -0
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -47,6 +47,9 @@ struct zone_padding {
 #define ZONE_PADDING(name)
 #endif
 
+enum zone_stat_item {
+	NR_VM_ZONE_STAT_ITEMS };
+
 struct per_cpu_pages {
 	int count;		/* number of pages in the list */
 	int high;		/* high watermark, emptying needed */
@@ -56,6 +59,10 @@ struct per_cpu_pages {
 
 struct per_cpu_pageset {
 	struct per_cpu_pages pcp[2];	/* 0: hot.  1: cold */
+#ifdef CONFIG_SMP
+	s8 vm_stat_diff[NR_VM_ZONE_STAT_ITEMS];
+#endif
+
 #ifdef CONFIG_NUMA
 	unsigned long numa_hit;		/* allocated in intended node */
 	unsigned long numa_miss;	/* allocated in non intended node */
@@ -166,6 +173,8 @@ struct zone {
 	/* A count of how many reclaimers are scanning this zone */
 	atomic_t		reclaim_in_progress;
 
+	/* Zone statistics */
+	atomic_long_t		vm_stat[NR_VM_ZONE_STAT_ITEMS];
 	/*
 	 * timestamp (in jiffies) of the last zone reclaim that did not
 	 * result in freeing of pages. This is used to avoid repeated scans
mm/page_alloc.c +2 -0
Index: linux-2.6.17-mm1/mm/page_alloc.c
===================================================================
@@ -1971,6 +1971,7 @@ static void __init free_area_init_core(s
 		zone->nr_scan_inactive = 0;
 		zone->nr_active = 0;
 		zone->nr_inactive = 0;
+		zap_zone_vm_stats(zone);
 		atomic_set(&zone->reclaim_in_progress, 0);
 		if (!size)
 			continue;
@@ -2072,6 +2073,7 @@ static int page_alloc_cpu_notify(struct 
 		}
 
 		local_irq_enable();
+		refresh_cpu_vm_stats(cpu);
 	}
 	return NOTIFY_OK;
 }
mm/slab.c +1 -0
Index: linux-2.6.17-mm1/mm/slab.c
===================================================================
@@ -3763,6 +3763,7 @@ next:
 	check_irq_on();
 	mutex_unlock(&cache_chain_mutex);
 	next_reap_node();
+	refresh_cpu_vm_stats(smp_processor_id());
 	/* Set up the next iteration */
 	schedule_delayed_work(&__get_cpu_var(reap_work), REAPTIMEOUT_CPUC);
 }
include/linux/vmstat.h +82 -0

93 lines not on this page · raw message

arch/ia64/Kconfig +5 -0
Index: linux-2.6.17-mm1/arch/ia64/Kconfig
===================================================================
@@ -70,6 +70,11 @@ config DMA_IS_DMA32
 	bool
 	default y
 
+config DMA_IS_NORMAL
+	bool
+	depends on IA64_SGI_SN2
+	default y
+
 choice
 	prompt "System type"
 	default IA64_GENERIC
mm/vmstat.c +276 -4

323 lines not on this page · raw message

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

Re: [PATCH 02/14] Basic ZVC (zoned vm counter) implementation

Quoted 1 line · Andrew Morton

These patches were a bit of a disaster - what happened?

I am not sure what you did to them. You removed pieces that were not touched by this patch. Could you undo your additional patches and tell me what the original problem was?

Quoted 10 lines

mm/vmstat.c void get_zone_counts(unsigned long *acti · line 163

Quoted 8 lines
mm/vmstat.c +8 -0
+atomic_long_t vm_stat[NR_VM_ZONE_STAT_ITEMS];
+
+static inline void zone_page_state_add(long x, struct zone *zone,
+				 enum zone_stat_item item)
+{
+	atomic_long_add(x, &zone->vm_stat[item]);
+	atomic_long_add(x, &vm_stat[item]);
+}

I'd have thought it'd be worth padding vm_stat out to one-per-cacheline.

On the other hand if the whole vm_stat is in one cacheline then multiple references to counters can be satisfied from one cacheline.

Quoted 17 lines

mm/vmstat.c void get_zone_counts(unsigned long *acti · line 198

Quoted 9 lines
mm/vmstat.c +9 -0
+	x = delta + *p;
+
+	if (unlikely(x > STAT_THRESHOLD || x < -STAT_THRESHOLD)) {
+		zone_page_state_add(x, zone, item);
+		x = 0;
+	}
+
+	*p = x;
+}

We'd get a little more efficiency with the same accuracy by doing:

	if (x > STAT_THRESHOLD) {
		zone_page_state_add(x + STAT_THRESHOLD/2, zone, item);
		x = -STAT_THRESHOLD/2;
	}
Quoted 4 lines

because these things aren't random - a CPU tends to do a lot of identical operations in sequence, then a batch of the opposite operation happens a relatively long time later. Memory allocations, pagetable instantiation, etc. Plus some counters only go in one direction.

0 is already in the middle of the interval. The interval is from -STAT_THRESHOLD to +STAT_THRESHOLD not from 0 to STAT_THRESHOLD.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#5 lore raw

PATCH 3/14 Convert nr_mapped to per zone counter

In reply to Christoph Lameter

nr_mapped is important because it allows a determination of how many pages of a zone are not mapped, which would allow a more efficient means of determining when we need to reclaim memory in a zone.

We take the nr_mapped field out of the page state structure and define a new per zone counter named NR_FILE_MAPPED (the anonymous pages will be split off from NR_MAPPED in the next patch).

We replace the use of nr_mapped in various kernel locations. This avoids the looping over all processors in try_to_free_pages(), writeback, reclaim (swap + zone reclaim).

arch/i386/mm/pgtable.c +1 -1
Index: linux-2.6.17-mm1/arch/i386/mm/pgtable.c
===================================================================
@@ -61,7 +61,7 @@ void show_mem(void)
 	get_page_state(&ps);
 	printk(KERN_INFO "%lu pages dirty\n", ps.nr_dirty);
 	printk(KERN_INFO "%lu pages writeback\n", ps.nr_writeback);
-	printk(KERN_INFO "%lu pages mapped\n", ps.nr_mapped);
+	printk(KERN_INFO "%lu pages mapped\n", global_page_state(NR_FILE_MAPPED));
 	printk(KERN_INFO "%lu pages slab\n", ps.nr_slab);
 	printk(KERN_INFO "%lu pages pagetables\n", ps.nr_page_table_pages);
 }
drivers/base/node.c +1 -3
Index: linux-2.6.17-mm1/drivers/base/node.c
===================================================================
@@ -53,8 +53,6 @@ static ssize_t node_read_meminfo(struct 
 		ps.nr_dirty = 0;
 	if ((long)ps.nr_writeback < 0)
 		ps.nr_writeback = 0;
-	if ((long)ps.nr_mapped < 0)
-		ps.nr_mapped = 0;
 	if ((long)ps.nr_slab < 0)
 		ps.nr_slab = 0;
 
@@ -83,7 +81,7 @@ static ssize_t node_read_meminfo(struct 
 		       nid, K(i.freeram - i.freehigh),
 		       nid, K(ps.nr_dirty),
 		       nid, K(ps.nr_writeback),
-		       nid, K(ps.nr_mapped),
+		       nid, K(node_page_state(nid, NR_FILE_MAPPED)),
 		       nid, K(ps.nr_slab));
 	n += hugetlb_report_node_meminfo(nid, buf + n);
 	return n;
fs/proc/proc_misc.c +1 -1
Index: linux-2.6.17-mm1/fs/proc/proc_misc.c
===================================================================
@@ -190,7 +190,7 @@ static int meminfo_read_proc(char *page,
 		K(i.freeswap),
 		K(ps.nr_dirty),
 		K(ps.nr_writeback),
-		K(ps.nr_mapped),
+		K(global_page_state(NR_FILE_MAPPED)),
 		K(ps.nr_slab),
 		K(allowed),
 		K(committed),
include/linux/mmzone.h +3 -0
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -48,6 +48,9 @@ struct zone_padding {
 #endif
 
 enum zone_stat_item {
+	NR_FILE_MAPPED,	/* mapped into pagetables.
+			   only modified from process context */
+
 	NR_VM_ZONE_STAT_ITEMS };
 
 struct per_cpu_pages {
mm/page_alloc.c +1 -1
Index: linux-2.6.17-mm1/mm/page_alloc.c
===================================================================
@@ -1314,7 +1314,7 @@ void show_free_areas(void)
 		ps.nr_unstable,
 		nr_free_pages(),
 		ps.nr_slab,
-		ps.nr_mapped,
+		global_page_state(NR_FILE_MAPPED),
 		ps.nr_page_table_pages);
 
 	for_each_zone(zone) {
mm/page-writeback.c +1 -1
Index: linux-2.6.17-mm1/mm/page-writeback.c
===================================================================
@@ -111,7 +111,7 @@ static void get_writeback_state(struct w
 {
 	wbs->nr_dirty = read_page_state(nr_dirty);
 	wbs->nr_unstable = read_page_state(nr_unstable);
-	wbs->nr_mapped = read_page_state(nr_mapped);
+	wbs->nr_mapped = global_page_state(NR_FILE_MAPPED);
 	wbs->nr_writeback = read_page_state(nr_writeback);
 }
 
mm/rmap.c +3 -3
Index: linux-2.6.17-mm1/mm/rmap.c
===================================================================
@@ -493,7 +493,7 @@ static void __page_set_anon_rmap(struct 
 	 * nr_mapped state can be updated without turning off
 	 * interrupts because it is not modified via interrupt.
 	 */
-	__inc_page_state(nr_mapped);
+	__inc_zone_page_state(page, NR_FILE_MAPPED);
 }
 
 /**
@@ -537,7 +537,7 @@ void page_add_new_anon_rmap(struct page 
 void page_add_file_rmap(struct page *page)
 {
 	if (atomic_inc_and_test(&page->_mapcount))
-		__inc_page_state(nr_mapped);
+		__inc_zone_page_state(page, NR_FILE_MAPPED);
 }
 
 /**
@@ -569,7 +569,7 @@ void page_remove_rmap(struct page *page)
 		 */
 		if (page_test_and_clear_dirty(page))
 			set_page_dirty(page);
-		__dec_page_state(nr_mapped);
+		__dec_zone_page_state(page, NR_FILE_MAPPED);
 	}
 }
 
mm/vmscan.c +3 -3
Index: linux-2.6.17-mm1/mm/vmscan.c
===================================================================
@@ -972,7 +972,7 @@ unsigned long try_to_free_pages(struct z
 	}
 
 	for (priority = DEF_PRIORITY; priority >= 0; priority--) {
-		sc.nr_mapped = read_page_state(nr_mapped);
+		sc.nr_mapped = global_page_state(NR_FILE_MAPPED);
 		sc.nr_scanned = 0;
 		if (!priority)
 			disable_swap_token();
@@ -1062,7 +1062,7 @@ loop_again:
 	total_scanned = 0;
 	nr_reclaimed = 0;
 	sc.may_writepage = !laptop_mode;
-	sc.nr_mapped = read_page_state(nr_mapped);
+	sc.nr_mapped = global_page_state(NR_FILE_MAPPED);
 
 	inc_page_state(pageoutrun);
 
@@ -1412,7 +1412,7 @@ static int __zone_reclaim(struct zone *z
 	struct scan_control sc = {
 		.may_writepage = !!(zone_reclaim_mode & RECLAIM_WRITE),
 		.may_swap = !!(zone_reclaim_mode & RECLAIM_SWAP),
-		.nr_mapped = read_page_state(nr_mapped),
+		.nr_mapped = global_page_state(NR_FILE_MAPPED),
 		.swap_cluster_max = max_t(unsigned long, nr_pages,
 					SWAP_CLUSTER_MAX),
 		.gfp_mask = gfp_mask,
mm/vmstat.c +1 -1
Index: linux-2.6.17-mm1/mm/vmstat.c
===================================================================
@@ -463,13 +463,13 @@ struct seq_operations fragmentation_op =
 
 static char *vmstat_text[] = {
 	/* Zoned VM counters */
+	"nr_mapped",
 
 	/* Page state */
 	"nr_dirty",
 	"nr_writeback",
 	"nr_unstable",
 	"nr_page_table_pages",
-	"nr_mapped",
 	"nr_slab",
 
 	"pgpgin",

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#6 lore raw

PATCH 4/14 Conversion of nr_pagecache to per zone counter

In reply to Christoph Lameter

Currently a single atomic variable is used to establish the size of the page cache in the whole machine. The zoned VM counters have the same method of implementation as the nr_pagecache code but also allow the determination of the pagecache size per zone.

Remove the special implementation for nr_pagecache and make it a zoned counter named NR_FILE_PAGES.

Updates of the page cache counters are always performed with interrupts off. We can therefore use the __ variant here.

arch/sparc64/kernel/sys_sunos32.c +1 -1
Index: linux-2.6.17-mm1/arch/sparc64/kernel/sys_sunos32.c
===================================================================
@@ -155,7 +155,7 @@ asmlinkage int sunos_brk(u32 baddr)
 	 * simple, it hopefully works in most obvious cases.. Easy to
 	 * fool it, but this should catch most mistakes.
 	 */
-	freepages = get_page_cache_size();
+	freepages = global_page_state(NR_FILE_PAGES);
 	freepages >>= 1;
 	freepages += nr_free_pages();
 	freepages += nr_swap_pages;
arch/sparc/kernel/sys_sunos.c +1 -1
Index: linux-2.6.17-mm1/arch/sparc/kernel/sys_sunos.c
===================================================================
@@ -196,7 +196,7 @@ asmlinkage int sunos_brk(unsigned long b
 	 * simple, it hopefully works in most obvious cases.. Easy to
 	 * fool it, but this should catch most mistakes.
 	 */
-	freepages = get_page_cache_size();
+	freepages = global_page_state(NR_FILE_PAGES);
 	freepages >>= 1;
 	freepages += nr_free_pages();
 	freepages += nr_swap_pages;
fs/proc/proc_misc.c +2 -1
Index: linux-2.6.17-mm1/fs/proc/proc_misc.c
===================================================================
@@ -142,7 +142,8 @@ static int meminfo_read_proc(char *page,
 	allowed = ((totalram_pages - hugetlb_total_pages())
 		* sysctl_overcommit_ratio / 100) + total_swap_pages;
 
-	cached = get_page_cache_size() - total_swapcache_pages - i.bufferram;
+	cached = global_page_state(NR_FILE_PAGES) -
+			total_swapcache_pages - i.bufferram;
 	if (cached < 0)
 		cached = 0;
 
include/linux/pagemap.h +0 -45

51 lines not on this page · raw message

mm/filemap.c +2 -2
Index: linux-2.6.17-mm1/mm/filemap.c
===================================================================
@@ -120,7 +120,7 @@ void __remove_from_page_cache(struct pag
 	radix_tree_delete(&mapping->page_tree, page->index);
 	page->mapping = NULL;
 	mapping->nrpages--;
-	pagecache_acct(-1);
+	__dec_zone_page_state(page, NR_FILE_PAGES);
 }
 
 void remove_from_page_cache(struct page *page)
@@ -415,7 +415,7 @@ int add_to_page_cache(struct page *page,
 			page->mapping = mapping;
 			page->index = offset;
 			mapping->nrpages++;
-			pagecache_acct(1);
+			__inc_zone_page_state(page, NR_FILE_PAGES);
 		}
 		write_unlock_irq(&mapping->tree_lock);
 		radix_tree_preload_end();
mm/mmap.c +1 -1
Index: linux-2.6.17-mm1/mm/mmap.c
===================================================================
@@ -96,7 +96,7 @@ int __vm_enough_memory(long pages, int c
 	if (sysctl_overcommit_memory == OVERCOMMIT_GUESS) {
 		unsigned long n;
 
-		free = get_page_cache_size();
+		free = global_page_state(NR_FILE_PAGES);
 		free += nr_swap_pages;
 
 		/*
mm/nommu.c +1 -1
Index: linux-2.6.17-mm1/mm/nommu.c
===================================================================
@@ -1122,7 +1122,7 @@ int __vm_enough_memory(long pages, int c
 	if (sysctl_overcommit_memory == OVERCOMMIT_GUESS) {
 		unsigned long n;
 
-		free = get_page_cache_size();
+		free = global_page_state(NR_FILE_PAGES);
 		free += nr_swap_pages;
 
 		/*
mm/page_alloc.c +0 -5
Index: linux-2.6.17-mm1/mm/page_alloc.c
===================================================================
@@ -2049,16 +2049,11 @@ static int page_alloc_cpu_notify(struct 
 				 unsigned long action, void *hcpu)
 {
 	int cpu = (unsigned long)hcpu;
-	long *count;
 	unsigned long *src, *dest;
 
 	if (action == CPU_DEAD) {
 		int i;
 
-		/* Drain local pagecache count. */
-		count = &per_cpu(nr_pagecache_local, cpu);
-		atomic_add(*count, &nr_pagecache);
-		*count = 0;
 		local_irq_disable();
 		__drain_pages(cpu);
 
mm/swap_state.c +2 -2
Index: linux-2.6.17-mm1/mm/swap_state.c
===================================================================
@@ -87,7 +87,7 @@ static int __add_to_swap_cache(struct pa
 			SetPageSwapCache(page);
 			set_page_private(page, entry.val);
 			total_swapcache_pages++;
-			pagecache_acct(1);
+			__inc_zone_page_state(page, NR_FILE_PAGES);
 		}
 		write_unlock_irq(&swapper_space.tree_lock);
 		radix_tree_preload_end();
@@ -132,7 +132,7 @@ void __delete_from_swap_cache(struct pag
 	set_page_private(page, 0);
 	ClearPageSwapCache(page);
 	total_swapcache_pages--;
-	pagecache_acct(-1);
+	__dec_zone_page_state(page, NR_FILE_PAGES);
 	INC_CACHE_INFO(del_total);
 }
 
include/linux/mmzone.h +1 -1
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -50,7 +50,7 @@ struct zone_padding {
 enum zone_stat_item {
 	NR_FILE_MAPPED,	/* mapped into pagetables.
 			   only modified from process context */
-
+	NR_FILE_PAGES,
 	NR_VM_ZONE_STAT_ITEMS };
 
 struct per_cpu_pages {
arch/s390/appldata/appldata_mem.c +2 -1
Index: linux-2.6.17-mm1/arch/s390/appldata/appldata_mem.c
===================================================================
@@ -130,7 +130,8 @@ static void appldata_get_mem_data(void *
 	mem_data->totalhigh = P2K(val.totalhigh);
 	mem_data->freehigh  = P2K(val.freehigh);
 	mem_data->bufferram = P2K(val.bufferram);
-	mem_data->cached    = P2K(atomic_read(&nr_pagecache) - val.bufferram);
+	mem_data->cached    = P2K(global_page_state(NR_FILE_PAGES)
+				- val.bufferram);
 
 	si_swapinfo(&val);
 	mem_data->totalswap = P2K(val.totalswap);
drivers/base/node.c +2 -0
Index: linux-2.6.17-mm1/drivers/base/node.c
===================================================================
@@ -68,6 +68,7 @@ static ssize_t node_read_meminfo(struct 
 		       "Node %d LowFree:      %8lu kB\n"
 		       "Node %d Dirty:        %8lu kB\n"
 		       "Node %d Writeback:    %8lu kB\n"
+		       "Node %d FilePages:    %8lu kB\n"
 		       "Node %d Mapped:       %8lu kB\n"
 		       "Node %d Slab:         %8lu kB\n",
 		       nid, K(i.totalram),
@@ -81,6 +82,7 @@ static ssize_t node_read_meminfo(struct 
 		       nid, K(i.freeram - i.freehigh),
 		       nid, K(ps.nr_dirty),
 		       nid, K(ps.nr_writeback),
+		       nid, K(node_page_state(nid, NR_FILE_PAGES)),
 		       nid, K(node_page_state(nid, NR_FILE_MAPPED)),
 		       nid, K(ps.nr_slab));
 	n += hugetlb_report_node_meminfo(nid, buf + n);
mm/vmstat.c +1 -6
Index: linux-2.6.17-mm1/mm/vmstat.c
===================================================================
@@ -20,12 +20,6 @@
  */
 static DEFINE_PER_CPU(struct page_state, page_states) = {0};
 
-atomic_t nr_pagecache = ATOMIC_INIT(0);
-EXPORT_SYMBOL(nr_pagecache);
-#ifdef CONFIG_SMP
-DEFINE_PER_CPU(long, nr_pagecache_local) = 0;
-#endif
-
 static void __get_page_state(struct page_state *ret, int nr, cpumask_t *cpumask)
 {
 	unsigned cpu;
@@ -464,6 +458,7 @@ struct seq_operations fragmentation_op =
 static char *vmstat_text[] = {
 	/* Zoned VM counters */
 	"nr_mapped",
+	"nr_file_pages",
 
 	/* Page state */
 	"nr_dirty",

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#7 lore raw

PATCH 5/14 Remove NR_FILE_MAPPED from scan control structure

In reply to Christoph Lameter

We can now access the number of pages in a mapped state in an inexpensive way in shrink_active_list. So drop the nr_mapped field from scan_control.

mm/vmscan.c +2 -7
Index: linux-2.6.17-mm1/mm/vmscan.c
===================================================================
@@ -46,8 +46,6 @@ struct scan_control {
 	/* Incremented by the number of inactive pages that were scanned */
 	unsigned long nr_scanned;
 
-	unsigned long nr_mapped;	/* From page_state */
-
 	/* This context's GFP mask */
 	gfp_t gfp_mask;
 
@@ -727,7 +725,8 @@ static void shrink_active_list(unsigned 
 		 * how much memory
 		 * is mapped.
 		 */
-		mapped_ratio = (sc->nr_mapped * 100) / total_memory;
+		mapped_ratio = (global_page_state(NR_FILE_MAPPED) * 100) /
+					total_memory;
 
 		/*
 		 * Now decide how much we really want to unmap some pages.  The
@@ -972,7 +971,6 @@ unsigned long try_to_free_pages(struct z
 	}
 
 	for (priority = DEF_PRIORITY; priority >= 0; priority--) {
-		sc.nr_mapped = global_page_state(NR_FILE_MAPPED);
 		sc.nr_scanned = 0;
 		if (!priority)
 			disable_swap_token();
@@ -1062,8 +1060,6 @@ loop_again:
 	total_scanned = 0;
 	nr_reclaimed = 0;
 	sc.may_writepage = !laptop_mode;
-	sc.nr_mapped = global_page_state(NR_FILE_MAPPED);
-
 	inc_page_state(pageoutrun);
 
 	for (i = 0; i < pgdat->nr_zones; i++) {
@@ -1412,7 +1408,6 @@ static int __zone_reclaim(struct zone *z
 	struct scan_control sc = {
 		.may_writepage = !!(zone_reclaim_mode & RECLAIM_WRITE),
 		.may_swap = !!(zone_reclaim_mode & RECLAIM_SWAP),
-		.nr_mapped = global_page_state(NR_FILE_MAPPED),
 		.swap_cluster_max = max_t(unsigned long, nr_pages,
 					SWAP_CLUSTER_MAX),
 		.gfp_mask = gfp_mask,

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#8 lore raw

PATCH 6/14 Split NR_ANON_PAGES off from NR_FILE_MAPPED

In reply to Christoph Lameter

The current NR_FILE_MAPPED is used by zone reclaim and the dirty load calculation as the number of mapped pagecache pages. However, that is not true. NR_FILE_MAPPED includes the mapped anonymous pages. This patch separates those and therefore allows an accurate tracking of the anonymous pages per zone.

It then becomes possible to determine the number of unmapped pages per zone and we can avoid scanning for unmapped pages if there are none.

Also it may now be possible to determine the mapped/unmapped ratio in get_dirty_limit. Isnt the number of anonymous pages irrelevant in that calculation?

Note that this will change the meaning of the number of mapped pages reported in /proc/vmstat /proc/meminfo and in the per node statistics. This may affect user space tools that monitor these counters! NR_FILE_MAPPED works like NR_FILE_DIRTY. It is only valid for pagecache pages.

fs/proc/proc_misc.c +2 -0
Index: linux-2.6.17-mm1/fs/proc/proc_misc.c
===================================================================
@@ -168,6 +168,7 @@ static int meminfo_read_proc(char *page,
 		"SwapFree:     %8lu kB\n"
 		"Dirty:        %8lu kB\n"
 		"Writeback:    %8lu kB\n"
+		"AnonPages:    %8lu kB\n"
 		"Mapped:       %8lu kB\n"
 		"Slab:         %8lu kB\n"
 		"CommitLimit:  %8lu kB\n"
@@ -191,6 +192,7 @@ static int meminfo_read_proc(char *page,
 		K(i.freeswap),
 		K(ps.nr_dirty),
 		K(ps.nr_writeback),
+		K(global_page_state(NR_ANON_PAGES)),
 		K(global_page_state(NR_FILE_MAPPED)),
 		K(ps.nr_slab),
 		K(allowed),
include/linux/mmzone.h +2 -1
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -48,7 +48,8 @@ struct zone_padding {
 #endif
 
 enum zone_stat_item {
-	NR_FILE_MAPPED,	/* mapped into pagetables.
+	NR_ANON_PAGES,	/* Mapped anonymous pages */
+	NR_FILE_MAPPED,	/* pagecache pages mapped into pagetables.
 			   only modified from process context */
 	NR_FILE_PAGES,
 	NR_VM_ZONE_STAT_ITEMS };
mm/rmap.c +3 -2
Index: linux-2.6.17-mm1/mm/rmap.c
===================================================================
@@ -493,7 +493,7 @@ static void __page_set_anon_rmap(struct 
 	 * nr_mapped state can be updated without turning off
 	 * interrupts because it is not modified via interrupt.
 	 */
-	__inc_zone_page_state(page, NR_FILE_MAPPED);
+	__inc_zone_page_state(page, NR_ANON_PAGES);
 }
 
 /**
@@ -569,7 +569,8 @@ void page_remove_rmap(struct page *page)
 		 */
 		if (page_test_and_clear_dirty(page))
 			set_page_dirty(page);
-		__dec_zone_page_state(page, NR_FILE_MAPPED);
+		__dec_zone_page_state(page,
+				PageAnon(page) ? NR_ANON_PAGES : NR_FILE_MAPPED);
 	}
 }
 
mm/vmscan.c +2 -1
Index: linux-2.6.17-mm1/mm/vmscan.c
===================================================================
@@ -725,7 +725,8 @@ static void shrink_active_list(unsigned 
 		 * how much memory
 		 * is mapped.
 		 */
-		mapped_ratio = (global_page_state(NR_FILE_MAPPED) * 100) /
+		mapped_ratio = ((global_page_state(NR_FILE_MAPPED) +
+				global_page_state(NR_ANON_PAGES)) * 100) /
 					total_memory;
 
 		/*
drivers/base/node.c +2 -0
Index: linux-2.6.17-mm1/drivers/base/node.c
===================================================================
@@ -70,6 +70,7 @@ static ssize_t node_read_meminfo(struct 
 		       "Node %d Writeback:    %8lu kB\n"
 		       "Node %d FilePages:    %8lu kB\n"
 		       "Node %d Mapped:       %8lu kB\n"
+		       "Node %d AnonPages:    %8lu kB\n"
 		       "Node %d Slab:         %8lu kB\n",
 		       nid, K(i.totalram),
 		       nid, K(i.freeram),
@@ -84,6 +85,7 @@ static ssize_t node_read_meminfo(struct 
 		       nid, K(ps.nr_writeback),
 		       nid, K(node_page_state(nid, NR_FILE_PAGES)),
 		       nid, K(node_page_state(nid, NR_FILE_MAPPED)),
+		       nid, K(node_page_state(nid, NR_ANON_PAGES)),
 		       nid, K(ps.nr_slab));
 	n += hugetlb_report_node_meminfo(nid, buf + n);
 	return n;
mm/page-writeback.c +2 -1
Index: linux-2.6.17-mm1/mm/page-writeback.c
===================================================================
@@ -111,7 +111,8 @@ static void get_writeback_state(struct w
 {
 	wbs->nr_dirty = read_page_state(nr_dirty);
 	wbs->nr_unstable = read_page_state(nr_unstable);
-	wbs->nr_mapped = global_page_state(NR_FILE_MAPPED);
+	wbs->nr_mapped = global_page_state(NR_FILE_MAPPED) +
+				global_page_state(NR_ANON_PAGES);
 	wbs->nr_writeback = read_page_state(nr_writeback);
 }
 
mm/vmstat.c +1 -0
Index: linux-2.6.17-mm1/mm/vmstat.c
===================================================================
@@ -457,6 +457,7 @@ struct seq_operations fragmentation_op =
 
 static char *vmstat_text[] = {
 	/* Zoned VM counters */
+	"nr_anon_pages",
 	"nr_mapped",
 	"nr_file_pages",
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

A message that is not in the archive (sent off-list or not yet received): [email protected]

#10 lore raw

Re: [PATCH 06/14] Split NR_ANON_PAGES off from NR_FILE_MAPPED

Quoted 2 lines · Martin J. Bligh

Isn't this still the number of mapped anon pages, rather than the total number of anon pages?

All anonymous pages are mapped. If an anonymous page is unmapped then it is freed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#11 lore raw

PATCH 7/14 zone_reclaim: remove /proc/sys/vm/zone_reclaim_interval

In reply to Christoph Lameter

The zone_reclaim_interval was necessary because we were not able to determine how many unmapped pages exist in a zone. Therefore we had to scan in intervals to figure out if any pages were unmapped.

With the zoned counters and NR_ANON_PAGES we now know the number of pagecache pages and the number of mapped pages in a zone. So we can simply skip the reclaim if there is an insufficient number of unmapped pages. We use SWAP_CLUSTER_MAX as the boundary.

Drop all support for /proc/sys/vm/zone_reclaim_interval.

include/linux/mmzone.h +0 -6
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -179,12 +179,6 @@ struct zone {
 
 	/* Zone statistics */
 	atomic_long_t		vm_stat[NR_VM_ZONE_STAT_ITEMS];
-	/*
-	 * timestamp (in jiffies) of the last zone reclaim that did not
-	 * result in freeing of pages. This is used to avoid repeated scans
-	 * if all memory in the zone is in use.
-	 */
-	unsigned long		last_unsuccessful_zone_reclaim;
 
 	/*
 	 * prev_priority holds the scanning priority for this zone.  It is
include/linux/swap.h +0 -1
Index: linux-2.6.17-mm1/include/linux/swap.h
===================================================================
@@ -194,7 +194,6 @@ extern pageout_t pageout(struct page *pa
 
 #ifdef CONFIG_NUMA
 extern int zone_reclaim_mode;
-extern int zone_reclaim_interval;
 extern int zone_reclaim(struct zone *, gfp_t, unsigned int);
 #else
 #define zone_reclaim_mode 0
kernel/sysctl.c +0 -9
Index: linux-2.6.17-mm1/kernel/sysctl.c
===================================================================
@@ -905,15 +905,6 @@ static ctl_table vm_table[] = {
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 	},
-	{
-		.ctl_name	= VM_ZONE_RECLAIM_INTERVAL,
-		.procname	= "zone_reclaim_interval",
-		.data		= &zone_reclaim_interval,
-		.maxlen		= sizeof(zone_reclaim_interval),
-		.mode		= 0644,
-		.proc_handler	= &proc_dointvec_jiffies,
-		.strategy	= &sysctl_jiffies,
-	},
 #endif
 	{ .ctl_name = 0 }
 };
mm/vmscan.c +10 -21
Index: linux-2.6.17-mm1/mm/vmscan.c
===================================================================
@@ -1384,11 +1384,6 @@ int zone_reclaim_mode __read_mostly;
 #define RECLAIM_SLAB (1<<3)	/* Do a global slab shrink if the zone is out of memory */
 
 /*
- * Mininum time between zone reclaim scans
- */
-int zone_reclaim_interval __read_mostly = 30*HZ;
-
-/*
  * Priority for ZONE_RECLAIM. This determines the fraction of pages
  * of a node considered for each zone_reclaim. 4 scans 1/16th of
  * a zone.
@@ -1452,16 +1447,6 @@ static int __zone_reclaim(struct zone *z
 
 	p->reclaim_state = NULL;
 	current->flags &= ~(PF_MEMALLOC | PF_SWAPWRITE);
-
-	if (nr_reclaimed == 0) {
-		/*
-		 * We were unable to reclaim enough pages to stay on node.  We
-		 * now allow off node accesses for a certain time period before
-		 * trying again to reclaim pages from the local zone.
-		 */
-		zone->last_unsuccessful_zone_reclaim = jiffies;
-	}
-
 	return nr_reclaimed >= nr_pages;
 }
 
@@ -1471,13 +1456,17 @@ int zone_reclaim(struct zone *zone, gfp_
 	int node_id;
 
 	/*
-	 * Do not reclaim if there was a recent unsuccessful attempt at zone
-	 * reclaim.  In that case we let allocations go off node for the
-	 * zone_reclaim_interval.  Otherwise we would scan for each off-node
-	 * page allocation.
+	 * Do not reclaim if there are not enough reclaimable pages in this
+	 * zone that would satify this allocations.
+	 *
+	 * All unmapped pagecache pages are reclaimable.
+	 *
+	 * Both counters may be temporarily off a bit so we use
+	 * SWAP_CLUSTER_MAX as the boundary. It may also be good to
+	 * leave a few frequently used unmapped pagecache pages around.
 	 */
-	if (time_before(jiffies,
-		zone->last_unsuccessful_zone_reclaim + zone_reclaim_interval))
+	if (zone_page_state(zone, NR_FILE_PAGES) -
+		zone_page_state(zone, NR_FILE_MAPPED) < SWAP_CLUSTER_MAX)
 			return 0;
 
 	/*
Documentation/sysctl/vm.txt +0 -13
Index: linux-2.6.17-mm1/Documentation/sysctl/vm.txt
===================================================================
@@ -28,7 +28,6 @@ Currently, these files are in /proc/sys/
 - block_dump
 - drop-caches
 - zone_reclaim_mode
-- zone_reclaim_interval
 
 ==============================================================
 
@@ -166,15 +165,3 @@ use of files and builds up large slab ca
 shrink operation is global, may take a long time and free slabs
 in all nodes of the system.
 
-================================================================
-
-zone_reclaim_interval:
-
-The time allowed for off node allocations after zone reclaim
-has failed to reclaim enough pages to allow a local allocation.
-
-Time is set in seconds and set by default to 30 seconds.
-
-Reduce the interval if undesired off node allocations occur. However, too
-frequent scans will have a negative impact onoff node allocation performance.
-

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#12 lore raw
In reply to Christoph Lameter
Quoted 11 lines

The patchset introduces a framework for counters that is a cross between the existing page_stats --which are simply global counters split per cpu-- and the approach of deferred incremental updates implemented for nr_pagecache.

Small per cpu 8 bit counters are added to struct zone. If the counter exceeds certain thresholds then the counters are accumulated in an array of atomic_long in the zone and in a global array that sums up all zone values. The small 8 bit counters are next to the per cpu page pointers and so they will be in high in the cpu cache when pages are allocated and freed.

Quoted 4 lines

Access to VM counter information for a zone and for the whole machine is then possible by simply indexing an array (Thanks to Nick Piggin for pointing out that approach). The access to the total number of pages of various types does no longer require the summing up of all per cpu counters.

Having the per-cpu counters with a global overflow seems like a really nice way to do counters to me - is it worth doing this as a more generalized counter type so that others could use it?

OTOH, I'm unsure why we're only using 8 bits in struct zone, which isn't
size critical. Is it just so you can pack vast numbers of different stats into a single cacheline?

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#13 lore raw
Quoted 3 lines · Martin J. Bligh

Having the per-cpu counters with a global overflow seems like a really nice way to do counters to me - is it worth doing this as a more generalized counter type so that others could use it?

Yes later patches also use the counters for other things. Please check out the patch that uses these for numa counters etc.

Quoted 3 lines

OTOH, I'm unsure why we're only using 8 bits in struct zone, which isn't size critical. Is it just so you can pack vast numbers of different stats into a single cacheline?

I would like to add some stats in the future. 8 bits is sufficient if the threshold is less than 64 (currently its 32). If we ever get higher then we can simply go to a bigger base size.

However, the space used by that array is

<nr-of-counters>*<nr_of_processors>*<nr_of_zones>

There are systems that have around 1k nodes and 4k processors. Lets say we have 16 counters then we get to

1k*4k*16 = 64Mbyte just for the counters.

This doubles for a short and quadruples for an int.

Also smaller counters help keep the pcp structure in one cacheline and reduces the cache footprint.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#14 lore raw
Quoted 9 lines · Christoph Lameter
Quoted 3 lines · Martin J. Bligh

Having the per-cpu counters with a global overflow seems like a really nice way to do counters to me - is it worth doing this as a more generalized counter type so that others could use it?

Yes later patches also use the counters for other things. Please check out the patch that uses these for numa counters etc.

OK, but looked like the original implementation was sort of tied to zones / the VM, at least in terminology, and code placement. I'll look at it again ...

Quoted 11 lines
Quoted 3 lines

OTOH, I'm unsure why we're only using 8 bits in struct zone, which isn't size critical. Is it just so you can pack vast numbers of different stats into a single cacheline?

I would like to add some stats in the future. 8 bits is sufficient if the threshold is less than 64 (currently its 32). If we ever get higher then we can simply go to a bigger base size.

However, the space used by that array is

<nr-of-counters>*<nr_of_processors>*<nr_of_zones>

Quoted 7 lines

There are systems that have around 1k nodes and 4k processors. Lets say we have 16 counters then we get to

1k*4k*16 = 64Mbyte just for the counters.

This doubles for a short and quadruples for an int.

Also smaller counters help keep the pcp structure in one cacheline and reduces the cache footprint.

Sure, but for a normal sized system, the smaller the per-cpu portion, the more atomic ops you'll end up doing, surely?

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#15 lore raw
Quoted 6 lines · Martin Bligh
Quoted 2 lines

Yes later patches also use the counters for other things. Please check out the patch that uses these for numa counters etc.

OK, but looked like the original implementation was sort of tied to zones / the VM, at least in terminology, and code placement. I'll look at it again ...

Yes it is. This one is useful only for zone related information.

Quoted 5 lines
Quoted 2 lines

Also smaller counters help keep the pcp structure in one cacheline and reduces the cache footprint.

Sure, but for a normal sized system, the smaller the per-cpu portion, the more atomic ops you'll end up doing, surely?

So we now do two atomic ops for every 32 increments (threshold). If we increment the threshhold then we reduce the atomic overhead but this also influences the inaccurary of the global and per zone counter because there is the potential of more counter update deferrals. Keeping the threshold low makes the global and per zone counter more up to date. I think 32 is a good compromise.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#16 lore raw
Quoted 17 lines · Christoph Lameter
Quoted 6 lines · Martin Bligh
Quoted 2 lines

Yes later patches also use the counters for other things. Please check out the patch that uses these for numa counters etc.

OK, but looked like the original implementation was sort of tied to zones / the VM, at least in terminology, and code placement. I'll look at it again ...

Yes it is. This one is useful only for zone related information.

Quoted 3 lines
Quoted 2 lines

Also smaller counters help keep the pcp structure in one cacheline and reduces the cache footprint.

Quoted 6 lines
Quoted 2 lines

Sure, but for a normal sized system, the smaller the per-cpu portion, the more atomic ops you'll end up doing, surely?

So we now do two atomic ops for every 32 increments (threshold). If we increment the threshhold then we reduce the atomic overhead but this also influences the inaccurary of the global and per zone counter because there is the potential of more counter update deferrals. Keeping the threshold low makes the global and per zone counter more up to date. I think 32 is a good compromise.

Ah, makes sense ... so the intent is get an an approximation. If you want accurate numbers, you still need to walk the per-cpu counters ... is that too slow for where you're reading it? didn't seem like a fastpath to me, given that balance_dirty_pages was already ratelimited? I guess it's still better than what we're doing now though ;-)

M.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#17 lore raw
Quoted 7 lines · Martin Bligh
Quoted 6 lines

So we now do two atomic ops for every 32 increments (threshold). If we increment the threshhold then we reduce the atomic overhead but this also influences the inaccurary of the global and per zone counter because there is the potential of more counter update deferrals. Keeping the threshold low makes the global and per zone counter more up to date.
I think 32 is a good compromise.

Quoted 5 lines

Ah, makes sense ... so the intent is get an an approximation. If you want accurate numbers, you still need to walk the per-cpu counters ... is that too slow for where you're reading it? didn't seem like a fastpath to me, given that balance_dirty_pages was already ratelimited? I guess it's still better than what we're doing now though ;-)

No you never walk per cpu counters. The advantage here is that the counters are always available without walking. Counters are regularly folded into the per zone and global counters via the cache reaper.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#18 lore raw

PATCH 8/14 Conversion of nr_slab to per zone counter

In reply to Christoph Lameter

- Allows reclaim to access counter without looping over processor counts.

- Allows accurate statistics on how many pages are used in a zone by the slab. This may become useful to balance slab allocations over various zones.

arch/i386/mm/pgtable.c +1 -1
Index: linux-2.6.17-mm1/arch/i386/mm/pgtable.c
===================================================================
@@ -62,7 +62,7 @@ void show_mem(void)
 	printk(KERN_INFO "%lu pages dirty\n", ps.nr_dirty);
 	printk(KERN_INFO "%lu pages writeback\n", ps.nr_writeback);
 	printk(KERN_INFO "%lu pages mapped\n", global_page_state(NR_FILE_MAPPED));
-	printk(KERN_INFO "%lu pages slab\n", ps.nr_slab);
+	printk(KERN_INFO "%lu pages slab\n", global_page_state(NR_SLAB));
 	printk(KERN_INFO "%lu pages pagetables\n", ps.nr_page_table_pages);
 }
 
drivers/base/node.c +1 -3
Index: linux-2.6.17-mm1/drivers/base/node.c
===================================================================
@@ -53,8 +53,6 @@ static ssize_t node_read_meminfo(struct 
 		ps.nr_dirty = 0;
 	if ((long)ps.nr_writeback < 0)
 		ps.nr_writeback = 0;
-	if ((long)ps.nr_slab < 0)
-		ps.nr_slab = 0;
 
 	n = sprintf(buf, "\n"
 		       "Node %d MemTotal:     %8lu kB\n"
@@ -86,7 +84,7 @@ static ssize_t node_read_meminfo(struct 
 		       nid, K(node_page_state(nid, NR_FILE_PAGES)),
 		       nid, K(node_page_state(nid, NR_FILE_MAPPED)),
 		       nid, K(node_page_state(nid, NR_ANON_PAGES)),
-		       nid, K(ps.nr_slab));
+		       nid, K(node_page_state(nid, NR_SLAB)));
 	n += hugetlb_report_node_meminfo(nid, buf + n);
 	return n;
 }
fs/proc/proc_misc.c +1 -1
Index: linux-2.6.17-mm1/fs/proc/proc_misc.c
===================================================================
@@ -194,7 +194,7 @@ static int meminfo_read_proc(char *page,
 		K(ps.nr_writeback),
 		K(global_page_state(NR_ANON_PAGES)),
 		K(global_page_state(NR_FILE_MAPPED)),
-		K(ps.nr_slab),
+		K(global_page_state(NR_SLAB)),
 		K(allowed),
 		K(committed),
 		K(ps.nr_page_table_pages),
include/linux/mmzone.h +1 -0
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -52,6 +52,7 @@ enum zone_stat_item {
 	NR_FILE_MAPPED,	/* pagecache pages mapped into pagetables.
 			   only modified from process context */
 	NR_FILE_PAGES,
+	NR_SLAB,	/* Pages used by slab allocator */
 	NR_VM_ZONE_STAT_ITEMS };
 
 struct per_cpu_pages {
mm/page_alloc.c +1 -1
Index: linux-2.6.17-mm1/mm/page_alloc.c
===================================================================
@@ -1313,7 +1313,7 @@ void show_free_areas(void)
 		ps.nr_writeback,
 		ps.nr_unstable,
 		nr_free_pages(),
-		ps.nr_slab,
+		global_page_state(NR_SLAB),
 		global_page_state(NR_FILE_MAPPED),
 		ps.nr_page_table_pages);
 
mm/slab.c +2 -2
Index: linux-2.6.17-mm1/mm/slab.c
===================================================================
@@ -1469,7 +1469,7 @@ static void *kmem_getpages(struct kmem_c
 	i = (1 << cachep->gfporder);
 	if (cachep->flags & SLAB_RECLAIM_ACCOUNT)
 		atomic_add(i, &slab_reclaim_pages);
-	add_page_state(nr_slab, i);
+	add_zone_page_state(page_zone(page), NR_SLAB, i);
 	while (i--) {
 		__SetPageSlab(page);
 		page++;
@@ -1491,7 +1491,7 @@ static void kmem_freepages(struct kmem_c
 		__ClearPageSlab(page);
 		page++;
 	}
-	sub_page_state(nr_slab, nr_freed);
+	sub_zone_page_state(page_zone(page), NR_SLAB, nr_freed);
 	if (current->reclaim_state)
 		current->reclaim_state->reclaimed_slab += nr_freed;
 	free_pages((unsigned long)addr, cachep->gfporder);
mm/vmstat.c +1 -1
Index: linux-2.6.17-mm1/mm/vmstat.c
===================================================================
@@ -460,13 +460,13 @@ static char *vmstat_text[] = {
 	"nr_anon_pages",
 	"nr_mapped",
 	"nr_file_pages",
+	"nr_slab",
 
 	/* Page state */
 	"nr_dirty",
 	"nr_writeback",
 	"nr_unstable",
 	"nr_page_table_pages",
-	"nr_slab",
 
 	"pgpgin",
 	"pgpgout",

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#19 lore raw

PATCH 9/14 Conversion of nr_pagetables to per zone counter

In reply to Christoph Lameter

Conversion of nr_page_table_pages to a per zone counter

arch/i386/mm/pgtable.c +2 -1
Index: linux-2.6.17-mm1/arch/i386/mm/pgtable.c
===================================================================
@@ -63,7 +63,8 @@ void show_mem(void)
 	printk(KERN_INFO "%lu pages writeback\n", ps.nr_writeback);
 	printk(KERN_INFO "%lu pages mapped\n", global_page_state(NR_FILE_MAPPED));
 	printk(KERN_INFO "%lu pages slab\n", global_page_state(NR_SLAB));
-	printk(KERN_INFO "%lu pages pagetables\n", ps.nr_page_table_pages);
+	printk(KERN_INFO "%lu pages pagetables\n",
+					global_page_state(NR_PAGETABLE));
 }
 
 /*
fs/proc/proc_misc.c +2 -2
Index: linux-2.6.17-mm1/fs/proc/proc_misc.c
===================================================================
@@ -171,9 +171,9 @@ static int meminfo_read_proc(char *page,
 		"AnonPages:    %8lu kB\n"
 		"Mapped:       %8lu kB\n"
 		"Slab:         %8lu kB\n"
+		"PageTables:   %8lu kB\n"
 		"CommitLimit:  %8lu kB\n"
 		"Committed_AS: %8lu kB\n"
-		"PageTables:   %8lu kB\n"
 		"VmallocTotal: %8lu kB\n"
 		"VmallocUsed:  %8lu kB\n"
 		"VmallocChunk: %8lu kB\n",
@@ -195,9 +195,9 @@ static int meminfo_read_proc(char *page,
 		K(global_page_state(NR_ANON_PAGES)),
 		K(global_page_state(NR_FILE_MAPPED)),
 		K(global_page_state(NR_SLAB)),
+		K(global_page_state(NR_PAGETABLE)),
 		K(allowed),
 		K(committed),
-		K(ps.nr_page_table_pages),
 		(unsigned long)VMALLOC_TOTAL >> 10,
 		vmi.used >> 10,
 		vmi.largest_chunk >> 10
include/linux/mmzone.h +1 -0
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -53,6 +53,7 @@ enum zone_stat_item {
 			   only modified from process context */
 	NR_FILE_PAGES,
 	NR_SLAB,	/* Pages used by slab allocator */
+	NR_PAGETABLE,	/* used for pagetables */
 	NR_VM_ZONE_STAT_ITEMS };
 
 struct per_cpu_pages {
mm/memory.c +2 -2
Index: linux-2.6.17-mm1/mm/memory.c
===================================================================
@@ -126,7 +126,7 @@ static void free_pte_range(struct mmu_ga
 	pmd_clear(pmd);
 	pte_lock_deinit(page);
 	pte_free_tlb(tlb, page);
-	dec_page_state(nr_page_table_pages);
+	dec_zone_page_state(page, NR_PAGETABLE);
 	tlb->mm->nr_ptes--;
 }
 
@@ -311,7 +311,7 @@ int __pte_alloc(struct mm_struct *mm, pm
 		pte_free(new);
 	} else {
 		mm->nr_ptes++;
-		inc_page_state(nr_page_table_pages);
+		inc_zone_page_state(new, NR_PAGETABLE);
 		pmd_populate(mm, pmd, new);
 	}
 	spin_unlock(&mm->page_table_lock);
mm/page_alloc.c +1 -1
Index: linux-2.6.17-mm1/mm/page_alloc.c
===================================================================
@@ -1315,7 +1315,7 @@ void show_free_areas(void)
 		nr_free_pages(),
 		global_page_state(NR_SLAB),
 		global_page_state(NR_FILE_MAPPED),
-		ps.nr_page_table_pages);
+		global_page_state(NR_PAGETABLE));
 
 	for_each_zone(zone) {
 		int i;
drivers/base/node.c +2 -0
Index: linux-2.6.17-mm1/drivers/base/node.c
===================================================================
@@ -69,6 +69,7 @@ static ssize_t node_read_meminfo(struct 
 		       "Node %d FilePages:    %8lu kB\n"
 		       "Node %d Mapped:       %8lu kB\n"
 		       "Node %d AnonPages:    %8lu kB\n"
+		       "Node %d PageTables:   %8lu kB\n"
 		       "Node %d Slab:         %8lu kB\n",
 		       nid, K(i.totalram),
 		       nid, K(i.freeram),
@@ -84,6 +85,7 @@ static ssize_t node_read_meminfo(struct 
 		       nid, K(node_page_state(nid, NR_FILE_PAGES)),
 		       nid, K(node_page_state(nid, NR_FILE_MAPPED)),
 		       nid, K(node_page_state(nid, NR_ANON_PAGES)),
+		       nid, K(node_page_state(nid, NR_PAGETABLE)),
 		       nid, K(node_page_state(nid, NR_SLAB)));
 	n += hugetlb_report_node_meminfo(nid, buf + n);
 	return n;
arch/um/kernel/skas/mmu.c +1 -1
Index: linux-2.6.17-mm1/arch/um/kernel/skas/mmu.c
===================================================================
@@ -152,7 +152,7 @@ void destroy_context_skas(struct mm_stru
 		free_page(mmu->id.stack);
 		pte_lock_deinit(virt_to_page(mmu->last_page_table));
 		pte_free_kernel((pte_t *) mmu->last_page_table);
-                dec_page_state(nr_page_table_pages);
+		dec_zone_page_state(virt_to_page(mmu->last_page_table), NR_PAGETABLE);
 #ifdef CONFIG_3_LEVEL_PGTABLES
 		pmd_free((pmd_t *) mmu->last_pmd);
 #endif
arch/arm/mm/mm-armv.c +1 -1
Index: linux-2.6.17-mm1/arch/arm/mm/mm-armv.c
===================================================================
@@ -227,7 +227,7 @@ void free_pgd_slow(pgd_t *pgd)
 
 	pte = pmd_page(*pmd);
 	pmd_clear(pmd);
-	dec_page_state(nr_page_table_pages);
+	dec_zone_page_state(virt_to_page((unsigned long *)pgd), NR_PAGETABLE);
 	pte_lock_deinit(pte);
 	pte_free(pte);
 	pmd_free(pmd);
mm/vmstat.c +1 -1
Index: linux-2.6.17-mm1/mm/vmstat.c
===================================================================
@@ -461,12 +461,12 @@ static char *vmstat_text[] = {
 	"nr_mapped",
 	"nr_file_pages",
 	"nr_slab",
+	"nr_page_table_pages",
 
 	/* Page state */
 	"nr_dirty",
 	"nr_writeback",
 	"nr_unstable",
-	"nr_page_table_pages",
 
 	"pgpgin",
 	"pgpgout",

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#20 lore raw

PATCH 10/14 Conversion of nr_dirty to per zone counter

In reply to Christoph Lameter

This makes nr_dirty a per zone counter. Looping over all processors is avoided during writeback state determination.

The counter aggregation for nr_dirty had to be undone in the NFS layer since we summed up the page counts from multiple zones. Someone more familiar with NFS should probably review what I have done.

arch/i386/mm/pgtable.c +1 -1
Index: linux-2.6.17-mm1/arch/i386/mm/pgtable.c
===================================================================
@@ -59,7 +59,7 @@ void show_mem(void)
 	printk(KERN_INFO "%d pages swap cached\n", cached);
 
 	get_page_state(&ps);
-	printk(KERN_INFO "%lu pages dirty\n", ps.nr_dirty);
+	printk(KERN_INFO "%lu pages dirty\n", global_page_state(NR_FILE_DIRTY));
 	printk(KERN_INFO "%lu pages writeback\n", ps.nr_writeback);
 	printk(KERN_INFO "%lu pages mapped\n", global_page_state(NR_FILE_MAPPED));
 	printk(KERN_INFO "%lu pages slab\n", global_page_state(NR_SLAB));
drivers/base/node.c +1 -3
Index: linux-2.6.17-mm1/drivers/base/node.c
===================================================================
@@ -49,8 +49,6 @@ static ssize_t node_read_meminfo(struct 
 	__get_zone_counts(&active, &inactive, &free, NODE_DATA(nid));
 
 	/* Check for negative values in these approximate counters */
-	if ((long)ps.nr_dirty < 0)
-		ps.nr_dirty = 0;
 	if ((long)ps.nr_writeback < 0)
 		ps.nr_writeback = 0;
 
@@ -80,7 +78,7 @@ static ssize_t node_read_meminfo(struct 
 		       nid, K(i.freehigh),
 		       nid, K(i.totalram - i.totalhigh),
 		       nid, K(i.freeram - i.freehigh),
-		       nid, K(ps.nr_dirty),
+		       nid, K(node_page_state(nid, NR_FILE_DIRTY)),
 		       nid, K(ps.nr_writeback),
 		       nid, K(node_page_state(nid, NR_FILE_PAGES)),
 		       nid, K(node_page_state(nid, NR_FILE_MAPPED)),
fs/buffer.c +1 -1
Index: linux-2.6.17-mm1/fs/buffer.c
===================================================================
@@ -854,7 +854,7 @@ int __set_page_dirty_buffers(struct page
 		write_lock_irq(&mapping->tree_lock);
 		if (page->mapping) {	/* Race with truncate? */
 			if (mapping_cap_account_dirty(mapping))
-				inc_page_state(nr_dirty);
+				__inc_zone_page_state(page, NR_FILE_DIRTY);
 			radix_tree_tag_set(&mapping->page_tree,
 						page_index(page),
 						PAGECACHE_TAG_DIRTY);
fs/fs-writeback.c +1 -1
Index: linux-2.6.17-mm1/fs/fs-writeback.c
===================================================================
@@ -462,7 +462,7 @@ void sync_inodes_sb(struct super_block *
 	struct writeback_control wbc = {
 		.sync_mode	= wait ? WB_SYNC_ALL : WB_SYNC_HOLD,
 	};
-	unsigned long nr_dirty = read_page_state(nr_dirty);
+	unsigned long nr_dirty = global_page_state(NR_FILE_DIRTY);
 	unsigned long nr_unstable = read_page_state(nr_unstable);
 
 	wbc.nr_to_write = nr_dirty + nr_unstable +
fs/nfs/pagelist.c +1 -0
Index: linux-2.6.17-mm1/fs/nfs/pagelist.c
===================================================================
@@ -315,6 +315,7 @@ nfs_scan_lock_dirty(struct nfs_inode *nf
 						req->wb_index, NFS_PAGE_TAG_DIRTY);
 				nfs_list_remove_request(req);
 				nfs_list_add_request(req, dst);
+				dec_zone_page_state(req->wb_page, NR_FILE_DIRTY);
 				res++;
 			}
 		}
fs/nfs/write.c +1 -2
Index: linux-2.6.17-mm1/fs/nfs/write.c
===================================================================
@@ -501,7 +501,7 @@ nfs_mark_request_dirty(struct nfs_page *
 	nfs_list_add_request(req, &nfsi->dirty);
 	nfsi->ndirty++;
 	spin_unlock(&nfsi->req_lock);
-	inc_page_state(nr_dirty);
+	inc_zone_page_state(req->wb_page, NR_FILE_DIRTY);
 	mark_inode_dirty(inode);
 }
 
@@ -602,7 +602,6 @@ nfs_scan_dirty(struct inode *inode, stru
 	if (nfsi->ndirty != 0) {
 		res = nfs_scan_lock_dirty(nfsi, dst, idx_start, npages);
 		nfsi->ndirty -= res;
-		sub_page_state(nr_dirty,res);
 		if ((nfsi->ndirty == 0) != list_empty(&nfsi->dirty))
 			printk(KERN_ERR "NFS: desynchronized value of nfs_i.ndirty.\n");
 	}
fs/proc/proc_misc.c +1 -1
Index: linux-2.6.17-mm1/fs/proc/proc_misc.c
===================================================================
@@ -190,7 +190,7 @@ static int meminfo_read_proc(char *page,
 		K(i.freeram-i.freehigh),
 		K(i.totalswap),
 		K(i.freeswap),
-		K(ps.nr_dirty),
+		K(global_page_state(NR_FILE_DIRTY)),
 		K(ps.nr_writeback),
 		K(global_page_state(NR_ANON_PAGES)),
 		K(global_page_state(NR_FILE_MAPPED)),
include/linux/mmzone.h +1 -0
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -54,6 +54,7 @@ enum zone_stat_item {
 	NR_FILE_PAGES,
 	NR_SLAB,	/* Pages used by slab allocator */
 	NR_PAGETABLE,	/* used for pagetables */
+	NR_FILE_DIRTY,
 	NR_VM_ZONE_STAT_ITEMS };
 
 struct per_cpu_pages {
mm/page_alloc.c +1 -1
Index: linux-2.6.17-mm1/mm/page_alloc.c
===================================================================
@@ -1309,7 +1309,7 @@ void show_free_areas(void)
 		"unstable:%lu free:%u slab:%lu mapped:%lu pagetables:%lu\n",
 		active,
 		inactive,
-		ps.nr_dirty,
+		global_page_state(NR_FILE_DIRTY),
 		ps.nr_writeback,
 		ps.nr_unstable,
 		nr_free_pages(),
mm/page-writeback.c +5 -5
Index: linux-2.6.17-mm1/mm/page-writeback.c
===================================================================
@@ -109,7 +109,7 @@ struct writeback_state
 
 static void get_writeback_state(struct writeback_state *wbs)
 {
-	wbs->nr_dirty = read_page_state(nr_dirty);
+	wbs->nr_dirty = global_page_state(NR_FILE_DIRTY);
 	wbs->nr_unstable = read_page_state(nr_unstable);
 	wbs->nr_mapped = global_page_state(NR_FILE_MAPPED) +
 				global_page_state(NR_ANON_PAGES);
@@ -638,7 +638,7 @@ int __set_page_dirty_nobuffers(struct pa
 			if (mapping2) { /* Race with truncate? */
 				BUG_ON(mapping2 != mapping);
 				if (mapping_cap_account_dirty(mapping))
-					inc_page_state(nr_dirty);
+					__inc_zone_page_state(page, NR_FILE_DIRTY);
 				radix_tree_tag_set(&mapping->page_tree,
 					page_index(page), PAGECACHE_TAG_DIRTY);
 			}
@@ -725,9 +725,9 @@ int test_clear_page_dirty(struct page *p
 			radix_tree_tag_clear(&mapping->page_tree,
 						page_index(page),
 						PAGECACHE_TAG_DIRTY);
-			write_unlock_irqrestore(&mapping->tree_lock, flags);
 			if (mapping_cap_account_dirty(mapping))
-				dec_page_state(nr_dirty);
+				__dec_zone_page_state(page, NR_FILE_DIRTY);
+			write_unlock_irqrestore(&mapping->tree_lock, flags);
 			return 1;
 		}
 		write_unlock_irqrestore(&mapping->tree_lock, flags);
@@ -758,7 +758,7 @@ int clear_page_dirty_for_io(struct page 
 	if (mapping) {
 		if (TestClearPageDirty(page)) {
 			if (mapping_cap_account_dirty(mapping))
-				dec_page_state(nr_dirty);
+				dec_zone_page_state(page, NR_FILE_DIRTY);
 			return 1;
 		}
 		return 0;
mm/vmstat.c +1 -1
Index: linux-2.6.17-mm1/mm/vmstat.c
===================================================================
@@ -462,9 +462,9 @@ static char *vmstat_text[] = {
 	"nr_file_pages",
 	"nr_slab",
 	"nr_page_table_pages",
+	"nr_dirty",
 
 	/* Page state */
-	"nr_dirty",
 	"nr_writeback",
 	"nr_unstable",
 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

A message that is not in the archive (sent off-list or not yet received): [email protected]

#22 lore raw

Re: [PATCH 10/14] Conversion of nr_dirty to per zone counter

Quoted 7 lines · Andrew Morton
Quoted 3 lines

The counter aggregation for nr_dirty had to be undone in the NFS layer since we summed up the page counts from multiple zones. Someone more familiar with NFS should probably review what I have done.

After fixing all the bugs, and with just a trivial, trivial, trivial amount of testing, the NFS unstable pages count goes negative and the machine hangs in balance_dirty_pages().

Ahh... SMP / NUMA configurations may mask a problem here by returning counters <0 as 0. I did not do this specific test on an UP system when I chased down the NFS counter issues. When I tested this on SMP/NUMA NR_UNSTABLE correctly went up and down while copying a file.

On a UP system the counters do not have deltas and therefore no <0 checking is necessary. I guess we expose the problem that way.

Quoted 3 lines

Christoph, this just isn't good enough and we need to do something significant to reduce the number of compilation errors and runtime bugs which you keep on sending.

I should probably switch my main development environment to be exactly like yours so that bugs that are trivial to you are also trivial to me. So need to set up a i386 development machine I guess.
r
But them my company would like to have the major development platform to be IA64.

I hope to have a fix soon .... This only surfaces on UP as far as I can tell.

Temporary safe fix if its urgent (this messes up inter zone NR_UNSTABLE accounting but there is nothing that uses zone specific NR_UNSTABLE yet)

fs/nfs/pagelist.c +0 -1
Index: linux-2.6.17/fs/nfs/pagelist.c
===================================================================
@@ -154,7 +154,6 @@
 {
 	struct page *page = req->wb_page;
 	if (page != NULL) {
-		dec_zone_page_state(page, NR_UNSTABLE_NFS);
 		page_cache_release(page);
 		req->wb_page = NULL;
 	}
fs/nfs/write.c +3 -0
Index: linux-2.6.17/fs/nfs/write.c
===================================================================
@@ -1424,6 +1424,9 @@
 	next:
 		if (req->wb_page)
 			dec_zone_page_state(req->wb_page, NR_UNSTABLE_NFS);
+		else	/* Page is gone but we do not know which zone so use
+			 * the zone of ZERO_PAGE for now */
+			dec_zone_page_state(ZERO_PAGE(0), NR_UNSTABLE_NFS);
 		nfs_clear_page_writeback(req);
 	}
 }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#23 lore raw

Re: [PATCH 10/14] Conversion of nr_dirty to per zone counter

In reply to a message not in the archive

Fix NR_UNSTABLE_NFS accounting

Move the decrement of NR_UNSTABLE higher in the loop where the page is still available and get rid of the old hokus pokus that just caused us grief.

Tested on: NUMA IA64 and a x86_64 single processor configuration.

fs/nfs/pagelist.c +0 -1
Index: linux-2.6.17-mm1/fs/nfs/pagelist.c
===================================================================
@@ -154,7 +154,6 @@ void nfs_clear_request(struct nfs_page *
 {
 	struct page *page = req->wb_page;
 	if (page != NULL) {
-		dec_zone_page_state(page, NR_UNSTABLE_NFS);
 		page_cache_release(page);
 		req->wb_page = NULL;
 	}
fs/nfs/write.c +1 -2
Index: linux-2.6.17-mm1/fs/nfs/write.c
===================================================================
@@ -1397,6 +1397,7 @@ static void nfs_commit_done(struct rpc_t
 	while (!list_empty(&data->pages)) {
 		req = nfs_list_entry(data->pages.next);
 		nfs_list_remove_request(req);
+		dec_zone_page_state(req->wb_page, NR_UNSTABLE_NFS);
 
 		dprintk("NFS: commit (%s/%Ld %d@%Ld)",
 			req->wb_context->dentry->d_inode->i_sb->s_id,
@@ -1422,8 +1423,6 @@ static void nfs_commit_done(struct rpc_t
 		dprintk(" mismatch\n");
 		nfs_mark_request_dirty(req);
 	next:
-		if (req->wb_page)
-			dec_zone_page_state(req->wb_page, NR_UNSTABLE_NFS);
 		nfs_clear_page_writeback(req);
 	}
 }

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#24 lore raw

PATCH 11/14 Conversion of nr_writeback to per zone counter

In reply to Christoph Lameter

Conversion of nr_writeback to per zone counter.

This removes the last page_state counter from arch/i386/mm/pgtable.c so we drop the page_state from there.

arch/i386/mm/pgtable.c +2 -3
Index: linux-2.6.17-mm1/arch/i386/mm/pgtable.c
===================================================================
@@ -30,7 +30,6 @@ void show_mem(void)
 	struct page *page;
 	pg_data_t *pgdat;
 	unsigned long i;
-	struct page_state ps;
 	unsigned long flags;
 
 	printk(KERN_INFO "Mem-info:\n");
@@ -58,9 +57,9 @@ void show_mem(void)
 	printk(KERN_INFO "%d pages shared\n", shared);
 	printk(KERN_INFO "%d pages swap cached\n", cached);
 
-	get_page_state(&ps);
 	printk(KERN_INFO "%lu pages dirty\n", global_page_state(NR_FILE_DIRTY));
-	printk(KERN_INFO "%lu pages writeback\n", ps.nr_writeback);
+	printk(KERN_INFO "%lu pages writeback\n",
+					global_page_state(NR_WRITEBACK));
 	printk(KERN_INFO "%lu pages mapped\n", global_page_state(NR_FILE_MAPPED));
 	printk(KERN_INFO "%lu pages slab\n", global_page_state(NR_SLAB));
 	printk(KERN_INFO "%lu pages pagetables\n",
drivers/base/node.c +1 -4
Index: linux-2.6.17-mm1/drivers/base/node.c
===================================================================
@@ -48,9 +48,6 @@ static ssize_t node_read_meminfo(struct 
 	get_page_state_node(&ps, nid);
 	__get_zone_counts(&active, &inactive, &free, NODE_DATA(nid));
 
-	/* Check for negative values in these approximate counters */
-	if ((long)ps.nr_writeback < 0)
-		ps.nr_writeback = 0;
 
 	n = sprintf(buf, "\n"
 		       "Node %d MemTotal:     %8lu kB\n"
@@ -79,7 +76,7 @@ static ssize_t node_read_meminfo(struct 
 		       nid, K(i.totalram - i.totalhigh),
 		       nid, K(i.freeram - i.freehigh),
 		       nid, K(node_page_state(nid, NR_FILE_DIRTY)),
-		       nid, K(ps.nr_writeback),
+		       nid, K(node_page_state(nid, NR_WRITEBACK)),
 		       nid, K(node_page_state(nid, NR_FILE_PAGES)),
 		       nid, K(node_page_state(nid, NR_FILE_MAPPED)),
 		       nid, K(node_page_state(nid, NR_ANON_PAGES)),
fs/proc/proc_misc.c +1 -1
Index: linux-2.6.17-mm1/fs/proc/proc_misc.c
===================================================================
@@ -191,7 +191,7 @@ static int meminfo_read_proc(char *page,
 		K(i.totalswap),
 		K(i.freeswap),
 		K(global_page_state(NR_FILE_DIRTY)),
-		K(ps.nr_writeback),
+		K(global_page_state(NR_WRITEBACK)),
 		K(global_page_state(NR_ANON_PAGES)),
 		K(global_page_state(NR_FILE_MAPPED)),
 		K(global_page_state(NR_SLAB)),
include/linux/mmzone.h +1 -0
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -55,6 +55,7 @@ enum zone_stat_item {
 	NR_SLAB,	/* Pages used by slab allocator */
 	NR_PAGETABLE,	/* used for pagetables */
 	NR_FILE_DIRTY,
+	NR_WRITEBACK,
 	NR_VM_ZONE_STAT_ITEMS };
 
 struct per_cpu_pages {
include/linux/page-flags.h +4 -4
Index: linux-2.6.17-mm1/include/linux/page-flags.h
===================================================================
@@ -164,7 +164,7 @@
 	do {								\
 		if (!test_and_set_bit(PG_writeback,			\
 				&(page)->flags))			\
-			inc_page_state(nr_writeback);			\
+			inc_zone_page_state(page, NR_WRITEBACK);	\
 	} while (0)
 #define TestSetPageWriteback(page)					\
 	({								\
@@ -172,14 +172,14 @@
 		ret = test_and_set_bit(PG_writeback,			\
 					&(page)->flags);		\
 		if (!ret)						\
-			inc_page_state(nr_writeback);			\
+			inc_zone_page_state(page, NR_WRITEBACK);	\
 		ret;							\
 	})
 #define ClearPageWriteback(page)					\
 	do {								\
 		if (test_and_clear_bit(PG_writeback,			\
 				&(page)->flags))			\
-			dec_page_state(nr_writeback);			\
+			dec_zone_page_state(page, NR_WRITEBACK);	\
 	} while (0)
 #define TestClearPageWriteback(page)					\
 	({								\
@@ -187,7 +187,7 @@
 		ret = test_and_clear_bit(PG_writeback,			\
 				&(page)->flags);			\
 		if (ret)						\
-			dec_page_state(nr_writeback);			\
+			dec_zone_page_state(page, NR_WRITEBACK);	\
 		ret;							\
 	})
 
mm/page_alloc.c +1 -1
Index: linux-2.6.17-mm1/mm/page_alloc.c
===================================================================
@@ -1310,7 +1310,7 @@ void show_free_areas(void)
 		active,
 		inactive,
 		global_page_state(NR_FILE_DIRTY),
-		ps.nr_writeback,
+		global_page_state(NR_WRITEBACK),
 		ps.nr_unstable,
 		nr_free_pages(),
 		global_page_state(NR_SLAB),
mm/page-writeback.c +1 -1
Index: linux-2.6.17-mm1/mm/page-writeback.c
===================================================================
@@ -113,7 +113,7 @@ static void get_writeback_state(struct w
 	wbs->nr_unstable = read_page_state(nr_unstable);
 	wbs->nr_mapped = global_page_state(NR_FILE_MAPPED) +
 				global_page_state(NR_ANON_PAGES);
-	wbs->nr_writeback = read_page_state(nr_writeback);
+	wbs->nr_writeback = global_page_state(NR_WRITEBACK);
 }
 
 /*
mm/vmstat.c +1 -1
Index: linux-2.6.17-mm1/mm/vmstat.c
===================================================================
@@ -463,9 +463,9 @@ static char *vmstat_text[] = {
 	"nr_slab",
 	"nr_page_table_pages",
 	"nr_dirty",
+	"nr_writeback",
 
 	/* Page state */
-	"nr_writeback",
 	"nr_unstable",
 
 	"pgpgin",

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#25 lore raw

PATCH 12/14 Conversion of nr_unstable to per zone counter

In reply to Christoph Lameter

Conversion of nr_unstable to a per zone counter

We need to do some special modifications to the nfs code since there are multiple cases of disposition and we need to have a page ref for proper accounting.

This converts the last critical page state of the VM and therefore we need to remove several functions that were depending on GET_PAGE_STATE_LAST in order to make the kernel compile again. We are only left with event type counters in page state.

drivers/base/node.c +2 -2
Index: linux-2.6.17-mm1/drivers/base/node.c
===================================================================
@@ -39,13 +39,11 @@ static ssize_t node_read_meminfo(struct 
 	int n;
 	int nid = dev->id;
 	struct sysinfo i;
-	struct page_state ps;
 	unsigned long inactive;
 	unsigned long active;
 	unsigned long free;
 
 	si_meminfo_node(&i, nid);
-	get_page_state_node(&ps, nid);
 	__get_zone_counts(&active, &inactive, &free, NODE_DATA(nid));
 
 
@@ -65,6 +63,7 @@ static ssize_t node_read_meminfo(struct 
 		       "Node %d Mapped:       %8lu kB\n"
 		       "Node %d AnonPages:    %8lu kB\n"
 		       "Node %d PageTables:   %8lu kB\n"
+		       "Node %d NFS Unstable: %8lu kB\n"
 		       "Node %d Slab:         %8lu kB\n",
 		       nid, K(i.totalram),
 		       nid, K(i.freeram),
@@ -81,6 +80,7 @@ static ssize_t node_read_meminfo(struct 
 		       nid, K(node_page_state(nid, NR_FILE_MAPPED)),
 		       nid, K(node_page_state(nid, NR_ANON_PAGES)),
 		       nid, K(node_page_state(nid, NR_PAGETABLE)),
+		       nid, K(node_page_state(nid, NR_UNSTABLE_NFS)),
 		       nid, K(node_page_state(nid, NR_SLAB)));
 	n += hugetlb_report_node_meminfo(nid, buf + n);
 	return n;
fs/fs-writeback.c +1 -1
Index: linux-2.6.17-mm1/fs/fs-writeback.c
===================================================================
@@ -463,7 +463,7 @@ void sync_inodes_sb(struct super_block *
 		.sync_mode	= wait ? WB_SYNC_ALL : WB_SYNC_HOLD,
 	};
 	unsigned long nr_dirty = global_page_state(NR_FILE_DIRTY);
-	unsigned long nr_unstable = read_page_state(nr_unstable);
+	unsigned long nr_unstable = global_page_state(NR_UNSTABLE_NFS);
 
 	wbc.nr_to_write = nr_dirty + nr_unstable +
 			(inodes_stat.nr_inodes - inodes_stat.nr_unused) +
fs/nfs/write.c +3 -4
Index: linux-2.6.17-mm1/fs/nfs/write.c
===================================================================
@@ -529,7 +529,7 @@ nfs_mark_request_commit(struct nfs_page 
 	nfs_list_add_request(req, &nfsi->commit);
 	nfsi->ncommit++;
 	spin_unlock(&nfsi->req_lock);
-	inc_page_state(nr_unstable);
+	inc_zone_page_state(req->wb_page, NR_UNSTABLE_NFS);
 	mark_inode_dirty(inode);
 }
 #endif
@@ -1386,7 +1386,6 @@ static void nfs_commit_done(struct rpc_t
 {
 	struct nfs_write_data	*data = calldata;
 	struct nfs_page		*req;
-	int res = 0;
 
         dprintk("NFS: %4d nfs_commit_done (status %d)\n",
                                 task->tk_pid, task->tk_status);
@@ -1423,10 +1422,10 @@ static void nfs_commit_done(struct rpc_t
 		dprintk(" mismatch\n");
 		nfs_mark_request_dirty(req);
 	next:
+		if (req->wb_page)
+			dec_zone_page_state(req->wb_page, NR_UNSTABLE_NFS);
 		nfs_clear_page_writeback(req);
-		res++;
 	}
-	sub_page_state(nr_unstable,res);
 }
 
 static const struct rpc_call_ops nfs_commit_ops = {
include/linux/mmzone.h +1 -0
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -56,6 +56,7 @@ enum zone_stat_item {
 	NR_PAGETABLE,	/* used for pagetables */
 	NR_FILE_DIRTY,
 	NR_WRITEBACK,
+	NR_UNSTABLE_NFS,	/* NFS unstable pages */
 	NR_VM_ZONE_STAT_ITEMS };
 
 struct per_cpu_pages {
mm/page_alloc.c +1 -3
Index: linux-2.6.17-mm1/mm/page_alloc.c
===================================================================
@@ -1266,7 +1266,6 @@ void si_meminfo_node(struct sysinfo *val
  */
 void show_free_areas(void)
 {
-	struct page_state ps;
 	int cpu, temperature;
 	unsigned long active;
 	unsigned long inactive;
@@ -1298,7 +1297,6 @@ void show_free_areas(void)
 		}
 	}
 
-	get_page_state(&ps);
 	get_zone_counts(&active, &inactive, &free);
 
 	printk("Free pages: %11ukB (%ukB HighMem)\n",
@@ -1311,7 +1309,7 @@ void show_free_areas(void)
 		inactive,
 		global_page_state(NR_FILE_DIRTY),
 		global_page_state(NR_WRITEBACK),
-		ps.nr_unstable,
+		global_page_state(NR_UNSTABLE_NFS),
 		nr_free_pages(),
 		global_page_state(NR_SLAB),
 		global_page_state(NR_FILE_MAPPED),
mm/page-writeback.c +1 -1
Index: linux-2.6.17-mm1/mm/page-writeback.c
===================================================================
@@ -110,7 +110,7 @@ struct writeback_state
 static void get_writeback_state(struct writeback_state *wbs)
 {
 	wbs->nr_dirty = global_page_state(NR_FILE_DIRTY);
-	wbs->nr_unstable = read_page_state(nr_unstable);
+	wbs->nr_unstable = global_page_state(NR_UNSTABLE_NFS);
 	wbs->nr_mapped = global_page_state(NR_FILE_MAPPED) +
 				global_page_state(NR_ANON_PAGES);
 	wbs->nr_writeback = global_page_state(NR_WRITEBACK);
fs/proc/proc_misc.c +2 -2
Index: linux-2.6.17-mm1/fs/proc/proc_misc.c
===================================================================
@@ -120,7 +120,6 @@ static int meminfo_read_proc(char *page,
 {
 	struct sysinfo i;
 	int len;
-	struct page_state ps;
 	unsigned long inactive;
 	unsigned long active;
 	unsigned long free;
@@ -129,7 +128,6 @@ static int meminfo_read_proc(char *page,
 	struct vmalloc_info vmi;
 	long cached;
 
-	get_page_state(&ps);
 	get_zone_counts(&active, &inactive, &free);
 
 /*
@@ -172,6 +170,7 @@ static int meminfo_read_proc(char *page,
 		"Mapped:       %8lu kB\n"
 		"Slab:         %8lu kB\n"
 		"PageTables:   %8lu kB\n"
+		"NFS Unstable: %8lu kB\n"
 		"CommitLimit:  %8lu kB\n"
 		"Committed_AS: %8lu kB\n"
 		"VmallocTotal: %8lu kB\n"
@@ -196,6 +195,7 @@ static int meminfo_read_proc(char *page,
 		K(global_page_state(NR_FILE_MAPPED)),
 		K(global_page_state(NR_SLAB)),
 		K(global_page_state(NR_PAGETABLE)),
+		K(global_page_state(NR_UNSTABLE_NFS)),
 		K(allowed),
 		K(committed),
 		(unsigned long)VMALLOC_TOTAL >> 10,
mm/vmstat.c +1 -2
Index: linux-2.6.17-mm1/mm/vmstat.c
===================================================================
@@ -464,10 +464,9 @@ static char *vmstat_text[] = {
 	"nr_page_table_pages",
 	"nr_dirty",
 	"nr_writeback",
-
-	/* Page state */
 	"nr_unstable",
 
+	/* Event counters */
 	"pgpgin",
 	"pgpgout",
 	"pswpin",
fs/nfs/pagelist.c +1 -0
Index: linux-2.6.17-mm1/fs/nfs/pagelist.c
===================================================================
@@ -154,6 +154,7 @@ void nfs_clear_request(struct nfs_page *
 {
 	struct page *page = req->wb_page;
 	if (page != NULL) {
+		dec_zone_page_state(page, NR_UNSTABLE_NFS);
 		page_cache_release(page);
 		req->wb_page = NULL;
 	}

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#26 lore raw

PATCH 13/14 Conversion of nr_bounce to per zone counter

In reply to Christoph Lameter

Conversion of nr_bounce to a per zone counter

nr_bounce is only used for proc output. So it could be left as an event counter. However, the event counters may not be accurate and nr_bounce is categorizing types of pages in a zone. So we really need this to also be a per zone counter.

drivers/base/node.c +2 -0
Index: linux-2.6.17-mm1/drivers/base/node.c
===================================================================
@@ -64,6 +64,7 @@ static ssize_t node_read_meminfo(struct 
 		       "Node %d AnonPages:    %8lu kB\n"
 		       "Node %d PageTables:   %8lu kB\n"
 		       "Node %d NFS Unstable: %8lu kB\n"
+		       "Node %d Bounce:       %8lu kB\n"
 		       "Node %d Slab:         %8lu kB\n",
 		       nid, K(i.totalram),
 		       nid, K(i.freeram),
@@ -81,6 +82,7 @@ static ssize_t node_read_meminfo(struct 
 		       nid, K(node_page_state(nid, NR_ANON_PAGES)),
 		       nid, K(node_page_state(nid, NR_PAGETABLE)),
 		       nid, K(node_page_state(nid, NR_UNSTABLE_NFS)),
+		       nid, K(node_page_state(nid, NR_BOUNCE)),
 		       nid, K(node_page_state(nid, NR_SLAB)));
 	n += hugetlb_report_node_meminfo(nid, buf + n);
 	return n;
include/linux/mmzone.h +1 -0
Index: linux-2.6.17-mm1/include/linux/mmzone.h
===================================================================
@@ -57,6 +57,7 @@ enum zone_stat_item {
 	NR_FILE_DIRTY,
 	NR_WRITEBACK,
 	NR_UNSTABLE_NFS,	/* NFS unstable pages */
+	NR_BOUNCE,
 	NR_VM_ZONE_STAT_ITEMS };
 
 struct per_cpu_pages {
mm/highmem.c +2 -2
Index: linux-2.6.17-mm1/mm/highmem.c
===================================================================
@@ -316,7 +316,7 @@ static void bounce_end_io(struct bio *bi
 			continue;
 
 		mempool_free(bvec->bv_page, pool);	
-		dec_page_state(nr_bounce);
+		dec_zone_page_state(bvec->bv_page, NR_BOUNCE);
 	}
 
 	bio_endio(bio_orig, bio_orig->bi_size, err);
@@ -397,7 +397,7 @@ static void __blk_queue_bounce(request_q
 		to->bv_page = mempool_alloc(pool, q->bounce_gfp);
 		to->bv_len = from->bv_len;
 		to->bv_offset = from->bv_offset;
-		inc_page_state(nr_bounce);
+		inc_zone_page_state(to->bv_page, NR_BOUNCE);
 
 		if (rw == WRITE) {
 			char *vto, *vfrom;
mm/vmstat.c +1 -1
Index: linux-2.6.17-mm1/mm/vmstat.c
===================================================================
@@ -465,6 +465,7 @@ static char *vmstat_text[] = {
 	"nr_dirty",
 	"nr_writeback",
 	"nr_unstable",
+	"nr_bounce",
 
 	/* Event counters */
 	"pgpgin",
@@ -512,7 +513,6 @@ static char *vmstat_text[] = {
 	"allocstall",
 
 	"pgrotated",
-	"nr_bounce",
 };
 
 /*
fs/proc/proc_misc.c +2 -0
Index: linux-2.6.17-mm1/fs/proc/proc_misc.c
===================================================================
@@ -171,6 +171,7 @@ static int meminfo_read_proc(char *page,
 		"Slab:         %8lu kB\n"
 		"PageTables:   %8lu kB\n"
 		"NFS Unstable: %8lu kB\n"
+		"Bounce:       %8lu kB\n"
 		"CommitLimit:  %8lu kB\n"
 		"Committed_AS: %8lu kB\n"
 		"VmallocTotal: %8lu kB\n"
@@ -196,6 +197,7 @@ static int meminfo_read_proc(char *page,
 		K(global_page_state(NR_SLAB)),
 		K(global_page_state(NR_PAGETABLE)),
 		K(global_page_state(NR_UNSTABLE_NFS)),
+		K(global_page_state(NR_BOUNCE)),
 		K(allowed),
 		K(committed),
 		(unsigned long)VMALLOC_TOTAL >> 10,

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#27 lore raw

PATCH 14/14 Remove useless struct wbs

In reply to Christoph Lameter

Remove writeback state

We can remove some functions now that were needed to calculate the page state for writeback control since these statistics are now directly available.

mm/page-writeback.c +34 -51
Index: linux-2.6.17-mm1/mm/page-writeback.c
===================================================================
@@ -99,23 +99,6 @@ EXPORT_SYMBOL(laptop_mode);
 
 static void background_writeout(unsigned long _min_pages);
 
-struct writeback_state
-{
-	unsigned long nr_dirty;
-	unsigned long nr_unstable;
-	unsigned long nr_mapped;
-	unsigned long nr_writeback;
-};
-
-static void get_writeback_state(struct writeback_state *wbs)
-{
-	wbs->nr_dirty = global_page_state(NR_FILE_DIRTY);
-	wbs->nr_unstable = global_page_state(NR_UNSTABLE_NFS);
-	wbs->nr_mapped = global_page_state(NR_FILE_MAPPED) +
-				global_page_state(NR_ANON_PAGES);
-	wbs->nr_writeback = global_page_state(NR_WRITEBACK);
-}
-
 /*
  * Work out the current dirty-memory clamping and background writeout
  * thresholds.
@@ -134,8 +117,8 @@ static void get_writeback_state(struct w
  * clamping level.
  */
 static void
-get_dirty_limits(struct writeback_state *wbs, long *pbackground, long *pdirty,
-		struct address_space *mapping)
+get_dirty_limits(long *pbackground, long *pdirty,
+					struct address_space *mapping)
 {
 	int background_ratio;		/* Percentages */
 	int dirty_ratio;
@@ -145,8 +128,6 @@ get_dirty_limits(struct writeback_state 
 	unsigned long available_memory = total_pages;
 	struct task_struct *tsk;
 
-	get_writeback_state(wbs);
-
 #ifdef CONFIG_HIGHMEM
 	/*
 	 * If this mapping can only allocate from low memory,
@@ -157,7 +138,9 @@ get_dirty_limits(struct writeback_state 
 #endif
 
 
-	unmapped_ratio = 100 - (wbs->nr_mapped * 100) / total_pages;
+	unmapped_ratio = 100 - ((global_page_state(NR_FILE_MAPPED) +
+				global_page_state(NR_ANON_PAGES)) * 100) /
+					total_pages;
 
 	dirty_ratio = vm_dirty_ratio;
 	if (dirty_ratio > unmapped_ratio / 2)
@@ -190,7 +173,6 @@ get_dirty_limits(struct writeback_state 
  */
 static void balance_dirty_pages(struct address_space *mapping)
 {
-	struct writeback_state wbs;
 	long nr_reclaimable;
 	long background_thresh;
 	long dirty_thresh;
@@ -207,11 +189,12 @@ static void balance_dirty_pages(struct a
 			.nr_to_write	= write_chunk,
 		};
 
-		get_dirty_limits(&wbs, &background_thresh,
-					&dirty_thresh, mapping);
-		nr_reclaimable = wbs.nr_dirty + wbs.nr_unstable;
-		if (nr_reclaimable + wbs.nr_writeback <= dirty_thresh)
-			break;
+		get_dirty_limits(&background_thresh, &dirty_thresh, mapping);
+		nr_reclaimable = global_page_state(NR_FILE_DIRTY) +
+					global_page_state(NR_UNSTABLE_NFS);
+		if (nr_reclaimable + global_page_state(NR_WRITEBACK) <=
+			dirty_thresh)
+				break;
 
 		if (!dirty_exceeded)
 			dirty_exceeded = 1;
@@ -224,11 +207,14 @@ static void balance_dirty_pages(struct a
 		 */
 		if (nr_reclaimable) {
 			writeback_inodes(&wbc);
-			get_dirty_limits(&wbs, &background_thresh,
-					&dirty_thresh, mapping);
-			nr_reclaimable = wbs.nr_dirty + wbs.nr_unstable;
-			if (nr_reclaimable + wbs.nr_writeback <= dirty_thresh)
-				break;
+			get_dirty_limits(&background_thresh,
+					 	&dirty_thresh, mapping);
+			nr_reclaimable = global_page_state(NR_FILE_DIRTY) +
+					global_page_state(NR_UNSTABLE_NFS);
+			if (nr_reclaimable +
+				global_page_state(NR_WRITEBACK)
+					<= dirty_thresh)
+						break;
 			pages_written += write_chunk - wbc.nr_to_write;
 			if (pages_written >= write_chunk)
 				break;		/* We've done our duty */
@@ -236,8 +222,9 @@ static void balance_dirty_pages(struct a
 		blk_congestion_wait(WRITE, HZ/10);
 	}
 
-	if (nr_reclaimable + wbs.nr_writeback <= dirty_thresh && dirty_exceeded)
-		dirty_exceeded = 0;
+	if (nr_reclaimable + global_page_state(NR_WRITEBACK)
+		<= dirty_thresh && dirty_exceeded)
+			dirty_exceeded = 0;
 
 	if (writeback_in_progress(bdi))
 		return;		/* pdflush is already working this queue */
@@ -299,12 +286,11 @@ EXPORT_SYMBOL(balance_dirty_pages_rateli
 
 void throttle_vm_writeout(void)
 {
-	struct writeback_state wbs;
 	long background_thresh;
 	long dirty_thresh;
 
         for ( ; ; ) {
-		get_dirty_limits(&wbs, &background_thresh, &dirty_thresh, NULL);
+		get_dirty_limits(&background_thresh, &dirty_thresh, NULL);
 
                 /*
                  * Boost the allowable dirty threshold a bit for page
@@ -312,8 +298,9 @@ void throttle_vm_writeout(void)
                  */
                 dirty_thresh += dirty_thresh / 10;      /* wheeee... */
 
-                if (wbs.nr_unstable + wbs.nr_writeback <= dirty_thresh)
-                        break;
+                if (global_page_state(NR_UNSTABLE_NFS) +
+			global_page_state(NR_WRITEBACK) <= dirty_thresh)
+                        	break;
                 blk_congestion_wait(WRITE, HZ/10);
         }
 }
@@ -335,12 +322,12 @@ static void background_writeout(unsigned
 	};
 
 	for ( ; ; ) {
-		struct writeback_state wbs;
 		long background_thresh;
 		long dirty_thresh;
 
-		get_dirty_limits(&wbs, &background_thresh, &dirty_thresh, NULL);
-		if (wbs.nr_dirty + wbs.nr_unstable < background_thresh
+		get_dirty_limits(&background_thresh, &dirty_thresh, NULL);
+		if (global_page_state(NR_FILE_DIRTY) +
+			global_page_state(NR_UNSTABLE_NFS) < background_thresh
 				&& min_pages <= 0)
 			break;
 		wbc.encountered_congestion = 0;
@@ -364,12 +351,9 @@ static void background_writeout(unsigned
  */
 int wakeup_pdflush(long nr_pages)
 {
-	if (nr_pages == 0) {
-		struct writeback_state wbs;
-
-		get_writeback_state(&wbs);
-		nr_pages = wbs.nr_dirty + wbs.nr_unstable;
-	}
+	if (nr_pages == 0)
+		nr_pages = global_page_state(NR_FILE_DIRTY) +
+				global_page_state(NR_UNSTABLE_NFS);
 	return pdflush_operation(background_writeout, nr_pages);
 }
 
@@ -400,7 +384,6 @@ static void wb_kupdate(unsigned long arg
 	unsigned long start_jif;
 	unsigned long next_jif;
 	long nr_to_write;
-	struct writeback_state wbs;
 	struct writeback_control wbc = {
 		.bdi		= NULL,
 		.sync_mode	= WB_SYNC_NONE,
@@ -412,11 +395,11 @@ static void wb_kupdate(unsigned long arg
 
 	sync_supers();
 
-	get_writeback_state(&wbs);
 	oldest_jif = jiffies - dirty_expire_interval;
 	start_jif = jiffies;
 	next_jif = start_jif + dirty_writeback_interval;
-	nr_to_write = wbs.nr_dirty + wbs.nr_unstable +
+	nr_to_write = global_page_state(NR_FILE_DIRTY) +
+			global_page_state(NR_UNSTABLE_NFS) +
 			(inodes_stat.nr_inodes - inodes_stat.nr_unused);
 	while (nr_to_write > 0) {
 		wbc.encountered_congestion = 0;

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>