split file and anonymous page queues #3

RFC PATCH In review Rik van Riel 4 participants 13 messages Mar 20, 2007 – Mar 21, 2007

Subsystems (15)

Memory management VFS NFS client MM core +11 more subsystems

lore

Share & tools

lore

Rik van Riel

author

2007-03-20 22:08 UTC #0 lore raw

RFC PATCH 1/1 split file and anonymous page queues #3

[ OK, I suck. I edited yesterday's email with the new info, but forgot
to change the attachment to today's patch. Here is today's patch. ]

Split the anonymous and file backed pages out onto their own pageout queues. This we do not unnecessarily churn through lots of anonymous pages when we do not want to swap them out anyway.

This should (with additional tuning) be a great step forward in scalability, allowing Linux to run well on very large systems where scanning through the anonymous memory (on our way to the page cache memory we do want to evict) is slowing systems down significantly.

The file backed queues and anon/swap queues receive different pageout pressure. Basically the larger the fraction of pages on each queue that were recently referenced, the more we let off the pressure, since that queue contains useful data.

For example, if 75% of the pages scanned on the anon/swap queues were referenced, but only 25% of the pages scanned on the file queues were referenced (ignoring used-once pages), we will put 3 times as much pressure on each file page as on each anon page. This is further modified by the /proc/sys/vm/swappiness parameter.

---
This patch has been stress tested and seems to work, but has not been fine tuned or benchmarked yet. For now the swappiness parameter can be used to tweak swap aggressiveness up and down as desired, but in the long run we may want to simply measure IO cost of page cache and anonymous memory and auto-adjust.

Please take this patch for a spin and let me know what goes well and what goes wrong.

More info on the patch can be found on:

http://linux-mm.org/PageReplacementDesign

Signed-off-by Rik van Riel <[email protected]>

Changelog #3:
- Change some whitespace on Andrew's request.
- Use unsigned long, not ULL since the calculations in get_scan_ratio() no longer need numbers that big.
Changelog #2:
- Fix page_anon() to put all the file pages really on the file list.
- Fix get_scan_ratio() to return more stable numbers, by properly keeping track of the scanned anon and file pages.

Signature · 1 line

All Rights Reversed

fs/proc/proc_misc.c renamed from fs/proc/proc_misc.c.vmsplit +35 ~~-31~~

@@ -147,43 +147,47 @@ static int meminfo_read_proc(char *page,
 	 * Tagged format, for easy grepping and expansion.
 	 */
 	len = sprintf(page,
-		"MemTotal:     %8lu kB\n"
-		"MemFree:      %8lu kB\n"
-		"Buffers:      %8lu kB\n"
-		"Cached:       %8lu kB\n"
-		"SwapCached:   %8lu kB\n"
-		"Active:       %8lu kB\n"
-		"Inactive:     %8lu kB\n"
+		"MemTotal:       %8lu kB\n"
+		"MemFree:        %8lu kB\n"
+		"Buffers:        %8lu kB\n"
+		"Cached:         %8lu kB\n"
+		"SwapCached:     %8lu kB\n"
+		"Active(anon):   %8lu kB\n"
+		"Inactive(anon): %8lu kB\n"
+		"Active(file):   %8lu kB\n"
+		"Inactive(file): %8lu kB\n"
 #ifdef CONFIG_HIGHMEM
-		"HighTotal:    %8lu kB\n"
-		"HighFree:     %8lu kB\n"
-		"LowTotal:     %8lu kB\n"
-		"LowFree:      %8lu kB\n"
-#endif
-		"SwapTotal:    %8lu kB\n"
-		"SwapFree:     %8lu kB\n"
-		"Dirty:        %8lu kB\n"
-		"Writeback:    %8lu kB\n"
-		"AnonPages:    %8lu kB\n"
-		"Mapped:       %8lu kB\n"
-		"Slab:         %8lu kB\n"
-		"SReclaimable: %8lu kB\n"
-		"SUnreclaim:   %8lu kB\n"
-		"PageTables:   %8lu kB\n"
-		"NFS_Unstable: %8lu kB\n"
-		"Bounce:       %8lu kB\n"
-		"CommitLimit:  %8lu kB\n"
-		"Committed_AS: %8lu kB\n"
-		"VmallocTotal: %8lu kB\n"
-		"VmallocUsed:  %8lu kB\n"
-		"VmallocChunk: %8lu kB\n",
+		"HighTotal:      %8lu kB\n"
+		"HighFree:       %8lu kB\n"
+		"LowTotal:       %8lu kB\n"
+		"LowFree:        %8lu kB\n"
+#endif
+		"SwapTotal:      %8lu kB\n"
+		"SwapFree:       %8lu kB\n"
+		"Dirty:          %8lu kB\n"
+		"Writeback:      %8lu kB\n"
+		"AnonPages:      %8lu kB\n"
+		"Mapped:         %8lu kB\n"
+		"Slab:           %8lu kB\n"
+		"SReclaimable:   %8lu kB\n"
+		"SUnreclaim:     %8lu kB\n"
+		"PageTables:     %8lu kB\n"
+		"NFS_Unstable:   %8lu kB\n"
+		"Bounce:         %8lu kB\n"
+		"CommitLimit:    %8lu kB\n"
+		"Committed_AS:   %8lu kB\n"
+		"VmallocTotal:   %8lu kB\n"
+		"VmallocUsed:    %8lu kB\n"
+		"VmallocChunk:   %8lu kB\n",
 		K(i.totalram),
 		K(i.freeram),
 		K(i.bufferram),
 		K(cached),
 		K(total_swapcache_pages),
-		K(global_page_state(NR_ACTIVE)),
-		K(global_page_state(NR_INACTIVE)),
+		K(global_page_state(NR_ACTIVE_ANON)),
+		K(global_page_state(NR_INACTIVE_ANON)),
+		K(global_page_state(NR_ACTIVE_FILE)),
+		K(global_page_state(NR_INACTIVE_FILE)),
 #ifdef CONFIG_HIGHMEM
 		K(i.totalhigh),
 		K(i.freehigh),

fs/mpage.c renamed from fs/mpage.c.vmsplit +2 -2

@@ -408,12 +408,12 @@ mpage_readpages(struct address_space *ma
 					&first_logical_block,
 					get_block);
 			if (!pagevec_add(&lru_pvec, page))
-				__pagevec_lru_add(&lru_pvec);
+				__pagevec_lru_add_file(&lru_pvec);
 		} else {
 			page_cache_release(page);
 		}
 	}
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	BUG_ON(!list_empty(pages));
 	if (bio)
 		mpage_bio_submit(READ, bio);

fs/cifs/file.c renamed from fs/cifs/file.c.vmsplit +2 -2

@@ -1746,7 +1746,7 @@ static void cifs_copy_cache_pages(struct
 		SetPageUptodate(page);
 		unlock_page(page);
 		if (!pagevec_add(plru_pvec, page))
-			__pagevec_lru_add(plru_pvec);
+			__pagevec_lru_add_file(plru_pvec);
 		data += PAGE_CACHE_SIZE;
 	}
 	return;

@@ -1880,7 +1880,7 @@ static int cifs_readpages(struct file *f
 		bytes_read = 0;
 	}
 
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 
 /* need to free smb_read_data buf before exit */
 	if (smb_read_data) {

fs/exec.c renamed from fs/exec.c.vmsplit +1 -1

@@ -322,7 +322,7 @@ void install_arg_page(struct vm_area_str
 		goto out;
 	}
 	inc_mm_counter(mm, anon_rss);
-	lru_cache_add_active(page);
+	lru_cache_add_active_anon(page);
 	set_pte_at(mm, address, pte, pte_mkdirty(pte_mkwrite(mk_pte(
 					page, vma->vm_page_prot))));
 	page_add_new_anon_rmap(page, vma, address);

fs/ntfs/file.c renamed from fs/ntfs/file.c.vmsplit +2 -2

@@ -440,7 +440,7 @@ static inline int __ntfs_grab_cache_page
 			pages[nr] = *cached_page;
 			page_cache_get(*cached_page);
 			if (unlikely(!pagevec_add(lru_pvec, *cached_page)))
-				__pagevec_lru_add(lru_pvec);
+				__pagevec_lru_add_file(lru_pvec);
 			*cached_page = NULL;
 		}
 		index++;

@@ -2113,7 +2113,7 @@ err_out:
 						OSYNC_METADATA|OSYNC_DATA);
 		}
   	}
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	ntfs_debug("Done.  Returning %s (written 0x%lx, status %li).",
 			written ? "written" : "status", (unsigned long)written,
 			(long)status);

fs/splice.c renamed from fs/splice.c.vmsplit +1 -1

@@ -598,7 +598,7 @@ static int pipe_to_file(struct pipe_inod
 		page_cache_get(page);
 
 		if (!(buf->flags & PIPE_BUF_FLAG_LRU))
-			lru_cache_add(page);
+			lru_cache_add_file(page);
 	} else {
 find_page:
 		page = find_lock_page(mapping, index);

fs/nfs/dir.c renamed from fs/nfs/dir.c.vmsplit +1 -1

@@ -1556,7 +1556,7 @@ static int nfs_symlink(struct inode *dir
 	if (!add_to_page_cache(page, dentry->d_inode->i_mapping, 0,
 							GFP_KERNEL)) {
 		pagevec_add(&lru_pvec, page);
-		pagevec_lru_add(&lru_pvec);
+		pagevec_lru_add_file(&lru_pvec);
 		SetPageUptodate(page);
 		unlock_page(page);
 	} else

fs/ramfs/file-nommu.c renamed from fs/ramfs/file-nommu.c.vmsplit +2 -2

@@ -112,12 +112,12 @@ static int ramfs_nommu_expand_for_mappin
 			goto add_error;
 
 		if (!pagevec_add(&lru_pvec, page))
-			__pagevec_lru_add(&lru_pvec);
+			__pagevec_lru_add_anon(&lru_pvec);
 
 		unlock_page(page);
 	}
 
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_anon(&lru_pvec);
 	return 0;
 
  fsize_exceeded:

drivers/base/node.c renamed from drivers/base/node.c.vmsplit +26 ~~-22~~

59 lines not on this page · raw message

mm/memory.c renamed from mm/memory.c.vmsplit +3 -3

@@ -1650,7 +1650,7 @@ gotten:
 		ptep_clear_flush(vma, address, page_table);
 		set_pte_at(mm, address, page_table, entry);
 		update_mmu_cache(vma, address, entry);
-		lru_cache_add_active(new_page);
+		lru_cache_add_active_anon(new_page);
 		page_add_new_anon_rmap(new_page, vma, address);
 
 		/* Free the old page.. */

@@ -2147,7 +2147,7 @@ static int do_anonymous_page(struct mm_s
 		if (!pte_none(*page_table))
 			goto release;
 		inc_mm_counter(mm, anon_rss);
-		lru_cache_add_active(page);
+		lru_cache_add_active_anon(page);
 		page_add_new_anon_rmap(page, vma, address);
 	} else {
 		/* Map the ZERO_PAGE - vm_page_prot is readonly */

@@ -2294,7 +2294,7 @@ retry:
 		set_pte_at(mm, address, page_table, entry);
 		if (anon) {
 			inc_mm_counter(mm, anon_rss);
-			lru_cache_add_active(new_page);
+			lru_cache_add_active_anon(new_page);
 			page_add_new_anon_rmap(new_page, vma, address);
 		} else {
 			inc_mm_counter(mm, file_rss);

mm/page_alloc.c renamed from mm/page_alloc.c.vmsplit +24 ~~-11~~

@@ -1571,10 +1571,13 @@ void show_free_areas(void)
 		}
 	}
 
-	printk("Active:%lu inactive:%lu dirty:%lu writeback:%lu unstable:%lu\n"
+	printk("Active_anon:%lu active_file:%lu inactive_anon%lu\n"
+		" inactive_file:%lu dirty:%lu writeback:%lu unstable:%lu\n"
 		" free:%lu slab:%lu mapped:%lu pagetables:%lu bounce:%lu\n",
-		global_page_state(NR_ACTIVE),
-		global_page_state(NR_INACTIVE),
+		global_page_state(NR_ACTIVE_ANON),
+		global_page_state(NR_ACTIVE_FILE),
+		global_page_state(NR_INACTIVE_ANON),
+		global_page_state(NR_INACTIVE_FILE),
 		global_page_state(NR_FILE_DIRTY),
 		global_page_state(NR_WRITEBACK),
 		global_page_state(NR_UNSTABLE_NFS),

@@ -1597,8 +1600,10 @@ void show_free_areas(void)
 			" min:%lukB"
 			" low:%lukB"
 			" high:%lukB"
-			" active:%lukB"
-			" inactive:%lukB"
+			" active_anon:%lukB"
+			" inactive_anon:%lukB"
+			" active_file:%lukB"
+			" inactive_file:%lukB"
 			" present:%lukB"
 			" pages_scanned:%lu"
 			" all_unreclaimable? %s"

@@ -1608,8 +1613,10 @@ void show_free_areas(void)
 			K(zone->pages_min),
 			K(zone->pages_low),
 			K(zone->pages_high),
-			K(zone_page_state(zone, NR_ACTIVE)),
-			K(zone_page_state(zone, NR_INACTIVE)),
+			K(zone_page_state(zone, NR_ACTIVE_ANON)),
+			K(zone_page_state(zone, NR_INACTIVE_ANON)),
+			K(zone_page_state(zone, NR_ACTIVE_FILE)),
+			K(zone_page_state(zone, NR_INACTIVE_FILE)),
 			K(zone->present_pages),
 			zone->pages_scanned,
 			(zone->all_unreclaimable ? "yes" : "no")

@@ -2671,10 +2678,16 @@ static void __meminit free_area_init_cor
 		zone->prev_priority = DEF_PRIORITY;
 
 		zone_pcp_init(zone);
-		INIT_LIST_HEAD(&zone->active_list);
-		INIT_LIST_HEAD(&zone->inactive_list);
-		zone->nr_scan_active = 0;
-		zone->nr_scan_inactive = 0;
+		INIT_LIST_HEAD(&zone->active_anon_list);
+		INIT_LIST_HEAD(&zone->inactive_anon_list);
+		INIT_LIST_HEAD(&zone->active_file_list);
+		INIT_LIST_HEAD(&zone->inactive_file_list);
+		zone->nr_scan_active_anon = 0;
+		zone->nr_scan_inactive_anon = 0;
+		zone->nr_scan_active_file = 0;
+		zone->nr_scan_inactive_file = 0;
+		zone->recent_rotated_anon = 0;
+		zone->recent_rotated_file = 0;
 		zap_zone_vm_stats(zone);
 		atomic_set(&zone->reclaim_in_progress, 0);
 		if (!size)

mm/swap.c renamed from mm/swap.c.vmsplit +110 ~~-22~~

195 lines not on this page · raw message

mm/migrate.c renamed from mm/migrate.c.vmsplit +23 -6

@@ -53,10 +53,21 @@ int isolate_lru_page(struct page *page, 
 			ret = 0;
 			get_page(page);
 			ClearPageLRU(page);
-			if (PageActive(page))
-				del_page_from_active_list(zone, page);
-			else
-				del_page_from_inactive_list(zone, page);
+			if (PageActive(page)) {
+				if (page_anon(page)) 
+					del_page_from_active_anon_list(zone,
+									page);
+				else
+					del_page_from_active_file_list(zone,
+									page);
+			} else {
+				if (page_anon(page)) 
+					del_page_from_inactive_anon_list(zone,
+									page);
+				else
+					del_page_from_inactive_file_list(zone,
+									page);
+			}
 			list_add_tail(&page->lru, pagelist);
 		}
 		spin_unlock_irq(&zone->lru_lock);

@@ -89,9 +100,15 @@ static inline void move_to_lru(struct pa
 		 * the PG_active bit is off.
 		 */
 		ClearPageActive(page);
-		lru_cache_add_active(page);
+		if (page_anon(page))
+			lru_cache_add_active_anon(page);
+		else
+			lru_cache_add_active_file(page);
 	} else {
-		lru_cache_add(page);
+		if (page_anon(page))
+			lru_cache_add_anon(page);
+		else
+			lru_cache_add_file(page);
 	}
 	put_page(page);
 }

mm/readahead.c renamed from mm/readahead.c.vmsplit +5 -5

@@ -147,14 +147,14 @@ int read_cache_pages(struct address_spac
 		}
 		ret = filler(data, page);
 		if (!pagevec_add(&lru_pvec, page))
-			__pagevec_lru_add(&lru_pvec);
+			__pagevec_lru_add_file(&lru_pvec);
 		if (ret) {
 			put_pages_list(pages);
 			break;
 		}
 		task_io_account_read(PAGE_CACHE_SIZE);
 	}
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	return ret;
 }

@@ -182,11 +182,11 @@ static int read_pages(struct address_spa
 					page->index, GFP_KERNEL)) {
 			mapping->a_ops->readpage(filp, page);
 			if (!pagevec_add(&lru_pvec, page))
-				__pagevec_lru_add(&lru_pvec);
+				__pagevec_lru_add_file(&lru_pvec);
 		} else
 			page_cache_release(page);
 	}
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	ret = 0;
 out:
 	return ret;

@@ -575,6 +575,6 @@ void handle_ra_miss(struct address_space
  */
 unsigned long max_sane_readahead(unsigned long nr)
 {
-	return min(nr, (node_page_state(numa_node_id(), NR_INACTIVE)
+	return min(nr, (node_page_state(numa_node_id(), NR_INACTIVE_FILE)
 		+ node_page_state(numa_node_id(), NR_FREE_PAGES)) / 2);
 }

mm/filemap.c renamed from mm/filemap.c.vmsplit +3 -3

@@ -462,7 +462,7 @@ int add_to_page_cache_lru(struct page *p
 {
 	int ret = add_to_page_cache(page, mapping, offset, gfp_mask);
 	if (ret == 0)
-		lru_cache_add(page);
+		lru_cache_add_file(page);
 	return ret;
 }

@@ -1836,7 +1836,7 @@ repeat:
 			page = *cached_page;
 			page_cache_get(page);
 			if (!pagevec_add(lru_pvec, page))
-				__pagevec_lru_add(lru_pvec);
+				__pagevec_lru_add_file(lru_pvec);
 			*cached_page = NULL;
 		}
 	}

@@ -2197,7 +2197,7 @@ zero_length_segment:
 	if (unlikely(file->f_flags & O_DIRECT) && written)
 		status = filemap_write_and_wait(mapping);
 
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	return written ? written : status;
 }
 EXPORT_SYMBOL(generic_file_buffered_write);

mm/vmstat.c renamed from mm/vmstat.c.vmsplit +9 -4

@@ -432,8 +432,10 @@ const struct seq_operations fragmentatio
 static const char * const vmstat_text[] = {
 	/* Zoned VM counters */
 	"nr_free_pages",
-	"nr_active",
-	"nr_inactive",
+	"nr_active_anon",
+	"nr_inactive_anon",
+	"nr_active_file",
+	"nr_inactive_file",
 	"nr_anon_pages",
 	"nr_mapped",
 	"nr_file_pages",

@@ -509,7 +511,7 @@ static int zoneinfo_show(struct seq_file
 			   "\n        min      %lu"
 			   "\n        low      %lu"
 			   "\n        high     %lu"
-			   "\n        scanned  %lu (a: %lu i: %lu)"
+			   "\n        scanned  %lu (ao: %lu io: %lu af: %lu if: %lu)"
 			   "\n        spanned  %lu"
 			   "\n        present  %lu",
 			   zone_page_state(zone, NR_FREE_PAGES),

@@ -517,7 +519,10 @@ static int zoneinfo_show(struct seq_file
 			   zone->pages_low,
 			   zone->pages_high,
 			   zone->pages_scanned,
-			   zone->nr_scan_active, zone->nr_scan_inactive,
+			   zone->nr_scan_active_anon,
+			   zone->nr_scan_inactive_anon,
+			   zone->nr_scan_active_file,
+			   zone->nr_scan_inactive_file,
 			   zone->spanned_pages,
 			   zone->present_pages);

mm/shmem.c renamed from mm/shmem.c.vmsplit +2 -2

@@ -176,7 +176,7 @@ static inline void shmem_unacct_blocks(u
 }
 
 static const struct super_operations shmem_ops;
-static const struct address_space_operations shmem_aops;
+const struct address_space_operations shmem_aops;
 static const struct file_operations shmem_file_operations;
 static const struct inode_operations shmem_inode_operations;
 static const struct inode_operations shmem_dir_inode_operations;

@@ -2315,7 +2315,7 @@ static void destroy_inodecache(void)
 	kmem_cache_destroy(shmem_inode_cachep);
 }
 
-static const struct address_space_operations shmem_aops = {
+const struct address_space_operations shmem_aops = {
 	.writepage	= shmem_writepage,
 	.set_page_dirty	= __set_page_dirty_no_writeback,
 #ifdef CONFIG_TMPFS

mm/vmscan.c renamed from mm/vmscan.c.vmsplit +253 ~~-143~~

563 lines not on this page · raw message

mm/swap_state.c renamed from mm/swap_state.c.vmsplit +1 -1

@@ -354,7 +354,7 @@ struct page *read_swap_cache_async(swp_e
 			/*
 			 * Initiate read into locked page and return.
 			 */
-			lru_cache_add_active(new_page);
+			lru_cache_add_anon(new_page);
 			swap_readpage(NULL, new_page);
 			return new_page;
 		}

include/linux/mmzone.h renamed from include/linux/mmzone.h.vmsplit +17 -7

@@ -49,15 +49,17 @@ struct zone_padding {
 enum zone_stat_item {
 	/* First 128 byte cacheline (assuming 64 bit words) */
 	NR_FREE_PAGES,
-	NR_INACTIVE,
-	NR_ACTIVE,
+	NR_ACTIVE_ANON,
+	NR_INACTIVE_ANON,
+	NR_ACTIVE_FILE,
+	NR_INACTIVE_FILE,
 	NR_ANON_PAGES,	/* Mapped anonymous pages */
 	NR_FILE_MAPPED,	/* pagecache pages mapped into pagetables.
 			   only modified from process context */
 	NR_FILE_PAGES,
+	/* Second 128 byte cacheline */
 	NR_FILE_DIRTY,
 	NR_WRITEBACK,
-	/* Second 128 byte cacheline */
 	NR_SLAB_RECLAIMABLE,
 	NR_SLAB_UNRECLAIMABLE,
 	NR_PAGETABLE,		/* used for pagetables */

@@ -215,10 +217,18 @@ struct zone {
 
 	/* Fields commonly accessed by the page reclaim scanner */
 	spinlock_t		lru_lock;	
-	struct list_head	active_list;
-	struct list_head	inactive_list;
-	unsigned long		nr_scan_active;
-	unsigned long		nr_scan_inactive;
+	struct list_head	active_anon_list;
+	struct list_head	inactive_anon_list;
+	struct list_head	active_file_list;
+	struct list_head	inactive_file_list;
+	unsigned long		nr_scan_active_anon;
+	unsigned long		nr_scan_inactive_anon;
+	unsigned long		nr_scan_active_file;
+	unsigned long		nr_scan_inactive_file;
+	unsigned long		recent_rotated_anon;
+	unsigned long		recent_rotated_file;
+	unsigned long		recent_scanned_anon;
+	unsigned long		recent_scanned_file;
 	unsigned long		pages_scanned;	   /* since last reclaim */
 	int			all_unreclaimable; /* All pages pinned */

include/linux/mm.h renamed from include/linux/mm.h.vmsplit +21 -0

@@ -591,6 +591,27 @@ static inline int PageAnon(struct page *
 	return ((unsigned long)page->mapping & PAGE_MAPPING_ANON) != 0;
 }
 
+/* Returns true if this page is anonymous, tmpfs or otherwise swap backed. */
+extern const struct address_space_operations shmem_aops;
+static inline int page_anon(struct page *page)
+{
+	struct address_space *mapping;
+
+	if (PageAnon(page) || PageSwapCache(page))
+		return 1;
+	mapping = page_mapping(page);
+	if (!mapping || !mapping->a_ops)
+		return 0;
+	if (mapping->a_ops == &shmem_aops)
+		return 1;
+	/* Should ramfs pages go onto an mlocked list instead? */
+	if ((unlikely(mapping->a_ops->writepage == NULL && PageDirty(page))))
+		return 1;
+
+	/* The page is page cache backed by a normal filesystem. */
+	return 0;
+}
+
 /*
  * Return the pagecache index of the passed page.  Regular pagecache pages
  * use ->index whereas swapcache pages use ->private

include/linux/mm_inline.h renamed from include/linux/mm_inline.h.vmsplit +46 ~~-12~~

84 lines not on this page · raw message

include/linux/pagevec.h renamed from include/linux/pagevec.h.vmsplit +12 -4

@@ -23,8 +23,10 @@ struct pagevec {
 void __pagevec_release(struct pagevec *pvec);
 void __pagevec_release_nonlru(struct pagevec *pvec);
 void __pagevec_free(struct pagevec *pvec);
-void __pagevec_lru_add(struct pagevec *pvec);
-void __pagevec_lru_add_active(struct pagevec *pvec);
+void __pagevec_lru_add_file(struct pagevec *pvec);
+void __pagevec_lru_add_active_file(struct pagevec *pvec);
+void __pagevec_lru_add_anon(struct pagevec *pvec);
+void __pagevec_lru_add_active_anon(struct pagevec *pvec);
 void pagevec_strip(struct pagevec *pvec);
 void pagevec_swap_free(struct pagevec *pvec);
 unsigned pagevec_lookup(struct pagevec *pvec, struct address_space *mapping,

@@ -82,10 +84,16 @@ static inline void pagevec_free(struct p
 		__pagevec_free(pvec);
 }
 
-static inline void pagevec_lru_add(struct pagevec *pvec)
+static inline void pagevec_lru_add_file(struct pagevec *pvec)
 {
 	if (pagevec_count(pvec))
-		__pagevec_lru_add(pvec);
+		__pagevec_lru_add_file(pvec);
+}
+
+static inline void pagevec_lru_add_anon(struct pagevec *pvec)
+{
+	if (pagevec_count(pvec))
+		__pagevec_lru_add_anon(pvec);
 }
 
 #endif /* _LINUX_PAGEVEC_H */

include/linux/swap.h renamed from include/linux/swap.h.vmsplit +4 -2

@@ -178,8 +178,10 @@ extern unsigned int nr_free_pagecache_pa
 
 
 /* linux/mm/swap.c */
-extern void FASTCALL(lru_cache_add(struct page *));
-extern void FASTCALL(lru_cache_add_active(struct page *));
+extern void FASTCALL(lru_cache_add_file(struct page *));
+extern void FASTCALL(lru_cache_add_anon(struct page *));
+extern void FASTCALL(lru_cache_add_active_file(struct page *));
+extern void FASTCALL(lru_cache_add_active_anon(struct page *));
 extern void FASTCALL(activate_page(struct page *));
 extern void FASTCALL(mark_page_accessed(struct page *));
 extern void lru_add_drain(void);

Matt Mackall

2007-03-21 01:07 UTC #1 lore raw

Quoted 2 lines · Rik van Riel

fs/proc/proc_misc.c renamed from fs/proc/proc_misc.c.vmsplit +0 -2
-		"Active:       %8lu kB\n"
-		"Inactive:     %8lu kB\n"

...

fs/proc/proc_misc.c static int meminfo_read_proc(char *page, · line 158

Quoted 4 lines

fs/proc/proc_misc.c renamed from fs/proc/proc_misc.c.vmsplit +4 -0

+		"Active(anon):   %8lu kB\n"
+		"Inactive(anon): %8lu kB\n"
+		"Active(file):   %8lu kB\n"
+		"Inactive(file): %8lu kB\n"

Potentially incompatible change. How about preserving the original fields (by totalling), then adding the other fields in a second patch.

fs/mpage.c mpage_readpages(struct address_space *ma · line 411

Quoted 3 lines

fs/mpage.c renamed from fs/mpage.c.vmsplit +1 -1

 			if (!pagevec_add(&lru_pvec, page))
-				__pagevec_lru_add(&lru_pvec);
+				__pagevec_lru_add_file(&lru_pvec);

Wouldn't lru_file_add or file_lru_add be a better name? If the object is a "file lru" then sticking "add" in the middle is a little ugly.

Quoted 5 lines

mm/swap.c renamed from mm/swap.c.vmsplit +2 -1

 	spin_lock_irq(&zone->lru_lock);
 	if (PageLRU(page) && !PageActive(page)) {
-		del_page_from_inactive_list(zone, page);
+	if (page_anon(page)) {
+		del_page_from_inactive_anon_list(zone,page);

mm/swap.c void fastcall activate_page(struct page · line 159

Quoted 10 lines

mm/swap.c renamed from mm/swap.c.vmsplit +6 -1

 		SetPageActive(page);
-		add_page_to_active_list(zone, page);
+		add_page_to_active_anon_list(zone, page);
+	} else {
+		del_page_from_inactive_file_list(zone, page);
+		SetPageActive(page);
+		add_page_to_active_file_list(zone, page);
+	}
 		__count_vm_event(PGACTIVATE);
 	}

Missing a level of indentation.

Signature · 7 lines

Mathematics is the supreme nostalgia of our time.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Rik van Riel

author

2007-03-21 19:11 UTC #2 lore raw

Quoted 11 lines · Matt Mackall

Quoted 2 lines · Rik van Riel
(quoted diff) +0 -2
-		"Active:       %8lu kB\n"
-		"Inactive:     %8lu kB\n"
...
Review
Quoted 4 lines
(quoted diff) +4 -0
+		"Active(anon):   %8lu kB\n"
+		"Inactive(anon): %8lu kB\n"
+		"Active(file):   %8lu kB\n"
+		"Inactive(file): %8lu kB\n"
Potentially incompatible change. How about preserving the original fields (by totalling), then adding the other fields in a second patch.

Fixed in the attached patch.

Quoted 6 lines

Review
Quoted 3 lines
(quoted diff) +1 -1
 			if (!pagevec_add(&lru_pvec, page))
-				__pagevec_lru_add(&lru_pvec);
+				__pagevec_lru_add_file(&lru_pvec);
Wouldn't lru_file_add or file_lru_add be a better name? If the object is a "file lru" then sticking "add" in the middle is a little ugly.

Not sure about this one. Does anybody else have an opinion here?

Quoted 9 lines

(quoted diff) +4 -2

 	spin_lock_irq(&zone->lru_lock);
 	if (PageLRU(page) && !PageActive(page)) {
-		del_page_from_inactive_list(zone, page);
+	if (page_anon(page)) {
+		del_page_from_inactive_anon_list(zone,page);
 		SetPageActive(page);
-		add_page_to_active_list(zone, page);
+		add_page_to_active_anon_list(zone, page);
+	} else {

Quoted 8 lines

Review

Quoted 6 lines

(quoted diff) +4 -0

+		del_page_from_inactive_file_list(zone, page);
+		SetPageActive(page);
+		add_page_to_active_file_list(zone, page);
+	}
 		__count_vm_event(PGACTIVATE);
 	}

Missing a level of indentation.

Fixed.

Signature · 1 line

All Rights Reversed

fs/proc/proc_misc.c renamed from fs/proc/proc_misc.c.vmsplit +41 ~~-31~~

@@ -147,43 +147,53 @@ static int meminfo_read_proc(char *page,
 	 * Tagged format, for easy grepping and expansion.
 	 */
 	len = sprintf(page,
-		"MemTotal:     %8lu kB\n"
-		"MemFree:      %8lu kB\n"
-		"Buffers:      %8lu kB\n"
-		"Cached:       %8lu kB\n"
-		"SwapCached:   %8lu kB\n"
-		"Active:       %8lu kB\n"
-		"Inactive:     %8lu kB\n"
+		"MemTotal:       %8lu kB\n"
+		"MemFree:        %8lu kB\n"
+		"Buffers:        %8lu kB\n"
+		"Cached:         %8lu kB\n"
+		"SwapCached:     %8lu kB\n"
+		"Active:         %8lu kB\n"
+		"Inactive:       %8lu kB\n"
+		"Active(anon):   %8lu kB\n"
+		"Inactive(anon): %8lu kB\n"
+		"Active(file):   %8lu kB\n"
+		"Inactive(file): %8lu kB\n"
 #ifdef CONFIG_HIGHMEM
-		"HighTotal:    %8lu kB\n"
-		"HighFree:     %8lu kB\n"
-		"LowTotal:     %8lu kB\n"
-		"LowFree:      %8lu kB\n"
-#endif
-		"SwapTotal:    %8lu kB\n"
-		"SwapFree:     %8lu kB\n"
-		"Dirty:        %8lu kB\n"
-		"Writeback:    %8lu kB\n"
-		"AnonPages:    %8lu kB\n"
-		"Mapped:       %8lu kB\n"
-		"Slab:         %8lu kB\n"
-		"SReclaimable: %8lu kB\n"
-		"SUnreclaim:   %8lu kB\n"
-		"PageTables:   %8lu kB\n"
-		"NFS_Unstable: %8lu kB\n"
-		"Bounce:       %8lu kB\n"
-		"CommitLimit:  %8lu kB\n"
-		"Committed_AS: %8lu kB\n"
-		"VmallocTotal: %8lu kB\n"
-		"VmallocUsed:  %8lu kB\n"
-		"VmallocChunk: %8lu kB\n",
+		"HighTotal:      %8lu kB\n"
+		"HighFree:       %8lu kB\n"
+		"LowTotal:       %8lu kB\n"
+		"LowFree:        %8lu kB\n"
+#endif
+		"SwapTotal:      %8lu kB\n"
+		"SwapFree:       %8lu kB\n"
+		"Dirty:          %8lu kB\n"
+		"Writeback:      %8lu kB\n"
+		"AnonPages:      %8lu kB\n"
+		"Mapped:         %8lu kB\n"
+		"Slab:           %8lu kB\n"
+		"SReclaimable:   %8lu kB\n"
+		"SUnreclaim:     %8lu kB\n"
+		"PageTables:     %8lu kB\n"
+		"NFS_Unstable:   %8lu kB\n"
+		"Bounce:         %8lu kB\n"
+		"CommitLimit:    %8lu kB\n"
+		"Committed_AS:   %8lu kB\n"
+		"VmallocTotal:   %8lu kB\n"
+		"VmallocUsed:    %8lu kB\n"
+		"VmallocChunk:   %8lu kB\n",
 		K(i.totalram),
 		K(i.freeram),
 		K(i.bufferram),
 		K(cached),
 		K(total_swapcache_pages),
-		K(global_page_state(NR_ACTIVE)),
-		K(global_page_state(NR_INACTIVE)),
+		K(global_page_state(NR_ACTIVE_ANON) +
+				global_page_state(NR_ACTIVE_FILE)),
+		K(global_page_state(NR_INACTIVE_ANON) +
+				global_page_state(NR_INACTIVE_FILE)),
+		K(global_page_state(NR_ACTIVE_ANON)),
+		K(global_page_state(NR_INACTIVE_ANON)),
+		K(global_page_state(NR_ACTIVE_FILE)),
+		K(global_page_state(NR_INACTIVE_FILE)),
 #ifdef CONFIG_HIGHMEM
 		K(i.totalhigh),
 		K(i.freehigh),

fs/mpage.c renamed from fs/mpage.c.vmsplit +2 -2

@@ -408,12 +408,12 @@ mpage_readpages(struct address_space *ma
 					&first_logical_block,
 					get_block);
 			if (!pagevec_add(&lru_pvec, page))
-				__pagevec_lru_add(&lru_pvec);
+				__pagevec_lru_add_file(&lru_pvec);
 		} else {
 			page_cache_release(page);
 		}
 	}
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	BUG_ON(!list_empty(pages));
 	if (bio)
 		mpage_bio_submit(READ, bio);

fs/cifs/file.c renamed from fs/cifs/file.c.vmsplit +2 -2

@@ -1746,7 +1746,7 @@ static void cifs_copy_cache_pages(struct
 		SetPageUptodate(page);
 		unlock_page(page);
 		if (!pagevec_add(plru_pvec, page))
-			__pagevec_lru_add(plru_pvec);
+			__pagevec_lru_add_file(plru_pvec);
 		data += PAGE_CACHE_SIZE;
 	}
 	return;

@@ -1880,7 +1880,7 @@ static int cifs_readpages(struct file *f
 		bytes_read = 0;
 	}
 
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 
 /* need to free smb_read_data buf before exit */
 	if (smb_read_data) {

fs/exec.c renamed from fs/exec.c.vmsplit +1 -1

@@ -322,7 +322,7 @@ void install_arg_page(struct vm_area_str
 		goto out;
 	}
 	inc_mm_counter(mm, anon_rss);
-	lru_cache_add_active(page);
+	lru_cache_add_active_anon(page);
 	set_pte_at(mm, address, pte, pte_mkdirty(pte_mkwrite(mk_pte(
 					page, vma->vm_page_prot))));
 	page_add_new_anon_rmap(page, vma, address);

fs/ntfs/file.c renamed from fs/ntfs/file.c.vmsplit +2 -2

@@ -440,7 +440,7 @@ static inline int __ntfs_grab_cache_page
 			pages[nr] = *cached_page;
 			page_cache_get(*cached_page);
 			if (unlikely(!pagevec_add(lru_pvec, *cached_page)))
-				__pagevec_lru_add(lru_pvec);
+				__pagevec_lru_add_file(lru_pvec);
 			*cached_page = NULL;
 		}
 		index++;

@@ -2113,7 +2113,7 @@ err_out:
 						OSYNC_METADATA|OSYNC_DATA);
 		}
   	}
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	ntfs_debug("Done.  Returning %s (written 0x%lx, status %li).",
 			written ? "written" : "status", (unsigned long)written,
 			(long)status);

fs/splice.c renamed from fs/splice.c.vmsplit +1 -1

@@ -598,7 +598,7 @@ static int pipe_to_file(struct pipe_inod
 		page_cache_get(page);
 
 		if (!(buf->flags & PIPE_BUF_FLAG_LRU))
-			lru_cache_add(page);
+			lru_cache_add_file(page);
 	} else {
 find_page:
 		page = find_lock_page(mapping, index);

fs/nfs/dir.c renamed from fs/nfs/dir.c.vmsplit +1 -1

@@ -1556,7 +1556,7 @@ static int nfs_symlink(struct inode *dir
 	if (!add_to_page_cache(page, dentry->d_inode->i_mapping, 0,
 							GFP_KERNEL)) {
 		pagevec_add(&lru_pvec, page);
-		pagevec_lru_add(&lru_pvec);
+		pagevec_lru_add_file(&lru_pvec);
 		SetPageUptodate(page);
 		unlock_page(page);
 	} else

fs/ramfs/file-nommu.c renamed from fs/ramfs/file-nommu.c.vmsplit +2 -2

@@ -112,12 +112,12 @@ static int ramfs_nommu_expand_for_mappin
 			goto add_error;
 
 		if (!pagevec_add(&lru_pvec, page))
-			__pagevec_lru_add(&lru_pvec);
+			__pagevec_lru_add_anon(&lru_pvec);
 
 		unlock_page(page);
 	}
 
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_anon(&lru_pvec);
 	return 0;
 
  fsize_exceeded:

drivers/base/node.c renamed from drivers/base/node.c.vmsplit +26 ~~-22~~

59 lines not on this page · raw message

mm/memory.c renamed from mm/memory.c.vmsplit +3 -3

@@ -1650,7 +1650,7 @@ gotten:
 		ptep_clear_flush(vma, address, page_table);
 		set_pte_at(mm, address, page_table, entry);
 		update_mmu_cache(vma, address, entry);
-		lru_cache_add_active(new_page);
+		lru_cache_add_active_anon(new_page);
 		page_add_new_anon_rmap(new_page, vma, address);
 
 		/* Free the old page.. */

@@ -2147,7 +2147,7 @@ static int do_anonymous_page(struct mm_s
 		if (!pte_none(*page_table))
 			goto release;
 		inc_mm_counter(mm, anon_rss);
-		lru_cache_add_active(page);
+		lru_cache_add_active_anon(page);
 		page_add_new_anon_rmap(page, vma, address);
 	} else {
 		/* Map the ZERO_PAGE - vm_page_prot is readonly */

@@ -2294,7 +2294,7 @@ retry:
 		set_pte_at(mm, address, page_table, entry);
 		if (anon) {
 			inc_mm_counter(mm, anon_rss);
-			lru_cache_add_active(new_page);
+			lru_cache_add_active_anon(new_page);
 			page_add_new_anon_rmap(new_page, vma, address);
 		} else {
 			inc_mm_counter(mm, file_rss);

mm/page_alloc.c renamed from mm/page_alloc.c.vmsplit +24 ~~-11~~

@@ -1571,10 +1571,13 @@ void show_free_areas(void)
 		}
 	}
 
-	printk("Active:%lu inactive:%lu dirty:%lu writeback:%lu unstable:%lu\n"
+	printk("Active_anon:%lu active_file:%lu inactive_anon%lu\n"
+		" inactive_file:%lu dirty:%lu writeback:%lu unstable:%lu\n"
 		" free:%lu slab:%lu mapped:%lu pagetables:%lu bounce:%lu\n",
-		global_page_state(NR_ACTIVE),
-		global_page_state(NR_INACTIVE),
+		global_page_state(NR_ACTIVE_ANON),
+		global_page_state(NR_ACTIVE_FILE),
+		global_page_state(NR_INACTIVE_ANON),
+		global_page_state(NR_INACTIVE_FILE),
 		global_page_state(NR_FILE_DIRTY),
 		global_page_state(NR_WRITEBACK),
 		global_page_state(NR_UNSTABLE_NFS),

@@ -1597,8 +1600,10 @@ void show_free_areas(void)
 			" min:%lukB"
 			" low:%lukB"
 			" high:%lukB"
-			" active:%lukB"
-			" inactive:%lukB"
+			" active_anon:%lukB"
+			" inactive_anon:%lukB"
+			" active_file:%lukB"
+			" inactive_file:%lukB"
 			" present:%lukB"
 			" pages_scanned:%lu"
 			" all_unreclaimable? %s"

@@ -1608,8 +1613,10 @@ void show_free_areas(void)
 			K(zone->pages_min),
 			K(zone->pages_low),
 			K(zone->pages_high),
-			K(zone_page_state(zone, NR_ACTIVE)),
-			K(zone_page_state(zone, NR_INACTIVE)),
+			K(zone_page_state(zone, NR_ACTIVE_ANON)),
+			K(zone_page_state(zone, NR_INACTIVE_ANON)),
+			K(zone_page_state(zone, NR_ACTIVE_FILE)),
+			K(zone_page_state(zone, NR_INACTIVE_FILE)),
 			K(zone->present_pages),
 			zone->pages_scanned,
 			(zone->all_unreclaimable ? "yes" : "no")

@@ -2671,10 +2678,16 @@ static void __meminit free_area_init_cor
 		zone->prev_priority = DEF_PRIORITY;
 
 		zone_pcp_init(zone);
-		INIT_LIST_HEAD(&zone->active_list);
-		INIT_LIST_HEAD(&zone->inactive_list);
-		zone->nr_scan_active = 0;
-		zone->nr_scan_inactive = 0;
+		INIT_LIST_HEAD(&zone->active_anon_list);
+		INIT_LIST_HEAD(&zone->inactive_anon_list);
+		INIT_LIST_HEAD(&zone->active_file_list);
+		INIT_LIST_HEAD(&zone->inactive_file_list);
+		zone->nr_scan_active_anon = 0;
+		zone->nr_scan_inactive_anon = 0;
+		zone->nr_scan_active_file = 0;
+		zone->nr_scan_inactive_file = 0;
+		zone->recent_rotated_anon = 0;
+		zone->recent_rotated_file = 0;
 		zap_zone_vm_stats(zone);
 		atomic_set(&zone->reclaim_in_progress, 0);
 		if (!size)

mm/swap.c renamed from mm/swap.c.vmsplit +111 ~~-23~~

196 lines not on this page · raw message

mm/migrate.c renamed from mm/migrate.c.vmsplit +23 -6

@@ -53,10 +53,21 @@ int isolate_lru_page(struct page *page, 
 			ret = 0;
 			get_page(page);
 			ClearPageLRU(page);
-			if (PageActive(page))
-				del_page_from_active_list(zone, page);
-			else
-				del_page_from_inactive_list(zone, page);
+			if (PageActive(page)) {
+				if (page_anon(page)) 
+					del_page_from_active_anon_list(zone,
+									page);
+				else
+					del_page_from_active_file_list(zone,
+									page);
+			} else {
+				if (page_anon(page)) 
+					del_page_from_inactive_anon_list(zone,
+									page);
+				else
+					del_page_from_inactive_file_list(zone,
+									page);
+			}
 			list_add_tail(&page->lru, pagelist);
 		}
 		spin_unlock_irq(&zone->lru_lock);

@@ -89,9 +100,15 @@ static inline void move_to_lru(struct pa
 		 * the PG_active bit is off.
 		 */
 		ClearPageActive(page);
-		lru_cache_add_active(page);
+		if (page_anon(page))
+			lru_cache_add_active_anon(page);
+		else
+			lru_cache_add_active_file(page);
 	} else {
-		lru_cache_add(page);
+		if (page_anon(page))
+			lru_cache_add_anon(page);
+		else
+			lru_cache_add_file(page);
 	}
 	put_page(page);
 }

mm/readahead.c renamed from mm/readahead.c.vmsplit +5 -5

@@ -147,14 +147,14 @@ int read_cache_pages(struct address_spac
 		}
 		ret = filler(data, page);
 		if (!pagevec_add(&lru_pvec, page))
-			__pagevec_lru_add(&lru_pvec);
+			__pagevec_lru_add_file(&lru_pvec);
 		if (ret) {
 			put_pages_list(pages);
 			break;
 		}
 		task_io_account_read(PAGE_CACHE_SIZE);
 	}
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	return ret;
 }

@@ -182,11 +182,11 @@ static int read_pages(struct address_spa
 					page->index, GFP_KERNEL)) {
 			mapping->a_ops->readpage(filp, page);
 			if (!pagevec_add(&lru_pvec, page))
-				__pagevec_lru_add(&lru_pvec);
+				__pagevec_lru_add_file(&lru_pvec);
 		} else
 			page_cache_release(page);
 	}
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	ret = 0;
 out:
 	return ret;

@@ -575,6 +575,6 @@ void handle_ra_miss(struct address_space
  */
 unsigned long max_sane_readahead(unsigned long nr)
 {
-	return min(nr, (node_page_state(numa_node_id(), NR_INACTIVE)
+	return min(nr, (node_page_state(numa_node_id(), NR_INACTIVE_FILE)
 		+ node_page_state(numa_node_id(), NR_FREE_PAGES)) / 2);
 }

mm/filemap.c renamed from mm/filemap.c.vmsplit +3 -3

@@ -462,7 +462,7 @@ int add_to_page_cache_lru(struct page *p
 {
 	int ret = add_to_page_cache(page, mapping, offset, gfp_mask);
 	if (ret == 0)
-		lru_cache_add(page);
+		lru_cache_add_file(page);
 	return ret;
 }

@@ -1836,7 +1836,7 @@ repeat:
 			page = *cached_page;
 			page_cache_get(page);
 			if (!pagevec_add(lru_pvec, page))
-				__pagevec_lru_add(lru_pvec);
+				__pagevec_lru_add_file(lru_pvec);
 			*cached_page = NULL;
 		}
 	}

@@ -2197,7 +2197,7 @@ zero_length_segment:
 	if (unlikely(file->f_flags & O_DIRECT) && written)
 		status = filemap_write_and_wait(mapping);
 
-	pagevec_lru_add(&lru_pvec);
+	pagevec_lru_add_file(&lru_pvec);
 	return written ? written : status;
 }
 EXPORT_SYMBOL(generic_file_buffered_write);

mm/vmstat.c renamed from mm/vmstat.c.vmsplit +9 -4

@@ -432,8 +432,10 @@ const struct seq_operations fragmentatio
 static const char * const vmstat_text[] = {
 	/* Zoned VM counters */
 	"nr_free_pages",
-	"nr_active",
-	"nr_inactive",
+	"nr_active_anon",
+	"nr_inactive_anon",
+	"nr_active_file",
+	"nr_inactive_file",
 	"nr_anon_pages",
 	"nr_mapped",
 	"nr_file_pages",

@@ -509,7 +511,7 @@ static int zoneinfo_show(struct seq_file
 			   "\n        min      %lu"
 			   "\n        low      %lu"
 			   "\n        high     %lu"
-			   "\n        scanned  %lu (a: %lu i: %lu)"
+			   "\n        scanned  %lu (ao: %lu io: %lu af: %lu if: %lu)"
 			   "\n        spanned  %lu"
 			   "\n        present  %lu",
 			   zone_page_state(zone, NR_FREE_PAGES),

@@ -517,7 +519,10 @@ static int zoneinfo_show(struct seq_file
 			   zone->pages_low,
 			   zone->pages_high,
 			   zone->pages_scanned,
-			   zone->nr_scan_active, zone->nr_scan_inactive,
+			   zone->nr_scan_active_anon,
+			   zone->nr_scan_inactive_anon,
+			   zone->nr_scan_active_file,
+			   zone->nr_scan_inactive_file,
 			   zone->spanned_pages,
 			   zone->present_pages);

mm/shmem.c renamed from mm/shmem.c.vmsplit +2 -2

@@ -176,7 +176,7 @@ static inline void shmem_unacct_blocks(u
 }
 
 static const struct super_operations shmem_ops;
-static const struct address_space_operations shmem_aops;
+const struct address_space_operations shmem_aops;
 static const struct file_operations shmem_file_operations;
 static const struct inode_operations shmem_inode_operations;
 static const struct inode_operations shmem_dir_inode_operations;

@@ -2315,7 +2315,7 @@ static void destroy_inodecache(void)
 	kmem_cache_destroy(shmem_inode_cachep);
 }
 
-static const struct address_space_operations shmem_aops = {
+const struct address_space_operations shmem_aops = {
 	.writepage	= shmem_writepage,
 	.set_page_dirty	= __set_page_dirty_no_writeback,
 #ifdef CONFIG_TMPFS

mm/vmscan.c renamed from mm/vmscan.c.vmsplit +253 ~~-143~~

563 lines not on this page · raw message

mm/swap_state.c renamed from mm/swap_state.c.vmsplit +1 -1

@@ -354,7 +354,7 @@ struct page *read_swap_cache_async(swp_e
 			/*
 			 * Initiate read into locked page and return.
 			 */
-			lru_cache_add_active(new_page);
+			lru_cache_add_anon(new_page);
 			swap_readpage(NULL, new_page);
 			return new_page;
 		}

include/linux/mmzone.h renamed from include/linux/mmzone.h.vmsplit +17 -7

@@ -49,15 +49,17 @@ struct zone_padding {
 enum zone_stat_item {
 	/* First 128 byte cacheline (assuming 64 bit words) */
 	NR_FREE_PAGES,
-	NR_INACTIVE,
-	NR_ACTIVE,
+	NR_ACTIVE_ANON,
+	NR_INACTIVE_ANON,
+	NR_ACTIVE_FILE,
+	NR_INACTIVE_FILE,
 	NR_ANON_PAGES,	/* Mapped anonymous pages */
 	NR_FILE_MAPPED,	/* pagecache pages mapped into pagetables.
 			   only modified from process context */
 	NR_FILE_PAGES,
+	/* Second 128 byte cacheline */
 	NR_FILE_DIRTY,
 	NR_WRITEBACK,
-	/* Second 128 byte cacheline */
 	NR_SLAB_RECLAIMABLE,
 	NR_SLAB_UNRECLAIMABLE,
 	NR_PAGETABLE,		/* used for pagetables */

@@ -215,10 +217,18 @@ struct zone {
 
 	/* Fields commonly accessed by the page reclaim scanner */
 	spinlock_t		lru_lock;	
-	struct list_head	active_list;
-	struct list_head	inactive_list;
-	unsigned long		nr_scan_active;
-	unsigned long		nr_scan_inactive;
+	struct list_head	active_anon_list;
+	struct list_head	inactive_anon_list;
+	struct list_head	active_file_list;
+	struct list_head	inactive_file_list;
+	unsigned long		nr_scan_active_anon;
+	unsigned long		nr_scan_inactive_anon;
+	unsigned long		nr_scan_active_file;
+	unsigned long		nr_scan_inactive_file;
+	unsigned long		recent_rotated_anon;
+	unsigned long		recent_rotated_file;
+	unsigned long		recent_scanned_anon;
+	unsigned long		recent_scanned_file;
 	unsigned long		pages_scanned;	   /* since last reclaim */
 	int			all_unreclaimable; /* All pages pinned */

include/linux/mm.h renamed from include/linux/mm.h.vmsplit +21 -0

@@ -591,6 +591,27 @@ static inline int PageAnon(struct page *
 	return ((unsigned long)page->mapping & PAGE_MAPPING_ANON) != 0;
 }
 
+/* Returns true if this page is anonymous, tmpfs or otherwise swap backed. */
+extern const struct address_space_operations shmem_aops;
+static inline int page_anon(struct page *page)
+{
+	struct address_space *mapping;
+
+	if (PageAnon(page) || PageSwapCache(page))
+		return 1;
+	mapping = page_mapping(page);
+	if (!mapping || !mapping->a_ops)
+		return 0;
+	if (mapping->a_ops == &shmem_aops)
+		return 1;
+	/* Should ramfs pages go onto an mlocked list instead? */
+	if ((unlikely(mapping->a_ops->writepage == NULL && PageDirty(page))))
+		return 1;
+
+	/* The page is page cache backed by a normal filesystem. */
+	return 0;
+}
+
 /*
  * Return the pagecache index of the passed page.  Regular pagecache pages
  * use ->index whereas swapcache pages use ->private

include/linux/mm_inline.h renamed from include/linux/mm_inline.h.vmsplit +46 ~~-12~~

84 lines not on this page · raw message

include/linux/pagevec.h renamed from include/linux/pagevec.h.vmsplit +12 -4

@@ -23,8 +23,10 @@ struct pagevec {
 void __pagevec_release(struct pagevec *pvec);
 void __pagevec_release_nonlru(struct pagevec *pvec);
 void __pagevec_free(struct pagevec *pvec);
-void __pagevec_lru_add(struct pagevec *pvec);
-void __pagevec_lru_add_active(struct pagevec *pvec);
+void __pagevec_lru_add_file(struct pagevec *pvec);
+void __pagevec_lru_add_active_file(struct pagevec *pvec);
+void __pagevec_lru_add_anon(struct pagevec *pvec);
+void __pagevec_lru_add_active_anon(struct pagevec *pvec);
 void pagevec_strip(struct pagevec *pvec);
 void pagevec_swap_free(struct pagevec *pvec);
 unsigned pagevec_lookup(struct pagevec *pvec, struct address_space *mapping,

@@ -82,10 +84,16 @@ static inline void pagevec_free(struct p
 		__pagevec_free(pvec);
 }
 
-static inline void pagevec_lru_add(struct pagevec *pvec)
+static inline void pagevec_lru_add_file(struct pagevec *pvec)
 {
 	if (pagevec_count(pvec))
-		__pagevec_lru_add(pvec);
+		__pagevec_lru_add_file(pvec);
+}
+
+static inline void pagevec_lru_add_anon(struct pagevec *pvec)
+{
+	if (pagevec_count(pvec))
+		__pagevec_lru_add_anon(pvec);
 }
 
 #endif /* _LINUX_PAGEVEC_H */

include/linux/swap.h renamed from include/linux/swap.h.vmsplit +4 -2

@@ -178,8 +178,10 @@ extern unsigned int nr_free_pagecache_pa
 
 
 /* linux/mm/swap.c */
-extern void FASTCALL(lru_cache_add(struct page *));
-extern void FASTCALL(lru_cache_add_active(struct page *));
+extern void FASTCALL(lru_cache_add_file(struct page *));
+extern void FASTCALL(lru_cache_add_anon(struct page *));
+extern void FASTCALL(lru_cache_add_active_file(struct page *));
+extern void FASTCALL(lru_cache_add_active_anon(struct page *));
 extern void FASTCALL(activate_page(struct page *));
 extern void FASTCALL(mark_page_accessed(struct page *));
 extern void lru_add_drain(void);

Nikita Danilov

2007-03-21 08:44 UTC #3 lore raw

In reply to Rik van Riel

Quoted 6 lines · Rik van Riel

[ OK, I suck. I edited yesterday's email with the new info, but forgot
to change the attachment to today's patch. Here is today's patch. ]
Split the anonymous and file backed pages out onto their own pageout queues. This we do not unnecessarily churn through lots of anonymous pages when we do not want to swap them out anyway.

Won't this re-introduce problems similar to ones due to split inactive_clean/inactive_dirty queues we had in the past?

For example, by rotating anon queues faster than file queues, kernel would end up reclaiming anon pages that are hotter (in "absolute" LRU order) than some file pages.

Nikita.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Rik van Riel

author

2007-03-21 11:24 UTC #4 lore raw

Quoted 8 lines · Nikita Danilov

Quoted 6 lines · Rik van Riel
[ OK, I suck. I edited yesterday's email with the new info, but forgot
to change the attachment to today's patch. Here is today's patch. ]
Split the anonymous and file backed pages out onto their own pageout queues. This we do not unnecessarily churn through lots of anonymous pages when we do not want to swap them out anyway.

Quoted 6 lines

Won't this re-introduce problems similar to ones due to split inactive_clean/inactive_dirty queues we had in the past?
For example, by rotating anon queues faster than file queues, kernel would end up reclaiming anon pages that are hotter (in "absolute" LRU order) than some file pages.

That is why we check the fraction of referenced pages in each queue. Please look at the get_scan_ratio() and shrink_zone() code in my patch.

Signature · 9 lines

Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Nikita Danilov

2007-03-21 11:48 UTC #5 lore raw

Quoted 12 lines · Rik van Riel

Quoted 11 lines · Nikita Danilov
Quoted 6 lines · Rik van Riel
[ OK, I suck. I edited yesterday's email with the new info, but forgot
to change the attachment to today's patch. Here is today's patch. ]
Split the anonymous and file backed pages out onto their own pageout queues. This we do not unnecessarily churn through lots of anonymous pages when we do not want to swap them out anyway.
Won't this re-introduce problems similar to ones due to split inactive_clean/inactive_dirty queues we had in the past?

Quoted 7 lines

Quoted 3 lines
For example, by rotating anon queues faster than file queues, kernel would end up reclaiming anon pages that are hotter (in "absolute" LRU order) than some file pages.
That is why we check the fraction of referenced pages in each queue. Please look at the get_scan_ratio() and shrink_zone() code in my patch.

Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases anon_percent, and shrink_zone() starts scanning anon queue more aggressively. As a result, pages spend less time there, and have less chance of ever being accessed, reducing fraction of referenced anon pages further, and triggering further increase in the amount of scanning, etc. Doesn't this introduce positive feed-back loop?

Nikita.

Rik van Riel

author

2007-03-21 12:01 UTC #6 lore raw

Quoted 9 lines · Nikita Danilov

Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases anon_percent, and shrink_zone() starts scanning anon queue more aggressively. As a result, pages spend less time there, and have less chance of ever being accessed, reducing fraction of referenced anon pages further, and triggering further increase in the amount of scanning, etc. Doesn't this introduce positive feed-back loop?

It's a possibility, but I don't think it will be much of an issue in practice.

If it is, we can always use refaults as a correcting mechanism - which would have the added benefit of being able to do streaming IO without putting any pressure on the active list, essentially clock-pro replacement with just some tweaks to shrink_list()...

Signature · 9 lines

Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Rik van Riel

author

2007-03-21 12:03 UTC #7 lore raw

Quoted 12 lines · Rik van Riel

Quoted 9 lines · Nikita Danilov
Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases anon_percent, and shrink_zone() starts scanning anon queue more aggressively. As a result, pages spend less time there, and have less chance of ever being accessed, reducing fraction of referenced anon pages further, and triggering further increase in the amount of scanning, etc. Doesn't this introduce positive feed-back loop?

Quoted 8 lines

It's a possibility, but I don't think it will be much of an issue in practice.
If it is, we can always use refaults as a correcting mechanism - which would have the added benefit of being able to do streaming IO without putting any pressure on the active list, essentially clock-pro replacement with just some tweaks to shrink_list()...

As an aside, due to the use-once algorithm file pages are at a natural disadvantage already. I believe it would be really hard to construct a workload where anon pages suffer the positive feedback loop you describe...

Signature · 9 lines

Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Nikita Danilov

2007-03-21 15:29 UTC #8 lore raw

Quoted 22 lines · Rik van Riel

Quoted 12 lines · Rik van Riel
Quoted 9 lines · Nikita Danilov
Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases anon_percent, and shrink_zone() starts scanning anon queue more aggressively. As a result, pages spend less time there, and have less chance of ever being accessed, reducing fraction of referenced anon pages further, and triggering further increase in the amount of scanning, etc. Doesn't this introduce positive feed-back loop?
Quoted 8 lines
It's a possibility, but I don't think it will be much of an issue in practice.
If it is, we can always use refaults as a correcting mechanism - which would have the added benefit of being able to do streaming IO without putting any pressure on the active list, essentially clock-pro replacement with just some tweaks to shrink_list()...

Quoted 4 lines

As an aside, due to the use-once algorithm file pages are at a natural disadvantage already. I believe it would be really hard to construct a workload where anon pages suffer the positive feedback loop you describe...

That scenario works for file queues too. Of course, all this is but a theoretical speculation at this point, but I am concerned that

- that loop would tend to happen under various border conditions, making it hard to isolate, diagnose, and debug, and

- long before it becomes explicitly visible (say, as an excessive cpu consumption by scanner), it would ruin global lru ordering, degrading overall performance.

Generally speaking, multi-queue replacement mechanisms were tried in the past, and they all suffer from the common drawback: once scanning rate is different for different queues, so is the notion of "hotness", measured by scanner. As a result multi-queue scanner fails to capture working set properly.

Nikita.

Quoted 5 lines

Signature · 4 lines

-- 
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.

Rik van Riel

author

2007-03-21 16:08 UTC #9 lore raw

Quoted 5 lines · Nikita Danilov

Generally speaking, multi-queue replacement mechanisms were tried in the past, and they all suffer from the common drawback: once scanning rate is different for different queues, so is the notion of "hotness", measured by scanner. As a result multi-queue scanner fails to capture working set properly.

You realize that the current "single" queue in the 2.6 kernel has this problem in a much worse way: when swappiness is low and the kernel does not want to reclaim mapped pages, it will randomly rotate those pages around the list.

In addition, the referenced bit on unmapped page cache pages was ignored completely, making it impossible for the VM to separate the page cache working set from transient pages due to streaming IO.

I agree that we should put some more negative feedback in place if it turns out we need it. I have refault code ready that can be plugged into this patch, but I don't want to add the overhead of such code if it turns out we do not actually need it.

Signature · 7 lines

All Rights Reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Nikita Danilov

2007-03-21 22:17 UTC #10 lore raw

Quoted 8 lines · Rik van Riel

Quoted 5 lines · Nikita Danilov
Generally speaking, multi-queue replacement mechanisms were tried in the past, and they all suffer from the common drawback: once scanning rate is different for different queues, so is the notion of "hotness", measured by scanner. As a result multi-queue scanner fails to capture working set properly.

Quoted 4 lines

You realize that the current "single" queue in the 2.6 kernel has this problem in a much worse way: when swappiness is low and the kernel does not want to reclaim mapped pages, it will randomly rotate those pages around the list.

Agree. Some time ago I tried to solve this very problem with dont-rotate-active-list patch
(http://linuxhacker.ru/~nikita/patches/2.6.12-rc6/2005.06.11/vm_03-dont-rotate-active-list.patch),
but it had problems on its own.

Quoted 5 lines

In addition, the referenced bit on unmapped page cache pages was ignored completely, making it impossible for the VM to separate the page cache working set from transient pages due to streaming IO.

Yes, basically FIFO for clean file system pages and FIFO-second-chance for dirty file pages. Very bad.

Quoted 6 lines

I agree that we should put some more negative feedback in place if it turns out we need it. I have refault code ready that can be plugged into this patch, but I don't want to add the overhead of such code if it turns out we do not actually need it.

In my humble opinion VM already has too many mechanisms that are supposed to help in corner cases, but there is little to do with that, except for major rewrite.

Nikita.

Chuck Ebbert

2007-03-21 16:12 UTC #11 lore raw

In reply to Rik van Riel

Quoted 12 lines · Rik van Riel

Quoted 9 lines · Nikita Danilov
Probably I am missing something, but I don't see how that can help. For example, suppose (for simplicity) that we have swappiness of 100%, and that fraction of referenced anon pages gets slightly less than of file pages. get_scan_ratio() increases anon_percent, and shrink_zone() starts scanning anon queue more aggressively. As a result, pages spend less time there, and have less chance of ever being accessed, reducing fraction of referenced anon pages further, and triggering further increase in the amount of scanning, etc. Doesn't this introduce positive feed-back loop?

Quoted 9 lines

It's a possibility, but I don't think it will be much of an issue in practice.
If it is, we can always use refaults as a correcting mechanism - which would have the added benefit of being able to do streaming IO without putting any pressure on the active list, essentially clock-pro replacement with just some tweaks to shrink_list()...

I think you're going to have to use refault rates. AIX 3.5 had to add that. Something like:

if refault_rate(anonymous/mmap) > refault_rate(pagecache)
drop a pagecache page
else
drop either

You do have anonymous memory and mmapped executables in the same queue, right?

Rik van Riel

author

2007-03-21 16:56 UTC #12 lore raw

Quoted 7 lines · Chuck Ebbert

I think you're going to have to use refault rates. AIX 3.5 had to add that. Something like:
if refault_rate(anonymous/mmap) > refault_rate(pagecache)
drop a pagecache page
else
drop either

How about the opposite?

If the page cache refault rate is way higher than the anonymous refault rate, did you favor page cache?

Btw, just a higher fault rate will already make that cache grow faster than the other, simply because it will have more allocations than the other cache and they both get shrunk to some degree...

Quoted 2 lines

You do have anonymous memory and mmapped executables in the same queue, right?

Nope. It is very hard to see the difference between mmapped executables and mmapped data from any structure linked off the struct page...

Signature · 7 lines

All Rights Reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>