Skip to contentSkip to conversation

Page flags: Cleanup, reorg and introduce 5 new flags

RFC PATCH In review Christoph Lameter 2 participants 13 messages Mar 4, 2008

Subsystems (6)
Memory management x86 MM core Page allocator +2 more subsystems
lore
Share & tools
lore

Christoph Lameter

author
#0 lore raw

cover Page flags: Cleanup, reorg and introduce 5 new flags

A set of patches that attempts to improve page flag handling. First of all a method is introduced to generate the page flag functions using macros. Then the number of page flags used by sparsemem is reduced. All page flag operations will no longer be macros. All flags will use inline function.

Then we add a way to export enum constants to the preprocessor which allows us to get rid of __ZONE_COUNT and use the NR_PAGEFLAGS for the dynamic calculation of actually available page flags for fields.

Optimization of sparsemem vmemmap allows us to avoid the use of page flags for section ids. The newly available flags are taken for various ongoing VM projects.

RFC->V1
- Fix various things as suggested by Mel and Kame-san.
- Avoid the #idef CONFIG_PAGEFLAG_EXTENDED. The new page flags are always available.

author
#1 lore raw

RFC 1/10 Pageflags: Use an enum for the flags

Use an enum to ease the maintenance of page flags. This is going to change the numbering from 0 to 18.

RFC->V1
- Add missing aliases (noticed by Mel)

Signed-off-by Christoph Lameter <[email protected]>
---
 include/linux/page-flags.h |   56 ++++++++++++++++++++-------------------------
 1 file changed, 26 insertions(+), 30 deletions(-)
include/linux/page-flags.h +26 -30
Index: linux-2.6/include/linux/page-flags.h
===================================================================
@@ -67,35 +67,29 @@
  * FLAGS_RESERVED which defines the width of the fields section
  * (see linux/mmzone.h).  New flags must _not_ overlap with this area.
  */
-#define PG_locked	 	 0	/* Page is locked. Don't touch. */
-#define PG_error		 1
-#define PG_referenced		 2
-#define PG_uptodate		 3
-
-#define PG_dirty	 	 4
-#define PG_lru			 5
-#define PG_active		 6
-#define PG_slab			 7	/* slab debug (Suparna wants this) */
-
-#define PG_owner_priv_1		 8	/* Owner use. If pagecache, fs may use*/
-#define PG_arch_1		 9
-#define PG_reserved		10
-#define PG_private		11	/* If pagecache, has fs-private data */
-
-#define PG_writeback		12	/* Page is under writeback */
-#define PG_compound		14	/* Part of a compound page */
-#define PG_swapcache		15	/* Swap page: swp_entry_t in private */
-
-#define PG_mappedtodisk		16	/* Has blocks allocated on-disk */
-#define PG_reclaim		17	/* To be reclaimed asap */
-#define PG_buddy		19	/* Page is free, on buddy lists */
-
-/* PG_readahead is only used for file reads; PG_reclaim is only for writes */
-#define PG_readahead		PG_reclaim /* Reminder to do async read-ahead */
-
-/* PG_owner_priv_1 users should have descriptive aliases */
-#define PG_checked		PG_owner_priv_1 /* Used by some filesystems */
-#define PG_pinned		PG_owner_priv_1	/* Xen pinned pagetable */
+enum pageflags {
+	PG_locked,		/* Page is locked. Don't touch. */
+	PG_error,
+	PG_referenced,
+	PG_uptodate,
+	PG_dirty,
+	PG_lru,
+	PG_active,
+	PG_slab,
+	PG_owner_priv_1,	/* Owner use. If pagecache, fs may use*/
+	PG_checked = PG_owner_priv_1, /* Used by some filesystems */
+	PG_pinned = PG_owner_priv_1, /* Xen pinned pagetable */
+	PG_arch_1,
+	PG_reserved,
+	PG_private,		/* If pagecache, has fs-private data */
+	PG_writeback,		/* Page is under writeback */
+	PG_compound,		/* A compound page */
+	PG_swapcache,		/* Swap page: swp_entry_t in private */
+	PG_mappedtodisk,	/* Has blocks allocated on-disk */
+	PG_reclaim,		/* To be reclaimed asap */
+	/* PG_readahead is only used for file reads; PG_reclaim is only for writes */
+	PG_readahead = PG_reclaim, /* Reminder to do async read-ahead */
+	PG_buddy,		/* Page is free, on buddy lists */
 
 #if (BITS_PER_LONG > 32)
 /*

@@ -105,8 +99,10 @@
  * 64 bit  |           FIELDS             | ??????         FLAGS         |
  *         63                            32                              0
  */
-#define PG_uncached		31	/* Page has been mapped as uncached */
+	PG_uncached = 31,		/* Page has been mapped as uncached */
 #endif
+	NR_PAGEFLAGS
+};
 
 /*
  * Manipulation of page state flags
author
#2 lore raw

RFC 2/10 Pageflags: Introduce macros to generate page flag functions

In reply to Christoph Lameter

Introduce a set of macros that generate functions to handle page flags.

A page flag function group typically starts with either

SETPAGEFLAG(<part of function name>,<part of PG_ flagname>)

to create a set of page flag operations that are atomic. Or

__SETPAGEFLAG(<part of function name>,<part of PG_ flagname)

to create a set of page flag operations that are not atomic.

Then additional operations can be added using the following macros

TESTSCFLAG Create additional atomic test-and-set and
test-and-clear functions

TESTSETFLAG Create additional test and set function
TESTCLEARFLAG Create additional test and clear function
SETPAGEFLAG Create additional atomic set function
CLEARPAGEFLAG Create additional atomic clear function
__TESTPAGEFLAG Create additional non atomic set function
__SETPAGEFLAG Create additional non atomic clear function

Signed-off-by Christoph Lameter <[email protected]>
---
 include/linux/page-flags.h |   41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
include/linux/page-flags.h +41 -0
Index: linux-2.6/include/linux/page-flags.h
===================================================================
@@ -105,6 +105,47 @@ enum pageflags {
 };
 
 /*
+ * Macros to create function definitions for page flags
+ */
+#define TESTPAGEFLAG(uname, lname)					\
+static inline int Page##uname(struct page *page) 			\
+			{ return test_bit(PG_##lname, page); }
+
+#define SETPAGEFLAG(uname, lname)					\
+static inline void SetPage##uname(struct page *page)			\
+			{ set_bit(PG_##lname, page); }
+
+#define CLEARPAGEFLAG(uname, lname)					\
+static inline void ClearPage##uname(struct page *page)			\
+			{ clear_bit(PG_##lname, page); }
+
+#define __SETPAGEFLAG(uname, lname)					\
+static inline void __SetPage##uname(struct page *page)			\
+			{ __set_bit(PG_##lname, page); }
+
+#define __CLEARPAGEFLAG(uname, lname)					\
+static inline void __ClearPage##uname(struct page *page)		\
+			{ __clear_bit(PG_##lname, page); }
+
+#define TESTSETFLAG(uname, lname)					\
+static inline int TestSetPage##uname(struct page *page)			\
+		{ return test_and_set_bit(PG_##lname, &page->flags); }
+
+#define TESTCLEARFLAG(uname, lname)					\
+static inline int TestClearPage##uname(struct page *page)		\
+		{ return test_and_clear_bit(PG_##lname, &page->flags); }
+
+
+#define PAGEFLAG(uname, lname) TESTPAGEFLAG(uname, lname)		\
+	SETPAGEFLAG(uname, lname) CLEARPAGEFLAG(uname, lname)
+
+#define __PAGEFLAG(uname, lname) TESTPAGEFLAG(uname, lname)		\
+	__SETPAGEFLAG(uname, lname)  __CLEARPAGEFLAG(uname, lname)
+
+#define TESTSCFLAG(uname, lname)					\
+	TESTSETFLAG(uname, lname) TESTCLEARFLAG(uname, lname)
+
+/*
  * Manipulation of page state flags
  */
 #define PageLocked(page)		\
author
#3 lore raw

RFC 3/10 Pageflags: Convert to the use of new macros

In reply to Christoph Lameter

Replace explicit definitions of page flags through the use of macros. Significantly reduces the size of the definitions and removes a lot of opportunity for errors. Additonal page flags can typically be generated with a single line.

RFC->V1:
- Use inline function for PageHighMem() (suggested by Mel)

Signed-off-by Christoph Lameter <[email protected]>
---
 include/linux/page-flags.h |  191 +++++++++++++++------------------------------
 1 file changed, 64 insertions(+), 127 deletions(-)
include/linux/page-flags.h +64 -127
Index: linux-2.6/include/linux/page-flags.h
===================================================================
@@ -145,28 +145,54 @@ static inline int TestClearPage##uname(s
 #define TESTSCFLAG(uname, lname)					\
 	TESTSETFLAG(uname, lname) TESTCLEARFLAG(uname, lname)
 
+struct page;	/* forward declaration */
+
+PAGEFLAG(Locked, locked) TESTSCFLAG(Locked, locked)
+PAGEFLAG(Error, error)
+PAGEFLAG(Referenced, referenced) TESTCLEARFLAG(Referenced, referenced)
+PAGEFLAG(Dirty, dirty) TESTSCFLAG(Dirty, dirty) __CLEARPAGEFLAG(Dirty, dirty)
+PAGEFLAG(LRU, lru) __CLEARPAGEFLAG(LRU, lru)
+PAGEFLAG(Active, active) __CLEARPAGEFLAG(Active, active)
+__PAGEFLAG(Slab, slab)
+PAGEFLAG(Checked, checked)		/* Used by some filesystems */
+PAGEFLAG(Pinned, pinned)		/* Xen pinned pagetable */
+PAGEFLAG(Reserved, reserved) __CLEARPAGEFLAG(Reserved, reserved)
+PAGEFLAG(Private, private) __CLEARPAGEFLAG(Private, private)
+	__SETPAGEFLAG(Private, private)
+
 /*
- * Manipulation of page state flags
+ * Only test-and-set exist for PG_writeback.  The unconditional operators are
+ * risky: they bypass page accounting.
  */
-#define PageLocked(page)		\
-		test_bit(PG_locked, &(page)->flags)
-#define SetPageLocked(page)		\
-		set_bit(PG_locked, &(page)->flags)
-#define TestSetPageLocked(page)		\
-		test_and_set_bit(PG_locked, &(page)->flags)
-#define ClearPageLocked(page)		\
-		clear_bit(PG_locked, &(page)->flags)
-#define TestClearPageLocked(page)	\
-		test_and_clear_bit(PG_locked, &(page)->flags)
-
-#define PageError(page)		test_bit(PG_error, &(page)->flags)
-#define SetPageError(page)	set_bit(PG_error, &(page)->flags)
-#define ClearPageError(page)	clear_bit(PG_error, &(page)->flags)
-
-#define PageReferenced(page)	test_bit(PG_referenced, &(page)->flags)
-#define SetPageReferenced(page)	set_bit(PG_referenced, &(page)->flags)
-#define ClearPageReferenced(page)	clear_bit(PG_referenced, &(page)->flags)
-#define TestClearPageReferenced(page) test_and_clear_bit(PG_referenced, &(page)->flags)
+TESTPAGEFLAG(Writeback, writeback) TESTSCFLAG(Writeback, writeback)
+__PAGEFLAG(Buddy, buddy)
+PAGEFLAG(MappedToDisk, mappedtodisk)
+
+/* PG_readahead is only used for file reads; PG_reclaim is only for writes */
+PAGEFLAG(Reclaim, reclaim) TESTCLEARFLAG(Reclaim, reclaim)
+PAGEFLAG(Readahead, readahead)		/* Reminder to do async read-ahead */
+
+static inline int PageHighMem(struct page *page)
+{
+#ifdef CONFIG_HIGHMEM
+	return is_highmem(page_zone(page);
+#else
+	return 0; /* needed to optimize away at compile time */
+#endif
+}
+
+#ifdef CONFIG_SWAP
+PAGEFLAG(SwapCache, swapcache)
+#else
+static inline int PageSwapCache(struct page *page)
+{
+	return 0;
+}
+#endif
+
+#if (BITS_PER_LONG > 32)
+PAGEFLAG(Uncached, uncached)
+#endif
 
 static inline int PageUptodate(struct page *page)
 {

@@ -214,97 +240,37 @@ static inline void SetPageUptodate(struc
 #endif
 }
 
-#define ClearPageUptodate(page)	clear_bit(PG_uptodate, &(page)->flags)
+CLEARPAGEFLAG(Uptodate, uptodate)
 
-#define PageDirty(page)		test_bit(PG_dirty, &(page)->flags)
-#define SetPageDirty(page)	set_bit(PG_dirty, &(page)->flags)
-#define TestSetPageDirty(page)	test_and_set_bit(PG_dirty, &(page)->flags)
-#define ClearPageDirty(page)	clear_bit(PG_dirty, &(page)->flags)
-#define __ClearPageDirty(page)	__clear_bit(PG_dirty, &(page)->flags)
-#define TestClearPageDirty(page) test_and_clear_bit(PG_dirty, &(page)->flags)
-
-#define PageLRU(page)		test_bit(PG_lru, &(page)->flags)
-#define SetPageLRU(page)	set_bit(PG_lru, &(page)->flags)
-#define ClearPageLRU(page)	clear_bit(PG_lru, &(page)->flags)
-#define __ClearPageLRU(page)	__clear_bit(PG_lru, &(page)->flags)
-
-#define PageActive(page)	test_bit(PG_active, &(page)->flags)
-#define SetPageActive(page)	set_bit(PG_active, &(page)->flags)
-#define ClearPageActive(page)	clear_bit(PG_active, &(page)->flags)
-#define __ClearPageActive(page)	__clear_bit(PG_active, &(page)->flags)
-
-#define PageSlab(page)		test_bit(PG_slab, &(page)->flags)
-#define __SetPageSlab(page)	__set_bit(PG_slab, &(page)->flags)
-#define __ClearPageSlab(page)	__clear_bit(PG_slab, &(page)->flags)
+extern void cancel_dirty_page(struct page *page, unsigned int account_size);
 
-#ifdef CONFIG_HIGHMEM
-#define PageHighMem(page)	is_highmem(page_zone(page))
-#else
-#define PageHighMem(page)	0 /* needed to optimize away at compile time */
-#endif
+int test_clear_page_writeback(struct page *page);
+int test_set_page_writeback(struct page *page);
 
-#define PageChecked(page)	test_bit(PG_checked, &(page)->flags)
-#define SetPageChecked(page)	set_bit(PG_checked, &(page)->flags)
-#define ClearPageChecked(page)	clear_bit(PG_checked, &(page)->flags)
-
-#define PagePinned(page)	test_bit(PG_pinned, &(page)->flags)
-#define SetPagePinned(page)	set_bit(PG_pinned, &(page)->flags)
-#define ClearPagePinned(page)	clear_bit(PG_pinned, &(page)->flags)
-
-#define PageReserved(page)	test_bit(PG_reserved, &(page)->flags)
-#define SetPageReserved(page)	set_bit(PG_reserved, &(page)->flags)
-#define ClearPageReserved(page)	clear_bit(PG_reserved, &(page)->flags)
-#define __ClearPageReserved(page)	__clear_bit(PG_reserved, &(page)->flags)
-
-#define SetPagePrivate(page)	set_bit(PG_private, &(page)->flags)
-#define ClearPagePrivate(page)	clear_bit(PG_private, &(page)->flags)
-#define PagePrivate(page)	test_bit(PG_private, &(page)->flags)
-#define __SetPagePrivate(page)  __set_bit(PG_private, &(page)->flags)
-#define __ClearPagePrivate(page) __clear_bit(PG_private, &(page)->flags)
+static inline void set_page_writeback(struct page *page)
+{
+	test_set_page_writeback(page);
+}
 
-/*
- * Only test-and-set exist for PG_writeback.  The unconditional operators are
- * risky: they bypass page accounting.
- */
-#define PageWriteback(page)	test_bit(PG_writeback, &(page)->flags)
-#define TestSetPageWriteback(page) test_and_set_bit(PG_writeback,	\
-							&(page)->flags)
-#define TestClearPageWriteback(page) test_and_clear_bit(PG_writeback,	\
-							&(page)->flags)
-
-#define PageBuddy(page)		test_bit(PG_buddy, &(page)->flags)
-#define __SetPageBuddy(page)	__set_bit(PG_buddy, &(page)->flags)
-#define __ClearPageBuddy(page)	__clear_bit(PG_buddy, &(page)->flags)
-
-#define PageMappedToDisk(page)	test_bit(PG_mappedtodisk, &(page)->flags)
-#define SetPageMappedToDisk(page) set_bit(PG_mappedtodisk, &(page)->flags)
-#define ClearPageMappedToDisk(page) clear_bit(PG_mappedtodisk, &(page)->flags)
-
-#define PageReadahead(page)	test_bit(PG_readahead, &(page)->flags)
-#define SetPageReadahead(page)	set_bit(PG_readahead, &(page)->flags)
-#define ClearPageReadahead(page) clear_bit(PG_readahead, &(page)->flags)
-
-#define PageReclaim(page)	test_bit(PG_reclaim, &(page)->flags)
-#define SetPageReclaim(page)	set_bit(PG_reclaim, &(page)->flags)
-#define ClearPageReclaim(page)	clear_bit(PG_reclaim, &(page)->flags)
-#define TestClearPageReclaim(page) test_and_clear_bit(PG_reclaim, &(page)->flags)
-
-#define PageCompound(page)	test_bit(PG_compound, &(page)->flags)
-#define __SetPageCompound(page)	__set_bit(PG_compound, &(page)->flags)
-#define __ClearPageCompound(page) __clear_bit(PG_compound, &(page)->flags)
+TESTPAGEFLAG(Compound, compound)
+__PAGEFLAG(Head, compound)
 
 /*
  * PG_reclaim is used in combination with PG_compound to mark the
- * head and tail of a compound page
+ * head and tail of a compound page. This saves one page flag
+ * but makes it impossible to use compound pages for the page cache.
+ * The PG_reclaim bit would have to be used for reclaim or readahead
+ * if compound pages enter the page cache.
  *
  * PG_compound & PG_reclaim	=> Tail page
  * PG_compound & ~PG_reclaim	=> Head page
  */
-
 #define PG_head_tail_mask ((1L << PG_compound) | (1L << PG_reclaim))
 
-#define PageTail(page)	(((page)->flags & PG_head_tail_mask)	\
-				== PG_head_tail_mask)
+static inline int PageTail(struct page *page)
+{
+	return ((page->flags & PG_head_tail_mask) == PG_head_tail_mask);
+}
 
 static inline void __SetPageTail(struct page *page)
 {

@@ -316,33 +282,4 @@ static inline void __ClearPageTail(struc
 	page->flags &= ~PG_head_tail_mask;
 }
 
-#define PageHead(page)	(((page)->flags & PG_head_tail_mask)	\
-				== (1L << PG_compound))
-#define __SetPageHead(page)	__SetPageCompound(page)
-#define __ClearPageHead(page)	__ClearPageCompound(page)
-
-#ifdef CONFIG_SWAP
-#define PageSwapCache(page)	test_bit(PG_swapcache, &(page)->flags)
-#define SetPageSwapCache(page)	set_bit(PG_swapcache, &(page)->flags)
-#define ClearPageSwapCache(page) clear_bit(PG_swapcache, &(page)->flags)
-#else
-#define PageSwapCache(page)	0
-#endif
-
-#define PageUncached(page)	test_bit(PG_uncached, &(page)->flags)
-#define SetPageUncached(page)	set_bit(PG_uncached, &(page)->flags)
-#define ClearPageUncached(page)	clear_bit(PG_uncached, &(page)->flags)
-
-struct page;	/* forward declaration */
-
-extern void cancel_dirty_page(struct page *page, unsigned int account_size);
-
-int test_clear_page_writeback(struct page *page);
-int test_set_page_writeback(struct page *page);
-
-static inline void set_page_writeback(struct page *page)
-{
-	test_set_page_writeback(page);
-}
-
 #endif	/* PAGE_FLAGS_H */
author
#4 lore raw

RFC 4/10 Pageflags: Use proper page flag functions in Xen

In reply to Christoph Lameter

Xen uses bitops to manipulate page flags. Make it use proper page flag functions.

Signed-off-by Christoph Lameter <[email protected]>
---
 arch/x86/xen/mmu.c         |    4 ++--
 include/linux/page-flags.h |    2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/xen/mmu.c +2 -2
Index: linux-2.6/arch/x86/xen/mmu.c
===================================================================
@@ -425,7 +425,7 @@ static void xen_do_pin(unsigned level, u
 
 static int pin_page(struct page *page, enum pt_level level)
 {
-	unsigned pgfl = test_and_set_bit(PG_pinned, &page->flags);
+	unsigned pgfl = TestSetPagePinned(page);
 	int flush;
 
 	if (pgfl)

@@ -506,7 +506,7 @@ void __init xen_mark_init_mm_pinned(void
 
 static int unpin_page(struct page *page, enum pt_level level)
 {
-	unsigned pgfl = test_and_clear_bit(PG_pinned, &page->flags);
+	unsigned pgfl = TestClearPagePinned(page);
 
 	if (pgfl && !PageHighMem(page)) {
 		void *pt = lowmem_page_address(page);
include/linux/page-flags.h +1 -1
Index: linux-2.6/include/linux/page-flags.h
===================================================================
@@ -155,7 +155,7 @@ PAGEFLAG(LRU, lru) __CLEARPAGEFLAG(LRU, 
 PAGEFLAG(Active, active) __CLEARPAGEFLAG(Active, active)
 __PAGEFLAG(Slab, slab)
 PAGEFLAG(Checked, checked)		/* Used by some filesystems */
-PAGEFLAG(Pinned, pinned)		/* Xen pinned pagetable */
+PAGEFLAG(Pinned, pinned) TESTSCFLAG(Pinned, pinned) /* Xen pagetable */
 PAGEFLAG(Reserved, reserved) __CLEARPAGEFLAG(Reserved, reserved)
 PAGEFLAG(Private, private) __CLEARPAGEFLAG(Private, private)
 	__SETPAGEFLAG(Private, private)
author
#5 lore raw

RFC 5/10 Pageflags: Eliminate PG_xxx aliases

In reply to Christoph Lameter

Remove aliases of PG_xxx. We can easily drop those now and alias by specifying the PG_xxx flag in the macro that generates the functions.

Signed-off-by Christoph Lameter <[email protected]>
---
 include/linux/page-flags.h |   10 +++-------
 mm/page_alloc.c            |    2 +-
 2 files changed, 4 insertions(+), 8 deletions(-)
mm/page_alloc.c +1 -1
Index: linux-2.6/mm/page_alloc.c
===================================================================
@@ -623,7 +623,7 @@ static int prep_new_page(struct page *pa
 	if (PageReserved(page))
 		return 1;
 
-	page->flags &= ~(1 << PG_uptodate | 1 << PG_error | 1 << PG_readahead |
+	page->flags &= ~(1 << PG_uptodate | 1 << PG_error | 1 << PG_reclaim |
 			1 << PG_referenced | 1 << PG_arch_1 |
 			1 << PG_owner_priv_1 | 1 << PG_mappedtodisk);
 	set_page_private(page, 0);
include/linux/page-flags.h +3 -7
Index: linux-2.6/include/linux/page-flags.h
===================================================================
@@ -77,8 +77,6 @@ enum pageflags {
 	PG_active,
 	PG_slab,
 	PG_owner_priv_1,	/* Owner use. If pagecache, fs may use*/
-	PG_checked = PG_owner_priv_1, /* Used by some filesystems */
-	PG_pinned = PG_owner_priv_1, /* Xen pinned pagetable */
 	PG_arch_1,
 	PG_reserved,
 	PG_private,		/* If pagecache, has fs-private data */

@@ -87,8 +85,6 @@ enum pageflags {
 	PG_swapcache,		/* Swap page: swp_entry_t in private */
 	PG_mappedtodisk,	/* Has blocks allocated on-disk */
 	PG_reclaim,		/* To be reclaimed asap */
-	/* PG_readahead is only used for file reads; PG_reclaim is only for writes */
-	PG_readahead = PG_reclaim, /* Reminder to do async read-ahead */
 	PG_buddy,		/* Page is free, on buddy lists */
 
 #if (BITS_PER_LONG > 32)

@@ -154,8 +150,8 @@ PAGEFLAG(Dirty, dirty) TESTSCFLAG(Dirty,
 PAGEFLAG(LRU, lru) __CLEARPAGEFLAG(LRU, lru)
 PAGEFLAG(Active, active) __CLEARPAGEFLAG(Active, active)
 __PAGEFLAG(Slab, slab)
-PAGEFLAG(Checked, checked)		/* Used by some filesystems */
-PAGEFLAG(Pinned, pinned) TESTSCFLAG(Pinned, pinned) /* Xen pagetable */
+PAGEFLAG(Checked, owner_priv_1)		/* Used by some filesystems */
+PAGEFLAG(Pinned, owner_priv_1) TESTSCFLAG(Pinned, owner_priv_1) /* Xen */
 PAGEFLAG(Reserved, reserved) __CLEARPAGEFLAG(Reserved, reserved)
 PAGEFLAG(Private, private) __CLEARPAGEFLAG(Private, private)
 	__SETPAGEFLAG(Private, private)

@@ -170,7 +166,7 @@ PAGEFLAG(MappedToDisk, mappedtodisk)
 
 /* PG_readahead is only used for file reads; PG_reclaim is only for writes */
 PAGEFLAG(Reclaim, reclaim) TESTCLEARFLAG(Reclaim, reclaim)
-PAGEFLAG(Readahead, readahead)		/* Reminder to do async read-ahead */
+PAGEFLAG(Readahead, reclaim)		/* Reminder to do async read-ahead */
 
 static inline int PageHighMem(struct page *page)
 {
author
#6 lore raw

RFC 6/10 Sparsemem: Vmemmap does not need section bits

In reply to Christoph Lameter

Sparsemem vmemmap does not need any section bits. This patch has the effect of reducing the number of bits used in page->flags by at least 6.

Signed-off-by Christoph Lameter <[email protected]>
---
 include/linux/mm.h |   11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
include/linux/mm.h +7 -4
Index: linux-2.6/include/linux/mm.h
===================================================================
@@ -390,11 +390,11 @@ static inline void set_compound_order(st
  * we have run out of space and have to fall back to an
  * alternate (slower) way of determining the node.
  *
- *        No sparsemem: |       NODE     | ZONE | ... | FLAGS |
- * with space for node: | SECTION | NODE | ZONE | ... | FLAGS |
- *   no space for node: | SECTION |     ZONE    | ... | FLAGS |
+ * No sparsemem or sparsemem vmemmap: |       NODE     | ZONE | ... | FLAGS |
+ * classic sparse with space for node:| SECTION | NODE | ZONE | ... | FLAGS |
+ * classic sparse no space for node:  | SECTION |     ZONE    | ... | FLAGS |
  */
-#ifdef CONFIG_SPARSEMEM
+#if defined(CONFIG_SPARSEMEM) && !defined(CONFIG_SPARSEMEM_VMEMMAP)
 #define SECTIONS_WIDTH		SECTIONS_SHIFT
 #else
 #define SECTIONS_WIDTH		0

@@ -405,6 +405,9 @@ static inline void set_compound_order(st
 #if SECTIONS_WIDTH+ZONES_WIDTH+NODES_SHIFT <= FLAGS_RESERVED
 #define NODES_WIDTH		NODES_SHIFT
 #else
+#ifdef CONFIG_SPARSEMEM_VMEMMAP
+#error "Vmemmap: No space for nodes field in page flags"
+#endif
 #define NODES_WIDTH		0
 #endif
author
#7 lore raw

RFC 7/10 Kbuild: Create a way to create preprocessor constants from C expressions

In reply to Christoph Lameter

The use of enums create constants that are not available to the preprocessor when building the kernel (f.e. MAX_NR_ZONES).

Arch code already has a way to export constants calculated to the preprocessor through the asm-offsets.c file. Generate something similar for the core kernel through kbuild.

Cc Sam Ravnborg <[email protected]>
Signed-off-by Christoph Lameter <[email protected]>
---
 Kbuild                 |   31 +++++++++++++++++++++++++++++--
 include/linux/bounds.h |   10 ++++++++++
 kernel/bounds.c        |   16 ++++++++++++++++
 3 files changed, 55 insertions(+), 2 deletions(-)
Kbuild +29 -2
Index: linux-2.6/Kbuild
===================================================================
@@ -9,9 +9,10 @@
 #
 
 offsets-file := include/asm-$(SRCARCH)/asm-offsets.h
+bounds := include/linux/bounds.h
 
-always  := $(offsets-file)
-targets := $(offsets-file)
+always  := $(offsets-file) $(bounds)
+targets := $(offsets-file) $(bounds)
 targets += arch/$(SRCARCH)/kernel/asm-offsets.s
 clean-files := $(addprefix $(objtree)/,$(targets))
 

@@ -39,6 +40,23 @@ define cmd_offsets
 	 echo "#endif" ) > $@
 endef
 
+quiet_cmd_bounds = GEN     $@
+define cmd_bounds
+	(set -e; \
+	 echo "#ifndef __LINUX_BOUNDS_H__"; \
+	 echo "#define __LINUX_BOUNDS_H__"; \
+	 echo "/*"; \
+	 echo " * DO NOT MODIFY."; \
+	 echo " *"; \
+	 echo " * This file was generated by Kbuild"; \
+	 echo " *"; \
+	 echo " */"; \
+	 echo ""; \
+	 sed -ne $(sed-y) $<; \
+	 echo ""; \
+	 echo "#endif" ) > $@
+endef
+
 # We use internal kbuild rules to avoid the "is up to date" message from make
 arch/$(SRCARCH)/kernel/asm-offsets.s: arch/$(SRCARCH)/kernel/asm-offsets.c FORCE
 	$(Q)mkdir -p $(dir $@)

@@ -48,6 +66,15 @@ $(obj)/$(offsets-file): arch/$(SRCARCH)/
 	$(Q)mkdir -p $(dir $@)
 	$(call cmd,offsets)
 
+# We use internal kbuild rules to avoid the "is up to date" message from make
+kernel/bounds.s: kernel/bounds.c FORCE
+	$(Q)mkdir -p $(dir $@)
+	$(call if_changed_dep,cc_s_c)
+
+$(obj)/$(bounds): kernel/bounds.s Kbuild
+	$(Q)mkdir -p $(dir $@)
+	$(call cmd,bounds)
+
 #####
 # 2) Check for missing system calls
 #
kernel/bounds.c new file +16 -0
Index: linux-2.6/kernel/bounds.c
===================================================================
@@ -0,0 +1,16 @@
+/*
+ * Generate definitions needed by the preprocessor.
+ * This code generates raw asm output which is post-processed
+ * to extract and format the required data.
+ */
+
+#include <linux/mm.h>
+
+#define DEFINE(sym, val) \
+        asm volatile("\n->" #sym " %0 " #val : : "i" (val))
+
+#define BLANK() asm volatile("\n->" : : )
+
+void foo(void)
+{
+}
include/linux/bounds.h new file +10 -0
Index: linux-2.6/include/linux/bounds.h
===================================================================
@@ -0,0 +1,10 @@
+#ifndef __LINUX_BOUNDS_H__
+#define __LINUX_BOUNDS_H__
+/*
+ * DO NOT MODIFY.
+ *
+ * This file was generated by Kbuild
+ *
+ */
+
+#endif
author
#8 lore raw

RFC 8/10 Pageflags: Get rid of FLAGS_RESERVED

In reply to Christoph Lameter

NR_PAGEFLAGS specifies the number of page flags we are using.

Quoted 1 line

From that we can calculate the number of bits leftover that can

be used for zone, node (and maybe the sections id). There is no need anymore for FLAGS_RESERVED if we use NR_PAGEFLAGS.

Use the new methods to make NR_PAGEFLAGS available via the preprocessor. NR_PAGEFLAGS is used to calculate field boundaries in the page flags fields. These field widths have to be available to the preprocessor.

Signed-off-by Christoph Lameter <[email protected]>
---
 include/linux/bounds.h     |    2 ++
 include/linux/mm.h         |    6 +++---
 include/linux/mmzone.h     |   19 -------------------
 include/linux/page-flags.h |   14 +++++++-------
 kernel/bounds.c            |    1 +
 5 files changed, 13 insertions(+), 29 deletions(-)
include/linux/mm.h +3 -3
Index: linux-2.6/include/linux/mm.h
===================================================================
@@ -402,7 +402,7 @@ static inline void set_compound_order(st
 
 #define ZONES_WIDTH		ZONES_SHIFT
 
-#if SECTIONS_WIDTH+ZONES_WIDTH+NODES_SHIFT <= FLAGS_RESERVED
+#if SECTIONS_WIDTH+ZONES_WIDTH+NODES_SHIFT <= BITS_PER_LONG - NR_PAGEFLAGS
 #define NODES_WIDTH		NODES_SHIFT
 #else
 #ifdef CONFIG_SPARSEMEM_VMEMMAP

@@ -450,8 +450,8 @@ static inline void set_compound_order(st
 
 #define ZONEID_PGSHIFT		(ZONEID_PGOFF * (ZONEID_SHIFT != 0))
 
-#if SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH > FLAGS_RESERVED
-#error SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH > FLAGS_RESERVED
+#if SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH > BITS_PER_LONG - NR_PAGEFLAGS
+#error SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH > BITS_PER_LONG - NR_PAGEFLAGS
 #endif
 
 #define ZONES_MASK		((1UL << ZONES_WIDTH) - 1)
include/linux/mmzone.h +0 -19
Index: linux-2.6/include/linux/mmzone.h
===================================================================
@@ -735,25 +735,6 @@ extern struct zone *next_zone(struct zon
 #include <asm/sparsemem.h>
 #endif
 
-#if BITS_PER_LONG == 32
-/*
- * with 32 bit page->flags field, we reserve 9 bits for node/zone info.
- * there are 4 zones (3 bits) and this leaves 9-3=6 bits for nodes.
- */
-#define FLAGS_RESERVED		9
-
-#elif BITS_PER_LONG == 64
-/*
- * with 64 bit flags field, there's plenty of room.
- */
-#define FLAGS_RESERVED		32
-
-#else
-
-#error BITS_PER_LONG not defined
-
-#endif
-
 #if !defined(CONFIG_HAVE_ARCH_EARLY_PFN_TO_NID) && \
 	!defined(CONFIG_ARCH_POPULATES_NODE_MAP)
 #define early_pfn_to_nid(nid)  (0UL)
include/linux/page-flags.h +7 -7
Index: linux-2.6/include/linux/page-flags.h
===================================================================
@@ -7,6 +7,7 @@
 
 #include <linux/types.h>
 #include <linux/mm_types.h>
+#include <linux/bounds.h>
 
 /*
  * Various page->flags bits:

@@ -59,13 +60,12 @@
  * extends from the high bits downwards.
  *
  *  | FIELD | ... | FLAGS |
- *  N-1     ^             0
- *          (N-FLAGS_RESERVED)
+ *  N-1           ^       0
+ *               (NR_PAGEFLAGS)
  *
- * The fields area is reserved for fields mapping zone, node and SPARSEMEM
- * section.  The boundry between these two areas is defined by
- * FLAGS_RESERVED which defines the width of the fields section
- * (see linux/mmzone.h).  New flags must _not_ overlap with this area.
+ * The fields area is reserved for fields mapping zone, node (for NUMA) and
+ * SPARSEMEM section (for variants of SPARSEMEM that require section ids like
+ * SPARSEMEM_EXTREME with !SPARSEMEM_VMEMMAP).
  */
 enum pageflags {
 	PG_locked,		/* Page is locked. Don't touch. */

@@ -97,7 +97,7 @@ enum pageflags {
  */
 	PG_uncached = 31,		/* Page has been mapped as uncached */
 #endif
-	NR_PAGEFLAGS
+	__NR_PAGEFLAGS
 };
 
 /*
kernel/bounds.c +1 -0
Index: linux-2.6/kernel/bounds.c
===================================================================
@@ -13,4 +13,5 @@
 
 void foo(void)
 {
+	DEFINE(NR_PAGEFLAGS, __NR_PAGEFLAGS);
 }
include/linux/bounds.h +2 -0
Index: linux-2.6/include/linux/bounds.h
===================================================================
@@ -7,4 +7,6 @@
  *
  */
 
+#define NR_PAGEFLAGS 32 /* __NR_PAGEFLAGS	# */
+
 #endif
author
#9 lore raw

RFC 9/10 Get rid of __ZONE_COUNT

In reply to Christoph Lameter

It was used to compensate because MAX_NR_ZONES was not available to the #ifdefs. Export MAX_NR_ZONES via the new mechanism and get rid of __ZONE_COUNT.

Signed-off-by Christoph Lameter <[email protected]>
---
 include/linux/bounds.h |    1 +
 include/linux/mmzone.h |   22 +++++-----------------
 kernel/bounds.c        |    1 +
 3 files changed, 7 insertions(+), 17 deletions(-)
include/linux/mmzone.h +5 -17
Index: linux-2.6/include/linux/mmzone.h
===================================================================
@@ -17,6 +17,7 @@
 #include <linux/pageblock-flags.h>
 #include <asm/atomic.h>
 #include <asm/page.h>
+#include <linux/bounds.h>
 
 /* Free memory management - zoned buddy allocator.  */
 #ifndef CONFIG_FORCE_MAX_ZONEORDER

@@ -177,7 +178,7 @@ enum zone_type {
 	ZONE_HIGHMEM,
 #endif
 	ZONE_MOVABLE,
-	MAX_NR_ZONES
+	__MAX_NR_ZONES
 };
 
 /*

@@ -188,28 +189,15 @@ enum zone_type {
  * match the requested limits. See gfp_zone() in include/linux/gfp.h
  */
 
-/*
- * Count the active zones.  Note that the use of defined(X) outside
- * #if and family is not necessarily defined so ensure we cannot use
- * it later.  Use __ZONE_COUNT to work out how many shift bits we need.
- */
-#define __ZONE_COUNT (			\
-	  defined(CONFIG_ZONE_DMA)	\
-	+ defined(CONFIG_ZONE_DMA32)	\
-	+ 1				\
-	+ defined(CONFIG_HIGHMEM)	\
-	+ 1				\
-)
-#if __ZONE_COUNT < 2
+#if MAX_NR_ZONES < 2
 #define ZONES_SHIFT 0
-#elif __ZONE_COUNT <= 2
+#elif MAX_NR_ZONES <= 2
 #define ZONES_SHIFT 1
-#elif __ZONE_COUNT <= 4
+#elif MAX_NR_ZONES <= 4
 #define ZONES_SHIFT 2
 #else
 #error ZONES_SHIFT -- too many zones configured adjust calculation
 #endif
-#undef __ZONE_COUNT
 
 struct zone {
 	/* Fields commonly accessed by the page allocator */
kernel/bounds.c +1 -0
Index: linux-2.6/kernel/bounds.c
===================================================================
@@ -14,4 +14,5 @@
 void foo(void)
 {
 	DEFINE(NR_PAGEFLAGS, __NR_PAGEFLAGS);
+	DEFINE(MAX_NR_ZONES, __MAX_NR_ZONES);
 }
include/linux/bounds.h +1 -0
Index: linux-2.6/include/linux/bounds.h
===================================================================
@@ -8,5 +8,6 @@
  */
 
 #define NR_PAGEFLAGS 32 /* __NR_PAGEFLAGS	# */
+#define MAX_NR_ZONES 4 /* __MAX_NR_ZONES	# */
 
 #endif
author
#10 lore raw

RFC 10/10 Pageflags: Land grab

In reply to Christoph Lameter

Grab 5 page flags for some upcoming VM projects and convert the Compound page flag handling to use 2 bits (necessary so that page cache flag use does no longer overlap with compound flags).

This makes us use 24 page flags (plus one additional flag for 64bit)

On a 64 bit system 32 bits are used for page flags. Of those we use 25 flags. So 7 flags are still available.

The rest applies only to 32 bit systems:

In non NUMA configurations we need 2 bits for the zoneid. Meaning 30 bits are left. Of those 24 are used for page flags. So 6 flags are still available.

In NUMA configuration these 6 bits could be used for node numbers which would result in the ability to support 64 nodes. However, the highest number of suported nodes on 32 bit is NUMAQ with 16 nodes. This means we need to use only 4 bits. So 2 page flags are still available.

32bit Sparsemem without vmemmap:

The page flags situation becomes very tight. The remaining 6 bits must then be used as section ids. Via a lookup table we can determine the node ids from the section id. So it would work.

However, we would have no page flags left. Any additional page flag will reduce the number of available sparsemem sections to half.

It may be good if we could phase out sparsemem w/o vmemmap for 32 bit systems. It is likely that most of memory is backed by contiguous RAM given currently available memory sizes.

Without the 32bit sparsemem issues we would still have 2 page flags available. Which would be the same situation as before this patchset and the page flag land grab.

Signed-off-by Christoph Lameter <[email protected]>
---
 include/linux/page-flags.h |   41 ++++++++++++++---------------------------
 1 file changed, 14 insertions(+), 27 deletions(-)
include/linux/page-flags.h +14 -27
Index: linux-2.6/include/linux/page-flags.h
===================================================================
@@ -81,11 +81,16 @@ enum pageflags {
 	PG_reserved,
 	PG_private,		/* If pagecache, has fs-private data */
 	PG_writeback,		/* Page is under writeback */
-	PG_compound,		/* A compound page */
 	PG_swapcache,		/* Swap page: swp_entry_t in private */
 	PG_mappedtodisk,	/* Has blocks allocated on-disk */
 	PG_reclaim,		/* To be reclaimed asap */
 	PG_buddy,		/* Page is free, on buddy lists */
+	PG_mlock,		/* Page cannot be swapped out */
+	PG_pin,			/* Page cannot be moved in memory */
+	PG_tail,		/* Tail of a compound page */
+	PG_head,		/* Head of a compound page */
+	PG_vcompound,		/* Compound page is virtually mapped */
+	PG_filebacked,		/* Page is backed by an actual disk (not RAM) */
 
 #if (BITS_PER_LONG > 32)
 /*

@@ -248,34 +253,16 @@ static inline void set_page_writeback(st
 	test_set_page_writeback(page);
 }
 
-TESTPAGEFLAG(Compound, compound)
-__PAGEFLAG(Head, compound)
+__PAGEFLAG(Head, head)
+__PAGEFLAG(Tail, tail)
+__PAGEFLAG(Vcompound, vcompound)
+__PAGEFLAG(Mlock, mlock)
+__PAGEFLAG(Pin, pin)
+__PAGEFLAG(FileBacked, filebacked)
 
-/*
- * PG_reclaim is used in combination with PG_compound to mark the
- * head and tail of a compound page. This saves one page flag
- * but makes it impossible to use compound pages for the page cache.
- * The PG_reclaim bit would have to be used for reclaim or readahead
- * if compound pages enter the page cache.
- *
- * PG_compound & PG_reclaim	=> Tail page
- * PG_compound & ~PG_reclaim	=> Head page
- */
-#define PG_head_tail_mask ((1L << PG_compound) | (1L << PG_reclaim))
-
-static inline int PageTail(struct page *page)
-{
-	return ((page->flags & PG_head_tail_mask) == PG_head_tail_mask);
-}
-
-static inline void __SetPageTail(struct page *page)
-{
-	page->flags |= PG_head_tail_mask;
-}
-
-static inline void __ClearPageTail(struct page *page)
+static inline int PageCompound(struct page *page)
 {
-	page->flags &= ~PG_head_tail_mask;
+	return (page->flags & ((1 << PG_tail) | (1 << PG_head))) != 0;
 }
 
 #endif	/* PAGE_FLAGS_H */
#11 lore raw

Re: [rfc 10/10] Pageflags: Land grab

Quoted 7 lines · Christoph Lameter

The page flags situation becomes very tight. The remaining 6 bits must then be used as section ids. Via a lookup table we can determine the node ids from the section id. So it would work.

However, we would have no page flags left. Any additional page flag will reduce the number of available sparsemem sections to half.

I have an upcoming, soon to be posted, project that needs one page flag on 32bit and two on 64bit. I would request that you leave at least that many over.

-Andi

author
#12 lore raw

Re: [rfc 10/10] Pageflags: Land grab

Quoted 3 lines · Andi Kleen

I have an upcoming, soon to be posted, project that needs one page flag on 32bit and two on 64bit. I would request that you leave at least that many over.

There are 2 left on 32 bit (if one does not use classic sparsemem which x86_64 does not do) and 7 on 64 bit.