Skip to contentSkip to conversation

put_page() tries to handle hugepages but fails

PATCH David Gibson 3 participants 12 messages Apr 23, 2004 – Apr 27, 2004

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

Andrew, please apply.

The code of put_page() is misleading, in that it appears to have code handling PageCompound pages (i.e. hugepages). However it won't actually handle them correctly - __page_cache_release() will not work properly on a compound. Instead, hugepages should be and are released with huge_page_release() from mm/hugetlb.c. This patch removes the broken PageCompound path from put_page(), replacing it with a BUG_ON(). This also removes the initialization of page[1].mapping from compoound pages, which was only ever used in this broken code path.

mm/swap.c +1 -11
Index: working-2.6/mm/swap.c
===================================================================
@@ -38,17 +38,7 @@
 
 void put_page(struct page *page)
 {
-	if (unlikely(PageCompound(page))) {
-		page = (struct page *)page->private;
-		if (put_page_testzero(page)) {
-			if (page[1].mapping) {	/* destructor? */
-				(*(void (*)(struct page *))page[1].mapping)(page);
-			} else {
-				__page_cache_release(page);
-			}
-		}
-		return;
-	}
+	BUG_ON(PageCompound(page));
 	if (!PageReserved(page) && put_page_testzero(page))
 		__page_cache_release(page);
 }
mm/page_alloc.c +0 -1
Index: working-2.6/mm/page_alloc.c
===================================================================
@@ -118,7 +118,6 @@
 	int i;
 	int nr_pages = 1 << order;
 
-	page[1].mapping = 0;
 	page[1].index = order;
 	for (i = 0; i < nr_pages; i++) {
 		struct page *p = page + i;
Signature · 4 lines
David Gibson			| For every complex problem there is a
david AT gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson
#1 lore raw
Quoted 12 lines · David Gibson

Andrew, please apply.

The code of put_page() is misleading, in that it appears to have code handling PageCompound pages (i.e. hugepages). However it won't actually handle them correctly - __page_cache_release() will not work properly on a compound. Instead, hugepages should be and are released with huge_page_release() from mm/hugetlb.c. This patch removes the broken PageCompound path from put_page(), replacing it with a BUG_ON(). This also removes the initialization of page[1].mapping from compoound pages, which was only ever used in this broken code path.

We could certainly remove the test for a null destructor in there and require that compound pages have a destructor installed.

But the main reason why that code is in there is for transparently handling direct-io into hugepage regions. That code does perform put_page against 4k pageframes within the huge page and it does follow the pointer to the head page.

With your patch applied get_user_pages() and bio_release_pages() will manipulate the refcounts of the inner 4k pages rather than the head pages and things will explode.

We could change follow_hugetlb_page() to always take a ref against the head page and we could teach bio_release_pages() to perform appropriate pfn masking to locate the head page, and perform similar tricks for futexes-in-large-pages. But with the code as-is the refcounting works transparently.

If it's "broken" I wanna know why.

#2 lore raw
Quoted 7 lines · Andrew Morton

We could certainly remove the test for a null destructor in there and require that compound pages have a destructor installed.
But the main reason why that code is in there is for transparently handling direct-io into hugepage regions. That code does perform put_page against 4k pageframes within the huge page and it does follow the pointer to the head page.
With your patch applied get_user_pages() and bio_release_pages() will

Quoted 8 lines

manipulate the refcounts of the inner 4k pages rather than the head pages and things will explode.
We could change follow_hugetlb_page() to always take a ref against the head page and we could teach bio_release_pages() to perform appropriate pfn masking to locate the head page, and perform similar tricks for futexes-in-large-pages. But with the code as-is the refcounting works transparently.
If it's "broken" I wanna know why.

The destructor is never invoked from that path, but it's not the path that should free it anyway. To me it appears the call to __page_cache_release() on the head of the hugepage should just be removed in favor of doing nothing; at best it can only race against concurrent put_page(), see page_count(page) vanish, and accidentally call free_hot_page() against the head of the hugepage. As hugepages are never on the LRU, the remainder of __page_cache_release() should be a nop for them.

Untested patch below.

-- wli

mm/swap.c +5 -8
Index: wli-2.6.6-rc2-mm1/mm/swap.c
===================================================================
@@ -41,15 +41,12 @@
 	if (unlikely(PageCompound(page))) {
 		page = (struct page *)page->private;
 		if (put_page_testzero(page)) {
-			if (page[1].mapping) {	/* destructor? */
-				(*(void (*)(struct page *))page[1].mapping)(page);
-			} else {
-				__page_cache_release(page);
-			}
+			void (*destructor)(struct page *);
+			destructor = (void (*)(struct page *))page[1].mapping;
+			BUG_ON(!destructor);
+			(*destructor)(page);
 		}
-		return;
-	}
-	if (!PageReserved(page) && put_page_testzero(page))
+	} else if (!PageReserved(page) && put_page_testzero(page))
 		__page_cache_release(page);
 }
 EXPORT_SYMBOL(put_page);
#3 lore raw
Quoted 14 lines · William Lee Irwin III

--- wli-2.6.6-rc2-mm1.orig/mm/swap.c 2004-04-21 05:19:58.000000000 -0700
+++ wli-2.6.6-rc2-mm1/mm/swap.c 2004-04-23 03:21:22.000000000 -0700
@@ -41,15 +41,12 @@
if (unlikely(PageCompound(page))) {
page = (struct page *)page->private; if (put_page_testzero(page)) {
- if (page[1].mapping) { /* destructor? */
- (*(void (*)(struct page *))page[1].mapping)(page);
- } else {
- __page_cache_release(page);
- }
+ void (*destructor)(struct page *);
+ destructor = (void (*)(struct page *))page[1].mapping;

Quoted 8 lines

+ BUG_ON(!destructor);
+ (*destructor)(page);
}
- return;
- }
- if (!PageReserved(page) && put_page_testzero(page))
+ } else if (!PageReserved(page) && put_page_testzero(page))
__page_cache_release(page);

Sure.

This of course duplicates huge_page_release(), which can be killed off.

#4 lore raw
Quoted 2 lines · Andrew Morton

Sure.
This of course duplicates huge_page_release(), which can be killed off.

Ah, but mm/hugetlb.c is putting the destructor in head->lru.prev not head[1].mapping; fix below along with nuking huge_page_release().

-- wli

include/linux/hugetlb.h +0 -2
Index: wli-2.6.6-rc2-mm1/include/linux/hugetlb.h
===================================================================
@@ -18,7 +18,6 @@
 void zap_hugepage_range(struct vm_area_struct *, unsigned long, unsigned long);
 void unmap_hugepage_range(struct vm_area_struct *, unsigned long, unsigned long);
 int hugetlb_prefault(struct address_space *, struct vm_area_struct *);
-void huge_page_release(struct page *);
 int hugetlb_report_meminfo(char *);
 int is_hugepage_mem_enough(size_t);
 unsigned long hugetlb_total_pages(void);
@@ -70,7 +69,6 @@
 #define hugetlb_prefault(mapping, vma)		({ BUG(); 0; })
 #define zap_hugepage_range(vma, start, len)	BUG()
 #define unmap_hugepage_range(vma, start, end)	BUG()
-#define huge_page_release(page)			BUG()
 #define is_hugepage_mem_enough(size)		0
 #define hugetlb_report_meminfo(buf)		0
 #define mark_mm_hugetlb(mm, vma)		do { } while (0)
mm/swap.c +5 -8
Index: wli-2.6.6-rc2-mm1/mm/swap.c
===================================================================
@@ -41,15 +41,12 @@
 	if (unlikely(PageCompound(page))) {
 		page = (struct page *)page->private;
 		if (put_page_testzero(page)) {
-			if (page[1].mapping) {	/* destructor? */
-				(*(void (*)(struct page *))page[1].mapping)(page);
-			} else {
-				__page_cache_release(page);
-			}
+			void (*destructor)(struct page *);
+			destructor = (void (*)(struct page *))page[1].mapping;
+			BUG_ON(!destructor);
+			(*destructor)(page);
 		}
-		return;
-	}
-	if (!PageReserved(page) && put_page_testzero(page))
+	} else if (!PageReserved(page) && put_page_testzero(page))
 		__page_cache_release(page);
 }
 EXPORT_SYMBOL(put_page);
mm/hugetlb.c +1 -9
Index: wli-2.6.6-rc2-mm1/mm/hugetlb.c
===================================================================
@@ -78,20 +78,12 @@
 	free_huge_pages--;
 	spin_unlock(&hugetlb_lock);
 	set_page_count(page, 1);
-	page->lru.prev = (void *)free_huge_page;
+	page[1].mapping = (void *)free_huge_page;
 	for (i = 0; i < (HPAGE_SIZE/PAGE_SIZE); ++i)
 		clear_highpage(&page[i]);
 	return page;
 }
 
-void huge_page_release(struct page *page)
-{
-	if (!put_page_testzero(page))
-		return;
-
-	free_huge_page(page);
-}
-
 static int __init hugetlb_init(void)
 {
 	unsigned long i;
arch/i386/mm/hugetlbpage.c +1 -1
Index: wli-2.6.6-rc2-mm1/arch/i386/mm/hugetlbpage.c
===================================================================
@@ -220,7 +220,7 @@
 		if (pte_none(pte))
 			continue;
 		page = pte_page(pte);
-		huge_page_release(page);
+		put_page(page);
 	}
 	mm->rss -= (end - start) >> PAGE_SHIFT;
 	flush_tlb_range(vma, start, end);
arch/ia64/mm/hugetlbpage.c +1 -1
Index: wli-2.6.6-rc2-mm1/arch/ia64/mm/hugetlbpage.c
===================================================================
@@ -249,7 +249,7 @@
 		if (pte_none(*pte))
 			continue;
 		page = pte_page(*pte);
-		huge_page_release(page);
+		put_page(page);
 		pte_clear(pte);
 	}
 	mm->rss -= (end - start) >> PAGE_SHIFT;
arch/ppc64/mm/hugetlbpage.c +1 -1
Index: wli-2.6.6-rc2-mm1/arch/ppc64/mm/hugetlbpage.c
===================================================================
@@ -394,7 +394,7 @@
 			flush_hash_hugepage(mm->context, addr,
 					    pte, local);
 
-		huge_page_release(page);
+		put_page(page);
 	}
 
 	mm->rss -= (end - start) >> PAGE_SHIFT;
arch/sh/mm/hugetlbpage.c +1 -1
Index: wli-2.6.6-rc2-mm1/arch/sh/mm/hugetlbpage.c
===================================================================
@@ -200,7 +200,7 @@
 		if (pte_none(*pte))
 			continue;
 		page = pte_page(*pte);
-		huge_page_release(page);
+		put_page(page);
 		for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
 			pte_clear(pte);
 			pte++;
arch/sparc64/mm/hugetlbpage.c +1 -1
Index: wli-2.6.6-rc2-mm1/arch/sparc64/mm/hugetlbpage.c
===================================================================
@@ -198,7 +198,7 @@
 		if (pte_none(*pte))
 			continue;
 		page = pte_page(*pte);
-		huge_page_release(page);
+		put_page(page);
 		for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
 			pte_clear(pte);
 			pte++;
#5 lore raw
Quoted 2 lines · Andrew Morton
Quoted 2 lines

Sure.
This of course duplicates huge_page_release(), which can be killed off.

Quoted 2 lines · William Lee Irwin III

Ah, but mm/hugetlb.c is putting the destructor in head->lru.prev not head[1].mapping; fix below along with nuking huge_page_release().

I just noticed a general problem where ->mapping is often referred to naked and so either needs to be filled in for the base pages of the superpage or the examiners need to be referred to the head of the superpage. Likewise with ->index, which is apparently now used to hold the order. In general, if they're superpage properties, either the examiners must be referred to the head of the superpage, or (wastefully) the values must be replicated across the base pages. The current scheme to prevent radix tree deadlocks and other nastinesses with multiple insertions (e.g. inserting an entry into the radix tree for each base page, which I've seen naive abuses of shit themselves several times before) reduces the resolution of the pagecache indices, which is problematic in various ways with respect to finding the right base page without updating excessive numbers of callers. The reason why this is less of a problem than it sounds like is that the base page lookups usually go through pagetables, which effectively act either as radix trees with an insertion for each base page (normal cpus) or radix trees with low- resolution indices with procedural API's to recover offsets into the superpage and the proper base page (i386, ppc64).

I'll be off the net all weekend, so hopefully the interested parties can use this info to resolve the remainder of the hugetlb put_page() issues. The work here is just cleaning up self-inconsistencies so it's not hard.

-- wli

#6 lore raw
In reply to Andrew Morton
Quoted 14 lines · Andrew Morton
Quoted 12 lines · David Gibson

Andrew, please apply.

The code of put_page() is misleading, in that it appears to have code handling PageCompound pages (i.e. hugepages). However it won't actually handle them correctly - __page_cache_release() will not work properly on a compound. Instead, hugepages should be and are released with huge_page_release() from mm/hugetlb.c. This patch removes the broken PageCompound path from put_page(), replacing it with a BUG_ON(). This also removes the initialization of page[1].mapping from compoound pages, which was only ever used in this broken code path.

Quoted 2 lines

We could certainly remove the test for a null destructor in there and require that compound pages have a destructor installed.

We could, but there seems little point, given that we have to test for PageCompound anyway, and the destructor is always the same for hugepages.

Quoted 4 lines

But the main reason why that code is in there is for transparently handling direct-io into hugepage regions. That code does perform put_page against 4k pageframes within the huge page and it does follow the pointer to the head page.

Ah. This I did not know.

Quoted 3 lines

With your patch applied get_user_pages() and bio_release_pages() will manipulate the refcounts of the inner 4k pages rather than the head pages and things will explode.

Actually, no - bio_release_pages() will instantly BUG() with my patch.

Quoted 7 lines

We could change follow_hugetlb_page() to always take a ref against the head page and we could teach bio_release_pages() to perform appropriate pfn masking to locate the head page, and perform similar tricks for futexes-in-large-pages. But with the code as-is the refcounting works transparently.

If it's "broken" I wanna know why.

It's broken if put_page() ever actually reduces a hugepage's count to zero, because it will call __page_cache_release() which will do the wrong thing on hugepages. We could bypass that by requiring that a hugepage always has a destructor (and actually setting it in the correct place, which hugetlb.c currently doesn't as wli pointed out). But since we only have one destructor, why bother.

Patch below makes put_page() work correctly on hugepages, and uses it (via page_cache_release()) in place of huge_page_release(). It also abolishes the destructor pointer - free_huge_page() is always used. Please apply.

mm/swap.c +2 -7
Index: working-2.6/mm/swap.c
===================================================================
@@ -40,13 +40,8 @@
 {
 	if (unlikely(PageCompound(page))) {
 		page = (struct page *)page->private;
-		if (put_page_testzero(page)) {
-			if (page[1].mapping) {	/* destructor? */
-				(*(void (*)(struct page *))page[1].mapping)(page);
-			} else {
-				__page_cache_release(page);
-			}
-		}
+		if (put_page_testzero(page))
+			free_huge_page(page);
 		return;
 	}
 	if (!PageReserved(page) && put_page_testzero(page))
mm/hugetlb.c +0 -9
Index: working-2.6/mm/hugetlb.c
===================================================================
@@ -78,20 +78,11 @@
 	free_huge_pages--;
 	spin_unlock(&hugetlb_lock);
 	set_page_count(page, 1);
-	page->lru.prev = (void *)free_huge_page;
 	for (i = 0; i < (HPAGE_SIZE/PAGE_SIZE); ++i)
 		clear_highpage(&page[i]);
 	return page;
 }
 
-void huge_page_release(struct page *page)
-{
-	if (!put_page_testzero(page))
-		return;
-
-	free_huge_page(page);
-}
-
 static int __init hugetlb_init(void)
 {
 	unsigned long i;
arch/i386/mm/hugetlbpage.c +1 -1
Index: working-2.6/arch/i386/mm/hugetlbpage.c
===================================================================
@@ -221,7 +221,7 @@
 		if (pte_none(pte))
 			continue;
 		page = pte_page(pte);
-		huge_page_release(page);
+		page_cache_release(page);
 	}
 	mm->rss -= (end - start) >> PAGE_SHIFT;
 	flush_tlb_range(vma, start, end);
arch/ppc64/mm/hugetlbpage.c +1 -1
Index: working-2.6/arch/ppc64/mm/hugetlbpage.c
===================================================================
@@ -395,7 +395,7 @@
 			flush_hash_hugepage(mm->context, addr,
 					    pte, local);
 
-		huge_page_release(page);
+		page_cache_release(page);
 	}
 
 	mm->rss -= (end - start) >> PAGE_SHIFT;
arch/sh/mm/hugetlbpage.c +1 -1
Index: working-2.6/arch/sh/mm/hugetlbpage.c
===================================================================
@@ -200,7 +200,7 @@
 		if (pte_none(*pte))
 			continue;
 		page = pte_page(*pte);
-		huge_page_release(page);
+		page_cache_release(page);
 		for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
 			pte_clear(pte);
 			pte++;
arch/sparc64/mm/hugetlbpage.c +1 -1
Index: working-2.6/arch/sparc64/mm/hugetlbpage.c
===================================================================
@@ -198,7 +198,7 @@
 		if (pte_none(*pte))
 			continue;
 		page = pte_page(*pte);
-		huge_page_release(page);
+		page_cache_release(page);
 		for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
 			pte_clear(pte);
 			pte++;
fs/hugetlbfs/inode.c +2 -2
Index: working-2.6/fs/hugetlbfs/inode.c
===================================================================
@@ -142,7 +142,7 @@
 	int i;
 
 	for (i = 0; i < pagevec_count(pvec); ++i)
-		huge_page_release(pvec->pages[i]);
+		page_cache_release(pvec->pages[i]);
 
 	pagevec_reinit(pvec);
 }
@@ -152,7 +152,7 @@
 	clear_page_dirty(page);
 	ClearPageUptodate(page);
 	remove_from_page_cache(page);
-	huge_page_release(page);
+	page_cache_release(page);
 }
 
 void truncate_hugepages(struct address_space *mapping, loff_t lstart)
include/linux/hugetlb.h +0 -2
Index: working-2.6/include/linux/hugetlb.h
===================================================================
@@ -16,7 +16,6 @@
 void zap_hugepage_range(struct vm_area_struct *, unsigned long, unsigned long);
 void unmap_hugepage_range(struct vm_area_struct *, unsigned long, unsigned long);
 int hugetlb_prefault(struct address_space *, struct vm_area_struct *);
-void huge_page_release(struct page *);
 int hugetlb_report_meminfo(char *);
 int is_hugepage_mem_enough(size_t);
 unsigned long hugetlb_total_pages(void);
@@ -68,7 +67,6 @@
 #define hugetlb_prefault(mapping, vma)		({ BUG(); 0; })
 #define zap_hugepage_range(vma, start, len)	BUG()
 #define unmap_hugepage_range(vma, start, end)	BUG()
-#define huge_page_release(page)			BUG()
 #define is_hugepage_mem_enough(size)		0
 #define hugetlb_report_meminfo(buf)		0
 #define mark_mm_hugetlb(mm, vma)		do { } while (0)
Signature · 4 lines
David Gibson			| For every complex problem there is a
david AT gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson
#7 lore raw
Quoted 50 lines · David Gibson
Quoted 14 lines · Andrew Morton
Quoted 12 lines · David Gibson

Andrew, please apply.

The code of put_page() is misleading, in that it appears to have code handling PageCompound pages (i.e. hugepages). However it won't actually handle them correctly - __page_cache_release() will not work properly on a compound. Instead, hugepages should be and are released with huge_page_release() from mm/hugetlb.c. This patch removes the broken PageCompound path from put_page(), replacing it with a BUG_ON(). This also removes the initialization of page[1].mapping from compoound pages, which was only ever used in this broken code path.

Quoted 2 lines

We could certainly remove the test for a null destructor in there and require that compound pages have a destructor installed.

We could, but there seems little point, given that we have to test for PageCompound anyway, and the destructor is always the same for hugepages.

Quoted 4 lines

But the main reason why that code is in there is for transparently handling direct-io into hugepage regions. That code does perform put_page against 4k pageframes within the huge page and it does follow the pointer to the head page.

Ah. This I did not know.

Quoted 3 lines

With your patch applied get_user_pages() and bio_release_pages() will manipulate the refcounts of the inner 4k pages rather than the head pages and things will explode.

Actually, no - bio_release_pages() will instantly BUG() with my patch.

Quoted 7 lines

We could change follow_hugetlb_page() to always take a ref against the head page and we could teach bio_release_pages() to perform appropriate pfn masking to locate the head page, and perform similar tricks for futexes-in-large-pages. But with the code as-is the refcounting works transparently.

If it's "broken" I wanna know why.

It's broken if put_page() ever actually reduces a hugepage's count to zero, because it will call __page_cache_release() which will do the wrong thing on hugepages. We could bypass that by requiring that a hugepage always has a destructor (and actually setting it in the correct place, which hugetlb.c currently doesn't as wli pointed out). But since we only have one destructor, why bother.

Quoted 4 lines

Patch below makes put_page() work correctly on hugepages, and uses it (via page_cache_release()) in place of huge_page_release(). It also abolishes the destructor pointer - free_huge_page() is always used. Please apply.

Damn. Supplied patch generated a warning. Fixed version below:

mm/swap.c +3 -7
Index: working-2.6/mm/swap.c
===================================================================
@@ -30,6 +30,7 @@
 #include <linux/cpu.h>
 #include <linux/notifier.h>
 #include <linux/init.h>
+#include <linux/hugetlb.h>
 
 /* How many pages do we try to swap or page in/out together? */
 int page_cluster;
@@ -40,13 +41,8 @@
 {
 	if (unlikely(PageCompound(page))) {
 		page = (struct page *)page->private;
-		if (put_page_testzero(page)) {
-			if (page[1].mapping) {	/* destructor? */
-				(*(void (*)(struct page *))page[1].mapping)(page);
-			} else {
-				__page_cache_release(page);
-			}
-		}
+		if (put_page_testzero(page))
+			free_huge_page(page);
 		return;
 	}
 	if (!PageReserved(page) && put_page_testzero(page))
mm/hugetlb.c +0 -9
Index: working-2.6/mm/hugetlb.c
===================================================================
@@ -78,20 +78,11 @@
 	free_huge_pages--;
 	spin_unlock(&hugetlb_lock);
 	set_page_count(page, 1);
-	page->lru.prev = (void *)free_huge_page;
 	for (i = 0; i < (HPAGE_SIZE/PAGE_SIZE); ++i)
 		clear_highpage(&page[i]);
 	return page;
 }
 
-void huge_page_release(struct page *page)
-{
-	if (!put_page_testzero(page))
-		return;
-
-	free_huge_page(page);
-}
-
 static int __init hugetlb_init(void)
 {
 	unsigned long i;
arch/i386/mm/hugetlbpage.c +1 -1
Index: working-2.6/arch/i386/mm/hugetlbpage.c
===================================================================
@@ -221,7 +221,7 @@
 		if (pte_none(pte))
 			continue;
 		page = pte_page(pte);
-		huge_page_release(page);
+		page_cache_release(page);
 	}
 	mm->rss -= (end - start) >> PAGE_SHIFT;
 	flush_tlb_range(vma, start, end);
arch/ppc64/mm/hugetlbpage.c +1 -1
Index: working-2.6/arch/ppc64/mm/hugetlbpage.c
===================================================================
@@ -395,7 +395,7 @@
 			flush_hash_hugepage(mm->context, addr,
 					    pte, local);
 
-		huge_page_release(page);
+		page_cache_release(page);
 	}
 
 	mm->rss -= (end - start) >> PAGE_SHIFT;
arch/sh/mm/hugetlbpage.c +1 -1
Index: working-2.6/arch/sh/mm/hugetlbpage.c
===================================================================
@@ -200,7 +200,7 @@
 		if (pte_none(*pte))
 			continue;
 		page = pte_page(*pte);
-		huge_page_release(page);
+		page_cache_release(page);
 		for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
 			pte_clear(pte);
 			pte++;
arch/sparc64/mm/hugetlbpage.c +1 -1
Index: working-2.6/arch/sparc64/mm/hugetlbpage.c
===================================================================
@@ -198,7 +198,7 @@
 		if (pte_none(*pte))
 			continue;
 		page = pte_page(*pte);
-		huge_page_release(page);
+		page_cache_release(page);
 		for (i = 0; i < (1 << HUGETLB_PAGE_ORDER); i++) {
 			pte_clear(pte);
 			pte++;
fs/hugetlbfs/inode.c +2 -2
Index: working-2.6/fs/hugetlbfs/inode.c
===================================================================
@@ -142,7 +142,7 @@
 	int i;
 
 	for (i = 0; i < pagevec_count(pvec); ++i)
-		huge_page_release(pvec->pages[i]);
+		page_cache_release(pvec->pages[i]);
 
 	pagevec_reinit(pvec);
 }
@@ -152,7 +152,7 @@
 	clear_page_dirty(page);
 	ClearPageUptodate(page);
 	remove_from_page_cache(page);
-	huge_page_release(page);
+	page_cache_release(page);
 }
 
 void truncate_hugepages(struct address_space *mapping, loff_t lstart)
include/linux/hugetlb.h +0 -2
Index: working-2.6/include/linux/hugetlb.h
===================================================================
@@ -16,7 +16,6 @@
 void zap_hugepage_range(struct vm_area_struct *, unsigned long, unsigned long);
 void unmap_hugepage_range(struct vm_area_struct *, unsigned long, unsigned long);
 int hugetlb_prefault(struct address_space *, struct vm_area_struct *);
-void huge_page_release(struct page *);
 int hugetlb_report_meminfo(char *);
 int is_hugepage_mem_enough(size_t);
 unsigned long hugetlb_total_pages(void);
@@ -68,7 +67,6 @@
 #define hugetlb_prefault(mapping, vma)		({ BUG(); 0; })
 #define zap_hugepage_range(vma, start, len)	BUG()
 #define unmap_hugepage_range(vma, start, end)	BUG()
-#define huge_page_release(page)			BUG()
 #define is_hugepage_mem_enough(size)		0
 #define hugetlb_report_meminfo(buf)		0
 #define mark_mm_hugetlb(mm, vma)		do { } while (0)
Signature · 4 lines
David Gibson			| For every complex problem there is a
david AT gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson
#8 lore raw
In reply to David Gibson

Review

Quoted 3 lines · David Gibson
(quoted diff) +1 -0
+		if (put_page_testzero(page))
 +			free_huge_page(page);

Well yes, but this is assuming that compound pages are always hugetlb pages.

It's true at present, but it doesn't have to always be true. The cost of the destructor is zilch, so why not?

Please review the changes which went into 2.6.6-rc2-mm2.

#9 lore raw
Quoted 9 lines · Andrew Morton

Review

Quoted 3 lines · David Gibson
(quoted diff) +1 -0
+		if (put_page_testzero(page))
 +			free_huge_page(page);

Well yes, but this is assuming that compound pages are always hugetlb pages.

It's true at present, but it doesn't have to always be true. The cost of the destructor is zilch, so why not?

True enough, I guess I was just following the "don't build it till you need it" philosophy.

Quoted 1 line

Please review the changes which went into 2.6.6-rc2-mm2.

Sorry, should have done that earlier. Looks reasonable with two small
exceptions: 1) put_page() can still theoretically call __page_cache_release() which is wrong (and makes the code misleading)
- patch below replaces this with a BUG() if there is no destructor. 2) what about wli's concern that mapping may be accessed without first checking for a PageCompound?

mm/swap.c +3 -5
Index: working-2.6/mm/swap.c
===================================================================
@@ -41,11 +41,9 @@
 	if (unlikely(PageCompound(page))) {
 		page = (struct page *)page->private;
 		if (put_page_testzero(page)) {
-			if (page[1].mapping) {	/* destructor? */
-				(*(void (*)(struct page *))page[1].mapping)(page);
-			} else {
-				__page_cache_release(page);
-			}
+			BUG_ON(! page[1].mapping);
+
+			(*(void (*)(struct page *))page[1].mapping)(page);
 		}
 		return;
 	}
Signature · 4 lines
David Gibson			| For every complex problem there is a
david AT gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson
#10 lore raw
Quoted 3 lines · David Gibson

1) put_page() can still theoretically call __page_cache_release() which is wrong (and makes the code misleading)

Don't think so?

void put_page(struct page *page)

{
	if (unlikely(PageCompound(page))) {
		page = (struct page *)page->private;
		if (put_page_testzero(page)) {
			void (*dtor)(struct page *page);

			dtor = (void (*)(struct page *))page[1].mapping;
			(*dtor)(page);
		}
		return;
	}
	if (!PageReserved(page) && put_page_testzero(page))
		__page_cache_release(page);
}
Quoted 3 lines

- patch below replaces this with a BUG() if there is no destructor. 2) what about wli's concern that mapping may be accessed without first checking for a PageCompound?

That shouldn't be happening (should it?). If someone had such a page they'd need to lock it to play with ->mapping and I cannot think of any code which does all that and yet could stumble across a compound page?

#11 lore raw
Quoted 8 lines · Andrew Morton
Quoted 3 lines · David Gibson

1) put_page() can still theoretically call __page_cache_release() which is wrong (and makes the code misleading)

Don't think so?

void put_page(struct page *page)

Quoted 14 lines
{
	if (unlikely(PageCompound(page))) {
		page = (struct page *)page->private;
		if (put_page_testzero(page)) {
			void (*dtor)(struct page *page);

			dtor = (void (*)(struct page *))page[1].mapping;
			(*dtor)(page);
		}
		return;
	}
	if (!PageReserved(page) && put_page_testzero(page))
		__page_cache_release(page);
}

Dang. Missed that patch in the series. My mistake.

Signature · 4 lines
David Gibson			| For every complex problem there is a
david AT gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson