- deleting huge pages
PATCH 1/1 - deleting huge pages
I found this problem in 2.4,21, but AFAICT, the same problem exists in 2.6.5.
If you attempt to allocate a LOT more huge pages than are physically available, the kernel may reference invalid PGDs or PMDs.
Here is the 2.4 backtrace of a failure. If the mmap fails, do_mmap_pgoff attempts to unmap the vma range it was mapping. Depending on where it failed during the mmap, some of the higher level PGD/PMDs may not have been assigned.
The bug (at least in 2.4) exists on all platforms but on our platform attempts to dereference NULL pointers usually cause MCAs. (If a platform has zeros in page 0, you may be lucky & the code would appear to work, but it is still a bug).
Stack traceback for pid 6817 0xe00025307ba50000 6817 6663 0 148 D 0xe00025307ba50420 toy 0xe00000000445e180 unmap_hugepage_range+0x160 << mca surfaced here 0xe00000000445e300 zap_hugepage_range+0x80 0xe00000000452dbc0 do_mmap_pgoff+0xea0 0xe000000004432910 sys_mmap+0x210 0xe00000000440e2a0 ia64_ret_from_syscall
The MCA was caused by the NULL pmd dereference in huge_pte_offset. The MCA doesnt surface until the bad data is consumed.
A patch against 2.6.5:
Index: linux/arch/ia64/mm/hugetlbpage.c
=================================--- linux.orig/arch/ia64/mm/hugetlbpage.c 2004-05-01 20:51:52.000000000 -0500
+++ linux/arch/ia64/mm/hugetlbpage.c 2004-05-01 20:51:54.000000000 -0500
@@ -111,9 +111,16 @@
pte_t *pte = NULL;
pgd = pgd_offset(mm, taddr);
+ if (pgd_none(*pgd) || pgd_bad(*pgd))
+ goto out;
pmd = pmd_offset(pgd, taddr);
+ if (pmd_none(*pmd) || pmd_bad(*pmd))
+ goto out;
pte = pte_offset_map(pmd, taddr);
return pte;
+
+out:
+ return 0;
}
#define mk_pte_huge(entry) { pte_val(entry) |= _PAGE_P; }
@@ -331,7 +338,7 @@
for (address = start; address < end; address += HPAGE_SIZE) {
pte = huge_pte_offset(mm, address);
- if (pte_none(*pte))
+ if (!pte || pte_none(*pte))
continue;
page = pte_page(*pte); huge_page_release(page);
Signature · 4 lines
Thanks Jack Steiner ([email protected]) 651-683-5302 Principal Engineer SGI - Silicon Graphics, Inc.
Quoted 16 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Jack Steiner wrote on Sunday, May 02, 2004 5:30 AM
I found this problem in 2.4,21, but AFAICT, the same problem exists in 2.6.5.
If you attempt to allocate a LOT more huge pages than are physically available, the kernel may reference invalid PGDs or PMDs.
Here is the 2.4 backtrace of a failure. If the mmap fails,
do_mmap_pgoffattempts to unmap the vma range it was mapping. Depending on where it failed during the mmap, some of the higher level PGD/PMDs may not have been assigned.The bug (at least in 2.4) exists on all platforms but on our platform attempts to dereference NULL pointers usually cause MCAs. (If a platform has zeros in page 0, you may be lucky & the code would appear to work, but it is still a bug).
Quoted 4 lines
The MCA was caused by the NULL pmd dereference in
huge_pte_offset. The MCA doesnt surface until the bad data is consumed.A patch against 2.6.5:
Recent work on cleaning up hugepage_vma has at least one hunk covered here.
http://linux.bkbits.net:8080/linux-2.5/cset@40842336E3nkJ7cWJ0-3zQ7yP4WbHg
- Ken
Quoted 19 lines · Kenneth W
Quoted 19 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Jack Steiner wrote on Sunday, May 02, 2004 5:30 AM
I found this problem in 2.4,21, but AFAICT, the same problem exists in 2.6.5.
If you attempt to allocate a LOT more huge pages than are physically available, the kernel may reference invalid PGDs or PMDs.
Here is the 2.4 backtrace of a failure. If the mmap fails,
do_mmap_pgoffattempts to unmap the vma range it was mapping. Depending on where it failed during the mmap, some of the higher level PGD/PMDs may not have been assigned.The bug (at least in 2.4) exists on all platforms but on our platform attempts to dereference NULL pointers usually cause MCAs. (If a platform has zeros in page 0, you may be lucky & the code would appear to work, but it is still a bug).
The MCA was caused by the NULL pmd dereference in
huge_pte_offset. The MCA doesnt surface until the bad data is consumed.
Quoted 7 lines
Quoted 1 line
A patch against 2.6.5:
Recent work on cleaning up
hugepage_vmahas at least one hunk covered here.
http://linux.bkbits.net:8080/linux-2.5/cset@40842336E3nkJ7cWJ0-3zQ7yP4WbHg- Ken
Yep... Looks like the same problem has been fixed by David Gibson. Ignore my patch.
Signature · 4 lines
Thanks Jack Steiner ([email protected]) 651-683-5302 Principal Engineer SGI - Silicon Graphics, Inc.
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
Quoted 1 line
Quoted 1 line
Jack Steiner wrote on Monday, May 03, 2004 7:53 AM
I found this problem in 2.4,21, but AFAICT, the same problem exists in 2.6.5.
If you attempt to allocate a LOT more huge pages than are physically available, the kernel may reference invalid PGDs or PMDs.
Here is the 2.4 backtrace of a failure. If the mmap fails,
do_mmap_pgoffattempts to unmap the vma range it was mapping. Depending on where it failed during the mmap, some of the higher level PGD/PMDs may not have been assigned.The bug (at least in 2.4) exists on all platforms but on our platform attempts to dereference NULL pointers usually cause MCAs. (If a platform has zeros in page 0, you may be lucky & the code would appear to work, but it is still a bug).
The MCA was caused by the NULL pmd dereference in
huge_pte_offset. The MCA doesnt surface until the bad data is consumed.A patch against 2.6.5:
Quoted 7 lines
Quoted 4 lines
Recent work on cleaning up
hugepage_vmahas at least one hunk covered here.
http://linux.bkbits.net:8080/linux-2.5/cset@40842336E3nkJ7cWJ0-3zQ7yP4WbHgYep... Looks like the same problem has been fixed by David Gibson. Ignore my patch.
Jack, I don't mean to stir up the mud. The hunk in unmap_hugepage_range() in your original post is still needed for 2.6 kernel, plus Bjorn definitely need your fix for 2.4 tree ;-)
And all this just making hugetlb demanding paging patch smaller .... :-)
- Ken
(Update to previous mail - patch updated for latest tree)
If you attempt to allocate a LOT more huge pages than are physically available, the kernel may reference a NULL pte.
If the mmap fails, do_mmap_pgoff attempts to unmap the vma range it was mapping. Depending on where it failed during
the mmap, some of the higher level PGD/PMDs may not have been assigned.
The same problem exists in 2.4. I'll post a second patch in a few days....
arch/ia64/mm/hugetlbpage.c +1 -1
@@ -245,7 +245,7 @@for (address = start; address < end; address +=HPAGE_SIZE ) { pte =huge_pte_offset (mm, address); -if (pte_none (*pte)) +if (!pte ||pte_none (*pte))continue ; page =pte_page (*pte);put_page (page);
Signature · 4 lines
Thanks Jack Steiner ([email protected]) 651-683-5302 Principal Engineer SGI - Silicon Graphics, Inc.