x86,mm: fix pte_free()
Subsystems (2)
PATCH 1/1 x86,mm: fix pte_free()
On -rt we were seeing spurious bad page states like:
Bad page state in process 'firefox'
page:c1bc2380 flags:0x40000000 mapping:c1bc2390 mapcount:0 count:0 Trying to fix it up, but a reboot is needed
Backtrace: Pid: 503, comm: firefox Not tainted 2.6.26.8-rt13 #3 [<c043d0f3>] ? printk+0x14/0x19 [<c0272d4e>] bad_page+0x4e/0x79 [<c0273831>] free_hot_cold_page+0x5b/0x1d3 [<c02739f6>] free_hot_page+0xf/0x11 [<c0273a18>] __free_pages+0x20/0x2b [<c027d170>] __pte_alloc+0x87/0x91 [<c027d25e>] handle_mm_fault+0xe4/0x733 [<c043f680>] ? rt_mutex_down_read_trylock+0x57/0x63 [<c043f680>] ? rt_mutex_down_read_trylock+0x57/0x63 [<c0218875>] do_page_fault+0x36f/0x88a
This is the case where a concurrent fault already installed the PTE and we get to free the newly allocated one.
This is due to pgtable_page_ctor() doing the spin_lock_init(&page->ptl) which is overlaid with the {private, mapping} struct.
union {
struct {
unsigned long private;
struct address_space *mapping;
};
#if NR_CPUS >= CONFIG_SPLIT_PTLOCK_CPUS
spinlock_t ptl;
#endif
struct kmem_cache *slab;
struct page *first_page;
};Normally the spinlock is small enough to not stomp on page->mapping, but PREEMPT_RT=y has huge 'spin'locks.
But lockdep kernels should also be able to trigger this splat, as the lock tracking code grows the spinlock to cover page->mapping.
The obvious fix is calling pgtable_page_dtor() like the regular pte free path __pte_free_tlb() does.
It seems all architectures except x86 and nm10300 already do this, and nm10300 doesn't seem to use pgtable_page_ctor(), which suggests it doesn't do SMP or simply doesnt do MMU at all or something.
--- arch/x86/include/asm/pgalloc.h | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
arch/x86/include/asm/pgalloc.h +1 -0
@@ -42,6 +42,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)static inline void pte_free (struct mm_struct *mm,struct page *pte) { +pgtable_page_dtor ();__free_page (pte); }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 49 lines · Peter Zijlstra
On -rt we were seeing spurious bad page states like:
Bad page state in process 'firefox'
page:c1bc2380 flags:0x40000000 mapping:c1bc2390 mapcount:0 count:0 Trying to fix it up, but a reboot is neededBacktrace: Pid: 503, comm: firefox Not tainted 2.6.26.8-rt13 #3 [<c043d0f3>] ? printk+0x14/0x19 [<c0272d4e>] bad_page+0x4e/0x79 [<c0273831>] free_hot_cold_page+0x5b/0x1d3 [<c02739f6>] free_hot_page+0xf/0x11 [<c0273a18>] __free_pages+0x20/0x2b [<c027d170>] __pte_alloc+0x87/0x91 [<c027d25e>] handle_mm_fault+0xe4/0x733 [<c043f680>] ? rt_mutex_down_read_trylock+0x57/0x63 [<c043f680>] ? rt_mutex_down_read_trylock+0x57/0x63 [<c0218875>] do_page_fault+0x36f/0x88aThis is the case where a concurrent fault already installed the PTE and
we get to free the newly allocated one.This is due to
pgtable_page_ctor()doing thespin_lock_init(&page->ptl)
which is overlaid with the {private, mapping} struct.union { struct { unsigned long private; struct address_space *mapping; }; #if NR_CPUS >= CONFIG_SPLIT_PTLOCK_CPUS spinlock_t ptl; #endif struct kmem_cache *slab; struct page *first_page; };Normally the spinlock is small enough to not stomp on
page->mapping, but
PREEMPT_RT=y has huge 'spin'locks.But lockdep kernels should also be able to trigger this splat, as the lock tracking code grows the spinlock to cover
page->mapping.The obvious fix is calling
pgtable_page_dtor()like the regular pte free
path__pte_free_tlb()does.
Quoted 6 lines
It seems all architectures except x86 and nm10300 already do this, and nm10300 doesn't seem to use
pgtable_page_ctor(), which suggests it doesn't do SMP or simply doesnt do MMU at all or something.Signed-off-by Peter Zijlstra <[email protected]>
Now one that's not obviously borken,..
--- arch/x86/include/asm/pgalloc.h | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
arch/x86/include/asm/pgalloc.h +1 -0
@@ -42,6 +42,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)static inline void pte_free (struct mm_struct *mm,struct page *pte) { +pgtable_page_dtor (pte);__free_page (pte); }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hugh Dickins
Acked-byQuoted 50 lines · Peter Zijlstra
Quoted 49 lines · Peter Zijlstra
On -rt we were seeing spurious bad page states like:
Bad page state in process 'firefox'
page:c1bc2380 flags:0x40000000 mapping:c1bc2390 mapcount:0 count:0 Trying to fix it up, but a reboot is neededBacktrace: Pid: 503, comm: firefox Not tainted 2.6.26.8-rt13 #3 [<c043d0f3>] ? printk+0x14/0x19 [<c0272d4e>] bad_page+0x4e/0x79 [<c0273831>] free_hot_cold_page+0x5b/0x1d3 [<c02739f6>] free_hot_page+0xf/0x11 [<c0273a18>] __free_pages+0x20/0x2b [<c027d170>] __pte_alloc+0x87/0x91 [<c027d25e>] handle_mm_fault+0xe4/0x733 [<c043f680>] ? rt_mutex_down_read_trylock+0x57/0x63 [<c043f680>] ? rt_mutex_down_read_trylock+0x57/0x63 [<c0218875>] do_page_fault+0x36f/0x88aThis is the case where a concurrent fault already installed the PTE and
we get to free the newly allocated one.This is due to
pgtable_page_ctor()doing thespin_lock_init(&page->ptl)
which is overlaid with the {private, mapping} struct.union { struct { unsigned long private; struct address_space *mapping; }; #if NR_CPUS >= CONFIG_SPLIT_PTLOCK_CPUS spinlock_t ptl; #endif struct kmem_cache *slab; struct page *first_page; };Normally the spinlock is small enough to not stomp on
page->mapping, but
PREEMPT_RT=y has huge 'spin'locks.But lockdep kernels should also be able to trigger this splat, as the lock tracking code grows the spinlock to cover
page->mapping.The obvious fix is calling
pgtable_page_dtor()like the regular pte free
path__pte_free_tlb()does.
Quoted 6 lines
Quoted 6 lines
It seems all architectures except x86 and nm10300 already do this, and nm10300 doesn't seem to use
pgtable_page_ctor(), which suggests it doesn't do SMP or simply doesnt do MMU at all or something.Signed-off-by Peter Zijlstra <[email protected]>
Thanks, Peter: good catch. That pgtable_page_dtor() had long been there in pte_free(), then somehow got lost in one of 2.6.26's rearrangements.
Quoted 2 lines
Now one that's not obviously borken,..
And I can quite see why you voided the first version:
your mind rightly stalled on that foul "struct page *pte". Oh well, clean that up some other time.
Hugh
Quoted 10 lines
--- arch/x86/include/asm/pgalloc.h | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)arch/x86/include/asm/pgalloc.h +0
-0
arch/x86/include/asm/pgalloc.h static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) · line 47
Quoted 6 lines
arch/x86/include/asm/pgalloc.h +1
-0@@ -42,6 +42,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)static inline void pte_free (struct mm_struct *mm,struct page *pte) { +pgtable_page_dtor (pte);__free_page (pte); }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 60 lines · Peter Zijlstra
On -rt we were seeing spurious bad page states like:
Bad page state in process 'firefox'
page:c1bc2380 flags:0x40000000 mapping:c1bc2390 mapcount:0 count:0 Trying to fix it up, but a reboot is neededBacktrace: Pid: 503, comm: firefox Not tainted 2.6.26.8-rt13 #3 [<c043d0f3>] ? printk+0x14/0x19 [<c0272d4e>] bad_page+0x4e/0x79 [<c0273831>] free_hot_cold_page+0x5b/0x1d3 [<c02739f6>] free_hot_page+0xf/0x11 [<c0273a18>] __free_pages+0x20/0x2b [<c027d170>] __pte_alloc+0x87/0x91 [<c027d25e>] handle_mm_fault+0xe4/0x733 [<c043f680>] ? rt_mutex_down_read_trylock+0x57/0x63 [<c043f680>] ? rt_mutex_down_read_trylock+0x57/0x63 [<c0218875>] do_page_fault+0x36f/0x88aThis is the case where a concurrent fault already installed the PTE and we get to free the newly allocated one.
This is due to
pgtable_page_ctor()doing thespin_lock_init(&page->ptl)which is overlaid with the {private, mapping} struct.union { struct { unsigned long private; struct address_space *mapping; }; #if NR_CPUS >= CONFIG_SPLIT_PTLOCK_CPUS spinlock_t ptl; #endif struct kmem_cache *slab; struct page *first_page; };Normally the spinlock is small enough to not stomp on
page->mapping, but PREEMPT_RT=y has huge 'spin'locks.But lockdep kernels should also be able to trigger this splat, as the lock tracking code grows the spinlock to cover
page->mapping.The obvious fix is calling
pgtable_page_dtor()like the regular pte free path__pte_free_tlb()does.It seems all architectures except x86 and nm10300 already do this, and nm10300 doesn't seem to use
pgtable_page_ctor(), which suggests it doesn't do SMP or simply doesnt do MMU at all or something.Signed-off-by Peter Zijlstra <[email protected]>--- arch/x86/include/asm/pgalloc.h | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)arch/x86/include/asm/pgalloc.h +0
-0
arch/x86/include/asm/pgalloc.h static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) · line 45
Quoted 4 lines
arch/x86/include/asm/pgalloc.h +1
-0@@ -42,6 +42,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte)static inline void pte_free (struct mm_struct *mm,struct page *pte) { +pgtable_page_dtor ();
i suspect on lockdep we dont see this in practice because it initializes things to NULL, which hides the issue. On -rt we initialize list heads there which brings up the wrong warning in the page free logic.
So i agree with the fix, but the patch does not look right: shouldnt that be pgtable_page_dtor(pte), so that we get ->mapping cleared via pte_lock_deinit()? (which i guess your intention was here - this probably wont even build)
Ingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · Ingo Molnar
So i agree with the fix, but the patch does not look right: shouldnt that be
pgtable_page_dtor(pte), so that we get ->mapping cleared viapte_lock_deinit()? (which i guess your intention was here - this probably wont even build)
Yeah, I somehow fudged it, already send out a better one. -- One of them days I guess :-(
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Peter Zijlstra
Quoted 4 lines · Ingo Molnar
So i agree with the fix, but the patch does not look right: shouldnt that be
pgtable_page_dtor(pte), so that we get ->mapping cleared viapte_lock_deinit()? (which i guess your intention was here - this probably wont even build)Yeah, I somehow fudged it, already send out a better one. -- One of them days I guess :-(
no problem - applied to tip/x86/urgent, thanks Peter!
Ingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 10 lines · Peter Zijlstra
Quoted 8 lines · Ingo Molnar
Quoted 5 lines
So i agree with the fix, but the patch does not look right: shouldnt that be
pgtable_page_dtor(pte), so that we get ->mapping cleared viapte_lock_deinit()? (which i guess your intention was here - this probably wont even build)
Quoted 4 lines
Quoted 2 lines
Yeah, I somehow fudged it, already send out a better one. -- One of them days I guess :-(
no problem - applied to tip/x86/urgent, thanks Peter!
please fix typo. s/nm10300/MN10300/ :)
at first look, I don't understand his intention.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines · Peter Zijlstra
It seems all architectures except x86 and nm10300 already do this, and nm10300 doesn't seem to use
pgtable_page_ctor(), which suggests it doesn't do SMP or simply doesnt do MMU at all or something.
MN10300 does not, as yet, do SMP.
David
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>