mm: Fix apply_to_pte_range() vs lazy MMU mode
Subsystems (4)
Alexander Gordeev
maintainercover mm: Fix apply_to_pte_range() vs lazy MMU mode
Hi All,
This series is an attempt to fix the violation of lazy MMU mode context requirement as described for arch_enter_lazy_mmu_mode():
This mode can only be entered and left under the protection of the page table locks for all page tables which may be modified.
On s390 if I make arch_enter_lazy_mmu_mode() -> preempt_enable() and arch_leave_lazy_mmu_mode() -> preempt_disable() I am getting this:
[ 553.332108] preempt_count: 1, expected: 0
[ 553.332117] no locks held by multipathd/2116.
[ 553.332128] CPU: 24 PID: 2116 Comm: multipathd Kdump: loaded Tainted:
[ 553.332139] Hardware name: IBM 3931 A01 701 (LPAR)
[ 553.332146] Call Trace:
[ 553.332152] [<00000000158de23a>] dump_stack_lvl+0xfa/0x150
[ 553.332167] [<0000000013e10d12>] __might_resched+0x57a/0x5e8
[ 553.332178] [<00000000144eb6c2>] __alloc_pages+0x2ba/0x7c0
[ 553.332189] [<00000000144d5cdc>] __get_free_pages+0x2c/0x88
[ 553.332198] [<00000000145663f6>] kasan_populate_vmalloc_pte+0x4e/0x110
[ 553.332207] [<000000001447625c>] apply_to_pte_range+0x164/0x3c8
[ 553.332218] [<000000001448125a>] apply_to_pmd_range+0xda/0x318
[ 553.332226] [<000000001448181c>] __apply_to_page_range+0x384/0x768
[ 553.332233] [<0000000014481c28>] apply_to_page_range+0x28/0x38
[ 553.332241] [<00000000145665da>] kasan_populate_vmalloc+0x82/0x98
[ 553.332249] [<00000000144c88d0>] alloc_vmap_area+0x590/0x1c90
[ 553.332257] [<00000000144ca108>] __get_vm_area_node.constprop.0+0x138/0x260
[ 553.332265] [<00000000144d17fc>] __vmalloc_node_range+0x134/0x360
[ 553.332274] [<0000000013d5dbf2>] alloc_thread_stack_node+0x112/0x378
[ 553.332284] [<0000000013d62726>] dup_task_struct+0x66/0x430
[ 553.332293] [<0000000013d63962>] copy_process+0x432/0x4b80
[ 553.332302] [<0000000013d68300>] kernel_clone+0xf0/0x7d0
[ 553.332311] [<0000000013d68bd6>] __do_sys_clone+0xae/0xc8
[ 553.332400] [<0000000013d68dee>] __s390x_sys_clone+0xd6/0x118
[ 553.332410] [<0000000013c9d34c>] do_syscall+0x22c/0x328
[ 553.332419] [<00000000158e7366>] __do_syscall+0xce/0xf0
[ 553.332428] [<0000000015913260>] system_call+0x70/0x98This exposes a KASAN issue fixed with patch 1 and apply_to_pte_range() issue fixed with patches 2-3. Patch 4 is a debug improvement on top, that could have helped to notice the issue.
Commit b9ef323ea168 ("powerpc/64s: Disable preemption in hash lazy mmu mode") looks like powerpc-only fix, yet not entirely conforming to the above provided requirement (page tables itself are still not protected). If I am not mistaken, xen and sparc are alike.
Thanks!
Alexander Gordeev (4):
kasan: Avoid sleepable page allocation from atomic context
mm: Cleanup apply_to_pte_range() routine
mm: Protect kernel pgtables in apply_to_pte_range()
mm: Allow detection of wrong arch_enter_lazy_mmu_mode() context
include/linux/pgtable.h | 15 ++++++++++++--- mm/kasan/shadow.c | 9 +++------ mm/memory.c | 33 +++++++++++++++++++++------------ 3 files changed, 36 insertions(+), 21 deletions(-)
Signature · 1 line
2.45.2
Alexander Gordeev
maintainerPATCH 1/4 kasan: Avoid sleepable page allocation from atomic context
apply_to_page_range() enters lazy MMU mode and then invokes kasan_populate_vmalloc_pte() callback on each page table walk iteration. The lazy MMU mode may only be entered only under protection of the page table lock. However, the callback can go into sleep when trying to allocate a single page.
Change __get_free_page() allocation mode from GFP_KERNEL to GFP_ATOMIC to avoid scheduling out while in atomic context.
--- mm/kasan/shadow.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/kasan/shadow.c +1 -1
@@ -301,7 +301,7 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,if (likely (!pte_none (ptep_get (ptep))))return 0 ; - page =__get_free_page (GFP_KERNEL ); + page =__get_free_page (GFP_ATOMIC );if (!page)return -ENOMEM ;
Signature · 1 line
2.45.2
Re: [PATCH v1 1/4] kasan: Avoid sleepable page allocation from atomic context
Quoted 8 lines · Alexander Gordeev
apply_to_page_range()enters lazy MMU mode and then invokeskasan_populate_vmalloc_pte()callback on each page table walk iteration. The lazy MMU mode may only be entered only under protection of the page table lock. However, the callback can go into sleep when trying to allocate a single page.Change
__get_free_page()allocation mode fromGFP_KERNELtoGFP_ATOMICto avoid scheduling out while in atomic context.
It's a bit unfortunate to make this use atomic allocs for archs that don't need it.
Could you make it depend on __HAVE_ARCH_ENTER_LAZY_MMU_MODE or is that overkill?
I wanted to remove ppc64's per-CPU page array and replace it with on stack or dynaimc alloc array in the thread... but cost/benefit of working on ppc64 hash MMU code is not high :(
Fix itself for ppc64's requirement at least looks right to me so for that,
Quoted 19 lines
Signed-off-by Alexander Gordeev <[email protected]>--- mm/kasan/shadow.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/kasan/shadow.c +1
-1@@ -301,7 +301,7 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,if (likely (!pte_none (ptep_get (ptep))))return 0 ; - page =__get_free_page (GFP_KERNEL ); + page =__get_free_page (GFP_ATOMIC );if (!page)return -ENOMEM ;
Re: [PATCH v1 1/4] kasan: Avoid sleepable page allocation from atomic context
In reply to Alexander GordeevQuoted 19 lines · Alexander Gordeev
apply_to_page_range()enters lazy MMU mode and then invokeskasan_populate_vmalloc_pte()callback on each page table walk iteration. The lazy MMU mode may only be entered only under protection of the page table lock. However, the callback can go into sleep when trying to allocate a single page.Change
__get_free_page()allocation mode fromGFP_KERNELtoGFP_ATOMICto avoid scheduling out while in atomic context.Signed-off-by Alexander Gordeev <[email protected]>--- mm/kasan/shadow.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/kasan/shadow.c +0
-0
mm/kasan/shadow.c static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, · line 307
Quoted 8 lines
mm/kasan/shadow.c +1
-1@@ -301,7 +301,7 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,if (likely (!pte_none (ptep_get (ptep))))return 0 ; - page =__get_free_page (GFP_KERNEL ); + page =__get_free_page (GFP_ATOMIC );if (!page)return -ENOMEM ;
Oh of course you can't make it GFP_KERNEL after the patch to take ptl even for archs that don't use lazy mmu.
Thanks,
Nick
Alexander Gordeev
maintainerPATCH 2/4 mm: Cleanup apply_to_pte_range() routine
In reply to Alexander GordeevReverse 'create' vs 'mm == &init_mm' conditions and move page table mask modification out of the atomic context.
--- mm/memory.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-)
mm/memory.c +17 -11
@@ -2915,24 +2915,28 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, pte_fn_t fn, void *data, bool create, pgtbl_mod_mask *mask) { + int err = create ? -ENOMEM : -EINVAL;pte_t *pte, *mapped_pte; -int err =0 ;spinlock_t *ptl; -if (create) { - mapped_pte = pte = (mm == &init_mm) ? -pte_alloc_kernel_track (pmd, addr, mask) : -pte_alloc_map_lock (mm, pmd, addr, &ptl); +if (mm == &init_mm) { +if (create) + pte =pte_alloc_kernel_track (pmd, addr, mask); +else + pte =pte_offset_kernel (pmd, addr);if (!pte) -return -ENOMEM ; +return err; }else { - mapped_pte = pte = (mm == &init_mm) ? -pte_offset_kernel (pmd, addr) : -pte_offset_map_lock (mm, pmd, addr, &ptl); +if (create) + pte =pte_alloc_map_lock (mm, pmd, addr, &ptl); +else + pte =pte_offset_map_lock (mm, pmd, addr, &ptl);if (!pte) -return -EINVAL ; +return err; + mapped_pte = pte; } + err =0 ;arch_enter_lazy_mmu_mode ();if (fn) {
@@ -2944,12 +2948,14 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, } }while (addr +=PAGE_SIZE , addr != end); } - *mask |=PGTBL_PTE_MODIFIED ;arch_leave_lazy_mmu_mode ();if (mm != &init_mm)pte_unmap_unlock (mapped_pte, ptl); + + *mask |=PGTBL_PTE_MODIFIED ; +return err; }
Signature · 1 line
2.45.2
Re: [PATCH v1 2/4] mm: Cleanup apply_to_pte_range() routine
Quoted 13 lines · Alexander Gordeev
Reverse 'create' vs 'mm ==
&init_mm' conditions and move page table mask modification out of the atomic context.Signed-off-by Alexander Gordeev <[email protected]>--- mm/memory.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-)mm/memory.c +0
-0
mm/memory.c static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, · line 2918
Quoted 4 lines
mm/memory.c +1
-0@@ -2915,24 +2915,28 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, pte_fn_t fn, void *data, bool create, pgtbl_mod_mask *mask) { + int err = create ? -ENOMEM : -EINVAL;
Could you make this a new variable instead of reusing existing err? 'const int pte_err' or something?
Quoted 36 lines
mm/memory.c +13
-10pte_t *pte, *mapped_pte; -int err =0 ;spinlock_t *ptl; -if (create) { - mapped_pte = pte = (mm == &init_mm) ? -pte_alloc_kernel_track (pmd, addr, mask) : -pte_alloc_map_lock (mm, pmd, addr, &ptl); +if (mm == &init_mm) { +if (create) + pte =pte_alloc_kernel_track (pmd, addr, mask); +else + pte =pte_offset_kernel (pmd, addr);if (!pte) -return -ENOMEM ; +return err; }else { - mapped_pte = pte = (mm == &init_mm) ? -pte_offset_kernel (pmd, addr) : -pte_offset_map_lock (mm, pmd, addr, &ptl); +if (create) + pte =pte_alloc_map_lock (mm, pmd, addr, &ptl); +else + pte =pte_offset_map_lock (mm, pmd, addr, &ptl);if (!pte) -return -EINVAL ; +return err; + mapped_pte = pte; } + err =0 ;arch_enter_lazy_mmu_mode ();if (fn) {@@ -2944,12 +2948,14 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, }
mm/memory.c static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, · line 2957
Quoted 10 lines
mm/memory.c +2
-1}while (addr +=PAGE_SIZE , addr != end); } - *mask |=PGTBL_PTE_MODIFIED ;arch_leave_lazy_mmu_mode ();if (mm != &init_mm)pte_unmap_unlock (mapped_pte, ptl); + + *mask |=PGTBL_PTE_MODIFIED ;
This is done just because we might as well? Less work in critical section?
Quoted 4 lines
mm/memory.c +1
-0+return err; }
Alexander Gordeev
maintainerRe: [PATCH v1 2/4] mm: Cleanup apply_to_pte_range() routine
Quoted 9 lines · Nicholas Piggin
Quoted 8 lines · Alexander Gordeev
Reverse 'create' vs 'mm ==
&init_mm' conditions and move page table mask modification out of the atomic context.Signed-off-by Alexander Gordeev <[email protected]>--- mm/memory.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-)
Quoted 12 lines
mm/memory.c static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, · line 2918
Quoted 9 lines
mm/memory.c +1
-0@@ -2915,24 +2915,28 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, pte_fn_t fn, void *data, bool create, pgtbl_mod_mask *mask) { + int err = create ? -ENOMEM : -EINVAL;Could you make this a new variable instead of reusing existing err? 'const int
pte_err' or something?
Will do, when/if repost.
...
Quoted 8 lines
Quoted 8 lines
mm/memory.c +0
-1@@ -2944,12 +2948,14 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, } }while (addr +=PAGE_SIZE , addr != end); } - *mask |=PGTBL_PTE_MODIFIED ;arch_leave_lazy_mmu_mode ();
Quoted 7 lines
mm/memory.c static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, · line 2957
Quoted 4 lines
mm/memory.c +2
-0if (mm != &init_mm)pte_unmap_unlock (mapped_pte, ptl); + + *mask |=PGTBL_PTE_MODIFIED ;This is done just because we might as well? Less work in critical section?
Yes.
Thanks!
Alexander Gordeev
maintainerPATCH 3/4 mm: Protect kernel pgtables in apply_to_pte_range()
In reply to Alexander GordeevThe lazy MMU mode can only be entered and left under the protection of the page table locks for all page tables which may be modified. Yet, when it comes to kernel mappings apply_to_pte_range() does not take any locks. That does not conform arch_enter|leave_lazy_mmu_mode() semantics and could potentially lead to re-schedulling a process while in lazy MMU mode or racing on a kernel page table updates.
--- mm/kasan/shadow.c | 7 ++----- mm/memory.c | 5 ++++- 2 files changed, 6 insertions(+), 6 deletions(-)
mm/kasan/shadow.c +2 -5
@@ -308,14 +308,14 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,__memset ((void *)page,KASAN_VMALLOC_INVALID ,PAGE_SIZE ); pte =pfn_pte (PFN_DOWN (__pa (page)),PAGE_KERNEL ); -spin_lock (&init_mm.page_table_lock);if (likely (pte_none (ptep_get (ptep)))) {set_pte_at (&init_mm, addr, ptep, pte); page =0 ; } -spin_unlock (&init_mm.page_table_lock); +if (page)free_page (page); +return 0 ; }
@@ -401,13 +401,10 @@ static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, page = (unsigned long )__va (pte_pfn (ptep_get (ptep)) <<PAGE_SHIFT ); -spin_lock (&init_mm.page_table_lock); -if (likely (!pte_none (ptep_get (ptep)))) {pte_clear (&init_mm, addr, ptep);free_page (page); } -spin_unlock (&init_mm.page_table_lock);return 0 ; }
mm/memory.c +4 -1
@@ -2926,6 +2926,7 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, pte =pte_offset_kernel (pmd, addr);if (!pte)return err; +spin_lock (&init_mm.page_table_lock); }else {if (create) pte =pte_alloc_map_lock (mm, pmd, addr, &ptl);
@@ -2951,7 +2952,9 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd,arch_leave_lazy_mmu_mode (); -if (mm != &init_mm) +if (mm == &init_mm) +spin_unlock (&init_mm.page_table_lock); +else pte_unmap_unlock (mapped_pte, ptl); *mask |=PGTBL_PTE_MODIFIED ;
Signature · 1 line
2.45.2
Re: [PATCH v1 3/4] mm: Protect kernel pgtables in apply_to_pte_range()
Quoted 24 lines · Alexander Gordeev
The lazy MMU mode can only be entered and left under the protection of the page table locks for all page tables which may be modified. Yet, when it comes to kernel mappings
apply_to_pte_range()does not take any locks. That does not conform arch_enter|leave_lazy_mmu_mode() semantics and could potentially lead to re-schedulling a process while in lazy MMU mode or racing on a kernel page table updates.Signed-off-by Alexander Gordeev <[email protected]>--- mm/kasan/shadow.c | 7 ++----- mm/memory.c | 5 ++++- 2 files changed, 6 insertions(+), 6 deletions(-)mm/kasan/shadow.c +0
-1@@ -308,14 +308,14 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr,__memset ((void *)page,KASAN_VMALLOC_INVALID ,PAGE_SIZE ); pte =pfn_pte (PFN_DOWN (__pa (page)),PAGE_KERNEL ); -spin_lock (&init_mm.page_table_lock);if (likely (pte_none (ptep_get (ptep)))) {set_pte_at (&init_mm, addr, ptep, pte);
mm/kasan/shadow.c static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, · line 321
Quoted 10 lines
mm/kasan/shadow.c +2
-1page =0 ; } -spin_unlock (&init_mm.page_table_lock); +if (page)free_page (page); +return 0 ; }
kasan_populate_vmalloc_pte() is really the only thing that takes the ptl in the apply_to_page_range fn()... Looks like you may be right. I wonder why they do and nobody else? Just luck?
Seems okay.
Quoted 37 lines
mm/kasan/shadow.c +0
-3@@ -401,13 +401,10 @@ static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, page = (unsigned long )__va (pte_pfn (ptep_get (ptep)) <<PAGE_SHIFT ); -spin_lock (&init_mm.page_table_lock); -if (likely (!pte_none (ptep_get (ptep)))) {pte_clear (&init_mm, addr, ptep);free_page (page); } -spin_unlock (&init_mm.page_table_lock);return 0 ; }mm/memory.c +4
-1@@ -2926,6 +2926,7 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd, pte =pte_offset_kernel (pmd, addr);if (!pte)return err; +spin_lock (&init_mm.page_table_lock); }else {if (create) pte =pte_alloc_map_lock (mm, pmd, addr, &ptl);@@ -2951,7 +2952,9 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd,arch_leave_lazy_mmu_mode (); -if (mm != &init_mm) +if (mm == &init_mm) +spin_unlock (&init_mm.page_table_lock); +else pte_unmap_unlock (mapped_pte, ptl); *mask |=PGTBL_PTE_MODIFIED ;
Alexander Gordeev
maintainerPATCH 4/4 mm: Allow detection of wrong arch_enter_lazy_mmu_mode() context
In reply to Alexander GordeevThe lazy MMU batching may be only be entered and left under the protection of the page table locks for all page tables which may be modified. Yet, there were cases arch_enter_lazy_mmu_mode() was called without the locks taken, e.g. commit b9ef323ea168 ("powerpc/64s: Disable preemption in hash lazy mmu mode").
Make default arch_enter|leave|flush_lazy_mmu_mode() callbacks complain at least in case the preemption is enabled to detect wrong contexts.
Most platforms do not implement the callbacks, so to aovid a performance impact allow the complaint when CONFIG_DEBUG_VM option is enabled only.
--- include/linux/pgtable.h | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)
include/linux/pgtable.h +12 -3
@@ -232,9 +232,18 @@ static inline int pmd_dirty(pmd_t pmd)* and the mode cannot be used in interrupt context. */ #ifndef __HAVE_ARCH_ENTER_LAZY_MMU_MODE -#define arch_enter_lazy_mmu_mode () do {} while (0) -#define arch_leave_lazy_mmu_mode () do {} while (0) -#define arch_flush_lazy_mmu_mode () do {} while (0) +static inline void arch_enter_lazy_mmu_mode (void ) +{ +VM_WARN_ON (preemptible ()); +} +static inline void arch_leave_lazy_mmu_mode (void ) +{ +VM_WARN_ON (preemptible ()); +} +static inline void arch_flush_lazy_mmu_mode (void ) +{ +VM_WARN_ON (preemptible ()); +}#endif #ifndef pte_batch_hint
Signature · 1 line
2.45.2
Re: [PATCH v1 4/4] mm: Allow detection of wrong arch_enter_lazy_mmu_mode() context
Quoted 10 lines · Alexander Gordeev
The lazy MMU batching may be only be entered and left under the protection of the page table locks for all page tables which may be modified. Yet, there were cases
arch_enter_lazy_mmu_mode()was called without the locks taken, e.g. commitb9ef323ea168("powerpc/64s: Disable preemption in hash lazy mmu mode").Make default arch_enter|leave|flush_lazy_mmu_mode() callbacks complain at least in case the preemption is enabled to detect wrong contexts.
Quoted 5 lines
Most platforms do not implement the callbacks, so to aovid a performance impact allow the complaint when
CONFIG_DEBUG_VMoption is enabled only.Signed-off-by Alexander Gordeev <[email protected]>
This is a good debugging feature independent of how the fix is done. I would just warn once, since it's not a bug for the arch and could fire frequently if it fires at all.
Quoted 30 lines
--- include/linux/pgtable.h | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-)include/linux/pgtable.h +12
-3@@ -232,9 +232,18 @@ static inline int pmd_dirty(pmd_t pmd)* and the mode cannot be used in interrupt context. */ #ifndef __HAVE_ARCH_ENTER_LAZY_MMU_MODE -#define arch_enter_lazy_mmu_mode () do {} while (0) -#define arch_leave_lazy_mmu_mode () do {} while (0) -#define arch_flush_lazy_mmu_mode () do {} while (0) +static inline void arch_enter_lazy_mmu_mode (void ) +{ +VM_WARN_ON (preemptible ()); +} +static inline void arch_leave_lazy_mmu_mode (void ) +{ +VM_WARN_ON (preemptible ()); +} +static inline void arch_flush_lazy_mmu_mode (void ) +{ +VM_WARN_ON (preemptible ()); +}#endif #ifndef pte_batch_hint
Andrew Morton
maintainerQuoted 11 lines · On Mon, 7 Apr 2025 17:11:26 +0200 Alexander Gordeev <[email protected]>…
This series is an attempt to fix the violation of lazy MMU mode context requirement as described for
arch_enter_lazy_mmu_mode():This mode can only be entered and left under the protection of the page table locks for all page tables which may be modified.
On s390 if I make
arch_enter_lazy_mmu_mode()->preempt_enable()andarch_leave_lazy_mmu_mode()->preempt_disable()I am getting this:...
Could you please reorganize this into two series? One series which should be fast-tracked into 6.15-rcX and one series for 6.16-rc1?
And in the first series, please suggest whether its patches should be backported into -stable and see if we can come up with suitable Fixes:
targets?
Thanks.
Quoted 39 lines · Alexander Gordeev
Hi All,
This series is an attempt to fix the violation of lazy MMU mode context requirement as described for
arch_enter_lazy_mmu_mode():This mode can only be entered and left under the protection of the page table locks for all page tables which may be modified.
On s390 if I make
arch_enter_lazy_mmu_mode()->preempt_enable()andarch_leave_lazy_mmu_mode()->preempt_disable()I am getting this:[ 553.332108] preempt_count: 1, expected: 0 [ 553.332117] no locks held by multipathd/2116. [ 553.332128] CPU: 24 PID: 2116 Comm: multipathd Kdump: loaded Tainted: [ 553.332139] Hardware name: IBM 3931 A01 701 (LPAR) [ 553.332146] Call Trace: [ 553.332152] [<00000000158de23a>] dump_stack_lvl+0xfa/0x150 [ 553.332167] [<0000000013e10d12>] __might_resched+0x57a/0x5e8 [ 553.332178] [<00000000144eb6c2>] __alloc_pages+0x2ba/0x7c0 [ 553.332189] [<00000000144d5cdc>] __get_free_pages+0x2c/0x88 [ 553.332198] [<00000000145663f6>] kasan_populate_vmalloc_pte+0x4e/0x110 [ 553.332207] [<000000001447625c>] apply_to_pte_range+0x164/0x3c8 [ 553.332218] [<000000001448125a>] apply_to_pmd_range+0xda/0x318 [ 553.332226] [<000000001448181c>] __apply_to_page_range+0x384/0x768 [ 553.332233] [<0000000014481c28>] apply_to_page_range+0x28/0x38 [ 553.332241] [<00000000145665da>] kasan_populate_vmalloc+0x82/0x98 [ 553.332249] [<00000000144c88d0>] alloc_vmap_area+0x590/0x1c90 [ 553.332257] [<00000000144ca108>] __get_vm_area_node.constprop.0+0x138/0x260 [ 553.332265] [<00000000144d17fc>] __vmalloc_node_range+0x134/0x360 [ 553.332274] [<0000000013d5dbf2>] alloc_thread_stack_node+0x112/0x378 [ 553.332284] [<0000000013d62726>] dup_task_struct+0x66/0x430 [ 553.332293] [<0000000013d63962>] copy_process+0x432/0x4b80 [ 553.332302] [<0000000013d68300>] kernel_clone+0xf0/0x7d0 [ 553.332311] [<0000000013d68bd6>] __do_sys_clone+0xae/0xc8 [ 553.332400] [<0000000013d68dee>] __s390x_sys_clone+0xd6/0x118 [ 553.332410] [<0000000013c9d34c>] do_syscall+0x22c/0x328 [ 553.332419] [<00000000158e7366>] __do_syscall+0xce/0xf0 [ 553.332428] [<0000000015913260>] system_call+0x70/0x98
Quoted 8 lines
This exposes a KASAN issue fixed with patch 1 and
apply_to_pte_range()issue fixed with patches 2-3. Patch 4 is a debug improvement on top, that could have helped to notice the issue.Commit
b9ef323ea168("powerpc/64s: Disable preemption in hash lazy mmu mode") looks like powerpc-only fix, yet not entirely conforming to the above provided requirement (page tables itself are still not protected). If I am not mistaken, xen and sparc are alike.
Huh. powerpc actually has some crazy code in __switch_to() that is supposed to handle preemption while in lazy mmu mode. So we probably don't even need to disable preemption, just use the raw per-cpu accessors (or keep disabling preemption and remove the now dead code from context switch).
IIRC all this got built up over a long time with some TLB flush rules changing at the same time, we could probably stay in lazy mmu mode for a longer time until it was discovered we really need to flush before dropping the PTL.
ppc64 and sparc I think don't even need lazy mmu mode for kasan (TLBs do not require flushing) and will function just fine if not in lazy mode (they just flush one TLB at a time), not sure about xen. We could actually go the other way and require that archs operate properly when not in lazy mode (at least for kernel page tables) and avoid it for apply_to_page_range()?
Thanks,
Nick
Alexander Gordeev
maintainerOn Fri, Apr 11, 2025 at 05:12:28PM +1000, Nicholas Piggin wrote:
...
Quoted 5 lines
Huh. powerpc actually has some crazy code in
__switch_to()that is supposed to handle preemption while in lazy mmu mode. So we probably don't even need to disable preemption, just use the raw per-cpu accessors (or keep disabling preemption and remove the now dead code from context switch).
Well, I tried to do the latter ;)
https://lore.kernel.org/linuxppc-dev/3b4e3e28172f09165b19ee7cac67a860d7cc1c6e.1742915600.git.agordeev@linux.ibm.com/
Not sure how it is aligned with the current state (see below).
Quoted 11 lines
IIRC all this got built up over a long time with some TLB flush rules changing at the same time, we could probably stay in lazy mmu mode for a longer time until it was discovered we really need to flush before dropping the PTL.
ppc64 and sparc I think don't even need lazy mmu mode for kasan (TLBs do not require flushing) and will function just fine if not in lazy mode (they just flush one TLB at a time), not sure about xen. We could actually go the other way and require that archs operate properly when not in lazy mode (at least for kernel page tables) and avoid it for
apply_to_page_range()?
Ryan Roberts hopefully brought some order to the topic:
https://lore.kernel.org/linux-mm/[email protected]/
Quoted 2 lines
Thanks,
Nick