mm/rmap: initialize nr_pages to 1 at loop start in try_to_unmap_one
Subsystems (3)
PATCH 1/1 mm/rmap: initialize nr_pages to 1 at loop start in try_to_unmap_one
Initialize nr_pages to 1 at the start of each loop iteration, like folio_referenced_one() does.
Without this, nr_pages computed by a previous folio_unmap_pte_batch() call can be reused on a later iteration that does not run folio_unmap_pte_batch() again.
mmap a 64K large folio with MAP_ANONYMOUS | MAP_DROPPABLE, then call madvise(MADV_FREE), then make the last page device-exclusive via HMM_DMIRROR_EXCLUSIVE.
Trigger node reclaim through sysfs. Now, in try_to_unmap_one(), we will first clear the first 15 out of 16 entries mapping the lazyfree folio. This will set nr_pages to 15. In the next pvmw walk, this nr_pages gets reused on a device-exclusive pte, thus potentially corrupting folio refcount/mapcount.
At the moment, I have a userspace program which can make the kernel spit out a trace, but the blow up is in folio_referenced_one(), because there are existing bugs in the interaction between device-private and rmap (which too I am investigating). I did a one liner kernel change to avoid going into folio_referenced_one(), and the kernel blows up at folio_remove_rmap_ptes in try_to_unmap_one which is what I wanted.
Note that the bug is there not since file folio batching but lazyfree folio batching, since device-exclusive only works for anonymous folios.
Userspace visible effect is simply kernel crashing somewhere due to refcount/mapcount corruption.
---
Applies on mm-unstable. This patch was part of
https://lore.kernel.org/all/[email protected]/
mm/rmap.c | 2 ++ 1 file changed, 2 insertions(+)
mm/rmap.c +2 -0
@@ -2030,6 +2030,8 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,mmu_notifier_invalidate_range_start (&range);while (page_vma_mapped_walk (&pvmw)) { + nr_pages =1 ; +/* * If the folio is in an mlock()d vma, we must not swap it out. */
Signature · 1 line
2.43.0
Quoted 27 lines · Dev Jain
Initialize
nr_pagesto 1 at the start of each loop iteration, likefolio_referenced_one()does.Without this,
nr_pagescomputed by a previousfolio_unmap_pte_batch()call can be reused on a later iteration that does not runfolio_unmap_pte_batch()again.mmap a 64K large folio with
MAP_ANONYMOUS|MAP_DROPPABLE, then callmadvise(MADV_FREE), then make the last page device-exclusive viaHMM_DMIRROR_EXCLUSIVE.Trigger node reclaim through sysfs. Now, in
try_to_unmap_one(), we will first clear the first 15 out of 16 entries mapping the lazyfree folio. This will setnr_pagesto 15. In the next pvmw walk, thisnr_pagesgets reused on a device-exclusive pte, thus potentially corrupting folio refcount/mapcount.At the moment, I have a userspace program which can make the kernel spit out a trace, but the blow up is in
folio_referenced_one(), because there are existing bugs in the interaction between device-private and rmap (which too I am investigating). I did a one liner kernel change to avoid going intofolio_referenced_one(), and the kernel blows up atfolio_remove_rmap_ptesintry_to_unmap_onewhich is what I wanted.Note that the bug is there not since file folio batching but lazyfree folio batching, since device-exclusive only works for anonymous folios.
Quoted 8 lines
Userspace visible effect is simply kernel crashing somewhere due to refcount/mapcount corruption.
Fixes 354dffd29575 ("mm: support batched unmap for lazyfree large folios during reclamation")Signed-off-by Dev Jain <[email protected]>Acked-by Barry Song <[email protected]>---
Thanks!
Signature · 3 lines
Cheers, David
Quoted 6 lines · Dev Jain
Initialize
nr_pagesto 1 at the start of each loop iteration, likefolio_referenced_one()does.Without this,
nr_pagescomputed by a previousfolio_unmap_pte_batch()call can be reused on a later iteration that does not runfolio_unmap_pte_batch()again.
Yikes!
Quoted 18 lines
mmap a 64K large folio with
MAP_ANONYMOUS|MAP_DROPPABLE, then callmadvise(MADV_FREE), then make the last page device-exclusive viaHMM_DMIRROR_EXCLUSIVE.Trigger node reclaim through sysfs. Now, in
try_to_unmap_one(), we will first clear the first 15 out of 16 entries mapping the lazyfree folio. This will setnr_pagesto 15. In the next pvmw walk, thisnr_pagesgets reused on a device-exclusive pte, thus potentially corrupting folio refcount/mapcount.At the moment, I have a userspace program which can make the kernel spit out a trace, but the blow up is in
folio_referenced_one(), because there are existing bugs in the interaction between device-private and rmap (which too I am investigating). I did a one liner kernel change to avoid going intofolio_referenced_one(), and the kernel blows up atfolio_remove_rmap_ptesintry_to_unmap_onewhich is what I wanted.
Quoted 5 lines
Note that the bug is there not since file folio batching but lazyfree folio batching, since device-exclusive only works for anonymous folios.
Userspace visible effect is simply kernel crashing somewhere due to refcount/mapcount corruption.
Also yikes!
Thanks for the detailed commit message :)
Quoted 4 lines
Fixes 354dffd29575 ("mm: support batched unmap for lazyfree large folios during reclamation")Signed-off-by Dev Jain <[email protected]>
Thanks, LGTM so:
Quoted 22 lines
Acked-by Barry Song <[email protected]>---
Applies on mm-unstable. This patch was part of
https://lore.kernel.org/all/[email protected]/mm/rmap.c | 2 ++ 1 file changed, 2 insertions(+)mm/rmap.c +2
-1@@ -2030,6 +2030,8 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,mmu_notifier_invalidate_range_start (&range);while (page_vma_mapped_walk (&pvmw)) { + nr_pages =1 ; +/* * If the folio is in an mlock()d vma, we must not swap it out. */ --
Quoted 2 lines
2.43.0
Cheers, Lorenzo