Skip to contentSkip to conversation

mm/damon/ops-common: use a page-aligned address in damon_ptep_mkold()

PATCH In review Nathan Gao 4 participants 4 messages Sep 2

Subsystems (2)
lore
Share & tools
lore

Nathan Gao

author
#0 lore raw

PATCH v4 1/1 mm/damon/ops-common: use a page-aligned address in damon_ptep_mkold()

__damon_va_prepare_access_check() picks a random byte address within the region and stores it in r->sampling_addr. damon_va_mkold() passes it into a page table walk, which hands it to damon_ptep_mkold() as the address of the page to sample:

damon_va_mkold(mm, r->sampling_addr)
damon_va_walk_page_range(mm, addr, addr + 1)
damon_mkold_pmd_entry()
damon_ptep_mkold(pte, vma, addr)
ptep_test_and_clear_young(vma, addr, pte) mmu_notifier_clear_young(mm, addr, addr + PAGE_SIZE)

For arm64, before commit 6f0e1142173a ("arm64: mm: support batch clearing of the young flag for large folios"), the contpte helper walked exactly CONT_PTES entries from the aligned-down page table pointer and used @addr only to pass down to each entry, so an unaligned value was harmless:

        ptep = contpte_align_down(ptep);
        addr = ALIGN_DOWN(addr, CONT_PTE_SIZE);
        for (i = 0; i < CONT_PTES; i++, ptep++, addr += PAGE_SIZE)

Now the range to walk is derived from @addr instead: end = addr + nr * PAGE_SIZE, rounded up to CONT_PTE_SIZE. For a sample in the last page of a contpte block, the sub-page offset puts end just past the block boundary, so the round-up lands a whole block further and the walk clears PTE_AF in CONT_PTES entries beyond the sampled block. For the last block in a page table page, those entries are past the end of that page, so the walk writes into the page that follows.

Triggered by the full 7.1/7.2 kernel selftest suite on arm64 (EC2 c/m6g.4xlarge). The kernel sometimes crashes at or shortly after the DAMON test.

What the overrun does depends on the page that happens to follow the page table, so there is no single signature. If that page is read-only, the write faults in the sampling path itself:

  Unable to handle kernel write to read-only memory at virtual address ffff0003c5d2d000
    FSC = 0x0f: level 3 permission fault
    CM = 0, WnR = 1, TnD = 0, TagAccess = 0
  CPU: 10 UID: 0 PID: 3487 Comm: kdamond.2
  pc : contpte_test_and_clear_young_ptes+0x70/0xc0
  lr : damon_ptep_mkold+0x1e8/0x1f8
  Call trace:
   contpte_test_and_clear_young_ptes+0x70/0xc0 (P)
   damon_mkold_pmd_entry+0x150/0x170
   walk_pmd_range+0x110/0x2b0
   walk_pud_range+0x10c/0x208
   walk_pgd_range+0x134/0x258
   __walk_page_range+0x98/0x1b0
   walk_page_range_vma_unsafe+0x90/0x148
   walk_page_range_vma+0x28/0x40
   damon_va_walk_page_range+0x114/0x2b8
   damon_va_prepare_access_checks+0xec/0x1a8
   kdamond_fn+0x534/0x770
   kthread+0x128/0x138
   ret_from_fork+0x10/0x20

Otherwise the page is writable, the PTE_AF clearing succeeds silently and the damage only surfaces later, in whatever happened to own the page, so the backtrace is unrelated to DAMON and differs between runs.

Pass a page-aligned address to the ptep_test_and_clear_young() call in damon_ptep_mkold(), which is the only place DAMON can reach contpte_test_and_clear_young_ptes() from. Nothing else sees the aligned address, and r->sampling_addr itself is left as is, so the sampling and region bookkeeping semantics are unchanged.

Fixes 6f0e1142173a ("arm64: mm: support batch clearing of the young flag for large folios")
Cc Baolin Wang <[email protected]>
Cc David Hildenbrand (Arm) <[email protected]>
Cc Ryan Roberts <[email protected]>

---
V3 -> V4:
- Align only the address that is passed to ptep_test_and_clear_young(), rather than @addr for the whole of damon_ptep_mkold(), so the mmu_notifier_clear_young() call keeps seeing the sampled address (SJ)

V2 -> V3:
- Move the alignment into damon_ptep_mkold(), instead of aligning in damon_va_mkold() and damon_va_young(). The ptep_test_and_clear_young() call in damon_ptep_mkold() is DAMON's only path to contpte_test_and_clear_young_ptes(), so damon_ptep_mkold() is the closest place in DAMON to the function that requires an aligned address (SJ)

V1 -> V2:
- Align inside damon_va_mkold() and damon_va_young() rather than aligning r->sampling_addr itself, so that sub-page sampling addresses remain possible for future non-PTE access check primitives (SJ)
- Point Fixes: at 6f0e1142173a instead of 3f49584b262c, since the unaligned address was harmless before that commit (SJ)
- Describe how the issue was noticed and what it does to the kernel (SJ)

v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/

 mm/damon/ops-common.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
mm/damon/ops-common.c +6 -1
index fbda70d8ea4d..8fc61d06d358 100644
@@ -61,7 +61,12 @@ void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr
 	 * device aspects.
 	 */
 	if (likely(pte_present(pteval)))
-		young |= ptep_test_and_clear_young(vma, addr, pte);
+		/*
+		 * Arch implementation of ptep_test_and_clear_young() may
+		 * require aligned @addr
+		 */
+		young |= ptep_test_and_clear_young(vma, PAGE_ALIGN_DOWN(addr),
+				pte);
 	young |= mmu_notifier_clear_young(vma->vm_mm, addr, addr + PAGE_SIZE);
 	if (young)
 		folio_set_young(folio);
Signature · 1 line
2.50.1
#1 lore raw
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 15 lines

__damon_va_prepare_access_check() picks a random byte address within the region and stores it in r->sampling_addr. damon_va_mkold() passes it into a page table walk, which hands it to damon_ptep_mkold() as the address of the page to sample:

damon_va_mkold(mm, r->sampling_addr)
damon_va_walk_page_range(mm, addr, addr + 1)
damon_mkold_pmd_entry()
damon_ptep_mkold(pte, vma, addr)
ptep_test_and_clear_young(vma, addr, pte) mmu_notifier_clear_young(mm, addr, addr + PAGE_SIZE)

For arm64, before commit 6f0e1142173a ("arm64: mm: support batch clearing of the young flag for large folios"), the contpte helper walked exactly CONT_PTES entries from the aligned-down page table pointer and

Quoted 6 lines
[ ... ]
Cc Baolin Wang <[email protected]>
Cc David Hildenbrand (Arm) <[email protected]>
Cc Ryan Roberts <[email protected]>

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
maintainer Reviewed-by on v4 1/1
#2 lore raw
In reply to Nathan Gao

On Tue, 1 Sep 2026 20:16:55 -0700 Nathan Gao <[email protected]> wrote:

[...]

Quoted 33 lines

Triggered by the full 7.1/7.2 kernel selftest suite on arm64 (EC2 c/m6g.4xlarge). The kernel sometimes crashes at or shortly after the DAMON test.

What the overrun does depends on the page that happens to follow the page table, so there is no single signature. If that page is read-only, the write faults in the sampling path itself:

  Unable to handle kernel write to read-only memory at virtual address ffff0003c5d2d000
    FSC = 0x0f: level 3 permission fault
    CM = 0, WnR = 1, TnD = 0, TagAccess = 0
  CPU: 10 UID: 0 PID: 3487 Comm: kdamond.2
  pc : contpte_test_and_clear_young_ptes+0x70/0xc0
  lr : damon_ptep_mkold+0x1e8/0x1f8
  Call trace:
   contpte_test_and_clear_young_ptes+0x70/0xc0 (P)
   damon_mkold_pmd_entry+0x150/0x170
   walk_pmd_range+0x110/0x2b0
   walk_pud_range+0x10c/0x208
   walk_pgd_range+0x134/0x258
   __walk_page_range+0x98/0x1b0
   walk_page_range_vma_unsafe+0x90/0x148
   walk_page_range_vma+0x28/0x40
   damon_va_walk_page_range+0x114/0x2b8
   damon_va_prepare_access_checks+0xec/0x1a8
   kdamond_fn+0x534/0x770
   kthread+0x128/0x138
   ret_from_fork+0x10/0x20

Otherwise the page is writable, the PTE_AF clearing succeeds silently and the damage only surfaces later, in whatever happened to own the page, so the backtrace is unrelated to DAMON and differs between runs.

Quoted 5 lines

Pass a page-aligned address to the ptep_test_and_clear_young() call in damon_ptep_mkold(), which is the only place DAMON can reach contpte_test_and_clear_young_ptes() from. Nothing else sees the aligned address, and r->sampling_addr itself is left as is, so the sampling and region bookkeeping semantics are unchanged.

Thank you for finding this nasty bug and sharing this great fix, Nathan. Also appreicate again for your revisioning efforts that accepted my humble and picky revisioning requests.

Quoted 7 lines
Fixes 6f0e1142173a ("arm64: mm: support batch clearing of the young flag for large folios")
Cc Baolin Wang <[email protected]>
Cc David Hildenbrand (Arm) <[email protected]>
Cc Ryan Roberts <[email protected]>
Quoted 22 lines

---
V3 -> V4:
- Align only the address that is passed to ptep_test_and_clear_young(), rather than @addr for the whole of damon_ptep_mkold(), so the mmu_notifier_clear_young() call keeps seeing the sampled address (SJ)

V2 -> V3:
- Move the alignment into damon_ptep_mkold(), instead of aligning in damon_va_mkold() and damon_va_young(). The ptep_test_and_clear_young() call in damon_ptep_mkold() is DAMON's only path to contpte_test_and_clear_young_ptes(), so damon_ptep_mkold() is the closest place in DAMON to the function that requires an aligned address (SJ)

V1 -> V2:
- Align inside damon_va_mkold() and damon_va_young() rather than aligning r->sampling_addr itself, so that sub-page sampling addresses remain possible for future non-PTE access check primitives (SJ)
- Point Fixes: at 6f0e1142173a instead of 3f49584b262c, since the unaligned address was harmless before that commit (SJ)
- Describe how the issue was noticed and what it does to the kernel (SJ)

Quoted 3 lines

v3: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v1: https://lore.kernel.org/all/[email protected]/

FWIW, I insisted on this version to make clear the intention of this pin point important fix. I still believe this minimum and clear change is better as the hotfix.

Meanwhile, to v3, Baolin mentioned it would be prefered to keep page alignment from callers. As I replied to the comment, that makes sense to me. And it seems some DAMON code has unnecessary unaligned address usages that could be problem in future. I will try to revisit such DAMON code and fix/improve as a followup. As a result, the resulting code might more look like v3 or v2. If you are interested, please feel free to send patches, too.

This patch is applied to damon/next [1] tree. If this patch is not added to mm.git in short term (~2 days?), I will ask mm.git maintainer (Andrew Morton) to pick this. So, no action from your side is needed for now. If it seems I also forgot doing that or you cannot wait for my action, please feel free to directly ask that to Andrew.

[1] https://origin.kernel.org/doc/html/latest/mm/damon/maintainer-profile.html#scm-trees

Thanks,
SJ

[...]

Reviewed-by on v4 1/1
#3 lore raw
In reply to Nathan Gao
Quoted 69 lines · Nathan Gao

__damon_va_prepare_access_check() picks a random byte address within the region and stores it in r->sampling_addr. damon_va_mkold() passes it into a page table walk, which hands it to damon_ptep_mkold() as the address of the page to sample:

damon_va_mkold(mm, r->sampling_addr)
damon_va_walk_page_range(mm, addr, addr + 1)
damon_mkold_pmd_entry()
damon_ptep_mkold(pte, vma, addr)
ptep_test_and_clear_young(vma, addr, pte) mmu_notifier_clear_young(mm, addr, addr + PAGE_SIZE)

For arm64, before commit 6f0e1142173a ("arm64: mm: support batch clearing of the young flag for large folios"), the contpte helper walked exactly CONT_PTES entries from the aligned-down page table pointer and used @addr only to pass down to each entry, so an unaligned value was harmless:

         ptep = contpte_align_down(ptep);
         addr = ALIGN_DOWN(addr, CONT_PTE_SIZE);
         for (i = 0; i < CONT_PTES; i++, ptep++, addr += PAGE_SIZE)

Now the range to walk is derived from @addr instead: end = addr + nr * PAGE_SIZE, rounded up to CONT_PTE_SIZE. For a sample in the last page of a contpte block, the sub-page offset puts end just past the block boundary, so the round-up lands a whole block further and the walk clears PTE_AF in CONT_PTES entries beyond the sampled block. For the last block in a page table page, those entries are past the end of that page, so the walk writes into the page that follows.

Triggered by the full 7.1/7.2 kernel selftest suite on arm64 (EC2 c/m6g.4xlarge). The kernel sometimes crashes at or shortly after the DAMON test.

What the overrun does depends on the page that happens to follow the page table, so there is no single signature. If that page is read-only, the write faults in the sampling path itself:

   Unable to handle kernel write to read-only memory at virtual address ffff0003c5d2d000
     FSC = 0x0f: level 3 permission fault
     CM = 0, WnR = 1, TnD = 0, TagAccess = 0
   CPU: 10 UID: 0 PID: 3487 Comm: kdamond.2
   pc : contpte_test_and_clear_young_ptes+0x70/0xc0
   lr : damon_ptep_mkold+0x1e8/0x1f8
   Call trace:
    contpte_test_and_clear_young_ptes+0x70/0xc0 (P)
    damon_mkold_pmd_entry+0x150/0x170
    walk_pmd_range+0x110/0x2b0
    walk_pud_range+0x10c/0x208
    walk_pgd_range+0x134/0x258
    __walk_page_range+0x98/0x1b0
    walk_page_range_vma_unsafe+0x90/0x148
    walk_page_range_vma+0x28/0x40
    damon_va_walk_page_range+0x114/0x2b8
    damon_va_prepare_access_checks+0xec/0x1a8
    kdamond_fn+0x534/0x770
    kthread+0x128/0x138
    ret_from_fork+0x10/0x20

Otherwise the page is writable, the PTE_AF clearing succeeds silently and the damage only surfaces later, in whatever happened to own the page, so the backtrace is unrelated to DAMON and differs between runs.

Pass a page-aligned address to the ptep_test_and_clear_young() call in damon_ptep_mkold(), which is the only place DAMON can reach contpte_test_and_clear_young_ptes() from. Nothing else sees the aligned address, and r->sampling_addr itself is left as is, so the sampling and region bookkeeping semantics are unchanged.

Quoted 7 lines
Fixes 6f0e1142173a ("arm64: mm: support batch clearing of the young flag for large folios")
Cc Baolin Wang <[email protected]>
Cc David Hildenbrand (Arm) <[email protected]>
Cc Ryan Roberts <[email protected]>

---

LGTM.