mm/rmap: fix potential out-of-bounds page table access during batched unmap

From: Lance Yang <[email protected]>

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() can read past the end of a PTE table if a large folio is mapped starting at the last entry of that table. It would be quite rare in practice, as MADV_FREE typically splits the large folio ;)

So let's fix the potential out-of-bounds read by refactoring the logic into a new helper, folio_unmap_pte_batch().

The new helper now correctly calculates the safe number of pages to scan by limiting the operation to the boundaries of the current VMA and the PTE table.

In addition, the "all-or-nothing" batching restriction is removed to support partial batches. The reference counting is also cleaned up to use folio_put_refs().

[1] https://lore.kernel.org/linux-mm/[email protected]

Quoted 15 lines · Lance Yang

From: Lance Yang <[email protected]>

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() can read past the end of a PTE table if a large folio is mapped starting at the last entry of that table. It would be quite rare in practice, as MADV_FREE typically splits the large folio ;)

So let's fix the potential out-of-bounds read by refactoring the logic into a new helper, folio_unmap_pte_batch().

The new helper now correctly calculates the safe number of pages to scan by limiting the operation to the boundaries of the current VMA and the PTE table.

Quoted 6 lines

In addition, the "all-or-nothing" batching restriction is removed to support partial batches. The reference counting is also cleaned up to use folio_put_refs().

[1] https://lore.kernel.org/linux-mm/[email protected]

What about ?

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() may read past the end of a PTE table when a large folio spans across two PMDs, particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the PMD boundary.

It also refactors the logic into a new helper, folio_unmap_pte_batch(), which supports batching between 1 and folio_nr_pages. This improves code clarity. Note that such cases are rare in practice, as MADV_FREE typically splits large folios.

Thanks
Barry

Quoted 32 lines · Barry Song

Quoted 15 lines · Lance Yang

From: Lance Yang <[email protected]>

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() can read past the end of a PTE table if a large folio is mapped starting at the last entry of that table. It would be quite rare in practice, as MADV_FREE typically splits the large folio ;)

So let's fix the potential out-of-bounds read by refactoring the logic into a new helper, folio_unmap_pte_batch().

The new helper now correctly calculates the safe number of pages to scan by limiting the operation to the boundaries of the current VMA and the PTE table.

Quoted 6 lines

In addition, the "all-or-nothing" batching restriction is removed to support partial batches. The reference counting is also cleaned up to use folio_put_refs().

[1] https://lore.kernel.org/linux-mm/[email protected]

What about ?

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() may read past the end of a PTE table when a large folio spans across two PMDs, particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the PMD boundary.

Quoted 4 lines

It also refactors the logic into a new helper, folio_unmap_pte_batch(), which supports batching between 1 and folio_nr_pages. This improves code clarity. Note that such cases are rare in practice, as MADV_FREE typically splits large folios.

Sorry, I meant that MADV_FREE typically splits large folios if the specified range doesn't cover the entire folio.

Quoted 33 lines · Barry Song

Quoted 32 lines · Barry Song

Quoted 15 lines · Lance Yang

From: Lance Yang <[email protected]>

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() can read past the end of a PTE table if a large folio is mapped starting at the last entry of that table. It would be quite rare in practice, as MADV_FREE typically splits the large folio ;)

So let's fix the potential out-of-bounds read by refactoring the logic into a new helper, folio_unmap_pte_batch().

The new helper now correctly calculates the safe number of pages to scan by limiting the operation to the boundaries of the current VMA and the PTE table.

Quoted 6 lines

In addition, the "all-or-nothing" batching restriction is removed to support partial batches. The reference counting is also cleaned up to use folio_put_refs().

[1] https://lore.kernel.org/linux-mm/[email protected]

What about ?

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() may read past the end of a PTE table when a large folio spans across two PMDs, particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the PMD boundary.

Quoted 7 lines

Quoted 4 lines

It also refactors the logic into a new helper, folio_unmap_pte_batch(), which supports batching between 1 and folio_nr_pages. This improves code clarity. Note that such cases are rare in practice, as MADV_FREE typically splits large folios.

Sorry, I meant that MADV_FREE typically splits large folios if the specified range doesn't cover the entire folio.

Hmm... I got it wrong as well :( It's the partial coverage that triggers the split.

how about this revised version:

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() may read past the end of a PTE table when a large folio spans across two PMDs, particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the PMD boundary.

It also refactors the logic into a new helper, folio_unmap_pte_batch(), which supports batching between 1 and folio_nr_pages. This improves code clarity. Note that such boundary-straddling cases are rare in practice, as MADV_FREE will typically split a large folio if the advice range does not cover the entire folio.

Quoted 56 lines · Lance Yang

Quoted 33 lines · Barry Song

Quoted 32 lines · Barry Song

Quoted 15 lines · Lance Yang

From: Lance Yang <[email protected]>

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() can read past the end of a PTE table if a large folio is mapped starting at the last entry of that table. It would be quite rare in practice, as MADV_FREE typically splits the large folio ;)

So let's fix the potential out-of-bounds read by refactoring the logic into a new helper, folio_unmap_pte_batch().

The new helper now correctly calculates the safe number of pages to scan by limiting the operation to the boundaries of the current VMA and the PTE table.

Quoted 6 lines

In addition, the "all-or-nothing" batching restriction is removed to support partial batches. The reference counting is also cleaned up to use folio_put_refs().

[1] https://lore.kernel.org/linux-mm/[email protected]

What about ?

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() may read past the end of a PTE table when a large folio spans across two PMDs, particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the PMD boundary.

Quoted 7 lines

Quoted 4 lines

It also refactors the logic into a new helper, folio_unmap_pte_batch(), which supports batching between 1 and folio_nr_pages. This improves code clarity. Note that such cases are rare in practice, as MADV_FREE typically splits large folios.

Sorry, I meant that MADV_FREE typically splits large folios if the specified range doesn't cover the entire folio.

Hmm... I got it wrong as well :( It's the partial coverage that triggers the split.

how about this revised version:

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() may read past the end of a PTE table when a large folio spans across two PMDs, particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the PMD boundary.

Quoted 5 lines

It also refactors the logic into a new helper, folio_unmap_pte_batch(), which supports batching between 1 and folio_nr_pages. This improves code clarity. Note that such boundary-straddling cases are rare in practice, as MADV_FREE will typically split a large folio if the advice range does not cover the entire folio.

I assume the out-of-bounds access must be fixed, even though it is very unlikely. It might occur after a large folio is marked with MADV_FREE and then remapped to an unaligned address, potentially crossing two PTE tables.

A batch size between 2 and nr_pages - 1 is practically rare, as we typically split large folios when MADV_FREE does not cover the entire folio range. Cases where a batch of size 2 or nr_pages - 1 occurs may only happen if a large folio is partially unmapped after being marked MADV_FREE, which is quite an unusual pattern in userspace.

Quoted 63 lines · Barry Song

Quoted 61 lines · Lance Yang

Quoted 33 lines · Barry Song

Quoted 32 lines · Barry Song

Quoted 15 lines · Lance Yang

From: Lance Yang <[email protected]>

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() can read past the end of a PTE table if a large folio is mapped starting at the last entry of that table. It would be quite rare in practice, as MADV_FREE typically splits the large folio ;)

So let's fix the potential out-of-bounds read by refactoring the logic into a new helper, folio_unmap_pte_batch().

The new helper now correctly calculates the safe number of pages to scan by limiting the operation to the boundaries of the current VMA and the PTE table.

Quoted 6 lines

In addition, the "all-or-nothing" batching restriction is removed to support partial batches. The reference counting is also cleaned up to use folio_put_refs().

[1] https://lore.kernel.org/linux-mm/ [email protected]

What about ?

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() may read past the end of a PTE table when a large folio spans across two PMDs, particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the PMD boundary.

Quoted 7 lines

Quoted 4 lines

It also refactors the logic into a new helper, folio_unmap_pte_batch(), which supports batching between 1 and folio_nr_pages. This improves code clarity. Note that such cases are rare in practice, as MADV_FREE typically splits large folios.

Sorry, I meant that MADV_FREE typically splits large folios if the specified range doesn't cover the entire folio.

Hmm... I got it wrong as well :( It's the partial coverage that triggers the split.

how about this revised version:

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() may read past the end of a PTE table when a large folio spans across two PMDs, particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the PMD boundary.

It also refactors the logic into a new helper, folio_unmap_pte_batch(), which supports batching between 1 and folio_nr_pages. This improves code clarity. Note that such boundary-straddling cases are rare in practice, as MADV_FREE will typically split a large folio if the advice range does not cover the entire folio.

Quoted 3 lines

I assume the out-of-bounds access must be fixed, even though it is very unlikely. It might occur after a large folio is marked with MADV_FREE and then remapped to an unaligned address, potentially crossing two PTE tables.

Right. If it can be triggered from userspace, it doesn't matter how likely/common/whatever it is. It must be fixed.

Quoted 7 lines

A batch size between 2 and nr_pages - 1 is practically rare, as we typically split large folios when MADV_FREE does not cover the entire folio range. Cases where a batch of size 2 or nr_pages - 1 occurs may only happen if a large folio is partially unmapped after being marked MADV_FREE, which is quite an unusual pattern in userspace.

I think the point is rather "Simplify the code by not special-casing for completely mapped folios, there is no real reason why we cannot batch ranges that don't cover the complete folio.".

Quoted 83 lines · David Hildenbrand

Quoted 82 lines · Barry Song

Quoted 80 lines · Lance Yang

Quoted 52 lines · Barry Song

Quoted 43 lines · Barry Song

Quoted 20 lines · Lance Yang

From: Lance Yang <[email protected]>

As pointed out by David[1], the batched unmap logic in try_to_unmap_one()
can read past the end of a PTE table if a large folio is mapped starting at
the last entry of that table. It would be quite rare in practice, as MADV_FREE typically splits the large folio ;)

So let's fix the potential out-of-bounds read by refactoring the logic into
a new helper, folio_unmap_pte_batch().

The new helper now correctly calculates the safe number of pages to scan by
limiting the operation to the boundaries of the current VMA and the PTE
table.

Quoted 8 lines

In addition, the "all-or-nothing" batching restriction is removed to support partial batches. The reference counting is also cleaned up to use
folio_put_refs().

[1] https://lore.kernel.org/linux-mm/
[email protected]

What about ?

As pointed out by David[1], the batched unmap logic in try_to_unmap_one()
may read past the end of a PTE table when a large folio spans across two PMDs,
particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the PMD
boundary.

Quoted 7 lines

It also refactors the logic into a new helper, folio_unmap_pte_batch(),
which supports batching between 1 and folio_nr_pages. This improves code
clarity. Note that such cases are rare in practice, as MADV_FREE typically
splits large folios.

Quoted 3 lines

Sorry, I meant that MADV_FREE typically splits large folios if the specified
range doesn't cover the entire folio.

Hmm... I got it wrong as well :( It's the partial coverage that triggers the split.

how about this revised version:

As pointed out by David[1], the batched unmap logic in try_to_unmap_one()
may read past the end of a PTE table when a large folio spans across two PMDs, particularly after being remapped with mremap(). This patch fixes the potential out-of-bounds access by capping the batch at vm_end and the
PMD boundary.

It also refactors the logic into a new helper, folio_unmap_pte_batch(), which supports batching between 1 and folio_nr_pages. This improves code clarity. Note that such boundary-straddling cases are rare in practice, as
MADV_FREE will typically split a large folio if the advice range does not
cover the entire folio.

Quoted 7 lines

Quoted 4 lines

I assume the out-of-bounds access must be fixed, even though it is very unlikely. It might occur after a large folio is marked with MADV_FREE and then remapped to an unaligned address, potentially crossing two PTE tables.

Right. If it can be triggered from userspace, it doesn't matter how likely/common/whatever it is. It must be fixed.

Agreed. It must be fixed regardless of how rare the scenario is ;)

Quoted 9 lines

Quoted 8 lines

A batch size between 2 and nr_pages - 1 is practically rare, as we typically
split large folios when MADV_FREE does not cover the entire folio range. Cases where a batch of size 2 or nr_pages - 1 occurs may only happen if a large folio is partially unmapped after being marked MADV_FREE, which is quite an unusual pattern in userspace.

Quoted 3 lines

I think the point is rather "Simplify the code by not special-casing for completely mapped folios, there is no real reason why we cannot batch ranges that don't cover the complete folio.".

Yeah. That makes the code cleaner and more generic, as there is no strong reason to special-case for fully mapped folios ;)

Based on that, I think we're on the same page now. I'd like to post the following commit message for the next version:

```
As pointed out by David[1], the batched unmap logic in try_to_unmap_one()
may read past the end of a PTE table when a large folio's PTE mappings
are not fully contained within a single page table.

While this scenario might be rare, an issue triggerable from userspace must
be fixed regardless of its likelihood. This patch fixes the out-of-bounds
access by refactoring the logic into a new helper, folio_unmap_pte_batch().

The new helper correctly calculates the safe batch size by capping the
scan at both the VMA and PMD boundaries. To simplify the code, it also
supports partial batching (i.e., any number of pages from 1 up to the
calculated safe maximum), as there is no strong reason to special-case
for fully mapped folios.
```

So, wdyt?

Based on that, I think we're on the same page now. I'd like to post the following commit message for the next version:

```
As pointed out by David[1], the batched unmap logic in try_to_unmap_one()
may read past the end of a PTE table when a large folio's PTE mappings
are not fully contained within a single page table.

While this scenario might be rare, an issue triggerable from userspace must
be fixed regardless of its likelihood. This patch fixes the out-of-bounds
access by refactoring the logic into a new helper, folio_unmap_pte_batch().

The new helper correctly calculates the safe batch size by capping the
scan at both the VMA and PMD boundaries. To simplify the code, it also
supports partial batching (i.e., any number of pages from 1 up to the
calculated safe maximum), as there is no strong reason to special-case
for fully mapped folios.
```

So, wdyt?

As pointed out by David[1], the batched unmap logic in try_to_unmap_one() can read past the end of a PTE table if a large folio is mapped starting at the last entry of that table. It would be quite rare in practice, as MADV_FREE typically splits the large folio ;)

So let's fix the potential out-of-bounds read by refactoring the logic into a new helper, folio_unmap_pte_batch().

The new helper now correctly calculates the safe number of pages to scan by limiting the operation to the boundaries of the current VMA and the PTE table.

In addition, the "all-or-nothing" batching restriction is removed to support partial batches. The reference counting is also cleaned up to use folio_put_refs().