Skip to contentSkip to conversation

mm: folio_zero_user: clear contiguous pages

PATCH Ankur Arora 1 participant 1 message Oct 28, 2025 Memory management

lore
Share & tools
lore
#0 lore raw

References: <[email protected]>
<[email protected]>
User-agent: mu4e 1.4.10; emacs 27.2
In-reply-to: <[email protected]>

Quoted 31 lines · Andrew Morton
Quoted 24 lines · On Mon, 27 Oct 2025 13:21:02 -0700 Ankur Arora <[email protected]> wrote:

This series adds clearing of contiguous page ranges for hugepages, improving on the current page-at-a-time approach in two ways:

- amortizes the per-page setup cost over a larger extent
- when using string instructions, exposes the real region size to the processor.

A processor could use a knowledge of the extent to optimize the clearing. AMD Zen uarchs, as an example, elide allocation of cachelines for regions larger than L3-size.

Demand faulting a 64GB region shows performance improvements:

 $ perf bench mem map -p $pg-sz -f demand -s 64GB -l 5

baseline +series change

(GB/s +- %stdev) (GB/s +- %stdev)

   pg-sz=2MB       12.92  +- 2.55%        17.03  +-  0.70%       + 31.8%	preempt=*

   pg-sz=1GB       17.14  +- 2.27%        18.04  +-  1.05% [#]   +  5.2%	preempt=none|voluntary
   pg-sz=1GB       17.26  +- 1.24%        42.17  +-  4.21%       +144.3%	preempt=full|lazy
Quoted 4 lines

[#] Milan uses a threshold of LLC-size (~32MB) for eliding cacheline allocation, which is higher than the maximum extent used on x86 (ARCH_CONTIG_PAGE_NR=8MB), so preempt=none|voluntary sees no improvement with pg-sz=1GB.

Quoted 8 lines

I wasn't understanding this preemption thing at all, but then I saw this in the v4 series changelogging:

: [#] Only with preempt=full|lazy because cooperatively preempted models : need regular invocations of cond_resched(). This limits the extent : sizes that can be cleared as a unit.

Please put this back in!!

/me facepalms. Sorry you had to go that far back. Yeah, that doesn't make any kind of sense standalone. Will fix.

Quoted 3 lines

It's possible that we're being excessively aggressive with those cond_resched()s. Have you investigating tuning their frequency so we can use larger extent sizes with these preemption models?

folio_zero_user() does a small part of that: for 2MB pages the clearing is split in three parts with an intervening cond_resched() for each.

This is of course much simpler than the process_huge_page() approach where we do a left right dance around the faulting page.

I had implemented a version of process_huge_page() with larger extent sizes that narrowed as we got closer to the faulting page in [a] (x86 performance was similar to the current series. See [b]).

In hindsight however, that felt too elaborate and probably unnecessary on most modern systems where you have reasonably large caches. Where it might help, however, is on more cache constrained systems where the spatial locality really does matter.

So, my idea was to start with a simple version, get some testing and then fill in the gaps instead of starting with something like [a].

[a] https://lore.kernel.org/lkml/[email protected]/#r
[b] https://lore.kernel.org/lkml/[email protected]/

Quoted 14 lines
Quoted 14 lines

The anon-w-seq test in the vm-scalability benchmark, however, does show worse performance with utime increasing by ~9%:

stime utime

  baseline         1654.63 ( +- 3.84% )     811.00 ( +- 3.84% )
  +series          1630.32 ( +- 2.73% )     886.37 ( +- 5.19% )

In part this is because anon-w-seq runs with 384 processes zeroing anonymously mapped memory which they then access sequentially. As such this is a likely uncommon pattern where the memory bandwidth is saturated while also being cache limited because we access the entire region.

Quoted 4 lines
Quoted 1 line

Raghavendra also tested previous version of the series on AMD Genoa [1].

I suggest you paste Raghavendra's results into this [0/N] - it's important material.

Will do.

Quoted 35 lines
Quoted 26 lines

...

 arch/alpha/include/asm/page.h      |  1 -
 arch/arc/include/asm/page.h        |  2 +
 arch/arm/include/asm/page-nommu.h  |  1 -
 arch/arm64/include/asm/page.h      |  1 -
 arch/csky/abiv1/inc/abi/page.h     |  1 +
 arch/csky/abiv2/inc/abi/page.h     |  7 ---
 arch/hexagon/include/asm/page.h    |  1 -
 arch/loongarch/include/asm/page.h  |  1 -
 arch/m68k/include/asm/page_mm.h    |  1 +
 arch/m68k/include/asm/page_no.h    |  1 -
 arch/microblaze/include/asm/page.h |  1 -
 arch/mips/include/asm/page.h       |  1 +
 arch/nios2/include/asm/page.h      |  1 +
 arch/openrisc/include/asm/page.h   |  1 -
 arch/parisc/include/asm/page.h     |  1 -
 arch/powerpc/include/asm/page.h    |  1 +
 arch/riscv/include/asm/page.h      |  1 -
 arch/s390/include/asm/page.h       |  1 -
 arch/sparc/include/asm/page_32.h   |  2 +
 arch/sparc/include/asm/page_64.h   |  1 +
 arch/um/include/asm/page.h         |  1 -
 arch/x86/include/asm/page.h        |  6 ---
 arch/x86/include/asm/page_32.h     |  6 +++
Quoted 8 lines
 arch/x86/include/asm/page_64.h     | 64 ++++++++++++++++++-----
 arch/x86/lib/clear_page_64.S       | 39 +++-----------
 arch/xtensa/include/asm/page.h     |  1 -
 include/linux/highmem.h            | 29 +++++++++++
 include/linux/mm.h                 | 69 +++++++++++++++++++++++++
 mm/memory.c                        | 82 ++++++++++++++++++++++--------
 mm/util.c                          | 13 +++++
 30 files changed, 247 insertions(+), 91 deletions(-)
Quoted 1 line

I guess this is an mm.git thing, with x86 acks (please).

Ack that.

Quoted 2 lines

The documented review activity is rather thin at this time so I'll sit this out for a while. Please ping me next week and we can reassess,

Will do. And, thanks for the quick look!

--
ankur
Date: Tue, 28 Oct 2025 10:15:38 -0700
Message-ID: <[email protected]>