Skip to contentSkip to conversation

mm: Add MADV_NONZERO to skip redundant zeroing in MADV_DONTNEED

RFC Ayaan Mirza Baig 3 participants 3 messages Jun 4, 2025 Memory management

lore
Share & tools
lore
#0 lore raw

Hi Andrew,

I've been looking at MADV_DONTNEED and noticed we always zero pages, even when it might not be necessary for cases like:

- Media buffers (video/audio processing)
- Temporary compute workloads
- Pre-zeroed memory pools

This causes measurable overhead - in some synthetic tests I'm seeing ~40% extra latency from the zeroing. I'm thinking of adding a MADV_NONZERO flag that would let us skip the zeroing when:

1. It's explicitly requested (MADV_DONTNEED|MADV_NONZERO)
2. We're confident it's safe (VM_IO, VM_HUGETLB, etc)
3. The user has explicitly marked the region as non-sensitive

Before I go deeper on patches, I wanted to check:
- Does this sound reasonable?
- Any obvious security holes I'm missing?
- Would you prefer a new flag or a separate advice type?

Early prototype numbers look promising, but I want to make sure I'm not missing something fundamental. If the concept seems viable, I'll work up proper benchmarks and formal patches.

Thanks for your time!
- Ayaan Mirza Baig

#1 lore raw
Quoted 9 lines · Ayaan Mirza Baig

Hi Andrew,

I've been looking at MADV_DONTNEED and noticed we always zero pages, even when
it might not be necessary for cases like:

- Media buffers (video/audio processing)
- Temporary compute workloads
- Pre-zeroed memory pools

Hi Ayaan,

I don't think this "skipping zeroing" is limited to MADV_DONTNEED at all?

We don't zero pages during zapping (unless init_on_free is enabled), but zero pages when madvise(MADV_DONTNEED)'d area is accessed again (for anonymous mappings), because that's how faulting anonymous mappings works.

And if you're suggesting that you want to skip zeroing (on some conditions) when faulting an anonymous mapping, we can't do that because applications rely on the guarantee that such pages are zeroed.

Or am I missing something?

Quoted 15 lines

This causes measurable overhead - in some synthetic tests I'm seeing ~40% extra
latency from the zeroing. I'm thinking of adding a MADV_NONZERO flag that would
let us skip the zeroing when:

1. It's explicitly requested (MADV_DONTNEED|MADV_NONZERO)
2. We're confident it's safe (VM_IO, VM_HUGETLB, etc)
3. The user has explicitly marked the region as non-sensitive

Before I go deeper on patches, I wanted to check:
- Does this sound reasonable?
- Any obvious security holes I'm missing?
- Would you prefer a new flag or a separate advice type?

Quoted 5 lines

Early prototype numbers look promising, but I want to make sure I'm not missing
something fundamental. If the concept seems viable, I'll work up proper benchmarks
and formal patches.

...and I'm wondering what does the informal patches do?

Signature · 2 lines
Cheers,
Harry / Hyeonggon
#2 lore raw
Quoted 15 lines · Harry Yoo
Quoted 9 lines · Ayaan Mirza Baig

Hi Andrew,

I've been looking at MADV_DONTNEED and noticed we always zero pages, even when
it might not be necessary for cases like:

- Media buffers (video/audio processing)
- Temporary compute workloads
- Pre-zeroed memory pools

Hi Ayaan,

I don't think this "skipping zeroing" is limited to MADV_DONTNEED at all?

Quoted 8 lines

We don't zero pages during zapping (unless init_on_free is enabled), but zero pages when madvise(MADV_DONTNEED)'d area is accessed again (for anonymous mappings), because that's how faulting anonymous mappings works.

And if you're suggesting that you want to skip zeroing (on some conditions) when faulting an anonymous mapping, we can't do that because applications rely on the guarantee that such pages are zeroed.

Yeah, no way we're adding something like that.

What we discussed last years at plumbers IIRC in Rik's session was having a per-process cache of zapped pages. Essentially, during MADV_DONTNEED we could move the pages to that cache, and on re-access (if the process opted in) we could grab one of these pages from the per-process cache without zeroing them out. On memory pressure we would flush the cache.

There are some challenges (e.g., how to keep these pages migratable/swappable etc).

Signature · 3 lines
Cheers,

David / dhildenb