Skip to contentSkip to conversation

madvise.2: Document MADV_POPULATE_READ and MADV_POPULATE_WRITE

PATCH In review David Hildenbrand 2 participants 3 messages Aug 23, 2021 – Oct 2, 2021

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 madvise.2: Document MADV_POPULATE_READ and MADV_POPULATE_WRITE

MADV_POPULATE_READ and MADV_POPULATE_WRITE have been merged into upstream Linux via commit 4ca9b3859dac ("mm/madvise: introduce MADV_POPULATE_(READ|WRITE) to prefault page tables"), part of v5.14-rc1.

Further, commit eb2faa513c24 ("mm/madvise: report SIGBUS as -EFAULT for MADV_POPULATE_(READ|WRITE)"), part of v5.14-rc6, made sure that SIGBUS is converted to -EFAULT instead of -EINVAL.

Let's document the behavior and error conditions of these new madvise() options.

Cc Alejandro Colomar <[email protected]>
Cc Michael Kerrisk <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Jann Horn <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Linux API <[email protected]>

---

v2 -> v3:
- Refine what "populating readable/writable" means
- Compare each version with MAP_POPULATE and give an example use case
- Reword SIGBUS handling
- Reword comment regarding special mappings and also add memfd_secret(2)
- Reference MADV_HWPOISON when talking about HW poisoned pages
- Minor cosmetic fixes

v1 -> v2:
- Use semantic newlines in all cases
- Add two missing "
- Document -EFAULT handling
- Rephrase some parts to make it more generic: VM_PFNMAP and VM_IO are only examples for special mappings

---
 man2/madvise.2 | 156 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 156 insertions(+)
man2/madvise.2 +156 -0
index f1f384c0c..37f6dd6fa 100644
@@ -469,6 +469,106 @@ If a page is file-backed and dirty, it will be written back to the backing
 storage.
 The advice might be ignored for some pages in the range when it is not
 applicable.
+.TP
+.BR MADV_POPULATE_READ " (since Linux 5.14)"
+"Populate (prefault) page tables readable,
+faulting in all pages in the range just as if manually reading from each page;
+however,
+avoid the actual memory access that would have been performed after handling
+the fault.
+.IP
+In contrast to
+.BR MAP_POPULATE ,
+.B MADV_POPULATE_READ
+does not hide errors,
+can be applied to (parts of) existing mappings and will always populate
+(prefault) page tables readable.
+One example use case is prefaulting a file mapping,
+reading all file content from disk;
+however,
+pages won't be dirtied and consequently won't have to be written back to disk
+when evicting the pages from memory.
+.IP
+Depending on the underlying mapping,
+map the shared zeropage,
+preallocate memory or read the underlying file;
+files with holes might or might not preallocate blocks.
+If populating fails,
+a
+.B SIGBUS
+signal is not generated; instead, an error is returned.
+.IP
+If
+.B MADV_POPULATE_READ
+succeeds,
+all page tables have been populated (prefaulted) readable once.
+If
+.B MADV_POPULATE_READ
+fails,
+some page tables might have been populated.
+.IP
+.B MADV_POPULATE_READ
+cannot be applied to mappings without read permissions
+and special mappings,
+for example,
+mappings marked with kernel-internal flags such as
+.B VM_PFNMAP
+or
+.BR VM_IO ,
+or secret memory regions created using
+.BR memfd_secret(2) .
+.IP
+Note that with
+.BR MADV_POPULATE_READ ,
+the process can be killed at any moment when the system runs out of memory.
+.TP
+.BR MADV_POPULATE_WRITE " (since Linux 5.14)"
+Populate (prefault) page tables writable,
+faulting in all pages in the range just as if manually writing to each
+each page;
+however,
+avoid the actual memory access that would have been performed after handling
+the fault.
+.IP
+In contrast to
+.BR MAP_POPULATE ,
+MADV_POPULATE_WRITE does not hide errors,
+can be applied to (parts of) existing mappings and will always populate
+(prefault) page tables writable.
+One example use case is preallocating memory,
+breaking any CoW (Copy on Write).
+.IP
+Depending on the underlying mapping,
+preallocate memory or read the underlying file;
+files with holes will preallocate blocks.
+If populating fails,
+a
+.B SIGBUS
+signal is not generated; instead, an error is returned.
+.IP
+If
+.B MADV_POPULATE_WRITE
+succeeds,
+all page tables have been populated (prefaulted) writable once.
+If
+.B MADV_POPULATE_WRITE
+fails,
+some page tables might have been populated.
+.IP
+.B MADV_POPULATE_WRITE
+cannot be applied to mappings without write permissions
+and special mappings,
+for example,
+mappings marked with kernel-internal flags such as
+.B VM_PFNMAP
+or
+.BR VM_IO ,
+or secret memory regions created using
+.BR memfd_secret(2) .
+.IP
+Note that with
+.BR MADV_POPULATE_WRITE ,
+the process can be killed at any moment when the system runs out of memory.
 .SH RETURN VALUE
 On success,
 .BR madvise ()
@@ -490,6 +590,22 @@ A kernel resource was temporarily unavailable.
 .B EBADF
 The map exists, but the area maps something that isn't a file.
 .TP
+.B EFAULT
+.I advice
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+and populating (prefaulting) page tables failed because a
+.B SIGBUS
+would have been generated on actual memory access and the reason is not a
+HW poisoned page
+(HW poisoned pages can,
+for example,
+be created using the
+.B MADV_HWPOISON
+flag described elsewhere in this page).
+.TP
 .B EINVAL
 .I addr
 is not page-aligned or
@@ -533,6 +649,22 @@ or
 .BR VM_PFNMAP
 ranges.
 .TP
+.B EINVAL
+.I advice
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+but the specified address range includes ranges with insufficient permissions
+or special mappings,
+for example,
+mappings marked with kernel-internal flags such a
+.B VM_IO
+or
+.BR VM_PFNMAP ,
+or secret memory regions created using
+.BR memfd_secret(2) .
+.TP
 .B EIO
 (for
 .BR MADV_WILLNEED )
@@ -548,6 +680,15 @@ Not enough memory: paging in failed.
 Addresses in the specified range are not currently
 mapped, or are outside the address space of the process.
 .TP
+.B ENOMEM
+.I advice
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+and populating (prefaulting) page tables failed because there was not enough
+memory.
+.TP
 .B EPERM
 .I advice
 is
@@ -555,6 +696,20 @@ is
 but the caller does not have the
 .B CAP_SYS_ADMIN
 capability.
+.TP
+.B EHWPOISON
+.I advice
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+and populating (prefaulting) page tables failed because a HW poisoned page
+(HW poisoned pages can,
+for example,
+be created using the
+.B MADV_HWPOISON
+flag described elsewhere in this page)
+was encountered.
 .SH VERSIONS
 Since Linux 3.18,
 .\" commit d3ac21cacc24790eb45d735769f35753f5b56ceb
@@ -602,6 +757,7 @@ from the system call, as it should).
 .\" function first appeared in 4.4BSD.
 .SH SEE ALSO
 .BR getrlimit (2),
+.BR memfd_secret(2),
 .BR mincore (2),
 .BR mmap (2),
 .BR mprotect (2),
Signature · 1 line
2.31.1
#1 lore raw
Quoted 239 lines · David Hildenbrand

MADV_POPULATE_READ and MADV_POPULATE_WRITE have been merged into upstream Linux via commit 4ca9b3859dac ("mm/madvise: introduce MADV_POPULATE_(READ|WRITE) to prefault page tables"), part of v5.14-rc1.

Further, commit eb2faa513c24 ("mm/madvise: report SIGBUS as -EFAULT for MADV_POPULATE_(READ|WRITE)"), part of v5.14-rc6, made sure that SIGBUS is converted to -EFAULT instead of -EINVAL.

Let's document the behavior and error conditions of these new madvise() options.

Cc Alejandro Colomar <[email protected]>
Cc Michael Kerrisk <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Jann Horn <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Linux API <[email protected]>

---

v2 -> v3:
- Refine what "populating readable/writable" means
- Compare each version with MAP_POPULATE and give an example use case
- Reword SIGBUS handling
- Reword comment regarding special mappings and also add memfd_secret(2)
- Reference MADV_HWPOISON when talking about HW poisoned pages
- Minor cosmetic fixes

v1 -> v2:
- Use semantic newlines in all cases
- Add two missing "
- Document -EFAULT handling
- Rephrase some parts to make it more generic: VM_PFNMAP and VM_IO are only examples for special mappings

---
  man2/madvise.2 | 156 +++++++++++++++++++++++++++++++++++++++++++++++++
  1 file changed, 156 insertions(+)
man2/madvise.2 +155 -0
index f1f384c0c..37f6dd6fa 100644
@@ -469,6 +469,106 @@ If a page is file-backed and dirty, it will be written back to the backing
  storage.
  The advice might be ignored for some pages in the range when it is not
  applicable.
+.TP
+.BR MADV_POPULATE_READ " (since Linux 5.14)"
+"Populate (prefault) page tables readable,
+faulting in all pages in the range just as if manually reading from each page;
+however,
+avoid the actual memory access that would have been performed after handling
+the fault.
+.IP
+In contrast to
+.BR MAP_POPULATE ,
+.B MADV_POPULATE_READ
+does not hide errors,
+can be applied to (parts of) existing mappings and will always populate
+(prefault) page tables readable.
+One example use case is prefaulting a file mapping,
+reading all file content from disk;
+however,
+pages won't be dirtied and consequently won't have to be written back to disk
+when evicting the pages from memory.
+.IP
+Depending on the underlying mapping,
+map the shared zeropage,
+preallocate memory or read the underlying file;
+files with holes might or might not preallocate blocks.
+If populating fails,
+a
+.B SIGBUS
+signal is not generated; instead, an error is returned.
+.IP
+If
+.B MADV_POPULATE_READ
+succeeds,
+all page tables have been populated (prefaulted) readable once.
+If
+.B MADV_POPULATE_READ
+fails,
+some page tables might have been populated.
+.IP
+.B MADV_POPULATE_READ
+cannot be applied to mappings without read permissions
+and special mappings,
+for example,
+mappings marked with kernel-internal flags such as
+.B VM_PFNMAP
+or
+.BR VM_IO ,
+or secret memory regions created using
+.BR memfd_secret(2) .
+.IP
+Note that with
+.BR MADV_POPULATE_READ ,
+the process can be killed at any moment when the system runs out of memory.
+.TP
+.BR MADV_POPULATE_WRITE " (since Linux 5.14)"
+Populate (prefault) page tables writable,
+faulting in all pages in the range just as if manually writing to each
+each page;
+however,
+avoid the actual memory access that would have been performed after handling
+the fault.
+.IP
+In contrast to
+.BR MAP_POPULATE ,
+MADV_POPULATE_WRITE does not hide errors,
+can be applied to (parts of) existing mappings and will always populate
+(prefault) page tables writable.
+One example use case is preallocating memory,
+breaking any CoW (Copy on Write).
+.IP
+Depending on the underlying mapping,
+preallocate memory or read the underlying file;
+files with holes will preallocate blocks.
+If populating fails,
+a
+.B SIGBUS
+signal is not generated; instead, an error is returned.
+.IP
+If
+.B MADV_POPULATE_WRITE
+succeeds,
+all page tables have been populated (prefaulted) writable once.
+If
+.B MADV_POPULATE_WRITE
+fails,
+some page tables might have been populated.
+.IP
+.B MADV_POPULATE_WRITE
+cannot be applied to mappings without write permissions
+and special mappings,
+for example,
+mappings marked with kernel-internal flags such as
+.B VM_PFNMAP
+or
+.BR VM_IO ,
+or secret memory regions created using
+.BR memfd_secret(2) .
+.IP
+Note that with
+.BR MADV_POPULATE_WRITE ,
+the process can be killed at any moment when the system runs out of memory.
  .SH RETURN VALUE
  On success,
  .BR madvise ()
@@ -490,6 +590,22 @@ A kernel resource was temporarily unavailable.
  .B EBADF
  The map exists, but the area maps something that isn't a file.
  .TP
+.B EFAULT
+.I advice
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+and populating (prefaulting) page tables failed because a
+.B SIGBUS
+would have been generated on actual memory access and the reason is not a
+HW poisoned page
+(HW poisoned pages can,
+for example,
+be created using the
+.B MADV_HWPOISON
+flag described elsewhere in this page).
+.TP
  .B EINVAL
  .I addr
  is not page-aligned or
@@ -533,6 +649,22 @@ or
  .BR VM_PFNMAP
  ranges.
  .TP
+.B EINVAL
+.I advice
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+but the specified address range includes ranges with insufficient permissions
+or special mappings,
+for example,
+mappings marked with kernel-internal flags such a
+.B VM_IO
+or
+.BR VM_PFNMAP ,
+or secret memory regions created using
+.BR memfd_secret(2) .
+.TP
  .B EIO
  (for
  .BR MADV_WILLNEED )
@@ -548,6 +680,15 @@ Not enough memory: paging in failed.
  Addresses in the specified range are not currently
  mapped, or are outside the address space of the process.
  .TP
+.B ENOMEM
+.I advice
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+and populating (prefaulting) page tables failed because there was not enough
+memory.
+.TP
  .B EPERM
  .I advice
  is
@@ -555,6 +696,20 @@ is
  but the caller does not have the
  .B CAP_SYS_ADMIN
  capability.
+.TP
+.B EHWPOISON
+.I advice
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+and populating (prefaulting) page tables failed because a HW poisoned page
+(HW poisoned pages can,
+for example,
+be created using the
+.B MADV_HWPOISON
+flag described elsewhere in this page)
+was encountered.
  .SH VERSIONS
  Since Linux 3.18,
  .\" commit d3ac21cacc24790eb45d735769f35753f5b56ceb

man2/madvise.2 from the system call, as it should). · line 763

Quoted 7 lines
man2/madvise.2 +1 -0
@@ -602,6 +757,7 @@ from the system call, as it should).
  .\" function first appeared in 4.4BSD.
  .SH SEE ALSO
  .BR getrlimit (2),
+.BR memfd_secret(2),
  .BR mincore (2),
  .BR mmap (2),
  .BR mprotect (2),

Gentle ping.

Signature · 3 lines
Thanks,

David / dhildenb
#2 lore raw

Hi David,

Quoted 12 lines · David Hildenbrand
Quoted 11 lines · David Hildenbrand

MADV_POPULATE_READ and MADV_POPULATE_WRITE have been merged into upstream Linux via commit 4ca9b3859dac ("mm/madvise: introduce MADV_POPULATE_(READ|WRITE) to prefault page tables"), part of v5.14-rc1.

Further, commit eb2faa513c24 ("mm/madvise: report SIGBUS as -EFAULT for MADV_POPULATE_(READ|WRITE)"), part of v5.14-rc6, made sure that SIGBUS is converted to -EFAULT instead of -EINVAL.

Let's document the behavior and error conditions of these new madvise() options.

Quoted 12 lines
Quoted 11 lines
Cc Alejandro Colomar <[email protected]>
Cc Michael Kerrisk <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Oscar Salvador <[email protected]>
Cc Jann Horn <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Linux API <[email protected]>
Quoted 1 line

---

Patch applied.

Thanks,

Alex

Quoted 238 lines
Quoted 227 lines

v2 -> v3:
- Refine what "populating readable/writable" means
- Compare each version with MAP_POPULATE and give an example use case
- Reword SIGBUS handling
- Reword comment regarding special mappings and also add memfd_secret(2)
- Reference MADV_HWPOISON when talking about HW poisoned pages
- Minor cosmetic fixes

v1 -> v2:
- Use semantic newlines in all cases
- Add two missing "
- Document -EFAULT handling
- Rephrase some parts to make it more generic: VM_PFNMAP and VM_IO are only
   examples for special mappings

---
  man2/madvise.2 | 156 +++++++++++++++++++++++++++++++++++++++++++++++++

  1 file changed, 156 insertions(+)
man2/madvise.2 +0 -0
index f1f384c0c..37f6dd6fa 100644
@@ -469,6 +469,106 @@ If a page is file-backed and dirty, it will be 

written back to the backing
  storage.
  The advice might be ignored for some pages in the range when it is not applicable.
+.TP
+.BR MADV_POPULATE_READ " (since Linux 5.14)"
+"Populate (prefault) page tables readable,
+faulting in all pages in the range just as if manually reading from each page;

man2/madvise.2 +2 -0
+however,
+avoid the actual memory access that would have been performed after 

handling

man2/madvise.2 +12 -0
+the fault.
+.IP
+In contrast to
+.BR MAP_POPULATE ,
+.B MADV_POPULATE_READ
+does not hide errors,
+can be applied to (parts of) existing mappings and will always populate
+(prefault) page tables readable.
+One example use case is prefaulting a file mapping,
+reading all file content from disk;
+however,
+pages won't be dirtied and consequently won't have to be written back 

to disk

man2/madvise.2 +34 -0
+when evicting the pages from memory.
+.IP
+Depending on the underlying mapping,
+map the shared zeropage,
+preallocate memory or read the underlying file;
+files with holes might or might not preallocate blocks.
+If populating fails,
+a
+.B SIGBUS
+signal is not generated; instead, an error is returned.
+.IP
+If
+.B MADV_POPULATE_READ
+succeeds,
+all page tables have been populated (prefaulted) readable once.
+If
+.B MADV_POPULATE_READ
+fails,
+some page tables might have been populated.
+.IP
+.B MADV_POPULATE_READ
+cannot be applied to mappings without read permissions
+and special mappings,
+for example,
+mappings marked with kernel-internal flags such as
+.B VM_PFNMAP
+or
+.BR VM_IO ,
+or secret memory regions created using
+.BR memfd_secret(2) .
+.IP
+Note that with
+.BR MADV_POPULATE_READ ,
+the process can be killed at any moment when the system runs out of 

memory.
+.TP
+.BR MADV_POPULATE_WRITE " (since Linux 5.14)"

man2/madvise.2 +5 -0
+Populate (prefault) page tables writable,
+faulting in all pages in the range just as if manually writing to each
+each page;
+however,
+avoid the actual memory access that would have been performed after 

handling

man2/madvise.2 +41 -0
+the fault.
+.IP
+In contrast to
+.BR MAP_POPULATE ,
+MADV_POPULATE_WRITE does not hide errors,
+can be applied to (parts of) existing mappings and will always populate
+(prefault) page tables writable.
+One example use case is preallocating memory,
+breaking any CoW (Copy on Write).
+.IP
+Depending on the underlying mapping,
+preallocate memory or read the underlying file;
+files with holes will preallocate blocks.
+If populating fails,
+a
+.B SIGBUS
+signal is not generated; instead, an error is returned.
+.IP
+If
+.B MADV_POPULATE_WRITE
+succeeds,
+all page tables have been populated (prefaulted) writable once.
+If
+.B MADV_POPULATE_WRITE
+fails,
+some page tables might have been populated.
+.IP
+.B MADV_POPULATE_WRITE
+cannot be applied to mappings without write permissions
+and special mappings,
+for example,
+mappings marked with kernel-internal flags such as
+.B VM_PFNMAP
+or
+.BR VM_IO ,
+or secret memory regions created using
+.BR memfd_secret(2) .
+.IP
+Note that with
+.BR MADV_POPULATE_WRITE ,
+the process can be killed at any moment when the system runs out of 

memory.
  .SH RETURN VALUE
  On success,
  .BR madvise ()

man2/madvise.2 +0 -0
@@ -490,6 +590,22 @@ A kernel resource was temporarily unavailable.

  .B EBADF
  The map exists, but the area maps something that isn't a file. .TP
+.B EFAULT
+.I advice

man2/madvise.2 +7 -0
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+and populating (prefaulting) page tables failed because a
+.B SIGBUS
+would have been generated on actual memory access and the reason is 

not a

man2/madvise.2 +7 -0
+HW poisoned page
+(HW poisoned pages can,
+for example,
+be created using the
+.B MADV_HWPOISON
+flag described elsewhere in this page).
+.TP

  .B EINVAL
  .I addr
  is not page-aligned or

man2/madvise.2 +0 -0
@@ -533,6 +649,22 @@ or

  .BR VM_PFNMAP
  ranges.
  .TP
+.B EINVAL
+.I advice

man2/madvise.2 +5 -0
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+but the specified address range includes ranges with insufficient 

permissions

man2/madvise.2 +9 -0
+or special mappings,
+for example,
+mappings marked with kernel-internal flags such a
+.B VM_IO
+or
+.BR VM_PFNMAP ,
+or secret memory regions created using
+.BR memfd_secret(2) .
+.TP

  .B EIO
  (for
  .BR MADV_WILLNEED )

man2/madvise.2 +0 -0
@@ -548,6 +680,15 @@ Not enough memory: paging in failed.

  Addresses in the specified range are not currently mapped, or are outside the address space of the process. .TP
+.B ENOMEM
+.I advice

man2/madvise.2 +5 -0
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+and populating (prefaulting) page tables failed because there was not 

enough
+memory.
+.TP
  .B EPERM
  .I advice
  is

man2/madvise.2 +0 -0
@@ -555,6 +696,20 @@ is

  but the caller does not have the
  .B CAP_SYS_ADMIN
  capability.
+.TP
+.B EHWPOISON
+.I advice

man2/madvise.2 +5 -0
+is
+.B MADV_POPULATE_READ
+or
+.BR MADV_POPULATE_WRITE ,
+and populating (prefaulting) page tables failed because a HW poisoned 

page

man2/madvise.2 +6 -0
+(HW poisoned pages can,
+for example,
+be created using the
+.B MADV_HWPOISON
+flag described elsewhere in this page)
+was encountered.

  .SH VERSIONS
  Since Linux 3.18,
  .\" commit d3ac21cacc24790eb45d735769f35753f5b56ceb

man2/madvise.2 +0 -0
@@ -602,6 +757,7 @@ from the system call, as it should).
Quoted 8 lines

  .\" function first appeared in 4.4BSD.
  .SH SEE ALSO
  .BR getrlimit (2),
+.BR memfd_secret(2),
  .BR mincore (2),
  .BR mmap (2),
  .BR mprotect (2),

Gentle ping.

Signature · 3 lines
Alejandro Colomar
Linux man-pages comaintainer; https://www.kernel.org/doc/man-pages/
http://www.alejandro-colomar.es/