Skip to contentSkip to conversation

madvise.2: Add MADV_WIPEONFORK documentation

PATCH Superseded Rik van Riel 3 participants 9 messages Sep 14, 2017 – Oct 9, 2017

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 madvise.2: Add MADV_WIPEONFORK documentation

Add MADV_WIPEONFORK and MADV_KEEPONFORK documentation to madvise.2. The new functionality was recently merged by Linus, and should be in the 4.14 kernel.

While documenting what EINVAL means for MADV_WIPEONFORK, I realized that MADV_FREE has the same thing going on, so I documented EINVAL for both in the ERRORS section.

This patch documents the following kernel commit:

commit d2cd9ede6e193dd7d88b6d27399e96229a551b19

Author: Rik van Riel <[email protected]>
Date:   Wed Sep 6 16:25:15 2017 -0700

mm,fork: introduce MADV_WIPEONFORK

index dfb31b63dba3..4f987ddfae79 100644

man2/madvise.2 +30 -0
@@ -31,6 +31,8 @@
 .\" 2010-06-19, Andi Kleen, Add documentation of MADV_SOFT_OFFLINE.
 .\" 2011-09-18, Doug Goldstein <[email protected]>
 .\"     Document MADV_HUGEPAGE and MADV_NOHUGEPAGE
+.\" 2017-09-14, Rik van Riel <[email protected]>
+.\"     Document MADV_WIPEONFORK and MADV_KEEPONFORK
 .\"
 .TH MADVISE 2 2017-07-13 "Linux" "Linux Programmer's Manual"
 .SH NAME
@@ -405,6 +407,22 @@ can be applied only to private anonymous pages (see
 .BR mmap (2)).
 On a swapless system, freeing pages in a given range happens instantly,
 regardless of memory pressure.
+.TP
+.BR MADV_WIPEONFORK " (since Linux 4.14)"
+Present the child process with zero-filled memory in this range after a
+.BR fork (2).
+This is useful for per-process data in forking servers that should be
+re-initialized in the child process after a fork, for example PRNG seeds,
+cryptographic data, etc.
+.IP
+The
+.B MADV_WIPEONFORK
+operation can only be applied to private anonymous pages (see
+.BR mmap (2)).
+.TP
+.BR MADV_KEEPONFORK " (since Linux 4.14)"
+Undo the effect of an earlier
+.BR MADV_WIPEONFORK .
 .SH RETURN VALUE
 On success,
 .BR madvise ()
@@ -457,6 +475,18 @@ or
 but the kernel was not configured with
 .BR CONFIG_KSM .
 .TP
+.B EINVAL
+.I advice
+is
+.BR MADV_FREE
+or
+.BR MADV_WIPEONFORK
+but the specified address range includes file, Huge TLB,
+.BR MAP_SHARED ,
+or
+.BR VM_PFNMAP
+ranges.
+.TP
 .B EIO
 (for
 .BR MADV_WILLNEED )
#1 lore raw

Great change, just some suggestions ...

Quoted 22 lines · Rik van Riel

Add MADV_WIPEONFORK and MADV_KEEPONFORK documentation to madvise.2. The new functionality was recently merged by Linus, and should be in the 4.14 kernel.

While documenting what EINVAL means for MADV_WIPEONFORK, I realized that MADV_FREE has the same thing going on, so I documented EINVAL for both in the ERRORS section.

This patch documents the following kernel commit:

commit d2cd9ede6e193dd7d88b6d27399e96229a551b19

Author: Rik van Riel <[email protected]>
Date:   Wed Sep 6 16:25:15 2017 -0700

mm,fork: introduce MADV_WIPEONFORK

index dfb31b63dba3..4f987ddfae79 100644

man2/madvise.2 line 36

Quoted 6 lines
man2/madvise.2 +2 -0
@@ -31,6 +31,8 @@
 .\" 2010-06-19, Andi Kleen, Add documentation of MADV_SOFT_OFFLINE.
 .\" 2011-09-18, Doug Goldstein <[email protected]>
 .\"     Document MADV_HUGEPAGE and MADV_NOHUGEPAGE
+.\" 2017-09-14, Rik van Riel <[email protected]>
+.\"     Document MADV_WIPEONFORK and MADV_KEEPONFORK
 .\"

It seems to be idiomatic to reference the commit adding the options in the hidden man-page comments. Probably needs:

.\" commit d2cd9ede6e193dd7d88b6d27399e96229a551b19

here. (That's the commit adding MADV_WIPEONFORK/MADV_KEEPONFORK to Linus' tree.

man2/madvise.2 can be applied only to private anonymous pages (see · line 416

Quoted 13 lines
man2/madvise.2 +7 -0
 .TH MADVISE 2 2017-07-13 "Linux" "Linux Programmer's Manual"
 .SH NAME
@@ -405,6 +407,22 @@ can be applied only to private anonymous pages (see
 .BR mmap (2)).
 On a swapless system, freeing pages in a given range happens instantly,
 regardless of memory pressure.
+.TP
+.BR MADV_WIPEONFORK " (since Linux 4.14)"
+Present the child process with zero-filled memory in this range after a
+.BR fork (2).
+This is useful for per-process data in forking servers that should be
+re-initialized in the child process after a fork, for example PRNG seeds,
+cryptographic data, etc.

Instead of cryptographic data, I would say more broadly "secrets" - to help nudge best-practise. For example in an application that buffers decrypted plaintext, it's smart to mark it as WIPEONFORK so that there aren't unnecessary copies of the plaintext floating around.

I'd suggest patching fork.2 also, with something like:

index b5af58ca0..b11e750e3 100644

man2/fork.2 +6 -0
@@ -140,6 +140,12 @@ Memory mappings that have been marked with the
 flag are not inherited across a
 .BR fork ().
 .IP *
+Memory in mappings that have been marked with the
+.BR madvise (2)
+.B MADV_WIPEONFORK
+flag is zeroed in the child after a
+.BR fork ().
+.IP *
 The termination signal of the child is always
 .B SIGCHLD
 (see
Signature · 5 lines
Colm
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#2 lore raw

PATCH v2 1/1 madvise.2: Add MADV_WIPEONFORK documentation

v2: implement the improvements suggested by Colm, and add
Colm's text to the fork.2 man page
(Colm, I have added a signed-off-by in your name - is that ok?)

Add MADV_WIPEONFORK and MADV_KEEPONFORK documentation to madvise.2. The new functionality was recently merged by Linus, and should be in the 4.14 kernel.

While documenting what EINVAL means for MADV_WIPEONFORK, I realized that MADV_FREE has the same thing going on, so I documented EINVAL for both in the ERRORS section.

This patch documents the following kernel commit:

commit d2cd9ede6e193dd7d88b6d27399e96229a551b19

Author: Rik van Riel <[email protected]>
Date:   Wed Sep 6 16:25:15 2017 -0700

mm,fork: introduce MADV_WIPEONFORK

man2/fork.2 +6 -0
index b5af58ca08c0..b11e750e3876 100644
@@ -140,6 +140,12 @@ Memory mappings that have been marked with the
 flag are not inherited across a
 .BR fork ().
 .IP *
+Memory in mappings that have been marked with the
+.BR madvise (2)
+.B MADV_WIPEONFORK
+flag is zeroed in the child after a
+.BR fork ().
+.IP *
 The termination signal of the child is always
 .B SIGCHLD
 (see
man2/madvise.2 +31 -0
index dfb31b63dba3..bb0ac469c509 100644
@@ -31,6 +31,9 @@
 .\" 2010-06-19, Andi Kleen, Add documentation of MADV_SOFT_OFFLINE.
 .\" 2011-09-18, Doug Goldstein <[email protected]>
 .\"     Document MADV_HUGEPAGE and MADV_NOHUGEPAGE
+.\" 2017-09-14, Rik van Riel <[email protected]>
+.\"     Document MADV_WIPEONFORK and MADV_KEEPONFORK
+.\" commit d2cd9ede6e193dd7d88b6d27399e96229a551b19
 .\"
 .TH MADVISE 2 2017-07-13 "Linux" "Linux Programmer's Manual"
 .SH NAME
@@ -405,6 +408,22 @@ can be applied only to private anonymous pages (see
 .BR mmap (2)).
 On a swapless system, freeing pages in a given range happens instantly,
 regardless of memory pressure.
+.TP
+.BR MADV_WIPEONFORK " (since Linux 4.14)"
+Present the child process with zero-filled memory in this range after a
+.BR fork (2).
+This is useful for per-process data in forking servers that should be
+re-initialized in the child process after a fork, for example PRNG seeds,
+cryptographic secrets, etc.
+.IP
+The
+.B MADV_WIPEONFORK
+operation can only be applied to private anonymous pages (see
+.BR mmap (2)).
+.TP
+.BR MADV_KEEPONFORK " (since Linux 4.14)"
+Undo the effect of an earlier
+.BR MADV_WIPEONFORK .
 .SH RETURN VALUE
 On success,
 .BR madvise ()
@@ -457,6 +476,18 @@ or
 but the kernel was not configured with
 .BR CONFIG_KSM .
 .TP
+.B EINVAL
+.I advice
+is
+.BR MADV_FREE
+or
+.BR MADV_WIPEONFORK
+but the specified address range includes file, Huge TLB,
+.BR MAP_SHARED ,
+or
+.BR VM_PFNMAP
+ranges.
+.TP
 .B EIO
 (for
 .BR MADV_WILLNEED )
#3 lore raw
Quoted 3 lines · Rik van Riel

v2: implement the improvements suggested by Colm, and add
Colm's text to the fork.2 man page
(Colm, I have added a signed-off-by in your name - is that ok?)

Yep, that's ok! Whole thing LGTM.

Signature · 7 lines
Colm

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#4 lore raw
In reply to Rik van Riel

Hello Rik, (and Colm)

Quoted 12 lines · Rik van Riel

v2: implement the improvements suggested by Colm, and add
Colm's text to the fork.2 man page
(Colm, I have added a signed-off-by in your name - is that ok?)

Add MADV_WIPEONFORK and MADV_KEEPONFORK documentation to madvise.2. The new functionality was recently merged by Linus, and should be in the 4.14 kernel.

While documenting what EINVAL means for MADV_WIPEONFORK, I realized that MADV_FREE has the same thing going on, so I documented EINVAL for both in the ERRORS section.

Quoted 7 lines

This patch documents the following kernel commit:

commit d2cd9ede6e193dd7d88b6d27399e96229a551b19

Author: Rik van Riel <[email protected]>
Date:   Wed Sep 6 16:25:15 2017 -0700

mm,fork: introduce MADV_WIPEONFORK

Thanks. I applied this, and tweaked the madvise.2 text a little, to read as follows (please let me know if I messed anything up):

MADV_WIPEONFORK (since Linux 4.14)
Present the child process with zero-filled memory in this range after a fork(2). This is useful in forking servers in order to ensure that sensitive per-process data (for example, PRNG seeds, cryptographic secrets, and so on) is not handed to child processes.

The MADV_WIPEONFORK operation can be applied only to pri‐ vate anonymous pages (see mmap(2)).

Thanks,

Michael

Quoted 77 lines
man2/fork.2 +6 -0
index b5af58ca08c0..b11e750e3876 100644
@@ -140,6 +140,12 @@ Memory mappings that have been marked with the
 flag are not inherited across a
 .BR fork ().
 .IP *
+Memory in mappings that have been marked with the
+.BR madvise (2)
+.B MADV_WIPEONFORK
+flag is zeroed in the child after a
+.BR fork ().
+.IP *
 The termination signal of the child is always
 .B SIGCHLD
 (see
man2/madvise.2 +31 -0
index dfb31b63dba3..bb0ac469c509 100644
@@ -31,6 +31,9 @@
 .\" 2010-06-19, Andi Kleen, Add documentation of MADV_SOFT_OFFLINE.
 .\" 2011-09-18, Doug Goldstein <[email protected]>
 .\"     Document MADV_HUGEPAGE and MADV_NOHUGEPAGE
+.\" 2017-09-14, Rik van Riel <[email protected]>
+.\"     Document MADV_WIPEONFORK and MADV_KEEPONFORK
+.\" commit d2cd9ede6e193dd7d88b6d27399e96229a551b19
 .\"
 .TH MADVISE 2 2017-07-13 "Linux" "Linux Programmer's Manual"
 .SH NAME
@@ -405,6 +408,22 @@ can be applied only to private anonymous pages (see
 .BR mmap (2)).
 On a swapless system, freeing pages in a given range happens instantly,
 regardless of memory pressure.
+.TP
+.BR MADV_WIPEONFORK " (since Linux 4.14)"
+Present the child process with zero-filled memory in this range after a
+.BR fork (2).
+This is useful for per-process data in forking servers that should be
+re-initialized in the child process after a fork, for example PRNG seeds,
+cryptographic secrets, etc.
+.IP
+The
+.B MADV_WIPEONFORK
+operation can only be applied to private anonymous pages (see
+.BR mmap (2)).
+.TP
+.BR MADV_KEEPONFORK " (since Linux 4.14)"
+Undo the effect of an earlier
+.BR MADV_WIPEONFORK .
 .SH RETURN VALUE
 On success,
 .BR madvise ()
@@ -457,6 +476,18 @@ or
 but the kernel was not configured with
 .BR CONFIG_KSM .
 .TP
+.B EINVAL
+.I advice
+is
+.BR MADV_FREE
+or
+.BR MADV_WIPEONFORK
+but the specified address range includes file, Huge TLB,
+.BR MAP_SHARED ,
+or
+.BR VM_PFNMAP
+ranges.
+.TP
 .B EIO
 (for
 .BR MADV_WILLNEED )
Signature · 7 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#5 lore raw
Quoted 14 lines · Michael Kerrisk (man-pages)

Thanks. I applied this, and tweaked the madvise.2 text a little, to read as follows (please let me know if I messed anything up):

       MADV_WIPEONFORK (since Linux 4.14)
              Present the child process with zero-filled memory  in  this
              range  after  a fork(2).  This is useful in forking servers
              in order to ensure that  sensitive  per- process  data  (for
              example,  PRNG  seeds, cryptographic secrets, and so on) is
              not handed to child processes.

Quoted 3 lines

              The MADV_WIPEONFORK operation can be applied only  to  pri‐
              vate anonymous pages (see mmap(2)).

That looks great. Thank you, Michael!

Signature · 1 line
All rights reversed
#6 lore raw

Hi Rik,

I have a follow-up question re wipe-on-fork. What are the semantics for this setting with respect to fork() and exec()? That is, in the child of a fork(), does the flag remain set for the specified address range? (My quick read of the source suggests yes, but I have not tested.) And, when we do an exec(), my assumption is that the flag is cleared for the address range, but it would be good to have confirmation.

Thanks,

Michael

Quoted 24 lines · Rik van Riel
Quoted 14 lines · Michael Kerrisk (man-pages)

Thanks. I applied this, and tweaked the madvise.2 text a little, to read as follows (please let me know if I messed anything up):

MADV_WIPEONFORK (since Linux 4.14)
Present the child process with zero-filled
memory in this
range after a fork(2). This is useful in forking
servers
in order to ensure that sensitive per-
process data (for
example, PRNG seeds, cryptographic secrets, and so
on) is
not handed to child processes.

Quoted 3 lines

The MADV_WIPEONFORK operation can be applied
only to pri‐
vate anonymous pages (see mmap(2)).

That looks great. Thank you, Michael!

--
All rights reversed

Signature · 3 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
#7 lore raw
Quoted 9 lines · Michael Kerrisk (man-pages)

Hi Rik,

I have a follow-up question re wipe-on-fork. What are the semantics for this setting with respect to fork() and exec()? That is, in the child of a fork(), does the flag remain set for the specified address range? (My quick read of the source suggests yes, but I have not tested.) And, when we do an exec(), my assumption is that the flag is cleared for the address range, but it would be good to have confirmation.

Indeed, on exec() the flag is cleared, because all memory regions get replaced on exec().

The flag remains across a fork(), so if a child task
were to fork, the memory would be empty of contents
again in its child. This seems to most closely match
the use case of discarding things like cryptographic
secrets at fork time.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#8 lore raw

Hi Rik,

Thanks for the blazingly fast response :-)

Quoted 11 lines · Rik van Riel
Quoted 9 lines · Michael Kerrisk (man-pages)

Hi Rik,

I have a follow-up question re wipe-on-fork. What are the semantics for this setting with respect to fork() and exec()? That is, in the child of a fork(), does the flag remain set for the specified address range? (My quick read of the source suggests yes, but I have not tested.) And, when we do an exec(), my assumption is that the flag is cleared for the address range, but it would be good to have confirmation.

Quoted 2 lines

Indeed, on exec() the flag is cleared, because all memory regions get replaced on exec().

Thanks.

Quoted 5 lines

The flag remains across a fork(), so if a child task were to fork, the memory would be empty of contents again in its child. This seems to most closely match the use case of discarding things like cryptographic secrets at fork time.

Thanks!

I'll add this info to the madvise(2) page.

Cheers,

Michael

Signature · 3 lines
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/