MAP_POPULATE behavior change?
Hey Nick,
Ping!
Cheers,
Michael
-------- Original Message --------
Subject: MAP_POPULATE behavior change?
Date: Tue, 22 Apr 2008 15:16:19 +0200
From: Michael Kerrisk <[email protected]>
To: Nick Piggin <[email protected]>
CC: lkml <[email protected]>, mk <[email protected]>
Hi Nick,
In 2.6.23, you had the commit below, which seems to have changed the kernel-userland API.
Am I right to understand that with this patch MAP_POPULATE now also works with MAP_PRIVATE mappings? (Formerly it only worked with MAP_SHARED mappings, but some very quick testing suggests that MAP_POPULATE now also has an affect for MAP_PRIVATE.)
Also, can you summarize the status of MAP_NONBLOCK? Is it now a no-op?
Cheers,
Michael
==
commit 54cb8821de07f2ffcd28c380ce9b93d5784b40d7
Author: Nick Piggin <[email protected]> Date: Thu Jul 19 01:46:59 2007 -0700
mm: merge populate and nopage into fault (fixes nonlinear)
Nonlinear mappings are (AFAIKS) simply a virtual memory concept that encodes the virtual address -> file offset differently from linear mappings.
->populate is a layering violation because the filesystem/pagecache code should need to know anything about the virtual memory mapping. The hitch here is that the ->nopage handler didn't pass down enough information (ie. pgoff).
But it is more logical to pass pgoff rather than have the ->nopage function
calculate it itself anyway (because that's a similar layering violation).
Having the populate handler install the pte itself is likewise a nasty thing to be doing.
This patch introduces a new fault handler that replaces ->nopage and ->populate and (later) ->nopfn. Most of the old mechanism is still in place so there is a lot of duplication and nice cleanups that can be removed if everyone switches over.
The rationale for doing this in the first place is that nonlinear mappings are subject to the pagefault vs invalidate/truncate race too, and it seemed stupid to duplicate the synchronisation logic rather than just consolidate the two.
After this patch, MAP_NONBLOCK no longer sets up ptes for pages present in pagecache. Seems like a fringe functionality anyway.
NOPAGE_REFAULT is removed. This should be implemented with ->fault, and no users have hit mainline yet.
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
Quoted 18 lines · Michael Kerrisk
Hey Nick,
Ping!
Cheers,
Michael
-------- Original Message --------
Subject:MAP_POPULATEbehavior change?
Date: Tue, 22 Apr 2008 15:16:19 +0200
From: Michael Kerrisk <[email protected]>
To: Nick Piggin <[email protected]>
CC: lkml <[email protected]>, mk <[email protected]>Hi Nick,
Quoted 7 lines
In 2.6.23, you had the commit below, which seems to have changed the kernel-userland API.
Am I right to understand that with this patch
MAP_POPULATEnow also works withMAP_PRIVATEmappings? (Formerly it only worked withMAP_SHAREDmappings, but some very quick testing suggests thatMAP_POPULATEnow also has an affect forMAP_PRIVATE.)
Yeah it should also set up anonymous mappings. Very observant :)
Quoted 1 line
Also, can you summarize the status of
MAP_NONBLOCK? Is it now a no-op?
Yes, it is. The problem is you need to walk the page tables, and it is fairly annoying to have to make a special case for that to be non blocking (after just getting rid of the ->populate special case).
It is possible of course to implement. It might even be easier because you could pass in a NONBLOCK flag to ->fault from get_user_pages... but for now it is a noop.
Thanks,
Nick
Quoted 48 lines
Cheers,
Michael
==
commit
54cb8821de07f2ffcd28c380ce9b93d5784b40d7Author: Nick Piggin <[email protected]> Date: Thu Jul 19 01:46:59 2007 -0700mm: merge populate and nopage into fault (fixes nonlinear)
Nonlinear mappings are (AFAIKS) simply a virtual memory concept that encodes the virtual address -> file offset differently from linear mappings.
->populate is a layering violation because the filesystem/pagecache code should need to know anything about the virtual memory mapping. The hitch here is that the ->nopage handler didn't pass down enough information (ie. pgoff).
But it is more logical to pass pgoff rather than have the ->nopage function
calculate it itself anyway (because that's a similar layering violation).Having the populate handler install the pte itself is likewise a nasty thing to be doing.
This patch introduces a new fault handler that replaces ->nopage and ->populate and (later) ->nopfn. Most of the old mechanism is still in place so there is a lot of duplication and nice cleanups that can be removed if everyone switches over.
The rationale for doing this in the first place is that nonlinear mappings are subject to the pagefault vs invalidate/truncate race too, and it seemed stupid to duplicate the synchronisation logic rather than just consolidate the two.
After this patch,
MAP_NONBLOCKno longer sets up ptes for pages present in pagecache. Seems like a fringe functionality anyway.
NOPAGE_REFAULTis removed. This should be implemented with ->fault, and no users have hit mainline yet.Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
Hi Nick,
Quoted 11 lines · Nick Piggin
Quoted 7 lines · Michael Kerrisk
In 2.6.23, you had the commit below, which seems to have changed the kernel-userland API.
Am I right to understand that with this patch
MAP_POPULATEnow also works withMAP_PRIVATEmappings? (Formerly it only worked withMAP_SHAREDmappings, but some very quick testing suggests thatMAP_POPULATEnow also has an affect forMAP_PRIVATE.)Yeah it should also set up anonymous mappings. Very observant :)
Umm -- did it not work for anonymous *shared* mappings before? I understood that it did.
Quoted 3 lines
Quoted 1 line
Also, can you summarize the status of
MAP_NONBLOCK? Is it now a no-op?Yes, it is.
Hmm -- going back and re-reading the code, there is:
if ((flags & MAP_POPULATE) && !(flags & MAP_NONBLOCK))
make_pages_present(addr, addr + len);That means that MAP_NONBLOCK makes *MAP_POPULATE* a no-op. Was that intended?
Cheers,
Michael
Quoted 61 lines
The problem is you need to walk the page tables, and it
is fairly annoying to have to make a special case for that to be non blocking (after just getting rid of the ->populate special case).It is possible of course to implement. It might even be easier because you could pass in a NONBLOCK flag to ->fault from
get_user_pages... but for now it is a noop.Thanks,
NickQuoted 48 lines
Cheers,
Michael
==
commit
54cb8821de07f2ffcd28c380ce9b93d5784b40d7Author: Nick Piggin <[email protected]> Date: Thu Jul 19 01:46:59 2007 -0700mm: merge populate and nopage into fault (fixes nonlinear)
Nonlinear mappings are (AFAIKS) simply a virtual memory concept that encodes the virtual address -> file offset differently from linear mappings.
->populate is a layering violation because the filesystem/pagecache code should need to know anything about the virtual memory mapping. The hitch here is that the ->nopage handler didn't pass down enough information (ie. pgoff).
But it is more logical to pass pgoff rather than have the ->nopage function
calculate it itself anyway (because that's a similar layering violation).Having the populate handler install the pte itself is likewise a nasty thing to be doing.
This patch introduces a new fault handler that replaces ->nopage and ->populate and (later) ->nopfn. Most of the old mechanism is still in place so there is a lot of duplication and nice cleanups that can be removed if everyone switches over.
The rationale for doing this in the first place is that nonlinear mappings are subject to the pagefault vs invalidate/truncate race too, and it seemed stupid to duplicate the synchronisation logic rather than just consolidate the two.
After this patch,
MAP_NONBLOCKno longer sets up ptes for pages present in pagecache. Seems like a fringe functionality anyway.
NOPAGE_REFAULTis removed. This should be implemented with ->fault, and no users have hit mainline yet.--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
Quoted 13 lines · Michael Kerrisk
Hi Nick,
Quoted 10 lines · Nick Piggin
Quoted 7 lines · Michael Kerrisk
In 2.6.23, you had the commit below, which seems to have changed the kernel-userland API.
Am I right to understand that with this patch
MAP_POPULATEnow also works withMAP_PRIVATEmappings? (Formerly it only worked withMAP_SHAREDmappings, but some very quick testing suggests thatMAP_POPULATEnow also has an affect forMAP_PRIVATE.)
Quoted 4 lines
Quoted 1 line
Yeah it should also set up anonymous mappings. Very observant :)
Umm -- did it not work for anonymous *shared* mappings before? I understood that it did.
Oh... yeah it would have. anonymous shared mappings aren't really anonymous from the POV of most of mm/ :P
Quoted 10 lines
Quoted 3 lines
Quoted 1 line
Also, can you summarize the status of
MAP_NONBLOCK? Is it now a no-op?Yes, it is.
Hmm -- going back and re-reading the code, there is:
if ((flags & MAP_POPULATE) && !(flags & MAP_NONBLOCK)) make_pages_present(addr, addr + len);That means that
MAP_NONBLOCKmakes *MAP_POPULATE* a no-op. Was that intended?
Ah, my logic wasn't up to scratch there. Yes of course MAP_NONBLOCK is not a noop, but it makes MAP_POPULATE into a noop. So it still guarantees that the call will not block and that the kernel will make some kind of best effort to populate the mapping... for some values of "some kind of" (ie. "nothing" ;)).
It's done this way of course so it is still safe to call MAP_NONBLOCK if you don't want to block, and thus everything stays backwards and forward compatible.
Thanks,
Nick
[...]
Quoted 20 lines
Quoted 11 lines
Quoted 4 lines
Quoted 2 lines
Also, can you summarize the status of
MAP_NONBLOCK? Is it now a no-op?Yes, it is.
Hmm -- going back and re-reading the code, there is:
if ((flags & MAP_POPULATE) && !(flags & MAP_NONBLOCK)) make_pages_present(addr, addr + len);That means that
MAP_NONBLOCKmakes *MAP_POPULATE* a no-op.Was that intended?
Ah, my logic wasn't up to scratch there. Yes of course
MAP_NONBLOCKis not
a noop, but it makesMAP_POPULATEinto a noop. So it still guarantees that the call will not block and that the kernel will make some kind of best effort to populate the mapping... for some values of "some kind of" (ie. "nothing" ;)).
Quoted 3 lines
It's done this way of course so it is still safe to call
MAP_NONBLOCKif you don't want to block, and thus everything stays backwards and forward compatible.
Thanks.
Continuing on from there, what is the status of the flags argument for remap_file_pages() now. the man page curently carries this text:
The flags argument has the same meaning as for mmap(2), but all flags other than MAP_NONBLOCK are ignored.
My question is: is it still true that flags other than MAP_NONBLOCK are ignored? A quick glance at the code leads me to suspect not. (I'm not sure if you made the recent changes to that code or not?)
Cheers,
Michael
Signature · 3 lines
Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html