mm: hold page lock over page_mkwrite
Subsystems (3)
RFC PATCH 1/1 mm: hold page lock over page_mkwrite
I want to have the page be protected by page lock between page_mkwrite notification to the filesystem, and the actual setting of the page dirty. Do this by holding the page lock over page_mkwrite, and keep it held until after set_page_dirty.
I need this in fsblock because I am working to ensure filesystem metadata can be correctly allocated and refcounted. This means that page cleaning should not require memory allocation (to be really robust).
Without this patch, then for example we could get a concurrent writeout after the page_mkwrite (which allocates page metadata required to clean it), but before the set_page_dirty. The writeout will clean the page and notice that the metadata is now unused and may be deallocated (because it appears clean as set_page_dirty hasn't been called yet). So at this point the page may be dirtied via the pte without enough metadata to be able to write it back.
--- fs/buffer.c | 11 +++++------ mm/memory.c | 51 ++++++++++++++++++++++++++++----------------------- 2 files changed, 33 insertions(+), 29 deletions(-)
fs/buffer.c +5 -6
@@ -2474,12 +2474,12 @@ block_page_mkwrite(struct vm_area_structloff_t size;int ret = -EINVAL ; -lock_page (page); +BUG_ON (page->mapping != inode->i_mapping); + size =i_size_read (inode); -if ((page->mapping != inode->i_mapping) || - (page_offset (page) > size)) { +if (page_offset (page) > size) {/* page got truncated out from underneath us */ - goto out_unlock; + goto out; }/* page is wholly or partially inside EOF */
@@ -2492,8 +2492,7 @@ block_page_mkwrite(struct vm_area_structif (!ret) ret =block_commit_write (page,0 , end); -out_unlock: -unlock_page (page); +out:return ret; }
mm/memory.c +28 -23
102 lines not on this page · raw message
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · Nick Piggin
I want to have the page be protected by page lock between
page_mkwritenotification to the filesystem, and the actual setting of the page dirty. Do this by holding the page lock overpage_mkwrite, and keep it held until afterset_page_dirty.
I fear that this will end up creating lock inversions with file systems who grab cross-node locks, which are ordered outside of the page lock, inside their ->page_mkwrite(). See ocfs2's call of ocfs2_inode_lock() from ocfs2_page_mkwrite().
In a sense, it's prepare_write() all over again. Please don't call into file systems after having acquired the page lock.
Instead, can we get a helper that the file system would call after having acquired its locks and the page lock? Something like that.
- z
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 10 lines · Zach Brown
Quoted 4 lines · Nick Piggin
I want to have the page be protected by page lock between
page_mkwritenotification to the filesystem, and the actual setting of the page dirty. Do this by holding the page lock overpage_mkwrite, and keep it held until afterset_page_dirty.I fear that this will end up creating lock inversions with file systems who grab cross-node locks, which are ordered outside of the page lock, inside their ->page_mkwrite(). See ocfs2's call of
ocfs2_inode_lock()fromocfs2_page_mkwrite().
Is ocfs2 immune to the races that get covered by this patch?
Quoted 2 lines
In a sense, it's
prepare_write()all over again. Please don't call into file systems after having acquired the page lock.
It is very much an opt-out thing with page_mkwrite. The filesystem if it thinks it is really smart can unlock the page if it likes. I'm sure it will probably introduce some obscure race or another, but hey ;)
Quoted 2 lines
Instead, can we get a helper that the file system would call after having acquired its locks and the page lock? Something like that.
Well, the critical part is holding the same page lock over the "important" part of the page_mkwrite operation and setting the pte dirty and marking the page dirty. So there isn't really a helper it can call.
Hmm, actually possibly we can enter page_mkwrite with the page unlocked, but exit with the page locked? Slightly more complex, but should save complexity elsewhere. Yes I think this might be the best way to go.
Thanks,
Nick
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 2 lines
Is ocfs2 immune to the races that get covered by this patch?
I haven't the slightest idea.
Quoted 3 lines
Hmm, actually possibly we can enter
page_mkwritewith the page unlocked, but exit with the page locked? Slightly more complex, but should save complexity elsewhere. Yes I think this might be the best way to go.
That sounds like it would work on first glance, yeah. Mark will yell at us if we've gotten it wrong ;).
- z
(Chris, your point about lock ordering is good. Zach raised the same one. Thanks for the good catch there guys).
Quoted 4 lines · Zach Brown
Quoted 2 lines
Is ocfs2 immune to the races that get covered by this patch?
I haven't the slightest idea.
Well, they would be covered with the new scheme anyway:
Quoted 6 lines
Quoted 3 lines
Hmm, actually possibly we can enter
page_mkwritewith the page unlocked, but exit with the page locked? Slightly more complex, but should save complexity elsewhere. Yes I think this might be the best way to go.That sounds like it would work on first glance, yeah. Mark will yell at us if we've gotten it wrong ;).
OK, thanks. I'll go with that approach and see what happens.
Quoted 8 lines · Nick Piggin
Quoted 6 lines
Quoted 3 lines
Hmm, actually possibly we can enter
page_mkwritewith the page unlocked, but exit with the page locked? Slightly more complex, but should save complexity elsewhere. Yes I think this might be the best way to go.That sounds like it would work on first glance, yeah. Mark will yell at us if we've gotten it wrong ;).
OK, thanks. I'll go with that approach and see what happens.
Yeah, that sounds reasonable... Thanks for pointing this out Zach!
--Mark
--
Mark Fasheh
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · Nick Piggin
I want to have the page be protected by page lock between
page_mkwritenotification to the filesystem, and the actual setting of the page dirty. Do this by holding the page lock overpage_mkwrite, and keep it held until afterset_page_dirty.
Are any of the filesystems ordering the journal lock outside the page lock? I thought ocfs2 and ext4 were either doing this or discussing it. If they are, this will make fsblock hard to use for them.
-chris
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
mm/memory.c unlock: · line 2104
Quoted 8 lines · Nick Piggin
mm/memory.c +6
-0+if (!page_mkwrite) +wait_on_page_locked (dirty_page);set_page_dirty_balance (dirty_page, page_mkwrite);put_page (dirty_page); +if (page_mkwrite) { +unlock_page (old_page); +page_cache_release (old_page); + }
We're calling into the whole balance_dirty_pages() writeout path with a page locked.. is that sensible?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 10 lines · Peter Zijlstra
mm/memory.c unlock: · line 2104
Quoted 8 lines · Nick Piggin
mm/memory.c +6
-0+if (!page_mkwrite) +wait_on_page_locked (dirty_page);set_page_dirty_balance (dirty_page, page_mkwrite);put_page (dirty_page); +if (page_mkwrite) { +unlock_page (old_page); +page_cache_release (old_page); + }
Quoted 2 lines
We're calling into the whole
balance_dirty_pages()writeout path with a page locked.. is that sensible?
Yeah, probably should move the balance out of there.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines · Nick Piggin
I need this in fsblock because I am working to ensure filesystem metadata can be correctly allocated and refcounted. This means that page cleaning should not require memory allocation (to be really robust).
Which, unfortunately, is just a dream for any filesystem that uses delayed allocation. i.e. they have to walk the free space trees which may need to be read from disk and therefore require memory to succeed....
Cheers,
Dave.
Signature · 8 lines
Dave Chinner [email protected] -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Dave Chinner
Quoted 3 lines · Nick Piggin
I need this in fsblock because I am working to ensure filesystem metadata can be correctly allocated and refcounted. This means that page cleaning should not require memory allocation (to be really robust).
Which, unfortunately, is just a dream for any filesystem that uses delayed allocation. i.e. they have to walk the free space trees which may need to be read from disk and therefore require memory to succeed....
Well it's a dream because probably none of them get it right, but that doesn't mean its impossible.
You don't need complete memory allocation up-front to be robust, but having reserves or degraded modes that simply guarantee forward progress is enough.
For example, if you need to read/write filesystem metadata to find and allocate free space, then you really only need a page to do all the IO.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 14 lines · Nick Piggin
Quoted 9 lines · Dave Chinner
Quoted 3 lines · Nick Piggin
I need this in fsblock because I am working to ensure filesystem metadata can be correctly allocated and refcounted. This means that page cleaning should not require memory allocation (to be really robust).
Which, unfortunately, is just a dream for any filesystem that uses delayed allocation. i.e. they have to walk the free space trees which may need to be read from disk and therefore require memory to succeed....
Well it's a dream because probably none of them get it right, but that doesn't mean its impossible.
Quoted 7 lines
You don't need complete memory allocation up-front to be robust, but having reserves or degraded modes that simply guarantee forward progress is enough.
For example, if you need to read/write filesystem metadata to find and allocate free space, then you really only need a page to do all the IO.
For journalling filesystems, dirty metadata is pinned for at least the duration of the transaction and in many cases it is pinned for multiple transactions (i.e. in memory aggregation of commits like XFS does). And then once the transaction is complete, it can't be reused until it is written to disk.
For the worst case usage in XFS, think about a complete btree split of both free space trees, plus a complete btree split of the extent tree. That is two buffers per level per btree that are pinned by the transaction.
The free space trees are bound in depth by the AG size so the limit is (IIRC) 15 buffers per tree at 1TB AG size. However, the inode extent tree can be deeper than that (bound by filesystem size). In effect, writing back a single page could require memory allocation of 30-40 pages just for metadata that is dirtied by the allocation transaction.
And then the next page written back goes into a different AG and splits the trees there. And then the next does the same.
Luckily, this sort of thing doesn't happen very often, but it does serve to demonstrate how difficult it is to quantify how much memory the writeback path really needs to guarantee forward progress. Hence the dream......
Cheers,
Dave.
Signature · 8 lines
Dave Chinner [email protected] -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 41 lines · Dave Chinner
Quoted 14 lines · Nick Piggin
Quoted 9 lines · Dave Chinner
Quoted 3 lines · Nick Piggin
I need this in fsblock because I am working to ensure filesystem metadata can be correctly allocated and refcounted. This means that page cleaning should not require memory allocation (to be really robust).
Which, unfortunately, is just a dream for any filesystem that uses delayed allocation. i.e. they have to walk the free space trees which may need to be read from disk and therefore require memory to succeed....
Well it's a dream because probably none of them get it right, but that doesn't mean its impossible.
Quoted 7 lines
You don't need complete memory allocation up-front to be robust, but having reserves or degraded modes that simply guarantee forward progress is enough.
For example, if you need to read/write filesystem metadata to find and allocate free space, then you really only need a page to do all the IO.
For journalling filesystems, dirty metadata is pinned for at least the duration of the transaction and in many cases it is pinned for multiple transactions (i.e. in memory aggregation of commits like XFS does). And then once the transaction is complete, it can't be reused until it is written to disk.
For the worst case usage in XFS, think about a complete btree split of both free space trees, plus a complete btree split of the extent tree. That is two buffers per level per btree that are pinned by the transaction.
The free space trees are bound in depth by the AG size so the limit is (IIRC) 15 buffers per tree at 1TB AG size. However, the inode extent tree can be deeper than that (bound by filesystem size). In effect, writing back a single page could require memory allocation of 30-40 pages just for metadata that is dirtied by the allocation transaction.
Quoted 2 lines
And then the next page written back goes into a different AG and splits the trees there. And then the next does the same.
So assuming there is no reasonable way to do out of core algorithms on the filesystem metadata (and likely you don't want to anyway because it would be a significant slowdown or diverge of code paths), you still only need to reserve one set of those 30-40 pages for the entire kernel.
You only ever need to reserve enough memory for a *single* page to be processed. In the worst case that there are multiple pages under writeout but can't allocate memory, only one will be allowed access to reserves and the others will block until it is finished and can unpin them all.
Quoted 4 lines
Luckily, this sort of thing doesn't happen very often, but it does serve to demonstrate how difficult it is to quantify how much memory the writeback path really needs to guarantee forward progress. Hence the dream......
Well I'm not saying it is an immediate problem or it would be a good use of anybody's time to rush out and try to redesign their fs code to fix it ;) But at least for any new core/generic library functionality like fsblock, it would be silly not to close the hole there (not least because the problem is simpler here than in a complex fs).
Quoted 7 lines · Nick Piggin
So assuming there is no reasonable way to do out of core algorithms on the filesystem metadata (and likely you don't want to anyway because it would be a significant slowdown or diverge of code paths), you still only need to reserve one set of those 30-40 pages for the entire kernel.
Quoted 5 lines
You only ever need to reserve enough memory for a *single* page to be processed. In the worst case that there are multiple pages under writeout but can't allocate memory, only one will be allowed access to reserves and the others will block until it is finished and can unpin them all.
Sure, nobody will mind seeing lots of extra pinned memory ;)
Don't forget to add the space for data transforms and raid driver operations in the write stack, and whatever else we may not have thought of. With good engineering we can make it so "we can always make forward progress". But it won't matter because once a real user drives the system off this cliff there is no difference between "hung" and "really slow progress". They are going to crash it and report a hang.
Quoted 6 lines
Well I'm not saying it is an immediate problem or it would be a good use of anybody's time to rush out and try to redesign their fs code to fix it ;) But at least for any new core/generic library functionality like fsblock, it would be silly not to close the hole there (not least because the problem is simpler here than in a complex fs).
Hey, I appreciate anything you do in VM to make the ugly dance with filesystems (my area) a little less ugly.
I'm sure you also appreciate that every time VM tries to save 32 bytes, someone else tries to take 32 K-bytes. As they say... memory is cheap :)
jim
Quoted 8 lines · jim owens
Quoted 7 lines · Nick Piggin
So assuming there is no reasonable way to do out of core algorithms on the filesystem metadata (and likely you don't want to anyway because it would be a significant slowdown or diverge of code paths), you still only need to reserve one set of those 30-40 pages for the entire kernel.
Quoted 7 lines
Quoted 5 lines
You only ever need to reserve enough memory for a *single* page to be processed. In the worst case that there are multiple pages under writeout but can't allocate memory, only one will be allowed access to reserves and the others will block until it is finished and can unpin them all.
Sure, nobody will mind seeing lots of extra pinned memory ;)
40 pages (160k) isn't a huge amount. You could always have a boot option to disable the memory reserve if it is a big deal.
Quoted 3 lines
Don't forget to add the space for data transforms and raid driver operations in the write stack, and whatever else we may not have thought of. With good engineering we can make
The block layer below the filesystem should be robust. Well actually the core block layer is (except maybe for the new bio integrity stuff that looks pretty nasty). Not sure about md/dm, but they really should be safe (they use mempools etc).
Quoted 4 lines
it so "we can always make forward progress". But it won't matter because once a real user drives the system off this cliff there is no difference between "hung" and "really slow progress". They are going to crash it and report a hang.
I don't think that is the case. These are situations that would be *really* rare and transient. It is not like thrashing in that your working set size exceeds physical RAM, but just a combination of conditions that causes an unusual spike in the required memory to clean some dirty pages (eg. Dave's example of several IOs requiring btree splits over several AGs). Could cause a resource deadlock.
Quoted 9 lines
Quoted 6 lines
Well I'm not saying it is an immediate problem or it would be a good use of anybody's time to rush out and try to redesign their fs code to fix it ;) But at least for any new core/generic library functionality like fsblock, it would be silly not to close the hole there (not least because the problem is simpler here than in a complex fs).
Hey, I appreciate anything you do in VM to make the ugly dance with filesystems (my area) a little less ugly.
Well thanks.
Quoted 3 lines
I'm sure you also appreciate that every time VM tries to save 32 bytes, someone else tries to take 32 K-bytes. As they say... memory is cheap :)
Well that's OK. If core vm/fs code saves a little bit of memory that enables something else like a filesystem to use it to cache a tiny bit more useful data, then I think it is a good result :)
Quoted 12 lines · Nick Piggin
Quoted 7 lines
Quoted 5 lines
You only ever need to reserve enough memory for a *single* page to be processed. In the worst case that there are multiple pages under writeout but can't allocate memory, only one will be allowed access to reserves and the others will block until it is finished and can unpin them all.
Sure, nobody will mind seeing lots of extra pinned memory ;)
40 pages (160k) isn't a huge amount. You could always have a boot option to disable the memory reserve if it is a big deal.
Quoted 8 lines
Quoted 3 lines
Don't forget to add the space for data transforms and raid driver operations in the write stack, and whatever else we may not have thought of. With good engineering we can make
The block layer below the filesystem should be robust. Well actually the core block layer is (except maybe for the new bio integrity stuff that looks pretty nasty). Not sure about md/dm, but they really should be safe (they use mempools etc).
Are mempools fully safe, or just statistically safer?
Quoted 5 lines
Quoted 4 lines
it so "we can always make forward progress". But it won't matter because once a real user drives the system off this cliff there is no difference between "hung" and "really slow progress". They are going to crash it and report a hang.
Quoted 7 lines
I don't think that is the case. These are situations that would be *really* rare and transient. It is not like thrashing in that your working set size exceeds physical RAM, but just a combination of conditions that causes an unusual spike in the required memory to clean some dirty pages (eg. Dave's example of several IOs requiring btree splits over several AGs). Could cause a resource deadlock.
Suppose the systems has two pages to be written. The first must _reserve_ 40 pages of scratch space just in case the operation will need them. If the second page write is initiated concurrently with the first, the second must reserve another 40 pages concurrently.
If 10 page writes are concurrent, that's 400 pages of scratch space needed in reserve...
Your idea that the system can serialises page writes to limit the scratch used will limit this. But how does it know when to serialise page writes, if not by reserving 40 pages of memory per written page, until it knows better?
In other words, do you need 40 pages of metadata space times the number of concurrent page writes - until the write logic for each page reaches the point of "unreserving" because they've worked out which metadata they don't need to touch?
It sounds like a classic concurrency problem - how many things to start in parallel, each having a large upper bound on the memory they may need when they are started, even though the average is much lower.
A solution to that is each concurrent thing being able to reserve memory "I may need X memory later" and block at reservation time, so it doesn't have to block when it's later allocating and using memory. Then to keep up global concurrency, each thing is able to unreserve memory as it progresses. Or reserve some more. The point is reservation can block, while later processing which may need the memory doesn't block.
-- Jamie
Quoted 27 lines
Quoted 11 lines
Quoted 8 lines
Well I'm not saying it is an immediate problem or it would be a good use of anybody's time to rush out and try to redesign their fs code to fix it ;) But at least for any new core/generic library functionality like fsblock, it would be silly not to close the hole there (not least because the problem is simpler here than in a complex fs).
Hey, I appreciate anything you do in VM to make the ugly dance with filesystems (my area) a little less ugly.
Well thanks.
Quoted 3 lines
I'm sure you also appreciate that every time VM tries to save 32 bytes, someone else tries to take 32 K-bytes. As they say... memory is cheap :)
Well that's OK. If core vm/fs code saves a little bit of memory that enables something else like a filesystem to use it to cache a tiny bit more useful data, then I think it is a good result :)
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 13 lines · Jamie Lokier
Quoted 5 lines
Quoted 4 lines
it so "we can always make forward progress". But it won't matter because once a real user drives the system off this cliff there is no difference between "hung" and "really slow progress". They are going to crash it and report a hang.
Quoted 7 lines
I don't think that is the case. These are situations that would be *really* rare and transient. It is not like thrashing in that your working set size exceeds physical RAM, but just a combination of conditions that causes an unusual spike in the required memory to clean some dirty pages (eg. Dave's example of several IOs requiring btree splits over several AGs). Could cause a resource deadlock.
Quoted 7 lines
Suppose the systems has two pages to be written. The first must _reserve_ 40 pages of scratch space just in case the operation will need them. If the second page write is initiated concurrently with the first, the second must reserve another 40 pages concurrently.
If 10 page writes are concurrent, that's 400 pages of scratch space needed in reserve...
Therein lies the problem. XFS can do this in parallel in every AG at the same time. i.e. the reserve is per AG. The maximum number of AGs in XFS is 2^32, and I know of filesystems out there that have thousands of AGs in them. Hence reserving 40 pages per AG is definitely unreasonable. ;)
Even if we look at concurrent allocations as the upper bound, I've seen an 8p machine with several hundred concurrent allocation transactions in progress. Even that is unreasonable if you consider machines with 64k pages - it's hundreds of megabytes of RAM that are mostly going to be unused.
Specifying a pool of pages is not a guaranteed solution, either, as someone will always exhaust it as we can't guarantee any given transaction will complete before the pool is exhausted. i.e. the mempool design as it stands can't be used.
AFAIC, "should never allocate during writeback" is a great goal, but it is one that we will never be able to reach without throwing everything away and starting again. Minimising allocation is something we can do but we can't avoid it entirely. The higher layers need to understand this, not assert that the lower layers must conform to an impossible constraint and break if they don't.....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 7 lines · Jamie Lokier
Quoted 4 lines · Nick Piggin
The block layer below the filesystem should be robust. Well actually the core block layer is (except maybe for the new bio integrity stuff that looks pretty nasty). Not sure about md/dm, but they really should be safe (they use mempools etc).
Are mempools fully safe, or just statistically safer?
They will guarantee forward progress if used correctly, so yes fully safe.
Quoted 13 lines
Quoted 5 lines
Quoted 4 lines
it so "we can always make forward progress". But it won't matter because once a real user drives the system off this cliff there is no difference between "hung" and "really slow progress". They are going to crash it and report a hang.
Quoted 7 lines
I don't think that is the case. These are situations that would be *really* rare and transient. It is not like thrashing in that your working set size exceeds physical RAM, but just a combination of conditions that causes an unusual spike in the required memory to clean some dirty pages (eg. Dave's example of several IOs requiring btree splits over several AGs). Could cause a resource deadlock.
Quoted 7 lines
Suppose the systems has two pages to be written. The first must _reserve_ 40 pages of scratch space just in case the operation will need them. If the second page write is initiated concurrently with the first, the second must reserve another 40 pages concurrently.
If 10 page writes are concurrent, that's 400 pages of scratch space needed in reserve...
You only need to guarantee forward progress, so you would reserve 40 pages up front for the entire machine (some mempools have more memory than strictly needed to improve performance, so you could toy with that, but let's just describe the baseline).
So allocations happen as normal, except when an allocation fails, then the task which fails the allocation is given access to this reserve memory, any other task requiring reserve will then block.
Now the reserve provides enough pages to guarantee forward progress, so that one task is going to be able to proceed and eventually its pages will become freeable and can be returned to the reserve. Once the writeout has finished, the reserve will become available to other tasks.
So this way you only have to reserve enough to write out 1 page, and you only start blocking things when their memory allocations wolud have failed *anyway*. And you guarantee forward progress.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 23 lines · Nick Piggin
Quoted 7 lines
Suppose the systems has two pages to be written. The first must _reserve_ 40 pages of scratch space just in case the operation will need them. If the second page write is initiated concurrently with the first, the second must reserve another 40 pages concurrently.
If 10 page writes are concurrent, that's 400 pages of scratch space needed in reserve...
You only need to guarantee forward progress, so you would reserve 40 pages up front for the entire machine (some mempools have more memory than strictly needed to improve performance, so you could toy with that, but let's just describe the baseline).
So allocations happen as normal, except when an allocation fails, then the task which fails the allocation is given access to this reserve memory, any other task requiring reserve will then block.
Now the reserve provides enough pages to guarantee forward progress, so that one task is going to be able to proceed and eventually its pages will become freeable and can be returned to the reserve. Once the writeout has finished, the reserve will become available to other tasks.
Quoted 3 lines
So this way you only have to reserve enough to write out 1 page, and you only start blocking things when their memory allocations wolud have failed *anyway*. And you guarantee forward progress.
Ah. *light bulb* In the writing tasks, memory allocation is forbidden, but blocking is allowed. That's what I was missing :-)
Forward progress marches on.
-- Jamie
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>