Another wish item for your TODO list...
Hi,
Just a quick thought --- at some point it would be good if we could add logic to the core VM so that for sequentially accessed files, reclaiming any page of the file from cache would evict the _whole_ of the file from cache.
For large files, we're not going to try to cache the whole thing anyway. For small files, reading the whole file back in later isn't much more expensive than reading back a few fragments if the rest still happens to be in cache.
Cheers,
Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/
Quoted 4 lines · Stephen C. Tweedie
Just a quick thought --- at some point it would be good if we could add logic to the core VM so that for sequentially accessed files, reclaiming any page of the file from cache would evict the _whole_ of the file from cache.
I take it you mean "move all the pages from before the currently read page to the inactive list", so we preserve the pages we just read in with readahead ?
Quoted 4 lines
For large files, we're not going to try to cache the whole thing anyway. For small files, reading the whole file back in later isn't much more expensive than reading back a few fragments if the rest still happens to be in cache.
Indeed.
regards,
Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000
http://www.conectiva.com/ http://www.surriel.com/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/
Hi,
Quoted 9 lines · Rik van Riel
Quoted 5 lines
Just a quick thought --- at some point it would be good if we could add logic to the core VM so that for sequentially accessed files, reclaiming any page of the file from cache would evict the _whole_ of the file from cache.
I take it you mean "move all the pages from before the currently read page to the inactive list", so we preserve the pages we just read in with readahead ?
No, I mean that once we actually remove a page, we should also remove all the other pages IF the file has never been accessed in a non-sequential manner. The inactive management is separate.
It's an optimisation in CPU time as much as for anything else: there's just no point in doing expensive memory balancing/aging for pages which we know are next to useless.
Cheers,
Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/
Quoted 11 lines · Stephen C. Tweedie
Quoted 9 lines · Rik van Riel
Quoted 5 lines
Just a quick thought --- at some point it would be good if we could add logic to the core VM so that for sequentially accessed files, reclaiming any page of the file from cache would evict the _whole_ of the file from cache.
I take it you mean "move all the pages from before the currently read page to the inactive list", so we preserve the pages we just read in with readahead ?
Quoted 7 lines
No, I mean that once we actually remove a page, we should also remove all the other pages IF the file has never been accessed in a non-sequential manner. The inactive management is separate.
It's an optimisation in CPU time as much as for anything else:
there's just no point in doing expensive memory balancing/aging for pages which we know are next to useless.
The problem here is that we shouldn't remove the pages which are in the current readahead window, as those /will/ most likely be used in the near future.
One option could be to free all inactive pages in the address space of that file, but we'd still need to go around and find them...
regards,
Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000
http://www.conectiva.com/ http://www.surriel.com/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/
Hi,
Quoted 7 lines · Rik van Riel
Quoted 3 lines
It's an optimisation in CPU time as much as for anything else:
there's just no point in doing expensive memory balancing/aging for pages which we know are next to useless.The problem here is that we shouldn't remove the pages which are in the current readahead window, as those /will/ most likely be used in the near future.
It probably only makes real sense to do this for inodes which are not in use, anyway, which avoids that problem completely.
Cheers,
Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/
Quoted 6 lines · Stephen C. Tweedie
Quoted 4 lines · Rik van Riel
Quoted 3 lines
It's an optimisation in CPU time as much as for anything else:
there's just no point in doing expensive memory balancing/aging for pages which we know are next to useless.
Quoted 6 lines
Quoted 3 lines
The problem here is that we shouldn't remove the pages which are in the current readahead window, as those /will/ most likely be used in the near future.
It probably only makes real sense to do this for inodes which are not in use, anyway, which avoids that problem completely.
FreeBSD has a neat solution for this:
- if the memory item (file, shm segment, ...) is in use, move the page to the inactive list (equivalent to our inactive_dirty)
- if the item isn't in use, move the page to the cache list (equivalent to our inactive_clean list)
We could extend this by moving *every* page of an item which isn't in use to the inactive_clean list once we move ONE page of such an item to that list (or reclaim it?).
[IMHO we don't want to free the page ... no need to throw away the data in the page too soon]
regards,
Rik
--
"What you're running that piece of shit Gnome?!?!"
-- Miguel de Icaza, UKUUG 2000
http://www.conectiva.com/ http://www.surriel.com/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/
Hi,
Quoted 9 lines · Rik van Riel
- if the memory item (file, shm segment, ...) is in use, move the page to the inactive list (equivalent to our
inactive_dirty)
- if the item isn't in use, move the page to the cache list (equivalent to ourinactive_cleanlist)We could extend this by moving *every* page of an item which isn't in use to the
inactive_cleanlist once we move ONE page of such an item to that list (or reclaim it?).
For unused inodes which are never sequentially accessed, this should make a lot of sense.
--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/
Quoted 7 lines · Stephen C. Tweedie
Quoted 3 lines · Rik van Riel
I take it you mean "move all the pages from before the currently read page to the inactive list", so we preserve the pages we just read in with readahead ?
No, I mean that once we actually remove a page, we should also remove all the other pages IF the file has never been accessed in a non-sequential manner. The inactive management is separate.
*.h files, which are read in by the GCC are always accessed sequentielly (at least from the kernel POV) and while unmapping them is ok, they should at least remain in cache to speed up compiling. That's just one example for a workload which will suffer from this idea.
If you are going to include stuff like this, please make it at least a sysctl. It would be nice, if I could tell the kernel the workload I have, that these weird access patterns are quite normal for me and how to handle them.
Thanks & Regards
Ingo Oeser
Signature · 6 lines
Feel the power of the penguin - run [email protected] <esc>:x -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux.eu.org/Linux-MM/
Hi,
Quoted 9 lines · Ingo Oeser
Quoted 7 lines · Stephen C. Tweedie
Quoted 3 lines · Rik van Riel
I take it you mean "move all the pages from before the currently read page to the inactive list", so we preserve the pages we just read in with readahead ?
No, I mean that once we actually remove a page, we should also remove all the other pages IF the file has never been accessed in a non-sequential manner. The inactive management is separate.
Quoted 5 lines
*.h files, which are read in by the GCC are always accessed sequentielly (at least from the kernel POV) and while unmapping them is ok, they should at least remain in cache to speed up compiling. That's just one example for a workload which will suffer from this idea.
No. If they stay in cache, then that's fine: we won't change the caching behaviour at all.
All we do is change what happens _after_ the kernel has already decided to start moving pages of the file out of cache because they are old. For small sequential files like header files, there's absolutely no point in having just a few of the pages in cache --- you know, for sure, that if you need one of the pages, you'll need all of them. So, once one of the pages is old enough to be evicted from the cache, there's no point in keeping any of the other pages around.
Cheers,
Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/