OOM at low page cache?
Why is there no tunable to OOM at low page cache?
I have no swap configured. I have 16GB RAM. If Chrome or Gimp or some other stupid program goes off the deep end and eats up my RAM, I hit some 15.5GB or 15.75GB usage and stay there for about 40 minutes. Every time the program tries to do something to eat more RAM, it cranks disk hard; the disk starts thrashing, the mouse pointer stops moving, and nothing goes on. It's like swapping like crazy, except you're reading library files instead of paged anonymous RAM.
If only I could tell the system to OOM kill at 512MB or 1GB or 95% non-evictable RAM, it would recover on its own. As-is, I need to wait or trigger the OOM killer by sysrq.
Am I just the only person in the world who's ever had that problem? Or is it a matter of questions fast popping up when you try to do this *and* enable paging to disk? (In my experience, that's a matter of too much swap space: if you have 16GB RAM and your computer dies at 15.25GB usage, your swap space should be no larger than 750MB plus inactive working RAM; obviously, your computer can't handle paging 750MB back and forth. If you make it 8GB wide and you start swap thrashing at 2GB usage, you have too much swap available).
I guess you could try to detect excessive swap and page cache thrashing, but that's complex; if anyone really wanted to do that, it would be done by now. A low-barrier OOM is much simpler.
CC linux-mm in case somebody has a good answer but missed this in lkml traffic
Quoted 23 lines · John Moser
Why is there no tunable to OOM at low page cache?
I have no swap configured. I have 16GB RAM. If Chrome or Gimp or some other stupid program goes off the deep end and eats up my RAM, I hit some 15.5GB or 15.75GB usage and stay there for about 40 minutes. Every time the program tries to do something to eat more RAM, it cranks disk hard; the disk starts thrashing, the mouse pointer stops moving, and nothing goes on. It's like swapping like crazy, except you're reading library files instead of paged anonymous RAM.
If only I could tell the system to OOM kill at 512MB or 1GB or 95% non-evictable RAM, it would recover on its own. As-is, I need to wait or trigger the OOM killer by sysrq.
Am I just the only person in the world who's ever had that problem? Or is it a matter of questions fast popping up when you try to do this *and* enable paging to disk? (In my experience, that's a matter of too much swap space: if you have 16GB RAM and your computer dies at 15.25GB usage, your swap space should be no larger than 750MB plus inactive working RAM; obviously, your computer can't handle paging 750MB back and forth. If you make it 8GB wide and you start swap thrashing at 2GB usage, you have too much swap available).
Quoted 9 lines
I guess you could try to detect excessive swap and page cache thrashing, but that's complex; if anyone really wanted to do that, it would be done by now. A low-barrier OOM is much simpler.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hello,
Quoted 4 lines · Vlastimil Babka
CC linux-mm in case somebody has a good answer but missed this in lkml traffic
Quoted 1 line · John Moser
Why is there no tunable to OOM at low page cache?
AFAIR, there were several trial although there wasn't acceptable at that time. One thing I can remember is min_filelist_kbytes. FYI, http://lwn.net/Articles/412313/
Quoted 22 lines
Quoted 22 lines
I have no swap configured. I have 16GB RAM. If Chrome or Gimp or some other stupid program goes off the deep end and eats up my RAM, I hit some 15.5GB or 15.75GB usage and stay there for about 40 minutes. Every time the program tries to do something to eat more RAM, it cranks disk hard; the disk starts thrashing, the mouse pointer stops moving, and nothing goes on. It's like swapping like crazy, except you're reading library files instead of paged anonymous RAM.
If only I could tell the system to OOM kill at 512MB or 1GB or 95% non-evictable RAM, it would recover on its own. As-is, I need to wait or trigger the OOM killer by sysrq.
Am I just the only person in the world who's ever had that problem? Or is it a matter of questions fast popping up when you try to do this *and* enable paging to disk? (In my experience, that's a matter of too much swap space: if you have 16GB RAM and your computer dies at 15.25GB usage, your swap space should be no larger than 750MB plus inactive working RAM; obviously, your computer can't handle paging 750MB back and forth. If you make it 8GB wide and you start swap thrashing at 2GB usage, you have too much swap available).
Quoted 3 lines
Quoted 3 lines
I guess you could try to detect excessive swap and page cache thrashing, but that's complex; if anyone really wanted to do that, it would be done by now. A low-barrier OOM is much simpler.
I'm far away from reclaim code for a long time but when I read again, I found something strange.
With having swap in get_scan_count, we keep a mount of file LRU + free as above than high wmark to prevent file LRU thrashing but we don't with no swap. Why?
Anyway, I believe we should fix it and we now have workingset.c so there might be more ways to be smart than old(although I am concern about that shadow shrinker blows out lots of information to be useful to detect in heavy memory pressure like page thrashing)
Below could be band-aid until we find a elegant solution?
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Quoted 6 lines · Minchan Kim
Hello,
Quoted 3 lines · Vlastimil Babka
CC linux-mm in case somebody has a good answer but missed this in lkml traffic
Quoted 6 lines
Quoted 2 lines · John Moser
Quoted 1 line
Why is there no tunable to OOM at low page cache?
AFAIR, there were several trial although there wasn't acceptable at that time. One thing I can remember is
min_filelist_kbytes. FYI,
http://lwn.net/Articles/412313/
The Android low memory killer does exactly what you want, and for very much the same reasons.
See drivers/staging/android/lowmemorykiller.c
However, in the mainline kernel I think it does make sense to apply something like the patch that Minchan cooked up with, to OOM if freeing all the page cache could not bring us back up to the high watermark, across all the memory zones.
- --
All rights reversed
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBAgAGBQJUyNewAAoJEM553pKExN6DlKQH/3PprrXF7IOyjiXnO+2Qqbau wgWXO7mQWGFi+zNqSUzmWtfTCFVx6BxLi23MCQG1RqKGnQI4DehdOKMDidFwoC8D 2grKe9ELp04mEbyG0aipdxSw6FouIDFhC2FzmU7oQDZX5RKmLuxY7QPU4NTCitcR xHp6jWrvyY2CDiSpA2QSAaAAIG21BtPJvXQg3WvY/LI03N1edqZnExt5Po8CY7oe EeiO7ZtYISl/wRIoribEafZF4rMAfJ5A36kdbulqCqVtgCWEDPV0RCXimc5EtDIt bFDiv924+YMiuEFULJlEqLGqTJOtfJ+NlBIn8nVRk5P1pOGEbO05zE+XV1Vea6k= =8351
-----END PGP SIGNATURE-----
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 10 lines · Rik van Riel
Quoted 8 lines · Minchan Kim
Hello,
Quoted 5 lines · Vlastimil Babka
CC linux-mm in case somebody has a good answer but missed this in lkml traffic
Quoted 1 line · John Moser
Why is there no tunable to OOM at low page cache?
Quoted 9 lines
Quoted 3 lines
AFAIR, there were several trial although there wasn't acceptable at that time. One thing I can remember is
min_filelist_kbytes. FYI,
http://lwn.net/Articles/412313/The Android low memory killer does exactly what you want, and for very much the same reasons.
See
drivers/staging/android/lowmemorykiller.c
Haven't seen that; it's been a long time since I bothered myself with kernel code, so I'm out-of-touch.
Wow lots of good responses coming quick.
Quoted 6 lines
However, in the mainline kernel I think it does make sense to apply something like the patch that Minchan cooked up with, to OOM if freeing all the page cache could not bring us back up to the high watermark, across all the memory zones.
Quoted 11 lines · Minchan Kim
Hello,
Quoted 4 lines · Vlastimil Babka
CC linux-mm in case somebody has a good answer but missed this in lkml traffic
Quoted 1 line · John Moser
Why is there no tunable to OOM at low page cache?
AFAIR, there were several trial although there wasn't acceptable at that time. One thing I can remember is
min_filelist_kbytes. FYI, http://lwn.net/Articles/412313/
That looks more straight-forward than http://lwn.net/Articles/422291/
Quoted 7 lines
I'm far away from reclaim code for a long time but when I read again, I found something strange.
With having swap in
get_scan_count, we keep a mount of file LRU + free as above than high wmark to prevent file LRU thrashing but we don't with no swap. Why?
That's ... strange. That means having a token 1MB swap file changes the system's practical memory reclaim behavior dramatically?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hello,
Quoted 19 lines · John Moser
Quoted 11 lines · Minchan Kim
Hello,
Quoted 4 lines · Vlastimil Babka
CC linux-mm in case somebody has a good answer but missed this in lkml traffic
Quoted 1 line · John Moser
Why is there no tunable to OOM at low page cache?
AFAIR, there were several trial although there wasn't acceptable at that time. One thing I can remember is
min_filelist_kbytes. FYI, http://lwn.net/Articles/412313/That looks more straight-forward than http://lwn.net/Articles/422291/
Quoted 3 lines
I'm far away from reclaim code for a long time but when I read again, I found something strange.
Quoted 7 lines
Quoted 4 lines
With having swap in
get_scan_count, we keep a mount of file LRU + free as above than high wmark to prevent file LRU thrashing but we don't with no swap. Why?That's ... strange. That means having a token 1MB swap file changes the system's practical memory reclaim behavior dramatically?
Basically, yes but 1M is too small. If all of swap consumed, the behavior will be same so I think we need more explicit logic to prevent cache thrashing. Could you test below patch?
Thanks.
Quoted 4 lines · Minchan Kim
Below could be band-aid until we find a elegant solution?
I don't know about elegant; but I'd be impressed if anyone figured out how to just go Windows 95 with it and build a Task Master interface. It would be useful to have a kernel interface that allows a service to attach, delegate an interface program, etc., and then pull it up under certain conditions (low memory, heavy scheduling due to lots of fork()ing, etc.) and assign temporary high priority. Basically, nearly-pause the system and allow the user to select and kill/term processes, or bring a process forward (for like 10 seconds, then kick it back again) so the user can save their work and exit gracefully. At hard OOM, you could either OOM or pause everything (you'd need a zero-allocation path to kill things in a user-end OOM handler).
Yeah, imaginative fantasies. Totally doable, but probably too complex to bother. There's all kinds of semaphore inversion or some such to worry about; how do you ensure an X11 program is 100% snappy when the system is being thrashed by fork() bombs and memory pressure?
Actually, I have no idea what I'm talking about.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>