Free memory starvation in a zone?
In kreclaimd() there is a nice loop that looks like
for(i = 0; i < MAX_NR_ZONES; i++) {
zone_t *zone = pgdat->node_zones + i;
if (!zone->size)
continue;
while (zone->free_pages < zone->pages_low) {
struct page * page;
page = reclaim_page(zone);
if (!page)
break;
__free_page(page);
}
}I was playing around with the page age algorithm when i noticed that it appears that the machine will get into a state where the inner loop _NEVER_ exits the current zone because applications running in that zone are eating the memory as fast as it is being freed up. I imaging that this could be causing some pretty serious problems since the other zone's pages will only get recleaimed during a page alloc. Maybe there should be a max number of pages that can be reclaimed out of any given zone to force this loop to break? Something like 5 or 10% of the zone?
Any comments?
BTW: I started playing with the page age system when I noticed that it wasn't very evenly distributed. All the pages in a tend to fall into one or two age groups pretty close to PAGE_AGE_START with a significant number of them often below PAGE_AGE_START.
Quoted 16 lines · Jeremy Linton
In
kreclaimd()there is a nice loop that looks likefor(i = 0; i < MAX_NR_ZONES; i++) { zone_t *zone = pgdat->node_zones + i; if (!zone->size) continue; while (zone->free_pages < zone->pages_low) { struct page * page; page = reclaim_page(zone); if (!page) break; __free_page(page); } }
Quoted 4 lines
I was playing around with the page age algorithm when i noticed that it appears that the machine will get into a state where the inner loop _NEVER_ exits the current zone because applications running in that zone are eating the memory as fast as it is being freed up.
Normal allocations are going to block giving a chance for kreclaimd to run (and remember, the loop and the freeing routines are really fast).
Are you sure you're seeing kreclaimd looping too much here ?
I've never seen that, and if I did I would get really really
concerned: we rely on kreclaimd to avoid atomic allocations from failing in a fragile way.
Quoted 23 lines · Marcelo Tosatti
Quoted 16 lines · Jeremy Linton
In
kreclaimd()there is a nice loop that looks likefor(i = 0; i < MAX_NR_ZONES; i++) { zone_t *zone = pgdat->node_zones + i; if (!zone->size) continue; while (zone->free_pages < zone->pages_low) { struct page * page; page = reclaim_page(zone); if (!page) break; __free_page(page); } }Quoted 4 lines
I was playing around with the page age algorithm when i noticed that it appears that the machine will get into a state where the inner loop _NEVER_ exits the current zone because applications running in that zone are eating the memory as fast as it is being freed up.
Quoted 8 lines
Normal allocations are going to block giving a chance for kreclaimd to run (and remember, the loop and the freeing routines are really fast).
Are you sure you're seeing kreclaimd looping too much here ?
I've never seen that, and if I did I would get really really
concerned: we rely on kreclaimd to avoid atomic allocations from failing in a fragile way.
Snippet from one of Dirk's logs.
PID PPID USER PRI SIZE SWAP RSS SHARE D STAT %CPU %MEM TIME COMMA
3 1 root 20 0 0 0 0 0 RW 58.8 0.0 2:41 kswapd
1494 1421 novatest 15 2009M 640M 1.3G 51476 0M R N 40.8 34.5 6:26 ceqsim
1751 1747 root 14 1048 4 1044 824 55 R 28.0 0.0 0:02 top
4 1 root 14 0 0 0 0 0 SW 27.1 0.0 1:06 krecla-Mike
Quoted 34 lines · Mike Galbraith
Quoted 23 lines · Marcelo Tosatti
Quoted 16 lines · Jeremy Linton
In
kreclaimd()there is a nice loop that looks likefor(i = 0; i < MAX_NR_ZONES; i++) { zone_t *zone = pgdat->node_zones + i; if (!zone->size) continue; while (zone->free_pages < zone->pages_low) { struct page * page; page = reclaim_page(zone); if (!page) break; __free_page(page); } }Quoted 4 lines
I was playing around with the page age algorithm when i noticed that it appears that the machine will get into a state where the inner loop _NEVER_ exits the current zone because applications running in that zone are eating the memory as fast as it is being freed up.
Quoted 8 lines
Normal allocations are going to block giving a chance for kreclaimd to run (and remember, the loop and the freeing routines are really fast).
Are you sure you're seeing kreclaimd looping too much here ?
I've never seen that, and if I did I would get really really
concerned: we rely on kreclaimd to avoid atomic allocations from failing in a fragile way.
Quoted 7 lines
Snippet from one of Dirk's logs.
PID PPID USER PRI SIZE SWAP RSS SHARE D STAT %CPU %MEM TIME COMMA 3 1 root 20 0 0 0 0 0 RW 58.8 0.0 2:41 kswapd 1494 1421 novatest 15 2009M 640M 1.3G 51476 0M R N 40.8 34.5 6:26 ceqsim 1751 1747 root 14 1048 4 1044 824 55 R 28.0 0.0 0:02 top 4 1 root 14 0 0 0 0 0 SW 27.1 0.0 1:06 krecla
The problem is if we have kreclaimd not able to satisfy all zones needs.
CPU usage does not mean zone freemem starvation.
Quoted 9 lines · Mike Galbraith
Quoted 1 line
Are you sure you're seeing kreclaimd looping too much here ?
Snippet from one of Dirk's logs.
PID PPID USER PRI SIZE SWAP RSS SHARE D STAT %CPU %MEM TIME COMMA 3 1 root 20 0 0 0 0 0 RW 58.8 0.0 2:41 kswapd 1494 1421 novatest 15 2009M 640M 1.3G 51476 0M R N 40.8 34.5 6:26 ceqsim 1751 1747 root 14 1048 4 1044 824 55 R 28.0 0.0 0:02 top 4 1 root 14 0 0 0 0 0 SW 27.1 0.0 1:06 krecla
I'm pretty sure this is because kreclaimd is woken up from __alloc_pages() all the time and cannot find anything useful to do ...
regards,
Rik
--
Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...
http://www.surriel.com/ http://distro.conectiva.com/
Send all your spam to [email protected] (spam digging piggy)
Quoted 12 lines · Rik van Riel
Quoted 9 lines · Mike Galbraith
Quoted 1 line
Are you sure you're seeing kreclaimd looping too much here ?
Snippet from one of Dirk's logs.
PID PPID USER PRI SIZE SWAP RSS SHARE D STAT %CPU %MEM TIME COMMA 3 1 root 20 0 0 0 0 0 RW 58.8 0.0 2:41 kswapd 1494 1421 novatest 15 2009M 640M 1.3G 51476 0M R N 40.8 34.5 6:26 ceqsim 1751 1747 root 14 1048 4 1044 824 55 R 28.0 0.0 0:02 top 4 1 root 14 0 0 0 0 0 SW 27.1 0.0 1:06 krecla
Quoted 3 lines
I'm pretty sure this is because kreclaimd is woken up from
__alloc_pages()all the time and cannot find anything useful to do ...
Yes, it could be bouncing (eg) referenced pages instead of looping.
In any case though, seems it would it be better to take locks, evaluate once and free in batch. With the possibility to bounce off of locks, it does look like looping is possible (if unlikely). Just picking a quantity and freeing it instead of reevaluating constantly would close off that possibility without batch freeing.
-Mike