Skip to contentSkip to conversation

out_of_memory called to often

DISCUSSION Jason Li 2 participants 2 messages Mar 11, 2003

lore
Share & tools
lore

Jason Li

author
#0 lore raw

Hi,

We are running an embedded linux (2.4.19) with no swap sapce. And we haven't applied any rmap yet.

Currely when the system runs low (reached the pages_low water-mark), kswapd kicks in. But soon after kswapd kicks in, the out_of_memory is called due to swapd not being able to reclaim enough pages from the page cache from each run of it. I don't understand why sometimes swapd can't reclaim the SWAP_CLUSTER_MAX number of pages.

We know there are still some number of cache pages that can be reclaimed if we run the following c program:

int main() {
int i, j;
char *tmp[10000]

for (i=0; i<NUMBER_OF_PAGES_LEFT_BEFORE_pages_low;
i++) tmp[i]=malloc(4k);
for (i=0; i<NUMBER_OF_PAGES_LEFT_BEFORE_pages_low;
i++) free(tmp[i]);
}
To solve this problem, I am trying to make a conditional call to the out_of_memory() -- only if the pages_min is reached, then the out_of_memory() can be called.

If we can solve the swapd livelock problem, is this a feasible solution? If you have a better idea, can you please share with us?

Any input will be greatly appreciated.

Thanks very much in advance! Please include my email address in your reply.

-Jason

__________________________________________________
Do you Yahoo!?
Yahoo! Web Hosting - establish your business online
http://webhosting.yahoo.com

#1 lore raw
Quoted 5 lines · Jason Li

Hi,

We are running an embedded linux (2.4.19) with no swap sapce. And we haven't applied any rmap yet.

I believe you may be seeing the same issue I brought up in a previous email. Have you tried what I suggested? I've appended the email to this message with out the test code.

Ross

I've verified this in 2.4.21-pre5 by code inspection and can trigger the problem on 2.4.18. It appears to have been fixed in 2.5.

The folowing code vmscan.c assumes that there is available swap space.

      /*
       * this is the non-racy check for busy page.
       */
      if (!page->mapping || !is_page_cache_freeable(page)) {
          spin_unlock(&pagecache_lock);
          UnlockPage(page);
page_mapped:
          if (--max_mapped >= 0)
              continue;

          /*
           * Alert! We've found too many mapped pages on the
           * inactive list, so we start swapping out now!
           */
          spin_unlock(&pagemap_lru_lock);
          swap_out(priority, gfp_mask, classzone);
          return nr_pages;
      }

If there is no swap space, then unfreeable pages are left on the inactive queue and the vmtree is walked rather than going through the rest of the inactive queue. I believe something like

      /*
       * this is the non-racy check for busy page.
       */
      if (!page->mapping || !is_page_cache_freeable(page)) {
          spin_unlock(&pagecache_lock);
          UnlockPage(page);
page_mapped:
                      /* If we don't have any swap space left, there
                         is no reason to worry about pages that do
                         not have swap associated with them, there
                         is nothing we can do about it. */
                      if (!page->mapping && !swap_avail()) {
                              /* Let's make the page active since we
                                 cannot swap it out.  It get's it off
                                 the inactive list. */
                              spin_unlock(&pagemap_lru_lock);
                              activate_page(page);
                              ClearPageReferenced(page);
                              spin_lock(&pagemap_lru_lock);
                              continue;
                      }
          if (--max_mapped >= 0)
              continue;

          /*
           * Alert! We've found too many mapped pages on the
           * inactive list, so we start swapping out now!
           */
          spin_unlock(&pagemap_lru_lock);
          swap_out(priority, gfp_mask, classzone);
          return nr_pages;
      }

will work better when there is no swap space available. If this change is made, it may also be necessary to limit refill_inactive to prevent it from using too much cpu. This bug can be triggered with the attached code and the correct parameters. In particular on a 3 gigabyte machine with no swap,

for i in $(seq 0 9); do dd if=/dev/zero of=file$i bs=1024k count=512; done killmm 1032735283 2 9

Usually causes an out of memory error when there is hundreds of megabytes of cache.

Ross