Skip to contentSkip to conversation

rsync out of memory 2.6.8-rc2

DISCUSSION Klaus Dittrich 3 participants 4 messages Jul 19, 2004 – Jul 26, 2004 1 message not in the archive

lore
Share & tools
lore

Klaus Dittrich

author
#0 lore raw

Nick your mail got lost here, sorry.

I did rsync -av xeon2:/disc1 . on the target system. xeon2 is the system that gets out of memory. /disc1 is 60GB and contains several thousand files. I think this is a common backup situation.

I found out I could trigger the memory outage using du -s /disc1 too.

Signature · 1 line
Klaus
#1 lore raw

dentry cache leak? Re: rsync out of memory 2.6.8-rc2

Quoted 1 line · Klaus Dittrich

I found out I could trigger the memory outage using du -s /disc1 too.

Including crashing and/or running out of swap? That would indicate that the dentry cache is not cleaning itself up, or that something is wrong with reference counting.

Can you run 'cat fs/dentry-state' before and after the du -s? (assuming there is an 'after'. Also, which fs is /disc1 on? any messages in dmesg?

dentry-state
------------

Status of the directory cache. Since directory entries are dynamically allocated and deallocated, this file indicates the current status. It holds six values, in which the last two are not used and are always zero. The others are listed in table 2-1.

Table 2-1: Status files of the directory cache
..............................................................................

 File       Content                                                            
 nr_dentry  Almost always zero                                                 
 nr_unused  Number of unused cache entries

age_limit
in seconds after the entry may be reclaimed, when memory is
short
want_pages internally

Signature · 2 lines
http://www.PowerDNS.com      Open source, database driven DNS Software 
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#3 lore raw

Re: dentry cache leak? Re: rsync out of memory 2.6.8-rc2

Quoted 47 lines · bert hubert

/proc/fs/dentry-state of course, sorry

Quoted 39 lines · bert hubert
Quoted 3 lines · Klaus Dittrich

I found out I could trigger the memory outage using du -s /disc1 too.

Including crashing and/or running out of swap? That would indicate that the dentry cache is not cleaning itself up, or that something is wrong with reference counting.

Can you run 'cat fs/dentry-state' before and after the du -s? (assuming there is an 'after'. Also, which fs is /disc1 on? any messages in dmesg?

dentry-state
------------

Status of the directory cache. Since directory entries are dynamically allocated and deallocated, this file indicates the current status. It holds six values, in which the last two are not used and are always zero. The others are listed in table 2-1.

Table 2-1: Status files of the directory cache ..............................................................................
File Content nr_dentry Almost always zero nr_unused Number of unused cache entries age_limit
in seconds after the entry may be reclaimed, when memory is
short want_pages internally

Signature · 6 lines
http://www.PowerDNS.com      Open source, database driven DNS Software http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

The fs is ext2.cat /proc/sys/fs/dentry-state
Output of cat /proc/sys/fs/dentry-state before and after processes got killed.

891083  888395  45      0       0       0
1142933 1085759 45      0       0       0

--
Klaus

#4 lore raw

Re: dentry cache leak? Re: rsync out of memory 2.6.8-rc2

Quoted 6 lines · [email protected] (Klaus Dittrich) wrote:

The fs is ext2.cat /proc/sys/fs/dentry-state
Output of cat /proc/sys/fs/dentry-state before and after processes got killed.

 891083  888395  45      0       0       0
 1142933 1085759 45      0       0       0

This bug is probably unique to yourself and a couple of other people. If it was hitting everyone, all the machines in the world would be going oom during the nightly updatedb run.

Can you try a different compiler version? And try disabling any "unusual" config options, such as filesystem quotas, extended attributes, etc?

Can you narrow the onset of the problem down to any particular kernel snapshot?

Try disabling laptop_mode, if it's set.

Carefully review the logs for any oopses: oopsing while holding shrinker_sem will do this for sure.