Skip to contentSkip to conversation

slow directory listing

DISCUSSION Ron Peterson 2 participants 3 messages Jun 10, 2005 – Jun 13, 2005

lore
Share & tools
lore
#0 lore raw

I'm setting up a new mail server, and am testing/tweaking IO. I have two directories: /test/a which contains 750 mbox files totalling 8GB, and /test/a2, which contains the exact same number of files, same names, all zero length.

I am using ext3. I have done this experiment with both indexed and non-indexed directories (mke2fs -O dir_index ...). I have also tried setting the noatime mount option.

The times taken to do a directory listing are significantly different.

1037# time ls /test/a2 > /dev/null

real    0m0.006s
user    0m0.000s
sys     0m0.006s

1038# time ls /test/a > /dev/null

real    0m5.244s
user    0m4.875s
sys     0m0.346s

If I refer to a specific file, there's still a difference, but only 5x, vs. 875x above.

1044# time ls a/anmbox > /dev/null

real    0m0.010s
user    0m0.009s
sys     0m0.002s

Fri Jun 10 10:31:02 root@slush:/db/tmp
1045# time ls a2/anmbox > /dev/null

real    0m0.002s
user    0m0.001s
sys     0m0.001s

I'm assuming this is normal behaviour. (?) However, I'd like to understand what's happening a little better, and I'm wondering if there's anything I'm overlooking vis-a-vis tuning my filesystem properly for this type of application.

Linux 2.6.11.11 on Debian Sarge. Dell 2800 w/ LSI Megaraid on PCI/E to Utra320 SCSI disks.

Signature · 4 lines
Ron Peterson
Network & Systems Manager
Mount Holyoke College
http://www.mtholyoke.edu/~rpeterso
#1 lore raw
Quoted 6 lines · rpeterso

I'm setting up a new mail server, and am testing/tweaking IO. I have two directories: /test/a which contains 750 mbox files totalling 8GB, and /test/a2, which contains the exact same number of files, same names, all zero length.
...
The times taken to do a directory listing are significantly different.

I've become more confused, if that's possible. I was just editing some test script in emacs. As part of the script creation process I used the M-! command to pipe the output of 'ls /test/a' into a buffer. It snapped back almost instantly.

So this seems to have something to do w/ bash, not filesystems. I think. So seems OT for this list, but I still don't understand where the sluggishness comes from.

BTW, I just did the same test w/ XFS and the results were almost identical.

Best.

Signature · 4 lines
Ron Peterson
Network & Systems Manager
Mount Holyoke College
http://www.mtholyoke.edu/~rpeterso
#2 lore raw
In reply to Ron Peterson

On Fri, Jun 10, 2005 at 10:37:20AM -0400, Ron Peterson wrote:
[snip]

Quoted 3 lines

The times taken to do a directory listing are significantly different.

1037# time ls /test/a2 > /dev/null

[snip]

Do you get the same results if you use ls with the -U (unsorted) flag?

Regards: David Weinehall

Signature · 3 lines
 /) David Weinehall <[email protected]> /) Northern lights wander      (\
//  Maintainer of the v2.0 kernel   //  Dance across the winter sky //
\)  http://www.acc.umu.se/~tao/    (/   Full colour fire           (/