slow directory listing
I'm setting up a new mail server, and am testing/tweaking IO. I have two directories: /test/a which contains 750 mbox files totalling 8GB, and /test/a2, which contains the exact same number of files, same names, all zero length.
I am using ext3. I have done this experiment with both indexed and non-indexed directories (mke2fs -O dir_index ...). I have also tried setting the noatime mount option.
The times taken to do a directory listing are significantly different.
1037# time ls /test/a2 > /dev/null
real 0m0.006s user 0m0.000s sys 0m0.006s
1038# time ls /test/a > /dev/null
real 0m5.244s user 0m4.875s sys 0m0.346s
If I refer to a specific file, there's still a difference, but only 5x, vs. 875x above.
1044# time ls a/anmbox > /dev/null
real 0m0.010s user 0m0.009s sys 0m0.002s
Fri Jun 10 10:31:02 root@slush:/db/tmp
1045# time ls a2/anmbox > /dev/null
real 0m0.002s user 0m0.001s sys 0m0.001s
I'm assuming this is normal behaviour. (?) However, I'd like to understand what's happening a little better, and I'm wondering if there's anything I'm overlooking vis-a-vis tuning my filesystem properly for this type of application.
Linux 2.6.11.11 on Debian Sarge. Dell 2800 w/ LSI Megaraid on PCI/E to Utra320 SCSI disks.
Signature · 4 lines
Ron Peterson Network & Systems Manager Mount Holyoke College http://www.mtholyoke.edu/~rpeterso
Quoted 6 lines · rpeterso
I'm setting up a new mail server, and am testing/tweaking IO. I have two directories: /test/a which contains 750 mbox files totalling 8GB, and /test/a2, which contains the exact same number of files, same names, all zero length.
...
The times taken to do a directory listing are significantly different.
I've become more confused, if that's possible. I was just editing some test script in emacs. As part of the script creation process I used the M-! command to pipe the output of 'ls /test/a' into a buffer. It snapped back almost instantly.
So this seems to have something to do w/ bash, not filesystems. I think. So seems OT for this list, but I still don't understand where the sluggishness comes from.
BTW, I just did the same test w/ XFS and the results were almost identical.
Best.
Signature · 4 lines
Ron Peterson Network & Systems Manager Mount Holyoke College http://www.mtholyoke.edu/~rpeterso
On Fri, Jun 10, 2005 at 10:37:20AM -0400, Ron Peterson wrote:
[snip]
Quoted 3 lines
The times taken to do a directory listing are significantly different.
1037# time ls /test/a2 >
/dev/null
[snip]
Do you get the same results if you use ls with the -U (unsorted) flag?
Regards: David Weinehall
Signature · 3 lines
/) David Weinehall <[email protected]> /) Northern lights wander (\ // Maintainer of the v2.0 kernel // Dance across the winter sky // \) http://www.acc.umu.se/~tao/ (/ Full colour fire (/