IDE and locking
Hi,
I'm working on an embedded system here that
basically counts ethernet packets. I'm using
2.4.20 and I'm noticing that when files are
read from the compact flash (ext2), ethernet
packets are being dropped.
So to work around the problem, I'm precaching
all the files (12MB) at boot like:
find / -type f | while read file; do dd if=$file of=/dev/null; done
So questions.
1. Is this due to the BKL around the IDE subsystem?
2. Are writes as susceptible as reads to the problem? I'm guessing if the compact flash had a write cache buffer I would be OK as I only write max 2MB at a time?
3. There was talk about removing the BKL in 2.6 with reference to:
http://sourceforge.net/project/showfiles.php?group_id=8875
Has this work been included?
4. Is there a max number of files that can be cached by linux?
5. Will the files be removed at any stage from the cache if there is no memory pressure?
6. Can I reserve memory for the file cache?
cheers.
Signature · 1 line
Pádraig Brady - http://www.pixelbeat.org
This doesn't answer your real question, but I can't resist noticing that the above would be more efficient as:
find / -type f | xargs wc >/dev/null
Several times fewer system calls, with just a few exec's of 'wc', instead of exec'ing a 'dd' per file, and with just one read per 16 Kb, instead of both a read and write per 512 bytes.
Granted - I'm being silly to mention this - your compact flash device is obviously the bottleneck here. Doesn't really matter if the cpu spends 1% or 2% of its time outside the idle loop.
Quoted 1 line
4. Is there a max number of files that can be cached by linux?
As long as there is no memory pressure, I suspect it keeps caching more.
Quoted 2 lines
5. Will the files be removed at any stage from the cache if there is no memory pressure?
I don't think so - stuff seems to stay in cache 'forever', if no one else wants the memory.
Quoted 1 line
6. Can I reserve memory for the file cache?
Not that I know of. If it were read-only data, I might try a ram disk, but your application apparently is read-write.
It might be (long shot, here) that just caching the directories and inodes was enough, without caching the file contents:
find / ! -true
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373