Skip to contentSkip to conversation

IDE and locking

DISCUSSION [email protected] 2 participants 2 messages Feb 16, 2004 – Feb 22, 2004

lore
Share & tools
lore
#0 lore raw

Hi,

I'm working on an embedded system here that
basically counts ethernet packets. I'm using
2.4.20 and I'm noticing that when files are
read from the compact flash (ext2), ethernet
packets are being dropped.

So to work around the problem, I'm precaching
all the files (12MB) at boot like:
find / -type f | while read file; do dd if=$file of=/dev/null; done

So questions.

1. Is this due to the BKL around the IDE subsystem?
2. Are writes as susceptible as reads to the problem? I'm guessing if the compact flash had a write cache buffer I would be OK as I only write max 2MB at a time?
3. There was talk about removing the BKL in 2.6 with reference to:
http://sourceforge.net/project/showfiles.php?group_id=8875
Has this work been included?
4. Is there a max number of files that can be cached by linux?
5. Will the files be removed at any stage from the cache if there is no memory pressure?
6. Can I reserve memory for the file cache?

cheers.

Signature · 1 line
Pádraig Brady - http://www.pixelbeat.org
#1 lore raw

This doesn't answer your real question, but I can't resist noticing that the above would be more efficient as:

find / -type f | xargs wc >/dev/null

Several times fewer system calls, with just a few exec's of 'wc', instead of exec'ing a 'dd' per file, and with just one read per 16 Kb, instead of both a read and write per 512 bytes.

Granted - I'm being silly to mention this - your compact flash device is obviously the bottleneck here. Doesn't really matter if the cpu spends 1% or 2% of its time outside the idle loop.

Quoted 1 line

4. Is there a max number of files that can be cached by linux?

As long as there is no memory pressure, I suspect it keeps caching more.

Quoted 2 lines

5. Will the files be removed at any stage from the cache if there is no memory pressure?

I don't think so - stuff seems to stay in cache 'forever', if no one else wants the memory.

Quoted 1 line

6. Can I reserve memory for the file cache?

Not that I know of. If it were read-only data, I might try a ram disk, but your application apparently is read-write.

It might be (long shot, here) that just caching the directories and inodes was enough, without caching the file contents:

find / ! -true

Signature · 3 lines
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <[email protected]> 1.650.933.1373