Skip to contentSkip to conversation

Delaying writes to disk when there's no need

DISCUSSION Tim Connors 2 participants 2 messages Mar 28, 2003 – Mar 30, 2003 3 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw
Quoted 11 lines · you
Quoted 10 lines · Helge Hafting
Quoted 3 lines · Erik Hensema

In all kernels I've tested writes to disk are delayed a long time even when there's no need to do so.

Short answer - it is supposed to do that!

Quoted 4 lines

A very simple test shows this: on an otherwise idle system, create a tar of a NFS-mounted filesystem to a local disk. The kernel starts writing out the data after 30 seconds, while a slow and steady stream would be much nicer to the system, I think.

Agreed. We have a cluster which is writing on average something like 20 Megs/sec/node. We had to lower the write threshold from 30% to 0%, because with the constant writing, linux will buffer it for 30 secs, fill up RAM, try to empty the write-cache, stall, wash, rinse, repeat. Because it was being filled up at roughly the rate it was being emptied, once it got 30% behind, there was no catching up, so the realtime system would lose data. Ouch.

Quoted 3 lines
Quoted 3 lines

You're wrong then. There's no need for a slow steady stream, why do you want that. Of course you can set up cron to run sync at regular (short) intervals to achieve this.

Last time I checked, cron had 1 minute resolution.

Quoted 6 lines

I see that. However, I don't see why the kernel is writing out data as agressively as it does now. Delaying a write for 30 seconds isn't the
problem: the aggressive writes are. Since the disks are otherwise idle, the kernel can gently start writing out the dirty cache. No need to try and write 40 MB in 1 sec when you can write 10 MB/sec in 4 seconds.

Quoted 7 lines

[...]

Quoted 2 lines

For more detailed information, read a book about how filesystems and disk caching works.

I'm just reporting what's happening to me in practice, I don't really care about what should happen in theory.

Exactly.

Helge's comment about /tmp files and rewriting files multiple times:
in real life, how often does this happen? How often do you overwrite one file many times in 30 seconds? The occasional 20 kilobyte /tmp file perhaps, but I doubt it matters in real life. In real life, when writing to disk constantly (not just scientific applications - I believe this happens in the real world too!), waiting for 30 seconds is a liability!

Signature · 3 lines
TimC -- http://astronomy.swin.edu.au/staff/tconnors/

White dwarf seeks red giant star
#4 lore raw
Quoted 13 lines · Tim Connors
Quoted 11 lines · you
Quoted 10 lines · Helge Hafting
Quoted 3 lines · Erik Hensema

In all kernels I've tested writes to disk are delayed a long time even when there's no need to do so.

Short answer - it is supposed to do that!

Quoted 4 lines

A very simple test shows this: on an otherwise idle system, create a tar of a NFS-mounted filesystem to a local disk. The kernel starts writing out the data after 30 seconds, while a slow and steady stream would be much nicer to the system, I think.

Quoted 8 lines

Agreed. We have a cluster which is writing on average something like 20 Megs/sec/node. We had to lower the write threshold from 30% to 0%, because with the constant writing, linux will buffer it for 30 secs, fill up RAM, try to empty the write-cache, stall, wash, rinse, repeat. Because it was being filled up at roughly the rate it was being emptied, once it got 30% behind, there was no catching up, so the realtime system would lose data. Ouch.

Nothing can help you if you're getting input at the rate you can write it. You need ability to write at least somewhat faster no matter how buffering is done.

Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

You're wrong then. There's no need for a slow steady stream, why do you want that. Of course you can set up cron to run sync at regular (short) intervals to achieve this.

Last time I checked, cron had 1 minute resolution.

If you need to sync more often than once per minute, consider this shellscript:
for ((;;)) ; do sleep 1 ; sync ; done

[...]

Quoted 3 lines

Helge's comment about /tmp files and rewriting files multiple times:
in real life, how often does this happen? How often do you overwrite one file many times in 30 seconds?

_Every_ time you write a file in chunks not perfectly aligned on block boundaries.

Quoted 5 lines

The occasional 20 kilobyte /tmp
file perhaps, but I doubt it matters in real life. In real life, when writing to disk constantly (not just scientific applications - I believe this happens in the real world too!), waiting for 30 seconds is a liability!

Only if that 30-second wait makes linux buffer more than it can handle. That may indeed be a problem, but buffering up _some_ data before writing is still a good idea.

Helge Hafting