when are buffers/caches flushed?

DISCUSSION Steven Scholz 3 participants 9 messages Aug 18, 2005 libata

lore

Share & tools

lore

Steven Scholz

author

2005-08-18 10:51 UTC #0 lore raw

Hi there,

According to an (old) man page of sync(2)

According to the standard specification (e.g., SVID), sync() schedules the writes, but may return before the
actual writing is done. However, since version 1.3.20
Linux does actually wait. (This still does not guarantee data integrity: modern disks have large caches.)

How about recent kernels? Does sync() block until buffers are flushed? How can I find out if the disk caches are actually flushed?

I want to make sure that all data is flushed to my disk drive before powering down the system.

Thanks.

--
Steven

Tejun Heo

2005-08-18 11:24 UTC #1 lore raw

Quoted 10 lines · Steven Scholz

Hi there,
According to an (old) man page of sync(2)
According to the standard specification (e.g., SVID), sync() schedules the writes, but may return before the
actual writing is done. However, since version 1.3.20
Linux does actually wait. (This still does not guarantee data integrity: modern disks have large caches.)

Quoted 7 lines

How about recent kernels? Does sync() block until buffers are flushed? How can I find out if the disk caches are actually flushed?
I want to make sure that all data is flushed to my disk drive before powering down the system.
Thanks.

All disk caches are flushed before shutdown via the following path.

kernel/sys.c::sys_reboot() kernel/drivers/base/power/shutdown.c::device_shutdown()
driver specific ->shutdown callback, for ide disks, the path is
drivers/ide/ide-disk.c::ide_device_shutdown() drivers/ide/ide-disk.c::ide_cacheflush_p() drivers/ide/ide-disk.c::do_idedisk_flushcache()

And, AFAIK, sync() doesn't flush disk caches.

Signature · 1 line

tejun

Steven Scholz

author

2005-08-18 11:34 UTC #2 lore raw

Quoted 22 lines · Tejun Heo

Quoted 10 lines · Steven Scholz
Hi there,
According to an (old) man page of sync(2)
According to the standard specification (e.g., SVID), sync() schedules the writes, but may return before the
actual writing is done. However, since version 1.3.20
Linux does actually wait. (This still does not guarantee data integrity: modern disks have large caches.)
Quoted 7 lines
How about recent kernels? Does sync() block until buffers are flushed? How can I find out if the disk caches are actually flushed?
I want to make sure that all data is flushed to my disk drive before powering down the system.
Thanks.
All disk caches are flushed before shutdown via the following path.

Quoted 8 lines

kernel/sys.c::sys_reboot() kernel/drivers/base/power/shutdown.c::device_shutdown()
driver specific ->shutdown callback, for ide disks, the path is
drivers/ide/ide-disk.c::ide_device_shutdown() drivers/ide/ide-disk.c::ide_cacheflush_p() drivers/ide/ide-disk.c::do_idedisk_flushcache()
And, AFAIK, sync() doesn't flush disk caches.

How about umount?

--
Steven

Signature · 6 lines

Steven Scholz

imc Measurement & Control               imc Meßsysteme GmbH
Voltastr. 5                             Voltastr. 5
13355 Berlin                            13355 Berlin
Germany                                 Deutschland

Tejun Heo

2005-08-18 11:59 UTC #3 lore raw

Quoted 31 lines · Steven Scholz

Quoted 29 lines · Tejun Heo
Quoted 10 lines · Steven Scholz
Hi there,
According to an (old) man page of sync(2)
According to the standard specification (e.g., SVID), sync() schedules the writes, but may return before the
actual writing is done. However, since version 1.3.20
Linux does actually wait. (This still does not guarantee data integrity: modern disks have large caches.)
Quoted 7 lines
How about recent kernels? Does sync() block until buffers are flushed? How can I find out if the disk caches are actually flushed?
I want to make sure that all data is flushed to my disk drive before powering down the system.
Thanks.
All disk caches are flushed before shutdown via the following path.
kernel/sys.c::sys_reboot() kernel/drivers/base/power/shutdown.c::device_shutdown()
driver specific ->shutdown callback, for ide disks, the path is
drivers/ide/ide-disk.c::ide_device_shutdown() drivers/ide/ide-disk.c::ide_cacheflush_p() drivers/ide/ide-disk.c::do_idedisk_flushcache()

Quoted 5 lines

Quoted 1 line
And, AFAIK, sync() doesn't flush disk caches.
How about umount?

[CC'ing Bartlomiej (Hi!)]

Hmmm, umount doesn't. I think maybe adding cache flushing to sync and
umount can be helpful.

And one more thing, ide-disk doesn't flush cache when shutting down.
It flushes only when rebooting.

from ide_device_shutdown()...

#ifdef CONFIG_ALPHA
/* On Alpha, halt(8) doesn't actually turn the machine off,
it puts you into the sort of firmware monitor. Typically, it's used to boot another kernel image, so it's not much different from reboot(8). Therefore, we don't need to spin down the disk in this case, especially since Alpha firmware doesn't handle disks in standby mode properly. On the other hand, it's reasonably safe to turn the power off when the shutdown process reaches the firmware prompt, as the firmware initialization takes rather long time - at least 10 seconds, which should be sufficient for the disk to expire its write cache. */
if (system_state != SYSTEM_POWER_OFF) {

#else
	if (system_state == SYSTEM_RESTART) {
#endif
		ide_cacheflush_p(drive);
		return;
	}

It seems that the if clause are there due to some historic thing. The
comment suggests that the body in the if clause spun down the drive previously, and we needed the if to avoid spinning down unnecessarily, and the if is left there after we changed the body to flush cache.

Bartlomiej, am I missing something?

Signature · 1 line

tejun

Steven Scholz

author

2005-08-18 12:12 UTC #4 lore raw

Tejun,

Quoted 16 lines

Quoted 16 lines
Quoted 15 lines
Quoted 4 lines
How about recent kernels? Does sync() block until buffers are flushed? How can I find out if the disk caches are actually flushed?
I want to make sure that all data is flushed to my disk drive before powering down the system.
All disk caches are flushed before shutdown via the following path.
kernel/sys.c::sys_reboot() kernel/drivers/base/power/shutdown.c::device_shutdown()
driver specific ->shutdown callback, for ide disks, the path is
drivers/ide/ide-disk.c::ide_device_shutdown() drivers/ide/ide-disk.c::ide_cacheflush_p() drivers/ide/ide-disk.c::do_idedisk_flushcache()
And, AFAIK, sync() doesn't flush disk caches.

Quoted 7 lines

Quoted 2 lines
How about umount?
[CC'ing Bartlomiej (Hi!)]
Hmmm, umount doesn't. I think maybe adding cache flushing to sync and
umount can be helpful.

Ehm. So you're saying umount does _not_ flush the disk caches? BUT it will flush the the fs buffers, right?

Quoted 2 lines

And one more thing, ide-disk doesn't flush cache when shutting down. It
flushes only when rebooting.

How about ide-cs then? When I unregister a drive?

--
Steven

Tejun Heo

2005-08-18 12:19 UTC #5 lore raw

Quoted 23 lines · Steven Scholz

Tejun,
Quoted 21 lines
Quoted 20 lines
Quoted 8 lines
Quoted 4 lines
How about recent kernels? Does sync() block until buffers are flushed? How can I find out if the disk caches are actually flushed?
I want to make sure that all data is flushed to my disk drive before powering down the system.
All disk caches are flushed before shutdown via the following path.
Quoted 8 lines
kernel/sys.c::sys_reboot() kernel/drivers/base/power/shutdown.c::device_shutdown()
driver specific ->shutdown callback, for ide disks, the path is
drivers/ide/ide-disk.c::ide_device_shutdown() drivers/ide/ide-disk.c::ide_cacheflush_p() drivers/ide/ide-disk.c::do_idedisk_flushcache()
And, AFAIK, sync() doesn't flush disk caches.
How about umount?

Quoted 7 lines

Quoted 4 lines
[CC'ing Bartlomiej (Hi!)]
Hmmm, umount doesn't. I think maybe adding cache flushing to sync
and umount can be helpful.
Ehm. So you're saying umount does _not_ flush the disk caches? BUT it will flush the the fs buffers, right?

Yeap, but it seems that currently ide-disk seems to flush only on
reboot not on halt. As soon as Bartlomiej confirms it, I'll submit a patch or he'll fix it.

Quoted 7 lines

Quoted 3 lines
And one more thing, ide-disk doesn't flush cache when shutting down.
It flushes only when rebooting.
How about ide-cs then? When I unregister a drive?

ide-cs is low-level driver, it just registers ide device with upper
ide midlayer and ide-disk handles all the rest, so it's all the same there.

Signature · 1 line

tejun

Bartlomiej Zolnierkiewicz

2005-08-18 12:24 UTC #6 lore raw

In reply to Tejun Heo

Hi,

Quoted 36 lines · Tejun Heo

Quoted 35 lines · Steven Scholz
Quoted 31 lines · Tejun Heo
Quoted 10 lines · Steven Scholz
Hi there,
According to an (old) man page of sync(2)
According to the standard specification (e.g., SVID), sync() schedules the writes, but may return before the
actual writing is done. However, since version 1.3.20
Linux does actually wait. (This still does not guarantee data integrity: modern disks have large caches.)
Quoted 8 lines
How about recent kernels? Does sync() block until buffers are flushed? How can I find out if the disk caches are actually flushed?
I want to make sure that all data is flushed to my disk drive before powering down the system.
Thanks.
All disk caches are flushed before shutdown via the following path.
kernel/sys.c::sys_reboot() kernel/drivers/base/power/shutdown.c::device_shutdown()
driver specific ->shutdown callback, for ide disks, the path is
drivers/ide/ide-disk.c::ide_device_shutdown() drivers/ide/ide-disk.c::ide_cacheflush_p() drivers/ide/ide-disk.c::do_idedisk_flushcache()
And, AFAIK, sync() doesn't flush disk caches.

Quoted 7 lines

Quoted 2 lines
How about umount?
[CC'ing Bartlomiej (Hi!)]
Hmmm, umount doesn't. I think maybe adding cache flushing to sync and
umount can be helpful.

I think that cache is flushed on umount through idedisk_release().

Quoted 2 lines

And one more thing, ide-disk doesn't flush cache when shutting down.
It flushes only when rebooting.

Please see ide_disk_remove().

Quoted 22 lines

from ide_device_shutdown()...
#ifdef CONFIG_ALPHA
/* On Alpha, halt(8) doesn't actually turn the machine off,
it puts you into the sort of firmware monitor. Typically, it's used to boot another kernel image, so it's not much different from reboot(8). Therefore, we don't need to spin down the disk in this case, especially since Alpha firmware doesn't handle disks in standby mode properly. On the other hand, it's reasonably safe to turn the power off when the shutdown process reaches the firmware prompt, as the firmware initialization takes rather long time - at least 10 seconds, which should be sufficient for the disk to expire its write cache. */
if (system_state != SYSTEM_POWER_OFF) {
#else
        if (system_state == SYSTEM_RESTART) {
#endif
                ide_cacheflush_p(drive);
                return;
        }

Quoted 4 lines

It seems that the if clause are there due to some historic thing. The
comment suggests that the body in the if clause spun down the drive previously, and we needed the if to avoid spinning down unnecessarily, and the if is left there after we changed the body to flush cache.

It is a correct workaround for Alpha firmware, please see the original patch:
http://linux.bkbits.net:8080/linux-2.6/cset@40bec934PKpDSgX_q3NCTdKXxyk6bw?nav=index.html|src/|src/drivers|src/drivers/ide|related/drivers/ide/ide-disk.c

The thing is that we want to avoid calling dev->bus->suspend() on system_state == SYSTEM_HALT.

Bartlomiej

Tejun Heo

2005-08-18 12:44 UTC #7 lore raw

Quoted 42 lines · Bartlomiej Zolnierkiewicz

Hi,
Quoted 39 lines · Tejun Heo
Quoted 37 lines · Steven Scholz
Quoted 23 lines · Tejun Heo
Quoted 10 lines · Steven Scholz
Hi there,
According to an (old) man page of sync(2)
According to the standard specification (e.g., SVID), sync() schedules the writes, but may return before the
actual writing is done. However, since version 1.3.20
Linux does actually wait. (This still does not guarantee data integrity: modern disks have large caches.)
Quoted 8 lines
How about recent kernels? Does sync() block until buffers are flushed? How can I find out if the disk caches are actually flushed?
I want to make sure that all data is flushed to my disk drive before powering down the system.
Thanks.
All disk caches are flushed before shutdown via the following path.
Quoted 8 lines
kernel/sys.c::sys_reboot() kernel/drivers/base/power/shutdown.c::device_shutdown()
driver specific ->shutdown callback, for ide disks, the path is
drivers/ide/ide-disk.c::ide_device_shutdown() drivers/ide/ide-disk.c::ide_cacheflush_p() drivers/ide/ide-disk.c::do_idedisk_flushcache()
And, AFAIK, sync() doesn't flush disk caches.
How about umount?

Quoted 7 lines

Quoted 4 lines
[CC'ing Bartlomiej (Hi!)]
Hmmm, umount doesn't. I think maybe adding cache flushing to sync and
umount can be helpful.
I think that cache is flushed on umount through idedisk_release().

Ah... right. I was testing while another partition is still mounted.

Quoted 5 lines

Quoted 2 lines
And one more thing, ide-disk doesn't flush cache when shutting down.
It flushes only when rebooting.
Please see ide_disk_remove().

Hmmm, disks are not removed on shutdown, so it's not called on
shutdown.

Quoted 27 lines

Quoted 22 lines
from ide_device_shutdown()...
#ifdef CONFIG_ALPHA
/* On Alpha, halt(8) doesn't actually turn the machine off,
it puts you into the sort of firmware monitor. Typically, it's used to boot another kernel image, so it's not much different from reboot(8). Therefore, we don't need to spin down the disk in this case, especially since Alpha firmware doesn't handle disks in standby mode properly. On the other hand, it's reasonably safe to turn the power off when the shutdown process reaches the firmware prompt, as the firmware initialization takes rather long time - at least 10 seconds, which should be sufficient for the disk to expire its write cache. */
if (system_state != SYSTEM_POWER_OFF) {
#else
        if (system_state == SYSTEM_RESTART) {
#endif
                ide_cacheflush_p(drive);
                return;
        }
Quoted 4 lines
It seems that the if clause are there due to some historic thing. The
comment suggests that the body in the if clause spun down the drive previously, and we needed the if to avoid spinning down unnecessarily, and the if is left there after we changed the body to flush cache.

Quoted 5 lines

It is a correct workaround for Alpha firmware, please see the original patch:
http://linux.bkbits.net:8080/linux-2.6/cset@40bec934PKpDSgX_q3NCTdKXxyk6bw?nav=index.html|src/|src/drivers|src/drivers/ide|related/drivers/ide/ide-disk.c
The thing is that we want to avoid calling dev->bus->suspend() on system_state == SYSTEM_HALT.

Oh.. right. We're calling either cacheflush_p or suspend and as
suspending implies flushing, we should be okay. Sorry about unnecessary noise.

Steven, ide-disk's are correctly flushed on shutdown and umount.
Again, sorry about the confusion.

Thanks.

Signature · 1 line

tejun

Bartlomiej Zolnierkiewicz

2005-08-18 13:03 UTC #8 lore raw

Quoted 3 lines · Tejun Heo

Hmmm, disks are not removed on shutdown, so it's not called on
shutdown.

Yep, I got confused that you meant driver removal.

Quoted 1 line

Quoted 1 line
Quoted 1 line
from ide_device_shutdown()...

above is called on device shutdown

Thanks,
Bartlomiej