Skip to contentSkip to conversation

Why is one sync() not enough?

DISCUSSION Nico Schottelius 10 participants 13 messages Jun 14, 2005 – Jun 15, 2005

lore
Share & tools
lore
#0 lore raw

Hello again!

When my system shuts down and init calls sync() and after that umount and then reboot, the filesystem is left in an unclean state.

If I do sync() two times (one before umount, one after umount) it seems to work.

Can someboy explain that to me?

Architecture is ppc and x86.

Nico

Signature · 3 lines
Keep it simple & stupid, use what's available.
Please use pgp encryption: 8D0E 27A4 is my id.
http://nico.schotteli.us | http://linux.schottelius.org
#1 lore raw

hoi :)

Quoted 5 lines · Nico Schottelius

When my system shuts down and init calls sync() and after that umount and then reboot, the filesystem is left in an unclean state.

If I do sync() two times (one before umount, one after umount) it seems to work.

unmounting the filesystem writes to the disk.
If you don't wait for those writes to reach the disk, then you still have a dirty filesystem.

Signature · 1 line
Martin Waitz
#2 lore raw
Quoted 6 lines · Martin Waitz

hoi :)

Quoted 3 lines · Nico Schottelius

When my system shuts down and init calls sync() and after that umount and then reboot, the filesystem is left in an unclean state.

Quoted 7 lines
Quoted 2 lines

If I do sync() two times (one before umount, one after umount) it seems to work.

unmounting the filesystem writes to the disk.
If you don't wait for those writes to reach the disk, then you still have a dirty filesystem.

If I remember correctly, sync (at least in the past) schedules the dumping of buffers to disk, but may return before everything is actually written. Thus it was common in ages past to run sync ; sync ; halt when shutting down a system since even though the first sync might return before writing was done, the second sync wouldn't start before the first was really done, thus when the second sync returned you'd know that at least the first one had completed.
I believe sync in recent Linux's actually waits for the data to be written and never returns early, so I don't think this is relevant any more - and everyone uses shutdown these days anyway. But go read the implementation of sync if you really want to know.

Signature · 3 lines
Jesper Juhl <[email protected]>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html
#3 lore raw
Quoted 3 lines · Jesper Juhl

actually written. Thus it was common in ages past to run sync ; sync ; halt when shutting down a system since even though the first sync might return before writing was done

Actually, it was:

# sync
# sync
# sync
# halt

If you enter 'sync;sync;sync' all 3 will get scheduled almost on top of each other when you hit return. If you hit return on each one, the first one starts running when you hit return, and hopefully completes while you're typing the second and third ones.....

'sync' on recent Linux kernels appears to be synchronous and not return until the data is out the door. If you're seeing 'sync; unmount; sync' being needed, the most likely cause is that the *first* sync is bogus (as unmount will flush the buffers too) - and the *second* sync is flushing out blocks from some file that was opened in the directory *before you did the mount*:

# cd /usr/local  (with nothing mounted on it, so it's an empty dir in /usr)
# touch foo
# tail -f foo &
# mount /usr/local

Now that 'foo' is an open but unreachable file on the /usr filesystem, and will likely cause the need for a sync *for /usr* after unmounting /usr/local at system shutdown.

#4 lore raw
In reply to Jesper Juhl

Jesper Juhl [Tue, Jun 14, 2005 at 05:58:36PM +0200]:

Quoted 6 lines

[sync twice, sync waits for sync]

I believe sync in recent Linux's actually waits for the data to be written and never returns early, so I don't think this is relevant any more - and everyone uses shutdown these days anyway. But go read the implementation of sync if you really want to know.

Thanks for the detailled instructions. I'll have a look at the sync implementation to see how it really works.

Anyway: As I am writing an init-system, I also have to write a shutdown procedure ;-)

Greetings,

Nico

Signature · 3 lines
Keep it simple & stupid, use what's available.
Please use pgp encryption: 8D0E 27A4 is my id.
http://nico.schotteli.us | http://linux.schottelius.org
#5 lore raw
In reply to Nico Schottelius
Quoted 9 lines · Nico Schottelius

Hello again!

When my system shuts down and init calls sync() and after that umount and then reboot, the filesystem is left in an unclean state.

If I do sync() two times (one before umount, one after umount) it seems to work.

That's a bug.

The standards say that sync() is supposed to "start" I/O, or something similarly vague and waffly. The Linux implementation of sync() has always started all I/O and then waited upon all of it before returning from sync().

And umount() itself will sync everything to disk, so the additional sync() calls should be unnecessary.

That being said, if umount was leaving dirty filesystems then about 1000000 people would be complaining. So there's something unusual about your setup.

What filesystem? What kernel version? Any unusual bind mounts, loopback mounts, etc? There must be something there...

#6 lore raw

There are cases where bitmaps can dirty themselves in the FS's (I have had to deal with several of these issues in 2.6) after a sync, and I have found that:

#

# sync
# sync
#

is required on ext2 in some situations due to some race conditions with remote and local clients using device based FS's, so there are some holes -- and comments in ext2 and several other FS's seem to bear this assumption out.

:-)

Jeff

Quoted 38 lines · Andrew Morton
Quoted 10 lines · Nico Schottelius

Hello again!

When my system shuts down and init calls sync() and after that umount and then reboot, the filesystem is left in an unclean state.

If I do sync() two times (one before umount, one after umount) it seems to work.

That's a bug.

The standards say that sync() is supposed to "start" I/O, or something similarly vague and waffly. The Linux implementation of sync() has always started all I/O and then waited upon all of it before returning from sync().

And umount() itself will sync everything to disk, so the additional sync() calls should be unnecessary.

That being said, if umount was leaving dirty filesystems then about 1000000 people would be complaining. So there's something unusual about your setup.

What filesystem? What kernel version? Any unusual bind mounts, loopback mounts, etc? There must be something there...
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#8 lore raw
In reply to Andrew Morton

Hello Andrew,

Quoted 1 line · you

What filesystem?

jfs

Quoted 1 line

What kernel version?

2.6.11.11

Quoted 2 lines

Any unusual bind mounts, loopback
mounts, etc? There must be something there...>

Yes, dm-crypt-mounted-jfs.

So if I understood everything correctly, a simple umount() without a sync() before should be enough?

If so, I'll try that, I am happy about every less function call I need to do.

Nico

P.S.: Helge, nice hint for debugging, cat'ing /proc/mounts before
reboot/power-off/etc.!

Signature · 3 lines
Keep it simple & stupid, use what's available.
Please use pgp encryption: 8D0E 27A4 is my id.
http://nico.schotteli.us | http://linux.schottelius.org
#9 lore raw
Quoted 7 lines · Nico Schottelius

Hello Andrew,

Quoted 1 line · you

What filesystem?

jfs

It would be useful to test with a different filesystem (ext3 mounted with data=writeback is close to equivalent). That'll help us identify the bug.

Quoted 8 lines
Quoted 1 line

What kernel version?

2.6.11.11

Quoted 2 lines

Any unusual bind mounts, loopback
mounts, etc? There must be something there...>

Yes, dm-crypt-mounted-jfs.

Again, if you can temporarily eliminate dm-crypt as well it will help narrow it down.

Quoted 2 lines

So if I understood everything correctly, a simple umount() without a sync() before should be enough?

Yes.

Quoted 2 lines

If so, I'll try that, I am happy about every less function call I need to do.

OK. sync()+umount() shouldn't take any longer than a bare umount() (which has to do a sync too). Unless you have multiple disks, in which case there's no point in syncing the disks which aren't being umounted. But then, I have a feeling that standard util-linux umount(8) does a global sync anyway.

#10 lore raw
In reply to Nico Schottelius
Quoted 11 lines · Nico Schottelius

Hello again!

When my system shuts down and init calls sync() and after that umount and then reboot, the filesystem is left in an unclean state.

If I do sync() two times (one before umount, one after umount) it seems to work.

Can someboy explain that to me?

You shouldn't need those syncs, as umount does its own syncing. There may be other explanations:

* Your reboot actually powers down (or resets) the disk. IDE disks are known for caching stuff, they may indicate that data is written slightly before it actually happens. (The same applies to scsi - if you enable caching there for the little extra performance it buys.)

Rebooting really quickly after umount in such a case can cut power to the disk before it finishes writing. If this is the case, then a few seconds of sleep after umount before reboot will work just as well as that sync. I don't recommend this as a solution, but it is an easy diagnostic!

* Your startup script accidentally mounted the fs twice. (Yes - linux support that, and the first umount won't undo both mounts.) This simply means the fs isn't umounted when you reboot, but an extra sync and you might get lucky.

Check to see that nothing at all is mounted as you reboot.

Helge Hafting

#11 lore raw
Quoted 9 lines · Helge Hafting
Quoted 7 lines · Nico Schottelius

Hello again!

When my system shuts down and init calls sync() and after that umount and then reboot, the filesystem is left in an unclean state.

If I do sync() two times (one before umount, one after umount) it seems to work.

sync before umount is superfluous.

Quoted 18 lines
Quoted 3 lines

Can someboy explain that to me?

You shouldn't need those syncs, as umount does its own syncing. There may be other explanations:

* Your reboot actually powers down (or resets) the disk. IDE disks are known for caching stuff, they may indicate that data is written slightly before it actually happens. (The same applies to scsi - if you enable caching there for the little extra performance it buys.)

Rebooting really quickly after umount in such a case can cut power to the disk before it finishes writing. If this is the case, then a few seconds of sleep after umount before reboot will work just as well as that sync. I don't recommend this as a solution, but it is an easy diagnostic!

Quoted 4 lines

* Your startup script accidentally mounted the fs twice. (Yes - linux support that, and the first umount won't undo both mounts.) This simply means the fs isn't umounted when you reboot, but an extra sync and you might get lucky.

My reboot script is checking (/proc/mounts) for stray rw mounts on reboot, prints a warning and waits for a keypress. This helps spot such things. --
vda

#12 lore raw
Quoted 5 lines · Denis Vlasenko
Quoted 4 lines · Helge Hafting
Quoted 2 lines · Nico Schottelius

Hello again!

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines

When my system shuts down and init calls sync() and after that umount and then reboot, the filesystem is left in an unclean state.

If I do sync() two times (one before umount, one after umount) it seems to work.

sync before umount is superfluous.

BTW, how about sysrq-s and sysrq-u? The same?

Signature · 2 lines
Coywolf Qi Hunt
http://ahbl.org/~coywolf/