Fix for hanging resize on root filesystem

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello all -

I recently ran across a bug that was initially suspected to be a reiserfs bug, but turned out to be (I feel) a problem with the LVM userspace utilities.

When a user has their root filesystem on an LV and attempts to resize the LV, the kernel will deadlock while trying to update the atime on the device node.

The following sequence causes the lockup:
open /dev/somedisk
suspend the dm instance
~ -> *_write_super_lockfs flushes and disallows new transactions
read from /dev/somedisk
~ -> if needed, update atime
~ -> dirty inode
~ -> tries to start transaction and waits for transactions to be

~         allowed again (waits forever, unless the LV is unlocked by
~         another process, which will cease the deadlock but cause the
~         lvresize operation to occur with the dm device active)

The attach patch causes dev_open_flags to add the O_NOATIME to the open flags, which will avoid the deadlock by not updating the atime.

Note that the resize on the root LV, when using ext3, will succeed most of the time. This _isn't_ because of correctness, but because it manages to perform the operations without crossing a 1-second interval. If a delay is introduced between one of the reads, for example, it will also deadlock.

The reason I feel this isn't a kernel bug, but rather a userspace bug, is that the LVM code explicitly suspends the device, and then performs operations with implicit writes associated with them. The reads/writes from the device are totally valid, but the inode atime update isn't. The kernel behavior is as expected.

- -Jeff

- --
Jeff Mahoney
SuSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFBQaJcLPWxlyuTD7IRAgmzAJ4h28KLAjeKopQ8/OUkVa7Xg96yTACeL5YI Z2YrBse1UMvAJEfFMeXKAA8=
=YfD4
-----END PGP SIGNATURE-----

diff -rup LVM2.2.00.15/lib/device/dev-io.c LVM2.2.00.15-devel/lib/device/dev-io.c
@@ -300,6 +300,11 @@ int dev_open_flags(struct device *dev, i
 		flags |= O_DIRECT;
 #endif
 
+#ifndef O_NOATIME
+# define O_NOATIME       01000000
+#endif
+        flags |= O_NOATIME;
+
 	if ((dev->fd = open(name, flags, 0777)) < 0) {
 		log_sys_error("open", name);
 		return 0;

Indeed, and if you push harder you'll probably find some trickier problems there too. This is one of the reasons I've said numerous times we don't support pvmove/snapshots of the devices used by the tools themselves. I shall add a test to the tools to detect when people do this. [See also the old 2.4 kernel patch included in the source tree (but not applied by default) which disables the relevant *time updates - write as well as read. I've not had time to look at equivalents for 2.6 yet.]

The way I envisage snapshots/pvmoves being supported most safely is by using temporary copies of the relevant inodes on a filesystem that does not get suspended.

Alasdair

[email protected]