add fsck to util-linux

This adds fsck from latest e2fsprogs git to util-linux. There are only tiny changes to integrate it into the build system and nls setup of util-linux and fixing up the trailing whitespaces quilt is complaining about. I've not yet converted it to the fsprobe helpers as the discussion on those libs is still ongoing and I haven't read up on the fsprobe library either.

Index: util-linux-ng/Makefile.am
===================================================================
@@ -4,6 +4,7 @@ SUBDIRS = \
 	include \
 	disk-utils \
 	fdisk \
+	fsck \
 	getopt \
 	hwclock \
 	login-utils \

Index: util-linux-ng/configure.ac
===================================================================
@@ -564,6 +564,7 @@ AC_CONFIG_FILES([
 Makefile
 disk-utils/Makefile
 fdisk/Makefile
+fsck/Makefile
 getopt/Makefile
 hwclock/Makefile
 include/Makefile

Index: util-linux-ng/fsck/Makefile.am
===================================================================
@@ -0,0 +1,6 @@
+include $(top_srcdir)/config/include-Makefile.am
+
+usrsbinexec_PROGRAMS = fsck
+fsck_SOURCES = fsck.c fsck.h base_device.c
+fsck_LDADD = -lblkid
+man_MANS = fsck.8

It looks like you pulled fsck from the master branch of e2fsprogs git; there is one slight bug fix in the maint branch that hasn't been merged into master yet, commit ed773a263829493e4e4bf612dbec2380cf09349f:

fsck: Ignore /etc/fstab entries for bind mounts

If a user specifies a bind mount with a non-zero fsck pass number, for example:

/foo /bar ext3 bind,defaults 1 3

print a warning and ignore the fstab entry.

Addresses-Red-Hat-Bugzilla: #151533

BTW, I don't like this syntax in the fstab file AT ALL, but it is in use in the wild by at least some Fedora users, and it's not documented in the fstab man page. I'd suggest using a filesystem type of bind, rather than ext3, as the officially "blessed" way of specifying it in fstab, but it badly needs to be documented in the fstab and/or mount man pages. The above patch should probably get included, though, and backwards compatibility for allowing "bind" to be specified in the mount options, and with a warning message that the specifying "bind" in the options field has been deprecated.

So with the above patch included,

For future code movement, I don't mind fsck moving over, but I would like to manage moving over blkid to util-linux-ng myself, as I have some pretty strong feelings about the right way to do things. I am quite willing to add some low-level interfaces so that fsid can use the same fs probing logic, and I'm willing to add some code so that the high-level interfaces of libblkid, if the /dev/disk/by-* links are present and the user isn't asking for information which isn't in the blkid cache, will use the symlinks instead. However, I really don't want to encode a dependency on udev being there, and I think it should be possible to make the fallback be transparent instead of being a compile-time option.

Regards,

- Ted

I'll pick that up.

The syntax is indeed horrible. Is it supported by upstream util-linux?

I've started looking into this, and I think at least for the detect which filesystem we have part libblkid is complete overkill. libvolume_id has a really nice lowlevel API for that that is much more suitable.

So if it was up to me I'd do the following:

- move libvolume_id out of udev
- make mount/fsck use libvolume_id unconditionally for detecting the filesystem type. There's absolute no reason to use anything in libblkid for this, and caching the result doesn't help us at all as we're going to touch the disk anyway as part of the mount/fsck.
- make libblkid use libvolume_id internally for filesystem detection

note that the latter might aswell be a static inclusion of the code, I haven't looked at the details yet.

Another note on moving the libraries into util-linux vs a standalone package:
At least in xfs land people do upgrade xfsprogs frequently and sometimes independent os the distro because new features get added quite a bit, including new filesystem features that require support. Having to upgrade util-linux for that is not very helpful. So I'm not so sure about moving this to util-linux

Sure, it's supported.

mount(8) supports the "bind" as standard mount option (since util-linux 2.10). There is not difference between options from fstab and command line.

It depends.. for example I think that call blindly all probing functions is overkill. The libblkid is firstly trying to recognize FS type by magic string.

unconditionally ... absolute no reason...

Too strong words, especially when there are distributions that depend on some features from libblkid (for example device-mapper support).

I agree that the cache should be optional, rather than mandatory. For example for FS type detection is it overkill, but for LABEL/UUID translation is it good thing (especially on systems without /dev/disk/by-*).

I think we can always change this concept in dependence on feedback from distributors/developers.

Karel

 Karel Zak  <[email protected]>

I think the issue is that if a filesystem is put into /etc/fstab with type = "bind" then it will show up in /etc/mtab as "bind" and if the original filesystem is unmounted the "mount" output will not show the correct type anymore (I tested this with "bind" and "reiserfs" on an ext3 filesystem).

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

No, that's not true, since on a non-udev system (and there are those, including a number of kernel hackers, who don't use udev for a number of reasons --- like not trusting the sysfs/udev breakages, and who want a static /dev) you need an efficient way to do lookup by UUID or by LABEL without needing to search every single device. If you have a some large storage box with 30,000 LUN's, you don't want to search them all at boot time, whether you are using udev, especially if you are only mounting a small subset of the images on demand. So there will be circumstances where you really will want to use the cache, and *not* want to use /dev/disk/by-* as the lookup mechanism.

- Ted

I really don't want to see something like: "the user isn't asking for information which isn't in the blkid cache" thing near the code udev has to call. The cache must be part of the "high-level" blkid functions, and nothing like this should come near the plain probing functions.
The compile-time udev switch and the udev symlink logic is already in the util-linux fsprobe wrapper, and there is no need to move that into blkid now, especially not by trying to make anything "transparent".

The low-level functions need to handle 64bit probing offset and size values, so we can probe partitioned devices.

We also need all filesytem types, all raid metadata probing, and filesystem version number and classification information in the final library, so that udev is able to switch over.

A low-level probe must be able to return:
the classification (filesystem, other, raid, crypto) the filesystem name string (ext3, vfat, xfs, ...) the version of the filesystem (1.3, FAT32, 0.9, ...) a label string (unicode validated and translated to utf8) the raw bytes and the length of the label
a UUID string (some raid sets have free-text UUID's today) the raw UUID string and the length

And we need to be able to call the sets of raid probing and filesystem probing functions independent from each other, so that we are able to skip all raid probing.

I like to see the ~40 volume_id filesystem/raid probing files merged, and the content of probe.c replaced with calls to these files, while preserving the current probing order of blkid.

Thanks,
Kay

Here is another attempt to move fsck over to the util-linux-ng tree. A similar attempt has been done previously by Christoph Hellwig some time ago.

Patch 1 is the initial move with minor adjustments for the build system.
Patch 2 adds support for the fsprobe library wrapper. Patch 3 adds a new option which has been in Suse for some time to ignore mounted filesystems.

Matthias

Allows fsck to work with blkid or volume_id as defined by the configure invocation.

---

 fsck/Makefile.am |   17 +++++++++++++++--
 fsck/fsck.c      |   13 ++++++-------
 2 files changed, 21 insertions(+), 9 deletions(-)

index 59523fc..bf2ac6c 100644
@@ -1,7 +1,20 @@
 include $(top_srcdir)/config/include-Makefile.am
 
 usrsbinexec_PROGRAMS = fsck
-fsck_SOURCES = fsck.c fsck.h base_device.c
-fsck_LDADD = -lblkid
 man_MANS = fsck.8
 
+utils_common = ../mount/sundries.c ../mount/xmalloc.c ../mount/realpath.c ../mount/fsprobe.c
+
+fsck_SOURCES = fsck.c fsck.h base_device.c $(utils_common)
+fsck_LDADD =
+
+if HAVE_BLKID
+utils_common += ../mount/fsprobe_blkid.c
+fsck_LDADD += -lblkid
+endif
+
+if HAVE_VOLUME_ID
+utils_common += ../mount/fsprobe_volumeid.c ../lib/blkdev.c ../lib/linux_version.c
+fsck_LDADD += -lvolume_id
+endif
+

index 6b39b21..c7905c4 100644
@@ -45,7 +45,7 @@
 #include "../config.h"
 #include "nls.h"
 #include "fsck.h"
-#include "blkid/blkid.h"
+#include "../mount/fsprobe.h"
 
 #ifndef _PATH_MNTTAB
 #define	_PATH_MNTTAB	"/etc/fstab"

@@ -104,7 +104,6 @@ struct fs_info *filesys_info = NULL, *filesys_last = NULL;
 struct fsck_instance *instance_list;
 const char *fsck_prefix_path = "/sbin:/sbin/fs.d:/sbin/fs:/etc/fs:/etc";
 char *fsck_path = 0;
-blkid_cache cache = NULL;
 
 static char *string_copy(const char *s)
 {

@@ -282,7 +281,7 @@ static int parse_fstab_line(char *line, struct fs_info **ret_fs)
 	parse_escape(freq);
 	parse_escape(passno);
 
-	dev = blkid_get_devname(cache, device, NULL);
+	dev = fsprobe_get_devname(device);
 	if (dev)
 		device = dev;
 

@@ -307,7 +306,7 @@ static void interpret_type(struct fs_info *fs)
 	
 	if (strcmp(fs->type, "auto") != 0)
 		return;
-	t = blkid_get_tag_value(cache, "TYPE", fs->device);
+	t = fsprobe_get_fstype_by_devname(fs->device);
 	if (t) {
 		free(fs->type);
 		fs->type = t;

@@ -1090,7 +1089,7 @@ static void PRS(int argc, char *argv[])
 					progname);
 				exit(EXIT_ERROR);
 			}
-			dev = blkid_get_devname(cache, arg, NULL);
+			dev = fsprobe_get_devname(arg);
 			if (!dev && strchr(arg, '=')) {
 				/*
 				 * Check to see if we failed because

@@ -1235,7 +1234,7 @@ int main(int argc, char *argv[])
 	bindtextdomain(PACKAGE, LOCALEDIR);
 	textdomain(PACKAGE);
 
-	blkid_get_cache(&cache, NULL);
+	fsprobe_init();
 	PRS(argc, argv);
 
 	if (!notitle)

@@ -1304,6 +1303,6 @@ int main(int argc, char *argv[])
 	}
 	status |= wait_many(FLAG_WAIT_ALL);
 	free(fsck_path);
-	blkid_put_cache(cache);
+	fsprobe_exit();
 	return status;
 }

The big thing here is there needs to be a BIG FAT WARNING that if you want ext4 to work, it's best to make sure fsck is built against the very latest blkid library.

I was thinking about using opensuse for more of testing, based on Markus's encouragement, but given that it's using vol_id, and I need to make some very specific changes so that mount uses the right filesystem for ext4 filesystems (especially since the kernel will be changing the filesystem name from ext4dev to ext4 at some point in the next few months), and opensuse uses vol_id for mount, that became a show-stopper for me.

- Ted

Add option -m to ignore and return an exit code of 0 for mounted filesystems.

---

 fsck/fsck.8.in |    4 +++
 fsck/fsck.c    |   77 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 80 insertions(+), 1 deletions(-)

index 4a12677..087321b 100644
@@ -180,6 +180,10 @@ option,
 will use the specified filesystem type.  If this type is not
 available, then the default file system type (currently ext2) is used. 
 .TP
+.B \-m
+Do not check mounted filesystems and return an exit code of 0
+for mounted filesystems.
+.TP
 .B \-A
 Walk through the
 .I /etc/fstab

The fsck from the latest e2fsprogs git is util-linux-ng git now. It's possible to link it against libblkid (from e2fsprogs or util-linux-ng) and libvolume_id.

Karel

 Karel Zak  <[email protected]>
--
To unsubscribe from this list: send the line "unsubscribe util-linux-ng" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Did you speak to Ted to make sure it's from the right branch? I remember that the one on the master branch wasn't the latests one when I did the first patch.

---

 Makefile.am        |    1 
 configure.ac       |    2 
 fsck/Makefile.am   |    7 
 fsck/base_device.c |  169 +++++++
 fsck/fsck.8.in     |  410 ++++++++++++++++
 fsck/fsck.c        | 1309 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 fsck/fsck.h        |   70 +++
 7 files changed, 1968 insertions(+), 0 deletions(-)

index 245f90e..e2b5ed1 100644
@@ -3,6 +3,7 @@ include $(top_srcdir)/config/include-Makefile.am
 SUBDIRS = \
 	include \
 	disk-utils \
+	fsck \
 	fdisk \
 	getopt \
 	lib \

index a148e28..cacf790 100644
@@ -578,6 +578,8 @@ AC_CONFIG_HEADERS(config.h)
 AC_CONFIG_FILES([
 Makefile
 disk-utils/Makefile
+fsck/Makefile
+fsck/fsck.8
 fdisk/Makefile
 getopt/Makefile
 hwclock/Makefile

new file mode 100644
index 0000000..59523fc
@@ -0,0 +1,7 @@
+include $(top_srcdir)/config/include-Makefile.am
+
+usrsbinexec_PROGRAMS = fsck
+fsck_SOURCES = fsck.c fsck.h base_device.c
+fsck_LDADD = -lblkid
+man_MANS = fsck.8
+

Thanks (also thanks to Christoph)!

I didn't forget about this task. It's still in my TODO list.

It seems that the fsck wrapper in e2fsprogs is optional (--enable-fsck) so we needn't any extra change to e2fsprogs.

It would be nice to add a note

"fsck is deprecated in favor of fsck from util-linux-ng"

to RELEASE-NOTES in e2fsprogs.

What about to move the fsprobe library wrapper to lib/ ?

We also need a --disable-fsck configure option.

Karel

 Karel Zak  <[email protected]>

I don't want to deprecate fsck quite yet from e2fsprogs, because there are some ext4 issues with fsck (or more accurately, fsck's use of vol_id instead of blkid), where it is more convenient if at least for distros that are being aggressive with ext4 testing, if we don't try to move fsck just now.

I'm working with Eric Sandeen from Red Hat to make sure that we can get usable ext4 support for FC9 in rawhide, and making sure that vol_id is doing the right thing with ext4 support is one more moving part that I'd rather not have to coordinate with. So it's going to be much simpler for now if we keep fsck in e2fsprogs, and what I'm telling people who want to play with ext4 is that life will be much simpler if they use the latest e2fsprogs and build mount so that it uses blkid, and NOT vol_id.

Kay and I have had philosophical differences about how hard to try to make in-line detection of filesystems work, and when I've tried to submit patches that fine-tune filesystem id detection, by rejiggering the order in which various filesystems types are tried (based on how thoroughly their mkfs commands scrub the disk to remove false positives for previous filesystems on the disk), they've been rejected by Kay.

So these days, while I'll take any patches to improve blkid's filesystem detection, I've largely stopped trying to send patches in the other direction. And there are some rather delicate tests that have been encoded into blkid to coordinate with ext4 and ext4dev (since at some point when the code is production ready we will be changing the filesystem name from ext4dev to ext4), it's just much simpler if I tell people that in order to support ext4, the only approach I've validated uses fsck from e2fsprogs, and mount compiled to use libblkid.

I'd love to try to work on merging the functionality between vol_id and blkid, and at some point that's great, but I just don't have the time right now to try to coordinate yet another moving part.

- Ted

That's nothing urgent for me.

No problem to postpone this change until util-linux-ng 2.15 (at end of this year). I believe ext4 will be pretty stable at this time :-)

I'm already working on vol_id + blkid merge

http://git.kernel.org/?p=utils/util-linux-ng/util-linux-ng.git;a=shortlog;h=topic/fsprobe

My plan is follow ideas and code from blkid and add a new low-level API for udev. That's all.

 Karel Zak  <[email protected]>

could you also integrate the requirements for fstype klibc, so that we can kill the third place of needed fs recognition patches.

hpa formulated them a while ago, searching,
http://www.zytor.com/pipermail/klibc/2006-February/001345.html

concerning initramfs volume_id added --probe-all, before it used to guess wrongly, so currently we prefer klibc fstype.

Care to explain what ext4 development has to do with the generic fsck program? I don''t see any convincing reason not to move that now.

Kay

Fsck and mount (and especially mount) needs to be linked against the very latest blkid library in order to make sure things work, and when we spread out packages across the various different packages, it means updating things to assure correct functionality is that much harder.

It also means getting the right ext4 detection routines into vol_id, and in the past I've had real problems getting patches that fix real bugs in vol_id past you, and I'm just too busy right now to deal with that right now.

- Ted

Sorry, some major distrost don't use blkid at all, so this does not apply at all to the fact that fsck should not be part of util-linux now. Again provide facts and stop complaining about the useless probe-reorder patches that didn't make it into libvolume_id for good reason.

Kay