New fsck option to ignore device-mapper crypto devices

Hi,

Current practice in defining crypto devices in common distributions has:
1. A definition of the device-mapper name with the corresponding device in /etc/crypttab
2. A definition in /etc/fstab for the mountpoint of the dm device.

Steps involved into setting up the crypto devices are a. fsck local filesystems
b. mount local filesystems
c. device-mapper set up of crypto devices
d. fsck crypto filesystems
e. mount crypto filesystems

Steps a.+b. have to be done before the crypto device setup, because the crypto device could be in a file container on a local filesystem.

Now, the problem appears if /etc/fstab contains a mount point of a crypto device which is supposed to be fsck'd in step d. fsck will fail in step a., since this device does not exist at this point in the boot process (it will be set up in step c.)

In order to address this, I propose a new option for fsck, lets say '-X'. Enabling this will skip a device-mapper device which is currently nonexistent, but is defined in /etc/crypttab.
In this way crypto devices could be skipped without fsck failure when running fsck -A.
Proposed patch to implement this below.

Regards,
Matthias

Index: e2fsprogs-1.40.7/misc/fsck.c
===================================================================
@@ -102,6 +102,7 @@ int noexecute = 0;
 int serialize = 0;
 int skip_root = 0;
 int ignore_mounted = 0;
+int ignore_crypto = 0;
 int notitle = 0;
 int parallel_root = 0;
 int progress = 0;

@@ -937,6 +938,67 @@ static int device_already_active(char *d
 	return 0;
 }
 
+/*
+ * Check if a device exists
+ */
+static int device_exists(const char *device)
+{
+	struct stat st;
+
+	if (stat(device, &st) == -1)
+		return 0;
+
+	if (!S_ISBLK(st.st_mode))
+		return 0;
+
+	return 1;
+}
+
+static void skipline(FILE *file)
+{
+	int c;
+
+	do {
+		c = fgetc(file);
+	} while (c != '\n' && c != EOF);
+}
+
+/*
+ * Check if a device is defined in /etc/crypttab
+ */
+static int device_in_crypttab(const char *device)
+{
+	const char crypttab[]="/etc/crypttab";
+	const char dmpath[]="/dev/mapper/";
+	char name[1024];
+	FILE *file;
+
+	/* check if device is a dm device
+	 * we can skip parsing if not
+	 */
+	if (strncmp(dmpath, device, strlen(dmpath)) != 0)
+		return 0;
+
+	file = fopen(crypttab, "r");
+	if (!file)
+		return 0;
+
+	while (fscanf(file, "%1023s", name) == 1) {
+		if (name[0] == '#') {
+			skipline(file);
+			continue;
+		}
+
+		if (strcmp(device+strlen(dmpath), name) == 0) {
+			fclose(file);
+			return 1;
+		}
+		skipline(file);
+	}
+	fclose(file);
+	return 0;
+}
+
 /* Check all file systems, using the /etc/fstab table. */
 static int check_all(NOARGS)
 {

@@ -1005,6 +1067,17 @@ static int check_all(NOARGS)
 				not_done_yet++;
 				continue;
 			}
+			/* ignore devices which do not exist yet, but are defined
+			 * in /etc/crypttab
+			 */
+			if (ignore_crypto && !device_exists(fs->device) &&
+					device_in_crypttab(fs->device)) {
+				if (verbose)
+					printf("Skipping nonexistent crypto device %s\n",
+							fs->device);
+				fs->flags |= FLAG_DONE;
+				continue;
+			}
 			if (ignore_mounted && is_mounted(fs->device)) {
 				fs->flags |= FLAG_DONE;
 				continue;

@@ -1058,7 +1131,7 @@ static int check_all(NOARGS)
 
 static void usage(NOARGS)
 {
-	fputs(_("Usage: fsck [-AMNPRTV] [ -C [ fd ] ] [-t fstype] [fs-options] [filesys ...]\n"), stderr);
+	fputs(_("Usage: fsck [-AMNPRTVX] [ -C [ fd ] ] [-t fstype] [fs-options] [filesys ...]\n"), stderr);
 	exit(EXIT_USAGE);
 }
 

@@ -1202,6 +1275,9 @@ static void PRS(int argc, char *argv[])
 				fstype = string_copy(tmp);
 				compile_fs_type(fstype, &fs_type_compiled);
 				goto next_arg;
+			case 'X':
+				ignore_crypto = 1;
+				break;
 			case '-':
 				opts_for_fsck++;
 				break;

@@ -1303,6 +1379,17 @@ int main(int argc, char *argv[])
 			if (!fs)
 				continue;
 		}
+		/*
+		 * Ignore devices which do not exist,
+		 * but are defined in /etc/crypttab
+		 */
+		if (ignore_crypto && !device_exists(fs->device) &&
+			   device_in_crypttab(fs->device)) {
+			if (verbose)
+				printf("Skipping nonexistent crypto device %s\n",
+						fs->device);
+			continue;
+		}
 		if (ignore_mounted && is_mounted(fs->device))
 			continue;
 		fsck_device(fs, interactive);

Index: e2fsprogs-1.40.7/misc/fsck.8.in
===================================================================
@@ -263,6 +263,10 @@ Don't show the title on startup.
 Produce verbose output, including all file system-specific commands
 that are executed.
 .TP
+.B \-X
+Skip device-mapper devices which currently do not exist, but are defined in 
+.BR crypttab (5).
+.TP
 .B fs-specific-options
 Options which are not understood by 
 .B fsck 

How is fsck invoked here? Does it use the -A flag?

Should field 8 of /etc/fstab (fs_passno) be zero for these mount points? Is there any reason for it to be anything different?

Alternately, would it make sense to define a special value for this field that tells fsck to silently ignore it if the device does not exist?

Could it be simplified to simply skip non-existent devices? Should it really be crypttab-specific?

David Kleikamp
IBM Linux Technology Center

fsck is invoked with -A option only in step a.
Later on in step d. fsck is just called for the crypto filesystems in question.

Why? zero would mean that they should never get checked.
I think it is reasonable to have the choice to get your crypto filesystems checked. Current practise for SuSE has been to allow only 0, but checked this filesystem anyway, which has lead to complaints. So we want to do this more consistent.

Yes, this would be an alternative. The mount option nofail which has recently been added in util-linux-ng would be suitable for this. However, we prefer a solution to have fsck automatically do the right thing.

Well, I would not disagree.
Currently fsck does not check and calls the default filesystem specific checker fsck.ext2 if a device does not exist, which then fails with exit code 8.
An option to simply skip non-existent devices is Ok for me.
I just haven't had the heart to change this to skip any device, since the problem only relates to the crypto devices and I haven't been sure if the current behaviour is intentional.

Matthias

Zero tells fsck not to check the filesystem during reboot. It's what tells fsck -A which filesystems to check. If we don't expect the filesystem to be check-able during that phase, a non-zero value won't have any real meaning.

David Kleikamp
IBM Linux Technology Center

I'm now beginning to understand why SuSE wanted fsck -M/-m (ignore mounted filesystems). Looks like SuSE has a very strange and non-standard usage scenario with fsck -A which is *not* just at boot-time.

So before we try to figure out whether Yet Another FSCK option makes sense, maybe it would be good to get an explanation exactly *how* SuSE is using fsck -A, besides just at boot-time?

- Ted

I see, but what are we doing with crypto filesystems for which the devices simply do not exist at this phase in the boot process?
How should we specify that we want these filesystems to be checked or not at a later time in the boot process after the crypto devices have been set up?

Matthias

I don't understand the question. If the fs_passno field is zero, fsck isn't even going to try to check the filesystem, so having no device is no problem.

This is why I asked if fsck was being run with the -A flag in step d. If it's not, then I'm not clear on why fs_passno has anything to do with it. Is there some script that looks at this field in /etc/fstab for step d?

If that's the case, then I suggest a more general solution. Either some special value for fs_passno that defers the fsck for a later pass, or a simplified version of your proposed patch without the crypto-specific part.

David Kleikamp
IBM Linux Technology Center

Agreed, in case of zero there is no problem.

Right, fsck -A is *not* called in step d and the fs_passno is in principle unrelated to this.
There is a boot script for setting up, fscking and mounting the crypto devices. As the crypto fileystems are supposed to be defined in /etc/fstab looking at fs_passno by the script to find out if the filesystem should be checked or not seems reasonable to me.

1. Do we agree that fstab fs_passno could be used besides fsck -A by a separate script to decide if a filesystem (that could not be checked in the fsck -A phase) should be checked?

2. If yes, what would be a proper way?

a) special value for fs_passno?
I don't know if this is reasonable, we already have a "nofail" mount option. We can already call fsck with something like (assuming crypto devices being tagged with the nofail option) fsck -A -t noopts=nofail
So giving either a special fs_passno value or an additional option seems to be the same: one has to put an additional value into fstab.

b) a general "ignore if device does not exist" option? Still I am not sure if we really *generally* want to ignore devices which are not available at fsck -A phase. The behaviour of fsck to fail in this case is probably on purpose?

Note, the whole discussion applies to crypto filesystems living in a container file. Encrypted block devices can be set up earlier so they could be checked in the normal fsck -A stage (Suse currently does not do this, but I am planning to change this).
So, since the issue will remain only for file containers, it might be not worth adding a new option for fsck. But it would be nice have the fsck behaviour on non-existent device failure clarified.

Thanks,
Matthias

Well, historically there have been /etc/fstab entries which look like this:

/dev/zipdrive /media/zipdisk ext3 default,noauto 0 0

This is because there might not be a zip disk in the drive at the time of the boot. However, a root user could insert a zip disk, and then run the command "fsck /dev/zipdrive" and expect it to work, and the follow that up with "mount /dev/zipdrive".

So passno == 0 can't mean "don't mount the filesystem, ever". All it means is "don't mount it at boot". So I don't think trying to kludge new fields into fs_passno is the way to go.

------------

So here's another way you could do things that doesn't require any patches to fsck at all. Just add "comment=crypto" into the option field of /etc/fstab, like this.

/dev/mapper/crypto-frobnitz /crypto1 ext3 comment=crypto 1 1

Then you can skip filesystems that have the comment=crypto in the options field via:

fsck -AR -t opts=!comment=crypto

To check all of the filesystems that have the comment=crypto field via:

fsck -AR -t opts=comment=crypto

Why did I use "comment=crypto"? Because if you use "crypto", then mount will try to pass crypto to the kernel, and if the kernel doesn't understand an option, it will reject the mount. Mount doesn't send "comment" mount options to the kernel, so this works.

Of course, you'll need to modify mount to be able to only mount all crypto filesystems, or not to mount any crypto filesystems, so you could change it to understand that "crypto" is not to be passed down to the kernel, at which point you could just have this in /etc/fstab:

/dev/mapper/crypto-frobnitz /crypto1 ext3 crypto 1 1

And then the fsck invocations would become:

fsck -AR -t opts=!crypto

and

fsck -AR -t opts=crypto

Which of course looks much nicer.

I put in the "-t opts=" matching a long time ago (as in 10+ years ago), because someone had wanted to do something strange like this. I had completely forgotten about it until now....

- Ted

Oh by the way, in case you haven't tried this, the best way to test exactly what fsck will do with a set of arguments is by using the -NV options, i.e.:

fsck -ARNV -t opts=!comment=crypto

- Ted

This is just a nitpick that doesn't change your argument at all.

s/mount/check/ in the above paragraph. passno only tells fsck what to do. "noauto" tells mount what to mount at boot time.

David Kleikamp
IBM Linux Technology Center