Skip to contentSkip to conversation

btrfs: nodatasum drop compress

PATCH In review Satoru Takeuchi 2 participants 8 messages Sep 18, 2014 – Sep 19, 2014 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/5 btrfs: nodatasum drop compress

From: Naohiro Aota <[email protected]>

We can enable both compress option and nodatasum option at the same time with the following command.

===

$ sudo mount -o remount,compress,nodatasum /
$ mount | grep btrfs
/dev/vda2 on / type btrfs (rw,relatime,seclabel,nodatasum,compress=zlib,space_cache)
===

This behavior collides with Documentation/filesystems/btrfs.txt:

===
...
If compression is enabled, nodatacow and nodatasum are disabled.
...
===

This patch drops the compress flags upon nodatasum flag.

---
 fs/btrfs/super.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
fs/btrfs/super.c +9 -2
index c4124de..d1c5b6d 100644
@@ -440,8 +440,15 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
 			 */
 			break;
 		case Opt_nodatasum:
-			btrfs_set_and_info(root, NODATASUM,
-					   "setting nodatasum");
+			if (!btrfs_test_opt(root, NODATASUM)) {
+				if (btrfs_test_opt(root, COMPRESS))
+					btrfs_info(root->fs_info, "setting nodatasum, compression disabled");
+				else
+					btrfs_info(root->fs_info, "setting nodatasum");
+			}
+			btrfs_clear_opt(info->mount_opt, COMPRESS);
+			btrfs_clear_opt(info->mount_opt, FORCE_COMPRESS);
+			btrfs_set_opt(info->mount_opt, NODATASUM);
 			break;
 		case Opt_datasum:
 			if (btrfs_test_opt(root, NODATASUM)) {

-- 1.8.3.1

#1 lore raw

PATCH 2/5 btrfs: correct a message on setting nodatacow

From: Naohiro Aota <[email protected]>

If we set nodatacow mount option after compress-force option, we don't get compression disabling message.

===

$ sudo mount -o remount,compress-force,nodatacow /; dmesg|tail -n 3
[ 3845.719047] BTRFS info (device vda2): force zlib compression
[ 3845.719052] BTRFS info (device vda2): setting nodatacow
[ 3845.719055] BTRFS info (device vda2): disk space caching is enabled
===
---
 fs/btrfs/super.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/super.c +1 -2
index d1c5b6d..d131098 100644
@@ -462,8 +462,7 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
 			break;
 		case Opt_nodatacow:
 			if (!btrfs_test_opt(root, NODATACOW)) {
-				if (!btrfs_test_opt(root, COMPRESS) ||
-				    !btrfs_test_opt(root, FORCE_COMPRESS)) {
+				if (btrfs_test_opt(root, COMPRESS)) {
 					btrfs_info(root->fs_info,
 						   "setting nodatacow, compression disabled");
 				} else {

-- 1.8.3.1

#2 lore raw

Re: [PATCH 2/5] btrfs: correct a message on setting nodatacow

-------- Original Message --------
Subject: [PATCH 2/5] btrfs: correct a message on setting nodatacow
From: Satoru Takeuchi <[email protected]>
To: [email protected] <[email protected]>
Date: 2014年09月18日 16:28

Quoted 35 lines

From: Naohiro Aota <[email protected]>

If we set nodatacow mount option after compress-force option, we don't get compression disabling message.

===

$ sudo mount -o remount,compress-force,nodatacow /; dmesg|tail -n 3
[ 3845.719047] BTRFS info (device vda2): force zlib compression
[ 3845.719052] BTRFS info (device vda2): setting nodatacow
[ 3845.719055] BTRFS info (device vda2): disk space caching is enabled
===
---
  fs/btrfs/super.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/super.c +1 -4
index d1c5b6d..d131098 100644
@@ -462,8 +462,7 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
  			break;
  		case Opt_nodatacow:
  			if (!btrfs_test_opt(root, NODATACOW)) {
-				if (!btrfs_test_opt(root, COMPRESS) ||
-				    !btrfs_test_opt(root, FORCE_COMPRESS)) {
+				if (btrfs_test_opt(root, COMPRESS)) {
  					btrfs_info(root->fs_info,
  						   "setting nodatacow, compression disabled");
  				} else {
-- 1.8.3.1
 
--
Quoted 3 lines

To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Although the patch makes the output ok, the core problem is missing conflict options check.

compress-force mount options implies datacow and datasum, but following nodatasum will disable datasum and compress, in fact they are conflicting mount option...

Even the current behavior(later mount option will override previous ones) provides great tolerance, IMO there should better be some conflicting check for mount options.

For example, we first save all the mount options passed in into a temporary bitmaps to finds out the conflicting and only when they contains no conflicts, set the mount options to fs_info.
(Maybe bitmap is not enough for this case, since we can't distinguish default value and value to be set?)

What do you think about this idea ?

Thanks,
Qu

#3 lore raw

Re: [PATCH 2/5] btrfs: correct a message on setting nodatacow

Hi Qu,

Thank you for your comment.

Quoted 48 lines · Qu Wenruo

-------- Original Message --------
Subject: [PATCH 2/5] btrfs: correct a message on setting nodatacow
From: Satoru Takeuchi <[email protected]>
To: [email protected] <[email protected]>
Date: 2014年09月18日 16:28

Quoted 35 lines

From: Naohiro Aota <[email protected]>

If we set nodatacow mount option after compress-force option, we don't get compression disabling message.

===

$ sudo mount -o remount,compress-force,nodatacow /; dmesg|tail -n 3
[ 3845.719047] BTRFS info (device vda2): force zlib compression
[ 3845.719052] BTRFS info (device vda2): setting nodatacow
[ 3845.719055] BTRFS info (device vda2): disk space caching is enabled
===
---
  fs/btrfs/super.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/super.c +1 -4
index d1c5b6d..d131098 100644
@@ -462,8 +462,7 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
              break;
          case Opt_nodatacow:
              if (!btrfs_test_opt(root, NODATACOW)) {
-                if (!btrfs_test_opt(root, COMPRESS) ||
-                    !btrfs_test_opt(root, FORCE_COMPRESS)) {
+                if (btrfs_test_opt(root, COMPRESS)) {
                      btrfs_info(root->fs_info,
                             "setting nodatacow, compression disabled");
                  } else {
-- 1.8.3.1
 
--
Quoted 3 lines

To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Although the patch makes the output ok, the core problem is missing conflict options check.

compress-force mount options implies datacow and datasum, but following nodatasum will disable datasum and compress, in fact they are conflicting mount option...

Quoted 8 lines

Even the current behavior(later mount option will override previous ones) provides great tolerance, IMO there should better be some conflicting check for mount options.

For example, we first save all the mount options passed in into a temporary bitmaps to finds out the conflicting and only when they contains no conflicts, set the mount options to fs_info.
(Maybe bitmap is not enough for this case, since we can't distinguish default value and value to be set?)

What do you think about this idea ?

I'm against your idea for two reasons and it's better to stay in current behavior though it's a bit complex.

First, the rule "last one wins" is not only a conventional rule, but also is what mount(8) says.

https://git.kernel.org/cgit/utils/util-linux/util-linux.git/tree/sys-utils/mount.8#n253

======
The usual behavior is that the last option wins if there are conflicting ones.
======

Second, if we change the behavior, we would break existing systems. At worst case, users would fail to boot their system after updating kernel, because of the failure of mounting Btrfs at the init process.

Thanks,
Satoru

Quoted 7 lines

Thanks,
Qu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw

Re: [PATCH 2/5] btrfs: correct a message on setting nodatacow

-------- Original Message --------
Subject: Re: [PATCH 2/5] btrfs: correct a message on setting nodatacow
From: Satoru Takeuchi <[email protected]>
To: Qu Wenruo <[email protected]>, [email protected] <[email protected]>
Date: 2014年09月19日 14:45

Quoted 78 lines

Hi Qu,

Thank you for your comment.

Quoted 50 lines · Qu Wenruo

-------- Original Message --------
Subject: [PATCH 2/5] btrfs: correct a message on setting nodatacow
From: Satoru Takeuchi <[email protected]>
To: [email protected] <[email protected]>
Date: 2014年09月18日 16:28

Quoted 34 lines

From: Naohiro Aota <[email protected]>

If we set nodatacow mount option after compress-force option, we don't get compression disabling message.

===

$ sudo mount -o remount,compress-force,nodatacow /; dmesg|tail -n 3
[ 3845.719047] BTRFS info (device vda2): force zlib compression
[ 3845.719052] BTRFS info (device vda2): setting nodatacow
[ 3845.719055] BTRFS info (device vda2): disk space caching is enabled
===
---
  fs/btrfs/super.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/super.c +1 -2
index d1c5b6d..d131098 100644
@@ -462,8 +462,7 @@ int btrfs_parse_options(struct btrfs_root *root, 
 char *options)
              break;
          case Opt_nodatacow:
              if (!btrfs_test_opt(root, NODATACOW)) {
-                if (!btrfs_test_opt(root, COMPRESS) ||
-                    !btrfs_test_opt(root, FORCE_COMPRESS)) {
+                if (btrfs_test_opt(root, COMPRESS)) {
                      btrfs_info(root->fs_info,
                             "setting nodatacow, compression 
disabled");
                  } else {
Quoted 6 lines

-- 1.8.3.1

Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Although the patch makes the output ok, the core problem is missing conflict options check.

compress-force mount options implies datacow and datasum, but following nodatasum will disable datasum and compress, in fact they are conflicting mount option...

Quoted 8 lines

Even the current behavior(later mount option will override previous ones) provides great tolerance, IMO there should better be some conflicting check for mount options.

For example, we first save all the mount options passed in into a temporary bitmaps to finds out the conflicting and only when they contains no conflicts, set the mount options to fs_info.
(Maybe bitmap is not enough for this case, since we can't distinguish default value and value to be set?)

What do you think about this idea ?

I'm against your idea for two reasons and it's better to stay in current behavior though it's a bit complex.

First, the rule "last one wins" is not only a conventional rule, but also is what mount(8) says.

https://git.kernel.org/cgit/utils/util-linux/util-linux.git/tree/sys-utils/mount.8#n253

======
The usual behavior is that the last option wins if there are conflicting ones.
======

Quoted 8 lines

Second, if we change the behavior, we would break existing systems. At worst case, users would fail to boot their system after updating kernel, because of the failure of mounting Btrfs at the init process.

Thanks,
Satoru

It really makes sense.

So I'm OK to keep things and it's true that the conflict check is somewhat overkilled.

Thanks,
Qu

Quoted 8 lines
Quoted 7 lines

Thanks,
Qu

Signature · 3 lines
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
#5 lore raw

PATCH 3/5 btrfs: notice nodatasum is also enabled with nodatacow

In reply to Satoru Takeuchi

From: Naohiro Aota <[email protected]>

Documentation/filesystems/btrfs.txt states the following:

===
...
Nodatacow implies nodatasum, and disables all compression.
...
===

However the current message only mentions nodatacow. It's better to also mention nodatasum.

---
 fs/btrfs/super.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
fs/btrfs/super.c +2 -2
index d131098..7ad4293 100644
@@ -464,9 +464,9 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
 			if (!btrfs_test_opt(root, NODATACOW)) {
 				if (btrfs_test_opt(root, COMPRESS)) {
 					btrfs_info(root->fs_info,
-						   "setting nodatacow, compression disabled");
+						   "setting nodatacow and nodatasum, compression disabled");
 				} else {
-					btrfs_info(root->fs_info, "setting nodatacow");
+					btrfs_info(root->fs_info, "setting nodatacow and nodatasum");
 				}
 			}
 			btrfs_clear_opt(info->mount_opt, COMPRESS);

-- 1.8.3.1

#6 lore raw

PATCH 4/5 btrfs: suppress a verbose message about enabling space cache on remounting

In reply to Satoru Takeuchi

From: Naohiro Aota <[email protected]>

Remount operation always tells you it enabled space cache. I consider it's a bit verbose and is better not to show this message on remounting.

---
 fs/btrfs/super.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
fs/btrfs/super.c +2 -3
index 7ad4293..03131c5 100644
@@ -404,7 +404,8 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
 
 	cache_gen = btrfs_super_cache_generation(root->fs_info->super_copy);
 	if (cache_gen)
-		btrfs_set_opt(info->mount_opt, SPACE_CACHE);
+		btrfs_set_and_info(root, SPACE_CACHE,
+				   "disk space caching is enabled");
 
 	if (!options)
 		goto out;
@@ -758,8 +759,6 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
 		}
 	}
 out:
-	if (!ret && btrfs_test_opt(root, SPACE_CACHE))
-		btrfs_info(root->fs_info, "disk space caching is enabled");
 	kfree(orig);
 	return ret;
 }

-- 1.8.3.1

#7 lore raw

PATCH 5/5 btrfs: rework compression related options processing

In reply to Satoru Takeuchi

From: Naohiro Aota <[email protected]>

There are mutual exclusive mount options corresponding to compression options. If we set these options at once, we get improper messages and states, and a verbose message.

* Improper states
- We can't disable "compress forcing" with compress={zlib,lzo} mount option.

====
# mount -o remount,compress-force=zlib,compress=lzo
[58852.210209] BTRFS info (device vda2): force zlib compression (We should see "use lzo compression" here)
# mount | grep btrfs
/dev/vda2 on / type btrfs (rw,relatime,seclabel,compress-force=lzo,space_cache) (We should also change "compress-force=lzo" to "compress=lzo")
====

* Improper messages
- We don't see compression enabled message when we enables it after disabling it.

====
# mount -o remount,compress=no,compress=zlib; dmesg -c [ 4077.130625] BTRFS info (device vda2): btrfs: use no compression (We should see "use zlib compression" here)
====

- We don't see any message when we change compression type.

====
# mount -o remount,compress=zlib,compress=lzo
(Since "BTRFS info" is at the beginning of this message, "btrfs: " is verbose
and we don't need it.)
====

- We don't see datacow and/or datasum enabling message when we

    enable compression.

    ====
    # mount -o remount,nodatacow,nodatasum,compress
    [58913.704536] BTRFS info (device vda2): setting nodatacow
    (We should see "use zlib compression, setting datacow and datasum" here)
    ====

    ====
    # mount -o remount,nodatasum,compress
    [58950.529392] BTRFS info (device vda2): setting nodatasum
    (We should see "use zlib compression, setting datasum" here)
    ====

* A verbose message
- We see verbose "btrfs: " prefix on disabling compression.

====
# mount -o remount,compress=zlib,compress=no [58790.727788] BTRFS info (device vda2): btrfs: use no compression ("btrfs: " is verbose here. We don't need it, since we already state this is btrfs with "BTRFS info")
====

This commit solves all these problems.

I separated the code in the following two parts:

a) Parse the options.
b) Set and clear flags of mount_opt and show proper messages.

I've confirmed that this patch doesn't cause any regression by running the following script, which tests all combinations of corresponding mount options (compress, compress-force, datacow, and datasum).

=====================
#!/bin/bash

BTRFS_ROOT=/

states=(
compress=zlib,datacow,datasum
compress=lzo,datacow,datasum
compress-force=zlib,datacow,datasum
compress-force=lzo,datacow,datasum
compress=no,nodatacow,nodatasum
compress=no,datacow,nodatasum
compress=no,datacow,datasum
)

options=(
compress
compress=zlib
compress=lzo
compress=no
compress=badopt
compress-force
compress-force=zlib
compress-force=lzo
compress-force=no
compress-force=badopt
nodatacow
datacow
nodatasum
datasum
)

for s in ${states[@]}; do
for o in ${options[@]}; do

    # set initial state
    echo initial: mount -o remount,$s
    mount -o remount,$s ${BTRFS_ROOT}
    # mount with each option
    echo mount -o remount,$o
    dmesg -C
    mount -o remount,$o ${BTRFS_ROOT}
    echo dmesg:
    dmesg -t | perl -ne 'if(/^BTRFS info \(device .+?\): (.+)$/){print "\t$1\n"}'
    echo -n 'mount options: '
    grep -v rootfs /proc/mounts | \
      awk "{if(\$2==\"${BTRFS_ROOT}\"){print \$4}}" | \
      perl -ne '@a=split(/,/); for(@a){print "$_ " if(/^compress(-force)?=|nodata(cow|sum)$/)}'
    echo
    echo
  done
done
=====================
---
 fs/btrfs/super.c | 57 +++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 40 insertions(+), 17 deletions(-)
fs/btrfs/super.c +40 -17
index 03131c5..83433e15 100644
@@ -401,6 +401,7 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
 	char *compress_type;
 	bool compress_force = false;
 	bool compress = false;
+	unsigned long old_compress_type;
 
 	cache_gen = btrfs_super_cache_generation(root->fs_info->super_copy);
 	if (cache_gen)
@@ -486,40 +487,62 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
 		case Opt_compress:
 		case Opt_compress_type:
 			compress = true;
+			old_compress_type = info->compress_type;
 			if (token == Opt_compress ||
 			    token == Opt_compress_force ||
 			    strcmp(args[0].from, "zlib") == 0) {
 				compress_type = "zlib";
 				info->compress_type = BTRFS_COMPRESS_ZLIB;
-				btrfs_set_opt(info->mount_opt, COMPRESS);
-				btrfs_clear_opt(info->mount_opt, NODATACOW);
-				btrfs_clear_opt(info->mount_opt, NODATASUM);
 			} else if (strcmp(args[0].from, "lzo") == 0) {
 				compress_type = "lzo";
 				info->compress_type = BTRFS_COMPRESS_LZO;
-				btrfs_set_opt(info->mount_opt, COMPRESS);
-				btrfs_clear_opt(info->mount_opt, NODATACOW);
-				btrfs_clear_opt(info->mount_opt, NODATASUM);
 				btrfs_set_fs_incompat(info, COMPRESS_LZO);
 			} else if (strncmp(args[0].from, "no", 2) == 0) {
 				compress_type = "no";
-				btrfs_clear_opt(info->mount_opt, COMPRESS);
-				btrfs_clear_opt(info->mount_opt, FORCE_COMPRESS);
-				compress_force = false;
+				compress = compress_force = false;
 			} else {
 				ret = -EINVAL;
 				goto out;
 			}
 
-			if (compress_force) {
-				btrfs_set_and_info(root, FORCE_COMPRESS,
-						   "force %s compression",
-						   compress_type);
-			} else if (compress) {
-				if (!btrfs_test_opt(root, COMPRESS))
+			if (compress_force || compress) {
+				const char *method = "", *setting = "";
+
+				if ((info->compress_type != old_compress_type) ||
+				    (compress_force &&
+				     !btrfs_test_opt(root, FORCE_COMPRESS)) ||
+				    (!compress_force &&
+				     btrfs_test_opt(root, FORCE_COMPRESS)) ||
+				    (compress && !btrfs_test_opt(root, COMPRESS))) {
+					method = compress_force?"force":"use";
+				}
+
+				if (btrfs_test_opt(root, NODATACOW))
+					setting = ", setting datacow and datasum";
+				else if (btrfs_test_opt(root, NODATASUM))
+					setting = ", setting datasum";
+
+				if (strcmp(method, "") != 0) {
 					btrfs_info(root->fs_info,
-						   "btrfs: use %s compression",
-						   compress_type);
+						   "%s %s compression%s",
+						   method, compress_type,
+						   setting);
+				}
+				if (compress_force) {
+					btrfs_set_opt(info->mount_opt,
+						      FORCE_COMPRESS);
+				} else {
+					btrfs_clear_opt(info->mount_opt,
+							FORCE_COMPRESS);
+				}
+				btrfs_set_opt(info->mount_opt, COMPRESS);
+				btrfs_clear_opt(info->mount_opt, NODATACOW);
+				btrfs_clear_opt(info->mount_opt, NODATASUM);
+			} else {
+				btrfs_clear_and_info(root, COMPRESS,
+						     "use no compression");
+				btrfs_clear_opt(info->mount_opt,
+						FORCE_COMPRESS);
 			}
 			break;
 		case Opt_ssd:

-- 1.8.3.1