Skip to contentSkip to conversation

btrfs-progs: tune: add --device and --noscan option

PATCH In review Anand Jain 1 participant 8 messages Jun 13, 2023 – Jun 28, 2023

Subsystems (2)
lore
Share & tools
lore

Anand Jain

author
#0 lore raw

cover btrfs-progs: tune: add --device and --noscan option

Separated out from the preparatory patch set. Depends on the preparatory patch set: ("btrfs-progs: cleanup and preparatory around device scan"). This set (along with its preparatory patch) has passed the btrfs-progs test suite.

By default, btrfstune scans all the block devices in the system.

To scan regular files without mapping them to a loop device, add the --device option.

The option arguments follow the same pattern as in the "mkfs.btrfs -O" option.

To indicate not to scan the system for other devices, add the --noscan option.

For example:

The command below will scan both regular files and the devices provided in the --device option, along with the system block devices.

	btrfstune -m --device /tdev/td1,/tdev/td2 /tdev/td3
  or
	btrfstune -m --device /tdev/td1 --device /tdev/td2 /tdev/td3

In some cases, if you need to avoid the default system scan for the block device, you can use the --noscan option.

	btrfstune -m --noscan --device /tdev/td1,/tdev/td2 /tdev/td3

Anand Jain (5):
btrfs-progs: tune: consolidate return goto free-out
btrfs-progs: tune: introduce --device option
btrfs-progs: docs: update btrfstune --device option
btrfs-progs: tune: introduce --noscan option
btrfs-progs: docs: update btrfstune --noscan option

 Documentation/btrfstune.rst |   7 +++
 tune/main.c                 | 103 ++++++++++++++++++++++++++++++------
 2 files changed, 95 insertions(+), 15 deletions(-)
Signature · 1 line
2.38.1
author
#1 lore raw

PATCH 1/5 btrfs-progs: tune: consolidate return goto free-out

The upcoming "--device" option requires memory to parse devices, which should be freed before returning from the main() function. As a preparation for adding the "--device" option to the "btrfstune" command, provide a consolidated error return exit from the main function with a "goto" labeled "free_out". The label "free_out" may not make sense currently, but it will when the "--device" option is added.

There are several return statements within the main function, and changing all of them in the main "--device" feature patch would deviate from the actual for the feature changes. Hence, it made sense to create a preparatory patch.

The return value for any failure remains the same as in the original code.

---
 tune/main.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)
tune/main.c +17 -10
index 0ca1e01282c9..0c8872dcdee5 100644
@@ -145,7 +145,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	bool to_fst = false;
 	int csum_type = -1;
 	char *new_fsid_str = NULL;
-	int ret;
+	int ret = 1;
 	u64 super_flags = 0;
 	int fd = -1;
 
@@ -233,18 +233,18 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	set_argv0(argv);
 	device = argv[optind];
 	if (check_argc_exact(argc - optind, 1))
-		return 1;
+		goto free_out;
 
 	if (random_fsid && new_fsid_str) {
 		error("random fsid can't be used with specified fsid");
-		return 1;
+		goto free_out;
 	}
 	if (!super_flags && !seeding_flag && !(random_fsid || new_fsid_str) &&
 	    !change_metadata_uuid && csum_type == -1 && !to_bg_tree &&
 	    !to_extent_tree && !to_fst) {
 		error("at least one option should be specified");
 		usage(&tune_cmd, 1);
-		return 1;
+		goto free_out;
 	}
 
 	if (new_fsid_str) {
@@ -253,18 +253,21 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		ret = uuid_parse(new_fsid_str, tmp);
 		if (ret < 0) {
 			error("could not parse UUID: %s", new_fsid_str);
-			return 1;
+			ret = 1;
+			goto free_out;
 		}
 		if (!test_uuid_unique(new_fsid_str)) {
 			error("fsid %s is not unique", new_fsid_str);
-			return 1;
+			ret = 1;
+			goto free_out;
 		}
 	}
 
 	fd = open(device, O_RDWR);
 	if (fd < 0) {
 		error("mount check: cannot open %s: %m", device);
-		return 1;
+		ret = 1;
+		goto free_out;
 	}
 
 	ret = check_mounted_where(fd, device, NULL, 0, NULL,
@@ -273,18 +276,21 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		errno = -ret;
 		error("could not check mount status of %s: %m", device);
 		close(fd);
-		return 1;
+		ret = 1;
+		goto free_out;
 	} else if (ret) {
 		error("%s is mounted", device);
 		close(fd);
-		return 1;
+		ret = 1;
+		goto free_out;
 	}
 
 	root = open_ctree_fd(fd, device, 0, ctree_flags);
 
 	if (!root) {
 		error("open ctree failed");
-		return 1;
+		ret = 1;
+		goto free_out;
 	}
 
  	if (to_bg_tree) {
@@ -431,5 +437,6 @@ out:
 	close_ctree(root);
 	btrfs_close_all_devices();
 
+free_out:
 	return ret;
 }
Signature · 1 line
2.38.1
author
#2 lore raw

PATCH 1/1 btrfs-progs: tune: introduce --device option

In reply to Anand Jain

As of now, btrfstune only accepts one device from the command line and then scans the system to find other parter devices if any.

However, this method mandates always accessing the file raw image as a loop device.

This patch modifies btrfstune to accept other devices or reg-files from the command line using an option --device and scans/registers them.

For example:

	btrfstune -m --device /tdev/td1,/tdev/td2 /tdev/td3
  or
	btrfstune -m --device /tdev/td1 --device /tdev/td2 /tdev/td3
---
 tune/main.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 64 insertions(+), 1 deletion(-)
tune/main.c +64 -1
index 0c8872dcdee5..c9bbada4f7b1 100644
@@ -23,6 +23,7 @@
 #include <getopt.h>
 #include <errno.h>
 #include <stdbool.h>
+#include <ctype.h>
 #include <uuid/uuid.h>
 #include "kernel-shared/ctree.h"
 #include "kernel-shared/disk-io.h"
@@ -63,6 +64,37 @@ static int set_super_incompat_flags(struct btrfs_root *root, u64 flags)
 	return ret;
 }
 
+static bool array_append(char **dest, char *src, int *cnt)
+{
+	char *this_tok = strtok(src, ",");
+	int ret_cnt = *cnt;
+
+	while(this_tok != NULL) {
+		ret_cnt++;
+		dest = realloc(dest, sizeof(char *) * ret_cnt);
+		if (!dest)
+			return false;
+
+		dest[ret_cnt - 1] = strdup(this_tok);
+		*cnt = ret_cnt;
+
+		this_tok = strtok(NULL, ",");
+	}
+
+	return true;
+}
+
+static void free_array(char **prt, int cnt)
+{
+	if (!prt)
+		return;
+
+	for (int i = 0; i < cnt; i++)
+		free(prt[i]);
+
+	free(prt);
+}
+
 static int convert_to_fst(struct btrfs_fs_info *fs_info)
 {
 	int ret;
@@ -117,6 +149,7 @@ static const char * const tune_usage[] = {
 	"",
 	"General:",
 	OPTLINE("-f", "allow dangerous operations, make sure that you are aware of the dangers"),
+	OPTLINE("--device", "devices or regular-files of the filesystem to be scanned"),
 	OPTLINE("--help", "print this help"),
 #if EXPERIMENTAL
 	"",
@@ -144,6 +177,8 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	bool to_bg_tree = false;
 	bool to_fst = false;
 	int csum_type = -1;
+	int argc_devices = 0;
+	char **argv_devices = NULL;
 	char *new_fsid_str = NULL;
 	int ret = 1;
 	u64 super_flags = 0;
@@ -155,7 +190,8 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		enum { GETOPT_VAL_CSUM = GETOPT_VAL_FIRST,
 		       GETOPT_VAL_ENABLE_BLOCK_GROUP_TREE,
 		       GETOPT_VAL_DISABLE_BLOCK_GROUP_TREE,
-		       GETOPT_VAL_ENABLE_FREE_SPACE_TREE };
+		       GETOPT_VAL_ENABLE_FREE_SPACE_TREE,
+		       GETOPT_VAL_DEVICE };
 		static const struct option long_options[] = {
 			{ "help", no_argument, NULL, GETOPT_VAL_HELP},
 			{ "convert-to-block-group-tree", no_argument, NULL,
@@ -167,6 +203,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 #if EXPERIMENTAL
 			{ "csum", required_argument, NULL, GETOPT_VAL_CSUM },
 #endif
+			{ "device", required_argument, NULL, GETOPT_VAL_DEVICE },
 			{ NULL, 0, NULL, 0 }
 		};
 		int c = getopt_long(argc, argv, "S:rxfuU:nmM:", long_options, NULL);
@@ -210,6 +247,21 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		case GETOPT_VAL_ENABLE_BLOCK_GROUP_TREE:
 			to_bg_tree = true;
 			break;
+		case GETOPT_VAL_DEVICE:
+			if (!argv_devices) {
+				argv_devices = malloc(sizeof(char *));
+				if (!argv_devices) {
+					error("memory alloc failed");
+					return 1;
+				}
+			}
+
+			if (!array_append(argv_devices, optarg,
+					  &argc_devices)) {
+				error("memory alloc failed");
+				goto free_out;
+			}
+			break;
 		case GETOPT_VAL_DISABLE_BLOCK_GROUP_TREE:
 			to_extent_tree = true;
 			break;
@@ -285,6 +337,16 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		goto free_out;
 	}
 
+	/*
+	 * check_mounted_where() with noscan == true frees the scanned devices
+	 * scan the command line provided device list now.
+	 */
+	if (argv_devices) {
+		ret = btrfs_scan_argv_devices(0, argc_devices, argv_devices);
+		if (ret)
+			goto free_out;
+	}
+
 	root = open_ctree_fd(fd, device, 0, ctree_flags);
 
 	if (!root) {
@@ -438,5 +500,6 @@ out:
 	btrfs_close_all_devices();
 
 free_out:
+	free_array(argv_devices, argc_devices);
 	return ret;
 }
Signature · 1 line
2.31.1
author
#4 lore raw

PATCH 3/5 btrfs-progs: docs: update btrfstune --device option

In reply to Anand Jain

Update the Documentation/btrsftune.rst to carry the new --device option.

---
 Documentation/btrfstune.rst | 3 +++
 1 file changed, 3 insertions(+)
Documentation/btrfstune.rst +3 -0
index 0510ad1f4c26..89f4494bbaf0 100644
@@ -46,6 +46,9 @@ OPTIONS
         Allow dangerous changes, e.g. clear the seeding flag or change fsid.
         Make sure that you are aware of the dangers.
 
+--device
+        List of block devices or regular files that are part of the filesystem.
+
 -m
         (since kernel: 5.0)
 
Signature · 1 line
2.38.1
author
#6 lore raw

PATCH 5/5 btrfs-progs: docs: update btrfstune --noscan option

In reply to Anand Jain

Update the Documentation/btrsftune.rst to carry the new --noscan option.

---
 Documentation/btrfstune.rst | 4 ++++
 1 file changed, 4 insertions(+)
Documentation/btrfstune.rst +4 -0
index 89f4494bbaf0..46793b3d4a77 100644
@@ -49,6 +49,10 @@ OPTIONS
 --device
         List of block devices or regular files that are part of the filesystem.
 
+--noscan
+        Do not automatically scan the system for other devices from the same
+        filesystem, only use the devices provided as the arguments.
+
 -m
         (since kernel: 5.0)
 
Signature · 1 line
2.38.1
author
#5 lore raw

PATCH 4/5 btrfs-progs: tune: introduce --noscan option

In reply to Anand Jain

The function check_where_mounted() scans the system for other btrfs devices, but in certain cases, we may need a way to instruct btrfstune not to perform the system scan and instead only work on the devices provided through the command line. And so, add an option --noscan.

For example:

  $ mkfs.btrfs -fq -draid0 -mraid0 ./td1 ./td2
  $ btrfstune -m ./td1
	  warning, device 2 is missing
	  ERROR: could not setup extent tree
	  ERROR: open ctree failed
  $ losetup --find --show ./td2
	/dev/loop4
  $ btrfstune -m ./td1

Or just

  $ btrfstune -m --device ./td1 ./td2

The 'noscan' option is optional because there may be scenarios where we have a copy that we don't want to modify the fsid. In the following scenario, we keep 'td2' out of the metadata_uuid changes, even though its loop device is created.

  $ cp td2 td3
  $ btrsftune --noscan --device ./td3 -m ./td1

Thanks

---
 tune/main.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
tune/main.c +9 -3
index f595bfee0e6a..af0e58973355 100644
@@ -150,6 +150,7 @@ static const char * const tune_usage[] = {
 	"General:",
 	OPTLINE("-f", "allow dangerous operations, make sure that you are aware of the dangers"),
 	OPTLINE("--device", "devices or regular-files of the filesystem to be scanned"),
+	OPTLINE("--noscan", "do not scan the devices from the system, use only the listed ones"),
 	OPTLINE("--help", "print this help"),
 #if EXPERIMENTAL
 	"",
@@ -176,6 +177,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	bool to_extent_tree = false;
 	bool to_bg_tree = false;
 	bool to_fst = false;
+	bool noscan = false;
 	int csum_type = -1;
 	int argc_devices = 0;
 	char **argv_devices = NULL;
@@ -191,7 +193,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		       GETOPT_VAL_ENABLE_BLOCK_GROUP_TREE,
 		       GETOPT_VAL_DISABLE_BLOCK_GROUP_TREE,
 		       GETOPT_VAL_ENABLE_FREE_SPACE_TREE,
-		       GETOPT_VAL_DEVICE };
+		       GETOPT_VAL_DEVICE, GETOPT_VAL_NOSCAN };
 		static const struct option long_options[] = {
 			{ "help", no_argument, NULL, GETOPT_VAL_HELP},
 			{ "convert-to-block-group-tree", no_argument, NULL,
@@ -204,6 +206,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 			{ "csum", required_argument, NULL, GETOPT_VAL_CSUM },
 #endif
 			{ "device", required_argument, NULL, GETOPT_VAL_DEVICE },
+			{ "noscan", no_argument, NULL, GETOPT_VAL_NOSCAN },
 			{ NULL, 0, NULL, 0 }
 		};
 		int c = getopt_long(argc, argv, "S:rxfuU:nmM:", long_options, NULL);
@@ -273,6 +276,10 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 			csum_type = parse_csum_type(optarg);
 			break;
 #endif
+		case GETOPT_VAL_NOSCAN:
+			ctree_flags |= OPEN_CTREE_NO_DEVICES;
+			noscan = true;
+			break;
 		case GETOPT_VAL_HELP:
 		default:
 			usage(&tune_cmd, c != GETOPT_VAL_HELP);
@@ -320,7 +327,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	}
 
 	ret = check_mounted_where(fd, device, NULL, 0, NULL,
-				  SBREAD_IGNORE_FSID_MISMATCH, false);
+				  SBREAD_IGNORE_FSID_MISMATCH, noscan);
 	if (ret < 0) {
 		errno = -ret;
 		error("could not check mount status of %s: %m", device);
@@ -345,7 +352,6 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	}
 
 	root = open_ctree_fd(fd, device, 0, ctree_flags);
-
 	if (!root) {
 		error("open ctree failed");
 		ret = 1;
Signature · 1 line
2.38.1
author
#7 lore raw
In reply to Anand Jain

Please ignore this patch bundle and instead use the new patch bundle.

[PATCH 00/10] btrfs-progs: check and tune: add device and noscan
options

Thanks.

Quoted 41 lines · Anand Jain

Separated out from the preparatory patch set. Depends on the preparatory patch set: ("btrfs-progs: cleanup and preparatory around device scan"). This set (along with its preparatory patch) has passed the btrfs-progs test suite.

By default, btrfstune scans all the block devices in the system.

To scan regular files without mapping them to a loop device, add the --device option.

The option arguments follow the same pattern as in the "mkfs.btrfs -O" option.

To indicate not to scan the system for other devices, add the --noscan option.

For example:

The command below will scan both regular files and the devices provided in the --device option, along with the system block devices.

	btrfstune -m --device /tdev/td1,/tdev/td2 /tdev/td3
   or
	btrfstune -m --device /tdev/td1 --device /tdev/td2 /tdev/td3

In some cases, if you need to avoid the default system scan for the block device, you can use the --noscan option.

	btrfstune -m --noscan --device /tdev/td1,/tdev/td2 /tdev/td3

Anand Jain (5):

   btrfs-progs: tune: consolidate return goto free-out
   btrfs-progs: tune: introduce --device option
   btrfs-progs: docs: update btrfstune --device option
   btrfs-progs: tune: introduce --noscan option
   btrfs-progs: docs: update btrfstune --noscan option
  Documentation/btrfstune.rst |   7 +++
  tune/main.c                 | 103 ++++++++++++++++++++++++++++++------
  2 files changed, 95 insertions(+), 15 deletions(-)