btrfs-progs: Auto resize fs after device replace
Subsystems (2)
v3 cover btrfs-progs: Auto resize fs after device replace
From: Marcos Paulo de Souza <[email protected]>
Changes from v2:
* Fixed the code format after moving function resize_filesystem in patch 0001 (suggested by David)
* Sorted the resize_filesystem function prototype in patch 0001 (suggested by David)
* Changed the -a into long argument --autoresize in patch 0002 (suggested by David)
* Translate srcdev if the argument is not a devid (suggested by David)
* This also changes the way we use the ioctl, now only passing devid to the kernel, instead of passing the path and letting the kernel to translate
* Add tests to check the autoresize functionality
Changes from v1:
* Reworded the help message and the docs telling the user that the fs will be resized to it's max size (suggested by Qu)
* Added a warning message saying that the resize failed, asking the user to resize manually. (suggested by Qu)
Both changes were done only in patch 0002.
Anand suggested this job to be done in kernel side, atomically, but as I received a good review from Qu I decided to send a v3 of this patchset.
Please review, thanks!
Original cover-letter[1]:
These two patches make possible to resize the fs after a successful replace finishes. The flag -a is responsible for doing it (-r is already use, so -a in this context means "automatically").
The first patch just moves the resize rationale to utils.c and the second patch adds the flag an calls resize if -a is informed replace finishes successfully.
Please review!
Marcos Paulo de Souza (3):
btrfs-progs: Move resize into functionaly into utils.c
btrfs-progs: replace: New argument to resize the fs after replace
btrfs-progs: tests: misc: Add some replace tests
Documentation/btrfs-replace.asciidoc | 5 +- cmds/filesystem.c | 58 +---------- cmds/replace.c | 105 +++++++++++++------- common/utils.c | 60 +++++++++++ common/utils.h | 2 + tests/misc-tests/039-replace-device/test.sh | 56 +++++++++++ 6 files changed, 192 insertions(+), 94 deletions(-) create mode 100755 tests/misc-tests/039-replace-device/test.sh
Signature · 1 line
2.25.1
PATCH v3 1/3 btrfs-progs: Move resize into functionaly into utils.c
From: Marcos Paulo de Souza <[email protected]>
This function will be used in the next patch to auto resize the filesystem when a bigger disk is added.
--- cmds/filesystem.c | 58 ++------------------------------------------- common/utils.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++ common/utils.h | 2 ++ 3 files changed, 64 insertions(+), 56 deletions(-)
cmds/filesystem.c +2 -56
@@ -1075,11 +1075,7 @@ static const char * const cmd_filesystem_resize_usage[] = {static int cmd_filesystem_resize (const struct cmd_struct *cmd,int argc,char **argv) { -struct btrfs_ioctl_vol_args args; -int fd, res, len, e; -char *amount, *path; -DIR *dirstream =NULL ; -struct stat st; +char *amount, *path;clean_args_no_options_relaxed (cmd, argc, argv);
@@ -1089,57 +1085,7 @@ static int cmd_filesystem_resize(const struct cmd_struct *cmd, amount = argv[optind]; path = argv[optind +1 ]; - len =strlen (amount); -if (len ==0 || len >=BTRFS_VOL_NAME_MAX ) { -error ("resize value too long (%s)" , amount); -return 1 ; - } - - res =stat (path, &st); -if (res <0 ) { -error ("resize: cannot stat %s: %m" , path); -return 1 ; - } -if (!S_ISDIR (st.st_mode)) { -error ("resize works on mounted filesystems and accepts only\n" -"directories as argument. Passing file containing a btrfs image\n" -"would resize the underlying filesystem instead of the image.\n" ); -return 1 ; - } - - fd =btrfs_open_dir (path, &dirstream,1 ); -if (fd <0 ) -return 1 ; - -printf ("Resize '%s' of '%s'\n" , path, amount); -memset (&args,0 ,sizeof (args)); -strncpy_null (args.name, amount); - res =ioctl (fd,BTRFS_IOC_RESIZE , &args); - e = errno; -close_file_or_dir (fd, dirstream); -if ( res <0 ){ -switch (e) { -case EFBIG : -error ("unable to resize '%s': no enough free space" , - path); -break ; -default : -error ("unable to resize '%s': %m" , path); -break ; - } -return 1 ; - }else if (res >0 ) { -const char *err_str =btrfs_err_str (res); - -if (err_str) { -error ("resizing of '%s' failed: %s" , path, err_str); - }else { -error ("resizing of '%s' failed: unknown error %d" , - path, res); - } -return 1 ; - } -return 0 ; +return resize_filesystem (amount, path); } static DEFINE_SIMPLE_COMMAND(filesystem_resize, "resize");
common/utils.c +60 -0
66 lines not on this page · raw message
common/utils.h +2 -0
@@ -147,4 +147,6 @@ static inline int btrfs_test_for_mixed_profiles_by_fd(int fd)int btrfs_check_for_mixed_profiles_by_path (const char *path);int btrfs_check_for_mixed_profiles_by_fd (int fd); +int resize_filesystem (const char *amount,const char *path); +#endif
Signature · 1 line
2.25.1
PATCH v3 2/3 btrfs-progs: replace: New argument to resize the fs after replace
In reply to Marcos Paulo de SouzaFrom: Marcos Paulo de Souza <[email protected]>
Add new --autoresize long opt to resize the fs automatically after replace. The code now checks if the srcdev is a path, and translates it to a devid, making it being consumed later on if auto resize is specified.
By using the --autoresize flag on replace makes btrfs issue a resize ioctl after the replace finishes. This argument is a shortcut for
btrfs replace start -f 3 /dev/sdf BTRFS/
btrfs fi resize 3:max BTRFS/
--- Documentation/btrfs-replace.asciidoc | 5 +- cmds/replace.c | 105 +++++++++++++++++---------- 2 files changed, 72 insertions(+), 38 deletions(-)
Documentation/btrfs-replace.asciidoc +4 -1
@@ -18,7 +18,7 @@ SUBCOMMAND *cancel* <mount_point>:: Cancel a running device replace operation. -*start* [-Bfr] <srcdev>|<devid> <targetdev> <path>:: +*start* [options] <srcdev>|<devid> <targetdev> <path>:: Replace device of a btrfs filesystem. + On a live filesystem, duplicate the data to the target device which
@@ -53,6 +53,9 @@ never allowed to be used as the <targetdev>. + -B:::: no background replace. +--autoresize:::: +automatically resizes the filesystem to it's max size if the <targetdev> is +bigger than <srcdev>. *status* [-1] <mount_point>:: Print status and progress information of a running device replace operation.
cmds/replace.c +68 -37
177 lines not on this page · raw message
Signature · 1 line
2.25.1
PATCH v3 3/3 btrfs-progs: tests: misc: Add some replace tests
In reply to Marcos Paulo de SouzaFrom: Marcos Paulo de Souza <[email protected]>
The new test includes a check for the new --autoresize option
--- tests/misc-tests/039-replace-device/test.sh | 56 +++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100755 tests/misc-tests/039-replace-device/test.sh
tests/misc-tests/039-replace-device/test.sh new file +56 -0
@@ -0,0 +1,56 @@ +#!/bin/bash +# +# test the different arguments accepted by "btrfs replace" + +source "$TEST_TOP/common" + +check_prereq mkfs.btrfs +check_prereq btrfs + +setup_root_helper + +setup_loopdevs 2 +prepare_loopdevs +dev1=${loopdevs[1]} +dev2=${loopdevs[2]} + +# resize only works with disk size bigger than the replaced disk +run_check_stdout truncate -s3g `pwd`/img3 +dev3=`run_check_stdout $SUDO_HELPER losetup --find --show $(pwd)/img3` + +test() +{ + local srcdev + local final_size + local resize_arg + srcdev="$1" + final_size="$2" + resize_arg="$3" + args="-B -f" + + if [ -n "$resize_arg" ]; then + args="$args --autoresize" + fi + + run_check $SUDO_HELPER "$TOP/mkfs.btrfs" -f "$dev1" "$dev2" + TEST_DEV="$dev1" + + run_check_mount_test_dev + run_check_stdout $SUDO_HELPER "$TOP/btrfs" replace start $args "$srcdev" "$dev3" "$TEST_MNT" + run_check_stdout $SUDO_HELPER "$TOP/btrfs" filesystem usage "$TEST_MNT" | head -2 | \ + grep -q "$final_size\\.00GiB" + [ $? -eq 1 ] && _fail "Device size don't match. Expected size: $final_size\\.00GiB" + run_check_umount_test_dev +} + +# test replace using devid and path, and also test the final fs size when +# --autoresize is passed, executing the replace + resize in just one command. +test 2 4 +test 2 5 true +test "$dev2" 4 +test "$dev2" 5 true + +run_check $SUDO_HELPER losetup -d "$dev3" +rm `pwd`/img3 + +cleanup_loopdevs
Signature · 1 line
2.25.1
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 70 lines · On Thu, 2020-04-16 at 07:18 +0100, [email protected] wrote:
Quoted 4 lines · Marcos Paulo de Souza
From: Marcos Paulo de Souza <[email protected]>
Changes from v2:
* Fixed the code format after moving functionresize_filesysteminpatch 0001
Quoted 2 lines
(suggested by David)
* Sorted theresize_filesystemfunction prototype in patch 0001(suggested by
Quoted 2 lines
David)
* Changed the -a into long argument --autoresize in patch 0002(suggested by
Quoted 2 lines
David)
* Translate srcdev if the argument is not a devid (suggested byDavid)
Quoted 1 line
* This also changes the way we use the ioctl, now only passing
devid to the
Quoted 1 line
kernel, instead of passing the path and letting the kernel to
translate
Quoted 4 lines
* Add tests to check the autoresize functionality
Changes from v1:
* Reworded the help message and the docs telling the user that thefs will be
Quoted 2 lines
resized to it's max size (suggested by Qu)
* Added a warning message saying that the resize failed, asking theuser to
Quoted 5 lines
resize manually. (suggested by Qu)
Both changes were done only in patch 0002.
Anand suggested this job to be done in kernel side, atomically, but
as I
Quoted 1 line
received a good review from Qu I decided to send a v3 of this
patchset.
Quoted 5 lines
Please review, thanks!
Original cover-letter[1]:
These two patches make possible to resize the fs after a successfulreplace
Quoted 1 line
finishes. The flag -a is responsible for doing it (-r is already
use, so -a in
Quoted 3 lines
this context means "automatically").
The first patch just moves the resize rationale to
utils.cand thesecond patch
Quoted 1 line
adds the flag an calls resize if -a is informed replace finishes
successfully.
Quoted 11 lines
Please review!
Marcos Paulo de Souza (3):
btrfs-progs: Move resize into functionaly intoutils.c
btrfs-progs: replace: New argument to resize the fs after replace
btrfs-progs: tests: misc: Add some replace testsDocumentation/btrfs-replace.asciidoc | 5 +- cmds/filesystem.c | 58 +---------- cmds/replace.c | 105 +++++++++++++-------
Quoted 6 lines
common/utils.c | 60 +++++++++++ common/utils.h | 2 + tests/misc-tests/039-replace-device/test.sh | 56 +++++++++++ 6 files changed, 192 insertions(+), 94 deletions(-) create mode 100755 tests/misc-tests/039-replace-device/test.shHmm,
Quoted 9 lines
Does this take into account if different raid levels are used for data
and meta data. When I last was upgrading a fs with larger drive it was
not just a case of making the over all file size different I had to also
change the ratio of metadata to data. And then I had to balance a whole
lot (which found a bug in memory at the time)
What do you mean about "ratio of metadata to data"? The new disk will receive the same content of the old disk, so, it the metadata wasn't balanced before the replace, it might be a good idea to execute balance after adding a maybe larger disk.
Can you please describe in more details you problem, or the problem you had at the time you did the replace last time? It seems it's not related to the resize itself, so these patches do not change the current behavior.
Quoted 5 lines
How do you deal with these edge cases ? Especially if you replacing with
a smaller disk and then have to shrink the metadata ratio?
Replace does not accept a smaller disk to replace a larger one.
Quoted 3 lines
Just curious.
Me too. These patches were meant to help people to avoid executing two different command for a pretty common action, all details behind replace and resize are the same, it's just a "helper" to make users to use the full capacity of a newly added disk.
Humble ping :)
Quoted 66 lines · Marcos Paulo de Souza
From: Marcos Paulo de Souza <[email protected]>
Changes from v2:
* Fixed the code format after moving functionresize_filesystemin patch 0001
(suggested by David)
* Sorted theresize_filesystemfunction prototype in patch 0001 (suggested by
David)
* Changed the -a into long argument --autoresize in patch 0002 (suggested by
David)
* Translate srcdev if the argument is not a devid (suggested by David)
* This also changes the way we use the ioctl, now only passing devid to the
kernel, instead of passing the path and letting the kernel to
translate
* Add tests to check the autoresize functionalityChanges from v1:
* Reworded the help message and the docs telling the user that the fs will be
resized to it's max size (suggested by Qu)
* Added a warning message saying that the resize failed, asking the user to
resize manually. (suggested by Qu)Both changes were done only in patch 0002.
Anand suggested this job to be done in kernel side, atomically, but as I
received a good review from Qu I decided to send a v3 of this patchset.Please review, thanks!
Original cover-letter[1]:
These two patches make possible to resize the fs after a successful replace
finishes. The flag -a is responsible for doing it (-r is already use, so -a in
this context means "automatically").The first patch just moves the resize rationale to
utils.cand the second patch
adds the flag an calls resize if -a is informed replace finishes successfully.Please review!
Marcos Paulo de Souza (3):
btrfs-progs: Move resize into functionaly intoutils.c
btrfs-progs: replace: New argument to resize the fs after replace
btrfs-progs: tests: misc: Add some replace testsDocumentation/btrfs-replace.asciidoc | 5 +- cmds/filesystem.c | 58 +---------- cmds/replace.c | 105 +++++++++++++----- -- common/utils.c | 60 +++++++++++ common/utils.h | 2 + tests/misc-tests/039-replace-device/test.sh | 56 +++++++++++ 6 files changed, 192 insertions(+), 94 deletions(-) create mode 100755 tests/misc-tests/039-replace-device/test.sh