Skip to contentSkip to conversation

btrfs-progs: tune: fix the leftover csum change item and add a test case for it

PATCH In review Qu Wenruo 3 participants 5 messages May 23, 2023 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover btrfs-progs: tune: fix the leftover csum change item and add a test case for it

David's cyclic csum change tests exposed a bug even with the latest csum change code, that I'm a total idiot and forgot to delete the csum change item.

This results a bug that if we have run multiple csum changes to the same target csum type (e.g. CRC32C->SHA256->CRC32C->SHA256), the second conversion to the same type would fail due to the left over csum change items.

This series would do the fix and add a test case to cover this bug.

Qu Wenruo (2):
btrfs-progs: tune: delete the csum change item after converting the fs
btrfs-progs: tests/misc: add a test case for csum change

 .../058-btrfstune-csum-change/test.sh         | 26 ++++++++++++
 tune/change-csum.c                            | 42 +++++++++++++++++--
 2 files changed, 64 insertions(+), 4 deletions(-)
 create mode 100755 tests/misc-tests/058-btrfstune-csum-change/test.sh
Signature · 1 line
2.40.1
author
#1 lore raw

PATCH 1/2 btrfs-progs: tune: delete the csum change item after converting the fs

[BUG]
Doing the following csum change in a row, it would fail:

 # mkfs.btrfs -f --csum crc32c $dev
 # btrfstune --csum sha256 $dev
 # btrfstune --csum crc32c $dev
 # btrfstune --csum sha256 $dev
 WARNING: Experimental build with unstable or unfinished features
 WARNING: Switching checksums is experimental, do not use for valuable data!

Proceed to switch checksums
ERROR: failed to insert csum change item: File exists
ERROR: failed to generate new data csums: File exists
WARNING: reserved space leaked, flag=0x4 bytes_reserved=16384 extent buffer leak: start 30572544 len 16384
extent buffer leak: start 30441472 len 16384
WARNING: dirty eb leak (aborted trans): start 30441472 len 16384

[CAUSE]
During every csum change operation, btrfstune would insert an temporaray csum change item into root tree.

But unfortunately after the conversion btrfstune doesn't properly delete the csum change item, result the following items in the root tree:

item 10 key (CSUM_CHANGE TEMPORARY_ITEM 0) itemoff 13423 itemsize 0
temporary item objectid CSUM_CHANGE offset 0 target csum type crc32c (0)
item 11 key (CSUM_CHANGE TEMPORARY_ITEM 2) itemoff 13423 itemsize 0
temporary item objectid CSUM_CHANGE offset 2 target csum type sha256 (2)

Thus at the last conversion try to go back to SHA256, we failed to insert the same item, and caused the above error.

[FIX]
After finishing the metadata csum conversion, do a proper removal of the csum item.

---
 tune/change-csum.c | 42 ++++++++++++++++++++++++++++++++++++++----
 1 file changed, 38 insertions(+), 4 deletions(-)
tune/change-csum.c +38 -4
index c8809300a143..b5efc3a8807f 100644
@@ -583,10 +583,13 @@ out:
 	btrfs_release_path(&path);
 
 	/*
-	 * Finish the change by clearing the csum change flag and update the superblock
-	 * csum type.
+	 * Finish the change by clearing the csum change flag, update the superblock
+	 * csum type, and delete the csum change item in the fs with new csum type.
 	 */
 	if (ret == 0) {
+		struct btrfs_root *tree_root = fs_info->tree_root;
+		struct btrfs_trans_handle *trans;
+
 		u64 super_flags = btrfs_super_flags(fs_info->super_copy);
 
 		btrfs_set_super_csum_type(fs_info->super_copy, new_csum_type);
@@ -596,11 +599,42 @@ out:
 
 		fs_info->csum_type = new_csum_type;
 		fs_info->csum_size = btrfs_csum_type_size(new_csum_type);
+		fs_info->skip_csum_check = 0;
 
-		ret = write_all_supers(fs_info);
+		trans = btrfs_start_transaction(tree_root, 1);
+		if (IS_ERR(trans)) {
+			ret = PTR_ERR(trans);
+			errno = -ret;
+			error("failed to start new transaction with new csum type: %m");
+			return ret;
+		}
+		key.objectid = BTRFS_CSUM_CHANGE_OBJECTID;
+		key.type = BTRFS_TEMPORARY_ITEM_KEY;
+		key.offset = new_csum_type;
+
+		ret = btrfs_search_slot(trans, tree_root, &key, &path, -1, 1);
+		if (ret > 0)
+			ret = -ENOENT;
 		if (ret < 0) {
 			errno = -ret;
-			error("failed to write super blocks: %m");
+			error("failed to locate the csum change item: %m");
+			btrfs_release_path(&path);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		ret = btrfs_del_item(trans, tree_root, &path);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to delete the csum change item: %m");
+			btrfs_release_path(&path);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		btrfs_release_path(&path);
+		ret = btrfs_commit_transaction(trans, tree_root);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to finalize the csum change: %m");
 		}
 	}
 	return ret;
Signature · 1 line
2.40.1
author
#2 lore raw

PATCH 2/2 btrfs-progs: tests/misc: add a test case for csum change

In reply to Qu Wenruo

The test case would:
- Populate the fs
- Run csum change for all the target csum type
- Run above csum change again

This should more or less cover the common paths, and detect any conflicting csum change item.

---
 .../058-btrfstune-csum-change/test.sh         | 26 +++++++++++++++++++
 1 file changed, 26 insertions(+)
 create mode 100755 tests/misc-tests/058-btrfstune-csum-change/test.sh
tests/misc-tests/058-btrfstune-csum-change/test.sh new file +26 -0
new file mode 100755
index 000000000000..925e71b86285
@@ -0,0 +1,26 @@
+#!/bin/bash
+# Test btrfstune --csum option
+
+source "$TEST_TOP/common" || exit
+source "$TEST_TOP/common.convert" || exit
+
+check_prereq mkfs.btrfs
+check_prereq btrfstune
+check_prereq btrfs
+
+setup_root_helper
+prepare_test_dev
+
+run_check_mkfs_test_dev --csum crc32c
+run_check_mount_test_dev
+populate_fs
+run_check_umount_test_dev
+
+for hash in "blake2" "crc32c" "xxhash" "sha256"; do
+	run_check $SUDO_HELPER "$TOP/btrfstune" --csum $hash "$TEST_DEV"
+	run_check $SUDO_HELPER "$TOP/btrfs" check --check-data-csum "$TEST_DEV"
+done
+for hash in "blake2" "crc32c" "xxhash" "sha256"; do
+	run_check $SUDO_HELPER "$TOP/btrfstune" --csum $hash "$TEST_DEV"
+	run_check $SUDO_HELPER "$TOP/btrfs" check --check-data-csum "$TEST_DEV"
+done
Signature · 1 line
2.40.1
#3 lore raw
In reply to Qu Wenruo
Quoted 8 lines · Qu Wenruo

David's cyclic csum change tests exposed a bug even with the latest csum change code, that I'm a total idiot and forgot to delete the csum change item.

This results a bug that if we have run multiple csum changes to the same target csum type (e.g. CRC32C->SHA256->CRC32C->SHA256), the second conversion to the same type would fail due to the left over csum change items.

Quoted 5 lines

This series would do the fix and add a test case to cover this bug.

Qu Wenruo (2):
btrfs-progs: tune: delete the csum change item after converting the fs
btrfs-progs: tests/misc: add a test case for csum change

Added to devel thanks, no more errors reported.

#4 lore raw
Quoted 9 lines · David Sterba
Quoted 8 lines · Qu Wenruo

David's cyclic csum change tests exposed a bug even with the latest csum change code, that I'm a total idiot and forgot to delete the csum change item.

This results a bug that if we have run multiple csum changes to the same target csum type (e.g. CRC32C->SHA256->CRC32C->SHA256), the second conversion to the same type would fail due to the left over csum change items.

Quoted 7 lines
Quoted 5 lines

This series would do the fix and add a test case to cover this bug.

Qu Wenruo (2):
btrfs-progs: tune: delete the csum change item after converting the fs
btrfs-progs: tests/misc: add a test case for csum change

Added to devel thanks, no more errors reported.

Finally I can work on the resume part (which is not much compared to the main workflow).

Thanks,
Qu