Skip to contentSkip to conversation

btrfs-progs: a more generic uuid tree rebuild ability

PATCH In review Qu Wenruo 4 participants 9 messages Jul 26, 2024 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover btrfs-progs: a more generic uuid tree rebuild ability

Inspired by Mark's previous attempt to add multi-subvolume mkfs support, one of the problem he hits is the uuid tree generation.

Currently the uuid tree is only generated for mkfs, but that's hard-coded for FS_TREE. Furthermore btrfs-convert doesn't really generate a UUID tree at all.

Thus this patchset introduces a more generic uuid tree rebuild, without the need to touch the existing subvolume creation code.

This is done by:

- Create a new uuid tree if there is not one

- Empty the existing uuid tree

- Iterate through all subvolumes
* If the subvolume has no valid UUID, regenerate one
* Add the uuid entry for the subvolume UUID
* If the subvolume has received UUID, also add it to UUID tree

This can handle all the situations I can think of, include the future multi-subvolume mkfs support.

The first two patches are just cleanup and preparation, the main dish is the last patch.

Qu Wenruo (3):
btrfs-progs: move uuid-tree definitions to kernel-shared/uuid-tree.h
btrfs-progs: cross-port btrfs_uuid_tree_add() from kernel
btrfs-progs: introduce btrfs_rebuild_uuid_tree() for mkfs and
btrfs-convert

 common/root-tree-utils.c  | 265 ++++++++++++++++++++++++++++++++++++++
 common/root-tree-utils.h  |   1 +
 common/send-utils.c       |   1 +
 convert/main.c            |   5 +
 kernel-shared/ctree.h     |  11 --
 kernel-shared/uuid-tree.c | 120 +++++++++++++++++
 kernel-shared/uuid-tree.h |  35 +++++
 mkfs/main.c               |  94 +-------------
 8 files changed, 431 insertions(+), 101 deletions(-)
 create mode 100644 kernel-shared/uuid-tree.h

--
2.45.2

author
#1 lore raw

PATCH 1/3 btrfs-progs: move uuid-tree definitions to kernel-shared/uuid-tree.h

Currently we already have a kernel-shared/uuid-tree.c, which is mostly shared with kernel already.

Meanwhile kernel also has a uuid-tree.h, but we are still using ctree.h for the header.

Just move all the uuid-tree related definitions to kernel-shared/uuid-tree.h, making future code sync easier.

---
 common/send-utils.c       |  1 +
 kernel-shared/ctree.h     | 11 -----------
 kernel-shared/uuid-tree.c |  1 +
 kernel-shared/uuid-tree.h | 33 +++++++++++++++++++++++++++++++++
 mkfs/main.c               |  1 +
 5 files changed, 36 insertions(+), 11 deletions(-)
 create mode 100644 kernel-shared/uuid-tree.h
common/send-utils.c +1 -0
index 173333e30a38..8c13ffa1e106 100644
@@ -29,6 +29,7 @@
 #include "kernel-shared/uapi/btrfs_tree.h"
 #include "kernel-shared/uapi/btrfs.h"
 #include "kernel-shared/ctree.h"
+#include "kernel-shared/uuid-tree.h"
 #include "common/send-utils.h"
 #include "common/messages.h"
 #include "common/utils.h"
kernel-shared/ctree.h +0 -11
index 2388879d1db3..7761b3f6fb1b 100644
@@ -1192,15 +1192,6 @@ int btrfs_del_inode_ref(struct btrfs_trans_handle *trans,
 			struct btrfs_root *root, const char *name, int name_len,
 			u64 ino, u64 parent_ino, u64 *index);
 
-/* uuid-tree.c, interface for mounted mounted filesystem */
-int btrfs_lookup_uuid_subvol_item(int fd, const u8 *uuid, u64 *subvol_id);
-int btrfs_lookup_uuid_received_subvol_item(int fd, const u8 *uuid,
-					   u64 *subvol_id);
-
-/* uuid-tree.c, interface for unmounte filesystem */
-int btrfs_uuid_tree_remove(struct btrfs_trans_handle *trans, u8 *uuid, u8 type,
-			   u64 subid);
-
 static inline int is_fstree(u64 rootid)
 {
 	if (rootid == BTRFS_FS_TREE_OBJECTID ||
@@ -1209,8 +1200,6 @@ static inline int is_fstree(u64 rootid)
 	return 0;
 }
 
-void btrfs_uuid_to_key(const u8 *uuid, u8 type, struct btrfs_key *key);
-
 /* inode.c */
 int btrfs_find_free_dir_index(struct btrfs_root *root, u64 dir_ino,
 			      u64 *ret_ino);
kernel-shared/uuid-tree.c +1 -0
index 766cd31e4234..26359520d57c 100644
@@ -29,6 +29,7 @@
 #include "kernel-shared/ctree.h"
 #include "kernel-shared/messages.h"
 #include "kernel-shared/transaction.h"
+#include "kernel-shared/uuid-tree.h"
 #include "common/messages.h"
 #include "common/utils.h"
 
kernel-shared/uuid-tree.h new file +33 -0
new file mode 100644
index 000000000000..0cdc2228c44f
@@ -0,0 +1,33 @@
+/*
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __BTRFS_UUID_TREE_H__
+#define __BTRFS_UUID_TREE_H__
+
+#include "kerncompat.h"
+#include "kernel-shared/uapi/btrfs_tree.h"
+
+/* uuid-tree.c, interface for mounted mounted filesystem */
+int btrfs_lookup_uuid_subvol_item(int fd, const u8 *uuid, u64 *subvol_id);
+int btrfs_lookup_uuid_received_subvol_item(int fd, const u8 *uuid,
+					   u64 *subvol_id);
+
+/* uuid-tree.c, interface for unmounte filesystem */
+int btrfs_uuid_tree_remove(struct btrfs_trans_handle *trans, u8 *uuid, u8 type,
+			   u64 subid);
+void btrfs_uuid_to_key(const u8 *uuid, u8 type, struct btrfs_key *key);
+
+#endif
mkfs/main.c +1 -0
index cf5cae45d02a..9a6047f7296b 100644
@@ -41,6 +41,7 @@
 #include "kernel-shared/volumes.h"
 #include "kernel-shared/transaction.h"
 #include "kernel-shared/zoned.h"
+#include "kernel-shared/uuid-tree.h"
 #include "crypto/hash.h"
 #include "common/defs.h"
 #include "common/internal.h"
Signature · 1 line
2.45.2
author
#2 lore raw

PATCH 2/3 btrfs-progs: cross-port btrfs_uuid_tree_add() from kernel

In reply to Qu Wenruo

The modification is minimal:

- Replace WARN_ON() with UASSERT()

- Remove the @trans parameter for btrfs_extend_item() and btrfs_mark_buffer_dirty()
As progs version doesn't need a transaction handler.

- Remove the btrfs_uuid_tree_add() in mkfs/main.c

---
 kernel-shared/uuid-tree.c | 119 ++++++++++++++++++++++++++++++++++++++
 kernel-shared/uuid-tree.h |   2 +
 mkfs/main.c               |  56 ------------------
 3 files changed, 121 insertions(+), 56 deletions(-)
kernel-shared/uuid-tree.c +119 -0
index 26359520d57c..3b00945d3981 100644
@@ -30,6 +30,7 @@
 #include "kernel-shared/messages.h"
 #include "kernel-shared/transaction.h"
 #include "kernel-shared/uuid-tree.h"
+#include "kernel-shared/disk-io.h"
 #include "common/messages.h"
 #include "common/utils.h"
 
@@ -199,3 +200,121 @@ out:
 	btrfs_free_path(path);
 	return ret;
 }
+
+/* return -ENOENT for !found, < 0 for errors, or 0 if an item was found */
+static int btrfs_uuid_tree_lookup(struct btrfs_root *uuid_root, const u8 *uuid,
+				  u8 type, u64 subid)
+{
+	int ret;
+	struct btrfs_path *path = NULL;
+	struct extent_buffer *eb;
+	int slot;
+	u32 item_size;
+	unsigned long offset;
+	struct btrfs_key key;
+
+	UASSERT(uuid_root);
+
+	path = btrfs_alloc_path();
+	if (!path) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	btrfs_uuid_to_key(uuid, type, &key);
+	ret = btrfs_search_slot(NULL, uuid_root, &key, path, 0, 0);
+	if (ret < 0) {
+		goto out;
+	} else if (ret > 0) {
+		ret = -ENOENT;
+		goto out;
+	}
+
+	eb = path->nodes[0];
+	slot = path->slots[0];
+	item_size = btrfs_item_size(eb, slot);
+	offset = btrfs_item_ptr_offset(eb, slot);
+	ret = -ENOENT;
+
+	if (!IS_ALIGNED(item_size, sizeof(u64))) {
+		btrfs_warn(uuid_root->fs_info,
+			   "uuid item with illegal size %lu!",
+			   (unsigned long)item_size);
+		goto out;
+	}
+	while (item_size) {
+		__le64 data;
+
+		read_extent_buffer(eb, &data, offset, sizeof(data));
+		if (le64_to_cpu(data) == subid) {
+			ret = 0;
+			break;
+		}
+		offset += sizeof(data);
+		item_size -= sizeof(data);
+	}
+
+out:
+	btrfs_free_path(path);
+	return ret;
+}
+
+int btrfs_uuid_tree_add(struct btrfs_trans_handle *trans, const u8 *uuid, u8 type,
+			u64 subid_cpu)
+{
+	struct btrfs_fs_info *fs_info = trans->fs_info;
+	struct btrfs_root *uuid_root = fs_info->uuid_root;
+	int ret;
+	struct btrfs_path *path = NULL;
+	struct btrfs_key key;
+	struct extent_buffer *eb;
+	int slot;
+	unsigned long offset;
+	__le64 subid_le;
+
+	ret = btrfs_uuid_tree_lookup(uuid_root, uuid, type, subid_cpu);
+	if (ret != -ENOENT)
+		return ret;
+
+	UASSERT(uuid_root);
+	btrfs_uuid_to_key(uuid, type, &key);
+
+	path = btrfs_alloc_path();
+	if (!path) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	ret = btrfs_insert_empty_item(trans, uuid_root, path, &key,
+				      sizeof(subid_le));
+	if (ret == 0) {
+		/* Add an item for the type for the first time */
+		eb = path->nodes[0];
+		slot = path->slots[0];
+		offset = btrfs_item_ptr_offset(eb, slot);
+	} else if (ret == -EEXIST) {
+		/*
+		 * An item with that type already exists.
+		 * Extend the item and store the new subid at the end.
+		 */
+		btrfs_extend_item(path, sizeof(subid_le));
+		eb = path->nodes[0];
+		slot = path->slots[0];
+		offset = btrfs_item_ptr_offset(eb, slot);
+		offset += btrfs_item_size(eb, slot) - sizeof(subid_le);
+	} else {
+		btrfs_warn(fs_info,
+			   "insert uuid item failed %d (0x%016llx, 0x%016llx) type %u!",
+			   ret, key.objectid, key.offset, type);
+		goto out;
+	}
+
+	ret = 0;
+	subid_le = cpu_to_le64(subid_cpu);
+	write_extent_buffer(eb, &subid_le, offset, sizeof(subid_le));
+	btrfs_mark_buffer_dirty(eb);
+
+out:
+	btrfs_free_path(path);
+	return ret;
+}
kernel-shared/uuid-tree.h +2 -0
index 0cdc2228c44f..904cabe7ea94 100644
@@ -29,5 +29,7 @@ int btrfs_lookup_uuid_received_subvol_item(int fd, const u8 *uuid,
 int btrfs_uuid_tree_remove(struct btrfs_trans_handle *trans, u8 *uuid, u8 type,
 			   u64 subid);
 void btrfs_uuid_to_key(const u8 *uuid, u8 type, struct btrfs_key *key);
+int btrfs_uuid_tree_add(struct btrfs_trans_handle *trans, const u8 *uuid, u8 type,
+			u64 subid_cpu);
 
 #endif
mkfs/main.c +0 -56

62 lines not on this page · raw message

Signature · 1 line
2.45.2
author
#3 lore raw

PATCH 3/3 btrfs-progs: introduce btrfs_rebuild_uuid_tree() for mkfs and btrfs-convert

In reply to Qu Wenruo

Currently mkfs uses its own create_uuid_tree(), but that function is only handling FS_TREE.

This means for btrfs-convert, we do not generate the uuid tree, nor add the UUID of the image subvolume.

This can be a problem if we're going to support multiple subvolumes during mkfs time.

To address this inconvenience, this patch introduces a new helper, btrfs_rebuild_uuid_tree(), which will:

- Create a new uuid tree if there is not one

- Empty the existing uuid tree

- Iterate through all subvolumes
* If the subvolume has no valid UUID, regenerate one
* Add the uuid entry for the subvolume UUID
* If the subvolume has received UUID, also add it to UUID tree

By this, this new helper can handle all the uuid tree generation needs for:

- Current mkfs
Only one uuid entry for FS_TREE

- Current btrfs-convert
Only FS_TREE and the image subvolume

- Future multi-subvolume mkfs
As we do the scan for all subvolumes.

- Future "btrfs rescue rebuild-uuid-tree"

---
 common/root-tree-utils.c | 265 +++++++++++++++++++++++++++++++++++++++
 common/root-tree-utils.h |   1 +
 convert/main.c           |   5 +
 mkfs/main.c              |  37 +-----
 4 files changed, 274 insertions(+), 34 deletions(-)
common/root-tree-utils.c +265 -0
index 6a57c51a8a74..13f89dbd5293 100644
@@ -15,9 +15,11 @@
  */
 
 #include <time.h>
+#include <uuid/uuid.h>
 #include "common/root-tree-utils.h"
 #include "common/messages.h"
 #include "kernel-shared/disk-io.h"
+#include "kernel-shared/uuid-tree.h"
 
 int btrfs_make_root_dir(struct btrfs_trans_handle *trans,
 			struct btrfs_root *root, u64 objectid)
@@ -212,3 +214,266 @@ abort:
 	btrfs_abort_transaction(trans, ret);
 	return ret;
 }
+
+static int empty_tree(struct btrfs_root *root)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_path path = { 0 };
+	struct btrfs_key key = { 0 };
+	int ret;
+
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		errno = -ret;
+		error_msg(ERROR_MSG_START_TRANS, "emptry tree: %m");
+		return ret;
+	}
+	while (true) {
+		int nr_items;
+
+		ret = btrfs_search_slot(trans, root, &key, &path, -1, 1);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to locate the first key of root %lld: %m",
+				root->root_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		UASSERT(ret > 0);
+		nr_items = btrfs_header_nritems(path.nodes[0]);
+		/* The tree is empty. */
+		if (nr_items == 0) {
+			btrfs_release_path(&path);
+			break;
+		}
+		ret = btrfs_del_items(trans, root, &path, 0, nr_items);
+		btrfs_release_path(&path);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to empty the first leaf of root %lld: %m",
+				root->root_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+	ret = btrfs_commit_transaction(trans, root);
+	if (ret < 0) {
+		errno = -ret;
+		error_msg(ERROR_MSG_COMMIT_TRANS, "empty tree: %m");
+	}
+	return ret;
+}
+
+static int rescan_subvol_uuid(struct btrfs_trans_handle *trans,
+			      struct btrfs_key *subvol_key)
+{
+	struct btrfs_fs_info *fs_info = trans->fs_info;
+	struct btrfs_root *subvol;
+	int ret;
+
+	UASSERT(is_fstree(subvol_key->objectid));
+
+	/*
+	 * Read out the subvolume root and updates root::root_item.
+	 * This is to avoid de-sync between in-memory and on-disk root_items.
+	 */
+	subvol = btrfs_read_fs_root(fs_info, subvol_key);
+	if (IS_ERR(subvol)) {
+		ret = PTR_ERR(subvol);
+		error("failed to read subvolume %llu: %m",
+			subvol_key->objectid);
+		btrfs_abort_transaction(trans, ret);
+		return ret;
+	}
+	/* The uuid is not set, regenerate one. */
+	if (uuid_is_null(subvol->root_item.uuid)) {
+		uuid_generate(subvol->root_item.uuid);
+		ret = btrfs_update_root(trans, fs_info->tree_root, &subvol->root_key,
+					&subvol->root_item);
+		if (ret < 0) {
+			error("failed to update subvolume %llu: %m",
+			      subvol_key->objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+	ret = btrfs_uuid_tree_add(trans, subvol->root_item.uuid,
+				  BTRFS_UUID_KEY_SUBVOL,
+				  subvol->root_key.objectid);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to add uuid for subvolume %llu: %m",
+		      subvol_key->objectid);
+		btrfs_abort_transaction(trans, ret);
+		return ret;
+	}
+	if (!uuid_is_null(subvol->root_item.received_uuid)) {
+		ret = btrfs_uuid_tree_add(trans, subvol->root_item.uuid,
+					  BTRFS_UUID_KEY_RECEIVED_SUBVOL,
+					  subvol->root_key.objectid);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to add received_uuid for subvol %llu: %m",
+			      subvol->root_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+	return 0;
+}
+
+static int rescan_uuid_tree(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_root *tree_root = fs_info->tree_root;
+	struct btrfs_root *uuid_root = fs_info->uuid_root;
+	struct btrfs_trans_handle *trans;
+	struct btrfs_path path = { 0 };
+	struct btrfs_key key = { 0 };
+	int ret;
+
+	UASSERT(uuid_root);
+	trans = btrfs_start_transaction(uuid_root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		errno = -ret;
+		error_msg(ERROR_MSG_START_TRANS, "rescan uuid tree: %m");
+		return ret;
+	}
+	key.objectid = BTRFS_LAST_FREE_OBJECTID;
+	key.type = BTRFS_ROOT_ITEM_KEY;
+	key.offset = (u64)-1;
+	/* Iterate through all subvolumes except fs tree. */
+	while (true) {
+		struct btrfs_key found_key;
+		struct extent_buffer *leaf;
+		int slot;
+
+		/* No more subvolume. */
+		if (key.objectid < BTRFS_FIRST_FREE_OBJECTID) {
+			ret = 0;
+			break;
+		}
+		ret = btrfs_search_slot(NULL, tree_root, &key, &path, 0, 0);
+		if (ret < 0) {
+			errno = -ret;
+			error_msg(ERROR_MSG_READ, "iterate subvolumes: %m");
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		if (ret > 0) {
+			ret = btrfs_previous_item(tree_root, &path,
+						  BTRFS_FIRST_FREE_OBJECTID,
+						  BTRFS_ROOT_ITEM_KEY);
+			if (ret < 0) {
+				errno = -ret;
+				btrfs_release_path(&path);
+				error_msg(ERROR_MSG_READ, "iterate subvolumes: %m");
+				btrfs_abort_transaction(trans, ret);
+				return ret;
+			}
+			/* No more subvolume. */
+			if (ret > 0) {
+				ret = 0;
+				btrfs_release_path(&path);
+				break;
+			}
+		}
+		leaf = path.nodes[0];
+		slot = path.slots[0];
+		btrfs_item_key_to_cpu(leaf, &found_key, slot);
+		btrfs_release_path(&path);
+		key.objectid = found_key.objectid - 1;
+
+		ret = rescan_subvol_uuid(trans, &found_key);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to rescan the uuid of subvolume %llu: %m",
+			      found_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+
+	/* Update fs tree uuid. */
+	key.objectid = BTRFS_FS_TREE_OBJECTID;
+	key.type = BTRFS_ROOT_ITEM_KEY;
+	key.offset = 0;
+	ret = rescan_subvol_uuid(trans, &key);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to rescan the uuid of subvolume %llu: %m",
+		      key.objectid);
+		btrfs_abort_transaction(trans, ret);
+		return ret;
+	}
+	ret = btrfs_commit_transaction(trans, uuid_root);
+	if (ret < 0) {
+		errno = -ret;
+		error_msg(ERROR_MSG_COMMIT_TRANS, "rescan uuid tree: %m");
+	}
+	return ret;
+}
+
+/*
+ * Rebuild the whole uuid tree.
+ *
+ * If no uuid tree is present, create a new one.
+ * If there is an existing uuid tree, all items would be deleted first.
+ *
+ * For all existing subvolumes (except fs tree), any uninitialized uuid
+ * (all zero) be generated using a random uuid, and inserted into the new tree.
+ * And if a subvolume has its UUID initialized, it would not be touched and
+ * be added to the new uuid tree.
+ */
+int btrfs_rebuild_uuid_tree(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_root *uuid_root;
+	struct btrfs_key key;
+	int ret;
+
+	if (!fs_info->uuid_root) {
+		struct btrfs_trans_handle *trans;
+
+		trans = btrfs_start_transaction(fs_info->tree_root, 1);
+		if (IS_ERR(trans)) {
+			ret = PTR_ERR(trans);
+			errno = -ret;
+			error_msg(ERROR_MSG_START_TRANS, "create uuid tree: %m");
+			return ret;
+		}
+		key.objectid = BTRFS_UUID_TREE_OBJECTID;
+		key.type = BTRFS_ROOT_ITEM_KEY;
+		key.offset = 0;
+		uuid_root = btrfs_create_tree(trans, &key);
+		if (IS_ERR(uuid_root)) {
+			ret = PTR_ERR(uuid_root);
+			errno = -ret;
+			error("failed to create uuid root: %m");
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		add_root_to_dirty_list(uuid_root);
+		fs_info->uuid_root = uuid_root;
+		ret = btrfs_commit_transaction(trans, fs_info->tree_root);
+		if (ret < 0) {
+			errno = -ret;
+			error_msg(ERROR_MSG_COMMIT_TRANS, "create uuid tree: %m");
+			return ret;
+		}
+	}
+	UASSERT(fs_info->uuid_root);
+	ret = empty_tree(fs_info->uuid_root);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to clear the existing uuid tree: %m");
+		return ret;
+	}
+	ret = rescan_uuid_tree(fs_info);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to rescan the uuid tree: %m");
+		return ret;
+	}
+	return 0;
+}
common/root-tree-utils.h +1 -0
index 0c4ece24c7cc..3cb508022e0c 100644
@@ -26,5 +26,6 @@ int btrfs_link_subvolume(struct btrfs_trans_handle *trans,
 			 struct btrfs_root *parent_root,
 			 u64 parent_dir, const char *name,
 			 int namelen, struct btrfs_root *subvol);
+int btrfs_rebuild_uuid_tree(struct btrfs_fs_info *fs_info);
 
 #endif
convert/main.c +5 -0
index 078ef64e41ca..aa253781ee99 100644
@@ -1339,6 +1339,11 @@ static int do_convert(const char *devname, u32 convert_flags, u32 nodesize,
 		goto fail;
 	}
 
+	ret = btrfs_rebuild_uuid_tree(image_root->fs_info);
+	if (ret < 0) {
+		errno = -ret;
+		goto fail;
+	}
 	memset(root->fs_info->super_copy->label, 0, BTRFS_LABEL_SIZE);
 	if (convert_flags & CONVERT_FLAG_COPY_LABEL) {
 		strncpy_null(root->fs_info->super_copy->label, cctx.label, BTRFS_LABEL_SIZE);
mkfs/main.c +3 -34
index b95f1c3372a3..00ccac14a41a 100644
@@ -736,35 +736,6 @@ static void update_chunk_allocation(struct btrfs_fs_info *fs_info,
 	}
 }
 
-static int create_uuid_tree(struct btrfs_trans_handle *trans)
-{
-	struct btrfs_fs_info *fs_info = trans->fs_info;
-	struct btrfs_root *root;
-	struct btrfs_key key = {
-		.objectid = BTRFS_UUID_TREE_OBJECTID,
-		.type = BTRFS_ROOT_ITEM_KEY,
-	};
-	int ret = 0;
-
-	UASSERT(fs_info->uuid_root == NULL);
-	root = btrfs_create_tree(trans, &key);
-	if (IS_ERR(root)) {
-		ret = PTR_ERR(root);
-		goto out;
-	}
-
-	add_root_to_dirty_list(root);
-	fs_info->uuid_root = root;
-	ret = btrfs_uuid_tree_add(trans, fs_info->fs_root->root_item.uuid,
-				  BTRFS_UUID_KEY_SUBVOL,
-				  fs_info->fs_root->root_key.objectid);
-	if (ret < 0)
-		btrfs_abort_transaction(trans, ret);
-
-out:
-	return ret;
-}
-
 static int create_global_root(struct btrfs_trans_handle *trans, u64 objectid,
 			      int root_id)
 {
@@ -1822,17 +1793,15 @@ raid_groups:
 		goto out;
 	}
 
-	ret = create_uuid_tree(trans);
-	if (ret)
-		warning(
-	"unable to create uuid tree, will be created after mount: %d", ret);
-
 	ret = btrfs_commit_transaction(trans, root);
 	if (ret) {
 		errno = -ret;
 		error_msg(ERROR_MSG_START_TRANS, "%m");
 		goto out;
 	}
+	ret = btrfs_rebuild_uuid_tree(fs_info);
+	if (ret < 0)
+		goto out;
 
 	ret = cleanup_temp_chunks(fs_info, &allocation, data_profile,
 				  metadata_profile, metadata_profile);
Signature · 1 line
2.45.2
author
#4 lore raw

Re: [PATCH 3/3] btrfs-progs: introduce btrfs_rebuild_uuid_tree() for mkfs and btrfs-convert

在 2024/7/26 19:29, Qu Wenruo 写道:
[...]

common/root-tree-utils.c abort: · line 229

Quoted 12 lines
common/root-tree-utils.c +12 -0
+static int empty_tree(struct btrfs_root *root)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_path path = { 0 };
+	struct btrfs_key key = { 0 };
+	int ret;
+
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		errno = -ret;
+		error_msg(ERROR_MSG_START_TRANS, "emptry tree: %m");

One typo, "emptry" -> "empty"

Fixed in my github repo.

Thanks for the awesome CI codespell check!
Qu

Quoted 338 lines
common/root-tree-utils.c +250 -0
+		return ret;
+	}
+	while (true) {
+		int nr_items;
+
+		ret = btrfs_search_slot(trans, root, &key, &path, -1, 1);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to locate the first key of root %lld: %m",
+				root->root_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		UASSERT(ret > 0);
+		nr_items = btrfs_header_nritems(path.nodes[0]);
+		/* The tree is empty. */
+		if (nr_items == 0) {
+			btrfs_release_path(&path);
+			break;
+		}
+		ret = btrfs_del_items(trans, root, &path, 0, nr_items);
+		btrfs_release_path(&path);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to empty the first leaf of root %lld: %m",
+				root->root_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+	ret = btrfs_commit_transaction(trans, root);
+	if (ret < 0) {
+		errno = -ret;
+		error_msg(ERROR_MSG_COMMIT_TRANS, "empty tree: %m");
+	}
+	return ret;
+}
+
+static int rescan_subvol_uuid(struct btrfs_trans_handle *trans,
+			      struct btrfs_key *subvol_key)
+{
+	struct btrfs_fs_info *fs_info = trans->fs_info;
+	struct btrfs_root *subvol;
+	int ret;
+
+	UASSERT(is_fstree(subvol_key->objectid));
+
+	/*
+	 * Read out the subvolume root and updates root::root_item.
+	 * This is to avoid de-sync between in-memory and on-disk root_items.
+	 */
+	subvol = btrfs_read_fs_root(fs_info, subvol_key);
+	if (IS_ERR(subvol)) {
+		ret = PTR_ERR(subvol);
+		error("failed to read subvolume %llu: %m",
+			subvol_key->objectid);
+		btrfs_abort_transaction(trans, ret);
+		return ret;
+	}
+	/* The uuid is not set, regenerate one. */
+	if (uuid_is_null(subvol->root_item.uuid)) {
+		uuid_generate(subvol->root_item.uuid);
+		ret = btrfs_update_root(trans, fs_info->tree_root, &subvol->root_key,
+					&subvol->root_item);
+		if (ret < 0) {
+			error("failed to update subvolume %llu: %m",
+			      subvol_key->objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+	ret = btrfs_uuid_tree_add(trans, subvol->root_item.uuid,
+				  BTRFS_UUID_KEY_SUBVOL,
+				  subvol->root_key.objectid);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to add uuid for subvolume %llu: %m",
+		      subvol_key->objectid);
+		btrfs_abort_transaction(trans, ret);
+		return ret;
+	}
+	if (!uuid_is_null(subvol->root_item.received_uuid)) {
+		ret = btrfs_uuid_tree_add(trans, subvol->root_item.uuid,
+					  BTRFS_UUID_KEY_RECEIVED_SUBVOL,
+					  subvol->root_key.objectid);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to add received_uuid for subvol %llu: %m",
+			      subvol->root_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+	return 0;
+}
+
+static int rescan_uuid_tree(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_root *tree_root = fs_info->tree_root;
+	struct btrfs_root *uuid_root = fs_info->uuid_root;
+	struct btrfs_trans_handle *trans;
+	struct btrfs_path path = { 0 };
+	struct btrfs_key key = { 0 };
+	int ret;
+
+	UASSERT(uuid_root);
+	trans = btrfs_start_transaction(uuid_root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		errno = -ret;
+		error_msg(ERROR_MSG_START_TRANS, "rescan uuid tree: %m");
+		return ret;
+	}
+	key.objectid = BTRFS_LAST_FREE_OBJECTID;
+	key.type = BTRFS_ROOT_ITEM_KEY;
+	key.offset = (u64)-1;
+	/* Iterate through all subvolumes except fs tree. */
+	while (true) {
+		struct btrfs_key found_key;
+		struct extent_buffer *leaf;
+		int slot;
+
+		/* No more subvolume. */
+		if (key.objectid < BTRFS_FIRST_FREE_OBJECTID) {
+			ret = 0;
+			break;
+		}
+		ret = btrfs_search_slot(NULL, tree_root, &key, &path, 0, 0);
+		if (ret < 0) {
+			errno = -ret;
+			error_msg(ERROR_MSG_READ, "iterate subvolumes: %m");
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		if (ret > 0) {
+			ret = btrfs_previous_item(tree_root, &path,
+						  BTRFS_FIRST_FREE_OBJECTID,
+						  BTRFS_ROOT_ITEM_KEY);
+			if (ret < 0) {
+				errno = -ret;
+				btrfs_release_path(&path);
+				error_msg(ERROR_MSG_READ, "iterate subvolumes: %m");
+				btrfs_abort_transaction(trans, ret);
+				return ret;
+			}
+			/* No more subvolume. */
+			if (ret > 0) {
+				ret = 0;
+				btrfs_release_path(&path);
+				break;
+			}
+		}
+		leaf = path.nodes[0];
+		slot = path.slots[0];
+		btrfs_item_key_to_cpu(leaf, &found_key, slot);
+		btrfs_release_path(&path);
+		key.objectid = found_key.objectid - 1;
+
+		ret = rescan_subvol_uuid(trans, &found_key);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to rescan the uuid of subvolume %llu: %m",
+			      found_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+
+	/* Update fs tree uuid. */
+	key.objectid = BTRFS_FS_TREE_OBJECTID;
+	key.type = BTRFS_ROOT_ITEM_KEY;
+	key.offset = 0;
+	ret = rescan_subvol_uuid(trans, &key);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to rescan the uuid of subvolume %llu: %m",
+		      key.objectid);
+		btrfs_abort_transaction(trans, ret);
+		return ret;
+	}
+	ret = btrfs_commit_transaction(trans, uuid_root);
+	if (ret < 0) {
+		errno = -ret;
+		error_msg(ERROR_MSG_COMMIT_TRANS, "rescan uuid tree: %m");
+	}
+	return ret;
+}
+
+/*
+ * Rebuild the whole uuid tree.
+ *
+ * If no uuid tree is present, create a new one.
+ * If there is an existing uuid tree, all items would be deleted first.
+ *
+ * For all existing subvolumes (except fs tree), any uninitialized uuid
+ * (all zero) be generated using a random uuid, and inserted into the new tree.
+ * And if a subvolume has its UUID initialized, it would not be touched and
+ * be added to the new uuid tree.
+ */
+int btrfs_rebuild_uuid_tree(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_root *uuid_root;
+	struct btrfs_key key;
+	int ret;
+
+	if (!fs_info->uuid_root) {
+		struct btrfs_trans_handle *trans;
+
+		trans = btrfs_start_transaction(fs_info->tree_root, 1);
+		if (IS_ERR(trans)) {
+			ret = PTR_ERR(trans);
+			errno = -ret;
+			error_msg(ERROR_MSG_START_TRANS, "create uuid tree: %m");
+			return ret;
+		}
+		key.objectid = BTRFS_UUID_TREE_OBJECTID;
+		key.type = BTRFS_ROOT_ITEM_KEY;
+		key.offset = 0;
+		uuid_root = btrfs_create_tree(trans, &key);
+		if (IS_ERR(uuid_root)) {
+			ret = PTR_ERR(uuid_root);
+			errno = -ret;
+			error("failed to create uuid root: %m");
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		add_root_to_dirty_list(uuid_root);
+		fs_info->uuid_root = uuid_root;
+		ret = btrfs_commit_transaction(trans, fs_info->tree_root);
+		if (ret < 0) {
+			errno = -ret;
+			error_msg(ERROR_MSG_COMMIT_TRANS, "create uuid tree: %m");
+			return ret;
+		}
+	}
+	UASSERT(fs_info->uuid_root);
+	ret = empty_tree(fs_info->uuid_root);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to clear the existing uuid tree: %m");
+		return ret;
+	}
+	ret = rescan_uuid_tree(fs_info);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to rescan the uuid tree: %m");
+		return ret;
+	}
+	return 0;
+}
common/root-tree-utils.h +1 -0
index 0c4ece24c7cc..3cb508022e0c 100644
@@ -26,5 +26,6 @@ int btrfs_link_subvolume(struct btrfs_trans_handle *trans,
  			 struct btrfs_root *parent_root,
  			 u64 parent_dir, const char *name,
  			 int namelen, struct btrfs_root *subvol);
+int btrfs_rebuild_uuid_tree(struct btrfs_fs_info *fs_info);
 
  #endif
convert/main.c +5 -0
index 078ef64e41ca..aa253781ee99 100644
@@ -1339,6 +1339,11 @@ static int do_convert(const char *devname, u32 convert_flags, u32 nodesize,
  		goto fail;
  	}
 
+	ret = btrfs_rebuild_uuid_tree(image_root->fs_info);
+	if (ret < 0) {
+		errno = -ret;
+		goto fail;
+	}
  	memset(root->fs_info->super_copy->label, 0, BTRFS_LABEL_SIZE);
  	if (convert_flags & CONVERT_FLAG_COPY_LABEL) {
  		strncpy_null(root->fs_info->super_copy->label, cctx.label, BTRFS_LABEL_SIZE);
mkfs/main.c +3 -34
index b95f1c3372a3..00ccac14a41a 100644
@@ -736,35 +736,6 @@ static void update_chunk_allocation(struct btrfs_fs_info *fs_info,
  	}
  }
 
-static int create_uuid_tree(struct btrfs_trans_handle *trans)
-{
-	struct btrfs_fs_info *fs_info = trans->fs_info;
-	struct btrfs_root *root;
-	struct btrfs_key key = {
-		.objectid = BTRFS_UUID_TREE_OBJECTID,
-		.type = BTRFS_ROOT_ITEM_KEY,
-	};
-	int ret = 0;
-
-	UASSERT(fs_info->uuid_root == NULL);
-	root = btrfs_create_tree(trans, &key);
-	if (IS_ERR(root)) {
-		ret = PTR_ERR(root);
-		goto out;
-	}
-
-	add_root_to_dirty_list(root);
-	fs_info->uuid_root = root;
-	ret = btrfs_uuid_tree_add(trans, fs_info->fs_root->root_item.uuid,
-				  BTRFS_UUID_KEY_SUBVOL,
-				  fs_info->fs_root->root_key.objectid);
-	if (ret < 0)
-		btrfs_abort_transaction(trans, ret);
-
-out:
-	return ret;
-}
-
  static int create_global_root(struct btrfs_trans_handle *trans, u64 objectid,
  			      int root_id)
  {
@@ -1822,17 +1793,15 @@ raid_groups:
  		goto out;
  	}
 
-	ret = create_uuid_tree(trans);
-	if (ret)
-		warning(
-	"unable to create uuid tree, will be created after mount: %d", ret);
-
  	ret = btrfs_commit_transaction(trans, root);
  	if (ret) {
  		errno = -ret;
  		error_msg(ERROR_MSG_START_TRANS, "%m");
  		goto out;
  	}
+	ret = btrfs_rebuild_uuid_tree(fs_info);
+	if (ret < 0)
+		goto out;
 
  	ret = cleanup_temp_chunks(fs_info, &allocation, data_profile,
  				  metadata_profile, metadata_profile);
#5 lore raw

Re: [PATCH 3/3] btrfs-progs: introduce btrfs_rebuild_uuid_tree() for mkfs and btrfs-convert

In reply to Qu Wenruo
Quoted 300 lines · Qu Wenruo

Currently mkfs uses its own create_uuid_tree(), but that function is only handling FS_TREE.

This means for btrfs-convert, we do not generate the uuid tree, nor add the UUID of the image subvolume.

This can be a problem if we're going to support multiple subvolumes during mkfs time.

To address this inconvenience, this patch introduces a new helper, btrfs_rebuild_uuid_tree(), which will:

- Create a new uuid tree if there is not one

- Empty the existing uuid tree

- Iterate through all subvolumes
* If the subvolume has no valid UUID, regenerate one
* Add the uuid entry for the subvolume UUID
* If the subvolume has received UUID, also add it to UUID tree

By this, this new helper can handle all the uuid tree generation needs for:

- Current mkfs
Only one uuid entry for FS_TREE

- Current btrfs-convert
Only FS_TREE and the image subvolume

- Future multi-subvolume mkfs
As we do the scan for all subvolumes.

- Future "btrfs rescue rebuild-uuid-tree"

---
 common/root-tree-utils.c | 265 +++++++++++++++++++++++++++++++++++++++
 common/root-tree-utils.h |   1 +
 convert/main.c           |   5 +
 mkfs/main.c              |  37 +-----
 4 files changed, 274 insertions(+), 34 deletions(-)
common/root-tree-utils.c +240 -0
index 6a57c51a8a74..13f89dbd5293 100644
@@ -15,9 +15,11 @@
  */
 
 #include <time.h>
+#include <uuid/uuid.h>
 #include "common/root-tree-utils.h"
 #include "common/messages.h"
 #include "kernel-shared/disk-io.h"
+#include "kernel-shared/uuid-tree.h"
 
 int btrfs_make_root_dir(struct btrfs_trans_handle *trans,
 			struct btrfs_root *root, u64 objectid)
@@ -212,3 +214,266 @@ abort:
 	btrfs_abort_transaction(trans, ret);
 	return ret;
 }
+
+static int empty_tree(struct btrfs_root *root)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_path path = { 0 };
+	struct btrfs_key key = { 0 };
+	int ret;
+
+	trans = btrfs_start_transaction(root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		errno = -ret;
+		error_msg(ERROR_MSG_START_TRANS, "emptry tree: %m");
+		return ret;
+	}
+	while (true) {
+		int nr_items;
+
+		ret = btrfs_search_slot(trans, root, &key, &path, -1, 1);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to locate the first key of root %lld: %m",
+				root->root_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		UASSERT(ret > 0);
+		nr_items = btrfs_header_nritems(path.nodes[0]);
+		/* The tree is empty. */
+		if (nr_items == 0) {
+			btrfs_release_path(&path);
+			break;
+		}
+		ret = btrfs_del_items(trans, root, &path, 0, nr_items);
+		btrfs_release_path(&path);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to empty the first leaf of root %lld: %m",
+				root->root_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+	ret = btrfs_commit_transaction(trans, root);
+	if (ret < 0) {
+		errno = -ret;
+		error_msg(ERROR_MSG_COMMIT_TRANS, "empty tree: %m");
+	}
+	return ret;
+}
+
+static int rescan_subvol_uuid(struct btrfs_trans_handle *trans,
+			      struct btrfs_key *subvol_key)
+{
+	struct btrfs_fs_info *fs_info = trans->fs_info;
+	struct btrfs_root *subvol;
+	int ret;
+
+	UASSERT(is_fstree(subvol_key->objectid));
+
+	/*
+	 * Read out the subvolume root and updates root::root_item.
+	 * This is to avoid de-sync between in-memory and on-disk root_items.
+	 */
+	subvol = btrfs_read_fs_root(fs_info, subvol_key);
+	if (IS_ERR(subvol)) {
+		ret = PTR_ERR(subvol);
+		error("failed to read subvolume %llu: %m",
+			subvol_key->objectid);
+		btrfs_abort_transaction(trans, ret);
+		return ret;
+	}
+	/* The uuid is not set, regenerate one. */
+	if (uuid_is_null(subvol->root_item.uuid)) {
+		uuid_generate(subvol->root_item.uuid);
+		ret = btrfs_update_root(trans, fs_info->tree_root, &subvol->root_key,
+					&subvol->root_item);
+		if (ret < 0) {
+			error("failed to update subvolume %llu: %m",
+			      subvol_key->objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+	ret = btrfs_uuid_tree_add(trans, subvol->root_item.uuid,
+				  BTRFS_UUID_KEY_SUBVOL,
+				  subvol->root_key.objectid);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to add uuid for subvolume %llu: %m",
+		      subvol_key->objectid);
+		btrfs_abort_transaction(trans, ret);
+		return ret;
+	}
+	if (!uuid_is_null(subvol->root_item.received_uuid)) {
+		ret = btrfs_uuid_tree_add(trans, subvol->root_item.uuid,
+					  BTRFS_UUID_KEY_RECEIVED_SUBVOL,
+					  subvol->root_key.objectid);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to add received_uuid for subvol %llu: %m",
+			      subvol->root_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+	return 0;
+}
+
+static int rescan_uuid_tree(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_root *tree_root = fs_info->tree_root;
+	struct btrfs_root *uuid_root = fs_info->uuid_root;
+	struct btrfs_trans_handle *trans;
+	struct btrfs_path path = { 0 };
+	struct btrfs_key key = { 0 };
+	int ret;
+
+	UASSERT(uuid_root);
+	trans = btrfs_start_transaction(uuid_root, 1);
+	if (IS_ERR(trans)) {
+		ret = PTR_ERR(trans);
+		errno = -ret;
+		error_msg(ERROR_MSG_START_TRANS, "rescan uuid tree: %m");
+		return ret;
+	}
+	key.objectid = BTRFS_LAST_FREE_OBJECTID;
+	key.type = BTRFS_ROOT_ITEM_KEY;
+	key.offset = (u64)-1;
+	/* Iterate through all subvolumes except fs tree. */
+	while (true) {
+		struct btrfs_key found_key;
+		struct extent_buffer *leaf;
+		int slot;
+
+		/* No more subvolume. */
+		if (key.objectid < BTRFS_FIRST_FREE_OBJECTID) {
+			ret = 0;
+			break;
+		}
+		ret = btrfs_search_slot(NULL, tree_root, &key, &path, 0, 0);
+		if (ret < 0) {
+			errno = -ret;
+			error_msg(ERROR_MSG_READ, "iterate subvolumes: %m");
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+		if (ret > 0) {
+			ret = btrfs_previous_item(tree_root, &path,
+						  BTRFS_FIRST_FREE_OBJECTID,
+						  BTRFS_ROOT_ITEM_KEY);
+			if (ret < 0) {
+				errno = -ret;
+				btrfs_release_path(&path);
+				error_msg(ERROR_MSG_READ, "iterate subvolumes: %m");
+				btrfs_abort_transaction(trans, ret);
+				return ret;
+			}
+			/* No more subvolume. */
+			if (ret > 0) {
+				ret = 0;
+				btrfs_release_path(&path);
+				break;
+			}
+		}
+		leaf = path.nodes[0];
+		slot = path.slots[0];
+		btrfs_item_key_to_cpu(leaf, &found_key, slot);
+		btrfs_release_path(&path);
+		key.objectid = found_key.objectid - 1;
+
+		ret = rescan_subvol_uuid(trans, &found_key);
+		if (ret < 0) {
+			errno = -ret;
+			error("failed to rescan the uuid of subvolume %llu: %m",
+			      found_key.objectid);
+			btrfs_abort_transaction(trans, ret);
+			return ret;
+		}
+	}
+
+	/* Update fs tree uuid. */
+	key.objectid = BTRFS_FS_TREE_OBJECTID;
+	key.type = BTRFS_ROOT_ITEM_KEY;
+	key.offset = 0;
+	ret = rescan_subvol_uuid(trans, &key);
+	if (ret < 0) {
+		errno = -ret;
+		error("failed to rescan the uuid of subvolume %llu: %m",
+		      key.objectid);
+		btrfs_abort_transaction(trans, ret);
+		return ret;
+	}
+	ret = btrfs_commit_transaction(trans, uuid_root);
+	if (ret < 0) {
+		errno = -ret;
+		error_msg(ERROR_MSG_COMMIT_TRANS, "rescan uuid tree: %m");
+	}
+	return ret;
+}
+
+/*
+ * Rebuild the whole uuid tree.
+ *
+ * If no uuid tree is present, create a new one.
+ * If there is an existing uuid tree, all items would be deleted first.
+ *
+ * For all existing subvolumes (except fs tree), any uninitialized uuid
+ * (all zero) be generated using a random uuid, and inserted into the new tree.
+ * And if a subvolume has its UUID initialized, it would not be touched and
+ * be added to the new uuid tree.
+ */
+int btrfs_rebuild_uuid_tree(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_root *uuid_root;
+	struct btrfs_key key;
+	int ret;
+
+	if (!fs_info->uuid_root) {
+		struct btrfs_trans_handle *trans;
+
+		trans = btrfs_start_transaction(fs_info->tree_root, 1);
+		if (IS_ERR(trans)) {
+			ret = PTR_ERR(trans);
+			errno = -ret;
+			error_msg(ERROR_MSG_START_TRANS, "create uuid tree: %m");
+			return ret;
+		}
+		key.objectid = BTRFS_UUID_TREE_OBJECTID;
+		key.type = BTRFS_ROOT_ITEM_KEY;
+		key.offset = 0;
+		uuid_root = btrfs_create_tree(trans, &key);
+		if (IS_ERR(uuid_root)) {
+			ret = PTR_ERR(uuid_root);
+			errno = -ret;
+			error("failed to create uuid root: %m");
+			btrfs_abort_transaction(trans, ret);
+			return ret;

common/root-tree-utils.c abort: · line 464

Quoted 10 lines
common/root-tree-utils.c +10 -0
+		}
+		add_root_to_dirty_list(uuid_root);
+		fs_info->uuid_root = uuid_root;
+		ret = btrfs_commit_transaction(trans, fs_info->tree_root);
+		if (ret < 0) {
+			errno = -ret;
+			error_msg(ERROR_MSG_COMMIT_TRANS, "create uuid tree: %m");
+			return ret;
+		}
+	}

This can just be

} else {
	ret = empty_tree();
}

ret = rescan_uuid_tree(fs_info).

Thanks,

Josef

#6 lore raw

Re: [PATCH 3/3] btrfs-progs: introduce btrfs_rebuild_uuid_tree() for mkfs and btrfs-convert

In reply to Qu Wenruo
Quoted 59 lines · Qu Wenruo

Currently mkfs uses its own create_uuid_tree(), but that function is only handling FS_TREE.

This means for btrfs-convert, we do not generate the uuid tree, nor add the UUID of the image subvolume.

This can be a problem if we're going to support multiple subvolumes during mkfs time.

To address this inconvenience, this patch introduces a new helper, btrfs_rebuild_uuid_tree(), which will:

- Create a new uuid tree if there is not one

- Empty the existing uuid tree

- Iterate through all subvolumes
* If the subvolume has no valid UUID, regenerate one
* Add the uuid entry for the subvolume UUID
* If the subvolume has received UUID, also add it to UUID tree

By this, this new helper can handle all the uuid tree generation needs for:

- Current mkfs
Only one uuid entry for FS_TREE

- Current btrfs-convert
Only FS_TREE and the image subvolume

- Future multi-subvolume mkfs
As we do the scan for all subvolumes.

- Future "btrfs rescue rebuild-uuid-tree"

---
 common/root-tree-utils.c | 265 +++++++++++++++++++++++++++++++++++++++
 common/root-tree-utils.h |   1 +
 convert/main.c           |   5 +
 mkfs/main.c              |  37 +-----
 4 files changed, 274 insertions(+), 34 deletions(-)
common/root-tree-utils.c +2 -0
index 6a57c51a8a74..13f89dbd5293 100644
@@ -15,9 +15,11 @@
  */
 
 #include <time.h>
+#include <uuid/uuid.h>
 #include "common/root-tree-utils.h"
 #include "common/messages.h"
 #include "kernel-shared/disk-io.h"
+#include "kernel-shared/uuid-tree.h"
 
 int btrfs_make_root_dir(struct btrfs_trans_handle *trans,
 			struct btrfs_root *root, u64 objectid)

common/root-tree-utils.c abort: · line 218

Quoted 5 lines
common/root-tree-utils.c +2 -0
@@ -212,3 +214,266 @@ abort:
 	btrfs_abort_transaction(trans, ret);
 	return ret;
 }
+
+static int empty_tree(struct btrfs_root *root)

Please rename this, it's confusing as empty can be a noun and verb so it's not clear if it's a meant as a predicate or action.

#7 lore raw

Re: [PATCH 3/3] btrfs-progs: introduce btrfs_rebuild_uuid_tree() for mkfs and btrfs-convert

Quoted 57 lines · David Sterba
Quoted 56 lines · Qu Wenruo

Currently mkfs uses its own create_uuid_tree(), but that function is only handling FS_TREE.

This means for btrfs-convert, we do not generate the uuid tree, nor add the UUID of the image subvolume.

This can be a problem if we're going to support multiple subvolumes during mkfs time.

To address this inconvenience, this patch introduces a new helper, btrfs_rebuild_uuid_tree(), which will:

- Create a new uuid tree if there is not one

- Empty the existing uuid tree

- Iterate through all subvolumes
* If the subvolume has no valid UUID, regenerate one
* Add the uuid entry for the subvolume UUID
* If the subvolume has received UUID, also add it to UUID tree

By this, this new helper can handle all the uuid tree generation needs for:

- Current mkfs
Only one uuid entry for FS_TREE

- Current btrfs-convert
Only FS_TREE and the image subvolume

- Future multi-subvolume mkfs
As we do the scan for all subvolumes.

- Future "btrfs rescue rebuild-uuid-tree"

---
  common/root-tree-utils.c | 265 +++++++++++++++++++++++++++++++++++++++
  common/root-tree-utils.h |   1 +
  convert/main.c           |   5 +
  mkfs/main.c              |  37 +-----
  4 files changed, 274 insertions(+), 34 deletions(-)
common/root-tree-utils.c +2 -0
index 6a57c51a8a74..13f89dbd5293 100644
@@ -15,9 +15,11 @@
   */
 
  #include <time.h>
+#include <uuid/uuid.h>
  #include "common/root-tree-utils.h"
  #include "common/messages.h"
  #include "kernel-shared/disk-io.h"
+#include "kernel-shared/uuid-tree.h"
 
Quoted 12 lines

common/root-tree-utils.c abort: · line 218

Quoted 8 lines
common/root-tree-utils.c +2 -0
  int btrfs_make_root_dir(struct btrfs_trans_handle *trans,
  			struct btrfs_root *root, u64 objectid)
@@ -212,3 +214,266 @@ abort:
  	btrfs_abort_transaction(trans, ret);
  	return ret;
  }
+
+static int empty_tree(struct btrfs_root *root)

Please rename this, it's confusing as empty can be a noun and verb so it's not clear if it's a meant as a predicate or action.

Any recommendation? clear_tree()?

Thanks,
Qu

#8 lore raw

Re: [PATCH 3/3] btrfs-progs: introduce btrfs_rebuild_uuid_tree() for mkfs and btrfs-convert

Quoted 72 lines · Qu Wenruo
Quoted 57 lines · David Sterba
Quoted 56 lines · Qu Wenruo

Currently mkfs uses its own create_uuid_tree(), but that function is only handling FS_TREE.

This means for btrfs-convert, we do not generate the uuid tree, nor add the UUID of the image subvolume.

This can be a problem if we're going to support multiple subvolumes during mkfs time.

To address this inconvenience, this patch introduces a new helper, btrfs_rebuild_uuid_tree(), which will:

- Create a new uuid tree if there is not one

- Empty the existing uuid tree

- Iterate through all subvolumes
* If the subvolume has no valid UUID, regenerate one
* Add the uuid entry for the subvolume UUID
* If the subvolume has received UUID, also add it to UUID tree

By this, this new helper can handle all the uuid tree generation needs for:

- Current mkfs
Only one uuid entry for FS_TREE

- Current btrfs-convert
Only FS_TREE and the image subvolume

- Future multi-subvolume mkfs
As we do the scan for all subvolumes.

- Future "btrfs rescue rebuild-uuid-tree"

---
  common/root-tree-utils.c | 265 +++++++++++++++++++++++++++++++++++++++
  common/root-tree-utils.h |   1 +
  convert/main.c           |   5 +
  mkfs/main.c              |  37 +-----
  4 files changed, 274 insertions(+), 34 deletions(-)
common/root-tree-utils.c +2 -0
index 6a57c51a8a74..13f89dbd5293 100644
@@ -15,9 +15,11 @@
   */
 
  #include <time.h>
+#include <uuid/uuid.h>
  #include "common/root-tree-utils.h"
  #include "common/messages.h"
  #include "kernel-shared/disk-io.h"
+#include "kernel-shared/uuid-tree.h"
 
Quoted 12 lines

common/root-tree-utils.c abort: · line 218

Quoted 8 lines
common/root-tree-utils.c +2 -0
  int btrfs_make_root_dir(struct btrfs_trans_handle *trans,
  			struct btrfs_root *root, u64 objectid)
@@ -212,3 +214,266 @@ abort:
  	btrfs_abort_transaction(trans, ret);
  	return ret;
  }
+
+static int empty_tree(struct btrfs_root *root)

Please rename this, it's confusing as empty can be a noun and verb so it's not clear if it's a meant as a predicate or action.

Quoted 1 line

Any recommendation? clear_tree()?

I've posted soem suggestions as the pull request review comments.