Skip to contentSkip to conversation

btrfs-progs: subvolume-list: add qgroup sizes output

PATCH In review Qu Wenruo 2 participants 7 messages Nov 28, 2023 – Mar 25, 2024

Subsystems (2)
lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover btrfs-progs: subvolume-list: add qgroup sizes output

ZFS' management tool is way better received than btrfs-progs, one of the user-friendly point is the default zpool list, which includes the size of each subvolume.

I'm not sure how ZFS handles it, but for btrfs we need qgroups (or the faster but slightly less accurate squota) to get the accurate numbers.

But considering a lot of distro is enabling qgroup by default for exactly the same reason, and during the years qgroup itself is also under a lot of optimization, I believe adding sizes output for btrfs subvolume list is an overall benefit for end uesrs.

This patch would do exactly so, the output example is:

 # ./btrfs subv list -t /mnt/btrfs/
 ID	gen	top level	rfer	excl	path
 --	---	---------	----	----	----
 256	11	5		1064960	1064960	subvol1
 257	11	5		4210688	4210688	subvol2

The extra columns are added depending on if qgroup is enabled, and we allow users to force such output, but if qgroup is not enabled and we're forced to output such sizes, a warning would be outputted and fill all the sizes value as 0.

Thanks William Brown for the UI suggestion.

Although there are still some pitfalls, mentioned in the last patch.

Qu Wenruo (3):
btrfs-progs: separate root attributes into a dedicated structure from
root_info
btrfs-progs: use root_attr structure to pass various attributes
btrfs-progs: subvolume-list: output qgroup sizes for subvolumes

 Documentation/btrfs-subvolume.rst |  12 +-
 cmds/subvolume-list.c             | 572 ++++++++++++++++++------------
 2 files changed, 349 insertions(+), 235 deletions(-)

--
2.42.1

author
#1 lore raw

PATCH 1/3 btrfs-progs: separate root attributes into a dedicated structure from root_info

Currently root_info structure contains both the rb-tree structures like rb_node and sort_node, and the info to describe all the attributes of the subvolume.

For the incoming expansion on subvolume attributes, we want to parse those info through a proper structure (without the unnecessary rb_nodes), other than over a dozen parameters.

Thus this patch would separate the root attributes into root_attr structure, and root_info would just hold a root_attr structure inside it.

---
 cmds/subvolume-list.c | 330 ++++++++++++++++++++++--------------------
 1 file changed, 170 insertions(+), 160 deletions(-)
cmds/subvolume-list.c +170 -160
index 5a91f41da998..e0a1c5d4f626 100644
@@ -102,12 +102,9 @@ enum btrfs_list_layout {
 };
 
 /*
- * one of these for each root we find.
+ * Describle all the info we found for a subvolume.
  */
-struct root_info {
-	struct rb_node rb_node;
-	struct rb_node sort_node;
-
+struct root_attr {
 	/* this root's id */
 	u64 root_id;
 
@@ -146,14 +143,24 @@ struct root_info {
 	/* the name of this root in the directory it lives in */
 	char *name;
 
+	/* length of the above name. */
+	int name_len;
+
 	char *full_path;
 
 	int deleted;
 };
 
-typedef int (*btrfs_list_filter_func)(struct root_info *, u64);
-typedef int (*btrfs_list_comp_func)(const struct root_info *a,
-				    const struct root_info *b);
+struct root_info {
+	struct rb_node rb_node;
+	struct rb_node sort_node;
+
+	struct root_attr attrs;
+};
+
+typedef int (*btrfs_list_filter_func)(struct root_attr *, u64);
+typedef int (*btrfs_list_comp_func)(const struct root_attr *a,
+				    const struct root_attr *b);
 
 struct btrfs_list_filter {
 	btrfs_list_filter_func filter_func;
@@ -309,8 +316,8 @@ static void btrfs_list_setup_print_column(enum btrfs_list_column_enum column)
 		btrfs_list_columns[i].need_print = 1;
 }
 
-static int comp_entry_with_rootid(const struct root_info *entry1,
-				  const struct root_info *entry2)
+static int comp_entry_with_rootid(const struct root_attr *entry1,
+				  const struct root_attr *entry2)
 {
 	if (entry1->root_id > entry2->root_id)
 		return 1;
@@ -319,8 +326,8 @@ static int comp_entry_with_rootid(const struct root_info *entry1,
 	return 0;
 }
 
-static int comp_entry_with_gen(const struct root_info *entry1,
-			       const struct root_info *entry2)
+static int comp_entry_with_gen(const struct root_attr *entry1,
+			       const struct root_attr *entry2)
 {
 	if (entry1->gen > entry2->gen)
 		return 1;
@@ -329,8 +336,8 @@ static int comp_entry_with_gen(const struct root_info *entry1,
 	return 0;
 }
 
-static int comp_entry_with_ogen(const struct root_info *entry1,
-				const struct root_info *entry2)
+static int comp_entry_with_ogen(const struct root_attr *entry1,
+				const struct root_attr *entry2)
 {
 	if (entry1->ogen > entry2->ogen)
 		return 1;
@@ -339,8 +346,8 @@ static int comp_entry_with_ogen(const struct root_info *entry1,
 	return 0;
 }
 
-static int comp_entry_with_path(const struct root_info *entry1,
-				const struct root_info *entry2)
+static int comp_entry_with_path(const struct root_attr *entry1,
+				const struct root_attr *entry2)
 {
 	if (strcmp(entry1->full_path, entry2->full_path) > 0)
 		return 1;
@@ -420,13 +427,13 @@ static int sort_comp(const struct root_info *entry1, const struct root_info *ent
 	int i, ret = 0;
 
 	if (!set || !set->ncomps)
-		return comp_entry_with_rootid(entry1, entry2);
+		return comp_entry_with_rootid(&entry1->attrs, &entry2->attrs);
 
 	for (i = 0; i < set->ncomps; i++) {
 		if (!set->comps[i].comp_func)
 			break;
 
-		ret = set->comps[i].comp_func(entry1, entry2);
+		ret = set->comps[i].comp_func(&entry1->attrs, &entry2->attrs);
 		if (set->comps[i].is_descending)
 			ret = -ret;
 		if (ret)
@@ -437,7 +444,7 @@ static int sort_comp(const struct root_info *entry1, const struct root_info *ent
 	}
 
 	if (!rootid_compared)
-		ret = comp_entry_with_rootid(entry1, entry2);
+		ret = comp_entry_with_rootid(&entry1->attrs, &entry2->attrs);
 
 	return ret;
 }
@@ -486,7 +493,7 @@ static int root_tree_insert(struct rb_root *root_tree,
 		parent = *p;
 		curr = to_root_info(parent);
 
-		ret = comp_entry_with_rootid(ins, curr);
+		ret = comp_entry_with_rootid(&ins->attrs, &curr->attrs);
 		if (ret < 0)
 			p = &(*p)->rb_left;
 		else if (ret > 0)
@@ -509,7 +516,7 @@ static struct root_info *root_tree_search(struct rb_root *root_tree,
 {
 	struct rb_node *n = root_tree->rb_node;
 	struct root_info *entry;
-	struct root_info tmp;
+	struct root_attr tmp;
 	int ret;
 
 	tmp.root_id = root_id;
@@ -517,7 +524,7 @@ static struct root_info *root_tree_search(struct rb_root *root_tree,
 	while(n) {
 		entry = to_root_info(n);
 
-		ret = comp_entry_with_rootid(&tmp, entry);
+		ret = comp_entry_with_rootid(&tmp, &entry->attrs);
 		if (ret < 0)
 			n = n->rb_left;
 		else if (ret > 0)
@@ -534,43 +541,47 @@ static int update_root(struct rb_root *root_lookup,
 		       time_t otime, u8 *uuid, u8 *puuid, u8 *ruuid)
 {
 	struct root_info *ri;
+	struct root_attr *attrs;
 
 	ri = root_tree_search(root_lookup, root_id);
-	if (!ri || ri->root_id != root_id)
+	if (!ri || ri->attrs.root_id != root_id)
 		return -ENOENT;
-	if (name && name_len > 0) {
-		free(ri->name);
 
-		ri->name = malloc(name_len + 1);
-		if (!ri->name) {
+	attrs = &ri->attrs;
+	if (name && name_len > 0) {
+		free(attrs->name);
+
+		attrs->name = malloc(name_len + 1);
+		if (!attrs->name) {
 			error_msg(ERROR_MSG_MEMORY, NULL);
 			exit(1);
 		}
-		strncpy(ri->name, name, name_len);
-		ri->name[name_len] = 0;
+		strncpy(attrs->name, name, name_len);
+		attrs->name[name_len] = 0;
+		attrs->name_len = name_len;
 	}
 	if (ref_tree)
-		ri->ref_tree = ref_tree;
+		attrs->ref_tree = ref_tree;
 	if (root_offset)
-		ri->root_offset = root_offset;
+		attrs->root_offset = root_offset;
 	if (flags)
-		ri->flags = flags;
+		attrs->flags = flags;
 	if (dir_id)
-		ri->dir_id = dir_id;
+		attrs->dir_id = dir_id;
 	if (gen)
-		ri->gen = gen;
+		attrs->gen = gen;
 	if (ogen)
-		ri->ogen = ogen;
-	if (!ri->ogen && root_offset)
-		ri->ogen = root_offset;
+		attrs->ogen = ogen;
+	if (!attrs->ogen && root_offset)
+		attrs->ogen = root_offset;
 	if (otime)
-		ri->otime = otime;
+		attrs->otime = otime;
 	if (uuid)
-		memcpy(&ri->uuid, uuid, BTRFS_UUID_SIZE);
+		memcpy(&attrs->uuid, uuid, BTRFS_UUID_SIZE);
 	if (puuid)
-		memcpy(&ri->puuid, puuid, BTRFS_UUID_SIZE);
+		memcpy(&attrs->puuid, puuid, BTRFS_UUID_SIZE);
 	if (ruuid)
-		memcpy(&ri->ruuid, ruuid, BTRFS_UUID_SIZE);
+		memcpy(&attrs->ruuid, ruuid, BTRFS_UUID_SIZE);
 
 	return 0;
 }
@@ -578,18 +589,7 @@ static int update_root(struct rb_root *root_lookup,
 /*
  * add_root - update the existed root, or allocate a new root and insert it
  *	      into the lookup tree.
- * root_id: object id of the root
- * ref_tree: object id of the referring root.
- * root_offset: offset value of the root'key
- * dir_id: inode id of the directory in ref_tree where this root can be found.
- * name: the name of root_id in that directory
- * name_len: the length of name
- * ogen: the original generation of the root
- * gen: the current generation of the root
- * otime: the original time (creation time) of the root
- * uuid: uuid of the root
- * puuid: uuid of the root parent if any
- * ruuid: uuid of the received subvol, if any
+ * attrs: the description for the new root.
  */
 static int add_root(struct rb_root *root_lookup,
 		    u64 root_id, u64 ref_tree, u64 root_offset, u64 flags,
@@ -597,6 +597,7 @@ static int add_root(struct rb_root *root_lookup,
 		    time_t otime, u8 *uuid, u8 *puuid, u8 *ruuid)
 {
 	struct root_info *ri;
+	struct root_attr *attrs;
 	int ret;
 
 	ret = update_root(root_lookup, root_id, ref_tree, root_offset, flags,
@@ -610,42 +611,44 @@ static int add_root(struct rb_root *root_lookup,
 		error_msg(ERROR_MSG_MEMORY, NULL);
 		exit(1);
 	}
-	ri->root_id = root_id;
+	attrs = &ri->attrs;
+	attrs->root_id = root_id;
 
 	if (name && name_len > 0) {
-		ri->name = malloc(name_len + 1);
-		if (!ri->name) {
+		attrs->name = malloc(name_len + 1);
+		if (!attrs->name) {
 			error_msg(ERROR_MSG_MEMORY, NULL);
 			exit(1);
 		}
-		strncpy(ri->name, name, name_len);
-		ri->name[name_len] = 0;
+		strncpy(attrs->name, name, name_len);
+		attrs->name[name_len] = 0;
+		attrs->name_len = name_len;
 	}
 	if (ref_tree)
-		ri->ref_tree = ref_tree;
+		attrs->ref_tree = ref_tree;
 	if (dir_id)
-		ri->dir_id = dir_id;
+		attrs->dir_id = dir_id;
 	if (root_offset)
-		ri->root_offset = root_offset;
+		attrs->root_offset = root_offset;
 	if (flags)
-		ri->flags = flags;
+		attrs->flags = flags;
 	if (gen)
-		ri->gen = gen;
+		attrs->gen = gen;
 	if (ogen)
-		ri->ogen = ogen;
-	if (!ri->ogen && root_offset)
-		ri->ogen = root_offset;
+		attrs->ogen = ogen;
+	if (!attrs->ogen && root_offset)
+		attrs->ogen = root_offset;
 	if (otime)
-		ri->otime = otime;
+		attrs->otime = otime;
 
 	if (uuid)
-		memcpy(&ri->uuid, uuid, BTRFS_UUID_SIZE);
+		memcpy(&attrs->uuid, uuid, BTRFS_UUID_SIZE);
 
 	if (puuid)
-		memcpy(&ri->puuid, puuid, BTRFS_UUID_SIZE);
+		memcpy(&attrs->puuid, puuid, BTRFS_UUID_SIZE);
 
 	if (ruuid)
-		memcpy(&ri->ruuid, ruuid, BTRFS_UUID_SIZE);
+		memcpy(&attrs->ruuid, ruuid, BTRFS_UUID_SIZE);
 
 	ret = root_tree_insert(root_lookup, ri);
 	if (ret < 0) {
@@ -672,9 +675,9 @@ static void free_root_info(struct rb_node *node)
 	struct root_info *ri;
 
 	ri = to_root_info(node);
-	free(ri->name);
-	free(ri->path);
-	free(ri->full_path);
+	free(ri->attrs.name);
+	free(ri->attrs.path);
+	free(ri->attrs.full_path);
 	free(ri);
 }
 
@@ -691,6 +694,7 @@ static int resolve_root(struct rb_root *rl, struct root_info *ri,
 	char *full_path = NULL;
 	int len = 0;
 	struct root_info *found;
+	struct root_attr *attrs;
 
 	/*
 	 * we go backwards from the root_info object and add pathnames
@@ -702,16 +706,17 @@ static int resolve_root(struct rb_root *rl, struct root_info *ri,
 		u64 next;
 		int add_len;
 
+		attrs = &found->attrs;
 		/*
 		 * ref_tree = 0 indicates the subvolume
 		 * has been deleted.
 		 */
-		if (!found->ref_tree) {
+		if (!attrs->ref_tree) {
 			free(full_path);
 			return -ENOENT;
 		}
 
-		add_len = strlen(found->path);
+		add_len = strlen(attrs->path);
 
 		if (full_path) {
 			/* room for / and for null */
@@ -722,19 +727,19 @@ static int resolve_root(struct rb_root *rl, struct root_info *ri,
 			}
 			memcpy(tmp + add_len + 1, full_path, len);
 			tmp[add_len] = '/';
-			memcpy(tmp, found->path, add_len);
+			memcpy(tmp, attrs->path, add_len);
 			tmp [add_len + len + 1] = '\0';
 			free(full_path);
 			full_path = tmp;
 			len += add_len + 1;
 		} else {
-			full_path = strdup(found->path);
+			full_path = strdup(attrs->path);
 			len = add_len;
 		}
-		if (!ri->top_id)
-			ri->top_id = found->ref_tree;
+		if (!ri->attrs.top_id)
+			ri->attrs.top_id = attrs->ref_tree;
 
-		next = found->ref_tree;
+		next = attrs->ref_tree;
 		if (next == top_id)
 			break;
 		/*
@@ -754,7 +759,7 @@ static int resolve_root(struct rb_root *rl, struct root_info *ri,
 		}
 	}
 
-	ri->full_path = full_path;
+	ri->attrs.full_path = full_path;
 
 	return 0;
 }
@@ -769,25 +774,26 @@ static int resolve_root(struct rb_root *rl, struct root_info *ri,
 static int lookup_ino_path(int fd, struct root_info *ri)
 {
 	struct btrfs_ioctl_ino_lookup_args args;
+	struct root_attr *attrs = &ri->attrs;
 	int ret;
 
-	if (ri->path)
+	if (attrs->path)
 		return 0;
 
-	if (!ri->ref_tree)
+	if (!attrs->ref_tree)
 		return -ENOENT;
 
 	memset(&args, 0, sizeof(args));
-	args.treeid = ri->ref_tree;
-	args.objectid = ri->dir_id;
+	args.treeid = attrs->ref_tree;
+	args.objectid = attrs->dir_id;
 
 	ret = ioctl(fd, BTRFS_IOC_INO_LOOKUP, &args);
 	if (ret < 0) {
 		if (errno == ENOENT) {
-			ri->ref_tree = 0;
+			attrs->ref_tree = 0;
 			return -ENOENT;
 		}
-		error("failed to lookup path for root %llu: %m", ri->ref_tree);
+		error("failed to lookup path for root %llu: %m", attrs->ref_tree);
 		return ret;
 	}
 
@@ -796,17 +802,17 @@ static int lookup_ino_path(int fd, struct root_info *ri)
 		 * we're in a subdirectory of ref_tree, the kernel ioctl
 		 * puts a / in there for us
 		 */
-		ri->path = malloc(strlen(ri->name) + strlen(args.name) + 1);
-		if (!ri->path) {
+		attrs->path = malloc(strlen(attrs->name) + strlen(args.name) + 1);
+		if (!attrs->path) {
 			error_msg(ERROR_MSG_MEMORY, NULL);
 			exit(1);
 		}
-		strcpy(ri->path, args.name);
-		strcat(ri->path, ri->name);
+		strcpy(attrs->path, args.name);
+		strcat(attrs->path, attrs->name);
 	} else {
 		/* we're at the root of ref_tree */
-		ri->path = strdup(ri->name);
-		if (!ri->path) {
+		attrs->path = strdup(attrs->name);
+		if (!attrs->path) {
 			perror("strdup failed");
 			exit(1);
 		}
@@ -932,87 +938,87 @@ static int list_subvol_search(int fd, struct rb_root *root_lookup)
 	return 0;
 }
 
-static int filter_by_rootid(struct root_info *ri, u64 data)
+static int filter_by_rootid(struct root_attr *ra, u64 data)
 {
-	return ri->root_id == data;
+	return ra->root_id == data;
 }
 
-static int filter_snapshot(struct root_info *ri, u64 data)
+static int filter_snapshot(struct root_attr *ra, u64 data)
 {
-	return !!ri->root_offset;
+	return !!ra->root_offset;
 }
 
-static int filter_flags(struct root_info *ri, u64 flags)
+static int filter_flags(struct root_attr *ra, u64 flags)
 {
-	return ri->flags & flags;
+	return ra->flags & flags;
 }
 
-static int filter_gen_more(struct root_info *ri, u64 data)
+static int filter_gen_more(struct root_attr *ra, u64 data)
 {
-	return ri->gen >= data;
+	return ra->gen >= data;
 }
 
-static int filter_gen_less(struct root_info *ri, u64 data)
+static int filter_gen_less(struct root_attr *ra, u64 data)
 {
-	return ri->gen <= data;
+	return ra->gen <= data;
 }
 
-static int filter_gen_equal(struct root_info  *ri, u64 data)
+static int filter_gen_equal(struct root_attr *ra, u64 data)
 {
-	return ri->gen == data;
+	return ra->gen == data;
 }
 
-static int filter_cgen_more(struct root_info *ri, u64 data)
+static int filter_cgen_more(struct root_attr *ra, u64 data)
 {
-	return ri->ogen >= data;
+	return ra->ogen >= data;
 }
 
-static int filter_cgen_less(struct root_info *ri, u64 data)
+static int filter_cgen_less(struct root_attr *ra, u64 data)
 {
-	return ri->ogen <= data;
+	return ra->ogen <= data;
 }
 
-static int filter_cgen_equal(struct root_info *ri, u64 data)
+static int filter_cgen_equal(struct root_attr *ra, u64 data)
 {
-	return ri->ogen == data;
+	return ra->ogen == data;
 }
 
-static int filter_topid_equal(struct root_info *ri, u64 data)
+static int filter_topid_equal(struct root_attr *ra, u64 data)
 {
-	return ri->top_id == data;
+	return ra->top_id == data;
 }
 
-static int filter_full_path(struct root_info *ri, u64 data)
+static int filter_full_path(struct root_attr *ra, u64 data)
 {
-	if (ri->full_path && ri->top_id != data) {
+	if (ra->full_path && ra->top_id != data) {
 		char *tmp;
 		char p[] = "<FS_TREE>";
 		int add_len = strlen(p);
-		int len = strlen(ri->full_path);
+		int len = strlen(ra->full_path);
 
 		tmp = malloc(len + add_len + 2);
 		if (!tmp) {
 			error_msg(ERROR_MSG_MEMORY, NULL);
 			exit(1);
 		}
-		memcpy(tmp + add_len + 1, ri->full_path, len);
+		memcpy(tmp + add_len + 1, ra->full_path, len);
 		tmp[len + add_len + 1] = '\0';
 		tmp[add_len] = '/';
 		memcpy(tmp, p, add_len);
-		free(ri->full_path);
-		ri->full_path = tmp;
+		free(ra->full_path);
+		ra->full_path = tmp;
 	}
 	return 1;
 }
 
-static int filter_by_parent(struct root_info *ri, u64 data)
+static int filter_by_parent(struct root_attr *ra, u64 data)
 {
-	return !uuid_compare(ri->puuid, (u8 *)(unsigned long)data);
+	return !uuid_compare(ra->puuid, (u8 *)(unsigned long)data);
 }
 
-static int filter_deleted(struct root_info *ri, u64 data)
+static int filter_deleted(struct root_attr *ra, u64 data)
 {
-	return ri->deleted;
+	return ra->deleted;
 }
 
 static btrfs_list_filter_func all_filter_funcs[] = {
@@ -1075,7 +1081,7 @@ static void btrfs_list_setup_filter(struct btrfs_list_filter_set **filter_set,
 	set->nfilters++;
 }
 
-static int filter_root(struct root_info *ri,
+static int filter_root(struct root_attr *ra,
 		       struct btrfs_list_filter_set *set)
 {
 	int i, ret;
@@ -1083,16 +1089,16 @@ static int filter_root(struct root_info *ri,
 	if (!set)
 		return 1;
 
-	if (set->only_deleted && !ri->deleted)
+	if (set->only_deleted && !ra->deleted)
 		return 0;
 
-	if (!set->only_deleted && ri->deleted)
+	if (!set->only_deleted && ra->deleted)
 		return 0;
 
 	for (i = 0; i < set->nfilters; i++) {
 		if (!set->filters[i].filter_func)
 			break;
-		ret = set->filters[i].filter_func(ri, set->filters[i].data);
+		ret = set->filters[i].filter_func(ra, set->filters[i].data);
 		if (!ret)
 			return 0;
 	}
@@ -1113,24 +1119,26 @@ static void filter_and_sort_subvol(struct rb_root *all_subvols,
 
 	n = rb_last(all_subvols);
 	while (n) {
+		struct root_attr *attrs;
 		entry = to_root_info(n);
+		attrs = &entry->attrs;
 
 		ret = resolve_root(all_subvols, entry, top_id);
 		if (ret == -ENOENT) {
-			if (entry->root_id != BTRFS_FS_TREE_OBJECTID) {
-				entry->full_path = strdup("DELETED");
-				entry->deleted = 1;
+			if (attrs->root_id != BTRFS_FS_TREE_OBJECTID) {
+				attrs->full_path = strdup("DELETED");
+				attrs->deleted = 1;
 			} else {
 				/*
 				 * The full path is not supposed to be printed,
 				 * but we don't want to print an empty string,
 				 * in case it appears somewhere.
 				 */
-				entry->full_path = strdup("TOPLEVEL");
-				entry->deleted = 0;
+				attrs->full_path = strdup("TOPLEVEL");
+				attrs->deleted = 0;
 			}
 		}
-		ret = filter_root(entry, filter_set);
+		ret = filter_root(attrs, filter_set);
 		if (ret)
 			sort_tree_insert(sort_tree, entry, comp_set);
 		n = rb_prev(n);
@@ -1142,59 +1150,60 @@ static void print_subvolume_column(struct root_info *subv,
 {
 	char tstr[256];
 	char uuidparse[BTRFS_UUID_UNPARSED_SIZE];
+	struct root_attr *attrs = &subv->attrs;
 
 	UASSERT(0 <= column && column < BTRFS_LIST_ALL);
 
 	switch (column) {
 	case BTRFS_LIST_OBJECTID:
-		pr_verbose(LOG_DEFAULT, "%llu", subv->root_id);
+		pr_verbose(LOG_DEFAULT, "%llu", attrs->root_id);
 		break;
 	case BTRFS_LIST_GENERATION:
-		pr_verbose(LOG_DEFAULT, "%llu", subv->gen);
+		pr_verbose(LOG_DEFAULT, "%llu", attrs->gen);
 		break;
 	case BTRFS_LIST_OGENERATION:
-		pr_verbose(LOG_DEFAULT, "%llu", subv->ogen);
+		pr_verbose(LOG_DEFAULT, "%llu", attrs->ogen);
 		break;
 	case BTRFS_LIST_PARENT:
-		pr_verbose(LOG_DEFAULT, "%llu", subv->ref_tree);
+		pr_verbose(LOG_DEFAULT, "%llu", attrs->ref_tree);
 		break;
 	case BTRFS_LIST_TOP_LEVEL:
-		pr_verbose(LOG_DEFAULT, "%llu", subv->top_id);
+		pr_verbose(LOG_DEFAULT, "%llu", attrs->top_id);
 		break;
 	case BTRFS_LIST_OTIME:
-		if (subv->otime) {
+		if (attrs->otime) {
 			struct tm tm;
 
-			localtime_r(&subv->otime, &tm);
+			localtime_r(&attrs->otime, &tm);
 			strftime(tstr, 256, "%Y-%m-%d %X", &tm);
 		} else
 			strcpy(tstr, "-");
 		pr_verbose(LOG_DEFAULT, "%s", tstr);
 		break;
 	case BTRFS_LIST_UUID:
-		if (uuid_is_null(subv->uuid))
+		if (uuid_is_null(attrs->uuid))
 			strcpy(uuidparse, "-");
 		else
-			uuid_unparse(subv->uuid, uuidparse);
+			uuid_unparse(attrs->uuid, uuidparse);
 		pr_verbose(LOG_DEFAULT, "%-36s", uuidparse);
 		break;
 	case BTRFS_LIST_PUUID:
-		if (uuid_is_null(subv->puuid))
+		if (uuid_is_null(attrs->puuid))
 			strcpy(uuidparse, "-");
 		else
-			uuid_unparse(subv->puuid, uuidparse);
+			uuid_unparse(attrs->puuid, uuidparse);
 		pr_verbose(LOG_DEFAULT, "%-36s", uuidparse);
 		break;
 	case BTRFS_LIST_RUUID:
-		if (uuid_is_null(subv->ruuid))
+		if (uuid_is_null(attrs->ruuid))
 			strcpy(uuidparse, "-");
 		else
-			uuid_unparse(subv->ruuid, uuidparse);
+			uuid_unparse(attrs->ruuid, uuidparse);
 		pr_verbose(LOG_DEFAULT, "%-36s", uuidparse);
 		break;
 	case BTRFS_LIST_PATH:
-		BUG_ON(!subv->full_path);
-		pr_verbose(LOG_DEFAULT, "%s", subv->full_path);
+		BUG_ON(!attrs->full_path);
+		pr_verbose(LOG_DEFAULT, "%s", attrs->full_path);
 		break;
 	default:
 		break;
@@ -1288,41 +1297,42 @@ static void print_subvol_json_key(struct format_ctx *fctx,
 				  const enum btrfs_list_column_enum column)
 {
 	const char *column_name;
+	struct root_attr const *attrs = &subv->attrs;
 
 	UASSERT(0 <= column && column < BTRFS_LIST_ALL);
 
 	column_name = btrfs_list_columns[column].name;
 	switch (column) {
 	case BTRFS_LIST_OBJECTID:
-		fmt_print(fctx, column_name, subv->root_id);
+		fmt_print(fctx, column_name, attrs->root_id);
 		break;
 	case BTRFS_LIST_GENERATION:
-		fmt_print(fctx, column_name, subv->gen);
+		fmt_print(fctx, column_name, attrs->gen);
 		break;
 	case BTRFS_LIST_OGENERATION:
-		fmt_print(fctx, column_name, subv->ogen);
+		fmt_print(fctx, column_name, attrs->ogen);
 		break;
 	case BTRFS_LIST_PARENT:
-		fmt_print(fctx, column_name, subv->ref_tree);
+		fmt_print(fctx, column_name, attrs->ref_tree);
 		break;
 	case BTRFS_LIST_TOP_LEVEL:
-		fmt_print(fctx, column_name, subv->top_id);
+		fmt_print(fctx, column_name, attrs->top_id);
 		break;
 	case BTRFS_LIST_OTIME:
-		fmt_print(fctx, column_name, subv->otime);
+		fmt_print(fctx, column_name, attrs->otime);
 		break;
 	case BTRFS_LIST_UUID:
-		fmt_print(fctx, column_name, subv->uuid);
+		fmt_print(fctx, column_name, attrs->uuid);
 		break;
 	case BTRFS_LIST_PUUID:
-		fmt_print(fctx, column_name, subv->puuid);
+		fmt_print(fctx, column_name, attrs->puuid);
 		break;
 	case BTRFS_LIST_RUUID:
-		fmt_print(fctx, column_name, subv->ruuid);
+		fmt_print(fctx, column_name, attrs->ruuid);
 		break;
 	case BTRFS_LIST_PATH:
-		BUG_ON(!subv->full_path);
-		fmt_print(fctx, column_name, subv->full_path);
+		BUG_ON(!attrs->full_path);
+		fmt_print(fctx, column_name, attrs->full_path);
 		break;
 	default:
 		break;
@@ -1366,7 +1376,7 @@ static void print_all_subvol_info(struct rb_root *sorted_tree,
 		entry = to_root_info_sorted(n);
 
 		/* The toplevel subvolume is not listed by default */
-		if (entry->root_id == BTRFS_FS_TREE_OBJECTID)
+		if (entry->attrs.root_id == BTRFS_FS_TREE_OBJECTID)
 			goto next;
 
 		switch (layout) {
Signature · 1 line
2.42.1
author
#2 lore raw

PATCH 2/3 btrfs-progs: use root_attr structure to pass various attributes

In reply to Qu Wenruo

Currently for the following functions, we have at least 14 parameters, just to describle various attributes:

- update_root()
- add_root()
- add_root_backref()

With the recent introduce of root_attr structure, we can easily replace those parameters with a root_attr structure pointer.

Furthermore, since add_root() and update_root() are both updating the same structure, we can also extract a update_root_attr() helper, to remove the duplicated code.

---
 cmds/subvolume-list.c | 188 ++++++++++++++++--------------------------
 1 file changed, 72 insertions(+), 116 deletions(-)
cmds/subvolume-list.c +72 -116
index e0a1c5d4f626..e05f7228b889 100644
@@ -535,54 +535,57 @@ static struct root_info *root_tree_search(struct rb_root *root_tree,
 	return NULL;
 }
 
-static int update_root(struct rb_root *root_lookup,
-		       u64 root_id, u64 ref_tree, u64 root_offset, u64 flags,
-		       u64 dir_id, char *name, int name_len, u64 ogen, u64 gen,
-		       time_t otime, u8 *uuid, u8 *puuid, u8 *ruuid)
+/* Update the root attributes of @found to @new, except the rootid. */
+static void update_root_attr(struct root_attr *found,
+			     const struct root_attr *new)
 {
-	struct root_info *ri;
-	struct root_attr *attrs;
+	if (new->name && new->name_len > 0) {
+		free(found->name);
 
-	ri = root_tree_search(root_lookup, root_id);
-	if (!ri || ri->attrs.root_id != root_id)
-		return -ENOENT;
-
-	attrs = &ri->attrs;
-	if (name && name_len > 0) {
-		free(attrs->name);
-
-		attrs->name = malloc(name_len + 1);
-		if (!attrs->name) {
+		found->name = malloc(new->name_len + 1);
+		if (!found->name) {
 			error_msg(ERROR_MSG_MEMORY, NULL);
 			exit(1);
 		}
-		strncpy(attrs->name, name, name_len);
-		attrs->name[name_len] = 0;
-		attrs->name_len = name_len;
+		strncpy(found->name, new->name, new->name_len);
+		found->name[new->name_len] = '\0';
+		found->name_len = new->name_len;
 	}
-	if (ref_tree)
-		attrs->ref_tree = ref_tree;
-	if (root_offset)
-		attrs->root_offset = root_offset;
-	if (flags)
-		attrs->flags = flags;
-	if (dir_id)
-		attrs->dir_id = dir_id;
-	if (gen)
-		attrs->gen = gen;
-	if (ogen)
-		attrs->ogen = ogen;
-	if (!attrs->ogen && root_offset)
-		attrs->ogen = root_offset;
-	if (otime)
-		attrs->otime = otime;
-	if (uuid)
-		memcpy(&attrs->uuid, uuid, BTRFS_UUID_SIZE);
-	if (puuid)
-		memcpy(&attrs->puuid, puuid, BTRFS_UUID_SIZE);
-	if (ruuid)
-		memcpy(&attrs->ruuid, ruuid, BTRFS_UUID_SIZE);
+	if (new->ref_tree)
+		found->ref_tree = new->ref_tree;
+	if (new->root_offset)
+		found->root_offset = new->root_offset;
+	if (new->flags)
+		found->flags = new->flags;
+	if (new->dir_id)
+		found->dir_id = new->dir_id;
+	if (new->gen)
+		found->gen = new->gen;
+	if (new->ogen)
+		found->ogen = new->ogen;
+	if (!found->ogen && new->root_offset)
+		found->ogen = new->root_offset;
+	if (new->otime)
+		found->otime = new->otime;
+	if (new->uuid[0])
+		memcpy(&found->uuid, new->uuid, BTRFS_UUID_SIZE);
+	if (new->puuid[0])
+		memcpy(&found->puuid, new->puuid, BTRFS_UUID_SIZE);
+	if (new->ruuid[0])
+		memcpy(&found->ruuid, new->ruuid, BTRFS_UUID_SIZE);
 
+}
+
+static int update_root(struct rb_root *root_lookup,
+		       struct root_attr const *new)
+{
+	struct root_info *ri;
+
+	ri = root_tree_search(root_lookup, new->root_id);
+	if (!ri || ri->attrs.root_id != new->root_id)
+		return -ENOENT;
+
+	update_root_attr(&ri->attrs, new);
 	return 0;
 }
 
@@ -591,18 +594,12 @@ static int update_root(struct rb_root *root_lookup,
  *	      into the lookup tree.
  * attrs: the description for the new root.
  */
-static int add_root(struct rb_root *root_lookup,
-		    u64 root_id, u64 ref_tree, u64 root_offset, u64 flags,
-		    u64 dir_id, char *name, int name_len, u64 ogen, u64 gen,
-		    time_t otime, u8 *uuid, u8 *puuid, u8 *ruuid)
+static int add_root(struct rb_root *root_lookup, const struct root_attr *new)
 {
 	struct root_info *ri;
-	struct root_attr *attrs;
 	int ret;
 
-	ret = update_root(root_lookup, root_id, ref_tree, root_offset, flags,
-			  dir_id, name, name_len, ogen, gen, otime,
-			  uuid, puuid, ruuid);
+	ret = update_root(root_lookup, new);
 	if (!ret)
 		return 0;
 
@@ -611,49 +608,14 @@ static int add_root(struct rb_root *root_lookup,
 		error_msg(ERROR_MSG_MEMORY, NULL);
 		exit(1);
 	}
-	attrs = &ri->attrs;
-	attrs->root_id = root_id;
+	ri->attrs.root_id = new->root_id;
 
-	if (name && name_len > 0) {
-		attrs->name = malloc(name_len + 1);
-		if (!attrs->name) {
-			error_msg(ERROR_MSG_MEMORY, NULL);
-			exit(1);
-		}
-		strncpy(attrs->name, name, name_len);
-		attrs->name[name_len] = 0;
-		attrs->name_len = name_len;
-	}
-	if (ref_tree)
-		attrs->ref_tree = ref_tree;
-	if (dir_id)
-		attrs->dir_id = dir_id;
-	if (root_offset)
-		attrs->root_offset = root_offset;
-	if (flags)
-		attrs->flags = flags;
-	if (gen)
-		attrs->gen = gen;
-	if (ogen)
-		attrs->ogen = ogen;
-	if (!attrs->ogen && root_offset)
-		attrs->ogen = root_offset;
-	if (otime)
-		attrs->otime = otime;
-
-	if (uuid)
-		memcpy(&attrs->uuid, uuid, BTRFS_UUID_SIZE);
-
-	if (puuid)
-		memcpy(&attrs->puuid, puuid, BTRFS_UUID_SIZE);
-
-	if (ruuid)
-		memcpy(&attrs->ruuid, ruuid, BTRFS_UUID_SIZE);
+	update_root_attr(&ri->attrs, new);
 
 	ret = root_tree_insert(root_lookup, ri);
 	if (ret < 0) {
 		errno = -ret;
-		error("failed to insert subvolume %llu to tree: %m", root_id);
+		error("failed to insert subvolume %llu to tree: %m", new->root_id);
 		exit(1);
 	}
 	return 0;
@@ -666,8 +628,15 @@ static int add_root(struct rb_root *root_lookup,
 static int add_root_backref(struct rb_root *root_lookup, u64 root_id,
 		u64 ref_tree, u64 dir_id, char *name, int name_len)
 {
-	return add_root(root_lookup, root_id, ref_tree, 0, 0, dir_id, name,
-			name_len, 0, 0, 0, NULL, NULL, NULL);
+	struct root_attr tmp = { 0 };
+
+	tmp.root_id = root_id;
+	tmp.ref_tree = ref_tree;
+	tmp.dir_id = dir_id;
+	tmp.name = name;
+	tmp.name_len = name_len;
+
+	return add_root(root_lookup, &tmp);
 }
 
 static void free_root_info(struct rb_node *node)
@@ -832,9 +801,6 @@ static int list_subvol_search(int fd, struct rb_root *root_lookup)
 	int name_len;
 	char *name;
 	u64 dir_id;
-	u64 gen = 0;
-	u64 ogen;
-	u64 flags;
 	int i;
 
 	root_lookup->rb_node = NULL;
@@ -878,39 +844,29 @@ static int list_subvol_search(int fd, struct rb_root *root_lookup)
 			} else if (sh.type == BTRFS_ROOT_ITEM_KEY &&
 				   (sh.objectid >= BTRFS_FIRST_FREE_OBJECTID ||
 				    sh.objectid == BTRFS_FS_TREE_OBJECTID)) {
-				time_t otime;
-				u8 uuid[BTRFS_UUID_SIZE];
-				u8 puuid[BTRFS_UUID_SIZE];
-				u8 ruuid[BTRFS_UUID_SIZE];
+				struct root_attr tmp = { 0 };
 
 				ri = (struct btrfs_root_item *)(args.buf + off);
-				gen = btrfs_root_generation(ri);
-				flags = btrfs_root_flags(ri);
+				tmp.root_id = sh.objectid;
+				tmp.root_offset = sh.offset;
+				tmp.gen = btrfs_root_generation(ri);
+				tmp.flags = btrfs_root_flags(ri);
 				if(sh.len >= sizeof(struct btrfs_root_item)) {
 					/*
 					 * The new full btrfs_root_item with
 					 * timestamp and UUID.
+					 *
+					 * For older v0 root item, those info is
+					 * not in root_item, and would remain 0.
 					 */
-					otime = btrfs_stack_timespec_sec(&ri->otime);
-					ogen = btrfs_root_otransid(ri);
-					memcpy(uuid, ri->uuid, BTRFS_UUID_SIZE);
-					memcpy(puuid, ri->parent_uuid, BTRFS_UUID_SIZE);
-					memcpy(ruuid, ri->received_uuid, BTRFS_UUID_SIZE);
-				} else {
-					/*
-					 * The old v0 root item, which doesn't
-					 * have timestamp nor UUID.
-					 */
-					otime = 0;
-					ogen = 0;
-					memset(uuid, 0, BTRFS_UUID_SIZE);
-					memset(puuid, 0, BTRFS_UUID_SIZE);
-					memset(ruuid, 0, BTRFS_UUID_SIZE);
+					tmp.otime = btrfs_stack_timespec_sec(&ri->otime);
+					tmp.ogen = btrfs_root_otransid(ri);
+					memcpy(tmp.uuid, ri->uuid, BTRFS_UUID_SIZE);
+					memcpy(tmp.puuid, ri->parent_uuid, BTRFS_UUID_SIZE);
+					memcpy(tmp.ruuid, ri->received_uuid, BTRFS_UUID_SIZE);
 				}
 
-				add_root(root_lookup, sh.objectid, 0,
-					 sh.offset, flags, 0, NULL, 0, ogen,
-					 gen, otime, uuid, puuid, ruuid);
+				add_root(root_lookup, &tmp);
 			}
 
 			off += sh.len;
Signature · 1 line
2.42.1
author
#3 lore raw

PATCH 3/3 btrfs-progs: subvolume-list: output qgroup sizes for subvolumes

In reply to Qu Wenruo

Inspired by ZFS zpool list, which would output the sizes of each subvolume, we can do better for out "btrfs subvolume list" UI/UX.

This patch would introduce the auto-detection and auto-output using qgroup sizes for subvolumes.

The output would look like this:

 # ./btrfs subv list -t /mnt/btrfs/
 ID	gen	top level	rfer	excl	path
 --	---	---------	----	----	----
 256	11	5		1064960	1064960	subvol1
 257	11	5		4210688	4210688	subvol2

Unfortunately there would be some pitfalls:

- No output for subvolume 5 (fs tree)
As we do not output subvolume 5 (fs tree) at all, thus if the end user wants the size of fs tree, they would still be upset.

- If qgroup is not enabled, the sizes would be omitted This may lead to different outputs for different use cases and can lead to some confusion.

- Over simplified column name
As a developer get too used to qgroup, I don't have any better names for the sizes right now.

- Table output can easily be screwed up by the larger sizes This requires some update to the table output to know the possible longest values to adjust.
Which is definitely worthy some other patchsets to address.

---
 Documentation/btrfs-subvolume.rst |  12 ++-
 cmds/subvolume-list.c             | 140 +++++++++++++++++++++++++++++-
 2 files changed, 150 insertions(+), 2 deletions(-)
Documentation/btrfs-subvolume.rst +11 -1
index eb116c4bdc95..8eaab5872d70 100644
@@ -130,7 +130,7 @@ list [options] [-G [\+|-]<value>] [-C [+|-]<value>] [--sort=rootid,gen,ogen,path
 
         For every subvolume the following information is shown by default:
 
-        ID *ID* gen *generation* top level *parent_ID* path *path*
+        ID *ID* gen *generation* top level *parent_ID* [size (referenced) *rfer* size (exclusive) *excl*] path *path*
 
         where *ID* is subvolume's (root)id, *generation* is an internal counter which is
         updated every transaction, *parent_ID* is the same as the parent subvolume's id,
@@ -138,6 +138,11 @@ list [options] [-G [\+|-]<value>] [-C [+|-]<value>] [--sort=rootid,gen,ogen,path
         The subvolume's ID may be used by the subvolume set-default command,
         or at mount time via the *subvolid=* option.
 
+	The sizes of a subvolume will only be outputted if qgroup is enabled,
+	as that's the only way to get accurate size of a subvolume.
+	The referenced size is the total bytes of every extent referred by that subvolume.
+	While the exclusive size is total bytes that is exclusive to that subvolume.
+
         ``Options``
 
         Path filtering:
@@ -163,6 +168,11 @@ list [options] [-G [\+|-]<value>] [-C [+|-]<value>] [--sort=rootid,gen,ogen,path
         -q
                 print the parent UUID of the subvolume
                 (*parent* here means subvolume of which this subvolume is a snapshot).
+
+	-Q
+		always print the qgroup sizes of the subvolume
+		If qgroup is not enabled, a warning would be outputted and all
+		qgroup sizes would be zero.
         -R
                 print the UUID of the sent subvolume, where the subvolume is the result of a receive operation.
 
cmds/subvolume-list.c +139 -1

229 lines not on this page · raw message

Signature · 1 line
2.42.1
#4 lore raw
In reply to Qu Wenruo
Quoted 3 lines · Qu Wenruo

ZFS' management tool is way better received than btrfs-progs, one of the user-friendly point is the default zpool list, which includes the size of each subvolume.

The output of 'subvol list' needs a rework, it's from the early times. Unfortunatelly lots of tools depend on the output format so it's not easy to change it.

There's a WIP https://github.com/kdave/btrfs-progs/issues/515 with outlined problems and proposed solutions.

To work around the compatibility problem and to bring a nice UI I've proposed to create a completely new command and shamelessly copying what
https://github.com/speed47/btrfs-list does. Instead of adding new features to current 'list' incrementally.

Adding the qgroups column is probably ok, I'll take another look.

author
#5 lore raw
In reply to Qu Wenruo

A gentle ping?

Any feedback on the new columns?

Thanks,
Qu

Quoted 43 lines · Qu Wenruo

ZFS' management tool is way better received than btrfs-progs, one of the user-friendly point is the default zpool list, which includes the size of each subvolume.

I'm not sure how ZFS handles it, but for btrfs we need qgroups (or the faster but slightly less accurate squota) to get the accurate numbers.

But considering a lot of distro is enabling qgroup by default for exactly the same reason, and during the years qgroup itself is also under a lot of optimization, I believe adding sizes output for btrfs subvolume list is an overall benefit for end uesrs.

This patch would do exactly so, the output example is:

  # ./btrfs subv list -t /mnt/btrfs/
  ID	gen	top level	rfer	excl	path
  --	---	---------	----	----	----
  256	11	5		1064960	1064960	subvol1
  257	11	5		4210688	4210688	subvol2

The extra columns are added depending on if qgroup is enabled, and we allow users to force such output, but if qgroup is not enabled and we're forced to output such sizes, a warning would be outputted and fill all the sizes value as 0.

Thanks William Brown for the UI suggestion.

Although there are still some pitfalls, mentioned in the last patch.

Qu Wenruo (3):
btrfs-progs: separate root attributes into a dedicated structure from
root_info
btrfs-progs: use root_attr structure to pass various attributes
btrfs-progs: subvolume-list: output qgroup sizes for subvolumes

  Documentation/btrfs-subvolume.rst |  12 +-
  cmds/subvolume-list.c             | 572 ++++++++++++++++++------------
  2 files changed, 349 insertions(+), 235 deletions(-)

--
2.42.1