Skip to contentSkip to conversation

btrfs-progs: simple quotas

PATCH Superseded Boris Burkov 2 participants 11 messages Sep 27, 2023 – Oct 2, 2023

Subsystems (3)

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover btrfs-progs: simple quotas

btrfs-progs changes for supporting simple quotas. Notably:
btrfs quota can enable squota via ioctl
mkfs can create an fs with squota enabled btrfstune can enable squota
btrfs inspect commands dump squota fields
btrfs check validates and repairs squota invariants

---
Changelog:
v3:
* rebase
* update enum values to match kernel changes
v2:
* fixed messed up list numbering in doc
* fixed broken formatting
* used new command in enable ioctl instead of status value
* introduced new qgroup status flag to mkfs/tune

Boris Burkov (8):
btrfs-progs: document squotas
btrfs-progs: simple quotas kernel definitions
btrfs-progs: simple quotas dump commands
btrfs-progs: simple quotas fsck
btrfs-progs: simple quotas mkfs
btrfs-progs: simple quotas btrfstune
btrfs-progs: simple quotas enable cmd
btrfs-progs: tree-checker: handle owner ref items

 Documentation/btrfs-quota.rst    |   7 +-
 Documentation/ch-quota-intro.rst |  59 +++++++++++
 Documentation/mkfs.btrfs.rst     |   6 ++
 Makefile                         |   2 +-
 check/main.c                     |   2 +
 check/qgroup-verify.c            |  86 +++++++++++++++-
 cmds/quota.c                     |  39 +++++--
 common/fsfeatures.c              |   9 ++
 kernel-shared/accessors.h        |   9 ++
 kernel-shared/ctree.h            |   6 +-
 kernel-shared/print-tree.c       |  27 ++++-
 kernel-shared/tree-checker.c     |   2 +
 kernel-shared/uapi/btrfs.h       |   3 +
 kernel-shared/uapi/btrfs_tree.h  |  16 ++-
 mkfs/main.c                      |  64 ++++++++++--
 tune/main.c                      |  18 +++-
 tune/quota.c                     | 172 +++++++++++++++++++++++++++++++
 tune/tune.h                      |   3 +
 18 files changed, 504 insertions(+), 26 deletions(-)
 create mode 100644 tune/quota.c
Signature · 1 line
2.42.0
#1 lore raw

PATCH v3 1/8 btrfs-progs: document squotas

Document the new options in btrfs quota and mkfs.btrfs. Also, add a section to the long form qgroups document about squotas.

---
 Documentation/btrfs-quota.rst    |  7 +++-
 Documentation/ch-quota-intro.rst | 59 ++++++++++++++++++++++++++++++++
 Documentation/mkfs.btrfs.rst     |  6 ++++
 3 files changed, 71 insertions(+), 1 deletion(-)
Documentation/btrfs-quota.rst +6 -1
index 04d53ef53..2103130fd 100644
@@ -47,9 +47,14 @@ SUBCOMMAND
 disable <path>
         Disable subvolume quota support for a filesystem.
 
-enable <path>
+enable [options] <path>
         Enable subvolume quota support for a filesystem.
 
+        ``Options``
+
+	-s|--simple
+		use simple quotas (squotas) instead of qgroups
+
 rescan [options] <path>
         Trash all qgroup numbers and scan the metadata again with the current config.
 
Documentation/ch-quota-intro.rst +59 -0

62 lines not on this page · raw message

Documentation/mkfs.btrfs.rst +6 -0
index 1fca7448b..da8e708ae 100644
@@ -326,6 +326,12 @@ block-group-tree
         enabled at *mkfs* time is possible, see :doc:`btrfstune`. Online
         conversion is not possible.
 
+squota
+	(kernel support since X.X)
+
+	Enable simple quota support (squotas). This is an alternative to qgroups with
+	a smaller performance impact but no notion of shared vs. exclusive usage.
+
 .. _mkfs-section-profiles:
 
 BLOCK GROUPS, CHUNKS, RAID
Signature · 1 line
2.42.0
#2 lore raw

PATCH v3 2/8 btrfs-progs: simple quotas kernel definitions

In reply to Boris Burkov

Copy over structs, accessors, and constants for simple quotas

---
 kernel-shared/accessors.h       |  9 +++++++++
 kernel-shared/ctree.h           |  6 ++++--
 kernel-shared/uapi/btrfs.h      |  1 +
 kernel-shared/uapi/btrfs_tree.h | 16 +++++++++++++++-
 4 files changed, 29 insertions(+), 3 deletions(-)
kernel-shared/accessors.h +9 -0
index 539c20d09..ab8c2d337 100644
@@ -379,9 +379,13 @@ static inline u32 btrfs_extent_inline_ref_size(int type)
 	if (type == BTRFS_EXTENT_DATA_REF_KEY)
 		return sizeof(struct btrfs_extent_data_ref) +
 		       offsetof(struct btrfs_extent_inline_ref, offset);
+	if (type == BTRFS_EXTENT_OWNER_REF_KEY)
+		return sizeof(struct btrfs_extent_inline_ref);
 	return 0;
 }
 
+BTRFS_SETGET_FUNCS(extent_owner_ref_root_id, struct btrfs_extent_owner_ref, root_id, 64);
+
 /* struct btrfs_node */
 BTRFS_SETGET_FUNCS(key_blockptr, struct btrfs_key_ptr, blockptr, 64);
 BTRFS_SETGET_FUNCS(key_generation, struct btrfs_key_ptr, generation, 64);
@@ -979,6 +983,9 @@ BTRFS_SETGET_FUNCS(qgroup_status_flags, struct btrfs_qgroup_status_item,
 		   flags, 64);
 BTRFS_SETGET_FUNCS(qgroup_status_rescan, struct btrfs_qgroup_status_item,
 		   rescan, 64);
+BTRFS_SETGET_FUNCS(qgroup_status_enable_gen, struct btrfs_qgroup_status_item,
+		   enable_gen, 64);
+
 BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_generation,
 			 struct btrfs_qgroup_status_item, generation, 64);
 BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_version,
@@ -987,6 +994,8 @@ BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_flags,
 			 struct btrfs_qgroup_status_item, flags, 64);
 BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_rescan,
 			 struct btrfs_qgroup_status_item, rescan, 64);
+BTRFS_SETGET_STACK_FUNCS(stack_qgroup_status_enable_gen,
+			 struct btrfs_qgroup_status_item, enable_gen, 64);
 
 /* btrfs_qgroup_info_item */
 BTRFS_SETGET_FUNCS(qgroup_info_generation, struct btrfs_qgroup_info_item,
kernel-shared/ctree.h +4 -2
index 59533879b..bdfebd665 100644
@@ -103,7 +103,8 @@ static inline u32 __BTRFS_LEAF_DATA_SIZE(u32 nodesize)
 	 BTRFS_FEATURE_INCOMPAT_RAID1C34 |		\
 	 BTRFS_FEATURE_INCOMPAT_METADATA_UUID |		\
 	 BTRFS_FEATURE_INCOMPAT_ZONED |			\
-	 BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2)
+	 BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2 | \
+	 BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA)
 #else
 #define BTRFS_FEATURE_INCOMPAT_SUPP			\
 	(BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF |		\
@@ -118,7 +119,8 @@ static inline u32 __BTRFS_LEAF_DATA_SIZE(u32 nodesize)
 	 BTRFS_FEATURE_INCOMPAT_NO_HOLES |		\
 	 BTRFS_FEATURE_INCOMPAT_RAID1C34 |		\
 	 BTRFS_FEATURE_INCOMPAT_METADATA_UUID |		\
-	 BTRFS_FEATURE_INCOMPAT_ZONED)
+	 BTRFS_FEATURE_INCOMPAT_ZONED | \
+	 BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA)
 #endif
 
 /*
kernel-shared/uapi/btrfs.h +1 -0
index 85b04f89a..7e0078a5d 100644
@@ -356,6 +356,7 @@ _static_assert(sizeof(struct btrfs_ioctl_fs_info_args) == 1024);
 #define BTRFS_FEATURE_INCOMPAT_RAID1C34		(1ULL << 11)
 #define BTRFS_FEATURE_INCOMPAT_ZONED		(1ULL << 12)
 #define BTRFS_FEATURE_INCOMPAT_EXTENT_TREE_V2	(1ULL << 13)
+#define BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA	(1ULL << 16)
 
 struct btrfs_ioctl_feature_flags {
 	__u64 compat_flags;
kernel-shared/uapi/btrfs_tree.h +15 -1
index ad555e705..29cf2f3d3 100644
@@ -227,6 +227,8 @@
 
 #define BTRFS_SHARED_DATA_REF_KEY	184
 
+#define BTRFS_EXTENT_OWNER_REF_KEY	188
+
 /*
  * block groups give us hints into the extent allocation trees.  Which
  * blocks are free etc etc
@@ -783,6 +785,10 @@ struct btrfs_shared_data_ref {
 	__le32 count;
 } __attribute__ ((__packed__));
 
+struct btrfs_extent_owner_ref {
+	__le64 root_id;
+} __attribute__ ((__packed__));
+
 struct btrfs_extent_inline_ref {
 	__u8 type;
 	__le64 offset;
@@ -1199,10 +1205,12 @@ static inline __u16 btrfs_qgroup_level(__u64 qgroupid)
  * Turning qouta off and on again makes it inconsistent, too.
  */
 #define BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT	(1ULL << 2)
+#define BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE	(1ULL << 3)
 
 #define BTRFS_QGROUP_STATUS_FLAGS_MASK	(BTRFS_QGROUP_STATUS_FLAG_ON |		\
 					 BTRFS_QGROUP_STATUS_FLAG_RESCAN |	\
-					 BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT)
+					 BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT |	\
+					 BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE)
 
 #define BTRFS_QGROUP_STATUS_VERSION        1
 
@@ -1224,6 +1232,12 @@ struct btrfs_qgroup_status_item {
 	 * of the scan. It contains a logical address
 	 */
 	__le64 rescan;
+
+	/*
+	 * Used by simple quotas to ignore old extent deletions
+	 * Present iff incompat flag SIMPLE_QUOTA is set
+	 */
+	__le64 enable_gen;
 } __attribute__ ((__packed__));
 
 struct btrfs_qgroup_info_item {
Signature · 1 line
2.42.0
#3 lore raw

PATCH v3 3/8 btrfs-progs: simple quotas dump commands

In reply to Boris Burkov

Add support to btrfs inspect-internal dump-super and dump-tree for the new structures and feature flags introduced by simple quotas

---
 kernel-shared/print-tree.c | 27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)
kernel-shared/print-tree.c +26 -1
index d7ffeccd1..666664868 100644
@@ -509,6 +509,10 @@ void print_extent_item(struct extent_buffer *eb, int slot, int metadata)
 			       (unsigned long long)offset,
 			       btrfs_shared_data_ref_count(eb, sref));
 			break;
+		case BTRFS_EXTENT_OWNER_REF_KEY:
+			printf("\t\textent owner root %llu\n",
+			       (unsigned long long)offset);
+			break;
 		default:
 			return;
 		}
@@ -661,6 +665,7 @@ void print_key_type(FILE *stream, u64 objectid, u8 type)
 		[BTRFS_EXTENT_DATA_REF_KEY]	= "EXTENT_DATA_REF",
 		[BTRFS_SHARED_DATA_REF_KEY]	= "SHARED_DATA_REF",
 		[BTRFS_EXTENT_REF_V0_KEY]	= "EXTENT_REF_V0",
+		[BTRFS_EXTENT_OWNER_REF_KEY]	= "EXTENT_OWNER_REF",
 		[BTRFS_CSUM_ITEM_KEY]		= "CSUM_ITEM",
 		[BTRFS_EXTENT_CSUM_KEY]		= "EXTENT_CSUM",
 		[BTRFS_EXTENT_DATA_KEY]		= "EXTENT_DATA",
@@ -1042,6 +1047,17 @@ static void print_shared_data_ref(struct extent_buffer *eb, int slot)
 		btrfs_shared_data_ref_count(eb, sref));
 }
 
+static void print_extent_owner_ref(struct extent_buffer *eb, int slot)
+{
+	struct btrfs_extent_owner_ref *oref;
+	u64 root_id;
+
+	oref = btrfs_item_ptr(eb, slot, struct btrfs_extent_owner_ref);
+	root_id = btrfs_extent_owner_ref_root_id(eb, oref);
+
+	printf("\t\textent owner root %llu\n", root_id);
+}
+
 static void print_free_space_info(struct extent_buffer *eb, int slot)
 {
 	struct btrfs_free_space_info *free_info;
@@ -1083,11 +1099,16 @@ static void print_qgroup_status(struct extent_buffer *eb, int slot)
 	memset(flags_str, 0, sizeof(flags_str));
 	qgroup_flags_to_str(btrfs_qgroup_status_flags(eb, qg_status),
 					flags_str);
-	printf("\t\tversion %llu generation %llu flags %s scan %llu\n",
+	printf("\t\tversion %llu generation %llu flags %s scan %llu",
 		(unsigned long long)btrfs_qgroup_status_version(eb, qg_status),
 		(unsigned long long)btrfs_qgroup_status_generation(eb, qg_status),
 		flags_str,
 		(unsigned long long)btrfs_qgroup_status_rescan(eb, qg_status));
+	if (btrfs_fs_incompat(eb->fs_info, SIMPLE_QUOTA))
+		printf(" enable_gen %llu\n",
+			   (unsigned long long)btrfs_qgroup_status_enable_gen(eb, qg_status));
+	else
+		printf("\n");
 }
 
 static void print_qgroup_info(struct extent_buffer *eb, int slot)
@@ -1407,6 +1428,9 @@ void btrfs_print_leaf(struct extent_buffer *eb, unsigned int mode)
 		case BTRFS_SHARED_DATA_REF_KEY:
 			print_shared_data_ref(eb, i);
 			break;
+		case BTRFS_EXTENT_OWNER_REF_KEY:
+			print_extent_owner_ref(eb, i);
+			break;
 		case BTRFS_EXTENT_REF_V0_KEY:
 			printf("\t\textent ref v0 (deprecated)\n");
 			break;
@@ -1708,6 +1732,7 @@ static struct readable_flag_entry incompat_flags_array[] = {
 	DEF_INCOMPAT_FLAG_ENTRY(RAID1C34),
 	DEF_INCOMPAT_FLAG_ENTRY(ZONED),
 	DEF_INCOMPAT_FLAG_ENTRY(EXTENT_TREE_V2),
+	DEF_INCOMPAT_FLAG_ENTRY(SIMPLE_QUOTA),
 };
 static const int incompat_flags_num = sizeof(incompat_flags_array) /
 				      sizeof(struct readable_flag_entry);
Signature · 1 line
2.42.0
#4 lore raw

PATCH v3 4/8 btrfs-progs: simple quotas fsck

In reply to Boris Burkov

Add simple quotas checks to btrfs check.

Like the kernel feature, these checks bypass most of the backref walking in the qgroups check. Instead, they enforce the invariant behind the design of simple quotas by scanning the extent tree and determining the owner of each extent:
Data: reading the owner ref inline item
Metadata: reading the tree block and reading its btrfs_header's owner

This gives us the expected count from squotas which we check against the on-disk state of the qgroup items.

---
 check/main.c          |  2 ++
 check/qgroup-verify.c | 79 +++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 78 insertions(+), 3 deletions(-)
check/main.c +2 -0
index d387eb25d..c3cfd199e 100644
@@ -5674,6 +5674,8 @@ static int process_extent_item(struct btrfs_root *root,
 					btrfs_shared_data_ref_count(eb, sref),
 					gen, 0, num_bytes);
 			break;
+		case BTRFS_EXTENT_OWNER_REF_KEY:
+			break;
 		default:
 			fprintf(stderr,
 				"corrupt extent record: key [%llu,%u,%llu]\n",
check/qgroup-verify.c +76 -3

151 lines not on this page · raw message

Signature · 1 line
2.42.0
#5 lore raw

PATCH v3 5/8 btrfs-progs: simple quotas mkfs

In reply to Boris Burkov

Add the ability to enable simple quotas from mkfs with '-O squota'

There is some complication around handling enable gen while still counting the root node of an fs. To handle this, employ a hack of doing a no-op write on the root node to bump its generation up above that of the qgroup enable generation, which results in counting it properly.

---
 check/qgroup-verify.c |  7 +++--
 common/fsfeatures.c   |  9 ++++++
 mkfs/main.c           | 64 +++++++++++++++++++++++++++++++++++++++----
 3 files changed, 72 insertions(+), 8 deletions(-)
check/qgroup-verify.c +5 -2
index c95e6f806..1cf01d2d0 100644
@@ -1695,6 +1695,8 @@ static int repair_qgroup_status(struct btrfs_fs_info *info, bool silent)
 	struct btrfs_path path;
 	struct btrfs_key key;
 	struct btrfs_qgroup_status_item *status_item;
+	bool simple = btrfs_fs_incompat(info, SIMPLE_QUOTA);
+	int flags = BTRFS_QGROUP_STATUS_FLAG_ON;
 
 	if (!silent)
 		printf("Repair qgroup status item\n");
@@ -1717,8 +1719,9 @@ static int repair_qgroup_status(struct btrfs_fs_info *info, bool silent)
 
 	status_item = btrfs_item_ptr(path.nodes[0], path.slots[0],
 				     struct btrfs_qgroup_status_item);
-	btrfs_set_qgroup_status_flags(path.nodes[0], status_item,
-				      BTRFS_QGROUP_STATUS_FLAG_ON);
+	if (simple)
+		flags |= BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE;
+	btrfs_set_qgroup_status_flags(path.nodes[0], status_item, flags);
 	btrfs_set_qgroup_status_rescan(path.nodes[0], status_item, 0);
 	btrfs_set_qgroup_status_generation(path.nodes[0], status_item,
 					   trans->transid);
common/fsfeatures.c +9 -0
index 1c993b594..6bf87937b 100644
@@ -109,6 +109,15 @@ static const struct btrfs_feature mkfs_features[] = {
 		VERSION_NULL(default),
 		.desc		= "quota support (qgroups)"
 	},
+	{
+		.name		= "squota",
+		.incompat_flag	= BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA,
+		.sysfs_name	= "squota",
+		VERSION_TO_STRING2(compat, 6,5),
+		VERSION_NULL(safe),
+		VERSION_NULL(default),
+		.desc		= "squota support (simple qgroups)"
+	},
 	{
 		.name		= "extref",
 		.incompat_flag	= BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF,
mkfs/main.c +58 -6

104 lines not on this page · raw message

Signature · 1 line
2.42.0
#7 lore raw

PATCH v3 6/8 btrfs-progs: simple quotas btrfstune

In reply to Boris Burkov

Add the ability to enable simple quotas on an existing file system at rest with btrfstune.

This is similar to the functionality in mkfs, except it must also find all the roots for which it must create qgroups. Note that this *does not* retroactively compute usage for existing extents as that is impossible for data. This is consistent with the behavior of the live enable ioctl.

---
 Makefile     |   2 +-
 tune/main.c  |  18 +++++-
 tune/quota.c | 172 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tune/tune.h  |   3 +
 4 files changed, 193 insertions(+), 2 deletions(-)
 create mode 100644 tune/quota.c
Makefile +1 -1
index 5dfd3c01d..7419475dd 100644
@@ -265,7 +265,7 @@ mkfs_objects = mkfs/main.o mkfs/common.o mkfs/rootdir.o
 image_objects = image/main.o image/sanitize.o image/image-create.o image/common.o \
 		image/image-restore.o
 tune_objects = tune/main.o tune/seeding.o tune/change-uuid.o tune/change-metadata-uuid.o \
-	       tune/convert-bgt.o tune/change-csum.o check/clear-cache.o
+	       tune/convert-bgt.o tune/change-csum.o tune/quota.o check/clear-cache.o
 all_objects = $(objects) $(cmds_objects) $(libbtrfs_objects) $(convert_objects) \
 	      $(mkfs_objects) $(image_objects) $(tune_objects) $(libbtrfsutil_objects)
 
tune/main.c +17 -1
index 63539b991..895e4da03 100644
@@ -103,6 +103,7 @@ static const char * const tune_usage[] = {
 	OPTLINE("-x", "enable skinny metadata extent refs (mkfs: skinny-metadata)"),
 	OPTLINE("-n", "enable no-holes feature (mkfs: no-holes, more efficient sparse file representation)"),
 	OPTLINE("-S <0|1>", "set/unset seeding status of a device"),
+	OPTLINE("-q", "enable simple quotas on the file system. (mkfs: squota)"),
 	OPTLINE("--convert-to-block-group-tree", "convert filesystem to track block groups in "
 			"the separate block-group-tree instead of extent tree (sets the incompat bit)"),
 	OPTLINE("--convert-from-block-group-tree",
@@ -153,6 +154,9 @@ enum btrfstune_group_enum {
 	 */
 	LEGACY,
 
+	/* Qgroup options */
+	QGROUP,
+
 	BTRFSTUNE_NR_GROUPS,
 };
 
@@ -189,6 +193,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 	char *new_fsid_str = NULL;
 	int ret;
 	u64 super_flags = 0;
+	int quota = 0;
 	int fd = -1;
 
 	btrfs_config_init();
@@ -211,7 +216,7 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 #endif
 			{ NULL, 0, NULL, 0 }
 		};
-		int c = getopt_long(argc, argv, "S:rxfuU:nmM:", long_options, NULL);
+		int c = getopt_long(argc, argv, "S:rxqfuU:nmM:", long_options, NULL);
 
 		if (c < 0)
 			break;
@@ -229,6 +234,10 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 			super_flags |= BTRFS_FEATURE_INCOMPAT_SKINNY_METADATA;
 			btrfstune_cmd_groups[LEGACY] = true;
 			break;
+		case 'q':
+			quota = 1;
+			btrfstune_cmd_groups[QGROUP] = true;
+			break;
 		case 'n':
 			super_flags |= BTRFS_FEATURE_INCOMPAT_NO_HOLES;
 			btrfstune_cmd_groups[LEGACY] = true;
@@ -499,6 +508,13 @@ int BOX_MAIN(btrfstune)(int argc, char *argv[])
 		ret = change_uuid(fs_info, new_fsid_str);
 		goto out;
 	}
+
+	if (quota) {
+		ret = enable_quota(root->fs_info, true);
+		if (ret)
+			goto out;
+	}
+
 out:
 	if (ret < 0) {
 		fs_info->readonly = 1;
tune/quota.c new file +172 -0

172 lines not on this page · raw message

tune/tune.h +3 -0
index 0ef249d89..cbf33b2e7 100644
@@ -33,4 +33,7 @@ int convert_to_bg_tree(struct btrfs_fs_info *fs_info);
 int convert_to_extent_tree(struct btrfs_fs_info *fs_info);
 
 int btrfs_change_csum_type(struct btrfs_fs_info *fs_info, u16 new_csum_type);
+
+int enable_quota(struct btrfs_fs_info *fs_info, bool simple);
+
 #endif
Signature · 1 line
2.42.0
#8 lore raw

PATCH v3 7/8 btrfs-progs: simple quotas enable cmd

In reply to Boris Burkov

Add a --simple flag to btrfs quota enable. If set, this enables simple quotas instead of full qgroups by using the new ioctl command value.

---
 cmds/quota.c               | 39 ++++++++++++++++++++++++++++++--------
 kernel-shared/uapi/btrfs.h |  2 ++
 2 files changed, 33 insertions(+), 8 deletions(-)
cmds/quota.c +31 -8
index cd874f9ed..a7d032a03 100644
@@ -34,17 +34,13 @@ static const char * const quota_cmd_group_usage[] = {
 	NULL
 };
 
-static int quota_ctl(int cmd, int argc, char **argv)
+static int quota_ctl(int cmd, char *path)
 {
 	int ret = 0;
 	int fd;
-	char *path = argv[1];
 	struct btrfs_ioctl_quota_ctl_args args;
 	DIR *dirstream = NULL;
 
-	if (check_argc_exact(argc, 2))
-		return -1;
-
 	memset(&args, 0, sizeof(args));
 	args.cmd = cmd;
 
@@ -67,16 +63,40 @@ static const char * const cmd_quota_enable_usage[] = {
 	"Any data already present on the filesystem will not count towards",
 	"the space usage numbers. It is recommended to enable quota for a",
 	"filesystem before writing any data to it.",
+	"",
+	"-s|--simple	simple qgroups account ownership by extent lifetime rather than backref walks",
 	NULL
 };
 
 static int cmd_quota_enable(const struct cmd_struct *cmd, int argc, char **argv)
 {
 	int ret;
+	int ctl_cmd = BTRFS_QUOTA_CTL_ENABLE;
 
-	clean_args_no_options(cmd, argc, argv);
+	optind = 0;
+	while (1) {
+		static const struct option long_options[] = {
+			{"simple", no_argument, NULL, 's'},
+			{NULL, 0, NULL, 0}
+		};
+		int c;
 
-	ret = quota_ctl(BTRFS_QUOTA_CTL_ENABLE, argc, argv);
+		c = getopt_long(argc, argv, "s", long_options, NULL);
+		if (c < 0)
+			break;
+
+		switch (c) {
+		case 's':
+			ctl_cmd = BTRFS_QUOTA_CTL_ENABLE_SIMPLE_QUOTA;
+			break;
+		default:
+			usage_unknown_option(cmd, argv);
+		}
+	}
+	if (check_argc_exact(argc - optind, 1))
+		return -1;
+
+	ret = quota_ctl(ctl_cmd, argv[optind]);
 
 	if (ret < 0)
 		usage(cmd, 1);
@@ -97,7 +117,10 @@ static int cmd_quota_disable(const struct cmd_struct *cmd,
 
 	clean_args_no_options(cmd, argc, argv);
 
-	ret = quota_ctl(BTRFS_QUOTA_CTL_DISABLE, argc, argv);
+	if (check_argc_exact(argc, 2))
+		return -1;
+
+	ret = quota_ctl(BTRFS_QUOTA_CTL_DISABLE, argv[1]);
 
 	if (ret < 0)
 		usage(cmd, 1);
kernel-shared/uapi/btrfs.h +2 -0
index 7e0078a5d..11ffd54d4 100644
@@ -786,9 +786,11 @@ struct btrfs_ioctl_get_dev_stats {
 };
 _static_assert(sizeof(struct btrfs_ioctl_get_dev_stats) == 1032);
 
+/* cmd values */
 #define BTRFS_QUOTA_CTL_ENABLE	1
 #define BTRFS_QUOTA_CTL_DISABLE	2
 #define BTRFS_QUOTA_CTL_RESCAN__NOTUSED	3
+#define BTRFS_QUOTA_CTL_ENABLE_SIMPLE_QUOTA 4
 struct btrfs_ioctl_quota_ctl_args {
 	__u64 cmd;
 	__u64 status;
Signature · 1 line
2.42.0
#9 lore raw

PATCH v3 8/8 btrfs-progs: tree-checker: handle owner ref items

In reply to Boris Burkov

Add the new OWNER_REF inline items to the tree-checker extent item checking code. We could somehow validate the root id for being a valid fstree id, but just skipping it seems fine as well.

---
 kernel-shared/tree-checker.c | 2 ++
 1 file changed, 2 insertions(+)
kernel-shared/tree-checker.c +2 -0
index 107975891..2f834cf33 100644
@@ -1477,6 +1477,8 @@ static int check_extent_item(struct extent_buffer *leaf,
 			}
 			inline_refs += btrfs_shared_data_ref_count(leaf, sref);
 			break;
+		case BTRFS_EXTENT_OWNER_REF_KEY:
+			break;
 		default:
 			extent_err(leaf, slot, "unknown inline ref type: %u",
 				   inline_type);
Signature · 1 line
2.42.0
#6 lore raw

Re: [PATCH v3 5/8] btrfs-progs: simple quotas mkfs

In reply to Boris Burkov
Quoted 91 lines · Boris Burkov

Add the ability to enable simple quotas from mkfs with '-O squota'

There is some complication around handling enable gen while still counting the root node of an fs. To handle this, employ a hack of doing a no-op write on the root node to bump its generation up above that of the qgroup enable generation, which results in counting it properly.

---
 check/qgroup-verify.c |  7 +++--
 common/fsfeatures.c   |  9 ++++++
 mkfs/main.c           | 64 +++++++++++++++++++++++++++++++++++++++----
 3 files changed, 72 insertions(+), 8 deletions(-)
check/qgroup-verify.c +5 -2
index c95e6f806..1cf01d2d0 100644
@@ -1695,6 +1695,8 @@ static int repair_qgroup_status(struct btrfs_fs_info *info, bool silent)
 	struct btrfs_path path;
 	struct btrfs_key key;
 	struct btrfs_qgroup_status_item *status_item;
+	bool simple = btrfs_fs_incompat(info, SIMPLE_QUOTA);
+	int flags = BTRFS_QGROUP_STATUS_FLAG_ON;
 
 	if (!silent)
 		printf("Repair qgroup status item\n");
@@ -1717,8 +1719,9 @@ static int repair_qgroup_status(struct btrfs_fs_info *info, bool silent)
 
 	status_item = btrfs_item_ptr(path.nodes[0], path.slots[0],
 				     struct btrfs_qgroup_status_item);
-	btrfs_set_qgroup_status_flags(path.nodes[0], status_item,
-				      BTRFS_QGROUP_STATUS_FLAG_ON);
+	if (simple)
+		flags |= BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE;
+	btrfs_set_qgroup_status_flags(path.nodes[0], status_item, flags);
 	btrfs_set_qgroup_status_rescan(path.nodes[0], status_item, 0);
 	btrfs_set_qgroup_status_generation(path.nodes[0], status_item,
 					   trans->transid);
common/fsfeatures.c +9 -0
index 1c993b594..6bf87937b 100644
@@ -109,6 +109,15 @@ static const struct btrfs_feature mkfs_features[] = {
 		VERSION_NULL(default),
 		.desc		= "quota support (qgroups)"
 	},
+	{
+		.name		= "squota",
+		.incompat_flag	= BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA,
+		.sysfs_name	= "squota",
+		VERSION_TO_STRING2(compat, 6,5),
+		VERSION_NULL(safe),
+		VERSION_NULL(default),
+		.desc		= "squota support (simple qgroups)"
+	},
 	{
 		.name		= "extref",
 		.incompat_flag	= BTRFS_FEATURE_INCOMPAT_EXTENDED_IREF,
mkfs/main.c +16 -0
index 1c5d668e1..5bb4d1a21 100644
@@ -59,6 +59,8 @@
 #include "mkfs/common.h"
 #include "mkfs/rootdir.h"
 
+#include "libbtrfs/ctree.h"
+
 struct mkfs_allocation {
 	u64 data;
 	u64 metadata;
@@ -882,6 +884,37 @@ static int insert_qgroup_items(struct btrfs_trans_handle *trans,
 	return ret;
 }
 
+static int touch_root_subvol(struct btrfs_fs_info *fs_info)
+{
+	struct btrfs_trans_handle *trans;
+	struct btrfs_key key = {
+		.objectid = BTRFS_FIRST_FREE_OBJECTID,
+		.type = BTRFS_INODE_ITEM_KEY,
+		.offset = 0,
+	};
+	struct extent_buffer *leaf;
+	int slot;
+	struct btrfs_path path;
+	int ret;
+
+	trans = btrfs_start_transaction(fs_info->fs_root, 1);

mkfs/main.c static int insert_qgroup_items(struct btrfs_trans_handle *trans, · line 910

Quoted 10 lines
mkfs/main.c +10 -0
+	btrfs_init_path(&path);
+	ret = btrfs_search_slot(trans, fs_info->fs_root, &key, &path, 0, 1);
+	if (ret)
+		goto fail;
+	leaf = path.nodes[0];
+	slot = path.slots[0];
+	btrfs_item_key_to_cpu(leaf, &key, slot);
+	btrfs_mark_buffer_dirty(leaf);
+	btrfs_commit_transaction(trans, fs_info->fs_root);
+	btrfs_release_path(&path);

Why there is no error handling for transaction start and commit?

Quoted 11 lines
mkfs/main.c +7 -0
+	return 0;
+fail:
+	btrfs_abort_transaction(trans, ret);
+	btrfs_release_path(&path);
+	return ret;
+}
+
 static int setup_quota_root(struct btrfs_fs_info *fs_info)
 {
 	struct btrfs_trans_handle *trans;

mkfs/main.c static int setup_quota_root(struct btrfs_fs_info *fs_info) · line 927

Quoted 5 lines
mkfs/main.c +2 -0
@@ -890,8 +923,11 @@ static int setup_quota_root(struct btrfs_fs_info *fs_info)
 	struct btrfs_path path;
 	struct btrfs_key key;
 	int qgroup_repaired = 0;
+	bool simple = btrfs_fs_incompat(fs_info, SIMPLE_QUOTA);
+	int flags;

Please use matching type for the structures, this is u64.

Quoted 32 lines
mkfs/main.c +12 -5
 	int ret;
 
+
 	/* One to modify tree root, one for quota root */
 	trans = btrfs_start_transaction(fs_info->tree_root, 2);
 	if (IS_ERR(trans)) {
@@ -921,13 +957,19 @@ static int setup_quota_root(struct btrfs_fs_info *fs_info)
 
 	qsi = btrfs_item_ptr(path.nodes[0], path.slots[0],
 			     struct btrfs_qgroup_status_item);
-	btrfs_set_qgroup_status_generation(path.nodes[0], qsi, 0);
+	btrfs_set_qgroup_status_generation(path.nodes[0], qsi, trans->transid);
 	btrfs_set_qgroup_status_rescan(path.nodes[0], qsi, 0);
+	flags = BTRFS_QGROUP_STATUS_FLAG_ON;
+	if (simple) {
+		btrfs_set_qgroup_status_enable_gen(path.nodes[0], qsi, trans->transid);
+		flags |= BTRFS_QGROUP_STATUS_FLAG_SIMPLE_MODE;
+	}
+	else {
+		flags |= BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT;
+	}
 
-	/* Mark current status info inconsistent, and fix it later */
-	btrfs_set_qgroup_status_flags(path.nodes[0], qsi,
-			BTRFS_QGROUP_STATUS_FLAG_ON |
-			BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT);
+	btrfs_set_qgroup_status_version(path.nodes[0], qsi, 1);
+	btrfs_set_qgroup_status_flags(path.nodes[0], qsi, flags);
 	btrfs_release_path(&path);
 
 	/* Currently mkfs will only create one subvolume */

mkfs/main.c static int setup_quota_root(struct btrfs_fs_info *fs_info) · line 989

Quoted 4 lines
mkfs/main.c +1 -0
@@ -944,6 +986,15 @@ static int setup_quota_root(struct btrfs_fs_info *fs_info)
 		return ret;
 	}
 
+	/* Hack to count the default subvol metadata by dirtying it */

This should try to explain how it works, not just that it's a hack. One day we may want to remove it so it should be possible if there is a need for it or not.

Quoted 23 lines
mkfs/main.c +10 -1
+	if (simple) {
+		ret = touch_root_subvol(fs_info);
+		if (ret) {
+			error("failed to touch root dir for simple quota accounting %d (%m)", ret);
+			goto fail;
+		}
+	}
+
 	/*
 	 * Qgroup is setup but with wrong info, use qgroup-verify
 	 * infrastructure to repair them.  (Just acts as offline rescan)
@@ -1743,7 +1794,8 @@ raid_groups:
 		}
 	}
 
-	if (features.runtime_flags & BTRFS_FEATURE_RUNTIME_QUOTA) {
+	if (features.runtime_flags & BTRFS_FEATURE_RUNTIME_QUOTA ||
+	    features.incompat_flags & BTRFS_FEATURE_INCOMPAT_SIMPLE_QUOTA) {
 		ret = setup_quota_root(fs_info);
 		if (ret < 0) {
 			error("failed to initialize quota: %d (%m)", ret);
Signature · 1 line
2.42.0
#10 lore raw
In reply to Boris Burkov
Quoted 18 lines · Boris Burkov

btrfs-progs changes for supporting simple quotas. Notably:
btrfs quota can enable squota via ioctl
mkfs can create an fs with squota enabled btrfstune can enable squota
btrfs inspect commands dump squota fields
btrfs check validates and repairs squota invariants

---
Changelog:
v3:
* rebase
* update enum values to match kernel changes
v2:
* fixed messed up list numbering in doc
* fixed broken formatting
* used new command in enable ioctl instead of status value
* introduced new qgroup status flag to mkfs/tune

Quoted 9 lines

Boris Burkov (8):
btrfs-progs: document squotas
btrfs-progs: simple quotas kernel definitions
btrfs-progs: simple quotas dump commands
btrfs-progs: simple quotas fsck
btrfs-progs: simple quotas mkfs
btrfs-progs: simple quotas btrfstune
btrfs-progs: simple quotas enable cmd
btrfs-progs: tree-checker: handle owner ref items

I've added the series to devel so we can do proper testing, however there are several style and convention problems, like the command line opiton names or long/short style. It'll be fixed in the branch.