Skip to contentSkip to conversation

Btrfs-progs: add support for mixed data+metadata block groups V3

PATCH In review Josef Bacik 3 participants 4 messages Oct 27, 2010 – Nov 6, 2010 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Btrfs-progs: add support for mixed data+metadata block groups V3

So alot of crazy people (I'm looking at you Meego) want to use btrfs on phones and such with small devices. Unfortunately the way we split out metadata/data chunks it makes space usage inefficient for volumes that are smaller than 1gigabyte. So add a -M option for mixing metadata+data, and default to this mixed mode if the filesystem is less than or equal to 1 gigabyte. I've tested this with xfstests on a 100mb filesystem and everything is a-ok.

---
V2->V3: Add an incompat flag since the kernel will panic with mixed block groups unless its had a few key fixes.

V1->V2: My original patch was screwing up if you specified the data or metadata profile. This update fixes that problem.

 btrfs-vol.c  |    4 +-
 btrfs_cmds.c |   13 +++++-
 ctree.h      |    4 +-
 mkfs.c       |  122 +++++++++++++++++++++++++++++++++++++++++-----------------
 utils.c      |   10 ++--
 utils.h      |    2 +-
 6 files changed, 109 insertions(+), 46 deletions(-)
btrfs-vol.c +3 -1
index 8069778..7200bbc 100644
@@ -129,7 +129,9 @@ int main(int ac, char **av)
 		exit(1);
 	}
 	if (cmd == BTRFS_IOC_ADD_DEV) {
-		ret = btrfs_prepare_device(devfd, device, 1, &dev_block_count);
+		int mixed = 0;
+
+		ret = btrfs_prepare_device(devfd, device, 1, &dev_block_count, &mixed);
 		if (ret) {
 			fprintf(stderr, "Unable to init %s\n", device);
 			exit(1);
btrfs_cmds.c +10 -3
index 8031c58..683aec0 100644
@@ -705,6 +705,7 @@ int do_add_volume(int nargs, char **args)
 		int	devfd, res;
 		u64 dev_block_count = 0;
 		struct stat st;
+		int mixed = 0;
 
 		devfd = open(args[i], O_RDWR);
 		if (!devfd) {
@@ -727,7 +728,7 @@ int do_add_volume(int nargs, char **args)
 			continue;
 		}
 
-		res = btrfs_prepare_device(devfd, args[i], 1, &dev_block_count);
+		res = btrfs_prepare_device(devfd, args[i], 1, &dev_block_count, &mixed);
 		if (res) {
 			fprintf(stderr, "ERROR: Unable to init '%s'\n", args[i]);
 			close(devfd);
@@ -889,8 +890,14 @@ int do_df_filesystem(int nargs, char **argv)
 		memset(description, 0, 80);
 
 		if (flags & BTRFS_BLOCK_GROUP_DATA) {
-			snprintf(description, 5, "%s", "Data");
-			written += 4;
+			if (flags & BTRFS_BLOCK_GROUP_METADATA) {
+				snprintf(description, 15, "%s",
+					 "Data+Metadata");
+				written += 14;
+			} else {
+				snprintf(description, 5, "%s", "Data");
+				written += 4;
+			}
 		} else if (flags & BTRFS_BLOCK_GROUP_SYSTEM) {
 			snprintf(description, 7, "%s", "System");
 			written += 6;
ctree.h +3 -1
index 4d0e1e2..8276e5e 100644
@@ -352,13 +352,15 @@ struct btrfs_super_block {
 #define BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF	(1ULL << 0)
 #define BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL	(1ULL << 1)
 #define BTRFS_FEATURE_INCOMPAT_SPACE_CACHE	(1ULL << 2)
+#define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS	(1ULL << 3)
 
 #define BTRFS_FEATURE_COMPAT_SUPP		0ULL
 #define BTRFS_FEATURE_COMPAT_RO_SUPP		0ULL
 #define BTRFS_FEATURE_INCOMPAT_SUPP			\
 	(BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF |		\
 	 BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL |	\
-	 BTRFS_FEATURE_INCOMPAT_SPACE_CACHE)
+	 BTRFS_FEATURE_INCOMPAT_SPACE_CACHE |		\
+	 BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS)
 
 /*
  * A leaf is full of items. offset and size tell us where to find
mkfs.c +87 -35

239 lines not on this page · raw message

utils.c +5 -5
index 2f4c6e1..0de43df 100644
@@ -507,7 +507,8 @@ int btrfs_add_to_fsid(struct btrfs_trans_handle *trans,
 	return 0;
 }
 
-int btrfs_prepare_device(int fd, char *file, int zero_end, u64 *block_count_ret)
+int btrfs_prepare_device(int fd, char *file, int zero_end, u64 *block_count_ret,
+			 int *mixed)
 {
 	u64 block_count;
 	u64 bytenr;
@@ -527,10 +528,9 @@ int btrfs_prepare_device(int fd, char *file, int zero_end, u64 *block_count_ret)
 	}
 	zero_end = 1;
 
-	if (block_count < 256 * 1024 * 1024) {
-		fprintf(stderr, "device %s is too small "
-		        "(must be at least 256 MB)\n", file);
-		exit(1);
+	if (block_count < 1024 * 1024 * 1024 && !(*mixed)) {
+		printf("SMALL VOLUME: forcing mixed metadata/data groups\n");
+		*mixed = 1;
 	}
 	ret = zero_dev_start(fd);
 	if (ret) {
utils.h +1 -1
index 7ff542b..b91140e 100644
@@ -27,7 +27,7 @@ int make_btrfs(int fd, const char *device, const char *label,
 int btrfs_make_root_dir(struct btrfs_trans_handle *trans,
 			struct btrfs_root *root, u64 objectid);
 int btrfs_prepare_device(int fd, char *file, int zero_end,
-			 u64 *block_count_ret);
+			 u64 *block_count_ret, int *mixed);
 int btrfs_add_to_fsid(struct btrfs_trans_handle *trans,
 		      struct btrfs_root *root, int fd, char *path,
 		      u64 block_count, u32 io_width, u32 io_align,
Signature · 1 line
1.6.6.1
#1 lore raw

On Wed, 27 Oct 2010 14:37:34 -0400 Josef Bacik <[email protected]> wrote= :

Quoted 1 line

So alot of crazy people (I'm looking at you Meego) want to use btrfs =

on phones

Quoted 1 line

and such with small devices. Unfortunately the way we split out meta=

data/data

Quoted 1 line

chunks it makes space usage inefficient for volumes that are smaller =

than

Quoted 1 line

1gigabyte. So add a -M option for mixing metadata+data, and default =

to this

Quoted 1 line

mixed mode if the filesystem is less than or equal to 1 gigabyte. I'=

ve tested

Quoted 2 lines

this with xfstests on a 100mb filesystem and everything is a-ok. =20

Hi,

Could you provide some rationale as to why btrfs should support these
two modes of data/metadata (mixed and separate) allocation instead of just always doing mixed data+metadata?

The reason I'm asking this is that having two separate modes implies
having more codepaths in the kernel, which naturally adds complexity to the filesystem and also more code to test, two things that should be avoided, if possible. I'd love if you could add a small explanation to your patch description (or a comment in the code) about why it is useful to keep the default mode of allocation, instead of simply droping it and doing mixed allocation in all cases.

Thanks in advance.

Best regards

Cl=C3=A1udio

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
Quoted 2 lines · On Fri, Oct 29, 2010 at 01:20:13AM +0100, Cl=E1udio Martins wrote:

=20
On Wed, 27 Oct 2010 14:37:34 -0400 Josef Bacik <[email protected]> wro=

te:

Quoted 1 line
Quoted 1 line

So alot of crazy people (I'm looking at you Meego) want to use btrf=

s on phones

Quoted 1 line
Quoted 1 line

and such with small devices. Unfortunately the way we split out me=

tadata/data

Quoted 1 line
Quoted 1 line

chunks it makes space usage inefficient for volumes that are smalle=

r than

Quoted 1 line
Quoted 1 line

1gigabyte. So add a -M option for mixing metadata+data, and defaul=

t to this

Quoted 1 line
Quoted 1 line

mixed mode if the filesystem is less than or equal to 1 gigabyte. =

I've tested

Quoted 6 lines
Quoted 2 lines

this with xfstests on a 100mb filesystem and everything is a-ok. =20

=20
Hi,
=20
Could you provide some rationale as to why btrfs should support thes=

e

Quoted 3 lines

two modes of data/metadata (mixed and separate) allocation instead of just always doing mixed data+metadata?

It makes the volume management stuff easier and gives us more flexibili= ty for
how we lay out data and metadata. Thanks,

Josef=20
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw

My apologies if I missed it, but this looks like the most recent version of this patch ([PATCH] Btrfs-progs: add support for mixed data+metadata block groups V3), and this version needs updating to take into account that BTRFS_FEATURE_INCOMPAT_SPACE_CACHE is no longer needed.

The kernel btrfs code expects:

#define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS (1ULL << 2)

But this version of the patch gives:

#define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS (1ULL << 3)

with (1ULL << 2) being occupied by the space_cache incompat flag.