Btrfs-progs: add support for mixed data+metadata block groups V3
PATCH 1/1 Btrfs-progs: add support for mixed data+metadata block groups V3
So alot of crazy people (I'm looking at you Meego) want to use btrfs on phones and such with small devices. Unfortunately the way we split out metadata/data chunks it makes space usage inefficient for volumes that are smaller than 1gigabyte. So add a -M option for mixing metadata+data, and default to this mixed mode if the filesystem is less than or equal to 1 gigabyte. I've tested this with xfstests on a 100mb filesystem and everything is a-ok.
---V2->V3: Add an incompat flag since the kernel will panic with mixed block groups unless its had a few key fixes.
V1->V2: My original patch was screwing up if you specified the data or metadata profile. This update fixes that problem.
btrfs-vol.c | 4 +- btrfs_cmds.c | 13 +++++- ctree.h | 4 +- mkfs.c | 122 +++++++++++++++++++++++++++++++++++++++++----------------- utils.c | 10 ++-- utils.h | 2 +- 6 files changed, 109 insertions(+), 46 deletions(-)
btrfs-vol.c +3 -1
@@ -129,7 +129,9 @@ int main(int ac, char **av)exit (1 ); }if (cmd ==BTRFS_IOC_ADD_DEV ) { - ret =btrfs_prepare_device (devfd, device,1 , &dev_block_count); +int mixed =0 ; + + ret =btrfs_prepare_device (devfd, device,1 , &dev_block_count, &mixed);if (ret) {fprintf (stderr,"Unable to init %s\n" , device);exit (1 );
btrfs_cmds.c +10 -3
@@ -705,6 +705,7 @@ int do_add_volume(int nargs, char **args)int devfd, res;u64 dev_block_count =0 ;struct stat st; +int mixed =0 ; devfd =open (args[i],O_RDWR );if (!devfd) {
@@ -727,7 +728,7 @@ int do_add_volume(int nargs, char **args)continue ; } - res =btrfs_prepare_device (devfd, args[i],1 , &dev_block_count); + res =btrfs_prepare_device (devfd, args[i],1 , &dev_block_count, &mixed);if (res) {fprintf (stderr,"ERROR: Unable to init '%s'\n" , args[i]);close (devfd);
@@ -889,8 +890,14 @@ int do_df_filesystem(int nargs, char **argv)memset (description,0 ,80 );if (flags &BTRFS_BLOCK_GROUP_DATA ) { -snprintf (description,5 ,"%s" ,"Data" ); - written +=4 ; +if (flags &BTRFS_BLOCK_GROUP_METADATA ) { +snprintf (description,15 ,"%s" , +"Data+Metadata" ); + written +=14 ; + }else { +snprintf (description,5 ,"%s" ,"Data" ); + written +=4 ; + } }else if (flags &BTRFS_BLOCK_GROUP_SYSTEM ) {snprintf (description,7 ,"%s" ,"System" ); written +=6 ;
ctree.h +3 -1
@@ -352,13 +352,15 @@ struct btrfs_super_block {#define BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF (1ULL << 0)#define BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL (1ULL << 1)#define BTRFS_FEATURE_INCOMPAT_SPACE_CACHE (1ULL << 2) +#define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS (1ULL << 3)#define BTRFS_FEATURE_COMPAT_SUPP 0ULL#define BTRFS_FEATURE_COMPAT_RO_SUPP 0ULL#define BTRFS_FEATURE_INCOMPAT_SUPP \ (BTRFS_FEATURE_INCOMPAT_MIXED_BACKREF | \ BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL | \ - BTRFS_FEATURE_INCOMPAT_SPACE_CACHE) + BTRFS_FEATURE_INCOMPAT_SPACE_CACHE | \ + BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS)/* * A leaf is full of items. offset and size tell us where to find
mkfs.c +87 -35
239 lines not on this page · raw message
utils.c +5 -5
@@ -507,7 +507,8 @@ int btrfs_add_to_fsid(struct btrfs_trans_handle *trans,return 0 ; } -int btrfs_prepare_device (int fd,char *file,int zero_end,u64 *block_count_ret) +int btrfs_prepare_device (int fd,char *file,int zero_end,u64 *block_count_ret, +int *mixed) {u64 block_count;u64 bytenr;
@@ -527,10 +528,9 @@ int btrfs_prepare_device(int fd, char *file, int zero_end, u64 *block_count_ret) } zero_end =1 ; -if (block_count <256 *1024 *1024 ) { -fprintf (stderr,"device %s is too small " -"(must be at least 256 MB)\n" , file); -exit (1 ); +if (block_count <1024 *1024 *1024 && !(*mixed)) { +printf ("SMALL VOLUME: forcing mixed metadata/data groups\n" ); + *mixed =1 ; } ret =zero_dev_start (fd);if (ret) {
utils.h +1 -1
@@ -27,7 +27,7 @@ int make_btrfs(int fd, const char *device, const char *label,int btrfs_make_root_dir (struct btrfs_trans_handle *trans,struct btrfs_root *root,u64 objectid);int btrfs_prepare_device (int fd,char *file,int zero_end, -u64 *block_count_ret); +u64 *block_count_ret,int *mixed); int btrfs_add_to_fsid(struct btrfs_trans_handle *trans, struct btrfs_root *root, int fd, char *path, u64 block_count, u32 io_width, u32 io_align,
Signature · 1 line
1.6.6.1
On Wed, 27 Oct 2010 14:37:34 -0400 Josef Bacik <[email protected]> wrote= :
Quoted 1 line
So alot of crazy people (I'm looking at you Meego) want to use btrfs =
on phones
Quoted 1 line
and such with small devices. Unfortunately the way we split out meta=
data/data
Quoted 1 line
chunks it makes space usage inefficient for volumes that are smaller =
than
Quoted 1 line
1gigabyte. So add a -M option for mixing metadata+data, and default =
to this
Quoted 1 line
mixed mode if the filesystem is less than or equal to 1 gigabyte. I'=
ve tested
Quoted 2 lines
this with xfstests on a 100mb filesystem and everything is a-ok. =20
Hi,
Could you provide some rationale as to why btrfs should support these
two modes of data/metadata (mixed and separate) allocation instead of just always doing mixed data+metadata?
The reason I'm asking this is that having two separate modes implies
having more codepaths in the kernel, which naturally adds complexity to the filesystem and also more code to test, two things that should be avoided, if possible. I'd love if you could add a small explanation to your patch description (or a comment in the code) about why it is useful to keep the default mode of allocation, instead of simply droping it and doing mixed allocation in all cases.
Thanks in advance.
Best regards
Cl=C3=A1udio
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 2 lines · On Fri, Oct 29, 2010 at 01:20:13AM +0100, Cl=E1udio Martins wrote:
=20
On Wed, 27 Oct 2010 14:37:34 -0400 Josef Bacik <[email protected]> wro=
te:
Quoted 1 line
Quoted 1 line
So alot of crazy people (I'm looking at you Meego) want to use btrf=
s on phones
Quoted 1 line
Quoted 1 line
and such with small devices. Unfortunately the way we split out me=
tadata/data
Quoted 1 line
Quoted 1 line
chunks it makes space usage inefficient for volumes that are smalle=
r than
Quoted 1 line
Quoted 1 line
1gigabyte. So add a -M option for mixing metadata+data, and defaul=
t to this
Quoted 1 line
Quoted 1 line
mixed mode if the filesystem is less than or equal to 1 gigabyte. =
I've tested
Quoted 6 lines
Quoted 2 lines
this with xfstests on a 100mb filesystem and everything is a-ok. =20
=20
Hi,
=20
Could you provide some rationale as to why btrfs should support thes=
e
Quoted 3 lines
two modes of data/metadata (mixed and separate) allocation instead of just always doing mixed data+metadata?
It makes the volume management stuff easier and gives us more flexibili= ty for
how we lay out data and metadata. Thanks,
Josef=20
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
My apologies if I missed it, but this looks like the most recent version of this patch ([PATCH] Btrfs-progs: add support for mixed data+metadata block groups V3), and this version needs updating to take into account that BTRFS_FEATURE_INCOMPAT_SPACE_CACHE is no longer needed.
The kernel btrfs code expects:
#define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS (1ULL << 2)
But this version of the patch gives:
#define BTRFS_FEATURE_INCOMPAT_MIXED_GROUPS (1ULL << 3)
with (1ULL << 2) being occupied by the space_cache incompat flag.