Btrfs: add support for mixed data+metadata block groups V3
PATCH 1/1 Btrfs: add support for mixed data+metadata block groups V3
There are just a few things that need to be fixed in the kernel to support mixed data+metadata block groups. Mostly we just need to make sure that if we are using mixed block groups that we continue to allocate mixed block groups as we need them. Also we need to make sure __find_space_info will find our space info if we search for DATA or METADATA only. Tested this with xfstests and it works nicely. Thanks,
---V2->V3: Add btrfs_mixed_space_info helper
V1->V2: In do_chunk_alloc I was changing flags to == space_info->flags, which isn't right since space_info doesn't carry the RAID profiles anymore, so instead check to see if the space info has DATA and METADATA set and if so set that in the flags as well.
fs/btrfs/ctree.h | 6 ++++++ fs/btrfs/extent-tree.c | 22 +++++++++++++++++++--- 2 files changed, 25 insertions(+), 3 deletions(-)
fs/btrfs/ctree.h +6 -0
@@ -2043,6 +2043,12 @@ static inline struct dentry *fdentry(struct file *file)return file->f_path.dentry; } +static inline bool btrfs_mixed_space_info (struct btrfs_space_info *space_info) +{ +return ((space_info->flags &BTRFS_BLOCK_GROUP_METADATA ) && + (space_info->flags &BTRFS_BLOCK_GROUP_DATA )); +} +/* extent-tree.c */ void btrfs_put_block_group (struct btrfs_block_group_cache *cache); int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,
fs/btrfs/extent-tree.c +19 -3
@@ -547,7 +547,7 @@ static struct btrfs_space_info *__find_space_info(struct btrfs_fs_info *info,rcu_read_lock ();list_for_each_entry_rcu (found, head, list) { -if (found->flags == flags) { +if (found->flags & flags) {rcu_read_unlock ();return found; }
@@ -3267,6 +3267,13 @@ static int do_chunk_alloc(struct btrfs_trans_handle *trans,spin_unlock (&space_info->lock);/* +* If we have mixed data/metadata chunks we want to make sure we keep +* allocating mixed chunks instead of individual chunks. +*/ +if (btrfs_mixed_space_info (space_info)) + flags |= (BTRFS_BLOCK_GROUP_DATA |BTRFS_BLOCK_GROUP_METADATA ); + +/* * if we're doing a data chunk, go ahead and make sure that * we keep a reasonable number of metadata chunks allocated in the * FS as well.
@@ -4793,6 +4800,7 @@ static noinline int find_free_extent(struct btrfs_trans_handle *trans,bool found_uncached_bg = false;bool failed_cluster_refill = false;bool failed_alloc = false; +bool use_cluster = true;u64 ideal_cache_percent =0 ;u64 ideal_cache_offset =0 ;
@@ -4807,16 +4815,24 @@ static noinline int find_free_extent(struct btrfs_trans_handle *trans,return -ENOSPC ; } +/* +* If the space info is for both data and metadata it means we have a +* small filesystem and we can't use the clustering stuff. +*/ +if (btrfs_mixed_space_info (space_info)) + use_cluster = false; +if (orig_root->ref_cows || empty_size) allowed_chunk_alloc =1 ; -if (data &BTRFS_BLOCK_GROUP_METADATA ) { +if (data &BTRFS_BLOCK_GROUP_METADATA && use_cluster) { last_ptr = &root->fs_info->meta_alloc_cluster;if (!btrfs_test_opt (root,SSD )) empty_cluster =64 *1024 ; } -if ((data &BTRFS_BLOCK_GROUP_DATA ) &&btrfs_test_opt (root,SSD )) { +if ((data &BTRFS_BLOCK_GROUP_DATA ) && use_cluster && +btrfs_test_opt (root,SSD )) { last_ptr = &root->fs_info->data_alloc_cluster; }
Signature · 1 line
1.6.6.1
I've been testing this patch (as well as the accompanying patch to btrfs-progs).
It seems to save a decent amount of space (maybe 10-20% according to df in my testing, YMMV), but I was also noticing a performance penalty of maybe 5-15%, depending on the application (in my case, I was timing the untar-ing of data to a btrfs partition).
I also developed the perception that I was encountering some minor latency issues when heavily using a drive formated for mixed data and metadata (more keyboard and mouse hesitations than normal, difficult stuff to quantify).
The main problem I encountered was a btrfs crash when booting with an un-patched kernel, and letting the boot process attempt to mount the drive with mixed data+metadata. I wasn't entirely surprised by this result, but it was unclear from the patch description whether a volume formatted with mixed data+metadata would be incompatible with an un-patched, older kernel.
Other than that, it seemed to perform well. I did not encounter any stability issues as long as I was using a patched kernel.
Does this mixing constitute a forbidden change of on-disk format, and if not how not?
Quoted 1 line · Mitch Harder
Other than that, it seemed to perform well. =C2=A0I did not encounter=
any
Quoted 1 line
stability issues as long as I was using a patched kernel.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Jueves, 21 de Octubre de 2010 17:46:58 David Nicol escribi=C3=B3:
Quoted 2 lines
Does this mixing constitute a forbidden change of on-disk format, and if not how not?
It doesn't need a format change. The difference between a data and a metadata block group is just an allocation hint AFAIK.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Oct 21, 2010 at 5:09 PM, Diego Calleja <[email protected]> wrot= e:
Quoted 2 lines
On Jueves, 21 de Octubre de 2010 17:46:58 David Nicol escribi=F3:
Quoted 1 line
Does this mixing constitute a forbidden change of on-disk format, an=
d
Quoted 6 lines
Quoted 1 line
if not how not?
It doesn't need a format change. The difference between a data and a metadata block group is just an allocation hint AFAIK.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
Quoted 3 lines
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
Let me know if the problems with an un-patched kernel were un-expected.
I can provide more information on the crash when booting an older kerne= l.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · Mitch Harder
On Thu, Oct 21, 2010 at 5:09 PM, Diego Calleja <[email protected]> wr=
ote:
Quoted 2 lines
Quoted 2 lines
On Jueves, 21 de Octubre de 2010 17:46:58 David Nicol escribi=F3:
Quoted 1 line
Does this mixing constitute a forbidden change of on-disk format, =
and
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
if not how not?
It doesn't need a format change. The difference between a data and a metadata block group is just an allocation hint AFAIK. --
To unsubscribe from this list: send the line "unsubscribe linux-btr=
fs" in
Quoted 2 lines
Quoted 2 lines
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.htm=
l
Quoted 3 lines
Quoted 1 line
=20
Let me know if the problems with an un-patched kernel were un-expecte=
d.
Quoted 2 lines
=20
I can provide more information on the crash when booting an older ker=
nel.
Nope they are expected, it's not a disk format change, but older kernel= s won't
deal with mixed block groups. Thanks,
Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 2 lines · Josef Bacik
Quoted 1 line · Mitch Harder
On Thu, Oct 21, 2010 at 5:09 PM, Diego Calleja <[email protected]> w=
rote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
On Jueves, 21 de Octubre de 2010 17:46:58 David Nicol escribi=F3:
Quoted 1 line
Does this mixing constitute a forbidden change of on-disk format,=
and
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
if not how not?
It doesn't need a format change. The difference between a data and a metadata block group is just an allocation hint AFAIK. --
To unsubscribe from this list: send the line "unsubscribe linux-bt=
rfs" in
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.ht=
ml
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Let me know if the problems with an un-patched kernel were un-expect=
ed.
Quoted 2 lines
Quoted 2 lines
I can provide more information on the crash when booting an older ke=
rnel.
Quoted 2 lines
Nope they are expected, it's not a disk format change, but older kern=
els won't
Quoted 1 line
deal with mixed block groups.
When something like this goes mainline, is it used by default/automatic= ally?
I ask because I maintain a btrfs-based rollback initramfs hook [1], and am currently updating it for extlinux, enabling kernel-level system rollbacks via btrfs set-default + reboot (or maybe kexec)...
rolling back to an old kernel will then blow up my machine (figuratively of course :-)?
C Anthony
[1] http://aur.archlinux.org/packages.php?ID=3D33376
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · C Anthony Risinger
On Thu, Oct 21, 2010 at 8:05 PM, Josef Bacik <[email protected]> wrote=
:
Quoted 2 lines
Quoted 2 lines · Mitch Harder
Quoted 1 line
On Thu, Oct 21, 2010 at 5:09 PM, Diego Calleja <[email protected]>=
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Jueves, 21 de Octubre de 2010 17:46:58 David Nicol escribi=F3=
:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Does this mixing constitute a forbidden change of on-disk forma=
t, and
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
if not how not?
It doesn't need a format change. The difference between a data a=
nd
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
a metadata block group is just an allocation hint AFAIK. --
To unsubscribe from this list: send the line "unsubscribe linux-=
btrfs" in
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.=
html
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Let me know if the problems with an un-patched kernel were un-expe=
cted.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I can provide more information on the crash when booting an older =
kernel.
Quoted 2 lines
Quoted 2 lines
Nope they are expected, it's not a disk format change, but older ke=
rnels won't
Quoted 3 lines
Quoted 1 line
deal with mixed block groups.
=20
When something like this goes mainline, is it used by default/automat=
ically?
Quoted 9 lines
=20
I ask because I maintain a btrfs-based rollback initramfs hook [1], and am currently updating it for extlinux, enabling kernel-level system rollbacks viabtrfs set-default+ reboot (or maybekexec)...
=20
rolling back to an old kernel will then blow up my machine (figuratively of course :-)?
The only way you get this feature is if you mkfs with the feature enabl= ed, and
is only meant for small filesystems (1 gig or smaller). Thanks,
Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 2 lines · Josef Bacik
Quoted 1 line · C Anthony Risinger
On Thu, Oct 21, 2010 at 8:05 PM, Josef Bacik <[email protected]> wrot=
e:
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines · Mitch Harder
Quoted 1 line
On Thu, Oct 21, 2010 at 5:09 PM, Diego Calleja <[email protected]=
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Jueves, 21 de Octubre de 2010 17:46:58 David Nicol escribi=F3=
:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Does this mixing constitute a forbidden change of on-disk form=
at, and
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
if not how not?
It doesn't need a format change. The difference between a data =
and
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
a metadata block group is just an allocation hint AFAIK. --
To unsubscribe from this list: send the line "unsubscribe linux=
-btrfs" in
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
the body of a message to [email protected] More majordomo info at =A0http://vger.kernel.org/majordomo-info=
=2Ehtml
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
Let me know if the problems with an un-patched kernel were un-exp=
ected.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
I can provide more information on the crash when booting an older=
kernel.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Nope they are expected, it's not a disk format change, but older k=
ernels won't
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
deal with mixed block groups.
When something like this goes mainline, is it used by default/automa=
tically?
Quoted 11 lines
Quoted 9 lines
I ask because I maintain a btrfs-based rollback initramfs hook [1], and am currently updating it for extlinux, enabling kernel-level system rollbacks via
btrfs set-default+ reboot (or maybekexec)...rolling back to an old kernel will then blow up my machine (figuratively of course :-)?
The only way you get this feature is if you mkfs with the feature ena=
bled, and
Quoted 1 line
is only meant for small filesystems (1 gig or smaller).
Ah right :-), thanks
C Anthony
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html