Skip to contentSkip to conversation

Stuck in CHANGING_BG_TREE state after interrupted btrfstune --convert-to-block-group-tree

DISCUSSION Tine Mezgec 2 participants 7 messages Jun 12, 2025 – Jun 18, 2025 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hi,

I have a Btrfs filesystem mounted at /media/storage with the following setup, that took minutes to mount:

Label: 'data' uuid: 7e1efcb0-96c7-4e8f-861c-ee3edba3e028 Total devices 11 FS bytes used 40.33TiB

devid    2 size 10.91TiB used  3.79TiB path /dev/sdd
devid    3 size  7.28TiB used     0.00B path /dev/sdc
devid    4 size  7.28TiB used     0.00B path /dev/sdg
devid    5 size  7.28TiB used   215.00GiB path /dev/sda
devid    8 size 16.37TiB used  8.93TiB path /dev/sdk
devid    9 size 16.37TiB used  9.50TiB path /dev/sdh
devid   12 size 14.55TiB used  7.41TiB path /dev/sdi
devid   13 size 14.55TiB used  7.71TiB path /dev/sdj
devid   15 size 18.19TiB used 10.74TiB path /dev/sdf
devid   16 size 23.65TiB used 16.20TiB path /dev/sdb
devid   17 size 23.65TiB used 16.20TiB path /dev/sde

I ran (after unmounting):

btrfstune --convert-to-block-group-tree /dev/sdd
(using btrfs-progs 6.6.3-1.1build2 from Ubuntu 24.04), but the system lost power during the conversion.

After reboot, rerunning the command gives:

ERROR: failed to find block group for bytenr 186297726533632
ERROR: failed to convert the filesystem to block group tree feature
ERROR: btrfstune failed

extent buffer leak: start 185256860958720 len 16384

Trying with kernel 6.15.2 and btrfs-progs 6.14 gives the same result.

The superblock flags now show:

0x4000000001 (WRITTEN | CHANGING_BG_TREE)

Attempting to revert:

btrfstune --convert-from-block-group-tree /dev/sdd fails with:
ERROR: filesystem doesn't have block-group-tree feature
ERROR: btrfstune failed

So I'm stuck with a filesystem in a half-converted state — not fully converted, but marked as changing.

What’s the best way to either complete the conversion or revert/abort it cleanly?

Thanks,
-Tine

#1 lore raw
Quoted 4 lines · Tine Mezgec

Hi,

I have a Btrfs filesystem mounted at /media/storage with the following setup, that took minutes to mount:

[...]

Quoted 2 lines

btrfstune --convert-to-block-group-tree /dev/sdd
(using btrfs-progs 6.6.3-1.1build2 from Ubuntu 24.04), but the system lost power during the conversion.

The progs is a little old, but I do not think there are bgt related fixes after that though.

Quoted 2 lines

After reboot, rerunning the command gives:

Rerunning the command is the correct way to resume, but something doesn't seem correct here.

Quoted 2 lines

ERROR: failed to find block group for bytenr 186297726533632

Please provide the following dump:

# btrfs ins dump-tree -t chunk <device>
# btrfs ins dump-tree -t extent <device>
  The above one is pretty large.
# btrfs ins dump-tree -t block-group <device>

I guess there is something wrong with the last converted bg checks, resulting btrfstune to trying to convert an already converted bg.

Thanks,
Qu

Quoted 25 lines

ERROR: failed to convert the filesystem to block group tree feature
ERROR: btrfstune failed

extent buffer leak: start 185256860958720 len 16384

Trying with kernel 6.15.2 and btrfs-progs 6.14 gives the same result.

The superblock flags now show:

0x4000000001 (WRITTEN | CHANGING_BG_TREE)

Attempting to revert:

btrfstune --convert-from-block-group-tree /dev/sdd fails with:
ERROR: filesystem doesn't have block-group-tree feature
ERROR: btrfstune failed

So I'm stuck with a filesystem in a half-converted state — not fully converted, but marked as changing.

What’s the best way to either complete the conversion or revert/abort it cleanly?

Thanks,
-Tine

#2 lore raw

Hi Qu,

Thanks for taking a look at this.

Here are the three tree dumps you asked for:

http://144.76.28.172/btrfs/block-group.dump.gz
http://144.76.28.172/btrfs/chunk.dump.gz
http://144.76.28.172/btrfs/extent.dump.gz

Let me know if you need anything else

Thanks,
-Tine

Quoted 62 lines · Qu Wenruo
Quoted 4 lines · Tine Mezgec

Hi,

I have a Btrfs filesystem mounted at /media/storage with the following setup, that took minutes to mount:

[...]

Quoted 2 lines

btrfstune --convert-to-block-group-tree /dev/sdd
(using btrfs-progs 6.6.3-1.1build2 from Ubuntu 24.04), but the system lost power during the conversion.

The progs is a little old, but I do not think there are bgt related fixes after that though.

Quoted 2 lines

After reboot, rerunning the command gives:

Rerunning the command is the correct way to resume, but something doesn't seem correct here.

Quoted 2 lines

ERROR: failed to find block group for bytenr 186297726533632

Please provide the following dump:

# btrfs ins dump-tree -t chunk <device>
# btrfs ins dump-tree -t extent <device>
   The above one is pretty large.
# btrfs ins dump-tree -t block-group <device>

I guess there is something wrong with the last converted bg checks, resulting btrfstune to trying to convert an already converted bg.

Thanks,
Qu

Quoted 25 lines

ERROR: failed to convert the filesystem to block group tree feature
ERROR: btrfstune failed

extent buffer leak: start 185256860958720 len 16384

Trying with kernel 6.15.2 and btrfs-progs 6.14 gives the same result.

The superblock flags now show:

0x4000000001 (WRITTEN | CHANGING_BG_TREE)

Attempting to revert:

btrfstune --convert-from-block-group-tree /dev/sdd fails with:
ERROR: filesystem doesn't have block-group-tree feature
ERROR: btrfstune failed

So I'm stuck with a filesystem in a half-converted state — not fully converted, but marked as changing.

What’s the best way to either complete the conversion or revert/abort it cleanly?

Thanks,
-Tine

#3 lore raw
Quoted 11 lines · Tine Mezgec

Hi Qu,

Thanks for taking a look at this.

Here are the three tree dumps you asked for:

http://144.76.28.172/btrfs/block-group.dump.gz
http://144.76.28.172/btrfs/chunk.dump.gz
http://144.76.28.172/btrfs/extent.dump.gz

Let me know if you need anything else

Thanks a lot.

The block-group dump shows that all block groups after 186298800275456 (including that bytenr) are converted to block group tree.

At the same time, chunk tree dump shows that block groups 186297726533632 is indeed the block group before the converted one.

Furthermore in side the extent tree dump, it shows there is indeed a block group item for 186297726533632 in the extent tree.

So the bug looks very likely inside the mount process which doesn't handle the last block group in the old tree correctly.

I crafted and attached a small patch, that will skip the the optimization to do a full extent tree search for old block groups. I hope this will allow you to resume the conversion.

You'll need to compile btrfs-progs with this patch applied, then use the compiled btrfstune to resume the conversion. (Better using the latest release of btrfs-progs)

This patch will cause a warning about an unused function but you can safely ignore it.

Thanks,
Qu

Quoted 68 lines

Thanks,
-Tine

Quoted 62 lines · Qu Wenruo
Quoted 4 lines · Tine Mezgec

Hi,

I have a Btrfs filesystem mounted at /media/storage with the following setup, that took minutes to mount:

[...]

Quoted 2 lines

btrfstune --convert-to-block-group-tree /dev/sdd
(using btrfs-progs 6.6.3-1.1build2 from Ubuntu 24.04), but the system lost power during the conversion.

The progs is a little old, but I do not think there are bgt related fixes after that though.

Quoted 2 lines

After reboot, rerunning the command gives:

Rerunning the command is the correct way to resume, but something doesn't seem correct here.

Quoted 2 lines

ERROR: failed to find block group for bytenr 186297726533632

Please provide the following dump:

# btrfs ins dump-tree -t chunk <device>
# btrfs ins dump-tree -t extent <device>
   The above one is pretty large.
# btrfs ins dump-tree -t block-group <device>

I guess there is something wrong with the last converted bg checks, resulting btrfstune to trying to convert an already converted bg.

Thanks,
Qu

Quoted 25 lines

ERROR: failed to convert the filesystem to block group tree feature
ERROR: btrfstune failed

extent buffer leak: start 185256860958720 len 16384

Trying with kernel 6.15.2 and btrfs-progs 6.14 gives the same result.

The superblock flags now show:

0x4000000001 (WRITTEN | CHANGING_BG_TREE)

Attempting to revert:

btrfstune --convert-from-block-group-tree /dev/sdd fails with:
ERROR: filesystem doesn't have block-group-tree feature
ERROR: btrfstune failed

So I'm stuck with a filesystem in a half-converted state — not fully converted, but marked as changing.

What’s the best way to either complete the conversion or revert/abort it cleanly?

Thanks,
-Tine

kernel-shared/extent-tree.c +1 -1
index 2b7a962f294b..b02bba4630b6 100644
@@ -2984,7 +2984,7 @@ static int read_converting_block_groups(struct btrfs_fs_info *fs_info)
 		return ret;
 	}
 
-	ret = read_old_block_groups_from_root(fs_info, old_root);
+	ret = read_block_groups_from_root(fs_info,  old_root);
 	if (ret < 0) {
 		error("failed to load block groups from the old root: %d", ret);
 		return ret;
#4 lore raw

Hi Qu,

Your patch did the trick, the conversion completed cleanly, the CHANGING_BG_TREE flag is gone, and the incompat BG_TREE bit is now set. Mounting the 11‑device volume takes ~5 seconds instead of several minutes, so everything is back to normal or even better than before.

Thanks a lot for the quick diagnosis and the fix.

One thought: the optimisation that skips the full extent‑tree scan clearly helps in the common case, but when it backfires the only recovery path is recompiling progs with a patch. Would it make sense to expose a --full-scan (or --no-fast-scan) switch in btrfstune so an interrupted conversion can be resumed with stock binaries?

Either way, I really appreciate the help.

Best regards,
-Tine

#5 lore raw
Quoted 7 lines · Tine Mezgec

Hi Qu,

Your patch did the trick, the conversion completed cleanly, the CHANGING_BG_TREE flag is gone, and the incompat BG_TREE bit is now set. Mounting the 11‑device volume takes ~5 seconds instead of several minutes, so everything is back to normal or even better than before.

Thanks a lot for the quick diagnosis and the fix.

One thought: the optimisation that skips the full extent‑tree scan clearly helps in the common case, but when it backfires the only recovery path is recompiling progs with a patch. Would it make sense to expose a --full-scan (or --no-fast-scan) switch in btrfstune so an interrupted conversion can be resumed with stock binaries?

No need for a new option, because it's definitely a bug, and a bug should and will be hunt down.

The only problem is, I still didn't get a full chain on why the last block group in the old tree is skipped.

The old code seems correct, but it isn't. I'll keep digging and send out a proper fix later.

Thanks,
Qu

Quoted 5 lines

Either way, I really appreciate the help.

Best regards,
-Tine

#6 lore raw
Quoted 7 lines · Qu Wenruo
Quoted 4 lines · Tine Mezgec

Hi Qu,

Your patch did the trick, the conversion completed cleanly, the CHANGING_BG_TREE flag is gone, and the incompat BG_TREE bit is now set. Mounting the 11‑device volume takes ~5 seconds instead of several minutes, so everything is back to normal or even better than before.

Quoted 7 lines
Quoted 3 lines

Thanks a lot for the quick diagnosis and the fix.

One thought: the optimisation that skips the full extent‑tree scan clearly helps in the common case, but when it backfires the only recovery path is recompiling progs with a patch. Would it make sense to expose a --full-scan (or --no-fast-scan) switch in btrfstune so an interrupted conversion can be resumed with stock binaries?

No need for a new option, because it's definitely a bug, and a bug should and will be hunt down.

The only problem is, I still didn't get a full chain on why the last block group in the old tree is skipped.

The final update, the proper fix is sent here:
https://lore.kernel.org/linux-btrfs/[email protected]/

The final fix is just a single one line fix, for a stupid uninitialized structure.

But I also added an optimization so that for resume case, the fs opening would benefit a lot from the block group tree.

Thank you for the report and all the help pinning down the problematic function.

Thanks,
Qu

Quoted 13 lines

The old code seems correct, but it isn't. I'll keep digging and send out a proper fix later.

Thanks,
Qu

Quoted 5 lines

Either way, I really appreciate the help.

Best regards,
-Tine