Skip to contentSkip to conversation

btrfs-progs: convert: Prevent bit overflow for cctx->total_bytes

PATCH In review Qu Wenruo 6 participants 17 messages Jul 20, 2020 – Jul 29, 2020 Btrfs file system

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

PATCH 1/2 btrfs-progs: convert: Prevent bit overflow for cctx->total_bytes

[BUG]
When convert is called on a 64GiB ext4 fs, it fails like this:

  $ btrfs-convert  /dev/loop0p1
  create btrfs filesystem:
          blocksize: 4096
          nodesize:  16384
          features:  extref, skinny-metadata (default)
          checksum:  crc32c
  creating ext2 image file
  ERROR: missing data block for bytenr 1048576
  ERROR: failed to create ext2_saved/image: -2
  WARNING: an error occurred during conversion, filesystem is partially created but not finalized and not mountable

Btrfs-convert also corrupts the source fs:

  $ LANG=C e2fsck /dev/loop0p1 -f
  e2fsck 1.45.6 (20-Mar-2020)
  Resize inode not valid.  Recreate<y>? yes
  Pass 1: Checking inodes, blocks, and sizes
  Deleted inode 3681 has zero dtime.  Fix<y>? yes
  Inodes that were part of a corrupted orphan linked list found.  Fix<y>? yes
  Inode 3744 was part of the orphaned inode list.  FIXED.
  Deleted inode 3745 has zero dtime.  Fix<y>? yes
  Inode 3747 has INLINE_DATA_FL flag on filesystem without inline data support.
  Clear<y>? yes
  ...

[CAUSE]
After some debugging, the first strange behavior is, the value of cctx->total_bytes is 0 in ext2_open_fs().

It turns out that, the value assign for cctx->total_bytes could lead to bit overflow for the unsigned int value.

And that 0 cctx->total_bytes leads to vairous problems for later free space calculation.
For example, in calculate_available_space(), we use cctx->total_bytes to ensure we won't create a data chunk beyond device end:

		cue_len = min(cctx->total_bytes - cur_off, cur_len);

If that cur_offset is also 0, we will create a cache_extent with 0 size, which could cause a lot of problems for cache tree search.

[FIX]
Do manual casting for the multiply operation, so we could got a real u64 result.
The fix will be applied to all supported fses (ext* and reiserfs).

---
 convert/main.c            | 1 +
 convert/source-ext2.c     | 3 ++-
 convert/source-reiserfs.c | 2 +-
 3 files changed, 4 insertions(+), 2 deletions(-)
convert/main.c +1 -0
index 7709e9a6c085..df6a2ae32722 100644
@@ -1136,6 +1136,7 @@ static int do_convert(const char *devname, u32 convert_flags, u32 nodesize,
 	if (ret)
 		goto fail;
 
+	ASSERT(cctx.total_bytes);
 	blocksize = cctx.blocksize;
 	total_bytes = (u64)blocksize * (u64)cctx.block_count;
 	if (blocksize < 4096) {
convert/source-ext2.c +2 -1
index f11ef651245a..a5e3a726863f 100644
@@ -87,7 +87,8 @@ static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name)
 	cctx->fs_data = ext2_fs;
 	cctx->blocksize = ext2_fs->blocksize;
 	cctx->block_count = ext2_fs->super->s_blocks_count;
-	cctx->total_bytes = ext2_fs->blocksize * ext2_fs->super->s_blocks_count;
+	cctx->total_bytes = (u64)ext2_fs->blocksize *
+			    (u64)ext2_fs->super->s_blocks_count;
 	cctx->volume_name = strndup((char *)ext2_fs->super->s_volume_name, 16);
 	cctx->first_data_block = ext2_fs->super->s_first_data_block;
 	cctx->inodes_count = ext2_fs->super->s_inodes_count;
convert/source-reiserfs.c +1 -1
index 9fd6b9abb9b4..8d9752f06ca9 100644
@@ -82,7 +82,7 @@ static int reiserfs_open_fs(struct btrfs_convert_context *cxt, const char *name)
 	cxt->fs_data = fs;
 	cxt->blocksize = fs->fs_blocksize;
 	cxt->block_count = get_sb_block_count(fs->fs_ondisk_sb);
-	cxt->total_bytes = cxt->blocksize * cxt->block_count;
+	cxt->total_bytes = (u64)cxt->blocksize * (u64)cxt->block_count;
 	cxt->volume_name = strndup(fs->fs_ondisk_sb->s_label, 16);
 	cxt->first_data_block = 0;
 	cxt->inodes_count = reiserfs_count_objectids(fs);
Signature · 1 line
2.27.0
author
#1 lore raw

PATCH 2/2 btrfs-progs: convert-tests: Add test case for multiply overflow

The new test case will test whether btrfs-convert can handle 64G ext* fs.

This exercise the cctx->total_bytes calculation where multiplying 4K (unsigned int) and 16777216 (u32) could lead to bit overflow for unsigned int and got 0, and screw up later free space calculation.

---
 .../018-fs-size-overflow/test.sh              | 22 +++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100755 tests/convert-tests/018-fs-size-overflow/test.sh
tests/convert-tests/018-fs-size-overflow/test.sh new file +22 -0
new file mode 100755
index 000000000000..2335225445f8
@@ -0,0 +1,22 @@
+#!/bin/bash
+# Check if btrfs-convert can handle an ext4 fs whose size is 64G.
+# That fs size could trigger a multiply overflow and screw up free space
+# calculation
+
+source "$TEST_TOP/common"
+source "$TEST_TOP/common.convert"
+
+setup_root_helper
+prepare_test_dev
+check_prereq btrfs-convert
+check_global_prereq mke2fs
+check_global_prereq truncate
+
+truncate -s 64g $TEST_DEV
+
+convert_test_prep_fs ext4 mke2fs -t ext4 -b 4096
+run_check_umount_test_dev
+
+# Unpatched btrfs-convert would fail half way due to corrupted free space
+# cache tree
+convert_test_do_convert
Signature · 1 line
2.27.0
Reviewed-by on v1 1/2
#2 lore raw
In reply to Qu Wenruo
Quoted 44 lines · Qu Wenruo

[BUG]
When convert is called on a 64GiB ext4 fs, it fails like this:

  $ btrfs-convert  /dev/loop0p1
  create btrfs filesystem:
          blocksize: 4096
          nodesize:  16384
          features:  extref, skinny-metadata (default)
          checksum:  crc32c
  creating ext2 image file
  ERROR: missing data block for bytenr 1048576
  ERROR: failed to create ext2_saved/image: -2
  WARNING: an error occurred during conversion, filesystem is partially created but not finalized and not mountable

Btrfs-convert also corrupts the source fs:

  $ LANG=C e2fsck /dev/loop0p1 -f
  e2fsck 1.45.6 (20-Mar-2020)
  Resize inode not valid.  Recreate<y>? yes
  Pass 1: Checking inodes, blocks, and sizes
  Deleted inode 3681 has zero dtime.  Fix<y>? yes
  Inodes that were part of a corrupted orphan linked list found.  Fix<y>? yes
  Inode 3744 was part of the orphaned inode list.  FIXED.
  Deleted inode 3745 has zero dtime.  Fix<y>? yes
  Inode 3747 has INLINE_DATA_FL flag on filesystem without inline data support.
  Clear<y>? yes
  ...

[CAUSE]
After some debugging, the first strange behavior is, the value of cctx->total_bytes is 0 in ext2_open_fs().

It turns out that, the value assign for cctx->total_bytes could lead to bit overflow for the unsigned int value.

And that 0 cctx->total_bytes leads to vairous problems for later free space calculation.
For example, in calculate_available_space(), we use cctx->total_bytes to ensure we won't create a data chunk beyond device end:

		cue_len = min(cctx->total_bytes - cur_off, cur_len);

If that cur_offset is also 0, we will create a cache_extent with 0 size, which could cause a lot of problems for cache tree search.

Quoted 7 lines

[FIX]
Do manual casting for the multiply operation, so we could got a real u64 result.
The fix will be applied to all supported fses (ext* and reiserfs).

#3 lore raw
In reply to Qu Wenruo

convert/source-ext2.c static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name) · line 91

Quoted 9 lines · Qu Wenruo
convert/source-ext2.c +2 -1
@@ -87,7 +87,8 @@ static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name)
 	cctx->fs_data = ext2_fs;
 	cctx->blocksize = ext2_fs->blocksize;
 	cctx->block_count = ext2_fs->super->s_blocks_count;
-	cctx->total_bytes = ext2_fs->blocksize * ext2_fs->super->s_blocks_count;
+	cctx->total_bytes = (u64)ext2_fs->blocksize *
+			    (u64)ext2_fs->super->s_blocks_count;

Do you need to cast both? Once one of the types is wide enough for the result, there should be no loss.

#4 lore raw
Quoted 14 lines · David Sterba

convert/source-ext2.c static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name) · line 91

Quoted 9 lines · Qu Wenruo
convert/source-ext2.c +2 -1
@@ -87,7 +87,8 @@ static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name)
 	cctx->fs_data = ext2_fs;
 	cctx->blocksize = ext2_fs->blocksize;
 	cctx->block_count = ext2_fs->super->s_blocks_count;
-	cctx->total_bytes = ext2_fs->blocksize * ext2_fs->super->s_blocks_count;
+	cctx->total_bytes = (u64)ext2_fs->blocksize *
+			    (u64)ext2_fs->super->s_blocks_count;

Do you need to cast both? Once one of the types is wide enough for the result, there should be no loss.

I just want to be extra safe.

Feel free to reduce one.

Thanks,
Qu

#5 lore raw
Quoted 17 lines · Qu Wenruo
Quoted 14 lines · David Sterba

convert/source-ext2.c static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name) · line 91

Quoted 9 lines · Qu Wenruo
convert/source-ext2.c +2 -1
@@ -87,7 +87,8 @@ static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name)
 	cctx->fs_data = ext2_fs;
 	cctx->blocksize = ext2_fs->blocksize;
 	cctx->block_count = ext2_fs->super->s_blocks_count;
-	cctx->total_bytes = ext2_fs->blocksize * ext2_fs->super->s_blocks_count;
+	cctx->total_bytes = (u64)ext2_fs->blocksize *
+			    (u64)ext2_fs->super->s_blocks_count;

Do you need to cast both? Once one of the types is wide enough for the result, there should be no loss.

Quoted 1 line

I just want to be extra safe.

Typecasts in code raise questions why are they needed, 'to be extra' safe is not a good reason. One typecast in multiplication/shifts is a common pattern to widen the result but two look more like lack of understanding of the integer promotion rules.

author
#6 lore raw
Quoted 18 lines · David Sterba
Quoted 17 lines · Qu Wenruo
Quoted 14 lines · David Sterba

convert/source-ext2.c static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name) · line 91

Quoted 9 lines · Qu Wenruo
convert/source-ext2.c +2 -1
@@ -87,7 +87,8 @@ static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name)
 	cctx->fs_data = ext2_fs;
 	cctx->blocksize = ext2_fs->blocksize;
 	cctx->block_count = ext2_fs->super->s_blocks_count;
-	cctx->total_bytes = ext2_fs->blocksize * ext2_fs->super->s_blocks_count;
+	cctx->total_bytes = (u64)ext2_fs->blocksize *
+			    (u64)ext2_fs->super->s_blocks_count;

Do you need to cast both? Once one of the types is wide enough for the result, there should be no loss.

Quoted 7 lines
Quoted 1 line

I just want to be extra safe.

Typecasts in code raise questions why are they needed, 'to be extra' safe is not a good reason. One typecast in multiplication/shifts is a common pattern to widen the result but two look more like lack of understanding of the integer promotion rules.

My point here is, I don't want the reviewers or new contributors to bother about the promotion rules at all.

They only need to know that using blocksize and blocks_count directly to do multiply would lead to overflow.

Other details like whether the multiply follows the highest factor or the left operator or the right operator, shouldn't be the point and we don't really need to bother.

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

Thanks,
Qu

#7 lore raw
Quoted 21 lines · Qu Wenruo
Quoted 20 lines · David Sterba
Quoted 18 lines · Qu Wenruo
Quoted 14 lines · David Sterba

convert/source-ext2.c static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name) · line 91

Quoted 9 lines · Qu Wenruo
convert/source-ext2.c +2 -1
@@ -87,7 +87,8 @@ static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name)
 	cctx->fs_data = ext2_fs;
 	cctx->blocksize = ext2_fs->blocksize;
 	cctx->block_count = ext2_fs->super->s_blocks_count;
-	cctx->total_bytes = ext2_fs->blocksize * ext2_fs->super->s_blocks_count;
+	cctx->total_bytes = (u64)ext2_fs->blocksize *
+			    (u64)ext2_fs->super->s_blocks_count;

Do you need to cast both? Once one of the types is wide enough for the result, there should be no loss.

I just want to be extra safe.

Quoted 7 lines
Quoted 4 lines

Typecasts in code raise questions why are they needed, 'to be extra' safe is not a good reason. One typecast in multiplication/shifts is a common pattern to widen the result but two look more like lack of understanding of the integer promotion rules.

My point here is, I don't want the reviewers or new contributors to bother about the promotion rules at all.

Ouch, I hope you don't mean that contributors should ignore the trickier parts of C language. Especially reviewers _have_ to bother about all sorts of subtle behaviour.

Quoted 6 lines

They only need to know that using blocksize and blocks_count directly to do multiply would lead to overflow.

Other details like whether the multiply follows the highest factor or the left operator or the right operator, shouldn't be the point and we don't really need to bother.

... and introduce bugs?

Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

The opposite, if you send me code that's not following known schemes or idiomatic schemes I'll be highly suspicious and looking for the reasons why it's that way and making sure it's correct costs way more time.

#8 lore raw
Quoted 41 lines · David Sterba
Quoted 21 lines · Qu Wenruo
Quoted 20 lines · David Sterba
Quoted 18 lines · Qu Wenruo
Quoted 14 lines · David Sterba

convert/source-ext2.c static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name) · line 91

Quoted 9 lines · Qu Wenruo
convert/source-ext2.c +2 -1
@@ -87,7 +87,8 @@ static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name)
 	cctx->fs_data = ext2_fs;
 	cctx->blocksize = ext2_fs->blocksize;
 	cctx->block_count = ext2_fs->super->s_blocks_count;
-	cctx->total_bytes = ext2_fs->blocksize * ext2_fs->super->s_blocks_count;
+	cctx->total_bytes = (u64)ext2_fs->blocksize *
+			    (u64)ext2_fs->super->s_blocks_count;

Do you need to cast both? Once one of the types is wide enough for the result, there should be no loss.

I just want to be extra safe.

Quoted 7 lines
Quoted 4 lines

Typecasts in code raise questions why are they needed, 'to be extra' safe is not a good reason. One typecast in multiplication/shifts is a common pattern to widen the result but two look more like lack of understanding of the integer promotion rules.

My point here is, I don't want the reviewers or new contributors to bother about the promotion rules at all.

Ouch, I hope you don't mean that contributors should ignore the trickier parts of C language. Especially reviewers _have_ to bother about all sorts of subtle behaviour.

Quoted 6 lines

They only need to know that using blocksize and blocks_count directly to do multiply would lead to overflow.

Other details like whether the multiply follows the highest factor or the left operator or the right operator, shouldn't be the point and we don't really need to bother.

Quoted 9 lines

... and introduce bugs?

Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

The opposite, if you send me code that's not following known schemes or idiomatic schemes I'll be highly suspicious and looking for the reasons why it's that way and making sure it's correct costs way more time.

OK, then would you please remove one casting at merge time, or do I need to resend?

Thanks,
Qu

#9 lore raw
Quoted 9 lines · Qu Wenruo
Quoted 7 lines
Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

The opposite, if you send me code that's not following known schemes or idiomatic schemes I'll be highly suspicious and looking for the reasons why it's that way and making sure it's correct costs way more time.

OK, then would you please remove one casting at merge time, or do I need to resend?

Yeah, I fix such things routinely no need to resend.

#10 lore raw
Quoted 5 lines · David Sterba
Quoted 3 lines · Qu Wenruo
Quoted 3 lines
Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

The opposite, if you send me code that's not following known schemes or idiomatic schemes I'll be highly suspicious and looking for the reasons why it's that way and making sure it's correct costs way more time.

OK, then would you please remove one casting at merge time, or do I need to resend?

Yeah, I fix such things routinely no need to resend.

I have a report[1] that seems to look like this patch solves it, is that correct?

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1851674#c7

Signature · 1 line
真実はいつも一つ!/ Always, there's only one truth!
#11 lore raw
Quoted 13 lines · Neal Gompa
Quoted 12 lines · David Sterba
Quoted 9 lines · Qu Wenruo
Quoted 7 lines
Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

The opposite, if you send me code that's not following known schemes or idiomatic schemes I'll be highly suspicious and looking for the reasons why it's that way and making sure it's correct costs way more time.

OK, then would you please remove one casting at merge time, or do I need to resend?

Quoted 7 lines
Quoted 1 line

Yeah, I fix such things routinely no need to resend.

I have a report[1] that seems to look like this patch solves it, is that correct?

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1851674#c7

Yep, looks like the same bug.

Thanks,
Qu

#12 lore raw
Quoted 17 lines · Qu Wenruo
Quoted 13 lines · Neal Gompa
Quoted 12 lines · David Sterba
Quoted 9 lines · Qu Wenruo
Quoted 7 lines
Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

The opposite, if you send me code that's not following known schemes or idiomatic schemes I'll be highly suspicious and looking for the reasons why it's that way and making sure it's correct costs way more time.

OK, then would you please remove one casting at merge time, or do I need to resend?

Quoted 9 lines
Quoted 7 lines
Quoted 1 line

Yeah, I fix such things routinely no need to resend.

I have a report[1] that seems to look like this patch solves it, is that correct?

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1851674#c7

Yep, looks like the same bug.

So I backported this fix into btrfs-progs-5.7-4.fc32[1], and the reporter is still seeing issues[2].

Pasting from the bug comment[2]:

[liveuser@localhost-live ~]$ sudo btrfs-convert /dev/sda2 create btrfs filesystem:

    blocksize: 4096
    nodesize:  16384
    features:  extref, skinny-metadata (default)
    checksum:  crc32c

creating ext2 image file
creating btrfs metadata
convert/source-ext2.c:845: ext2_copy_inodes: BUG_ON ret triggered, value -28 btrfs-convert(+0x13589)[0x558cfe4a6589] btrfs-convert(+0x14549)[0x558cfe4a7549] btrfs-convert(main+0xfc0)[0x558cfe4a22b0] /lib64/libc.so.6(__libc_start_main+0xf2)[0x7fd2c6666042] btrfs-convert(_start+0x2e)[0x558cfe4a35ce]
Aborted

Any idea what's going on here?

[1]: https://src.fedoraproject.org/rpms/btrfs-progs/c/e4a3d39e87313954cb3e8214e28ee0ba50bee874
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1851674#c13

Signature · 1 line
真実はいつも一つ!/ Always, there's only one truth!
#13 lore raw
Quoted 35 lines · Neal Gompa
Quoted 17 lines · Qu Wenruo
Quoted 13 lines · Neal Gompa
Quoted 12 lines · David Sterba
Quoted 9 lines · Qu Wenruo
Quoted 7 lines
Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

The opposite, if you send me code that's not following known schemes or idiomatic schemes I'll be highly suspicious and looking for the reasons why it's that way and making sure it's correct costs way more time.

OK, then would you please remove one casting at merge time, or do I need to resend?

Quoted 9 lines
Quoted 7 lines
Quoted 1 line

Yeah, I fix such things routinely no need to resend.

I have a report[1] that seems to look like this patch solves it, is that correct?

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1851674#c7

Yep, looks like the same bug.

So I backported this fix into btrfs-progs-5.7-4.fc32[1], and the reporter is still seeing issues[2].

Pasting from the bug comment[2]:

[liveuser@localhost-live ~]$ sudo btrfs-convert /dev/sda2 create btrfs filesystem:

Quoted 7 lines
    blocksize: 4096
    nodesize:  16384
    features:  extref, skinny-metadata (default)
    checksum:  crc32c

creating ext2 image file
creating btrfs metadata
convert/source-ext2.c:845: ext2_copy_inodes: BUG_ON ret triggered, value -28

This means we have no space left.

We don't know if it's the fs already exhausted (little space left for EXT*), or it's btrfs' extent allocator not working.

Would it possible to update the image?

BTW, even btrfs-convert crashed, the fs should be completely fine, just as nothing happened to it (from the point of view of ext*)

Thanks,
Qu

Quoted 13 lines

btrfs-convert(+0x13589)[0x558cfe4a6589] btrfs-convert(+0x14549)[0x558cfe4a7549] btrfs-convert(main+0xfc0)[0x558cfe4a22b0] /lib64/libc.so.6(__libc_start_main+0xf2)[0x7fd2c6666042] btrfs-convert(_start+0x2e)[0x558cfe4a35ce] Aborted

Any idea what's going on here?

[1]: https://src.fedoraproject.org/rpms/btrfs-progs/c/e4a3d39e87313954cb3e8214e28ee0ba50bee874
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1851674#c13

#14 lore raw
Quoted 47 lines · Qu Wenruo
Quoted 35 lines · Neal Gompa
Quoted 17 lines · Qu Wenruo
Quoted 13 lines · Neal Gompa
Quoted 12 lines · David Sterba
Quoted 9 lines · Qu Wenruo
Quoted 7 lines
Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

The opposite, if you send me code that's not following known schemes or idiomatic schemes I'll be highly suspicious and looking for the reasons why it's that way and making sure it's correct costs way more time.

OK, then would you please remove one casting at merge time, or do I need to resend?

Quoted 9 lines
Quoted 7 lines
Quoted 1 line

Yeah, I fix such things routinely no need to resend.

I have a report[1] that seems to look like this patch solves it, is that correct?

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1851674#c7

Yep, looks like the same bug.

So I backported this fix into btrfs-progs-5.7-4.fc32[1], and the reporter is still seeing issues[2].

Pasting from the bug comment[2]:

[liveuser@localhost-live ~]$ sudo btrfs-convert /dev/sda2 create btrfs filesystem:

Quoted 7 lines
    blocksize: 4096
    nodesize:  16384
    features:  extref, skinny-metadata (default)
    checksum:  crc32c

creating ext2 image file
creating btrfs metadata
convert/source-ext2.c:845: ext2_copy_inodes: BUG_ON ret triggered, value -28

Quoted 7 lines

This means we have no space left.

We don't know if it's the fs already exhausted (little space left for EXT*), or it's btrfs' extent allocator not working.

Would it possible to update the image?

I'm not sure what you're asking here? Do you mean update the live environment? You can boot a Fedora 32 live environment and update the btrfs-progs package before using it like the bug reporter did...

Quoted 3 lines

BTW, even btrfs-convert crashed, the fs should be completely fine, just as nothing happened to it (from the point of view of ext*)

I figured as much, but we shouldn't have a case where btrfs-convert crashes like this...

--
真実はいつも一つ!/ Always, there's only one truth!

#15 lore raw
Quoted 53 lines · Neal Gompa
Quoted 52 lines · Qu Wenruo
Quoted 35 lines · Neal Gompa
Quoted 17 lines · Qu Wenruo
Quoted 13 lines · Neal Gompa
Quoted 12 lines · David Sterba
Quoted 9 lines · Qu Wenruo
Quoted 7 lines
Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

The opposite, if you send me code that's not following known schemes or idiomatic schemes I'll be highly suspicious and looking for the reasons why it's that way and making sure it's correct costs way more time.

OK, then would you please remove one casting at merge time, or do I need to resend?

Quoted 9 lines
Quoted 7 lines
Quoted 1 line

Yeah, I fix such things routinely no need to resend.

I have a report[1] that seems to look like this patch solves it, is that correct?

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1851674#c7

Yep, looks like the same bug.

So I backported this fix into btrfs-progs-5.7-4.fc32[1], and the reporter is still seeing issues[2].

Pasting from the bug comment[2]:

[liveuser@localhost-live ~]$ sudo btrfs-convert /dev/sda2 create btrfs filesystem:

Quoted 7 lines
    blocksize: 4096
    nodesize:  16384
    features:  extref, skinny-metadata (default)
    checksum:  crc32c

creating ext2 image file
creating btrfs metadata
convert/source-ext2.c:845: ext2_copy_inodes: BUG_ON ret triggered, value -28

This means we have no space left.

We don't know if it's the fs already exhausted (little space left for EXT*), or it's btrfs' extent allocator not working.

Quoted 6 lines
Quoted 2 lines

Would it possible to update the image?

I'm not sure what you're asking here? Do you mean update the live environment? You can boot a Fedora 32 live environment and update the btrfs-progs package before using it like the bug reporter did...

I mean, upload the binary dump or e2image dump of /dev/sda2. e2image -Q would be much smaller and without any data stored in it.

Sometimes that's the easiest way to debug.
Or we need to add tons of probes to btrfs-convert and let the reporter to try-and-report to pin down the bug.

Quoted 7 lines
Quoted 4 lines

BTW, even btrfs-convert crashed, the fs should be completely fine, just as nothing happened to it (from the point of view of ext*)

I figured as much, but we shouldn't have a case where btrfs-convert crashes like this...

Yep, the BUG_ON() should be removed and replaced with a more graceful exit.

I'll do that soon.

Thanks,
Qu

#16 lore raw
In reply to Qu Wenruo
Quoted 21 lines · Qu Wenruo
Quoted 20 lines · David Sterba
Quoted 18 lines · Qu Wenruo
Quoted 14 lines · David Sterba

convert/source-ext2.c static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name) · line 91

Quoted 9 lines · Qu Wenruo
convert/source-ext2.c +2 -1
@@ -87,7 +87,8 @@ static int ext2_open_fs(struct btrfs_convert_context *cctx, const char *name)
 	cctx->fs_data = ext2_fs;
 	cctx->blocksize = ext2_fs->blocksize;
 	cctx->block_count = ext2_fs->super->s_blocks_count;
-	cctx->total_bytes = ext2_fs->blocksize * ext2_fs->super->s_blocks_count;
+	cctx->total_bytes = (u64)ext2_fs->blocksize *
+			    (u64)ext2_fs->super->s_blocks_count;

Do you need to cast both? Once one of the types is wide enough for the result, there should be no loss.

I just want to be extra safe.

Quoted 8 lines
Quoted 5 lines

Typecasts in code raise questions why are they needed, 'to be extra' safe is not a good reason. One typecast in multiplication/shifts is a common pattern to widen the result but two look more like lack of understanding of the integer promotion rules.

My point here is, I don't want the reviewers or new contributors to bother about the promotion rules at all.

Reviewers and contributors need to understand the rules of C.

Quoted 6 lines

They only need to know that using blocksize and blocks_count directly to do multiply would lead to overflow.

Other details like whether the multiply follows the highest factor or the left operator or the right operator, shouldn't be the point and we don't really need to bother.

This is like suggesting to use brackets on every int-expression like ((5*3)+7) because the precedere rules are too complex. Do you think that writing ((5*3)+7) is "extra safe"?

Quoted 2 lines

Thus casting both would definitely be right, without the need to refer to the complex rule book, thus save the reviewer several minutes.

I don't think so.

Signature · 2 lines
  ciao - 
    Stefan