Skip to content

btrfs-progs: allow tree-checker to be triggered more frequently for btrfs-convert

Qu Wenruo linux-btrfs 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest3 patches 5 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3btrfs-progs: convert/ext2: new debug environment variable to finetune transaction sizecommon/utils.c common/utils.h convert/source-ext2.c+71 -14 replies
2/3btrfs-progs: new debug environment variable to finetune metadata cache sizekernel-shared/extent_io.c+4 -10 replies
3/3btrfs-progs: convert-tests: trigger tree-checker more frequentlytests/convert-tests/001-ext2-basic/test.sh tests/convert-tests/003-ext4-basic/test.sh+14 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/3] btrfs-progs: allow tree-checker to be triggered more frequently for btrfs-convert

We have issue #731, which is a large ext4 fs, and triggered tree-checker very reliably.

The root cause is the bad write behavior of btrfs-convert, which always insert inode refs/file extents/xattrs first, then the inode item.

Such bad behavior would normally trigger tree-checker, but for our regular convert tests, it doesn't get trigger because:

- We have metadata cache
The default size is system memory / 4, which can be very very large. And written tree blocks would still be cached thus no tree-checker triggered for those cached tree blocks.

- We won't commit transaction for every copied inodes
This means for a lot of cases, we may just commit a large transaction for all inodes, further reduce the chance to trigger tree checker.

This patchset would introduce two debug environment variables:

- BTRFS_PROGS_DEBUG_METADATA_CACHE_SIZE
Global metadata cache size, affecting all btrfs-progs tools that opens an unmounted btrfs.

- BTRFS_PROGS_DEBUG_BLOCKS_USED_THRESHOLD
This only affects ext2 conversion, which determine how many bytes of dirty tree blocks we need before commit a transaction.

Those two variables would affect the speed of btrfs-convert greatly, and we mostly use them for the selftests, thus they won't be documented anyway but the source code.

With those new debug environment variables, we can reduce those values to minimal, so that the existing convert tests are enough to trigger the bug of issue #731.

Although the cost is, we make btrfs-convert test case 001 and 003 much slower than usual (due to much frequent commit transaction commitment and more IO to read tree blocks).

But I'd say, it's still worthy, as long as it can detect bad btrfs-convert behaviors.

Qu Wenruo (3):
btrfs-progs: convert/ext2: new debug environment variable to finetune
transaction size
btrfs-progs: new debug environment variable to finetune metadata cache
size
btrfs-progs: convert-tests: trigger tree-checker more frequently

 common/utils.c                             | 62 ++++++++++++++++++++++
 common/utils.h                             |  1 +
 convert/source-ext2.c                      |  9 +++-
 kernel-shared/extent_io.c                  |  5 +-
 tests/convert-tests/001-ext2-basic/test.sh |  7 +++
 tests/convert-tests/003-ext4-basic/test.sh |  7 +++
 6 files changed, 89 insertions(+), 2 deletions(-)

--
2.43.0