Skip to contentSkip to conversation

data chunk_size ceiling and shrinking stripe sizes on large RAID1/10 arrays

DISCUSSION Forza 2 participants 2 messages May 23, 2025 Btrfs file system

lore
Share & tools
lore

Forza

author
#0 lore raw

Hi all,

Because the default

/sys/fs/btrfs/<fsid>/allocation/data/chunk_size

value is capped at 10 GiB (and cannot be raised), the allocator’s logic in decide_stripe_size_regular() progressively reduces the per-device stripe_size once data_stripes * stripe_size would exceed that 10 GiB ceiling.

Concrete example:

40-device RAID10 → data_stripes = 20, allocator shrinks stripe_size to ≈ 512 MiB

Functionally, I think everything works, but the result is a much smaller stripe per device than the customary 1 GiB. With even more drives the stripe continues to decrease.

Questions

1. Is this behaviour considered acceptable or just a stop-gap until a better solution is developed? The limit seems.to have been introduced in https://github.com/torvalds/linux/commit/fce466eab7ac6baa9d2dcd88abcf945be3d4a089 (btrfs: tree-checker: Verify block_group_item) but with no direct explanation to why 10GiB was chosen.

2. What are the risks if we increase BTRFS_MAX_DATA_CHUNK_SIZE limit beyond 10GiB?

3. Has anyone measured performance impacts once stripe_size drops below 1 GiB? Are there other issues that may happen on such a filesystem?

Thanks!

~ Forza

#1 lore raw
Quoted 11 lines · Forza

Hi all,

Because the default

/sys/fs/btrfs/<fsid>/allocation/data/chunk_size

value is capped at 10 GiB (and cannot be raised), the allocator’s logic in decide_stripe_size_regular() progressively reduces the per-device stripe_size once data_stripes * stripe_size would exceed that 10 GiB ceiling.

Concrete example:

40-device RAID10 → data_stripes = 20, allocator shrinks stripe_size to ≈ 512 MiB

40 devices, this is already something we do not expect.

Maybe limiting the device number of a RAID0/RAID5/RAID6 chunk to 10/11/12 will be better?

Quoted 6 lines

Functionally, I think everything works, but the result is a much smaller stripe per device than the customary 1 GiB. With even more drives the stripe continues to decrease.

Questions

1. Is this behaviour considered acceptable or just a stop-gap until a better solution is developed? The limit seems.to have been introduced in https://github.com/torvalds/linux/commit/fce466eab7ac6baa9d2dcd88abcf945be3d4a089 (btrfs: tree-checker: Verify block_group_item) but with no direct explanation to why 10GiB was chosen.

The 10GiB is a long existing limits, mostly from the beginning of btrfs, thus that tree check patch just keeps it, and that's also a good indicator of obviously wrong chunk size.

Quoted 2 lines

2. What are the risks if we increase BTRFS_MAX_DATA_CHUNK_SIZE limit beyond 10GiB?

Further slowing down balance I believe.

Especially the balance itself is single threaded read, won't fully utilize all that many devices in RAID10.

Quoted 2 lines

3. Has anyone measured performance impacts once stripe_size drops below 1 GiB? Are there other issues that may happen on such a filesystem?

No real world benchmark yet, but I guess it should be no big deal, as the stripe size is also limited by 10% of the device size.

Thus for fs < 10 GiB, their stripe size is already smaller than 1GiB.

Thanks,
Qu

Quoted 6 lines

Thanks!

~ Forza