Introduce per-profile available space array to avoid over-confident can_overcommit()
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v9 Latest5 patches 0 repliesNew
- v85 patches 1 replySuperseded
- v75 patches 4 repliesSuperseded
Earlier versions · 7
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v6 posting 2/2 | 5 patches | 6 replies | Superseded | |
| v6 posting 1/2 | 0 of 4 patches | 3 replies | Superseded | |
| v5 | 4 patches | 5 replies | Superseded | |
| v4 | 3 patches | 2 replies | Superseded | |
| v3 | 3 patches | 9 replies | Superseded | |
| v2 | 4 patches | 8 replies | Superseded | |
| v1 | 4 patches | 1 reply | Superseded |
Changes in v9 from the cover letter
- Rebased to latest misc-next branch
Minor conflicts due to chunk allocator refactor
- Remove redundant virtual allocation clear
Patches in v9 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/5 | btrfs: Introduce per-profile available space facilityfs/ fs/ | +204 | 0 replies | |
| 2/5 | btrfs: space-info: Use per-profile available space in can_overcommit()fs/ | +5 | 0 replies | |
| 3/5 | btrfs: statfs: Use pre-calculated per-profile available spacefs/ | +7 | 0 replies | |
| 4/5 | btrfs: Reset device size when btrfs_update_device() failed in btrfs_grow_device()fs/ | +20 | 0 replies | |
| 5/5 | btrfs: volumes: Revert device used bytes when calc_per_profile_avail() failedfs/ | +9 | 0 replies |
Reviews trailers per patch
1 of 5 patches has a Reviewed-by or Acked-by.
| Patch | Josef Bacik |
|---|---|
| 1/5 | |
| 2/5 | R-b |
| 3/5 | |
| 4/5 | |
| 5/5 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH v9 0/5] Introduce per-profile available space array to avoid over-confident can_overcommit()
There are several bug reports of ENOSPC error inbtrfs_run_delalloc_range().
With some extra info from one reporter, it turns out that can_overcommit() is using a wrong way to calculate allocatable metadata space.
The most typical case would look like:
devid 1 unallocated: 1G
devid 2 unallocated: 10G
metadata profile: RAID1
In above case, we can at most allocate 1G chunk for metadata, due to unbalanced disk free space.
But current can_overcommit() uses factor based calculation, which never consider the disk free space balance.
To address this problem, here comes the per-profile available space array, which gets updated every time a chunk get allocated/removed or a device get grown or shrunk.
This provides a quick way for hotter place like can_overcommit() to grab an estimation on how many bytes it can over-commit.
The per-profile available space calculation tries to keep the behavior of chunk allocator, thus it can handle uneven disks pretty well.
And statfs() can also grab that pre-calculated value for instance usage.
Since this patch introduced a new failure pattern, some new error handling are introduced:
- __btrfs_alloc_chunk()
At the end of that function where calc_per_profile_avail() get called, if it failed due to -ENOMEM, we will revert device used space, and remove the allocated chunk.
This is the only new error handling added by patch 5.
- btrfs_remove_chunk()
There is no good way to revert the change. So here we abort transaction, just like what the old error handling does.
- btrfs_grow_device()
This function has its problem by not reverting device used space from the very beginning.
This patchset will enhance it in patch 4.
- btrfs_shrink_device()
- btrfs_init_new_device()
- btrfs_rm_device()
This function already has good error handling, reuse it.
- btrfs_verify_dev_extents()
Mount time error will lead to mount failure, nothing to worry about.
Contents of the patchset:
Patch 1: Core facility, with basic (not perfect) error handling
Patch 2: Fix for over-confident can_overcommit()
Patch 3: Make statfs() more accurate
Patch 4: Better error handling for btrfs_grow_device()
Patch 5: Better error handling for __btrfs_alloc_chunk()
If needed, patch 4 and patch 5 can be merged into patch 1.
Changelog:
v1:
- Fix a bug where we forgot to update per-profile array after allocating a chunk.
To avoid ABBA deadlock, this introduce a small windows at the end __btrfs_alloc_chunk(), it's not elegant but should be good enough before we rework chunk and device list mutex.
- Make statfs() to use virtual chunk allocator to do better estimation Now statfs() can report not only more accurate result, but can also handle RAID5/6 better.
v2:
- Fix a deadlock caused by acquiring device_list_mutex under __btrfs_alloc_chunk()
There is no need to acquire device_list_mutex when holding chunk_mutex.
Fix it and remove the lockdep assert.
v3:
- Use proper chunk_mutex instead of device_list_mutex
Since they are protecting two different things, and we only care about alloc_list, we should only use chunk_mutex.
With improved lock situation, it's easier to fold calc_per_profile_available() calls into the first patch.
- Add performance benchmark for statfs() modification
As Facebook seems to run into some problems with statfs() calls, add some basic ftrace results.
v4:
- Keep the lock-free design for statfs()
As extra sleeping in statfs() may not be a good idea, keep the old lock-free design, and use factor based calculation as fall back.
v5:
- Enhance btrfs_update_device() error handling in btrfs_grow_device()
- Ensure all failure caused by calc_per_profile_available() is the same with existing error handling
- Fix a bug where chunk_mutex is not released in btrfs_shrink_device()
v6:
- Don't update the array if we hit any error.
To avoid calling calc_per_profile_avail() in error handling path.
- Re-order the patchset
Make the core facility the first patch.
Error handling improvement in later patches.
- Add better error handling
Improve one existing bad error handling, and provide a better solution for __btrfs_alloc_chunk()
v7:
- Remove btrfs_calc_avail_data_space() completely
Now we only need to grab the pre-calculated number, no need for a function over 100 lines.
- Keep the 0-avail-if-metadata-exhausted behavior
Now it's handled by space_info->full, which indicates if we can allocate new chunks in metadata space info.
We have no need to bother that now.
v8:
- Cosmetic changes
* Comment fixes
* Use rounddown() to replace one open-code
* while() loop reformat
* Remove one redundant 0-size check
* Add one lockdep_assert() for calc_one_profile_avail()
* Use atomic64_t to remove spinlock
- Add two more timing to call calc_per_profile_avail()
* btrfs_rm_device()
* btrfs_init_new_device()
v9:
- Rebased to latest misc-next branch
Minor conflicts due to chunk allocator refactor
- Remove redundant virtual allocation clear
Qu Wenruo (5):
btrfs: Introduce per-profile available space facility
btrfs: space-info: Use per-profile available space in can_overcommit()
btrfs: statfs: Use pre-calculated per-profile available space
btrfs: Reset device size when btrfs_update_device() failed in
btrfs_grow_device()
btrfs: volumes: Revert device used bytes when calc_per_profile_avail()
failed
fs/btrfs/space-info.c | 14 +-- fs/btrfs/super.c | 131 ++--------------------- fs/btrfs/volumes.c | 242 ++++++++++++++++++++++++++++++++++++++---- fs/btrfs/volumes.h | 10 ++ 4 files changed, 244 insertions(+), 153 deletions(-)
Signature · 1 line
2.26.2