btrfs: delay compression to bbio submission time
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v7 Latest6 patches 2 repliesIn review
- v66 patches 0 repliesSuperseded
- v56 patches 0 repliesSuperseded
Earlier versions · 5
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v4 posting 2/2 | 6 patches | 0 replies | Superseded | |
| v4 posting 1/2 | 6 patches | 5 replies | Superseded | |
| v3 | 6 patches | 1 reply | Superseded | |
| v2 | 6 patches | 7 replies | Superseded | |
| v1 | 6 patches | 6 replies | Superseded |
Changes in v7 from the cover letter
- Fix the error handling of run_delalloc_delayed() Again exposed by Sashiko and looks correct.
Patches in v7 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/6 | btrfs: add delayed ordered extent supportfs/ fs/ fs/ | +243 | 2 replies | |
| 2/6 | btrfs: add skeleton for delayed btrfs biofs/ fs/ fs/ +3 more | +86 | 0 replies | |
| 3/6 | btrfs: introduce the skeleton of delayed bbio endio functionfs/ fs/ fs/ | +90 | 0 replies | |
| 4/6 | btrfs: introduce compression for delayed bbiofs/ | +163 | 0 replies | |
| 5/6 | btrfs: implement uncompressed fallback for delayed bbiofs/ | +162 | 0 replies | |
| 6/6 | btrfs: enable experimental delayed compression supportfs/ fs/ fs/ | +95 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v7 yet.
Cover letter [PATCH v7 0/6] btrfs: delay compression to bbio submission time
[CHANGELOG]
v7:
- Fix the error handling of run_delalloc_delayed() Again exposed by Sashiko and looks correct.
Where other delalloc functions all lock the whole range, then alloc EM/OEs with extent locked.
And the error handling is already assuming the full range has extent lock held.
Follow the common behavior by locking the whole range first.
v6:
- Fix a bitmap race which is not protected
This is reported by Sashiko and looks correct.
There can be multiple bios submitted for the same delayed OE. Thus the child_cleanup_bitmap needs to be protected from races.
Re-use btrfs_inode::ordered_tree_lock to prevent race.
- Release qgroup reserved space on error path
- Release EXTENT_DELALLOC_NEW and EXTENT_DEFRAG flags on error path
- Call btrfs_drop_extent_map_range() during error handling The point here is even if we failed to create the real child EM, there is still a delayed EM created at run_delalloc_delayed().
Since the parent OE cleanup will not cover the error range, we also need to drop the delayed EM.
- Drop a duplicated xmpchg() call
btrfs_bio_end_io() is already doing it for us.
The remaining comments from Sashiko are incorrect:
- Double metadata delalloc releaseing
The out_free_reserve tag of try_submit_compressed() is only reached when an error happens before an OE created.
So there is no child OE to do the cleanup. Furthermore btrfs_removed_ordered_extent() won't release metadata on delayed parent OE.
Thus the situation mentioned in the review is incorrect.
- Data space leak
The data space is already converted from bytes_may_use to bytes_reserved at a succesful btrfs_reserve_extent().
And later on error path btrfs_free_reserved_extent() will decrease bytes_reserved.
Furthermore no parent OE will cleanup the range, so there is no leak on data reserved space.
Although the io tree bits (EXTENT_DELALLOC_NEW and EXTENT_DEFRAG) is still leaking.
v5:
- Fix an error handling path when compression writes failed after reserved data extents
The current fallback behavior is different than the old async path, and can lead to reserved space inbalance.
Follow the old async behavior that if an error is hit after reserved a data extent, error out without falling back to uncompressed writes.
- Fix an error handling path that both parent and failed child OEs are doing the same cleanup twice
If a delayed child OE range hits some critical error without creating an OE, e.g. failed to allocate the child OE itself, the child OE range will be cleaned up.
But later parent OE will also cleanup any range that has no child OE, it will clean the range again.
Fix it by changing the btrfs_ordered_extent::child_bitmap to child_cleanup bitmap, which is initialized to all 1 when allocating the OE, then cleared when:
* A new child OE is added
* A child OE range is already cleaned up
In either try_submit_compressed() or submit_one_uncompressed_range().
So that the parent OE cleanup will skip any range that is covered by an child OE or already cleaned up.
- Fix reserved metadata space leak in child OE error path
If the child OE range failed to allocate an OE, the reserved metadata space is not released, thus will require manual cleanup.
- Fix a lockdep warning where the aquire and release is not paired
- Mark the mapping as error when end_bbio_delayed() hit an error To follow end_bbio_data_write().
v4 RESEND:
- Fix a conflict due to some rewording in for-next Which prevents Sashiko to review the series.
v4:
- Fix a possible write path deadlock during reclaim
The delayed_bio_private structure is allocated during writeback, it needs GFP_NOFS to avoid deadlock.
- Use a dedicated workqueue for WQ_MEM_RECLAIM ability
v3:
- Various fixes to error handling
Mostly reported by LLM.
- Use delayed bbios' pending_io to tracking child bbios Which reduces the structure size of delayed_bio_private.
- Extra full sync requirement for delayed writes
To avoid fast sync path to grab a delayed OE, which has no reliable on-disk bytenr.
v2:
- Rebased to the latest for-next branch Several minor conflicts:
* The removal of folio ordered flag
* The refactor of btrfs_mod_oustanding_extents()
- Fix a random failure in btrfs/260
It turns out that the original filemap_flush() only triggers writeback of dirty pages, but since our new compression happens after the bios are submitted, there can be a race between inode->defrag_compression clearing and compression path reading inode->defrag_compression.
This can cause btrfs to use the mount option other than the specified defrag compression algo to do the compression.
Fix it by using filemap_write_and_wait_range(), which also avoids the quirky double flush behavior.
- Fix a use-after-free bug where bio->bi_status is accessed after bio_put()
- Remove a mapping_set_error() call when try_submit_compressed() failed As we still have uncompressed fallback, we should not set the mapping as error.
- Drop all allocated OEs along with the extent maps when run_delalloc_delayed() failed
- Slightly reword the cover letter
PoC->v1:
- Fix the ordered extent leak caused by incorrect ref count of child OEs
- Fix the reserved space leakage in ranges without a real OE
- Fix the hang caused by incorrect extent lock/unlock pair All exposed by fsstress runs
- Fix the OE range check in btrfs_wait_ordered_extents() that affects snapshot creation
All exposed by fstests runs
Qu Wenruo (6):
btrfs: add delayed ordered extent support
btrfs: add skeleton for delayed btrfs bio
btrfs: introduce the skeleton of delayed bbio endio function
btrfs: introduce compression for delayed bbio
btrfs: implement uncompressed fallback for delayed bbio
btrfs: enable experimental delayed compression support
fs/btrfs/bio.c | 1 + fs/btrfs/bio.h | 7 + fs/btrfs/btrfs_inode.h | 3 + fs/btrfs/defrag.c | 27 +- fs/btrfs/disk-io.c | 8 +- fs/btrfs/extent_io.c | 36 ++- fs/btrfs/extent_map.h | 9 +- fs/btrfs/fs.h | 5 + fs/btrfs/inode.c | 595 +++++++++++++++++++++++++++++++++++++++- fs/btrfs/ordered-data.c | 185 ++++++++++--- fs/btrfs/ordered-data.h | 20 ++ 11 files changed, 830 insertions(+), 66 deletions(-)
Signature · 1 line
2.55.0