btrfs: restrain lock extent usage during writeback
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest17 patches 27 repliesIn review
Patches in v1 No applies recorded yet
Reviews trailers per patch
16 of 17 patches have a Reviewed-by or Acked-by.
| Patch | Johannes Thumshirn | Goldwyn Rodrigues |
|---|---|---|
| 1/17 | R-b | |
| 2/17 | R-b | |
| 3/17 | R-b | |
| 4/17 | R-b | |
| 5/17 | ||
| 6/17 | R-b | |
| 7/17 | R-b | |
| 8/17 | R-b | |
| 9/17 | R-b | |
| 10/17 | R-b | |
| 11/17 | R-b | |
| 12/17 | R-b | |
| 13/17 | R-b | |
| 14/17 | R-b | |
| 15/17 | R-b | |
| 16/17 | R-b | |
| 17/17 | R-b R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH 00/17] btrfs: restrain lock extent usage during writeback
Hello,
We use the extent lock to cover a pretty wide range of operations during writeback, and this has made things like the iomap conversion more annoying. It's also relatively complex how the rules work, and in fact are different between compressed and normal writeback.
This series is meant to address this by pushing the extent lock down to cover a discreet set of operations. Being able to clearly define what the extent lock is protecting will give us a better idea of how to reduce it's usage or possibly replace it in the future.
The current state of affairs has been we lock the page for the range we're going to write back, and then we lock the extent, and then we start writeback. This looks a few different ways
COW:
lock page
lock extent
allocate the extent
insert the pinned extent
allocate the ordered extent
clear DELALLOC
unlock extent
unlock page
NOCOW
lock page
lock extent
lookup the file extent
check if the file extent can be used nocow
if no:
cow_file_range
if yes:
insert pinned extent if prealloc
allocate ordered extent
clear delalloc
unlock extent
unlock page
COMPRESS
lock page
lock extent
unlock extent
create async objects for the page ranges
compress the pages
submit the compressed extents
lock extent
allocate the extent
insert the pinned extent
allocate the ordered extent
clear DELALLOC
unlock extent
unlock pages
As you can see there's some variations above, but in essence the extent lock protects
- Calling into the allocator.
- The em_tree modifications (pinned extent).
- Allocating and attaching the ordered extent.
- Clearing the various bits on the extent_io_tree.
- Looking up the file extent items on disk in the NOCOW case.
We don't actually need to protect the allocator when we're allocating, that has it's own locking, we're not gaining anything from the lock being held there.
We also don't need to protect looking up the file extent items in the NOCOW case, we are protected by the page lock here. There are 3 cases we would need to be worried about here
- Prealloc. This locks the inode and calls btrfs_wait_ordered_range() before doing the preallocation, so there will be no dirty pages in this range before it begins the preallocation.
- Zero range. Same as above.
- O_DIRECT. We invalidate the page range before we start the write, so again we're protected here.
This allows us to reduce the scope of the extent lock to the 3 things that really matter
- Protecting the em tree modifications. This makes sense, this is directly related to the inode range.
- Protecting the extent_io_tree modifications. Again this is clear.
- Allocating and attaching the ordered extent. Also clear.
With this in mind we can push the extent locking down into the functions that do these operations, and drastically simplify the locking that we're doing here.
I've run this through several runs of the CI to validate that this doesn't break anything. Nothing has failed yet, but it is a rather scary looking change, so a sharp review is absolutely necessary, and having a decent soak time. I would like to get this merged ASAP so we can start pounding on it and make sure it's solid before I attempt other locking related changes. Thanks,
Josef
Josef Bacik (17):
btrfs: handle errors in btrfs_reloc_clone_csums properly
btrfs: push all inline logic into cow_file_range
btrfs: unlock all the pages with successful inline extent creation
btrfs: move extent bit and page cleanup into cow_file_range_inline
btrfs: lock extent when doing inline extent in compression
btrfs: push the extent lock into btrfs_run_delalloc_range
btrfs: push extent lock into run_delalloc_nocow
btrfs: adjust while loop condition in run_delalloc_nocow
btrfs: push extent lock down in run_delalloc_nocow
btrfs: remove unlock_extent from run_delalloc_compressed
btrfs: push extent lock into run_delalloc_cow
btrfs: push extent lock into cow_file_range
btrfs: push lock_extent into cow_file_range_inline
btrfs: move can_cow_file_range_inline() outside of the extent lock
btrfs: push lock_extent down in cow_file_range()
btrfs: push extent lock down in submit_one_async_extent
btrfs: add a cached state to extent_clear_unlock_delalloc
fs/btrfs/extent_io.c | 8 +- fs/btrfs/extent_io.h | 2 + fs/btrfs/inode.c | 279 +++++++++++++++++++++++----------------- fs/btrfs/ordered-data.c | 6 + fs/btrfs/ordered-data.h | 1 + fs/btrfs/relocation.c | 4 +- 6 files changed, 177 insertions(+), 123 deletions(-)
Signature · 1 line
2.43.0