Skip to content

btrfs: improve stalls under sudden writeback

Boris Burkov linux-btrfs 4 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest4 patches 13 repliesIn review
  2. v34 patches 7 repliesSuperseded
  3. v25 patches 6 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v15 patches4 repliesSuperseded

Changes in v4 from the cover letter

- Treat the extent tree data delayed ref as needing reservation for two cow operations.

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4btrfs: reserve space for delayed_refs in delallocfs/btrfs/delalloc-space.c fs/btrfs/delalloc-space.h fs/btrfs/inode.c +1 more+72 -211 reply
2/4btrfs: account for compression in delalloc extent reservationfs/btrfs/btrfs_inode.h fs/btrfs/delalloc-space.c fs/btrfs/fs.h +1 more+72 -330 replies
3/4btrfs: make inode->outstanding_extents a u64fs/btrfs/btrfs_inode.h fs/btrfs/delalloc-space.c fs/btrfs/inode.c +3 more+45 -361 reply
4/4btrfs: cap shrink_delalloc iterations to 128Mfs/btrfs/space-info.c+20 -1110 replies

Reviews trailers per patch

All 4 patches have a Reviewed-by or Acked-by.

PatchFilipe Manana
1/4R-b
2/4R-b
3/4R-b
4/4R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH v4 0/4] btrfs: improve stalls under sudden writeback

If you have a system with very large memory (TiBs) and a normal percentage based dirty_ratio/dirty_background_ratio like the defaults of 20%/10%, then we can theoretically rack up 100s of GiB of dirty pages before doing any writeback. This is further exacerbated if we also see a sudden drop in the free memory due to a large allocation. If we (relatively likely for a large ram system) also have a large disk, we are unlikely to do trigger much preemptive metadata reclaim either.

Once we do start doing writeback with such a large supply, the results are somewhat ugly. The delalloc work generates a huge amount of delayed refs without proper reservations which sends the metadata space system into a tailspin trying to run yet more delalloc to free space. Ultimately, the system stalls waiting for huge amounts of ordered extents and delayed refs blocking all users in start_transaction() on tickets in reserve_space().

This patch series aims to address these issues in a relatively targeted way by improving our reservations for delalloc delayed refs and by doing some very basic smoothing of the work in flush_space(). Further work could be done to improve flush_space() heuristics and latency but this is already a big help on my observed workloads.

I was able to reproduce stalls on a more "modest" system with 264GiB of ram by using a somewhat silly 80% dirty_ratio.

I was unfortunately unable to reproduce any stalls on a yet smaller system with only 32GiB of ram.

The first 2 patches do the delayed_ref rsv accounting on btrfs_inode, mirroring inode->block_rsv.
The 3th patch is a cleanup to the types counting max extents The 4th patch reduces the size of the unit of work in shrink_delalloc() to further reduce stalls.
---
Changelog:
v4:
- Treat the extent tree data delayed ref as needing reservation for two cow operations.
v3:
- Merge csum reservation patch (2) into main delalloc delrefs rsv patch (1)
- Add delayed refs reservations for RST and subvol tree metadata cow to patch 1.
- Do the migration in the nocow/prealloc finish_one_ordered() cases as there are still metadata delayed refs generated.
- Double delref rsv for cows (add+drop). This seems really conservative to me, but I think it is correct. If we like it, it needs to happen more places too...
- Upgrade ASSERTs in patch 3 (old patch 4) to log unexpected values.
- Remove unused return value in migrate function.
- Various stylistic issues in several patches.
v2:
- patch 1 no longer embeds a new block_rsv on btrfs_inode for the delayed reservation. Instead it does the reservation on inode->block_rsv and migrates it to trans->delayed_rsv at the moment of truth.

Boris Burkov (4):
btrfs: reserve space for delayed_refs in delalloc
btrfs: account for compression in delalloc extent reservation
btrfs: make inode->outstanding_extents a u64
btrfs: cap shrink_delalloc iterations to 128M

 fs/btrfs/btrfs_inode.h       | 20 ++++++--
 fs/btrfs/delalloc-space.c    | 79 +++++++++++++++++++++++++------
 fs/btrfs/delalloc-space.h    |  3 ++
 fs/btrfs/fs.h                | 13 ------
 fs/btrfs/inode.c             | 90 ++++++++++++++++++++++++++++--------
 fs/btrfs/ordered-data.c      |  4 +-
 fs/btrfs/space-info.c        | 31 ++++++++-----
 fs/btrfs/tests/inode-tests.c | 18 ++++----
 fs/btrfs/transaction.c       | 36 ++++++---------
 include/trace/events/btrfs.h |  8 ++--
 10 files changed, 205 insertions(+), 97 deletions(-)
Signature · 1 line
2.53.0