Skip to content

btrfs: scrub: improve the scrub performance

Qu Wenruo linux-btrfs 2 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest5 patches 2 repliesIn review
  2. v15 patches 10 repliesSuperseded

Changes in v2 from the cover letter

- Fix a double accounting error in the last patch scrub_stripe_report_errors() is called twice, thus doubling the accounting.

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5btrfs: scrub: avoid unnecessary extent tree search preparing stripesfs/btrfs/scrub.c+29 -120 replies
2/5btrfs: scrub: avoid unnecessary csum tree search preparing stripesfs/btrfs/file-item.c fs/btrfs/file-item.h fs/btrfs/raid56.c +1 more+45 -270 replies
3/5btrfs: scrub: fix grouping of read IOfs/btrfs/scrub.c+72 -230 replies
4/5btrfs: scrub: don't go ordered workqueue for dev-replacefs/btrfs/scrub.c+3 -70 replies
5/5btrfs: scrub: move write back of repaired sectors into scrub_stripe_read_repair_worker()fs/btrfs/scrub.c+27 -470 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [PATCH v2 0/5] btrfs: scrub: improve the scrub performance

[REPO]
https://github.com/adam900710/linux/tree/scrub_testing

[CHANGELOG]
v2:
- Fix a double accounting error in the last patch scrub_stripe_report_errors() is called twice, thus doubling the accounting.

v1:
- Rebased to latest misc-next

- Rework the read IO grouping patch
David has found some crashes mostly related to scrub performance fixes, meanwhile the original grouping patch has one extra flag, SCRUB_FLAG_READ_SUBMITTED, to avoid double submitting.

But this flag can be avoided as we can easily avoid double submitting just by properly checking the sctx->nr_stripe variable.

This reworked grouping read IO patch should be safer compared to the initial version, with better code structure.

Unfortunately, the final performance is worse than the initial version (2.2GiB/s vs 2.5GiB/s), but it should be less racy thus safer.

- Re-order the patches
The first 3 patches are the main fixes, and I put safer patches first, so even if David still found crash at certain patch, the remaining can be dropped if needed.

There is a huge scrub performance drop introduced by v6.4 kernel, that the scrub performance is only around 1/3 for large data extents.

There are several causes:

- Missing blk plug
This means read requests won't be merged by block layer, this can hugely reduce the read performance.

- Extra time spent on extent/csum tree search
This including extra path allocation/freeing and tree searchs. This is especially obvious for large data extents, as previously we only do one csum search per 512K, but now we do one csum search per 64K, an 8 times increase in csum tree search.

- Less concurrency
Mostly due to the fact that we're doing submit-and-wait, thus much lower queue depth, affecting things like NVME which benefits a lot from high concurrency.

The first 3 patches would greately improve the scrub read performance, but unfortunately it's still not as fast as the pre-6.4 kernels. (2.2GiB/s vs 3.0GiB/s), but still much better than 6.4 kernels (2.2GiB vs 1.0GiB/s).

Qu Wenruo (5):
btrfs: scrub: avoid unnecessary extent tree search preparing stripes
btrfs: scrub: avoid unnecessary csum tree search preparing stripes
btrfs: scrub: fix grouping of read IO
btrfs: scrub: don't go ordered workqueue for dev-replace
btrfs: scrub: move write back of repaired sectors into
scrub_stripe_read_repair_worker()

 fs/btrfs/file-item.c |  33 +++---
 fs/btrfs/file-item.h |   6 +-
 fs/btrfs/raid56.c    |   4 +-
 fs/btrfs/scrub.c     | 235 ++++++++++++++++++++++++++-----------------
 4 files changed, 169 insertions(+), 109 deletions(-)
Signature · 1 line
2.41.0