Skip to content

new on disk format versions, backpointers fsck scalability

Kent Overstreet linux-bcachefs 1 version New

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest8 patches 0 repliesNew

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/8bcachefs: bcachefs_metadata_version_backpointer_bucket_genfs/bcachefs/backpointers.c fs/bcachefs/backpointers.h fs/bcachefs/bcachefs_format.h +1 more+19 -230 replies
2/8bcachefs: bcachefs_metadata_version_disk_accounting_big_endianfs/bcachefs/bcachefs_format.h fs/bcachefs/disk_accounting.c fs/bcachefs/disk_accounting.h +2 more+61 -150 replies
3/8bcachefs: bch2_extent_ptr_to_bp() no longer depends on devicefs/bcachefs/backpointers.c fs/bcachefs/backpointers.h fs/bcachefs/buckets.c+14 -360 replies
4/8bcachefs: kill __bch2_extent_ptr_to_bp()fs/bcachefs/backpointers.h fs/bcachefs/buckets.c+6 -190 replies
5/8bcachefs: check_extents_to_backpointers() now only checks buckets with mismatchesfs/bcachefs/backpointers.c fs/bcachefs/bcachefs.h fs/bcachefs/btree_cache.c +1 more+263 -190 replies
6/8bcachefs: Add write buffer flush param to backpointer_get_key()fs/bcachefs/backpointers.c fs/bcachefs/backpointers.h fs/bcachefs/ec.c +1 more+26 -130 replies
7/8bcachefs: bch2_backpointer_get_key() now repairs dangling backpointersfs/bcachefs/backpointers.c+25 -410 replies
8/8bcachefs: Only run check_backpointers_to_extents in debug modefs/bcachefs/recovery_passes_types.h fs/bcachefs/sb-errors_format.h+51 -450 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH 0/8] new on disk format versions, backpointers fsck scalability

- the sort order of disk accounting keys was screwed up, making disk accounting read way more expensive than it needs to be

this is an expensive forced version upgrade, but necessary because we now have counters for every inode for tracking fragmentation - we can't be scanning all of those at startup time

backpointers fsck has been the limiting factor on fsck scalability:
checking every extent <-> backpointer reference inherently doesn't scale, since the references we follow do nearly random cross-btree access so we have to pin in ram, or run in multiple passes.

so:

- change bch_backpointer to include the bucket generation number

with this, we can sum up backpointers in a bucket and check against the bucket sector counts to determine if there's missing backpointers; we need the generation number to avoid counting stale backpointers since the backpointers btree uses the btree write buffer (updates are much faster, reads are always potentially stale).

having done so, we can mark out a bitmap and then only check extents -> backpointers references for those specific buckets.

for backpointers -> extents, those checks can be deferred until when the backpointer is used: the overall self healing project is "run fsck check/repair code at runtime whenever appropriate", so with that done we can switch that pass to only run in debug mode.

Kent Overstreet (8):
bcachefs: bcachefs_metadata_version_backpointer_bucket_gen
bcachefs: bcachefs_metadata_version_disk_accounting_big_endian
bcachefs: bch2_extent_ptr_to_bp() no longer depends on device
bcachefs: kill __bch2_extent_ptr_to_bp()
bcachefs: check_extents_to_backpointers() now only checks buckets with
mismatches
bcachefs: Add write buffer flush param to backpointer_get_key()
bcachefs: bch2_backpointer_get_key() now repairs dangling backpointers
bcachefs: Only run check_backpointers_to_extents in debug mode

 fs/bcachefs/backpointers.c          | 394 +++++++++++++++++++++-------
 fs/bcachefs/backpointers.h          |  32 +--
 fs/bcachefs/bcachefs.h              |   2 +
 fs/bcachefs/bcachefs_format.h       |   7 +-
 fs/bcachefs/btree_cache.c           |   1 -
 fs/bcachefs/buckets.c               |  10 +-
 fs/bcachefs/disk_accounting.c       |  25 +-
 fs/bcachefs/disk_accounting.h       |  25 +-
 fs/bcachefs/ec.c                    |  14 +-
 fs/bcachefs/errcode.h               |   1 +
 fs/bcachefs/move.c                  |   8 +-
 fs/bcachefs/recovery_passes_types.h |  92 ++++---
 fs/bcachefs/sb-downgrade.c          |  26 +-
 fs/bcachefs/sb-errors_format.h      |   4 +-
 fs/bcachefs/util.h                  |   9 +
 15 files changed, 452 insertions(+), 198 deletions(-)
Signature · 1 line
2.45.2