Skip to content

btrfs-progs: extent tree v2 gc tree and metadata ref changes

Josef Bacik linux-btrfs 2 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest15 patches 0 repliesNew
  2. v115 patches 0 repliesSuperseded

Changes in v2 from the cover letter

- rebased onto devel, depends on "[PATCH v5 00/19] btrfs-progs: extent tree v2 support, global roots"

Patches in v2 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/15btrfs-progs: extract out free extent accounting handlingkernel-shared/extent-tree.c+26 -190 replies
2/15btrfs-progs: check: skip owner ref check for extent tree v2check/main.c+7 -00 replies
3/15btrfs-progs: check: skip extent backref for metadata in extent tree v2check/main.c+7 -00 replies
4/15btrfs-progs: check: calculate normal flags for extent tree v2 metadatacheck/main.c+3 -00 replies
5/15btrfs-progs: check: make metadata ref counting extent tree v2 awarecheck/main.c+16 -20 replies
6/15btrfs-progs: check: update block group used properly for extent tree v2check/main.c+8 -00 replies
7/15btrfs-progs: do not insert extent items for metadata for extent tree v2kernel-shared/extent-tree.c+6 -20 replies
8/15btrfs-progs: do not remove metadata backrefs for extent tree v2kernel-shared/extent-tree.c+6 -10 replies
9/15btrfs-progs: repair: traverse tree blocks for extent tree v2common/repair.c+6 -00 replies
10/15btrfs-progs: cache the block group with the free space tree if possiblekernel-shared/extent-tree.c kernel-shared/free-space-cache.c+17 -00 replies
11/15btrfs-progs: make btrfs_lookup_extent_info extent tree v2 awarekernel-shared/extent-tree.c+8 -00 replies
12/15btrfs-progs: mkfs: don't populate extent tree with extent tree v2mkfs/common.c+110 -910 replies
13/15btrfs-progs: add on-disk items and read support for the gc treekernel-shared/ctree.h kernel-shared/disk-io.c kernel-shared/print-tree.c+20 -00 replies
14/15btrfs-progs: mkfs: create the gc tree at mkfs timemkfs/main.c+12 -10 replies
15/15btrfs-progs: deal with GC items in checkcheck/main.c check/mode-original.h+53 -30 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.

Read the v2 thread

Cover letter [PATCH v2 00/15] btrfs-progs: extent tree v2 gc tree and metadata ref changes

v1->v2:
- rebased onto devel, depends on "[PATCH v5 00/19] btrfs-progs: extent tree v2 support, global roots"

--- Original email ---

Hello,

Here's the next set of btrfs-progs patches for the next set of extent tree v2 features, no more metadata reference tracking and support for a garbge collection tree.

1. No more metadata reference tracking. For cowonly trees (global roots) we never have more than one reference for a block, so the reference counting adds a fair bit of overhead for these trees. This is especially noticeable with the extent tree, we have to update the extent tree to include references for every block, including blocks in the extent tree, which generates more modifications to the extent tree. In practice there's a stable state so we do not recursively use all the disk space describing the extent tree, however it generates a certain amount of churn.

For FS tree's that can be shared we still want a way to know when we can free blocks. That will be accomplished by drop trees in a future patchset and described more thoroughly there.

2. Garbage collection tree. Free'ing objects in btrfs is a more complicated operation than in other file systems. For truncate we can mete out the removal of the inode items, but the cleaning up of checksums has to be done in the transaction that the extent is freed. For large files this can be quite expensive and cause latencies because we have to hold the transaction open for these operations. Deferring this work to a GC tree will allow us to better throttle these long running operations without causing latencies for the rest of the system.

As alluded above, free'ing shared metadata extents via the drop trees will be one of these longer running operations, having a GC tree in place will allow us to do these heavier operations without requiring they be done in a single transaction.

These patches have been tested with the corresponding kernel patches and pass the xfstests that are able to be run. Thanks,

Josef

Josef Bacik (15):
btrfs-progs: extract out free extent accounting handling
btrfs-progs: check: skip owner ref check for extent tree v2
btrfs-progs: check: skip extent backref for metadata in extent tree v2
btrfs-progs: check: calculate normal flags for extent tree v2 metadata
btrfs-progs: check: make metadata ref counting extent tree v2 aware
btrfs-progs: check: update block group used properly for extent tree

    v2
  btrfs-progs: do not insert extent items for metadata for extent tree
    v2

btrfs-progs: do not remove metadata backrefs for extent tree v2
btrfs-progs: repair: traverse tree blocks for extent tree v2
btrfs-progs: cache the block group with the free space tree if
possible
btrfs-progs: make btrfs_lookup_extent_info extent tree v2 aware
btrfs-progs: mkfs: don't populate extent tree with extent tree v2
btrfs-progs: add on-disk items and read support for the gc tree
btrfs-progs: mkfs: create the gc tree at mkfs time
btrfs-progs: deal with GC items in check

 check/main.c                     |  98 ++++++++++++++-
 check/mode-original.h            |   1 +
 common/repair.c                  |   6 +
 kernel-shared/ctree.h            |   6 +
 kernel-shared/disk-io.c          |  10 ++
 kernel-shared/extent-tree.c      |  82 +++++++++----
 kernel-shared/free-space-cache.c |   3 +
 kernel-shared/print-tree.c       |   4 +
 mkfs/common.c                    | 201 +++++++++++++++++--------------
 mkfs/main.c                      |  13 +-
 10 files changed, 305 insertions(+), 119 deletions(-)
Signature · 1 line
2.26.3