btrfs-progs: extent tree v2 support, global roots
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v5 Latest19 patches 9 repliesIn review
- v2 posting 3/311 patches 8 repliesSuperseded
- v422 patches 0 repliesSuperseded
Earlier versions · 4
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v2 posting 2/3 | 22 patches | 0 replies | Superseded | |
| v3 | 8 patches | 0 replies | Superseded | |
| v2 posting 1/3 | 30 patches | 1 reply | Superseded | |
| v1 | 8 patches | 12 replies | Superseded |
v4->v3 from the cover letter
- Rebase onto devel, depends on "btrfs-progs: cleanup btrfs_item* accessors".
- Dropped the various patches that have already been merged into -progs.
Patches in v5 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/19 | btrfs-progs: add support for loading the block group rootkernel-shared/ kernel-shared/ kernel-shared/ +2 more | +138 | 0 replies | |
| 2/19 | btrfs-progs: add print support for the block group treecmds/ kernel-shared/ | +48 | 0 replies | |
| 3/19 | btrfs-progs: mkfs: use the btrfs_block_group_root helpercheck/ mkfs/ | +3 | 0 replies | |
| 4/19 | btrfs-progs: check-lowmem: use the btrfs_block_group_root helpercheck/ | +4 | 0 replies | |
| 5/19 | btrfs-progs: handle no bg item in extent tree for free space treekernel-shared/ | +8 | 0 replies | |
| 6/19 | btrfs-progs: mkfs: add support for the block group treemkfs/ mkfs/ mkfs/ | +91 | 0 replies | |
| 7/19 | btrfs-progs: check: add block group tree supportcheck/ check/ | +22 | 0 replies | |
| 8/19 | btrfs-progs: qgroup-verify: scan extents based on block groupscheck/ | +24 | 0 replies | |
| 9/19 | btrfs-progs: check: make free space tree validation extent tree v2 awarecheck/ | +31 | 0 replies | |
| 10/19 | btrfs-progs: check: add helper to reinit the root based on a keycheck/ | +58 | 0 replies | |
| 11/19 | btrfs-progs: check: handle the block group tree properlycheck/ common/ | +20 | 0 replies | |
| 12/19 | btrfs-progs: set the number of global roots in the super blockkernel-shared/ kernel-shared/ mkfs/ | +10 | 6 replies | |
| 13/19 | btrfs-progs: handle the per-block group global root idkernel-shared/ kernel-shared/ kernel-shared/ +2 more | +50 | 0 replies | |
| 14/19 | btrfs-progs: add a btrfs_delete_and_free_root helperkernel-shared/ kernel-shared/ kernel-shared/ | +30 | 0 replies | |
| 15/19 | btrfs-progs: make btrfs_clear_free_space_tree extent tree v2 awarekernel-shared/ | +27 | 0 replies | |
| 16/19 | btrfs-progs: make btrfs_create_tree take a key for the root keykernel-shared/ kernel-shared/ kernel-shared/ +1 more | +24 | 0 replies | |
| 17/19 | btrfs-progs: mkfs: set chunk_item_objectid properly for extent tree v2mkfs/ | +4 | 0 replies | |
| 18/19 | btrfs-progs: mkfs: create the global root'smkfs/ | +72 | 2 replies | |
| 19/19 | btrfs-progs: check: don't do the root item check for extent tree v2check/ | +3 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v5 yet.
Cover letter [PATCH v5 00/19] btrfs-progs: extent tree v2 support, global roots
v4->v3:
- Rebase onto devel, depends on "btrfs-progs: cleanup btrfs_item* accessors".
- Dropped the various patches that have already been merged into -progs.
v3->v4:
- Rebase onto devel, depends on the v3 prep patches that were sent on December 1st which has the rest of the "don't access ->*_root" patches.
- I think I screwed up the versioning of this, but I lost the other submission, so call this v3.
v1->v2:
- These depend on the v3 of the prep patches (it's marked as v2 because I'm stupid, but the second v2 posting I sent.)
- I've moved the global root rb tree patches into this series to differentiate them from the actual fixes in the prep series.
--- Original email ---
Hello,
These patches are the first chunk of the extent tree v2 format changes. This includes the separate block group root which will hold all of the block group items. This also includes the global root support, which is the work to allow us to have multiple extent, csum, and free space trees in the same file system.
The goal of these two changes are straightforward. For the block group root, on very large file systems the block group items are very widely separated, which means it takes a very long time to mount the file system on large, slow disks. Putting the block group items in their own root will allow us to densely populate the tree and dramatically increase mount times in these cases.
The global roots change is motivated by lock contention on the root nodes of these global roots. I've had to make many changes to how we run delayed refs to speed up things like the transaction commit because of all the delayed refs going into one tree and contending on the root node of the extent tree. In the same token you can have heavy lock contention on the csum roots when writing to many files. Allowing for multiple roots will let us spread the lock contention load around.
I have disabled a few key features, namely balance and qgroups. There will be more to come as I make more and more invasive changes, and then they will slowly be re-enabled as the work is added. These are disabled to avoid a bunch of work that would be thrown away by future changes.
These patches have passed xfstests without panicing, but clearly failing a lot of tests because of the disabled features. I've also run it through fsperf to validate that there are no major performance regressions.
WARNING: there are many more format changes planned, this is just the first batch. If you want to test then please feel free, but know that the format is still in flux. Thanks,
Josef
Josef Bacik (19):
btrfs-progs: add support for loading the block group root
btrfs-progs: add print support for the block group tree
btrfs-progs: mkfs: use the btrfs_block_group_root helper
btrfs-progs: check-lowmem: use the btrfs_block_group_root helper
btrfs-progs: handle no bg item in extent tree for free space tree
btrfs-progs: mkfs: add support for the block group tree
btrfs-progs: check: add block group tree support
btrfs-progs: qgroup-verify: scan extents based on block groups
btrfs-progs: check: make free space tree validation extent tree v2
aware
btrfs-progs: check: add helper to reinit the root based on a key
btrfs-progs: check: handle the block group tree properly
btrfs-progs: set the number of global roots in the super block
btrfs-progs: handle the per-block group global root id
btrfs-progs: add a btrfs_delete_and_free_root helper
btrfs-progs: make btrfs_clear_free_space_tree extent tree v2 aware
btrfs-progs: make btrfs_create_tree take a key for the root key
btrfs-progs: mkfs: set chunk_item_objectid properly for extent tree v2
btrfs-progs: mkfs: create the global root's
btrfs-progs: check: don't do the root item check for extent tree v2
check/main.c | 233 +++++++++++++++++-------------- check/mode-lowmem.c | 12 +- check/qgroup-verify.c | 32 +++-- cmds/inspect-dump-tree.c | 30 +++- common/repair.c | 3 + kernel-shared/ctree.h | 9 +- kernel-shared/disk-io.c | 235 ++++++++++++++++++++++++-------- kernel-shared/disk-io.h | 15 +- kernel-shared/extent-tree.c | 32 ++++- kernel-shared/free-space-tree.c | 72 +++++----- kernel-shared/print-tree.c | 23 +++- kernel-shared/transaction.c | 2 + mkfs/common.c | 94 ++++++++++--- mkfs/common.h | 12 ++ mkfs/main.c | 93 ++++++++++++- 15 files changed, 658 insertions(+), 239 deletions(-)
Signature · 1 line
2.26.3