btrfs-progs: handle orphan directories properly
v2 cover btrfs-progs: handle orphan directories properly
v1->v2:
- use "is_orphan" not "has_orphan_item" in the mode-lowmem code. Somehow the compiler didn't warn me of this until I switched to a different branch.
--- Original email ---
Hello,
While implementing the garbage collection tree I started getting btrfsck errors when a test would do rm -rf DIRECTORY and then immediately unmount. This is because we stop processing GC items during umount, and thus we left the directory with an nlink == 0 and all of its children in the fs tree.
However this isn't a problem with just the GC tree, we can have this happen if we fail to do the eviction work at evict time, and we leave the orphan entry in place on disk. btrfsck properly ignores problems with inodes that have orphan items, except for directory inodes.
Fix this by making sure we don't add any backrefs we find from directory inodes that we've find the inode item for and have an nlink == 0.
I generated a test image for this by simply skipping the btrfs_truncate_inode_items() work in evict in a kernel and rm -rf'ing a directory on a test file system.
With this patch both make test-check and make test-check-lowmem pass all the tests, including the new testcase. Thanks,
Josef
Josef Bacik (2):
btrfs-progs: handle orphan directories properly
btrfs-progs: add a test to check orphaned directories
check/main.c | 15 ++++++++++++- check/mode-lowmem.c | 6 ++++-- .../052-orphan-directory/default.img.xz | Bin 0 -> 1432 bytes tests/fsck-tests/052-orphan-directory/test.sh | 20 ++++++++++++++++++ 4 files changed, 38 insertions(+), 3 deletions(-) create mode 100644 tests/fsck-tests/052-orphan-directory/default.img.xz create mode 100755 tests/fsck-tests/052-orphan-directory/test.sh
Signature · 1 line
2.26.3
PATCH v2 1/2 btrfs-progs: handle orphan directories properly
When implementing the GC tree I started getting btrfsck errors when a test rm -rf <directory> with files inside of it and immediately unmount, leaving behind orphaned directory items that have GC items for them.
This made me realize that we don't actually handle this case currently for our normal orphan path. If we fail to clean everything up and leave behind the orphan items we'll fail fsck.
Fix this by not processing any backrefs we find if we found an inode item and its nlink is 0. This allows us to pass the test case I've provided to validate this patch.
--- check/main.c | 15 ++++++++++++++- check/mode-lowmem.c | 6 ++++-- 2 files changed, 18 insertions(+), 3 deletions(-)
check/main.c +14 -1
@@ -1018,6 +1018,16 @@ static int merge_inode_recs(struct inode_record *src, struct inode_record *dst,int ret =0 ; dst->merging =1 ; + +/* +* If we wandered into a shared node while we were processing an inode +* we may have added backrefs for a directory that had nlink == 0, so +* skip adding these backrefs to our src inode if we have nlink == 0 and +* we actually found the inode item. +*/ +if (src->found_inode_item && src->nlink ==0 ) + goto skip_backrefs; +list_for_each_entry (backref, &src->backrefs, list) {if (backref->found_dir_index) { add_inode_backref(dst_cache, dst->ino, backref->dir,
@@ -1039,7 +1049,7 @@ static int merge_inode_recs(struct inode_record *src, struct inode_record *dst, backref->ref_type, backref->errors); } } - +skip_backrefs:if (src->found_dir_item) dst->found_dir_item =1 ;if (src->found_file_extent)
@@ -1394,6 +1404,9 @@ static int process_dir_item(struct extent_buffer *eb, rec = active_node->current; rec->found_dir_item =1 ; +if (rec->found_inode_item && rec->nlink ==0 ) +return 0 ; + di = btrfs_item_ptr(eb, slot, struct btrfs_dir_item); total =btrfs_item_size_nr (eb, slot);while (cur < total) {
check/mode-lowmem.c +4 -2
@@ -2727,6 +2727,8 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path)imode_to_type (mode), key.objectid, key.offset); } +if (is_orphan && key.type ==BTRFS_DIR_INDEX_KEY ) +break ; ret =check_dir_item (root, &key, path, &size); err |= ret;break ;
@@ -2769,7 +2771,7 @@ out: &nlink); } -if (nlink !=1 ) { +if (nlink >1 ) { err |=LINK_COUNT_ERROR ;error ("root %llu DIR INODE[%llu] shouldn't have more than one link(%llu)" , root->objectid, inode_id, nlink);
@@ -2785,7 +2787,7 @@ out: gfs_info->nodesize); } -if (isize != size) { +if (isize != size && !is_orphan) {if (repair) ret =repair_dir_isize_lowmem (root, path, inode_id, size);
Signature · 1 line
2.26.3
PATCH v2 2/2 btrfs-progs: add a test to check orphaned directories
In reply to Josef BacikWhen adding the GC support I noticed we were failing fsck when we had a directory that hadn't been cleaned up yet because of rm -rf. However this isn't limited to extent-tree-v2, we'll actually fail in the same way if we were unable to do the evict portion of the deletion and left the orphan items around for everybody.
This is a valid file system, it'll be cleaned up properly at mount time, so fsck shouldn't fail in this case.
--- .../052-orphan-directory/default.img.xz | Bin 0 -> 1432 bytes tests/fsck-tests/052-orphan-directory/test.sh | 20 ++++++++++++++++++ 2 files changed, 20 insertions(+) create mode 100644 tests/fsck-tests/052-orphan-directory/default.img.xz create mode 100755 tests/fsck-tests/052-orphan-directory/test.sh
tests/fsck-tests/052-orphan-directory/default.img.xz new file binary +0 -0
tests/fsck-tests/052-orphan-directory/test.sh new file +20 -0
@@ -0,0 +1,20 @@ +#!/bin/bash +# We could potentially have a directory and it's children with ORPHAN items left +# for them without having been cleaned up. +# +# fsck shouldn't complain about this or attempt to do anything about it, the +# orphan cleanup will do the correct thing. +# +# To create this image I simply modified the kernel to skip doing the +# btrfs_truncate_inode_items() and removing the orphan item at evict time, and +# then rm -rf'ed a directory. + +source "$TEST_TOP/common" + +check_prereq btrfs + +check_image() { + run_check "$TOP/btrfs" check "$1" +} + +check_all_images
Signature · 1 line
2.26.3
Quoted 24 lines · Josef Bacik
v1->v2:
- use "is_orphan" not "has_orphan_item" in the mode-lowmem code. Somehow the compiler didn't warn me of this until I switched to a different branch.--- Original email ---
Hello,While implementing the garbage collection tree I started getting btrfsck errors when a test would do rm -rf DIRECTORY and then immediately unmount. This is because we stop processing GC items during umount, and thus we left the directory with an nlink == 0 and all of its children in the fs tree.
However this isn't a problem with just the GC tree, we can have this happen if we fail to do the eviction work at evict time, and we leave the orphan entry in place on disk. btrfsck properly ignores problems with inodes that have orphan items, except for directory inodes.
Fix this by making sure we don't add any backrefs we find from directory inodes that we've find the inode item for and have an nlink == 0.
I generated a test image for this by simply skipping the
btrfs_truncate_inode_items()work in evict in a kernel and rm -rf'ing a directory on a test file system.
Quoted 8 lines
With this patch both make test-check and make test-check-lowmem pass all the tests, including the new testcase. Thanks,
Josef
Josef Bacik (2):
btrfs-progs: handle orphan directories properly
btrfs-progs: add a test to check orphaned directories
Added to devel, thanks.