Skip to contentSkip to conversation

btrfs-progs: handle orphan directories properly

PATCH RESEND In review Josef Bacik 2 participants 4 messages Feb 22, 2022 – Mar 8, 2022 Btrfs file system

lore
Share & tools
lore
#0 lore raw

v2 cover btrfs-progs: handle orphan directories properly

v1->v2:
- use "is_orphan" not "has_orphan_item" in the mode-lowmem code. Somehow the compiler didn't warn me of this until I switched to a different branch.

--- Original email ---
Hello,

While implementing the garbage collection tree I started getting btrfsck errors when a test would do rm -rf DIRECTORY and then immediately unmount. This is because we stop processing GC items during umount, and thus we left the directory with an nlink == 0 and all of its children in the fs tree.

However this isn't a problem with just the GC tree, we can have this happen if we fail to do the eviction work at evict time, and we leave the orphan entry in place on disk. btrfsck properly ignores problems with inodes that have orphan items, except for directory inodes.

Fix this by making sure we don't add any backrefs we find from directory inodes that we've find the inode item for and have an nlink == 0.

I generated a test image for this by simply skipping the btrfs_truncate_inode_items() work in evict in a kernel and rm -rf'ing a directory on a test file system.

With this patch both make test-check and make test-check-lowmem pass all the tests, including the new testcase. Thanks,

Josef

Josef Bacik (2):
btrfs-progs: handle orphan directories properly
btrfs-progs: add a test to check orphaned directories

 check/main.c                                  |  15 ++++++++++++-
 check/mode-lowmem.c                           |   6 ++++--
 .../052-orphan-directory/default.img.xz       | Bin 0 -> 1432 bytes
 tests/fsck-tests/052-orphan-directory/test.sh |  20 ++++++++++++++++++
 4 files changed, 38 insertions(+), 3 deletions(-)
 create mode 100644 tests/fsck-tests/052-orphan-directory/default.img.xz
 create mode 100755 tests/fsck-tests/052-orphan-directory/test.sh
Signature · 1 line
2.26.3
#1 lore raw

PATCH v2 1/2 btrfs-progs: handle orphan directories properly

When implementing the GC tree I started getting btrfsck errors when a test rm -rf <directory> with files inside of it and immediately unmount, leaving behind orphaned directory items that have GC items for them.

This made me realize that we don't actually handle this case currently for our normal orphan path. If we fail to clean everything up and leave behind the orphan items we'll fail fsck.

Fix this by not processing any backrefs we find if we found an inode item and its nlink is 0. This allows us to pass the test case I've provided to validate this patch.

---
 check/main.c        | 15 ++++++++++++++-
 check/mode-lowmem.c |  6 ++++--
 2 files changed, 18 insertions(+), 3 deletions(-)
check/main.c +14 -1
index abe208df..e9ac94cc 100644
@@ -1018,6 +1018,16 @@ static int merge_inode_recs(struct inode_record *src, struct inode_record *dst,
 	int ret = 0;
 
 	dst->merging = 1;
+
+	/*
+	 * If we wandered into a shared node while we were processing an inode
+	 * we may have added backrefs for a directory that had nlink == 0, so
+	 * skip adding these backrefs to our src inode if we have nlink == 0 and
+	 * we actually found the inode item.
+	 */
+	if (src->found_inode_item && src->nlink == 0)
+		goto skip_backrefs;
+
 	list_for_each_entry(backref, &src->backrefs, list) {
 		if (backref->found_dir_index) {
 			add_inode_backref(dst_cache, dst->ino, backref->dir,
@@ -1039,7 +1049,7 @@ static int merge_inode_recs(struct inode_record *src, struct inode_record *dst,
 					backref->ref_type, backref->errors);
 		}
 	}
-
+skip_backrefs:
 	if (src->found_dir_item)
 		dst->found_dir_item = 1;
 	if (src->found_file_extent)
@@ -1394,6 +1404,9 @@ static int process_dir_item(struct extent_buffer *eb,
 	rec = active_node->current;
 	rec->found_dir_item = 1;
 
+	if (rec->found_inode_item && rec->nlink == 0)
+		return 0;
+
 	di = btrfs_item_ptr(eb, slot, struct btrfs_dir_item);
 	total = btrfs_item_size_nr(eb, slot);
 	while (cur < total) {
check/mode-lowmem.c +4 -2
index f354a29b..0cdf24cd 100644
@@ -2727,6 +2727,8 @@ static int check_inode_item(struct btrfs_root *root, struct btrfs_path *path)
 					imode_to_type(mode), key.objectid,
 					key.offset);
 			}
+			if (is_orphan && key.type == BTRFS_DIR_INDEX_KEY)
+				break;
 			ret = check_dir_item(root, &key, path, &size);
 			err |= ret;
 			break;
@@ -2769,7 +2771,7 @@ out:
 				&nlink);
 		}
 
-		if (nlink != 1) {
+		if (nlink > 1) {
 			err |= LINK_COUNT_ERROR;
 			error("root %llu DIR INODE[%llu] shouldn't have more than one link(%llu)",
 			      root->objectid, inode_id, nlink);
@@ -2785,7 +2787,7 @@ out:
 				gfs_info->nodesize);
 		}
 
-		if (isize != size) {
+		if (isize != size && !is_orphan) {
 			if (repair)
 				ret = repair_dir_isize_lowmem(root, path,
 							      inode_id, size);
Signature · 1 line
2.26.3
#2 lore raw

PATCH v2 2/2 btrfs-progs: add a test to check orphaned directories

In reply to Josef Bacik

When adding the GC support I noticed we were failing fsck when we had a directory that hadn't been cleaned up yet because of rm -rf. However this isn't limited to extent-tree-v2, we'll actually fail in the same way if we were unable to do the evict portion of the deletion and left the orphan items around for everybody.

This is a valid file system, it'll be cleaned up properly at mount time, so fsck shouldn't fail in this case.

---
 .../052-orphan-directory/default.img.xz       | Bin 0 -> 1432 bytes
 tests/fsck-tests/052-orphan-directory/test.sh |  20 ++++++++++++++++++
 2 files changed, 20 insertions(+)
 create mode 100644 tests/fsck-tests/052-orphan-directory/default.img.xz
 create mode 100755 tests/fsck-tests/052-orphan-directory/test.sh
tests/fsck-tests/052-orphan-directory/default.img.xz new file binary +0 -0
new file mode 100644
index 0000000000000000000000000000000000000000..4fac926c579034d29d51065f07ede7f1528f67f7
GIT binary patch
tests/fsck-tests/052-orphan-directory/test.sh new file +20 -0
new file mode 100755
index 00000000..f3d380d9
@@ -0,0 +1,20 @@
+#!/bin/bash
+# We could potentially have a directory and it's children with ORPHAN items left
+# for them without having been cleaned up.
+#
+# fsck shouldn't complain about this or attempt to do anything about it, the
+# orphan cleanup will do the correct thing.
+#
+# To create this image I simply modified the kernel to skip doing the
+# btrfs_truncate_inode_items() and removing the orphan item at evict time, and
+# then rm -rf'ed a directory.
+
+source "$TEST_TOP/common"
+
+check_prereq btrfs
+
+check_image() {
+	run_check "$TOP/btrfs" check "$1"
+}
+
+check_all_images
Signature · 1 line
2.26.3
#3 lore raw
In reply to Josef Bacik
Quoted 24 lines · Josef Bacik

v1->v2:
- use "is_orphan" not "has_orphan_item" in the mode-lowmem code. Somehow the compiler didn't warn me of this until I switched to a different branch.

--- Original email ---
Hello,

While implementing the garbage collection tree I started getting btrfsck errors when a test would do rm -rf DIRECTORY and then immediately unmount. This is because we stop processing GC items during umount, and thus we left the directory with an nlink == 0 and all of its children in the fs tree.

However this isn't a problem with just the GC tree, we can have this happen if we fail to do the eviction work at evict time, and we leave the orphan entry in place on disk. btrfsck properly ignores problems with inodes that have orphan items, except for directory inodes.

Fix this by making sure we don't add any backrefs we find from directory inodes that we've find the inode item for and have an nlink == 0.

I generated a test image for this by simply skipping the btrfs_truncate_inode_items() work in evict in a kernel and rm -rf'ing a directory on a test file system.

Quoted 8 lines

With this patch both make test-check and make test-check-lowmem pass all the tests, including the new testcase. Thanks,

Josef

Josef Bacik (2):
btrfs-progs: handle orphan directories properly
btrfs-progs: add a test to check orphaned directories

Added to devel, thanks.