Skip to contentSkip to conversation

generic/707: Test moving directory while being grown

PATCH In review Jan Kara 2 participants 6 messages Jan 31, 2023 – Oct 19, 2023

lore
Share & tools
lore

Jan Kara

author
#0 lore raw

PATCH v3 1/1 generic/707: Test moving directory while being grown

Test how the filesystem can handle moving a directory to a different directory (so that parent pointer gets updated) while it is grown. Ext4 and UDF had a bug where if the directory got converted to a different type due to growth while rename is running, the filesystem got corrupted.

---
 tests/generic/707     | 70 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/707.out |  2 ++
 2 files changed, 72 insertions(+)
 create mode 100755 tests/generic/707
 create mode 100644 tests/generic/707.out
tests/generic/707 new file +70 -0
new file mode 100755
index 000000000000..ad1592a1b957
@@ -0,0 +1,70 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Jan Kara, SUSE Linux.  All Rights Reserved.
+#
+# FS QA Test 707
+#
+# This is a test verifying whether the filesystem can gracefully handle
+# modifying of a directory while it is being moved, in particular the cases
+# where directory format changes
+#
+. ./common/preamble
+_begin_fstest auto
+
+_supported_fs generic
+_require_scratch
+
+_fixed_by_kernel_commit f950fd052913 \
+	"udf: Protect rename against modification of moved directory"
+_fixed_by_kernel_commit XXXXXXXXXXXX \
+	"ext4: fix possible corruption when moving a directory"
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	if [ -n "$BGPID" ]; then
+		# Stop background process
+		kill -9 $BGPID &>/dev/null
+		wait
+	fi
+}
+
+# Loop multiple times trying to hit the race
+loops=$((100*TIME_FACTOR))
+files=500
+moves=500
+
+create_files()
+{
+	# We use slightly longer file name to make directory grow faster and
+	# hopefully convert between various types
+	for (( i = 0; i < $files; i++ )); do
+		touch somewhatlongerfilename$i
+	done
+}
+
+for (( i = 0; i <= $moves; i++ )); do
+	mkdir $SCRATCH_MNT/dir$i
+done
+
+for (( l = 0; l < $loops; l++ )); do
+	mkdir $SCRATCH_MNT/dir0/dir
+	pushd $SCRATCH_MNT/dir0/dir &>/dev/null
+	create_files &
+	BGPID=$!
+	popd &>/dev/null
+	for (( i = 0; i < $moves; i++ )); do
+		mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir
+	done
+	wait
+	unset BGPID
+	rm -r $SCRATCH_MNT/dir$moves/dir
+done
+
+echo "Silence is golden"
+status=0
+exit
tests/generic/707.out new file +2 -0
new file mode 100644
index 000000000000..8e57a1d8c971
@@ -0,0 +1,2 @@
+QA output created by 707
+Silence is golden
Signature · 1 line
2.35.3
Reviewed-by on v3 1/1
#1 lore raw
Quoted 9 lines · Jan Kara

Test how the filesystem can handle moving a directory to a different directory (so that parent pointer gets updated) while it is grown. Ext4 and UDF had a bug where if the directory got converted to a different type due to growth while rename is running, the filesystem got corrupted.

---

Thanks, this version looks good to me,

Quoted 93 lines
 tests/generic/707     | 70 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/707.out |  2 ++
 2 files changed, 72 insertions(+)
 create mode 100755 tests/generic/707
 create mode 100644 tests/generic/707.out
tests/generic/707 new file +70 -0
new file mode 100755
index 000000000000..ad1592a1b957
@@ -0,0 +1,70 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Jan Kara, SUSE Linux.  All Rights Reserved.
+#
+# FS QA Test 707
+#
+# This is a test verifying whether the filesystem can gracefully handle
+# modifying of a directory while it is being moved, in particular the cases
+# where directory format changes
+#
+. ./common/preamble
+_begin_fstest auto
+
+_supported_fs generic
+_require_scratch
+
+_fixed_by_kernel_commit f950fd052913 \
+	"udf: Protect rename against modification of moved directory"
+_fixed_by_kernel_commit XXXXXXXXXXXX \
+	"ext4: fix possible corruption when moving a directory"
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	if [ -n "$BGPID" ]; then
+		# Stop background process
+		kill -9 $BGPID &>/dev/null
+		wait
+	fi
+}
+
+# Loop multiple times trying to hit the race
+loops=$((100*TIME_FACTOR))
+files=500
+moves=500
+
+create_files()
+{
+	# We use slightly longer file name to make directory grow faster and
+	# hopefully convert between various types
+	for (( i = 0; i < $files; i++ )); do
+		touch somewhatlongerfilename$i
+	done
+}
+
+for (( i = 0; i <= $moves; i++ )); do
+	mkdir $SCRATCH_MNT/dir$i
+done
+
+for (( l = 0; l < $loops; l++ )); do
+	mkdir $SCRATCH_MNT/dir0/dir
+	pushd $SCRATCH_MNT/dir0/dir &>/dev/null
+	create_files &
+	BGPID=$!
+	popd &>/dev/null
+	for (( i = 0; i < $moves; i++ )); do
+		mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir
+	done
+	wait
+	unset BGPID
+	rm -r $SCRATCH_MNT/dir$moves/dir
+done
+
+echo "Silence is golden"
+status=0
+exit
tests/generic/707.out new file +2 -0
new file mode 100644
index 000000000000..8e57a1d8c971
@@ -0,0 +1,2 @@
+QA output created by 707
+Silence is golden
Signature · 1 line
2.35.3
author
#2 lore raw
Quoted 7 lines · Zorro Lang
Quoted 6 lines · Jan Kara

Test how the filesystem can handle moving a directory to a different directory (so that parent pointer gets updated) while it is grown. Ext4 and UDF had a bug where if the directory got converted to a different type due to growth while rename is running, the filesystem got corrupted.

Quoted 7 lines
Quoted 3 lines

---

Thanks, this version looks good to me,

Guys, I have just noticed this patch didn't get merged. Can you please pick it up? Thanks!

Honza

Quoted 95 lines
Quoted 94 lines
 tests/generic/707     | 70 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/707.out |  2 ++
 2 files changed, 72 insertions(+)
 create mode 100755 tests/generic/707
 create mode 100644 tests/generic/707.out
tests/generic/707 new file +70 -0
new file mode 100755
index 000000000000..ad1592a1b957
@@ -0,0 +1,70 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Jan Kara, SUSE Linux.  All Rights Reserved.
+#
+# FS QA Test 707
+#
+# This is a test verifying whether the filesystem can gracefully handle
+# modifying of a directory while it is being moved, in particular the cases
+# where directory format changes
+#
+. ./common/preamble
+_begin_fstest auto
+
+_supported_fs generic
+_require_scratch
+
+_fixed_by_kernel_commit f950fd052913 \
+	"udf: Protect rename against modification of moved directory"
+_fixed_by_kernel_commit XXXXXXXXXXXX \
+	"ext4: fix possible corruption when moving a directory"
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	if [ -n "$BGPID" ]; then
+		# Stop background process
+		kill -9 $BGPID &>/dev/null
+		wait
+	fi
+}
+
+# Loop multiple times trying to hit the race
+loops=$((100*TIME_FACTOR))
+files=500
+moves=500
+
+create_files()
+{
+	# We use slightly longer file name to make directory grow faster and
+	# hopefully convert between various types
+	for (( i = 0; i < $files; i++ )); do
+		touch somewhatlongerfilename$i
+	done
+}
+
+for (( i = 0; i <= $moves; i++ )); do
+	mkdir $SCRATCH_MNT/dir$i
+done
+
+for (( l = 0; l < $loops; l++ )); do
+	mkdir $SCRATCH_MNT/dir0/dir
+	pushd $SCRATCH_MNT/dir0/dir &>/dev/null
+	create_files &
+	BGPID=$!
+	popd &>/dev/null
+	for (( i = 0; i < $moves; i++ )); do
+		mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir
+	done
+	wait
+	unset BGPID
+	rm -r $SCRATCH_MNT/dir$moves/dir
+done
+
+echo "Silence is golden"
+status=0
+exit
tests/generic/707.out new file +2 -0
new file mode 100644
index 000000000000..8e57a1d8c971
@@ -0,0 +1,2 @@
+QA output created by 707
+Silence is golden
Signature · 1 line
2.35.3
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#3 lore raw
Quoted 12 lines · Jan Kara
Quoted 11 lines · Zorro Lang
Quoted 9 lines · Jan Kara

Test how the filesystem can handle moving a directory to a different directory (so that parent pointer gets updated) while it is grown. Ext4 and UDF had a bug where if the directory got converted to a different type due to growth while rename is running, the filesystem got corrupted.

---

Quoted 6 lines
Quoted 3 lines

Thanks, this version looks good to me,

Guys, I have just noticed this patch didn't get merged. Can you please pick it up? Thanks!

Hi Jan, I think it has been in fstests v2023.02.05 release, as:

commit da69ce70dfcdbd88d6f940d7c1ab346d5a1e5f82

Author: Jan Kara <[email protected]>
Date:   Tue Jan 31 13:54:16 2023 +0100

generic/707: Test moving directory while being grown

Isn't it same with this patch? Please check, and feel free to tell me if anything wrong.

Thanks,
Zorro

Quoted 102 lines

Honza

Quoted 95 lines
Quoted 94 lines
 tests/generic/707     | 70 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/707.out |  2 ++
 2 files changed, 72 insertions(+)
 create mode 100755 tests/generic/707
 create mode 100644 tests/generic/707.out
tests/generic/707 new file +70 -0
new file mode 100755
index 000000000000..ad1592a1b957
@@ -0,0 +1,70 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2023 Jan Kara, SUSE Linux.  All Rights Reserved.
+#
+# FS QA Test 707
+#
+# This is a test verifying whether the filesystem can gracefully handle
+# modifying of a directory while it is being moved, in particular the cases
+# where directory format changes
+#
+. ./common/preamble
+_begin_fstest auto
+
+_supported_fs generic
+_require_scratch
+
+_fixed_by_kernel_commit f950fd052913 \
+	"udf: Protect rename against modification of moved directory"
+_fixed_by_kernel_commit XXXXXXXXXXXX \
+	"ext4: fix possible corruption when moving a directory"
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+	if [ -n "$BGPID" ]; then
+		# Stop background process
+		kill -9 $BGPID &>/dev/null
+		wait
+	fi
+}
+
+# Loop multiple times trying to hit the race
+loops=$((100*TIME_FACTOR))
+files=500
+moves=500
+
+create_files()
+{
+	# We use slightly longer file name to make directory grow faster and
+	# hopefully convert between various types
+	for (( i = 0; i < $files; i++ )); do
+		touch somewhatlongerfilename$i
+	done
+}
+
+for (( i = 0; i <= $moves; i++ )); do
+	mkdir $SCRATCH_MNT/dir$i
+done
+
+for (( l = 0; l < $loops; l++ )); do
+	mkdir $SCRATCH_MNT/dir0/dir
+	pushd $SCRATCH_MNT/dir0/dir &>/dev/null
+	create_files &
+	BGPID=$!
+	popd &>/dev/null
+	for (( i = 0; i < $moves; i++ )); do
+		mv $SCRATCH_MNT/dir$i/dir $SCRATCH_MNT/dir$((i+1))/dir
+	done
+	wait
+	unset BGPID
+	rm -r $SCRATCH_MNT/dir$moves/dir
+done
+
+echo "Silence is golden"
+status=0
+exit
tests/generic/707.out new file +2 -0
new file mode 100644
index 000000000000..8e57a1d8c971
@@ -0,0 +1,2 @@
+QA output created by 707
+Silence is golden
Signature · 1 line
2.35.3
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
author
#4 lore raw
Quoted 22 lines · Zorro Lang
Quoted 12 lines · Jan Kara
Quoted 11 lines · Zorro Lang
Quoted 9 lines · Jan Kara

Test how the filesystem can handle moving a directory to a different directory (so that parent pointer gets updated) while it is grown. Ext4 and UDF had a bug where if the directory got converted to a different type due to growth while rename is running, the filesystem got corrupted.

---

Quoted 6 lines
Quoted 3 lines

Thanks, this version looks good to me,

Guys, I have just noticed this patch didn't get merged. Can you please pick it up? Thanks!

Hi Jan, I think it has been in fstests v2023.02.05 release, as:

Quoted 8 lines

commit da69ce70dfcdbd88d6f940d7c1ab346d5a1e5f82

Author: Jan Kara <[email protected]>
Date:   Tue Jan 31 13:54:16 2023 +0100

generic/707: Test moving directory while being grown

Isn't it same with this patch? Please check, and feel free to tell me if anything wrong.

Bah, sorry. Not sure why I didn't find it when grepping :-|. I'm sorry for the noise.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#5 lore raw
Quoted 27 lines · Jan Kara
Quoted 26 lines · Zorro Lang
Quoted 12 lines · Jan Kara
Quoted 11 lines · Zorro Lang
Quoted 9 lines · Jan Kara

Test how the filesystem can handle moving a directory to a different directory (so that parent pointer gets updated) while it is grown. Ext4 and UDF had a bug where if the directory got converted to a different type due to growth while rename is running, the filesystem got corrupted.

---

Quoted 6 lines
Quoted 3 lines

Thanks, this version looks good to me,

Guys, I have just noticed this patch didn't get merged. Can you please pick it up? Thanks!

Hi Jan, I think it has been in fstests v2023.02.05 release, as:

commit da69ce70dfcdbd88d6f940d7c1ab346d5a1e5f82

Author: Jan Kara <[email protected]>
Date:   Tue Jan 31 13:54:16 2023 +0100
Quoted 7 lines
Quoted 4 lines

generic/707: Test moving directory while being grown

Isn't it same with this patch? Please check, and feel free to tell me if anything wrong.

Bah, sorry. Not sure why I didn't find it when grepping :-|. I'm sorry for the noise.

No problem, Feel free to ping me if any of your patches get missed :)

Quoted 6 lines

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR