Skip to contentSkip to conversation

bcachefs support

PATCH In review Kent Overstreet 7 participants 19 messages Apr 27, 2021 – May 24, 2021 VFS

lore
Share & tools
lore
#0 lore raw

cover bcachefs support

A small patch adding bcachefs support, and two other patches for consideration:

Kent Overstreet (3):
Initial bcachefs support
Improved .gitignore
Use --yes option to lvcreate

 .gitignore         |  3 +++
 common/attr        |  6 ++++++
 common/config      |  3 +++
 common/dmlogwrites |  7 +++++++
 common/quota       |  4 ++--
 common/rc          | 31 +++++++++++++++++++++++++++++++
 tests/generic/042  |  3 ++-
 tests/generic/081  |  2 +-
 tests/generic/108  |  2 +-
 tests/generic/425  |  3 +++
 tests/generic/441  |  2 +-
 tests/generic/482  | 27 ++++++++++++++++++++-------
 tests/generic/558  |  2 ++
 13 files changed, 82 insertions(+), 13 deletions(-)
Signature · 1 line
2.31.1
#1 lore raw

PATCH 1/3 Initial bcachefs support

From: Kent Overstreet <[email protected]>

---
 common/attr        |  6 ++++++
 common/config      |  3 +++
 common/dmlogwrites |  7 +++++++
 common/quota       |  4 ++--
 common/rc          | 31 +++++++++++++++++++++++++++++++
 tests/generic/042  |  3 ++-
 tests/generic/425  |  3 +++
 tests/generic/441  |  2 +-
 tests/generic/482  | 27 ++++++++++++++++++++-------
 tests/generic/558  |  2 ++
 10 files changed, 77 insertions(+), 11 deletions(-)
common/attr +6 -0
index 669909d600..42ceab9233 100644
@@ -33,6 +33,9 @@ _acl_get_max()
 			echo 506
 		fi
 		;;
+	bcachefs)
+		echo 251
+		;;
 	*)
 		echo 0
 		;;
@@ -273,6 +276,9 @@ pvfs2)
 9p|ceph|nfs)
 	MAX_ATTRVAL_SIZE=65536
 	;;
+bcachefs)
+	MAX_ATTRVAL_SIZE=1024
+	;;
 *)
 	# Assume max ~1 block of attrs
 	BLOCK_SIZE=`_get_block_size $TEST_DIR`
common/config +3 -0
index a47e462c77..8153301483 100644
@@ -415,6 +415,9 @@ _mkfs_opts()
 	btrfs)
 		export MKFS_OPTIONS="$BTRFS_MKFS_OPTIONS"
 		;;
+	bcachefs)
+		export MKFS_OPTIONS="--errors=panic"
+		;;
 	*)
 		;;
 	esac
common/dmlogwrites +7 -0
index 573f4b8a56..668d49e995 100644
@@ -111,6 +111,13 @@ _log_writes_replay_log()
 	[ -z "$_blkdev" ] && _fail \
 	"block dev must be specified for _log_writes_replay_log"
 
+	if [ "$FSTYP" = "bcachefs" ]; then
+		# bcachefs gets confused if we're replaying the history out of
+		# order, and we see writes on the device from a newer point in
+		# time than what the superblock points to:
+		dd if=/dev/zero of=$SCRATCH_DEV bs=1M oflag=direct >& /dev/null
+	fi
+
 	$here/src/log-writes/replay-log --log $LOGWRITES_DEV --find \
 		--end-mark $_mark >> $seqres.full 2>&1
 	[ $? -ne 0 ] && _fail "mark '$_mark' does not exist"
common/quota +2 -2
index 32a9a55593..883a28a20d 100644
@@ -17,7 +17,7 @@ _require_quota()
 	    _notrun "Installed kernel does not support quotas"
 	fi
 	;;
-    gfs2|ocfs2)
+    gfs2|ocfs2|bcachefs)
 	;;
     xfs)
 	if [ ! -f /proc/fs/xfs/xqmstat ]; then
@@ -278,7 +278,7 @@ _check_quota_usage()
 
 	VFS_QUOTA=0
 	case $FSTYP in
-	ext2|ext3|ext4|ext4dev|f2fs|reiserfs|gfs2)
+	ext2|ext3|ext4|ext4dev|f2fs|reiserfs|gfs2|bcachefs)
 		VFS_QUOTA=1
 		quotaon -f -u -g $SCRATCH_MNT 2>/dev/null
 		;;
common/rc +31 -0
index 2cf550ec68..0e03846aeb 100644
@@ -334,6 +334,7 @@ _try_scratch_mount()
 		return $?
 	fi
 	_mount -t $FSTYP `_scratch_mount_options $*`
+	return
 }
 
 # mount scratch device with given options and _fail if mount fails
@@ -667,6 +668,9 @@ _test_mkfs()
     ext2|ext3|ext4)
 	$MKFS_PROG -t $FSTYP -- -F $MKFS_OPTIONS $* $TEST_DEV
 	;;
+    bcachefs)
+	$MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* $TEST_DEV
+	;;
     *)
 	yes | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* $TEST_DEV
 	;;
@@ -706,6 +710,10 @@ _mkfs_dev()
 	$MKFS_PROG -t $FSTYP -- -f $MKFS_OPTIONS $* \
 		2>$tmp.mkfserr 1>$tmp.mkfsstd
 	;;
+    bcachefs)
+	$MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* \
+		2>$tmp_dir.mkfserr 1>$tmp_dir.mkfsstd
+	;;
     *)
 	yes | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* \
 		2>$tmp.mkfserr 1>$tmp.mkfsstd
@@ -803,6 +811,10 @@ _scratch_mkfs()
 		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="grep -v -e ^mkfs\.ocfs2"
 		;;
+	bcachefs)
+		mkfs_cmd="$MKFS_PROG -t $FSTYP --"
+		mkfs_filter="cat"
+		;;
 	*)
 		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="cat"
@@ -1065,6 +1077,9 @@ _scratch_mkfs_sized()
 		fi
 		export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
 		;;
+	bcachefs)
+		$MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS --fs_size=$fssize --block_size=$blocksize $SCRATCH_DEV
+		;;
 	*)
 		_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
 		;;
@@ -1133,6 +1148,9 @@ _scratch_mkfs_blocksized()
     ocfs2)
 	yes | ${MKFS_PROG} -t $FSTYP -F $MKFS_OPTIONS -b $blocksize -C $blocksize $SCRATCH_DEV
 	;;
+    bcachefs)
+	${MKFS_PROG}.$FSTYP $MKFS_OPTIONS --block_size=$blocksize $SCRATCH_DEV
+	;;
     *)
 	_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_blocksized"
 	;;
@@ -1179,6 +1197,19 @@ _repair_scratch_fs()
 	fi
 	return $res
         ;;
+    bcachefs)
+	fsck -t $FSTYP -n $SCRATCH_DEV 2>&1
+	res=$?
+	case $res in
+	0)
+		res=0
+		;;
+	*)
+		_dump_err2 "fsck.$FSTYP failed, err=$res"
+		;;
+	esac
+	return $res
+	;;
     *)
 	local dev=$SCRATCH_DEV
 	local fstyp=$FSTYP
tests/generic/042 +2 -1
index 35727bcbc6..42919e2313 100755
@@ -63,7 +63,8 @@ _crashtest()
 
 	# We should /never/ see 0xCD in the file, because we wrote that pattern
 	# to the filesystem image to expose stale data.
-	if hexdump -v -e '/1 "%02X "' $file | grep -q "CD"; then
+	# The file is not required to exist since we didn't sync before going down:
+	if [[ -f $file ]] && hexdump -v -e '/1 "%02X "' $file | grep -q "CD"; then
 		echo "Saw stale data!!!"
 		hexdump $file
 	fi
tests/generic/425 +3 -0
index 51cbe1c67d..be2bc1b02e 100755
@@ -30,6 +30,9 @@ _cleanup()
 
 # real QA test starts here
 _supported_fs generic
+
+[ $FSTYP = bcachefs ] && _notrun "bcachefs does not store xattrs in blocks"
+
 _require_scratch
 _require_attrs
 _require_xfs_io_command "fiemap" "-a"
tests/generic/441 +1 -1
index bedbcb0817..814387b2a9 100755
@@ -40,7 +40,7 @@ case $FSTYP in
 	btrfs)
 		_notrun "btrfs has a specialized test for this"
 		;;
-	ext3|ext4|xfs)
+	ext3|ext4|xfs|bcachefs)
 		# Do the more thorough test if we have a logdev
 		_has_logdev && sflag=''
 		;;
tests/generic/482 +20 -7
index 86941e8468..3cbe187f2e 100755
@@ -77,16 +77,29 @@ prev=$(_log_writes_mark_to_entry_number mkfs)
 cur=$(_log_writes_find_next_fua $prev)
 [ -z "$cur" ] && _fail "failed to locate next FUA write"
 
+if [ "$FSTYP" = "bcachefs" ]; then
+    _dmthin_cleanup
+    _dmthin_init $devsize $devsize $csize $lowspace
+fi
+
 while [ ! -z "$cur" ]; do
 	_log_writes_replay_log_range $cur $DMTHIN_VOL_DEV >> $seqres.full
 
-	# Here we need extra mount to replay the log, mainly for journal based
-	# fs, as their fsck will report dirty log as error.
-	# We don't care to preserve any data on the replay dev, as we can replay
-	# back to the point we need, and in fact sometimes creating/deleting
-	# snapshots repeatedly can be slower than replaying the log.
-	_dmthin_mount
-	_dmthin_check_fs
+	if [ "$FSTYP" = "bcachefs" ]; then
+	    # bcachefs will get confused if fsck does writes to replay the log,
+	    # but then we replay writes from an earlier point in time on the
+	    # same fs - but  fsck in -n mode won't do any writes:
+	    _check_scratch_fs -n $DMTHIN_VOL_DEV
+	else
+	    # Here we need extra mount to replay the log, mainly for journal based
+	    # fs, as their fsck will report dirty log as error.
+	    # We don't care to preserve any data on the replay dev, as we can replay
+	    # back to the point we need, and in fact sometimes creating/deleting
+	    # snapshots repeatedly can be slower than replaying the log.
+
+	    _dmthin_mount
+	    _dmthin_check_fs
+	fi
 
 	prev=$cur
 	cur=$(_log_writes_find_next_fua $(($cur + 1)))
tests/generic/558 +2 -0
index 4bed90e2a7..2eaa8f7686 100755
@@ -55,6 +55,8 @@ _scratch_mount
 i=0
 free_inode=`_get_free_inode $SCRATCH_MNT`
 file_per_dir=1000
+[ $FSTYP = bcachefs ] && file_per_dir=10000
+
 loop=$((free_inode / file_per_dir + 1))
 mkdir -p $SCRATCH_MNT/testdir
 
Signature · 1 line
2.31.1
#2 lore raw

Re: [PATCH 1/3] Initial bcachefs support

Quoted 1 line · Kent Overstreet

From: Kent Overstreet <[email protected]>

Better to add commit logs at least to give an example about how to setup fstests to test bcachefs.

Quoted 44 lines
---
 common/attr        |  6 ++++++
 common/config      |  3 +++
 common/dmlogwrites |  7 +++++++
 common/quota       |  4 ++--
 common/rc          | 31 +++++++++++++++++++++++++++++++
 tests/generic/042  |  3 ++-
 tests/generic/425  |  3 +++
 tests/generic/441  |  2 +-
 tests/generic/482  | 27 ++++++++++++++++++++-------
 tests/generic/558  |  2 ++
 10 files changed, 77 insertions(+), 11 deletions(-)
common/attr +6 -0
index 669909d600..42ceab9233 100644
@@ -33,6 +33,9 @@ _acl_get_max()
 			echo 506
 		fi
 		;;
+	bcachefs)
+		echo 251
+		;;
 	*)
 		echo 0
 		;;
@@ -273,6 +276,9 @@ pvfs2)
 9p|ceph|nfs)
 	MAX_ATTRVAL_SIZE=65536
 	;;
+bcachefs)
+	MAX_ATTRVAL_SIZE=1024
+	;;
 *)
 	# Assume max ~1 block of attrs
 	BLOCK_SIZE=`_get_block_size $TEST_DIR`
common/config +0 -0
index a47e462c77..8153301483 100644

common/config _mkfs_opts() · line 419

Quoted 5 lines
common/config +2 -0
@@ -415,6 +415,9 @@ _mkfs_opts()
 	btrfs)
 		export MKFS_OPTIONS="$BTRFS_MKFS_OPTIONS"
 		;;
+	bcachefs)
+		export MKFS_OPTIONS="--errors=panic"

I think this might be useful for bcachefs developers to find bugs earlier, but not suitable for people like QA, as this may crash their test machines and stop the whole test.

You could always set MKFS_OPTIONS to "--errors=panic" explicitly when needed.

Quoted 9 lines
common/config +1 -0
+		;;
 	*)
 		;;
 	esac
common/dmlogwrites +0 -0
index 573f4b8a56..668d49e995 100644

common/dmlogwrites _log_writes_replay_log() · line 118

Quoted 8 lines
common/dmlogwrites +5 -0
@@ -111,6 +111,13 @@ _log_writes_replay_log()
 	[ -z "$_blkdev" ] && _fail \
 	"block dev must be specified for _log_writes_replay_log"
 
+	if [ "$FSTYP" = "bcachefs" ]; then
+		# bcachefs gets confused if we're replaying the history out of
+		# order, and we see writes on the device from a newer point in
+		# time than what the superblock points to:
+		dd if=/dev/zero of=$SCRATCH_DEV bs=1M oflag=direct >& /dev/null

I don't know bcachefs internals, I'm not sure I understand this, clearing the first 1M of SCRATCH_DEV seems to clear superblock, but I'm still not sure why it's needed. Does wipefs work?

Quoted 32 lines
common/dmlogwrites +2 -0
+	fi
+
 	$here/src/log-writes/replay-log --log $LOGWRITES_DEV --find \
 		--end-mark $_mark >> $seqres.full 2>&1
 	[ $? -ne 0 ] && _fail "mark '$_mark' does not exist"
common/quota +2 -2
index 32a9a55593..883a28a20d 100644
@@ -17,7 +17,7 @@ _require_quota()
 	    _notrun "Installed kernel does not support quotas"
 	fi
 	;;
-    gfs2|ocfs2)
+    gfs2|ocfs2|bcachefs)
 	;;
     xfs)
 	if [ ! -f /proc/fs/xfs/xqmstat ]; then
@@ -278,7 +278,7 @@ _check_quota_usage()
 
 	VFS_QUOTA=0
 	case $FSTYP in
-	ext2|ext3|ext4|ext4dev|f2fs|reiserfs|gfs2)
+	ext2|ext3|ext4|ext4dev|f2fs|reiserfs|gfs2|bcachefs)
 		VFS_QUOTA=1
 		quotaon -f -u -g $SCRATCH_MNT 2>/dev/null
 		;;
common/rc +0 -0
index 2cf550ec68..0e03846aeb 100644

common/rc _try_scratch_mount() · line 337

Quoted 4 lines
common/rc +1 -0
@@ -334,6 +334,7 @@ _try_scratch_mount()
 		return $?
 	fi
 	_mount -t $FSTYP `_scratch_mount_options $*`
+	return

Seems not necessary.

common/rc _test_mkfs() · line 673

Quoted 10 lines
common/rc +3 -0
 }
 
 # mount scratch device with given options and _fail if mount fails
@@ -667,6 +668,9 @@ _test_mkfs()
     ext2|ext3|ext4)
 	$MKFS_PROG -t $FSTYP -- -F $MKFS_OPTIONS $* $TEST_DEV
 	;;
+    bcachefs)
+	$MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* $TEST_DEV
+	;;

I think we could just use the default mkfs command below. The only difference is dropping the "yes | " part, but that does nothing if mkfs doesn't read "yes" or "no" from stdin.

common/rc _mkfs_dev() · line 716

Quoted 11 lines
common/rc +4 -0
     *)
 	yes | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* $TEST_DEV
 	;;
@@ -706,6 +710,10 @@ _mkfs_dev()
 	$MKFS_PROG -t $FSTYP -- -f $MKFS_OPTIONS $* \
 		2>$tmp.mkfserr 1>$tmp.mkfsstd
 	;;
+    bcachefs)
+	$MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* \
+		2>$tmp_dir.mkfserr 1>$tmp_dir.mkfsstd
+	;;

Same as above.

common/rc _scratch_mkfs() · line 817

Quoted 11 lines
common/rc +4 -0
     *)
 	yes | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* \
 		2>$tmp.mkfserr 1>$tmp.mkfsstd
@@ -803,6 +811,10 @@ _scratch_mkfs()
 		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="grep -v -e ^mkfs\.ocfs2"
 		;;
+	bcachefs)
+		mkfs_cmd="$MKFS_PROG -t $FSTYP --"
+		mkfs_filter="cat"
+		;;

Same here.

Quoted 14 lines
common/rc +3 -0
 	*)
 		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="cat"
@@ -1065,6 +1077,9 @@ _scratch_mkfs_sized()
 		fi
 		export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
 		;;
+	bcachefs)
+		$MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS --fs_size=$fssize --block_size=$blocksize $SCRATCH_DEV
+		;;
 	*)
 		_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
 		;;

common/rc _scratch_mkfs_blocksized() · line 1153

Quoted 6 lines
common/rc +3 -0
@@ -1133,6 +1148,9 @@ _scratch_mkfs_blocksized()
     ocfs2)
 	yes | ${MKFS_PROG} -t $FSTYP -F $MKFS_OPTIONS -b $blocksize -C $blocksize $SCRATCH_DEV
 	;;
+    bcachefs)
+	${MKFS_PROG}.$FSTYP $MKFS_OPTIONS --block_size=$blocksize $SCRATCH_DEV
+	;;

Better to be consistent and use "${MKFS_PROG} -t $FSTYP ..." instead of ${MKFS_PROG}.$FSTYP

common/rc _repair_scratch_fs() · line 1201

Quoted 9 lines
common/rc +2 -0
     *)
 	_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_blocksized"
 	;;
@@ -1179,6 +1197,19 @@ _repair_scratch_fs()
 	fi
 	return $res
         ;;
+    bcachefs)
+	fsck -t $FSTYP -n $SCRATCH_DEV 2>&1

_repair_scratch_fs() is supposed to actually fix the errors, does "fsck -n" fix errors for bcachefs?

Quoted 19 lines
common/rc +11 -0
+	res=$?
+	case $res in
+	0)
+		res=0
+		;;
+	*)
+		_dump_err2 "fsck.$FSTYP failed, err=$res"
+		;;
+	esac
+	return $res
+	;;
     *)
 	local dev=$SCRATCH_DEV
 	local fstyp=$FSTYP
tests/generic/042 +0 -0
index 35727bcbc6..42919e2313 100755

tests/generic/042 _crashtest() · line 70

Quoted 9 lines
tests/generic/042 +2 -1
@@ -63,7 +63,8 @@ _crashtest()
 
 	# We should /never/ see 0xCD in the file, because we wrote that pattern
 	# to the filesystem image to expose stale data.
-	if hexdump -v -e '/1 "%02X "' $file | grep -q "CD"; then
+	# The file is not required to exist since we didn't sync before going down:
+	if [[ -f $file ]] && hexdump -v -e '/1 "%02X "' $file | grep -q "CD"; then
 		echo "Saw stale data!!!"
 		hexdump $file
 	fi

Updates for individual test should be in a separate patch.

Quoted 73 lines
tests/generic/425 +3 -0
index 51cbe1c67d..be2bc1b02e 100755
@@ -30,6 +30,9 @@ _cleanup()
 
 # real QA test starts here
 _supported_fs generic
+
+[ $FSTYP = bcachefs ] && _notrun "bcachefs does not store xattrs in blocks"
+
 _require_scratch
 _require_attrs
 _require_xfs_io_command "fiemap" "-a"
tests/generic/441 +1 -1
index bedbcb0817..814387b2a9 100755
@@ -40,7 +40,7 @@ case $FSTYP in
 	btrfs)
 		_notrun "btrfs has a specialized test for this"
 		;;
-	ext3|ext4|xfs)
+	ext3|ext4|xfs|bcachefs)
 		# Do the more thorough test if we have a logdev
 		_has_logdev && sflag=''
 		;;
tests/generic/482 +20 -7
index 86941e8468..3cbe187f2e 100755
@@ -77,16 +77,29 @@ prev=$(_log_writes_mark_to_entry_number mkfs)
 cur=$(_log_writes_find_next_fua $prev)
 [ -z "$cur" ] && _fail "failed to locate next FUA write"
 
+if [ "$FSTYP" = "bcachefs" ]; then
+    _dmthin_cleanup
+    _dmthin_init $devsize $devsize $csize $lowspace
+fi
+
 while [ ! -z "$cur" ]; do
 	_log_writes_replay_log_range $cur $DMTHIN_VOL_DEV >> $seqres.full
 
-	# Here we need extra mount to replay the log, mainly for journal based
-	# fs, as their fsck will report dirty log as error.
-	# We don't care to preserve any data on the replay dev, as we can replay
-	# back to the point we need, and in fact sometimes creating/deleting
-	# snapshots repeatedly can be slower than replaying the log.
-	_dmthin_mount
-	_dmthin_check_fs
+	if [ "$FSTYP" = "bcachefs" ]; then
+	    # bcachefs will get confused if fsck does writes to replay the log,
+	    # but then we replay writes from an earlier point in time on the
+	    # same fs - but  fsck in -n mode won't do any writes:
+	    _check_scratch_fs -n $DMTHIN_VOL_DEV
+	else
+	    # Here we need extra mount to replay the log, mainly for journal based
+	    # fs, as their fsck will report dirty log as error.
+	    # We don't care to preserve any data on the replay dev, as we can replay
+	    # back to the point we need, and in fact sometimes creating/deleting
+	    # snapshots repeatedly can be slower than replaying the log.
+
+	    _dmthin_mount
+	    _dmthin_check_fs
+	fi
 
 	prev=$cur
 	cur=$(_log_writes_find_next_fua $(($cur + 1)))
tests/generic/558 +0 -0
index 4bed90e2a7..2eaa8f7686 100755

tests/generic/558 _scratch_mount · line 59

Quoted 5 lines
tests/generic/558 +2 -0
@@ -55,6 +55,8 @@ _scratch_mount
 i=0
 free_inode=`_get_free_inode $SCRATCH_MNT`
 file_per_dir=1000
+[ $FSTYP = bcachefs ] && file_per_dir=10000
+

Some comments would be good.

Thanks,
Eryu

Quoted 5 lines
tests/generic/558 +0 -0
 loop=$((free_inode / file_per_dir + 1))
 mkdir -p $SCRATCH_MNT/testdir
 
Signature · 1 line
2.31.1
#3 lore raw

Re: [PATCH 1/3] Initial bcachefs support

Quoted 8 lines · Eryu Guan
Quoted 1 line · Kent Overstreet

From: Kent Overstreet <[email protected]>

Better to add commit logs at least to give an example about how to setup fstests to test bcachefs.

You could always set MKFS_OPTIONS to "--errors=panic" explicitly when needed.

Forgot that was an option - doing that now.

Quoted 13 lines
Quoted 13 lines
common/config +1 -0
+		;;
 	*)
 		;;
 	esac
common/dmlogwrites +0 -0
index 573f4b8a56..668d49e995 100644
@@ -111,6 +111,13 @@ _log_writes_replay_log()
 	[ -z "$_blkdev" ] && _fail \
 	"block dev must be specified for _log_writes_replay_log"
 
Quoted 9 lines

common/dmlogwrites _log_writes_replay_log() · line 118

Quoted 5 lines
common/dmlogwrites +5 -0
+	if [ "$FSTYP" = "bcachefs" ]; then
+		# bcachefs gets confused if we're replaying the history out of
+		# order, and we see writes on the device from a newer point in
+		# time than what the superblock points to:
+		dd if=/dev/zero of=$SCRATCH_DEV bs=1M oflag=direct >& /dev/null

I don't know bcachefs internals, I'm not sure I understand this, clearing the first 1M of SCRATCH_DEV seems to clear superblock, but I'm still not sure why it's needed. Does wipefs work?

It's not just the superblock we need to clear, it's really all metadata - the journal, and btree nodes are also log structured in bcachefs. So 1M actually isn't sufficient - the better solution would be to either

- change the tests to check the markers in the log in the correct order, so we never see metadata from a future point in time, also making sure we don't do any writes to the filesystem when we're checking the different markers, or
- just replay to a new dm-thin device

This is basically what I did for generic/482, the 1M zerout is really just a hack for 455 and 457 and should probably be moved there, unless you've got another suggestion.

Quoted 11 lines

common/rc _try_scratch_mount() · line 337

Quoted 9 lines
common/rc +1 -0
index 2cf550ec68..0e03846aeb 100644
@@ -334,6 +334,7 @@ _try_scratch_mount()
 		return $?
 	fi
 	_mount -t $FSTYP `_scratch_mount_options $*`
+	return

Seems not necessary.

Not sure how that got in, dropped it.

Quoted 7 lines

common/rc _test_mkfs() · line 673

Quoted 3 lines
common/rc +3 -0
+    bcachefs)
+	$MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* $TEST_DEV
+	;;

I think we could just use the default mkfs command below. The only difference is dropping the "yes | " part, but that does nothing if mkfs doesn't read "yes" or "no" from stdin.

That dates from when my test environment had SIGPIPE set up wrong (systemd!), it's fixed now so I've dropped these.

Quoted 4 lines
Quoted 4 lines
common/rc +0 -0
     *)
 	_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_blocksized"
 	;;
@@ -1179,6 +1197,19 @@ _repair_scratch_fs()
Quoted 8 lines

common/rc _repair_scratch_fs() · line 1201

Quoted 5 lines
common/rc +2 -0
 	fi
 	return $res
         ;;
+    bcachefs)
+	fsck -t $FSTYP -n $SCRATCH_DEV 2>&1

_repair_scratch_fs() is supposed to actually fix the errors, does "fsck -n" fix errors for bcachefs?

No - but with bcachefs fsck finding errors _always_ indicates a bug, so for the purposes of these tests I think this is the right thing to do - I don't want the tests to pass if fsck is finding and fixing errors.

Quoted 6 lines
Quoted 6 lines
tests/generic/042 +0 -0
index 35727bcbc6..42919e2313 100755
@@ -63,7 +63,8 @@ _crashtest()
 
Quoted 10 lines

tests/generic/042 _crashtest() · line 70

Quoted 8 lines
tests/generic/042 +2 -1
 	# We should /never/ see 0xCD in the file, because we wrote that pattern
 	# to the filesystem image to expose stale data.
-	if hexdump -v -e '/1 "%02X "' $file | grep -q "CD"; then
+	# The file is not required to exist since we didn't sync before going down:
+	if [[ -f $file ]] && hexdump -v -e '/1 "%02X "' $file | grep -q "CD"; then
 		echo "Saw stale data!!!"
 		hexdump $file
 	fi

Updates for individual test should be in a separate patch.

Ok, I'll split those out.

#4 lore raw

Re: [PATCH 1/3] Initial bcachefs support

On Sun, May 23, 2021 at 06:51:49PM -0400, Kent Overstreet wrote:
[snip]

Quoted 11 lines
Quoted 10 lines

common/rc _repair_scratch_fs() · line 1201

Quoted 7 lines
common/rc +2 -0
 	;;
@@ -1179,6 +1197,19 @@ _repair_scratch_fs()
 	fi
 	return $res
         ;;
+    bcachefs)
+	fsck -t $FSTYP -n $SCRATCH_DEV 2>&1

_repair_scratch_fs() is supposed to actually fix the errors, does "fsck -n" fix errors for bcachefs?

Quoted 3 lines

No - but with bcachefs fsck finding errors _always_ indicates a bug, so for the purposes of these tests I think this is the right thing to do - I don't want the tests to pass if fsck is finding and fixing errors.

Then _check_scratch_fs() should be used instead, which will fail the test if any fsck finds any corruptions. _repair_scratch_fs() is meant to fix errors, and only report failure when there's unfixable errors.

Thanks,
Eryu

Quoted 19 lines
Quoted 7 lines
Quoted 7 lines
tests/generic/042 +0 -0
index 35727bcbc6..42919e2313 100755
@@ -63,7 +63,8 @@ _crashtest()
 
Quoted 10 lines

tests/generic/042 _crashtest() · line 70

Quoted 8 lines
tests/generic/042 +2 -1
 	# We should /never/ see 0xCD in the file, because we wrote that pattern
 	# to the filesystem image to expose stale data.
-	if hexdump -v -e '/1 "%02X "' $file | grep -q "CD"; then
+	# The file is not required to exist since we didn't sync before going down:
+	if [[ -f $file ]] && hexdump -v -e '/1 "%02X "' $file | grep -q "CD"; then
 		echo "Saw stale data!!!"
 		hexdump $file
 	fi

Updates for individual test should be in a separate patch.

Ok, I'll split those out.

#5 lore raw

Re: [PATCH 1/3] Initial bcachefs support

Quoted 14 lines · Eryu Guan

On Sun, May 23, 2021 at 06:51:49PM -0400, Kent Overstreet wrote:
[snip]

Quoted 11 lines
Quoted 10 lines

common/rc _repair_scratch_fs() · line 1201

Quoted 7 lines
common/rc +2 -0
 	;;
@@ -1179,6 +1197,19 @@ _repair_scratch_fs()
 	fi
 	return $res
         ;;
+    bcachefs)
+	fsck -t $FSTYP -n $SCRATCH_DEV 2>&1

_repair_scratch_fs() is supposed to actually fix the errors, does "fsck -n" fix errors for bcachefs?

Quoted 7 lines
Quoted 3 lines

No - but with bcachefs fsck finding errors _always_ indicates a bug, so for the purposes of these tests I think this is the right thing to do - I don't want the tests to pass if fsck is finding and fixing errors.

Then _check_scratch_fs() should be used instead, which will fail the test if any fsck finds any corruptions. _repair_scratch_fs() is meant to fix errors, and only report failure when there's unfixable errors.

I see no reason to make such a change to generic tests that were written for other filesystems, when this gets me exactly what I want.

#6 lore raw

Re: [PATCH 1/3] Initial bcachefs support

Quoted 19 lines · Kent Overstreet
Quoted 18 lines · Eryu Guan

On Sun, May 23, 2021 at 06:51:49PM -0400, Kent Overstreet wrote:
[snip]

Quoted 14 lines
Quoted 10 lines

common/rc _repair_scratch_fs() · line 1201

Quoted 7 lines
common/rc +2 -0
 	;;
@@ -1179,6 +1197,19 @@ _repair_scratch_fs()
 	fi
 	return $res
         ;;
+    bcachefs)
+	fsck -t $FSTYP -n $SCRATCH_DEV 2>&1

_repair_scratch_fs() is supposed to actually fix the errors, does "fsck -n" fix errors for bcachefs?

No - but with bcachefs fsck finding errors _always_ indicates a bug, so for the purposes of these tests I think this is the right thing to do - I don't want the tests to pass if fsck is finding and fixing errors.

Quoted 6 lines
Quoted 3 lines

Then _check_scratch_fs() should be used instead, which will fail the test if any fsck finds any corruptions. _repair_scratch_fs() is meant to fix errors, and only report failure when there's unfixable errors.

I see no reason to make such a change to generic tests that were written for other filesystems, when this gets me exactly what I want.

Oh, I noticed that "with bcachefs fsck finding errors _always_ indicates a bug" now.. Then I think using fsck -n is fine here, but better with comments to describe why this is ok for bcachefs.

Also, we could just use _check_scratch_fs here instead of the open-coded fsck command. As _check_scratch_fs calls _check_generic_filesystem() which calls fsck -n internally, and handles mount/umount device correctly and prints pretty log if there's any errors.

Thanks,
Eryu

#7 lore raw

Re: [PATCH 1/3] Initial bcachefs support

Quoted 8 lines · Eryu Guan

Oh, I noticed that "with bcachefs fsck finding errors _always_ indicates a bug" now.. Then I think using fsck -n is fine here, but better with comments to describe why this is ok for bcachefs.

Also, we could just use _check_scratch_fs here instead of the open-coded fsck command. As _check_scratch_fs calls _check_generic_filesystem() which calls fsck -n internally, and handles mount/umount device correctly and prints pretty log if there's any errors.

That should work.

#9 lore raw

PATCH 3/3 Use --yes option to lvcreate

In reply to Kent Overstreet

This fixes spurious test failures caused by broken pipe messages.

---
 tests/generic/081 | 2 +-
 tests/generic/108 | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
tests/generic/081 +1 -1
index 5dff079852..26702007ab 100755
@@ -70,7 +70,7 @@ _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
+$LVM_PROG lvcreate --yes -L 256M -n $lvname $vgname >>$seqres.full 2>&1
 # wait for lvcreation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1
 
tests/generic/108 +1 -1
index 6fb194f43c..74945fdf3c 100755
@@ -56,7 +56,7 @@ $LVM_PROG pvcreate -f $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -i 2 -I 4m -L 275m -n $lvname $vgname \
+$LVM_PROG lvcreate --yes -i 2 -I 4m -L 275m -n $lvname $vgname \
 	>>$seqres.full 2>&1
 # wait for lv creation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1
Signature · 1 line
2.31.1
#10 lore raw

Re: [PATCH 3/3] Use --yes option to lvcreate

Quoted 13 lines · Kent Overstreet

This fixes spurious test failures caused by broken pipe messages.

---
 tests/generic/081 | 2 +-
 tests/generic/108 | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
tests/generic/081 +0 -0
index 5dff079852..26702007ab 100755

tests/generic/081 _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1 · line 73

Quoted 5 lines
tests/generic/081 +1 -1
@@ -70,7 +70,7 @@ _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
+$LVM_PROG lvcreate --yes -L 256M -n $lvname $vgname >>$seqres.full 2>&1

Please see above comments, we use yes pipe intentionally. I don't see how this would result in broken pipe. Would you please provide more details? And let's see if we could fix the broken pipe issue.

Thanks,
Eryu

Quoted 18 lines
tests/generic/081 +0 -0
 # wait for lvcreation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1
 
tests/generic/108 +1 -1
index 6fb194f43c..74945fdf3c 100755
@@ -56,7 +56,7 @@ $LVM_PROG pvcreate -f $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -i 2 -I 4m -L 275m -n $lvname $vgname \
+$LVM_PROG lvcreate --yes -i 2 -I 4m -L 275m -n $lvname $vgname \
 	>>$seqres.full 2>&1
 # wait for lv creation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1
Signature · 1 line
2.31.1
#11 lore raw

Re: [PATCH 3/3] Use --yes option to lvcreate

Quoted 9 lines · Eryu Guan
Quoted 8 lines · Kent Overstreet

This fixes spurious test failures caused by broken pipe messages.

---
 tests/generic/081 | 2 +-
 tests/generic/108 | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
Quoted 14 lines

tests/generic/081 _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1 · line 73

Quoted 10 lines
tests/generic/081 +1 -1
index 5dff079852..26702007ab 100755
@@ -70,7 +70,7 @@ _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
+$LVM_PROG lvcreate --yes -L 256M -n $lvname $vgname >>$seqres.full 2>&1

Please see above comments, we use yes pipe intentionally. I don't see how this would result in broken pipe. Would you please provide more details? And let's see if we could fix the broken pipe issue.

If lvcreate never ask y/n - never reads from standard input, then echo sees a broken pipe when it tries to write. That's what I get without this patch.

#12 lore raw

Re: [PATCH 3/3] Use --yes option to lvcreate

Quoted 25 lines · Kent Overstreet
Quoted 9 lines · Eryu Guan
Quoted 8 lines · Kent Overstreet

This fixes spurious test failures caused by broken pipe messages.

---
 tests/generic/081 | 2 +-
 tests/generic/108 | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
Quoted 14 lines

tests/generic/081 _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1 · line 73

Quoted 10 lines
tests/generic/081 +1 -1
index 5dff079852..26702007ab 100755
@@ -70,7 +70,7 @@ _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
+$LVM_PROG lvcreate --yes -L 256M -n $lvname $vgname >>$seqres.full 2>&1

Please see above comments, we use yes pipe intentionally. I don't see how this would result in broken pipe. Would you please provide more details? And let's see if we could fix the broken pipe issue.

Quoted 2 lines

If lvcreate never ask y/n - never reads from standard input, then echo sees a broken pipe when it tries to write. That's what I get without this patch.

I think it's something in how ktest sets up the environment. I also see the SIGPIPEs when using your ktest scripts, but not when ssh'ing into the guest and running the test.

What that thing is, I don't know. I'm not tall enough to understand signal handling.

#13 lore raw

Re: [PATCH 3/3] Use --yes option to lvcreate

Quoted 29 lines · Matthew Wilcox
Quoted 25 lines · Kent Overstreet
Quoted 9 lines · Eryu Guan
Quoted 8 lines · Kent Overstreet

This fixes spurious test failures caused by broken pipe messages.

---
 tests/generic/081 | 2 +-
 tests/generic/108 | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
Quoted 14 lines

tests/generic/081 _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1 · line 73

Quoted 10 lines
tests/generic/081 +1 -1
index 5dff079852..26702007ab 100755
@@ -70,7 +70,7 @@ _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
+$LVM_PROG lvcreate --yes -L 256M -n $lvname $vgname >>$seqres.full 2>&1

Please see above comments, we use yes pipe intentionally. I don't see how this would result in broken pipe. Would you please provide more details? And let's see if we could fix the broken pipe issue.

Quoted 2 lines

If lvcreate never ask y/n - never reads from standard input, then echo sees a broken pipe when it tries to write. That's what I get without this patch.

Quoted 6 lines

I think it's something in how ktest sets up the environment. I also see the SIGPIPEs when using your ktest scripts, but not when ssh'ing into the guest and running the test.

What that thing is, I don't know. I'm not tall enough to understand signal handling.

See xfstests commit 9bcb266cd778 ("generic/397: be compatible with ignored SIGPIPE") for an example of the same problem being fixed in another test, with some more explanation.

But it's better to just not execute xfstests (or any shell script, for that matter) in an environment where SIGPIPE is ignored.

- Eric

#14 lore raw

Re: [PATCH 3/3] Use --yes option to lvcreate

Quoted 37 lines · Eric Biggers
Quoted 29 lines · Matthew Wilcox
Quoted 25 lines · Kent Overstreet
Quoted 9 lines · Eryu Guan
Quoted 8 lines · Kent Overstreet

This fixes spurious test failures caused by broken pipe messages.

---
 tests/generic/081 | 2 +-
 tests/generic/108 | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
Quoted 14 lines

tests/generic/081 _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1 · line 73

Quoted 10 lines
tests/generic/081 +1 -1
index 5dff079852..26702007ab 100755
@@ -70,7 +70,7 @@ _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
+$LVM_PROG lvcreate --yes -L 256M -n $lvname $vgname >>$seqres.full 2>&1

Please see above comments, we use yes pipe intentionally. I don't see how this would result in broken pipe. Would you please provide more details? And let's see if we could fix the broken pipe issue.

Quoted 2 lines

If lvcreate never ask y/n - never reads from standard input, then echo sees a broken pipe when it tries to write. That's what I get without this patch.

Quoted 6 lines

I think it's something in how ktest sets up the environment. I also see the SIGPIPEs when using your ktest scripts, but not when ssh'ing into the guest and running the test.

What that thing is, I don't know. I'm not tall enough to understand signal handling.

Quoted 6 lines

See xfstests commit 9bcb266cd778 ("generic/397: be compatible with ignored SIGPIPE") for an example of the same problem being fixed in another test, with some more explanation.

But it's better to just not execute xfstests (or any shell script, for that matter) in an environment where SIGPIPE is ignored.

Ah, thanks for the info. I'll see if I can figure out what's mucking with the signal handler (probably systemd...)

#15 lore raw
In reply to Kent Overstreet
Quoted 1 line · Kent Overstreet

A small patch adding bcachefs support, and two other patches for consideration:

As bcachefs is not upstream yet, I think we should re-visit bcachefs support after it's in upstream.

OTOH, I have some minor comments go to patch 1.

Thanks,
Eryu

Quoted 23 lines

Kent Overstreet (3):
Initial bcachefs support
Improved .gitignore
Use --yes option to lvcreate

 .gitignore         |  3 +++
 common/attr        |  6 ++++++
 common/config      |  3 +++
 common/dmlogwrites |  7 +++++++
 common/quota       |  4 ++--
 common/rc          | 31 +++++++++++++++++++++++++++++++
 tests/generic/042  |  3 ++-
 tests/generic/081  |  2 +-
 tests/generic/108  |  2 +-
 tests/generic/425  |  3 +++
 tests/generic/441  |  2 +-
 tests/generic/482  | 27 ++++++++++++++++++++-------
 tests/generic/558  |  2 ++
 13 files changed, 82 insertions(+), 13 deletions(-)
Signature · 1 line
2.31.1
#16 lore raw
Quoted 5 lines · Eryu Guan
Quoted 1 line · Kent Overstreet

A small patch adding bcachefs support, and two other patches for consideration:

As bcachefs is not upstream yet, I think we should re-visit bcachefs support after it's in upstream.

I disagree completely. I've been waiting for this to land for some time so I can actually run fstests against bcachefs easily to evaluate it's current state of stability and support. The plans are to get bcachefs merged upstream, and so having support already in fstests makes it much easier for reviewers and developers to actually run tests and find problems prior to merging.

As an upstream developer and someone who will be reviewing bcachefs when it is next proposed for merge, I would much prefer to see extensive and long term fstests coverage *before* the code is even merged upstream. Given that filesystems take years to develop to the point where they are stable and ready for merge, saying "can't enable the test environment until it is merged upstream" is not very helpful.

As a general principle, we want developers of new filesystems to start using fstests early in the development process of their filesystem. We should be encouraging new filesystems to be added to fstests, not saying "we only support upstream filesystems". If the filesystem plans to be merged upstream, then fstests support for that filesystem should be there long before the filesytsem is even proposed for merge. We need to help people get new filesystems upstream, not place arbitrary "not upstream so not supported" catch-22s in their way...

Hence I ask that you merge bcachefs support to help the process of getting bcachefs suport upstream.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#17 lore raw
Quoted 32 lines · Dave Chinner
Quoted 5 lines · Eryu Guan
Quoted 1 line · Kent Overstreet

A small patch adding bcachefs support, and two other patches for consideration:

As bcachefs is not upstream yet, I think we should re-visit bcachefs support after it's in upstream.

I disagree completely. I've been waiting for this to land for some time so I can actually run fstests against bcachefs easily to evaluate it's current state of stability and support. The plans are to get bcachefs merged upstream, and so having support already in fstests makes it much easier for reviewers and developers to actually run tests and find problems prior to merging.

As an upstream developer and someone who will be reviewing bcachefs when it is next proposed for merge, I would much prefer to see extensive and long term fstests coverage *before* the code is even merged upstream. Given that filesystems take years to develop to the point where they are stable and ready for merge, saying "can't enable the test environment until it is merged upstream" is not very helpful.

As a general principle, we want developers of new filesystems to start using fstests early in the development process of their filesystem. We should be encouraging new filesystems to be added to fstests, not saying "we only support upstream filesystems". If the filesystem plans to be merged upstream, then fstests support for that filesystem should be there long before the filesytsem is even proposed for merge. We need to help people get new filesystems upstream, not place arbitrary "not upstream so not supported" catch-22s in their way...

Quoted 2 lines

Hence I ask that you merge bcachefs support to help the process of getting bcachefs suport upstream.

/me notes that both Eryu and Dave have been willing to merge surprisingly large quantities of code for XFS reflink and online fsck long before either of those features landed in mainline, so I think it's perfectly fine to merge Kent's relatively small changes to enable 'FSTYP=bcachefs'.

(With all of Eryu's review comments fixed, obviously...)

--D

Quoted 6 lines

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#18 lore raw
Quoted 23 lines · Darrick J. Wong
Quoted 22 lines · Dave Chinner
Quoted 5 lines · Eryu Guan
Quoted 1 line · Kent Overstreet

A small patch adding bcachefs support, and two other patches for consideration:

As bcachefs is not upstream yet, I think we should re-visit bcachefs support after it's in upstream.

I disagree completely. I've been waiting for this to land for some time so I can actually run fstests against bcachefs easily to evaluate it's current state of stability and support. The plans are to get bcachefs merged upstream, and so having support already in fstests makes it much easier for reviewers and developers to actually run tests and find problems prior to merging.

As an upstream developer and someone who will be reviewing bcachefs when it is next proposed for merge, I would much prefer to see extensive and long term fstests coverage *before* the code is even merged upstream. Given that filesystems take years to develop to the point where they are stable and ready for merge, saying "can't enable the test environment until it is merged upstream" is not very helpful.

Quoted 9 lines
Quoted 9 lines

As a general principle, we want developers of new filesystems to start using fstests early in the development process of their filesystem. We should be encouraging new filesystems to be added to fstests, not saying "we only support upstream filesystems". If the filesystem plans to be merged upstream, then fstests support for that filesystem should be there long before the filesytsem is even proposed for merge. We need to help people get new filesystems upstream, not place arbitrary "not upstream so not supported" catch-22s in their way...

OK, that makes sense. Actually, I should have made my "upstream first" more clear. I'd love to merge non-upstream features/new fs supports if the proposed new feature/new filesystem has been developmented actively and the community has generally made the agreement that will merge the new feature/filesystem when it's in a good shape. IOW, it's not some random new features/filesystems, which are very likely to be dropped in the half way.

And providing such info in the patch is very helpful to reviewers.

Quoted 3 lines
Quoted 3 lines

Hence I ask that you merge bcachefs support to help the process of getting bcachefs suport upstream.

Sure, bcachefs looks promising to me now :)

Quoted 4 lines

/me notes that both Eryu and Dave have been willing to merge surprisingly large quantities of code for XFS reflink and online fsck long before either of those features landed in mainline, so I think it's

Because I know you and the xfs commnity will work on it continuously and very unlikely make the new tests dead code :)

Thanks,
Eryu

Quoted 13 lines

perfectly fine to merge Kent's relatively small changes to enable 'FSTYP=bcachefs'.

(With all of Eryu's review comments fixed, obviously...)

--D

Quoted 6 lines

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]