Skip to contentSkip to conversation

fstests: introduce MKFS_BCACHEFS_PROG for bcachefs

PATCH Superseded Su Yue 3 participants 6 messages Jan 29, 2024

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Su Yue

author
#0 lore raw

PATCH v3 1/2 fstests: introduce MKFS_BCACHEFS_PROG for bcachefs

From: Su Yue <[email protected]>

mkfs.bcachefs supports force overwrite when option '-f' is given:

$ mkfs.bcachefs --help | grep force
  -f, --force

There are some tests which call _scratch_mkfs multiple times e.g. tests/generic/171. Without '-f' in MKFS_OPTIONS, these tests just hang in overwrite confirmation. After this commit, MKFS_BCACHEFS_PROG will contains ' -f' so we don't have to add '-f' to MKFS_OPTIONS manually to make these tests pass.

It also fixes generic/466 which unsets MKFS_OPTIONS causing that test hangs in mfks.bcachefs waiting for confirmation of the force overwrite.

---
changelog:
v3:
Add Brian's Reviewed-by.
v2:
Add more details about why MKFS_BCACHEFS_PROG should contain '-f'.

 common/config |  3 ++-
 common/rc     | 12 +++++++++---
 2 files changed, 11 insertions(+), 4 deletions(-)
common/config +2 -1
index c9771ff934cb..1f9edceec57a 100644
@@ -105,7 +105,7 @@ set_mkfs_prog_path_with_opts()
 	# Note: mkfs.f2fs doesn't support the --help option yet, but it doesn't
 	# matter since it also prints the help when an invalid option is given.
 	if [ "$p" != "" ] && \
-		$p --help |& grep -q "[[:space:]]-f[[:space:]|]"; then
+		$p --help |& grep -q "[[:space:]]-f[[:space:]|,]"; then
 		echo "$p -f"
 	else
 		echo $p
@@ -313,6 +313,7 @@ export MKFS_REISER4_PROG=$(type -P mkfs.reiser4)
 export E2FSCK_PROG=$(type -P e2fsck)
 export TUNE2FS_PROG=$(type -P tune2fs)
 export FSCK_OVERLAY_PROG=$(type -P fsck.overlay)
+export MKFS_BCACHEFS_PROG=$(set_mkfs_prog_path_with_opts bcachefs)
 
 # SELinux adds extra xattrs which can mess up our expected output.
 # So, mount with a context, and they won't be created.
common/rc +9 -3
index 524ffa02aa6a..31c21d2a8360 100644
@@ -611,6 +611,9 @@ _test_mkfs()
     xfs)
 	$MKFS_PROG -t $FSTYP -- -f $MKFS_OPTIONS $* $TEST_DEV
 	;;
+    bcachefs)
+    $MKFS_BCACHEFS_PROG $MKFS_OPTIONS $* $TEST_DEV > /dev/null
+	;;
     *)
 	yes | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* $TEST_DEV
 	;;
@@ -753,6 +756,10 @@ _scratch_mkfs()
 		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="grep -v -e ^mkfs\.ocfs2"
 		;;
+    bcachefs)
+		mkfs_cmd="$MKFS_BCACHEFS_PROG"
+		mkfs_filter="cat"
+		;;
 	*)
 		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="cat"
@@ -1044,7 +1051,7 @@ _scratch_mkfs_sized()
 		export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
 		;;
 	bcachefs)
-		$MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS --fs_size=$fssize --block_size=$blocksize $SCRATCH_DEV
+		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize --block_size=$blocksize $SCRATCH_DEV
 		;;
 	*)
 		_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
@@ -1128,8 +1135,7 @@ _scratch_mkfs_blocksized()
 						-C $blocksize $SCRATCH_DEV
 		;;
 	bcachefs)
-		${MKFS_PROG} -t $FSTYP $MKFS_OPTIONS --block_size=$blocksize \
-								$SCRATCH_DEV
+		_scratch_mkfs --block_size=$blocksize
 		;;
 	udf)
 		_scratch_mkfs -b $blocksize
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v3 1/2 fstests: introduce MKFS_BCACHEFS_PROG for bcachefs

From: Su Yue <[email protected]>

mkfs.bcachefs supports force overwrite when option '-f' is given:

$ mkfs.bcachefs --help | grep force
  -f, --force

There are some tests which call _scratch_mkfs multiple times e.g. tests/generic/171. Without '-f' in MKFS_OPTIONS, these tests just hang in overwrite confirmation. After this commit, MKFS_BCACHEFS_PROG will contains ' -f' so we don't have to add '-f' to MKFS_OPTIONS manually to make these tests pass.

It also fixes generic/466 which unsets MKFS_OPTIONS causing that test hangs in mfks.bcachefs waiting for confirmation of the force overwrite.

---
changelog:
v3:
Add Brian's Reviewed-by.
v2:
Add more details about why MKFS_BCACHEFS_PROG should contain '-f'.

 common/config |  3 ++-
 common/rc     | 12 +++++++++---
 2 files changed, 11 insertions(+), 4 deletions(-)
common/config +2 -1
index c9771ff934cb..1f9edceec57a 100644
@@ -105,7 +105,7 @@ set_mkfs_prog_path_with_opts()
 	# Note: mkfs.f2fs doesn't support the --help option yet, but it doesn't
 	# matter since it also prints the help when an invalid option is given.
 	if [ "$p" != "" ] && \
-		$p --help |& grep -q "[[:space:]]-f[[:space:]|]"; then
+		$p --help |& grep -q "[[:space:]]-f[[:space:]|,]"; then
 		echo "$p -f"
 	else
 		echo $p
@@ -313,6 +313,7 @@ export MKFS_REISER4_PROG=$(type -P mkfs.reiser4)
 export E2FSCK_PROG=$(type -P e2fsck)
 export TUNE2FS_PROG=$(type -P tune2fs)
 export FSCK_OVERLAY_PROG=$(type -P fsck.overlay)
+export MKFS_BCACHEFS_PROG=$(set_mkfs_prog_path_with_opts bcachefs)
 
 # SELinux adds extra xattrs which can mess up our expected output.
 # So, mount with a context, and they won't be created.
common/rc +9 -3
index 524ffa02aa6a..31c21d2a8360 100644
@@ -611,6 +611,9 @@ _test_mkfs()
     xfs)
 	$MKFS_PROG -t $FSTYP -- -f $MKFS_OPTIONS $* $TEST_DEV
 	;;
+    bcachefs)
+    $MKFS_BCACHEFS_PROG $MKFS_OPTIONS $* $TEST_DEV > /dev/null
+	;;
     *)
 	yes | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* $TEST_DEV
 	;;
@@ -753,6 +756,10 @@ _scratch_mkfs()
 		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="grep -v -e ^mkfs\.ocfs2"
 		;;
+    bcachefs)
+		mkfs_cmd="$MKFS_BCACHEFS_PROG"
+		mkfs_filter="cat"
+		;;
 	*)
 		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="cat"
@@ -1044,7 +1051,7 @@ _scratch_mkfs_sized()
 		export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
 		;;
 	bcachefs)
-		$MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS --fs_size=$fssize --block_size=$blocksize $SCRATCH_DEV
+		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize --block_size=$blocksize $SCRATCH_DEV
 		;;
 	*)
 		_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
@@ -1128,8 +1135,7 @@ _scratch_mkfs_blocksized()
 						-C $blocksize $SCRATCH_DEV
 		;;
 	bcachefs)
-		${MKFS_PROG} -t $FSTYP $MKFS_OPTIONS --block_size=$blocksize \
-								$SCRATCH_DEV
+		_scratch_mkfs --block_size=$blocksize
 		;;
 	udf)
 		_scratch_mkfs -b $blocksize
Signature · 1 line
2.43.0
author
#2 lore raw

PATCH v3 2/2 common/rc: improve block_size support for bcachefs

In reply to Su Yue

From: Su Yue <[email protected]>

mkfs.bcachefs now supports option '--block_size' to allow custom block_size.

Add the pattern to set def_blksz if MKFS_OPTIONS contains the option in _scratch_mkfs_sized.
Also let mkfs.bcachefs decide blocksize if no option is given in local.config or _scratch_mkfs_sized parameter.

---
changelog:
v3:
Add logic to Let mkfs.bcachefs decide blocksize if no option is given in local.config or _scratch_mkfs_sized parameter.
v2:
Born.

---
 common/rc | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
common/rc +9 -1
index 31c21d2a8360..315a2413f963 100644
@@ -930,6 +930,7 @@ _scratch_mkfs_sized()
 	local fssize=$1
 	local blocksize=$2
 	local def_blksz
+	local blocksize_opt
 
 	case $FSTYP in
 	xfs)
@@ -950,6 +951,13 @@ _scratch_mkfs_sized()
 	jfs)
 		def_blksz=4096
 		;;
+	bcachefs)
+		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*(--block_size)[ =]?+([0-9]+).*/\2/p'`
+		[ -n "$def_blksize" ] && blocksize_opt="--block_size=$def_blksize"
+		[ -n "$blocksize" ] && blocksize_opt="--block_size=$blocksize"
+		# If no block size is given by local.confg or parameter, blocksize_opt is empty.
+		# Let MKFS_BCACHEFS_PROG decide block size on its own.
+		;;
 	esac
 
 	[ -n "$def_blksz" ] && blocksize=$def_blksz
@@ -1051,7 +1059,7 @@ _scratch_mkfs_sized()
 		export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
 		;;
 	bcachefs)
-		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize --block_size=$blocksize $SCRATCH_DEV
+		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize $blocksize_opt $SCRATCH_DEV
 		;;
 	*)
 		_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
Signature · 1 line
2.43.0
author
#3 lore raw

PATCH v3 2/2 common/rc: improve block_size support for bcachefs

In reply to Su Yue

From: Su Yue <[email protected]>

mkfs.bcachefs now supports option '--block_size' to allow custom block_size.

Add the pattern to set def_blksz if MKFS_OPTIONS contains the option in _scratch_mkfs_sized.
Also let mkfs.bcachefs decide blocksize if no option is given in local.config or _scratch_mkfs_sized parameter.

---
changelog:
v3:
Add logic to Let mkfs.bcachefs decide blocksize if no option is given in local.config or _scratch_mkfs_sized parameter.
v2:
Born.

---
 common/rc | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
common/rc +9 -1
index 31c21d2a8360..315a2413f963 100644
@@ -930,6 +930,7 @@ _scratch_mkfs_sized()
 	local fssize=$1
 	local blocksize=$2
 	local def_blksz
+	local blocksize_opt
 
 	case $FSTYP in
 	xfs)
@@ -950,6 +951,13 @@ _scratch_mkfs_sized()
 	jfs)
 		def_blksz=4096
 		;;
+	bcachefs)
+		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*(--block_size)[ =]?+([0-9]+).*/\2/p'`
+		[ -n "$def_blksize" ] && blocksize_opt="--block_size=$def_blksize"
+		[ -n "$blocksize" ] && blocksize_opt="--block_size=$blocksize"
+		# If no block size is given by local.confg or parameter, blocksize_opt is empty.
+		# Let MKFS_BCACHEFS_PROG decide block size on its own.
+		;;
 	esac
 
 	[ -n "$def_blksz" ] && blocksize=$def_blksz
@@ -1051,7 +1059,7 @@ _scratch_mkfs_sized()
 		export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
 		;;
 	bcachefs)
-		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize --block_size=$blocksize $SCRATCH_DEV
+		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize $blocksize_opt $SCRATCH_DEV
 		;;
 	*)
 		_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
Signature · 1 line
2.43.0
#4 lore raw

Re: [PATCH v3 2/2] common/rc: improve block_size support for bcachefs

Quoted 11 lines · Su Yue

From: Su Yue <[email protected]>

mkfs.bcachefs now supports option '--block_size' to allow custom block_size.

Add the pattern to set def_blksz if MKFS_OPTIONS contains the option in _scratch_mkfs_sized.
Also let mkfs.bcachefs decide blocksize if no option is given in local.config or _scratch_mkfs_sized parameter.

Quoted 8 lines

---
changelog:
v3:
Add logic to Let mkfs.bcachefs decide blocksize if no option is given in local.config or _scratch_mkfs_sized parameter.
v2:
Born.
---

Looks like the series duplicates the patches for some reason..

Quoted 16 lines
 common/rc | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
common/rc +1 -0
index 31c21d2a8360..315a2413f963 100644
@@ -930,6 +930,7 @@ _scratch_mkfs_sized()
 	local fssize=$1
 	local blocksize=$2
 	local def_blksz
+	local blocksize_opt
 
 	case $FSTYP in
 	xfs)

common/rc _scratch_mkfs_sized() · line 957

Quoted 7 lines
common/rc +4 -0
@@ -950,6 +951,13 @@ _scratch_mkfs_sized()
 	jfs)
 		def_blksz=4096
 		;;
+	bcachefs)
+		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*(--block_size)[ =]?+([0-9]+).*/\2/p'`
+		[ -n "$def_blksize" ] && blocksize_opt="--block_size=$def_blksize"
+		[ -n "$blocksize" ] && blocksize_opt="--block_size=$blocksize"

This seems reasonable to me, but if I follow this function correctly the behavior when both the param ($blocksize) and MKFS_OPTIONS specify a block size is that MKFS_OPTIONS overrides the former. For bcachefs it looks like the above does the opposite. Should we switch around the above two statements?

Brian

Quoted 19 lines
common/rc +4 -1
+		# If no block size is given by local.confg or parameter, blocksize_opt is empty.
+		# Let MKFS_BCACHEFS_PROG decide block size on its own.
+		;;
 	esac
 
 	[ -n "$def_blksz" ] && blocksize=$def_blksz
@@ -1051,7 +1059,7 @@ _scratch_mkfs_sized()
 		export MOUNT_OPTIONS="-o size=$fssize $TMPFS_MOUNT_OPTIONS"
 		;;
 	bcachefs)
-		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize --block_size=$blocksize $SCRATCH_DEV
+		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize $blocksize_opt $SCRATCH_DEV
 		;;
 	*)
 		_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_sized"
Signature · 1 line
2.43.0
#5 lore raw

Re: [PATCH v3 2/2] common/rc: improve block_size support for bcachefs

Quoted 22 lines · Brian Foster
Quoted 11 lines · Su Yue

From: Su Yue <[email protected]>

mkfs.bcachefs now supports option '--block_size' to allow custom block_size.

Add the pattern to set def_blksz if MKFS_OPTIONS contains the option in _scratch_mkfs_sized.
Also let mkfs.bcachefs decide blocksize if no option is given in local.config or _scratch_mkfs_sized parameter.

Quoted 9 lines

---
changelog:
v3:
Add logic to Let mkfs.bcachefs decide blocksize if no option is given in
local.config or _scratch_mkfs_sized parameter.
v2:
Born.
---

Quoted 2 lines

Looks like the series duplicates the patches for some reason..

Yeah..I sent two patches but 4 patches in
https://lore.kernel.org/fstests/[email protected]/T/#t

Quoted 25 lines
Quoted 25 lines
 common/rc | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
common/rc +3 -0
index 31c21d2a8360..315a2413f963 100644
@@ -930,6 +930,7 @@ _scratch_mkfs_sized()
 	local fssize=$1
 	local blocksize=$2
 	local def_blksz
+	local blocksize_opt
 
 	case $FSTYP in
 	xfs)
@@ -950,6 +951,13 @@ _scratch_mkfs_sized()
 	jfs)
 		def_blksz=4096
 		;;
+	bcachefs)
+		def_blksz=`echo $MKFS_OPTIONS | sed -rn 

's/.*(--block_size)[ =]?+([0-9]+).*/\2/p'`

common/rc +1 -0
+		[ -n "$def_blksize" ] && 

blocksize_opt="--block_size=$def_blksize"

Quoted 9 lines
Quoted 2 lines
common/rc +1 -0
+		[ -n "$blocksize" ] && 

blocksize_opt="--block_size=$blocksize"

This seems reasonable to me, but if I follow this function correctly the behavior when both the param ($blocksize) and MKFS_OPTIONS specify a block size is that MKFS_OPTIONS overrides the former. For bcachefs it looks like the above does the opposite. Should we switch around the above two statements?

Thanks for the catch. Nights turned my brain to mush.

V4 was sent.
--
Su

Quoted 26 lines

Brian

Quoted 24 lines
common/rc +1 -0
+		# If no block size is given by local.confg or 

parameter, blocksize_opt is empty.

common/rc +3 -2
+		# Let MKFS_BCACHEFS_PROG decide block size on its own.
+		;;
 	esac
 
 	[ -n "$def_blksz" ] && blocksize=$def_blksz
@@ -1051,7 +1059,7 @@ _scratch_mkfs_sized()
 		export MOUNT_OPTIONS="-o size=$fssize 
 $TMPFS_MOUNT_OPTIONS"
 		;;
 	bcachefs)
-		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize 
--block_size=$blocksize $SCRATCH_DEV
+		$MKFS_BCACHEFS_PROG $MKFS_OPTIONS --fs_size=$fssize 

$blocksize_opt $SCRATCH_DEV

common/rc +0 -1
 		;;
 	*)
 		_notrun "Filesystem $FSTYP not supported in 
 _scratch_mkfs_sized"
--

2.43.0