Skip to contentSkip to conversation

fstests: _require_dm_target should not always skip DAX capable devices

PATCH Superseded Dave Chinner 3 participants 7 messages Oct 26, 2021 – Oct 27, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fstests: _require_dm_target should not always skip DAX capable devices

From: Dave Chinner <[email protected]>

Recent changes have turned off all dm-error, dm-thin and dm-flakey tests on pmem devices even when we are not explicitly testing DAX. This is a regression resulting in a large number of log recovery tests no longer running on my pmem-based test VMs. I added the "-o dax=never" mount options to these test configs, only to find it still would not run the dm tests even though the filesystem will never use DAX.

Fix this so that the dm target DAX test explicitly ignores the the block device DAX capability when the filesystem is mounted with dax=never and hence we can use all the dm targets when the tests are being run with FSDAX disabled.

---
 common/rc | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
common/rc +15 -6
index 7f693d39..0cbe8a7d 100644
@@ -1965,12 +1965,19 @@ _require_sane_bdev_flush()
 }
 
 # Decide if the scratch filesystem is likely to be mounted in fsdax mode.
-# If there's a dax clause in the mount options we assume the test runner
-# wants us to test DAX; or if the scratch device itself advertises dax mode
-# in sysfs.
-__detect_scratch_fsdax()
+# It goes 3 ways based on mount options::
+#	1. "dax" or "dax=always" means always test using DAX
+#	2. "dax=never" means we'll never use DAX
+#	3. "dax=inode" or nothing means "use scratch dev capability" to
+#	    determine whether DAX is going to be used.
+#
+# Returns 0 if DAX will be used, 1 if DAX is not going to be used.
+__scratch_uses_fsdax()
 {
-	_normalize_mount_options | egrep -q "dax(=always| |$)" && return 0
+	local ops=$(_normalize_mount_options)
+
+	echo $ops | egrep -q "dax(=always| |$)" && return 0
+	echo $ops | grep -q "dax=never" && return 1
 
 	local sysfs="/sys/block/$(_short_dev $SCRATCH_DEV)"
 	test -e "${sysfs}/dax" && return 0
@@ -1982,6 +1989,8 @@ __detect_scratch_fsdax()
 _require_dm_target()
 {
 	local target=$1
+	local fsdax
+	local bdevdax
 
 	# require SCRATCH_DEV to be a valid block device with sane BLKFLSBUF
 	# behaviour
@@ -1989,7 +1998,7 @@ _require_dm_target()
 	_require_sane_bdev_flush $SCRATCH_DEV
 	_require_command "$DMSETUP_PROG" dmsetup
 
-	if __detect_scratch_fsdax; then
+	if __scratch_uses_fsdax; then
 		case $target in
 		stripe|linear|log-writes)
 			;;
Signature · 1 line
2.33.0
#1 lore raw
Quoted 36 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Recent changes have turned off all dm-error, dm-thin and dm-flakey tests on pmem devices even when we are not explicitly testing DAX. This is a regression resulting in a large number of log recovery tests no longer running on my pmem-based test VMs. I added the "-o dax=never" mount options to these test configs, only to find it still would not run the dm tests even though the filesystem will never use DAX.

Fix this so that the dm target DAX test explicitly ignores the the block device DAX capability when the filesystem is mounted with dax=never and hence we can use all the dm targets when the tests are being run with FSDAX disabled.

---
 common/rc | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
common/rc +4 -4
index 7f693d39..0cbe8a7d 100644
@@ -1965,12 +1965,19 @@ _require_sane_bdev_flush()
 }
 
 # Decide if the scratch filesystem is likely to be mounted in fsdax mode.
-# If there's a dax clause in the mount options we assume the test runner
-# wants us to test DAX; or if the scratch device itself advertises dax mode
-# in sysfs.
-__detect_scratch_fsdax()
+# It goes 3 ways based on mount options::
+#	1. "dax" or "dax=always" means always test using DAX
+#	2. "dax=never" means we'll never use DAX
+#	3. "dax=inode" or nothing means "use scratch dev capability" to

common/rc _require_sane_bdev_flush() · line 1980

Quoted 10 lines
common/rc +8 -1
+#	    determine whether DAX is going to be used.
+#
+# Returns 0 if DAX will be used, 1 if DAX is not going to be used.
+__scratch_uses_fsdax()
 {
-	_normalize_mount_options | egrep -q "dax(=always| |$)" && return 0
+	local ops=$(_normalize_mount_options)
+
+	echo $ops | egrep -q "dax(=always| |$)" && return 0
+	echo $ops | grep -q "dax=never" && return 1

Question: Do you want grep -w here so that grep won't match on anything that isn't a whole "word"?

e.g. MOUNT_OPTIONS="-o ihatedax=alwaysp*****gmeoff" shouldn't be a match, right?

--D

Quoted 24 lines
common/rc +3 -1
 	local sysfs="/sys/block/$(_short_dev $SCRATCH_DEV)"
 	test -e "${sysfs}/dax" && return 0
@@ -1982,6 +1989,8 @@ __detect_scratch_fsdax()
 _require_dm_target()
 {
 	local target=$1
+	local fsdax
+	local bdevdax
 
 	# require SCRATCH_DEV to be a valid block device with sane BLKFLSBUF
 	# behaviour
@@ -1989,7 +1998,7 @@ _require_dm_target()
 	_require_sane_bdev_flush $SCRATCH_DEV
 	_require_command "$DMSETUP_PROG" dmsetup
 
-	if __detect_scratch_fsdax; then
+	if __scratch_uses_fsdax; then
 		case $target in
 		stripe|linear|log-writes)
 			;;
Signature · 1 line
2.33.0
#2 lore raw
Quoted 51 lines · Darrick J. Wong
Quoted 36 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Recent changes have turned off all dm-error, dm-thin and dm-flakey tests on pmem devices even when we are not explicitly testing DAX. This is a regression resulting in a large number of log recovery tests no longer running on my pmem-based test VMs. I added the "-o dax=never" mount options to these test configs, only to find it still would not run the dm tests even though the filesystem will never use DAX.

Fix this so that the dm target DAX test explicitly ignores the the block device DAX capability when the filesystem is mounted with dax=never and hence we can use all the dm targets when the tests are being run with FSDAX disabled.

---
 common/rc | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
common/rc +4 -4
index 7f693d39..0cbe8a7d 100644
@@ -1965,12 +1965,19 @@ _require_sane_bdev_flush()
 }
 
 # Decide if the scratch filesystem is likely to be mounted in fsdax mode.
-# If there's a dax clause in the mount options we assume the test runner
-# wants us to test DAX; or if the scratch device itself advertises dax mode
-# in sysfs.
-__detect_scratch_fsdax()
+# It goes 3 ways based on mount options::
+#	1. "dax" or "dax=always" means always test using DAX
+#	2. "dax=never" means we'll never use DAX
+#	3. "dax=inode" or nothing means "use scratch dev capability" to

common/rc _require_sane_bdev_flush() · line 1980

Quoted 10 lines
common/rc +8 -1
+#	    determine whether DAX is going to be used.
+#
+# Returns 0 if DAX will be used, 1 if DAX is not going to be used.
+__scratch_uses_fsdax()
 {
-	_normalize_mount_options | egrep -q "dax(=always| |$)" && return 0
+	local ops=$(_normalize_mount_options)
+
+	echo $ops | egrep -q "dax(=always| |$)" && return 0
+	echo $ops | grep -q "dax=never" && return 1

Question: Do you want grep -w here so that grep won't match on anything that isn't a whole "word"?

Quoted 2 lines

e.g. MOUNT_OPTIONS="-o ihatedax=alwaysp*****gmeoff" shouldn't be a match, right?

At which point fstests will be useless because the scratch device won't mount.... :/

But, sure, if we care about people fuzzing fstests mount options that much we can make it "grep -qw"....

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#3 lore raw
Quoted 52 lines · Dave Chinner
Quoted 51 lines · Darrick J. Wong
Quoted 36 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Recent changes have turned off all dm-error, dm-thin and dm-flakey tests on pmem devices even when we are not explicitly testing DAX. This is a regression resulting in a large number of log recovery tests no longer running on my pmem-based test VMs. I added the "-o dax=never" mount options to these test configs, only to find it still would not run the dm tests even though the filesystem will never use DAX.

Fix this so that the dm target DAX test explicitly ignores the the block device DAX capability when the filesystem is mounted with dax=never and hence we can use all the dm targets when the tests are being run with FSDAX disabled.

---
 common/rc | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
common/rc +4 -4
index 7f693d39..0cbe8a7d 100644
@@ -1965,12 +1965,19 @@ _require_sane_bdev_flush()
 }
 
 # Decide if the scratch filesystem is likely to be mounted in fsdax mode.
-# If there's a dax clause in the mount options we assume the test runner
-# wants us to test DAX; or if the scratch device itself advertises dax mode
-# in sysfs.
-__detect_scratch_fsdax()
+# It goes 3 ways based on mount options::
+#	1. "dax" or "dax=always" means always test using DAX
+#	2. "dax=never" means we'll never use DAX
+#	3. "dax=inode" or nothing means "use scratch dev capability" to

common/rc _require_sane_bdev_flush() · line 1980

Quoted 10 lines
common/rc +8 -1
+#	    determine whether DAX is going to be used.
+#
+# Returns 0 if DAX will be used, 1 if DAX is not going to be used.
+__scratch_uses_fsdax()
 {
-	_normalize_mount_options | egrep -q "dax(=always| |$)" && return 0
+	local ops=$(_normalize_mount_options)
+
+	echo $ops | egrep -q "dax(=always| |$)" && return 0
+	echo $ops | grep -q "dax=never" && return 1

Question: Do you want grep -w here so that grep won't match on anything that isn't a whole "word"?

Quoted 8 lines
Quoted 2 lines

e.g. MOUNT_OPTIONS="-o ihatedax=alwaysp*****gmeoff" shouldn't be a match, right?

At which point fstests will be useless because the scratch device won't mount.... :/

But, sure, if we care about people fuzzing fstests mount options that much we can make it "grep -qw"....

I actually meant someone creating a legitimate mount option for another filesystem that happens to match 'dax=always', not someone feeding bad options. Maybe I should have given as an example:

mount -t deepspace9 -o everydax=alwaystrill /dev/sda /mnt

or something...

--D

Quoted 6 lines

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#4 lore raw

PATCH v2 1/1 fstests: _require_dm_target should not always skip DAX capable devices

From: Dave Chinner <[email protected]>

Recent changes have turned off all dm-error, dm-thin and dm-flakey tests on pmem devices even when we are not explicitly testing DAX. This is a regression resulting in a large number of log recovery tests no longer running on my pmem-based test VMs. I added the "-o dax=never" mount options to these test configs, only to find it still would not run the dm tests even though the filesystem will never use DAX.

Fix this so that the dm target DAX test explicitly ignores the the block device DAX capability when the filesystem is mounted with dax=never and hence we can use all the dm targets when the tests are being run with FSDAX disabled.

---
Version 2:
-use -w for grep command to do exact word matching for options.

 common/rc | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
common/rc +15 -6
index 7f693d39..75197453 100644
@@ -1965,12 +1965,19 @@ _require_sane_bdev_flush()
 }
 
 # Decide if the scratch filesystem is likely to be mounted in fsdax mode.
-# If there's a dax clause in the mount options we assume the test runner
-# wants us to test DAX; or if the scratch device itself advertises dax mode
-# in sysfs.
-__detect_scratch_fsdax()
+# It goes 3 ways based on mount options::
+#	1. "dax" or "dax=always" means always test using DAX
+#	2. "dax=never" means we'll never use DAX
+#	3. "dax=inode" or nothing means "use scratch dev capability" to
+#	    determine whether DAX is going to be used.
+#
+# Returns 0 if DAX will be used, 1 if DAX is not going to be used.
+__scratch_uses_fsdax()
 {
-	_normalize_mount_options | egrep -q "dax(=always| |$)" && return 0
+	local ops=$(_normalize_mount_options)
+
+	echo $ops | egrep -qw "dax(=always| |$)" && return 0
+	echo $ops | grep -qw "dax=never" && return 1
 
 	local sysfs="/sys/block/$(_short_dev $SCRATCH_DEV)"
 	test -e "${sysfs}/dax" && return 0
@@ -1982,6 +1989,8 @@ __detect_scratch_fsdax()
 _require_dm_target()
 {
 	local target=$1
+	local fsdax
+	local bdevdax
 
 	# require SCRATCH_DEV to be a valid block device with sane BLKFLSBUF
 	# behaviour
@@ -1989,7 +1998,7 @@ _require_dm_target()
 	_require_sane_bdev_flush $SCRATCH_DEV
 	_require_command "$DMSETUP_PROG" dmsetup
 
-	if __detect_scratch_fsdax; then
+	if __scratch_uses_fsdax; then
 		case $target in
 		stripe|linear|log-writes)
 			;;
Reviewed-by on v2 1/1
#5 lore raw
Quoted 10 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Recent changes have turned off all dm-error, dm-thin and dm-flakey tests on pmem devices even when we are not explicitly testing DAX. This is a regression resulting in a large number of log recovery tests no longer running on my pmem-based test VMs. I added the "-o dax=never" mount options to these test configs, only to find it still would not run the dm tests even though the filesystem will never use DAX.

Quoted 6 lines

Fix this so that the dm target DAX test explicitly ignores the the block device DAX capability when the filesystem is mounted with dax=never and hence we can use all the dm targets when the tests are being run with FSDAX disabled.

Looks good to me,

--D

Quoted 54 lines

---
Version 2:

(quoted diff) +0 -1
-use -w for grep command to do exact word matching for options.
 
 common/rc | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
common/rc +15 -6
index 7f693d39..75197453 100644
@@ -1965,12 +1965,19 @@ _require_sane_bdev_flush()
 }
 
 # Decide if the scratch filesystem is likely to be mounted in fsdax mode.
-# If there's a dax clause in the mount options we assume the test runner
-# wants us to test DAX; or if the scratch device itself advertises dax mode
-# in sysfs.
-__detect_scratch_fsdax()
+# It goes 3 ways based on mount options::
+#	1. "dax" or "dax=always" means always test using DAX
+#	2. "dax=never" means we'll never use DAX
+#	3. "dax=inode" or nothing means "use scratch dev capability" to
+#	    determine whether DAX is going to be used.
+#
+# Returns 0 if DAX will be used, 1 if DAX is not going to be used.
+__scratch_uses_fsdax()
 {
-	_normalize_mount_options | egrep -q "dax(=always| |$)" && return 0
+	local ops=$(_normalize_mount_options)
+
+	echo $ops | egrep -qw "dax(=always| |$)" && return 0
+	echo $ops | grep -qw "dax=never" && return 1
 
 	local sysfs="/sys/block/$(_short_dev $SCRATCH_DEV)"
 	test -e "${sysfs}/dax" && return 0
@@ -1982,6 +1989,8 @@ __detect_scratch_fsdax()
 _require_dm_target()
 {
 	local target=$1
+	local fsdax
+	local bdevdax
 
 	# require SCRATCH_DEV to be a valid block device with sane BLKFLSBUF
 	# behaviour
@@ -1989,7 +1998,7 @@ _require_dm_target()
 	_require_sane_bdev_flush $SCRATCH_DEV
 	_require_command "$DMSETUP_PROG" dmsetup
 
-	if __detect_scratch_fsdax; then
+	if __scratch_uses_fsdax; then
 		case $target in
 		stripe|linear|log-writes)
 			;;
Reviewed-by on v2 1/1
#6 lore raw
In reply to Dave Chinner
Quoted 10 lines · Dave Chinner

From: Dave Chinner <[email protected]>

Recent changes have turned off all dm-error, dm-thin and dm-flakey tests on pmem devices even when we are not explicitly testing DAX. This is a regression resulting in a large number of log recovery tests no longer running on my pmem-based test VMs. I added the "-o dax=never" mount options to these test configs, only to find it still would not run the dm tests even though the filesystem will never use DAX.

Quoted 7 lines

Fix this so that the dm target DAX test explicitly ignores the the block device DAX capability when the filesystem is mounted with dax=never and hence we can use all the dm targets when the tests are being run with FSDAX disabled.

---

Make sense.

Quoted 54 lines

Version 2:

(quoted diff) +0 -1
-use -w for grep command to do exact word matching for options.
 
 common/rc | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
common/rc +15 -6
index 7f693d39..75197453 100644
@@ -1965,12 +1965,19 @@ _require_sane_bdev_flush()
 }
 
 # Decide if the scratch filesystem is likely to be mounted in fsdax mode.
-# If there's a dax clause in the mount options we assume the test runner
-# wants us to test DAX; or if the scratch device itself advertises dax mode
-# in sysfs.
-__detect_scratch_fsdax()
+# It goes 3 ways based on mount options::
+#	1. "dax" or "dax=always" means always test using DAX
+#	2. "dax=never" means we'll never use DAX
+#	3. "dax=inode" or nothing means "use scratch dev capability" to
+#	    determine whether DAX is going to be used.
+#
+# Returns 0 if DAX will be used, 1 if DAX is not going to be used.
+__scratch_uses_fsdax()
 {
-	_normalize_mount_options | egrep -q "dax(=always| |$)" && return 0
+	local ops=$(_normalize_mount_options)
+
+	echo $ops | egrep -qw "dax(=always| |$)" && return 0
+	echo $ops | grep -qw "dax=never" && return 1
 
 	local sysfs="/sys/block/$(_short_dev $SCRATCH_DEV)"
 	test -e "${sysfs}/dax" && return 0
@@ -1982,6 +1989,8 @@ __detect_scratch_fsdax()
 _require_dm_target()
 {
 	local target=$1
+	local fsdax
+	local bdevdax
 
 	# require SCRATCH_DEV to be a valid block device with sane BLKFLSBUF
 	# behaviour
@@ -1989,7 +1998,7 @@ _require_dm_target()
 	_require_sane_bdev_flush $SCRATCH_DEV
 	_require_command "$DMSETUP_PROG" dmsetup
 
-	if __detect_scratch_fsdax; then
+	if __scratch_uses_fsdax; then
 		case $target in
 		stripe|linear|log-writes)
 			;;