common/rc: explicitly test for engine availability in _require_fio
PATCH v2 1/1 common/rc: explicitly test for engine availability in _require_fio
The current test in _require_fio (--warnings-fatal --showcmd) does not fail if an invalid/unavailable io engine is specified.
Add an explicit test that every requested io engine in the job file is actually available.
Remove the "ioe_e4defrag" entries in ext4 tests - an engine with this name has seemingly never existed, but in each case later stanzas overrode the io engine, so it did not cause problems without this explicit parsing and checking.
---
V2: Be Better at Bash, thanks Zorro. Also add | sort | uniq to ensure
we only test each requested engine once.
common/rc +7 -0
@@ -3977,12 +3977,19 @@ _require_scratch_dev_pool_equal_size() _require_fio() { local job=$1 + local eng _require_command "$FIO_PROG" fio if [ -z "$1" ]; then return 1; fi + # Explicitly check for every ioengine listed in the job + for eng in `sed -n "s/ioengine=\(.*\)/\1/p" $job | sort | uniq`; do + grep -wq $eng <($FIO_PROG --enghelp 2>/dev/null) || \ + _notrun "fio engine $eng not available" + done + $FIO_PROG --warnings-fatal --showcmd $job >> $seqres.full 2>&1 [ $? -eq 0 ] || _notrun "$FIO_PROG too old, see $seqres.full" }
tests/ext4/301 +0 -1
@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}
tests/ext4/302 +0 -1
@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}
tests/ext4/303 +0 -1
@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (3+1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}
tests/ext4/304 +0 -1
@@ -32,7 +32,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}
Quoted 10 lines · Eric Sandeen
The current test in
_require_fio(--warnings-fatal --showcmd) does not fail if an invalid/unavailable io engine is specified.Add an explicit test that every requested io engine in the job file is actually available.
Remove the "
ioe_e4defrag" entries in ext4 tests - an engine with this name has seemingly never existed, but in each case later stanzas overrode the io engine, so it did not cause problems without this explicit parsing and checking.
Well, they override it with the correct name "e4defrag":
$ fio --enghelp
Available IO engines:
cpuio
mmap
sync
<snip>
falloc
e4defragI think the commit message should call that out specifically "...but in each case later stanzas provide the correct ioengine name 'e4defrag', so...").
Quoted 1 line
Signed-off-by Eric Sandeen <[email protected]>
With that amended, this all looks good to me so
--D
Quoted 79 lines
---
V2: Be Better at Bash, thanks Zorro. Also add | sort | uniq to ensure
we only test each requested engine once.common/rc +7
-0@@ -3977,12 +3977,19 @@ _require_scratch_dev_pool_equal_size() _require_fio() { local job=$1 + local eng _require_command "$FIO_PROG" fio if [ -z "$1" ]; then return 1; fi + # Explicitly check for every ioengine listed in the job + for eng in `sed -n "s/ioengine=\(.*\)/\1/p" $job | sort | uniq`; do + grep -wq $eng <($FIO_PROG --enghelp 2>/dev/null) || \ + _notrun "fio engine $eng not available" + done + $FIO_PROG --warnings-fatal --showcmd $job >> $seqres.full 2>&1 [ $? -eq 0 ] || _notrun "$FIO_PROG too old, see $seqres.full" }tests/ext4/301 +0
-1@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}tests/ext4/302 +0
-1@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}tests/ext4/303 +0
-1@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (3+1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}tests/ext4/304 +0
-1@@ -32,7 +32,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}
Quoted 7 lines · Darrick J. Wong
Quoted 6 lines · Eric Sandeen
The current test in
_require_fio(--warnings-fatal --showcmd) does not fail if an invalid/unavailable io engine is specified.Add an explicit test that every requested io engine in the job file is actually available.
Quoted 6 lines
Quoted 4 lines
Remove the "
ioe_e4defrag" entries in ext4 tests - an engine with this name has seemingly never existed, but in each case later stanzas overrode the io engine, so it did not cause problems without this explicit parsing and checking.Well, they override it with the correct name "e4defrag":
and/or with libaio - it doesn't matter what it's overridden with, just that the incorrect name in the global stanza is never used due to re-specification in each subsequent stanza.
I guess I don't really understand how it helps to explicitly describe the contents of each ioengine= line that clearly still exist in each stanza, or why that's critical to the commit log...
Not saying your suggested text is bad. Just saying I don't think it warrants a V3, unless that's a showstopper to get your RVB, in which case I will go back again and take the time to do so.
Thanks,
-Eric
Quoted 100 lines
$ fio --enghelp Available IO engines: cpuio mmap sync <snip> falloc e4defragI think the commit message should call that out specifically "...but in each case later stanzas provide the correct ioengine name 'e4defrag', so...").
Quoted 1 line
Signed-off-by Eric Sandeen <[email protected]>With that amended, this all looks good to me so
Reviewed-by "Darrick J. Wong" <[email protected]>--D
Quoted 79 lines
---
V2: Be Better at Bash, thanks Zorro. Also add | sort | uniq to ensure
we only test each requested engine once.common/rc +7
-0@@ -3977,12 +3977,19 @@ _require_scratch_dev_pool_equal_size() _require_fio() { local job=$1 + local eng _require_command "$FIO_PROG" fio if [ -z "$1" ]; then return 1; fi + # Explicitly check for every ioengine listed in the job + for eng in `sed -n "s/ioengine=\(.*\)/\1/p" $job | sort | uniq`; do + grep -wq $eng <($FIO_PROG --enghelp 2>/dev/null) || \ + _notrun "fio engine $eng not available" + done + $FIO_PROG --warnings-fatal --showcmd $job >> $seqres.full 2>&1 [ $? -eq 0 ] || _notrun "$FIO_PROG too old, see $seqres.full" }tests/ext4/301 +0
-1@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}tests/ext4/302 +0
-1@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}tests/ext4/303 +0
-1@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (3+1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}tests/ext4/304 +0
-1@@ -32,7 +32,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}
Quoted 19 lines · Eric Sandeen
Quoted 7 lines · Darrick J. Wong
Quoted 6 lines · Eric Sandeen
The current test in
_require_fio(--warnings-fatal --showcmd) does not fail if an invalid/unavailable io engine is specified.Add an explicit test that every requested io engine in the job file is actually available.
Quoted 6 lines
Quoted 4 lines
Remove the "
ioe_e4defrag" entries in ext4 tests - an engine with this name has seemingly never existed, but in each case later stanzas overrode the io engine, so it did not cause problems without this explicit parsing and checking.Well, they override it with the correct name "e4defrag":
and/or with libaio - it doesn't matter what it's overridden with, just that the incorrect name in the global stanza is never used due to re-specification in each subsequent stanza.
Quoted 7 lines
I guess I don't really understand how it helps to explicitly describe the contents of each ioengine= line that clearly still exist in each stanza, or why that's critical to the commit log...
Not saying your suggested text is bad. Just saying I don't think it warrants a V3, unless that's a showstopper to get your RVB, in which case I will go back again and take the time to do so.
Oh, the reason I wanted the commit message to mention that the fio config files in ext4/30[1-4] eventually picks the correct engine name is that I looked at the patch, saw the removal of "ioengine=ioe_e4defrag" and wondered "Doesn't that break the test?" The patch doesn't break anything, but that's not immediately obvious from the diff context.
At least it wasn't to me :P
--D
Quoted 104 lines
Thanks,
-EricQuoted 100 lines
$ fio --enghelp Available IO engines: cpuio mmap sync <snip> falloc e4defragI think the commit message should call that out specifically "...but in each case later stanzas provide the correct ioengine name 'e4defrag', so...").
Quoted 1 line
Signed-off-by Eric Sandeen <[email protected]>With that amended, this all looks good to me so
Reviewed-by "Darrick J. Wong" <[email protected]>--D
Quoted 79 lines
---
V2: Be Better at Bash, thanks Zorro. Also add | sort | uniq to ensure
we only test each requested engine once.common/rc +7
-0@@ -3977,12 +3977,19 @@ _require_scratch_dev_pool_equal_size() _require_fio() { local job=$1 + local eng _require_command "$FIO_PROG" fio if [ -z "$1" ]; then return 1; fi + # Explicitly check for every ioengine listed in the job + for eng in `sed -n "s/ioengine=\(.*\)/\1/p" $job | sort | uniq`; do + grep -wq $eng <($FIO_PROG --enghelp 2>/dev/null) || \ + _notrun "fio engine $eng not available" + done + $FIO_PROG --warnings-fatal --showcmd $job >> $seqres.full 2>&1 [ $? -eq 0 ] || _notrun "$FIO_PROG too old, see $seqres.full" }tests/ext4/301 +0
-1@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}tests/ext4/302 +0
-1@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}tests/ext4/303 +0
-1@@ -31,7 +31,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (3+1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}tests/ext4/304 +0
-1@@ -32,7 +32,6 @@ FILE_SIZE=$((BLK_DEV_SIZE * (512 / (2 + 1)))) cat >$fio_config <<EOF # Common e4defrag regression tests [global] -ioengine=ioe_e4defrag iodepth=1 directory=${SCRATCH_MNT} filesize=${FILE_SIZE}