Skip to contentSkip to conversation

docs/devel/testing: iotests check script should be run via 'run'

PATCH In review Peter Maydell 4 participants 5 messages Nov 6 – Nov 11

lore
Share & tools
lore

Peter Maydell

author
#0 lore raw

PATCH v2 1/1 docs/devel/testing: iotests check script should be run via 'run'

The iotests section of the testing documentation suggests directly running the "./check" script. However if you do this then the script complains:

WARNING: iotests is being run from outside of the configure-time python virtual environment [...]
Please use the meson run script:

        /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check

Update the documentation to match this. While we are here, standardize on "run from the root of the build tree", which is how we generally document running things under "run" (e.g. the functional tests), since the iotests don't have a dependency on their current working directory.

Signed-off-by Peter Maydell <[email protected]>

---
v1->v2: Use "./run ./tests-qemu-iotests/check", for consistency with other uses of the run script

---
 docs/devel/testing/main.rst | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)
docs/devel/testing/main.rst +12 -8
index b0ca37fe2ba..3195f997e85 100644
@@ -271,9 +271,13 @@ than "make check" tests and 99% of the code is written in bash or Python
 scripts.  The testing success criteria is golden output comparison, and the
 test files are named with numbers.
 
-To run iotests, make sure QEMU is built successfully, then switch to the
-``tests/qemu-iotests`` directory under the build directory, and run ``./check``
-with desired arguments from there.
+To run iotests, make sure QEMU is built successfully, then switch to
+the build tree and run the ``tests/qemu-iotests/check`` script via the
+``run`` wrapper which sets up the environment:
+
+.. code::
+
+   ./run ./tests/qemu-iotests/check [args]
 
 By default, "raw" format and "file" protocol is used; all tests will be
 executed, except the unsupported ones. You can override the format and protocol

@@ -282,22 +286,22 @@ with arguments:
 .. code::
 
   # test with qcow2 format
-  ./check -qcow2
+  ./run ./tests/qemu-iotests/check -qcow2
   # or test a different protocol
-  ./check -nbd
+  .run ./tests/qemu-iotests/check -nbd
 
 It's also possible to list test numbers explicitly:
 
 .. code::
 
   # run selected cases with qcow2 format
-  ./check -qcow2 001 030 153
+  ./run ./tests/qemu-iotests/check -qcow2 001 030 153
 
 Cache mode can be selected with the "-c" option, which may help reveal bugs
 that are specific to certain cache mode.
 
-More options are supported by the ``./check`` script, run ``./check -h`` for
-help.
+More options are supported by the ``check`` script; run
+``./tests/qemu-iotests/check -h`` for help.
 
 Writing a new test case
 ~~~~~~~~~~~~~~~~~~~~~~~
Signature · 1 line
2.43.0
Reviewed-by on v2 1/1
#1 lore raw
Quoted 15 lines · Peter Maydell

The iotests section of the testing documentation suggests directly running the "./check" script. However if you do this then the script complains:

WARNING: iotests is being run from outside of the configure-time python virtual environment [...]
Please use the meson run script:

        /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check

Update the documentation to match this. While we are here, standardize on "run from the root of the build tree", which is how we generally document running things under "run" (e.g. the functional tests), since the iotests don't have a dependency on their current working directory.

Quoted 7 lines
Signed-off-by Peter Maydell <[email protected]>

---
v1->v2: Use "./run ./tests-qemu-iotests/check", for consistency with other uses of the run script

---
 docs/devel/testing/main.rst | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)
Reviewed-by Daniel P. Berrangé <[email protected]>

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|
Reviewed-by on v2 1/1
#2 lore raw
In reply to Peter Maydell
Quoted 15 lines · Peter Maydell

The iotests section of the testing documentation suggests directly running the "./check" script. However if you do this then the script complains:

WARNING: iotests is being run from outside of the configure-time python virtual environment [...]
Please use the meson run script:

        /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check

Update the documentation to match this. While we are here, standardize on "run from the root of the build tree", which is how we generally document running things under "run" (e.g. the functional tests), since the iotests don't have a dependency on their current working directory.

Quoted 8 lines
Signed-off-by Peter Maydell <[email protected]>

---
v1->v2: Use "./run ./tests-qemu-iotests/check", for consistency with other uses of the run script

---
 docs/devel/testing/main.rst | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)
Reviewed-by Pierrick Bouvier <[email protected]>
Reviewed-by on v2 1/1
#3 lore raw
In reply to Peter Maydell
Quoted 8 lines · Peter Maydell

The iotests section of the testing documentation suggests directly running the "./check" script. However if you do this then the script complains:

WARNING: iotests is being run from outside of the configure-time python virtual environment [...]
Please use the meson run script:

        /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check

Huh. I did a git checkout, mkdir build, cd build && ../configure, then:

$ build/run tests/qemu-iotests/check 5

WARNING: iotests is being run from outside of the configure-time python virtual environment

current python: /home/eblake/qemu/build/pyvenv/bin/python3
pyvenv python:  /home/pyvenv/bin/python3

...

but running directly from the configured directory:

$ cd build
$ ./run tets/qemu-iotests/check 5

does what this patch recommends.

Quoted 10 lines

Update the documentation to match this. While we are here, standardize on "run from the root of the build tree", which is how we generally document running things under "run" (e.g. the functional tests), since the iotests don't have a dependency on their current working directory.

Signed-off-by Peter Maydell <[email protected]>

---
v1->v2: Use "./run ./tests-qemu-iotests/check", for consistency

Typo in the notes - but those don't make it into git, so that one is irrelevant.

Quoted 19 lines

+++ b/docs/devel/testing/main.rst

docs/devel/testing/main.rst +7 -3
@@ -271,9 +271,13 @@ than "make check" tests and 99% of the code is written in bash or Python
 scripts.  The testing success criteria is golden output comparison, and the
 test files are named with numbers.
 
-To run iotests, make sure QEMU is built successfully, then switch to the
-``tests/qemu-iotests`` directory under the build directory, and run ``./check``
-with desired arguments from there.
+To run iotests, make sure QEMU is built successfully, then switch to
+the build tree and run the ``tests/qemu-iotests/check`` script via the
+``run`` wrapper which sets up the environment:
+
+.. code::
+
+   ./run ./tests/qemu-iotests/check [args]
 
 By default, "raw" format and "file" protocol is used; all tests will be
 executed, except the unsupported ones. You can override the format and protocol

docs/devel/testing/main.rst with arguments: · line 291

Quoted 8 lines
docs/devel/testing/main.rst +2 -2
@@ -282,22 +286,22 @@ with arguments:
 .. code::
 
   # test with qcow2 format
-  ./check -qcow2
+  ./run ./tests/qemu-iotests/check -qcow2
   # or test a different protocol
-  ./check -nbd
+  .run ./tests/qemu-iotests/check -nbd

"./run", not ".run".

With that typo fixed,

Reviewed-by Eric Blake <[email protected]>

Technically, "tests/qemu-iotests" and "./tests/qemu-iotests" will have the same behavior to ./run, but while your variant is slightly longer to type, I don't see a problem in spelling it that way.

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization:  qemu.org | libguestfs.org
#4 lore raw
Quoted 12 lines · Eric Blake
Quoted 8 lines · Peter Maydell

The iotests section of the testing documentation suggests directly running the "./check" script. However if you do this then the script complains:

WARNING: iotests is being run from outside of the configure-time python virtual environment [...]
Please use the meson run script:

        /home/pm215/qemu/build/x86-tgt-san-dbg/run ./check

Huh. I did a git checkout, mkdir build, cd build && ../configure, then:

Quoted 7 lines
$ build/run tests/qemu-iotests/check 5

WARNING: iotests is being run from outside of the configure-time python virtual environment

current python: /home/eblake/qemu/build/pyvenv/bin/python3
pyvenv python:  /home/pyvenv/bin/python3

...

Beyond that warning, the 'check' script has bad heuristics for finding the build directory, looking for 'cwd' in this particular case which is wrong. I've sent a path that addresses this which does the job in my testing at least.

Quoted 7 lines

but running directly from the configured directory:

$ cd build
$ ./run tets/qemu-iotests/check 5

does what this patch recommends.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|