Skip to content

RFC tests: run python tests under the build/tests/venv environment

John Snow qemu-devel 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest9 patches 34 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/9python: update for mypy 0.950python/qemu/qmp/util.py python/setup.cfg+4 -12 replies
2/9tests: add "TESTS_PYTHON" variable to Makefiletests/Makefile.include+4 -31 reply
3/9tests: install "qemu" namespace package into venvtests/Makefile.include tests/avocado/avocado_qemu/__init__.py tests/avocado/virtio_check_params.py +2 more+10 -92 replies
4/9tests: silence pip upgrade warnings during venv creationtests/Makefile.include+3 -32 replies
5/9tests: use tests/venv to run basevm.py-based scriptstests/vm/Makefile.include tests/vm/basevm.py+10 -91 reply
6/9tests: add check-venv as a dependency of check and check-blocktests/Makefile.include+8 -14 replies
7/9tests: add check-venv to build-tcg-disabled CI recipe.gitlab-ci.d/buildtest.yml+1 -01 reply
8/9iotests: fix source directory locationtests/qemu-iotests/testenv.py+2 -21 reply
9/9iotests: use tests/venv for running teststests/qemu-iotests/testenv.py+17 -78 replies

Reviews trailers per patch

6 of 9 patches have a Reviewed-by or Acked-by.

PatchPaolo Bonzini
1/9R-b
2/9R-b
3/9R-b
4/9
5/9R-b
6/9
7/9R-b
8/9R-b
9/9

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [RFC PATCH 0/9] tests: run python tests under the build/tests/venv environment

RFC: This is a very early, crude attempt at switching over to an external Python package dependency for QMP. This series does not actually make the switch in and of itself, but instead just switches to the paradigm of using a venv in general to install the QEMU python packages instead of using PYTHONPATH to load them from the source tree.

(By installing the package, we can process dependencies.)

I'm sending it to the list so I can show you some of what's ugly so far and my notes on how I might make it less ugly.

(1) This doesn't trigger venv creation *from* iotests, it merely prints a friendly error message if "make check-venv" has not been run first. Not the greatest.

(2) This isn't acceptable for SRPM builds, because it uses PyPI to fetch packages just-in-time. My thought is to use an environment variable like QEMU_CHECK_NO_INTERNET that changes the behavior of the venv setup process. We can use "--system-site-packages" as an argument to venv creation and "--no-index" as an argument to pip installation to achieve good behavior in SRPM building scenarios. It'd be up to the spec-writer to opt into that behavior.

(3) Using one venv for *all* tests means that avocado comes as a pre-req for iotests -- which adds avocado as a BuildRequires for the Fedora SRPM. That's probably not ideal. It may be better to model the test venv as something that can be created in stages: the "core" venv first, and the avocado packages only when needed.

You can see in these patches that I wasn't really sure how to tie the check-venv step as a dependency of 'check' or 'check-block', and it winds up feeling kind of hacky and fragile as a result.

(Patches 6 and 7 feel particularly fishy.)

What I think I would like to do is replace the makefile logic with a Python bootstrapping script. This will allow me to add in environment variable logic to accommodate #2 pretty easily. It will also allow iotests to call into the bootstrap script whenever it detects the venv isn't set up, which it needed to do anyway in order to print a user-friendly error message. Lastly, it will make it easier to create a "tiered" venv that layers in the avocado dependencies only as-needed, which avoids us having to bloat the SRPM build dependencies.

In the end, I think that approach will:

- Allow us to run iotests without having to run a manual prep step
- Keep additional SRPM deps to a minimum
- Keep makefile hacks to a minimum

The only downside I am really frowning at is that I will have to replicate some "update the venv if it's outdated" logic that is usually handled by the Make system in the venv bootstrapper. Still, I think it's probably the only way to hit all of the requirements here without trying to concoct a fairly complex Makefile.

any thoughts? If not, I'll just move on to trying to hack up that version next instead.

--js

John Snow (9):
python: update for mypy 0.950
tests: add "TESTS_PYTHON" variable to Makefile
tests: install "qemu" namespace package into venv
tests: silence pip upgrade warnings during venv creation
tests: use tests/venv to run basevm.py-based scripts
tests: add check-venv as a dependency of check and check-block
tests: add check-venv to build-tcg-disabled CI recipe
iotests: fix source directory location
iotests: use tests/venv for running tests

 .gitlab-ci.d/buildtest.yml             |  1 +
 python/qemu/qmp/util.py                |  4 +++-
 python/setup.cfg                       |  1 +
 tests/Makefile.include                 | 23 +++++++++++++++------
 tests/avocado/avocado_qemu/__init__.py | 11 +++++-----
 tests/avocado/virtio_check_params.py   |  1 -
 tests/avocado/virtio_version.py        |  1 -
 tests/qemu-iotests/testenv.py          | 28 +++++++++++++++++---------
 tests/requirements.txt                 |  1 +
 tests/vm/Makefile.include              | 13 ++++++------
 tests/vm/basevm.py                     |  6 +++---
 11 files changed, 57 insertions(+), 33 deletions(-)
Signature · 1 line
2.34.1