avocado: use sha1 for fc31 imgs to avoid first time re-download
PATCH 1/1 avocado: use sha1 for fc31 imgs to avoid first time re-download
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1], DEFAULT_HASH_ALGORITHM is set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.
Back in QEMU, the LinuxDistro class defines the images it is going to use in the KNOWN_DISTROS attribute. All images have the 'checksum' attribute, defined using sha256, that will make avocado check if it has the right asset in the cache. So this means that we're downloading a sha1 image during check-avocado, then trying to fetch it using a sha256 hash.
This doesn't work. Avocado will match the provided checksum, a sha256 hash, with the sha1 hash from the existing CHECKSUM file for the asset, and will re-download the image again. Avocado doesn't take into account that we're using a different hash algorithm than the one used to fetch the image via get-vm-image-fedora31 (in avocado: utils/asset.py, self.find_asset_file() and fetch(self, timeout=None). The new download will take into account the chosen hash algorithm, and subsequent avocado runs will not re-download the image again because now it's comparing sha256 with sha256.
All of this, of course, assuming that the system was able to re-download the image before the avocado test timeouts. A more "fun" scenario happens if the network is not fast enough to download the image during the avocado timeout, the test will fail and the sha1 image will remain in the cache. And the test will keep trying to re-fetch it, and will keep failing with a timeout.
There's something to be said about avocado matching different sha algorithms instead of re-calculating the hash with the chosen algorithm during fetch(), especially considering that the CHECKSUM file contains the name of the algorithm used. An avocado feature request [2] was opened to see if this is a feasible change to do.
For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
[1] https://github.com/avocado-framework/avocado.git @ 942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496
--- tests/avocado/avocado_qemu/__init__.py | 30 ++++++++++++++++++-------- 1 file changed, 21 insertions(+), 9 deletions(-)
tests/avocado/avocado_qemu/__init__.py +21 -9
@@ -423,12 +423,24 @@ class LinuxDistro: Holds information of known distros. """ #: A collection of known distros and their respective image checksum + # + # 'get-vm-image-fedora-31' target from Makefile.include will + # download the images using avocado default 'sha1' algorithm. + # This happens because the command line 'avocado vmimage get' + # from Makefile.include is not able to handle an extra --algorithm + # argument. + # + # To avoid a re-download of a recent image download due to a sha + # mismatch in the first test run, all Fedora 31 images are using + # 'sha1' instead of 'sha256'. + # + # FIXME: revisit this change if/when avocado learns to do CHECKSUM + # matches using different algorithms. KNOWN_DISTROS = { 'fedora': { '31': { 'x86_64': - {'checksum': ('e3c1b309d9203604922d6e255c2c5d09' - '8a309c2d46215d8fc026954f3c5c27a0'), + {'checksum': ('f62f8eabbf3687ea610c495bd59551a0025f99b7'), 'pxeboot_url': ('https://archives.fedoraproject.org/' 'pub/archive/fedora/linux/releases/31/' 'Everything/x86_64/os/images/pxeboot/'),
@@ -438,8 +450,7 @@ class LinuxDistro: 'console=ttyS0,115200n8'), }, 'aarch64': - {'checksum': ('1e18d9c0cf734940c4b5d5ec592facae' - 'd2af0ad0329383d5639c997fdf16fe49'), + {'checksum': ('39d1fe099cdecacae894480d421be57d18b4e854'), 'pxeboot_url': 'https://archives.fedoraproject.org/' 'pub/archive/fedora/linux/releases/31/' 'Everything/aarch64/os/images/pxeboot/',
@@ -450,11 +461,9 @@ class LinuxDistro: ' console=ttyAMA0'), }, 'ppc64': - {'checksum': ('7c3528b85a3df4b2306e892199a9e1e4' - '3f991c506f2cc390dc4efa2026ad2f58')}, + {'checksum': ('9993dc28e7c49ceb52125f9513130dfe2ace026c')}, 's390x': - {'checksum': ('4caaab5a434fd4d1079149a072fdc789' - '1e354f834d355069ca982fdcaf5a122d')}, + {'checksum': ('455f017b82decf32f366e06e7a7d0f6da86f96a7')}, }, '32': { 'aarch64':
@@ -595,15 +604,18 @@ def download_boot(self): self.log.info('Downloading/preparing boot image') # Fedora 31 only provides ppc64le images image_arch = self.arch + hash_algorithm = 'sha256' if self.distro.name == 'fedora': if image_arch == 'ppc64': image_arch = 'ppc64le' + if self.distro.version == '31': + hash_algorithm = 'sha1' try: boot = vmimage.get( self.distro.name, arch=image_arch, version=self.distro.version, checksum=self.distro.checksum, - algorithm='sha256', + algorithm=hash_algorithm, cache_dir=self.cache_dirs[0], snapshot_dir=self.workdir) except:
Signature · 1 line
2.37.3
Quoted 10 lines · Daniel Henrique Barboza
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1],
DEFAULT_HASH_ALGORITHMis set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.
Quoted 6 lines
For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
[1] https://github.com/avocado-framework/avocado.git @
942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496
Can we just ask Avocado maintainers to fix this problem on their side to allow use of a modern hash alg as a priority item. We've already had this problem in QEMU for over a year AFAICT, so doesn't seem like we need to urgently do a workaround on QEMU side, so we can get Avocado devs to commit to fixing it in the next month.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 15 lines · Daniel P. Berrangé
Quoted 14 lines · Daniel Henrique Barboza
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1],
DEFAULT_HASH_ALGORITHMis set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
Quoted 8 lines
Quoted 2 lines
[1] https://github.com/avocado-framework/avocado.git @
942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496Can we just ask Avocado maintainers to fix this problem on their side to allow use of a modern hash alg as a priority item. We've already had this problem in QEMU for over a year AFAICT, so doesn't seem like we need to urgently do a workaround on QEMU side, so we can get Avocado devs to commit to fixing it in the next month.
Sure, I think it would be better if we can get an Avocado side fix to make the current QEMU usage viable.
For now I'll use this patch locally because this issue is 100% reproducible on my home machine :( apparently my internet is capable of 4k streaming but, for some reason, it doesn't manage to download 350Mb from the Fedora mirror in less that 180 seconds. 'make check-avocado' always gives me a timeout in 'test_pseries_tcg'.
Thanks,
Daniel
Quoted 4 lines
With regards,
Daniel
Quoted 15 lines · Daniel P. Berrangé
Quoted 14 lines · Daniel Henrique Barboza
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1],
DEFAULT_HASH_ALGORITHMis set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
Quoted 8 lines
Quoted 2 lines
[1] https://github.com/avocado-framework/avocado.git @
942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496Can we just ask Avocado maintainers to fix this problem on their side to allow use of a modern hash alg as a priority item. We've already had this problem in QEMU for over a year AFAICT, so doesn't seem like we need to urgently do a workaround on QEMU side, so we can get Avocado devs to commit to fixing it in the next month.
Do we have such a commitment? ... The avocado version in QEMU is completely backlevel these days, it's still using version 88.1 from May 2021, i.e. there hasn't been any update since more than a year. I recently tried to bump it to a newer version on my own (since I'm still suffering from the problem that find_free_port() does not work if you don't have a local IPv6 address), but it's not that straight forward since the recent versions of avocado changed a lot of things (e.g. the new nrunner - do we want to run tests in parallel? If so it breaks a lot of the timeout settings, I think), so an update needs a lot of careful testing...
So unless someone is really committing to spend a lot of time on updating Avocado in QEMU in the near future, I don't think that such a fix for the hash algorithm will happen any time soon, and thus I think we should consider to include this work-around for the time being.
Thomas
Quoted 25 lines · Thomas Huth
Quoted 15 lines · Daniel P. Berrangé
Quoted 14 lines · Daniel Henrique Barboza
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1],
DEFAULT_HASH_ALGORITHMis set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
Quoted 8 lines
Quoted 2 lines
[1] https://github.com/avocado-framework/avocado.git @
942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496Can we just ask Avocado maintainers to fix this problem on their side to allow use of a modern hash alg as a priority item. We've already had this problem in QEMU for over a year AFAICT, so doesn't seem like we need to urgently do a workaround on QEMU side, so we can get Avocado devs to commit to fixing it in the next month.
Quoted 9 lines
Do we have such a commitment? ... The avocado version in QEMU is completely backlevel these days, it's still using version 88.1 from May 2021, i.e. there hasn't been any update since more than a year. I recently tried to bump it to a newer version on my own (since I'm still suffering from the problem that
find_free_port()does not work if you don't have a local IPv6 address), but it's not that straight forward since the recent versions of avocado changed a lot of things (e.g. the new nrunner - do we want to run tests in parallel? If so it breaks a lot of the timeout settings, I think), so an update needs a lot of careful testing...
That it is so difficult to update Avocado after barely more than 1 year is not exactly a strong vote of confidence in our continued use of Avocado long term :-(
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Quoted 36 lines · Daniel P. Berrangé
Quoted 25 lines · Thomas Huth
Quoted 15 lines · Daniel P. Berrangé
Quoted 14 lines · Daniel Henrique Barboza
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1],
DEFAULT_HASH_ALGORITHMis set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
Quoted 8 lines
Quoted 2 lines
[1] https://github.com/avocado-framework/avocado.git @
942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496Can we just ask Avocado maintainers to fix this problem on their side to allow use of a modern hash alg as a priority item. We've already had this problem in QEMU for over a year AFAICT, so doesn't seem like we need to urgently do a workaround on QEMU side, so we can get Avocado devs to commit to fixing it in the next month.
Quoted 9 lines
Do we have such a commitment? ... The avocado version in QEMU is completely backlevel these days, it's still using version 88.1 from May 2021, i.e. there hasn't been any update since more than a year. I recently tried to bump it to a newer version on my own (since I'm still suffering from the problem that
find_free_port()does not work if you don't have a local IPv6 address), but it's not that straight forward since the recent versions of avocado changed a lot of things (e.g. the new nrunner - do we want to run tests in parallel? If so it breaks a lot of the timeout settings, I think), so an update needs a lot of careful testing...
Quoted 3 lines
That it is so difficult to update Avocado after barely more than 1 year is not exactly a strong vote of confidence in our continued use of Avocado long term :-(
By the way, Avocado just provided a fix for the problem this patch is trying to amend:
https://github.com/avocado-framework/avocado/pull/5515#issuecomment-1308872846
Is there an easy way to plug upstream Avocado into QEMU? I would like to test tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg to see if the problem is fixed by Avocado upstream.
Thanks,
Daniel
Quoted 3 lines
With regards,
Daniel
Quoted 42 lines · Daniel Henrique Barboza
Quoted 36 lines · Daniel P. Berrangé
Quoted 25 lines · Thomas Huth
Quoted 15 lines · Daniel P. Berrangé
Quoted 14 lines · Daniel Henrique Barboza
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1],
DEFAULT_HASH_ALGORITHMis set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
Quoted 8 lines
Quoted 2 lines
[1] https://github.com/avocado-framework/avocado.git @
942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496Can we just ask Avocado maintainers to fix this problem on their side to allow use of a modern hash alg as a priority item. We've already had this problem in QEMU for over a year AFAICT, so doesn't seem like we need to urgently do a workaround on QEMU side, so we can get Avocado devs to commit to fixing it in the next month.
Quoted 9 lines
Do we have such a commitment? ... The avocado version in QEMU is completely backlevel these days, it's still using version 88.1 from May 2021, i.e. there hasn't been any update since more than a year. I recently tried to bump it to a newer version on my own (since I'm still suffering from the problem that
find_free_port()does not work if you don't have a local IPv6 address), but it's not that straight forward since the recent versions of avocado changed a lot of things (e.g. the new nrunner - do we want to run tests in parallel? If so it breaks a lot of the timeout settings, I think), so an update needs a lot of careful testing...Quoted 3 lines
That it is so difficult to update Avocado after barely more than 1 year is not exactly a strong vote of confidence in our continued use of Avocado long term :-(
Quoted 4 lines
By the way, Avocado just provided a fix for the problem this patch is trying to amend:
https://github.com/avocado-framework/avocado/pull/5515#issuecomment-1308872846
Thanks Jan!
Quoted 3 lines
Is there an easy way to plug upstream Avocado into QEMU? I would like to test tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg to see if the problem is fixed by Avocado upstream.
See https://lore.kernel.org/qemu-devel/[email protected]/
For your case:
-- >8 --
tests/requirements.txt +1 -1
@@ -4,3 +4,3 @@ # Note that qemu.git/python/ is always implicitly installed. -avocado-framework==88.1 +-e git+https://github.com/avocado-framework/avocado.git@b31b868c882d4650d3b7d2fbfc9b8ac0f2c3672b#egg=avocado-framework pycdlib==1.11.0
---
Regards,
Phil.
Quoted 28 lines · Philippe Mathieu-Daudé
Quoted 27 lines · Daniel Henrique Barboza
Quoted 26 lines · Daniel P. Berrangé
Quoted 25 lines · Thomas Huth
Quoted 23 lines · Daniel P. Berrangé
Quoted 10 lines · Daniel Henrique Barboza
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1],
DEFAULT_HASH_ALGORITHMis set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.Quoted 6 lines
For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
[1] https://github.com/avocado-framework/avocado.git @
942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496Can we just ask Avocado maintainers to fix this problem on their side to allow use of a modern hash alg as a priority item. We've already had this problem in QEMU for over a year AFAICT, so doesn't seem like we need to urgently do a workaround on QEMU side, so we can get Avocado devs to commit to fixing it in the next month.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Do we have such a commitment? ... The avocado version in QEMU is completely backlevel these days, it's still using version 88.1 from May 2021, i.e. there hasn't been any update since more than a year. I recently tried to bump it to a newer version on my own (since I'm still suffering from the problem that
find_free_port()does not work if you don't have a local IPv6 address), but it's not that straight forward since the recent versions of avocado changed a lot of things (e.g. the new nrunner - do we want to run tests in parallel? If so it breaks a lot of the timeout settings, I think), so an update needs a lot of careful testing...
Hi Daniel,
if the problem of migrating avocado to latest version on qemu is only in parallel run, I would suggest to disable it with nrunner.max_parallel_tasks [1]. Even that the differences between avocado legacy runner and nrunner is huge, the migration should be straight forward. So if you have more issues with migration to the nrunner, I would be happy to help you with that.
[1] https://avocado-framework.readthedocs.io/en/latest/config/index.html#nrunner-max-parallel-tasks
- Jan
Quoted 38 lines
Quoted 10 lines
Quoted 4 lines
That it is so difficult to update Avocado after barely more than 1 year is not exactly a strong vote of confidence in our continued use of Avocado long term :-(
By the way, Avocado just provided a fix for the problem this patch is trying to amend:
https://github.com/avocado-framework/avocado/pull/5515#issuecomment-1308872846
Thanks Jan!
Quoted 3 lines
Is there an easy way to plug upstream Avocado into QEMU? I would like to test tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg to see if the problem is fixed by Avocado upstream.
See https://lore.kernel.org/qemu-devel/[email protected]/
For your case:
-- >8 --
tests/requirements.txt +1
-1@@ -4,3 +4,3 @@ # Note that qemu.git/python/ is always implicitly installed. -avocado-framework==88.1 +-egit+https://github.com/avocado-framework/avocado.git@b31b868c882d4650d3b7d2fbfc9b8ac0f2c3672b#egg=avocado-framework
pycdlib==1.11.0
---Regards,
Phil.
Quoted 41 lines · Jan Richter
Quoted 28 lines · Philippe Mathieu-Daudé
Quoted 27 lines · Daniel Henrique Barboza
Quoted 26 lines · Daniel P. Berrangé
Quoted 25 lines · Thomas Huth
Quoted 23 lines · Daniel P. Berrangé
Quoted 10 lines · Daniel Henrique Barboza
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1],
DEFAULT_HASH_ALGORITHMis set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.Quoted 6 lines
For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
[1] https://github.com/avocado-framework/avocado.git @
942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496Can we just ask Avocado maintainers to fix this problem on their side to allow use of a modern hash alg as a priority item. We've already had this problem in QEMU for over a year AFAICT, so doesn't seem like we need to urgently do a workaround on QEMU side, so we can get Avocado devs to commit to fixing it in the next month.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Do we have such a commitment? ... The avocado version in QEMU is completely backlevel these days, it's still using version 88.1 from May 2021, i.e. there hasn't been any update since more than a year. I recently tried to bump it to a newer version on my own (since I'm still suffering from the problem that
find_free_port()does not work if you don't have a local IPv6 address), but it's not that straight forward since the recent versions of avocado changed a lot of things (e.g. the new nrunner - do we want to run tests in parallel? If so it breaks a lot of the timeout settings, I think), so an update needs a lot of careful testing...
Quoted 5 lines
Hi Daniel,
if the problem of migrating avocado to latest version on qemu is only in parallel run, I would suggest to disable it with
nrunner.max_parallel_tasks[1]. Even that the differences between avocado legacy runner and nrunner is huge, the migration should be straight forward. So if you have more issues with migration to the nrunner, I would be happy to help you with that.[1] https://avocado-framework.readthedocs.io/en/latest/config/index.html#nrunner-max-parallel-tasks
Thanks Jan and Phil for the infos. I didn't manage to do a successful Avocado run with the QEMU test though.
What I did, aside from the changes that Phil mentioned in tests/requirements.txt:
- created a /etc/avocado/avocado.conf to store the settings
- copied python/avocado.cfg from QEMU to avocado.conf
- added the following in avocado.conf:
[nrunner]max_parallel_tasks = 1
This allowed me to set Avocado as it would be if running with QEMU avocado, but even then I had no success. The test get stuck indefinitely at this point:
(...)
2022-11-10 16:00:20,758 avocado.test INFO | Temporary dir: /var/tmp/avocado_tmp_znhvpbh0/avocado_job_ywyf7v30 2022-11-10 16:00:20,758 avocado.test INFO | 2022-11-10 16:00:20,758 avocado.test INFO | Job ID: 4bb3e2a12c05d84a0a06849ecef435d547a198a0 2022-11-10 16:00:20,758 avocado.test INFO | 2022-11-10 16:00:21,041 avocado.core.task.statemachine DEBUG| <Worker spawner="<avocado.plugins.spawners.process.ProcessSpawner object at 0x7fdad5da5840>" max_triaging=12 max_running=1 task_timeout=None> has been initialized 2022-11-10 16:00:21,041 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": requested -> triaging 2022-11-10 16:00:21,042 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": requirements OK (will proceed to check dependencies) 2022-11-10 16:00:21,042 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": about to be spawned with "<avocado.plugins.spawners.process.ProcessSpawner object at 0x7fdad5da5840>" 2022-11-10 16:00:21,043 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": spawned successfully
No error is provided. Tried to run the test for 10+ minutes and nothing happens. Hitting CTRL+C aborts the test:
$ make check-avocado AVOCADO_TESTS='tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg' GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc AVOCADO Downloading avocado tests VM image for ppc64le The image was downloaded: Provider Version Architecture File fedora 31 ppc64le /home/danielhb/avocado/data/cache/by_location/d73d707673a941f5892e4bb0fa7fabaf0d41fd58/Fedora-Cloud-Base-31-1.9.ppc64le.qcow2 AVOCADO tests/avocado JOB ID : 4bb3e2a12c05d84a0a06849ecef435d547a198a0 JOB LOG : /home/danielhb/kvm-project/qemu/build/tests/results/job-2022-11-10T16.00-4bb3e2a/job.log ^C RESULTS : PASS 0 | ERROR 0 | FAIL 0 | SKIP 1 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : /home/danielhb/kvm-project/qemu/build/tests/results/job-2022-11-10T16.00-4bb3e2a/results.html JOB TIME : 396.78 s make: *** [/home/danielhb/kvm-project/qemu/tests/Makefile.include:142: check-avocado] Error 8
And the log registers the following:
2022-11-10 16:00:21,043 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": spawned successfully 2022-11-10 16:06:54,071 avocado.test INFO | 2022-11-10 16:06:54,123 avocado.test INFO | Test results available in /home/danielhb/kvm-project/qemu/build/tests/results/job-2022-11-10T16.00-4bb3e2a 2022-11-10 16:06:55,694 avocado.sysinfo DEBUG| Not logging /proc/slabinfo (lack of permissions) 2022-11-10 16:06:55,696 avocado.sysinfo DEBUG| Not logging /proc/pci (file not found) 2022-11-10 16:06:55,712 avocado.sysinfo DEBUG| Not logging /sys/kernel/debug/sched_features (file not found)
[---end of log---]
I'm probably missing more settings.
Daniel
Quoted 41 lines
- Jan
Quoted 37 lines
Quoted 10 lines
Quoted 4 lines
That it is so difficult to update Avocado after barely more than 1 year is not exactly a strong vote of confidence in our continued use of Avocado long term :-(
By the way, Avocado just provided a fix for the problem this patch is trying to amend:
https://github.com/avocado-framework/avocado/pull/5515#issuecomment-1308872846
Thanks Jan!
Quoted 3 lines
Is there an easy way to plug upstream Avocado into QEMU? I would like to test tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg to see if the problem is fixed by Avocado upstream.
See https://lore.kernel.org/qemu-devel/[email protected]/
For your case:
-- >8 --
tests/requirements.txt +1
-1@@ -4,3 +4,3 @@ # Note that qemu.git/python/ is always implicitly installed. -avocado-framework==88.1 +-e git+https://github.com/avocado-framework/avocado.git@b31b868c882d4650d3b7d2fbfc9b8ac0f2c3672b#egg=avocado-frameworkpycdlib==1.11.0
---Regards,
Phil.
Quoted 107 lines · Daniel Henrique Barboza
Quoted 41 lines · Jan Richter
Quoted 28 lines · Philippe Mathieu-Daudé
Quoted 27 lines · Daniel Henrique Barboza
Quoted 26 lines · Daniel P. Berrangé
Quoted 25 lines · Thomas Huth
Quoted 23 lines · Daniel P. Berrangé
Quoted 10 lines · Daniel Henrique Barboza
'make check-avocado' will download any images that aren't present in the cache via 'get-vm-images' in tests/Makefile.include. The target that downloads fedora 31 images, get-vm-image-fedora-31, will use 'avocado vmimage get --distro=fedora --distro-version=31 --arch=(...)' to download the image for each arch. Note that this command does not support any argument to set the hash algorithm used and, based on the avocado source code [1],
DEFAULT_HASH_ALGORITHMis set to "sha1". The sha1 hash is stored in a Fedora-Cloud-Base-31-1.9.{ARCH}.qcow2-CHECKSUM in the cache.Quoted 6 lines
For now, in QEMU, let's use sha1 for all Fedora 31 images. This will immediately spares us at least one extra download for each Fedora 31 image that we're doing in all our CI runs.
[1] https://github.com/avocado-framework/avocado.git @
942a5d6972906
[2] https://github.com/avocado-framework/avocado/issues/5496Can we just ask Avocado maintainers to fix this problem on their side to allow use of a modern hash alg as a priority item. We've already had this problem in QEMU for over a year AFAICT, so doesn't seem like we need to urgently do a workaround on QEMU side, so we can get Avocado devs to commit to fixing it in the next month.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Do we have such a commitment? ... The avocado version in QEMU is completely backlevel these days, it's still using version 88.1 from May 2021, i.e. there hasn't been any update since more than a year. I recently tried to bump it to a newer version on my own (since I'm still suffering from the problem that
find_free_port()does not work if you don't have a local IPv6 address), but it's not that straight forward since the recent versions of avocado changed a lot of things (e.g. the new nrunner - do we want to run tests in parallel? If so it breaks a lot of the timeout settings, I think), so an update needs a lot of careful testing...Quoted 5 lines
Hi Daniel,
if the problem of migrating avocado to latest version on qemu is only in parallel run, I would suggest to disable it with
nrunner.max_parallel_tasks[1]. Even that the differences between avocado legacy runner and nrunner is huge, the migration should be straight forward. So if you have more issues with migration to the nrunner, I would be happy to help you with that.[1] https://avocado-framework.readthedocs.io/en/latest/config/index.html#nrunner-max-parallel-tasks
Thanks Jan and Phil for the infos. I didn't manage to do a successful Avocado run with the QEMU test though.
What I did, aside from the changes that Phil mentioned in
tests/requirements.txt:- created a
/etc/avocado/avocado.confto store the settings
- copied python/avocado.cfg from QEMU to avocado.conf
- added the following in avocado.conf:[nrunner]
max_parallel_tasks = 1This allowed me to set Avocado as it would be if running with QEMU avocado, but even then I had no success. The test get stuck indefinitely at this point:
(...)
2022-11-10 16:00:20,758 avocado.test INFO | Temporary dir: /var/tmp/avocado_tmp_znhvpbh0/avocado_job_ywyf7v30 2022-11-10 16:00:20,758 avocado.test INFO | 2022-11-10 16:00:20,758 avocado.test INFO | Job ID: 4bb3e2a12c05d84a0a06849ecef435d547a198a0 2022-11-10 16:00:20,758 avocado.test INFO | 2022-11-10 16:00:21,041 avocado.core.task.statemachine DEBUG| <Worker spawner="<avocado.plugins.spawners.process.ProcessSpawner object at 0x7fdad5da5840>" max_triaging=12 max_running=1 task_timeout=None> has been initialized 2022-11-10 16:00:21,041 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": requested -> triaging 2022-11-10 16:00:21,042 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": requirements OK (will proceed to check dependencies) 2022-11-10 16:00:21,042 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": about to be spawned with "<avocado.plugins.spawners.process.ProcessSpawner object at 0x7fdad5da5840>" 2022-11-10 16:00:21,043 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": spawned successfullyNo error is provided. Tried to run the test for 10+ minutes and nothing happens. Hitting CTRL+C aborts the test:
$ make check-avocado AVOCADO_TESTS='tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg' GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc AVOCADO Downloading avocado tests VM image for ppc64le The image was downloaded: Provider Version Architecture File fedora 31 ppc64le /home/danielhb/avocado/data/cache/by_location/d73d707673a941f5892e4bb0fa7fabaf0d41fd58/Fedora-Cloud-Base-31-1.9.ppc64le.qcow2 AVOCADO tests/avocado JOB ID : 4bb3e2a12c05d84a0a06849ecef435d547a198a0 JOB LOG : /home/danielhb/kvm-project/qemu/build/tests/results/job-2022-11-10T16.00-4bb3e2a/job.log ^C RESULTS : PASS 0 | ERROR 0 | FAIL 0 | SKIP 1 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : /home/danielhb/kvm-project/qemu/build/tests/results/job-2022-11-10T16.00-4bb3e2a/results.html JOB TIME : 396.78 s make: *** [/home/danielhb/kvm-project/qemu/tests/Makefile.include:142: check-avocado] Error 8And the log registers the following:
2022-11-10 16:00:21,043 avocado.core.task.statemachine DEBUG| Task "1-tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg": spawned successfully 2022-11-10 16:06:54,071 avocado.test INFO | 2022-11-10 16:06:54,123 avocado.test INFO | Test results available in /home/danielhb/kvm-project/qemu/build/tests/results/job-2022-11-10T16.00-4bb3e2a 2022-11-10 16:06:55,694 avocado.sysinfo DEBUG| Not logging /proc/slabinfo (lack of permissions) 2022-11-10 16:06:55,696 avocado.sysinfo DEBUG| Not logging /proc/pci (file not found) 2022-11-10 16:06:55,712 avocado.sysinfo DEBUG| Not logging /sys/kernel/debug/sched_features (file not found)
Quoted 4 lines
[---end of log---]
I'm probably missing more settings.
Hi Daniel,
I discussed this problem on Avocado meeting today and I got information that Cleber Rosa is working on a patch for qemu to update scripts to Avocado 99.0 where he will fix those things, and he will publish it soon.
- Jan
Quoted 48 lines
Daniel
Quoted 42 lines
- Jan
Quoted 38 lines
Quoted 10 lines
Quoted 4 lines
That it is so difficult to update Avocado after barely more than 1 year is not exactly a strong vote of confidence in our continued use of Avocado long term :-(
By the way, Avocado just provided a fix for the problem this patch is trying to amend:
https://github.com/avocado-framework/avocado/pull/5515#issuecomment-1308872846
Thanks Jan!
Quoted 3 lines
Is there an easy way to plug upstream Avocado into QEMU? I would like to test tests/avocado/boot_linux.py:BootLinuxPPC64.test_pseries_tcg to see if the problem is fixed by Avocado upstream.
See https://lore.kernel.org/qemu-devel/[email protected]/
For your case:
-- >8 --
tests/requirements.txt +1
-1@@ -4,3 +4,3 @@ # Note that qemu.git/python/ is always implicitly installed. -avocado-framework==88.1 +-egit+https://github.com/avocado-framework/avocado.git@b31b868c882d4650d3b7d2fbfc9b8ac0f2c3672b#egg=avocado-framework pycdlib==1.11.0
---Regards,
Phil.