Skip to contentSkip to conversation

tests/acceptance/boot_linux: Switch to Fedora 32

PATCH In review Daniele Buono 4 participants 6 messages Jan 27, 2021 – Feb 5, 2021

lore
Share & tools
lore
#0 lore raw

cover tests/acceptance/boot_linux: Switch to Fedora 32

Local acceptance tests run with "make check-acceptance" are now showing some cases canceled like the following:

(01/39) tests/acceptance/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg: CANCEL: Failed to download/prepare boot image (0.25 s)

Turns out, every full-vm test in boot_linux.py is trying to use a Fedora 31 cloud image and is failing, with Avocado refusing to download it, presumably because Fedora 31 is EOL.

This patch moves to Fedora 32, which is still supported. And seem to work fine

The script has the image checksums hardcoded. I downloaded and verified the checksums with the Fedora 32 GPG key, but I would feel more confident if someone else wants to verify it too.

The checksum files are here:
https://download-ib01.fedoraproject.org/pub/fedora-secondary/releases/32/Cloud/ppc64le/images/Fedora-Cloud-32-1.6-ppc64le-CHECKSUM
https://download-ib01.fedoraproject.org/pub/fedora-secondary/releases/32/Cloud/s390x/images/Fedora-Cloud-32-1.6-s390x-CHECKSUM
https://download-ib01.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/aarch64/images/Fedora-Cloud-32-1.6-aarch64-CHECKSUM
https://download-ib01.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-32-1.6-x86_64-CHECKSUM
and the GPG keys are available here:
https://getfedora.org/en/security/

NOTE: I tried moving to Fedora 33, but the aarch64 VM cannot boot properly. May be worth investigating but I have no experience with ARM so I'll leave that to someone else, if interested.

Daniele Buono (1):
tests/acceptance/boot_linux: Switch to Fedora 32

 tests/acceptance/boot_linux.py | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
Signature · 1 line
2.30.0
#1 lore raw

PATCH 1/1 tests/acceptance/boot_linux: Switch to Fedora 32

Some acceptance test require a full VM image. The tests were using Fedora 31, which is now EOL. Probably as a consequence, the Avocado framework is not able to download such images anymore.

Switch to Fedora 32, which is still supported.

---
 tests/acceptance/boot_linux.py | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)
tests/acceptance/boot_linux.py +5 -6
index 1da4a53d6a..b954f95f0b 100644
@@ -48,7 +48,7 @@ class BootLinuxBase(Test):
             image_arch = 'ppc64le'
         try:
             boot = vmimage.get(
-                'fedora', arch=image_arch, version='31',
+                'fedora', arch=image_arch, version='32',
                 checksum=self.chksum,
                 algorithm='sha256',
                 cache_dir=self.cache_dirs[0],
@@ -111,7 +111,7 @@ class BootLinuxX8664(BootLinux):
     :avocado: tags=arch:x86_64
     """
 
-    chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
+    chksum = '423a4ce32fa32c50c11e3d3ff392db97a762533b81bef9d00599de518a7469c8'
 
     def test_pc_i440fx_tcg(self):
         """
@@ -160,8 +160,7 @@ class BootLinuxAarch64(BootLinux):
     :avocado: tags=machine:virt
     :avocado: tags=machine:gic-version=2
     """
-
-    chksum = '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'
+    chksum = 'b367755c664a2d7a26955bbfff985855adfa2ca15e908baf15b4b176d68d3967'
 
     def add_common_args(self):
         self.vm.add_args('-bios',
@@ -217,7 +216,7 @@ class BootLinuxPPC64(BootLinux):
     :avocado: tags=arch:ppc64
     """
 
-    chksum = '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'
+    chksum = 'dd989a078d641713c55720ba3e4320b204ade6954e2bfe4570c8058dc36e2e5d'
 
     def test_pseries_tcg(self):
         """
@@ -235,7 +234,7 @@ class BootLinuxS390X(BootLinux):
     :avocado: tags=arch:s390x
     """
 
-    chksum = '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'
+    chksum = '93e49b98fa016797a6864a95cbb7beaec86ffd61dbcd42a951158ae732dae1ec'
 
     @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
     def test_s390_ccw_virtio_tcg(self):
Signature · 1 line
2.30.0
#2 lore raw
In reply to Daniele Buono

Hi,

Quoted 11 lines · Daniele Buono

Local acceptance tests run with "make check-acceptance" are now showing some cases canceled like the following:

(01/39) tests/acceptance/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg: CANCEL: Failed to download/prepare boot image (0.25 s)

Turns out, every full-vm test in boot_linux.py is trying to use a Fedora 31 cloud image and is failing, with Avocado refusing to download it, presumably because Fedora 31 is EOL.

This patch moves to Fedora 32, which is still supported. And seem to work fine

While ago it was discussed about updating the Fedora version which, in my opinion, ended up without a conclusion. Please see the complete thread in:

https://www.mail-archive.com/[email protected]/msg763986.html

I'm CC'ing Daniel Berrrangé so that, perhaps, we could resume the discussion.

Thanks!

- Wainer

Quoted 23 lines

The script has the image checksums hardcoded. I downloaded and verified the checksums with the Fedora 32 GPG key, but I would feel more confident if someone else wants to verify it too.

The checksum files are here:
https://download-ib01.fedoraproject.org/pub/fedora-secondary/releases/32/Cloud/ppc64le/images/Fedora-Cloud-32-1.6-ppc64le-CHECKSUM
https://download-ib01.fedoraproject.org/pub/fedora-secondary/releases/32/Cloud/s390x/images/Fedora-Cloud-32-1.6-s390x-CHECKSUM
https://download-ib01.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/aarch64/images/Fedora-Cloud-32-1.6-aarch64-CHECKSUM
https://download-ib01.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-32-1.6-x86_64-CHECKSUM
and the GPG keys are available here:
https://getfedora.org/en/security/

NOTE: I tried moving to Fedora 33, but the aarch64 VM cannot boot properly. May be worth investigating but I have no experience with ARM so I'll leave that to someone else, if interested.

Daniele Buono (1):
tests/acceptance/boot_linux: Switch to Fedora 32

  tests/acceptance/boot_linux.py | 11 +++++------
  1 file changed, 5 insertions(+), 6 deletions(-)
#3 lore raw
Quoted 12 lines · Wainer dos Santos Moschetta

Hi,

Quoted 9 lines · Daniele Buono

Local acceptance tests run with "make check-acceptance" are now showing some cases canceled like the following:

(01/39) tests/acceptance/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg: CANCEL: Failed to download/prepare boot image (0.25 s)

Turns out, every full-vm test in boot_linux.py is trying to use a Fedora 31 cloud image and is failing, with Avocado refusing to download it, presumably because Fedora 31 is EOL.

Quoted 6 lines
Quoted 2 lines

This patch moves to Fedora 32, which is still supported. And seem to work fine

While ago it was discussed about updating the Fedora version which, in my opinion, ended up without a conclusion. Please see the complete thread in:

https://www.mail-archive.com/[email protected]/msg763986.html

Oops, didn't notice the previous thread. Apologies for the duplicate!

Quoted 35 lines

I'm CC'ing Daniel Berrrangé so that, perhaps, we could resume the discussion.

Thanks!

- Wainer

Quoted 27 lines

The script has the image checksums hardcoded. I downloaded and verified the checksums with the Fedora 32 GPG key, but I would feel more confident if someone else wants to verify it too.

The checksum files are here:
https://download-ib01.fedoraproject.org/pub/fedora-secondary/releases/32/Cloud/ppc64le/images/Fedora-Cloud-32-1.6-ppc64le-CHECKSUM

https://download-ib01.fedoraproject.org/pub/fedora-secondary/releases/32/Cloud/s390x/images/Fedora-Cloud-32-1.6-s390x-CHECKSUM

https://download-ib01.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/aarch64/images/Fedora-Cloud-32-1.6-aarch64-CHECKSUM

https://download-ib01.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-32-1.6-x86_64-CHECKSUM

and the GPG keys are available here:
https://getfedora.org/en/security/

NOTE: I tried moving to Fedora 33, but the aarch64 VM cannot boot properly. May be worth investigating but I have no experience with ARM so I'll leave that to someone else, if interested.

Daniele Buono (1):
   tests/acceptance/boot_linux: Switch to Fedora 32

  tests/acceptance/boot_linux.py | 11 +++++------

  1 file changed, 5 insertions(+), 6 deletions(-)
#4 lore raw

Hi Wainer,

Quoted 18 lines · Daniele Buono
Quoted 17 lines · Wainer dos Santos Moschetta

Hi,

Quoted 13 lines · Daniele Buono

Local acceptance tests run with "make check-acceptance" are now showing some cases canceled like the following:

(01/39) tests/acceptance/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg:
CANCEL: Failed to download/prepare boot image (0.25 s)

Turns out, every full-vm test in boot_linux.py is trying to use a Fedora 31 cloud image and is failing, with Avocado refusing to download it, presumably because Fedora 31 is EOL.

This patch moves to Fedora 32, which is still supported. And seem to work fine

Quoted 11 lines
Quoted 5 lines

While ago it was discussed about updating the Fedora version which, in my opinion, ended up without a conclusion. Please see the complete thread in:

https://www.mail-archive.com/[email protected]/msg763986.html

Oops, didn't notice the previous thread. Apologies for the duplicate!

Quoted 3 lines

I'm CC'ing Daniel Berrrangé so that, perhaps, we could resume the discussion.

The first question I'd like to figure out is how/where can we archive the artifacts being tested by the project. As we maintain stable tags, I'm more worried about regressions affecting LTS use rather than recently added features which get more coverage and activity. Is this too conservative for acceptance testing?

Regards,

Phil.

#5 lore raw
Quoted 30 lines · Philippe Mathieu-Daudé

Hi Wainer,

Quoted 27 lines · Daniele Buono
Quoted 14 lines · Wainer dos Santos Moschetta

Hi,

Quoted 11 lines · Daniele Buono

Local acceptance tests run with "make check-acceptance" are now showing some cases canceled like the following:

(01/39) tests/acceptance/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg:
CANCEL: Failed to download/prepare boot image (0.25 s)

Turns out, every full-vm test in boot_linux.py is trying to use a Fedora 31 cloud image and is failing, with Avocado refusing to download it, presumably because Fedora 31 is EOL.

Quoted 8 lines
Quoted 2 lines

This patch moves to Fedora 32, which is still supported. And seem to work fine

While ago it was discussed about updating the Fedora version which, in my opinion, ended up without a conclusion. Please see the complete thread in:

https://www.mail-archive.com/[email protected]/msg763986.html

Oops, didn't notice the previous thread. Apologies for the duplicate!

Quoted 1 line
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

I'm CC'ing Daniel Berrrangé so that, perhaps, we could resume the discussion.

The first question I'd like to figure out is how/where can we archive the artifacts being tested by the project. As we maintain stable tags, I'm more worried about regressions affecting LTS use rather than recently added features which get more coverage and activity. Is this too conservative for acceptance testing?

I think there's multiple issues to worry about, one short term, the others long term.

The most immediate issue is that we are pointing to an EOL Fedora release with broken download URL.

While we could update to F 32, that doesn't solve the problem. It will burn us again in just a few months when F34 comes out making F32 EOL. We need a long term solution.

The broader long term question is what our goals are for the acceptance tests.

I think the primary goal is detect regressions in QEMU where we break something which used to work. To achieve this we don't need to be chasing distro releases. It is fine for us to be testing an EOL distro. The fact that it is EOL doesn't invalidate the test behaviour.

The only problem switch EOL distros is the URL breakge. That is trivially solved in the Fedora case by downloading from the archive.fedoraproject.org server instead of main server. Problem solved forever.

The more important long term question is whether the selection of distros we are testing gives us coverage which exercises all the scenarios that we care about.

For example, considering virtio devices, we need

- Guest OS which *only* has virtio legacy mode implemented
- pc for PCI
- q35 for PCI-e
Proves deices work in legacy mode on both PCI and PCIe

- Guest OS which has both legacy and modern mode implemented
- pc for PCI
- q35 for PCI-e
Proves devices work in transitional mode (pc) and modern-only mode (PCI-e)

So for virtio coverage, we'll need 2 guest OS, each with two scenarios as a starting point. RHEL-6 is an example of a distro that was legacy only. Anything newer does the other case.

But then we need to expand the axis to non-x86 too, most especially for ppc64 and s390, which have non-PCI based options as well as PCI.

Periodically a new virtio device will arrive (eg virtiofs) and that might mean we need to add another test scenario, or upgrade the more modern OS in an existing test scenario. This is reasonably infrequent though, so in general we won't need to be chasing the bleeding edge distros.

The matrix can grow pretty damn fast, and we need to keep it under control. One way to deal with that is to have one test case cover multiple features. eg don't run a separate test for each of virtio-net, virtio-blk, virtio-scsi. Launch a single VM in which we can test all of them at once.

Also we don't need to test all of Ubuntu, Fedora, RHEL, SUSE if all those distros are basically exercising the same set of QEMU device code paths. ie just becuase the distro is different, doesn't mean it adds value to the test coverage.

We should be strongly driven by whether the distro exercises a feature that isn't already covered by an existing test.

So the next question is what needs to be in our matrix ? Most downstream vendors will say it needs to be all KVM related features. QEMU as a project though also cares more broadly about many more devices for emulating older platforms. There's some balance to be struck here.

On many previous occassions we've talked about classifying QEMU features into support tiers. If we had a view of what our support tiers were for each feature, this would in turn show us where to spend effort in building up testing coverage.

I made an abortive start at trying to define what tiers might mean but didn't get into really classifying features

https://wiki.qemu.org/Support_Tiers

don't pay attention to the host/guest classifications I did there - I was mostly just playing with how to represent it.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|