Skip to contentSkip to conversation

Acceptance tests: bump Fedora to 32

PATCH In review Cleber Rosa 5 participants 12 messages Dec 2, 2020 – Dec 4, 2020

lore
Share & tools
lore
#0 lore raw

cover Acceptance tests: bump Fedora to 32

I believe this may be a candidate for "right now" because the code changes here simply sync with external infrastructure changes, that is, the retirement of Fedora 31 from the official repository locations).

The following jobs contain a validation of this bump:

- https://gitlab.com/cleber.gnu/qemu/-/jobs/886864642#L72
- https://gitlab.com/cleber.gnu/qemu/-/jobs/886864633#L72

Thanks,
- Cleber.

Cleber Rosa (1):
Acceptance tests: bump Fedora to 32

 tests/acceptance/boot_linux.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
Signature · 1 line
2.25.4
#1 lore raw

PATCH 1/1 Acceptance tests: bump Fedora to 32

Currently in use Fedora 31 has been moved out of the standard download locations that are supported by the functionality provided by avocado.utils.vmimage. So right now, the boot_linux.py tests will get canceled by not being able to find those specific images.

Ideally, this would be bumped to version 33. But, I've found issues with the aarch64 images, with various systemd services failing to start. So to keep all archs consistent, I've settled on 32.

---
 tests/acceptance/boot_linux.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
tests/acceptance/boot_linux.py +6 -6
index 1da4a53d6a..0824de008e 100644
@@ -42,13 +42,13 @@ class BootLinuxBase(Test):
         vmimage.QEMU_IMG = qemu_img
 
         self.log.info('Downloading/preparing boot image')
-        # Fedora 31 only provides ppc64le images
+        # Fedora 32 only provides ppc64le images
         image_arch = self.arch
         if image_arch == 'ppc64':
             image_arch = 'ppc64le'
         try:
             boot = vmimage.get(
-                'fedora', arch=image_arch, version='31',
+                'fedora', arch=image_arch, version='32',
                 checksum=self.chksum,
                 algorithm='sha256',
                 cache_dir=self.cache_dirs[0],
@@ -111,7 +111,7 @@ class BootLinuxX8664(BootLinux):
     :avocado: tags=arch:x86_64
     """
 
-    chksum = 'e3c1b309d9203604922d6e255c2c5d098a309c2d46215d8fc026954f3c5c27a0'
+    chksum = '423a4ce32fa32c50c11e3d3ff392db97a762533b81bef9d00599de518a7469c8'
 
     def test_pc_i440fx_tcg(self):
         """
@@ -161,7 +161,7 @@ class BootLinuxAarch64(BootLinux):
     :avocado: tags=machine:gic-version=2
     """
 
-    chksum = '1e18d9c0cf734940c4b5d5ec592facaed2af0ad0329383d5639c997fdf16fe49'
+    chksum = 'b367755c664a2d7a26955bbfff985855adfa2ca15e908baf15b4b176d68d3967'
 
     def add_common_args(self):
         self.vm.add_args('-bios',
@@ -217,7 +217,7 @@ class BootLinuxPPC64(BootLinux):
     :avocado: tags=arch:ppc64
     """
 
-    chksum = '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'
+    chksum = 'dd989a078d641713c55720ba3e4320b204ade6954e2bfe4570c8058dc36e2e5d'
 
     def test_pseries_tcg(self):
         """
@@ -235,7 +235,7 @@ class BootLinuxS390X(BootLinux):
     :avocado: tags=arch:s390x
     """
 
-    chksum = '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'
+    chksum = '93e49b98fa016797a6864a95cbb7beaec86ffd61dbcd42a951158ae732dae1ec'
 
     @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
     def test_s390_ccw_virtio_tcg(self):
Signature · 1 line
2.25.4
#2 lore raw
Quoted 21 lines · Cleber Rosa

Currently in use Fedora 31 has been moved out of the standard download locations that are supported by the functionality provided by avocado.utils.vmimage. So right now, the boot_linux.py tests will get canceled by not being able to find those specific images.

Ideally, this would be bumped to version 33. But, I've found issues with the aarch64 images, with various systemd services failing to start. So to keep all archs consistent, I've settled on 32.

---
 tests/acceptance/boot_linux.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
tests/acceptance/boot_linux.py +0 -0
index 1da4a53d6a..0824de008e 100644
@@ -42,13 +42,13 @@ class BootLinuxBase(Test):
         vmimage.QEMU_IMG = qemu_img
 

tests/acceptance/boot_linux.py class BootLinuxBase(Test): · line 51

Quoted 10 lines
tests/acceptance/boot_linux.py +2 -2
         self.log.info('Downloading/preparing boot image')
-        # Fedora 31 only provides ppc64le images
+        # Fedora 32 only provides ppc64le images
         image_arch = self.arch
         if image_arch == 'ppc64':
             image_arch = 'ppc64le'
         try:
             boot = vmimage.get(
-                'fedora', arch=image_arch, version='31',
+                'fedora', arch=image_arch, version='32',

I already expressed my view on this (latest QEMU should be able to use at least f31 - which was tested - and eventually f33 - which is coverage extension). I'm not going to vouch this change. If other maintainers are happy with it, I don't mind this gets merged.

BTW I don't see why this is urgent for 5.2.

Phil.

#3 lore raw
Quoted 39 lines · Philippe Mathieu-Daudé
Quoted 21 lines · Cleber Rosa

Currently in use Fedora 31 has been moved out of the standard download locations that are supported by the functionality provided by avocado.utils.vmimage. So right now, the boot_linux.py tests will get canceled by not being able to find those specific images.

Ideally, this would be bumped to version 33. But, I've found issues with the aarch64 images, with various systemd services failing to start. So to keep all archs consistent, I've settled on 32.

---
 tests/acceptance/boot_linux.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
tests/acceptance/boot_linux.py +0 -0
index 1da4a53d6a..0824de008e 100644
@@ -42,13 +42,13 @@ class BootLinuxBase(Test):
         vmimage.QEMU_IMG = qemu_img
 

tests/acceptance/boot_linux.py class BootLinuxBase(Test): · line 51

Quoted 10 lines
tests/acceptance/boot_linux.py +2 -2
         self.log.info('Downloading/preparing boot image')
-        # Fedora 31 only provides ppc64le images
+        # Fedora 32 only provides ppc64le images
         image_arch = self.arch
         if image_arch == 'ppc64':
             image_arch = 'ppc64le'
         try:
             boot = vmimage.get(
-                'fedora', arch=image_arch, version='31',
+                'fedora', arch=image_arch, version='32',

I already expressed my view on this (latest QEMU should be able to use at least f31 - which was tested - and eventually f33 - which is coverage extension). I'm not going to vouch this change. If other maintainers are happy with it, I don't mind this gets merged.

Quoted 4 lines

BTW I don't see why this is urgent for 5.2.

Phil.

Hi Phil,

Are you implying that, in your opinion, QEMU (say 5.2) should somehow provide compatibility with Fedora 31 because it was used during the entire cycle? I sympathize with that, but, QEMU is not really advertising compatibility support with specific Linux Distros, is it?

And, assuming that the issues I found on the Fedora 33 aarch64 image can not be worked around, would you suggest not moving to 32? I mean, I don't see a reason why QEMU shouldn't be able to use at least Fedora 32, which is a currently *active* version (different from 31).

Thanks,
- Cleber.

#4 lore raw
Quoted 52 lines · Cleber Rosa
Quoted 39 lines · Philippe Mathieu-Daudé
Quoted 21 lines · Cleber Rosa

Currently in use Fedora 31 has been moved out of the standard download locations that are supported by the functionality provided by avocado.utils.vmimage. So right now, the boot_linux.py tests will get canceled by not being able to find those specific images.

Ideally, this would be bumped to version 33. But, I've found issues with the aarch64 images, with various systemd services failing to start. So to keep all archs consistent, I've settled on 32.

---
 tests/acceptance/boot_linux.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
tests/acceptance/boot_linux.py +0 -0
index 1da4a53d6a..0824de008e 100644
@@ -42,13 +42,13 @@ class BootLinuxBase(Test):
         vmimage.QEMU_IMG = qemu_img
 

tests/acceptance/boot_linux.py class BootLinuxBase(Test): · line 51

Quoted 10 lines
tests/acceptance/boot_linux.py +2 -2
         self.log.info('Downloading/preparing boot image')
-        # Fedora 31 only provides ppc64le images
+        # Fedora 32 only provides ppc64le images
         image_arch = self.arch
         if image_arch == 'ppc64':
             image_arch = 'ppc64le'
         try:
             boot = vmimage.get(
-                'fedora', arch=image_arch, version='31',
+                'fedora', arch=image_arch, version='32',

I already expressed my view on this (latest QEMU should be able to use at least f31 - which was tested - and eventually f33 - which is coverage extension). I'm not going to vouch this change. If other maintainers are happy with it, I don't mind this gets merged.

Quoted 4 lines

BTW I don't see why this is urgent for 5.2.

Phil.

Hi Phil,

Are you implying that, in your opinion, QEMU (say 5.2) should somehow provide compatibility with Fedora 31 because it was used during the entire cycle? I sympathize with that, but, QEMU is not really advertising compatibility support with specific Linux Distros, is it?

Quoted 4 lines

And, assuming that the issues I found on the Fedora 33 aarch64 image can not be worked around, would you suggest not moving to 32? I mean, I don't see a reason why QEMU shouldn't be able to use at least Fedora 32, which is a currently *active* version (different from 31).

I think the problem with the Fedora acceptance is that we'll be constantly chasing a moving target. Every URL we pick will go away 6-12 months later. IOW, while the acceptance test pass today, in 6 months time they'll be failing. IOW, switching to F32 doesn't solve the root cause, it just pushs the problem down the road for 6 months until F32 is EOL and hits the same URL change problem.

One way to avoid this is to *not* actually test a current Fedora. Instead intentionally point at an EOL Fedora release whose URL has already moved to the archive site which is long term stable.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#5 lore raw
Quoted 7 lines · Daniel P. Berrangé

I think the problem with the Fedora acceptance is that we'll be constantly chasing a moving target. Every URL we pick will go away 6-12 months later. IOW, while the acceptance test pass today, in 6 months time they'll be failing. IOW, switching to F32 doesn't solve the root cause, it just pushs the problem down the road for 6 months until F32 is EOL and hits the same URL change problem.

Just FIY, the tests will not FAIL when the images are removed from the official locations. This is what happens Today:

JOB ID : e85527a9d75023070f15b833eac0f91f803afc83
JOB LOG : /home/cleber/avocado/job-results/job-2020-12-03T12.21-e85527a/job.log
(1/1) tests/acceptance/boot_linux.py:BootLinuxX8664.test_pc_q35_kvm: CANCEL: Failed to download/prepare boot image (0.33 s)

   RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 1
   JOB HTML   : /home/cleber/avocado/job-results/job-2020-12-03T12.21-e85527a/results.html
   JOB TIME   : 0.76 s

And *normally*, we'd have 12+ months between updates, that is from Fedora 31 -> 33, 33 -> 35, etc.

Quoted 4 lines

One way to avoid this is to *not* actually test a current Fedora. Instead intentionally point at an EOL Fedora release whose URL has already moved to the archive site which is long term stable.

So the tradeoff is, a patch every 6 or 12 months, versus using a more modern guest. With other tests, such as virtiofs_submounts.py, already depending on the same decision (to avoid multiple guest images downloaded), I think this tradeoff decision needs more visibility.

IMO, the cost of such a simple patch every 6 or 12 months is very low provided we'll benefit from the newer guests.

Cheers,
- Cleber.

Quoted 8 lines

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#6 lore raw
Quoted 27 lines · Cleber Rosa
Quoted 7 lines · Daniel P. Berrangé

I think the problem with the Fedora acceptance is that we'll be constantly chasing a moving target. Every URL we pick will go away 6-12 months later. IOW, while the acceptance test pass today, in 6 months time they'll be failing. IOW, switching to F32 doesn't solve the root cause, it just pushs the problem down the road for 6 months until F32 is EOL and hits the same URL change problem.

Just FIY, the tests will not FAIL when the images are removed from the official locations. This is what happens Today:

   JOB ID     : e85527a9d75023070f15b833eac0f91f803afc83
   JOB LOG    : /home/cleber/avocado/job-results/job-2020-12-03T12.21-e85527a/job.log
    (1/1) tests/acceptance/boot_linux.py:BootLinuxX8664.test_pc_q35_kvm: CANCEL: Failed to download/prepare boot image (0.33 s)
   RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 1
   JOB HTML   : /home/cleber/avocado/job-results/job-2020-12-03T12.21-e85527a/results.html
   JOB TIME   : 0.76 s

And *normally*, we'd have 12+ months between updates, that is from Fedora 31 -> 33, 33 -> 35, etc.

Quoted 4 lines

One way to avoid this is to *not* actually test a current Fedora. Instead intentionally point at an EOL Fedora release whose URL has already moved to the archive site which is long term stable.

Quoted 7 lines

So the tradeoff is, a patch every 6 or 12 months, versus using a more modern guest. With other tests, such as virtiofs_submounts.py, already depending on the same decision (to avoid multiple guest images downloaded), I think this tradeoff decision needs more visibility.

IMO, the cost of such a simple patch every 6 or 12 months is very low provided we'll benefit from the newer guests.

I don't think changing the OS version typically changes the level of coverage in aggregate. The new OS may exercise new code paths, but it will stop exercising old code paths, so most of the time it'll be net-zero. The ideal would be to test a representative selection of both old and new versions but capacity limits.

The only time there's probably a notable difference is if we need to access to a new type of device that the old OS doesn't have, but that's relatively rare.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#7 lore raw
In reply to Cleber Rosa
Quoted 13 lines · Cleber Rosa
Quoted 7 lines · Daniel P. Berrangé

I think the problem with the Fedora acceptance is that we'll be constantly chasing a moving target. Every URL we pick will go away 6-12 months later. IOW, while the acceptance test pass today, in 6 months time they'll be failing. IOW, switching to F32 doesn't solve the root cause, it just pushs the problem down the road for 6 months until F32 is EOL and hits the same URL change problem.

Just FIY, the tests will not FAIL when the images are removed from the official locations. This is what happens Today:

Quoted 7 lines
   JOB ID     : e85527a9d75023070f15b833eac0f91f803afc83
   JOB LOG    : /home/cleber/avocado/job-results/job-2020-12-03T12.21-e85527a/job.log
    (1/1) tests/acceptance/boot_linux.py:BootLinuxX8664.test_pc_q35_kvm: CANCEL: Failed to download/prepare boot image (0.33 s)
   RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 1
   JOB HTML   : /home/cleber/avocado/job-results/job-2020-12-03T12.21-e85527a/results.html
   JOB TIME   : 0.76 s

In the CI, enabling the GitLab cache will help, but will also hide the URL change problem when it happens.

It is also true if the person ran the test and it successfully downloaded the image once. For people running the test for the first time, it will look like your output.

#8 lore raw
In reply to Daniel P. Berrangé
Quoted 46 lines · Daniel P. Berrangé
Quoted 45 lines · Cleber Rosa
Quoted 43 lines · Philippe Mathieu-Daudé
Quoted 21 lines · Cleber Rosa

Currently in use Fedora 31 has been moved out of the standard download locations that are supported by the functionality provided by avocado.utils.vmimage. So right now, the boot_linux.py tests will get canceled by not being able to find those specific images.

Ideally, this would be bumped to version 33. But, I've found issues with the aarch64 images, with various systemd services failing to start. So to keep all archs consistent, I've settled on 32.

---
 tests/acceptance/boot_linux.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
tests/acceptance/boot_linux.py +0 -0
index 1da4a53d6a..0824de008e 100644
@@ -42,13 +42,13 @@ class BootLinuxBase(Test):
         vmimage.QEMU_IMG = qemu_img
 

tests/acceptance/boot_linux.py class BootLinuxBase(Test): · line 51

Quoted 10 lines
tests/acceptance/boot_linux.py +2 -2
         self.log.info('Downloading/preparing boot image')
-        # Fedora 31 only provides ppc64le images
+        # Fedora 32 only provides ppc64le images
         image_arch = self.arch
         if image_arch == 'ppc64':
             image_arch = 'ppc64le'
         try:
             boot = vmimage.get(
-                'fedora', arch=image_arch, version='31',
+                'fedora', arch=image_arch, version='32',

I already expressed my view on this (latest QEMU should be able to use at least f31 - which was tested - and eventually f33 - which is coverage extension). I'm not going to vouch this change. If other maintainers are happy with it, I don't mind this gets merged.

BTW I don't see why this is urgent for 5.2.

Phil.

Quoted 6 lines
Quoted 6 lines

Hi Phil,

Are you implying that, in your opinion, QEMU (say 5.2) should somehow provide compatibility with Fedora 31 because it was used during the entire cycle? I sympathize with that, but, QEMU is not really advertising compatibility support with specific Linux Distros, is it?

What I don't understand is why you remove F31 and not simply add a F32 test. Why should I stop testing F31 if I have it cached?

Quoted 13 lines
Quoted 5 lines

And, assuming that the issues I found on the Fedora 33 aarch64 image can not be worked around, would you suggest not moving to 32? I mean, I don't see a reason why QEMU shouldn't be able to use at least Fedora 32, which is a currently *active* version (different from 31).

I think the problem with the Fedora acceptance is that we'll be constantly chasing a moving target. Every URL we pick will go away 6-12 months later. IOW, while the acceptance test pass today, in 6 months time they'll be failing. IOW, switching to F32 doesn't solve the root cause, it just pushs the problem down the road for 6 months until F32 is EOL and hits the same URL change problem.

Quoted 3 lines

One way to avoid this is to *not* actually test a current Fedora. Instead intentionally point at an EOL Fedora release whose URL has already moved to the archive site which is long term stable.

I agree with Daniel, 'acceptance' test must not fail.

Having an archived image allow us to not rely on external disappearing storage.

Quoted 4 lines

Regards,
Daniel

#9 lore raw
Quoted 49 lines · Philippe Mathieu-Daudé
Quoted 48 lines · Daniel P. Berrangé
Quoted 47 lines · Cleber Rosa
Quoted 39 lines · Philippe Mathieu-Daudé
Quoted 21 lines · Cleber Rosa

Currently in use Fedora 31 has been moved out of the standard download locations that are supported by the functionality provided by avocado.utils.vmimage. So right now, the boot_linux.py tests will get canceled by not being able to find those specific images.

Ideally, this would be bumped to version 33. But, I've found issues with the aarch64 images, with various systemd services failing to start. So to keep all archs consistent, I've settled on 32.

---
 tests/acceptance/boot_linux.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
tests/acceptance/boot_linux.py +0 -0
index 1da4a53d6a..0824de008e 100644
@@ -42,13 +42,13 @@ class BootLinuxBase(Test):
         vmimage.QEMU_IMG = qemu_img
 

tests/acceptance/boot_linux.py class BootLinuxBase(Test): · line 51

Quoted 10 lines
tests/acceptance/boot_linux.py +2 -2
         self.log.info('Downloading/preparing boot image')
-        # Fedora 31 only provides ppc64le images
+        # Fedora 32 only provides ppc64le images
         image_arch = self.arch
         if image_arch == 'ppc64':
             image_arch = 'ppc64le'
         try:
             boot = vmimage.get(
-                'fedora', arch=image_arch, version='31',
+                'fedora', arch=image_arch, version='32',

I already expressed my view on this (latest QEMU should be able to use at least f31 - which was tested - and eventually f33 - which is coverage extension). I'm not going to vouch this change. If other maintainers are happy with it, I don't mind this gets merged.

Quoted 4 lines

BTW I don't see why this is urgent for 5.2.

Phil.

Hi Phil,

Quoted 7 lines
Quoted 4 lines
Quoted 4 lines

Are you implying that, in your opinion, QEMU (say 5.2) should somehow provide compatibility with Fedora 31 because it was used during the entire cycle? I sympathize with that, but, QEMU is not really advertising compatibility support with specific Linux Distros, is it?

What I don't understand is why you remove F31 and not simply add a F32 test. Why should I stop testing F31 if I have it cached?

Well we can't let the list of OS increase unbounded.

IMHO there's not enough changes between each Fedora release to justify keeping them all. Same for any other short life distro.

If we're trying to get a variety of coverage, I'd suggest we focus on the long life distros for old stuff. eg RHEL-5, RHEL-6, RHEL-7, etc all have large enough differences in their virtual hardware to justify having a test for each major version.

We could then just have 1 recent-ish Fedora as a way to get coverage of the cutting edge.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#10 lore raw
In reply to Cleber Rosa
Quoted 5 lines · Cleber Rosa

I believe this may be a candidate for "right now" because the code changes here simply sync with external infrastructure changes, that is, the retirement of Fedora 31 from the official repository locations).

Strong "no" at this point in the release cycle. Only "this is absolutely critical because QEMU sets users' machines on fire" bug fixes at this point.

thanks
-- PMM

#11 lore raw
Quoted 7 lines · Peter Maydell
Quoted 5 lines · Cleber Rosa

I believe this may be a candidate for "right now" because the code changes here simply sync with external infrastructure changes, that is, the retirement of Fedora 31 from the official repository locations).

Quoted 7 lines

Strong "no" at this point in the release cycle. Only "this is absolutely critical because QEMU sets users' machines on fire" bug fixes at this point.

thanks
-- PMM

Sure,

Thanks for taking the time to consider it and for the response.

Cheers,
- Cleber.