Skip to contentSkip to conversation

tests/acceptance: Disable tests dependent of unreliable apt.armbian.com

PATCH In review Philippe Mathieu-Daudé 5 participants 9 messages Sep 17, 2020 – Sep 18, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 tests/acceptance: Disable tests dependent of unreliable apt.armbian.com

Armbian servers are not very reliable and confused the GitLab CI users a few times this month (path updated, archives moved, and now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired" error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add them to the cache).

---
 tests/acceptance/boot_linux_console.py | 10 ++++++++++
 tests/acceptance/replay_kernel.py      |  2 ++
 2 files changed, 12 insertions(+)
tests/acceptance/boot_linux_console.py +10 -0
index 4a366ce93e4..68534693109 100644
@@ -485,6 +485,8 @@ def test_arm_exynos4210_initrd(self):
         self.wait_for_console_pattern('Boot successful.')
         # TODO user command, for now the uart is stuck
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_initrd(self):
         """
         :avocado: tags=arch:arm
@@ -525,6 +527,8 @@ def test_arm_cubieboard_initrd(self):
                                                 'system-control@1c00000')
         # cubieboard's reboot is not functioning; omit reboot test.
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_sata(self):
         """
         :avocado: tags=arch:arm
@@ -651,6 +655,8 @@ def test_arm_quanta_gsj_initrd(self):
         self.wait_for_console_pattern(
                 'Give root password for system maintenance')
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_orangepi(self):
         """
         :avocado: tags=arch:arm
@@ -676,6 +682,8 @@ def test_arm_orangepi(self):
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_orangepi_initrd(self):
         """
         :avocado: tags=arch:arm
@@ -718,6 +726,8 @@ def test_arm_orangepi_initrd(self):
         # Wait for VM to shut down gracefully
         self.vm.wait()
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_orangepi_sd(self):
         """
         :avocado: tags=arch:arm
tests/acceptance/replay_kernel.py +2 -0
index b79fc8daf8e..7b4310fccf3 100644
@@ -127,6 +127,8 @@ def test_arm_virt(self):
         self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
 
     @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_initrd(self):
         """
         :avocado: tags=arch:arm
Signature · 1 line
2.26.2
#1 lore raw
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series seems to have some coding style problems. See output below for more information:

N/A. Internal error while reading log file

The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

Acked-by on v1 1/1
#2 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 7 lines · Philippe Mathieu-Daudé

Armbian servers are not very reliable and confused the GitLab CI users a few times this month (path updated, archives moved, and now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired" error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add them to the cache).

Quoted 5 lines
---
 tests/acceptance/boot_linux_console.py | 10 ++++++++++
 tests/acceptance/replay_kernel.py      |  2 ++
 2 files changed, 12 insertions(+)

These failures in the Gitlab-CI are quite annoying, indeed.

#3 lore raw
Quoted 8 lines · Thomas Huth
Quoted 7 lines · Philippe Mathieu-Daudé

Armbian servers are not very reliable and confused the GitLab CI users a few times this month (path updated, archives moved, and now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired" error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add them to the cache).

Quoted 7 lines
Quoted 5 lines
---
 tests/acceptance/boot_linux_console.py | 10 ++++++++++
 tests/acceptance/replay_kernel.py      |  2 ++
 2 files changed, 12 insertions(+)

These failures in the Gitlab-CI are quite annoying, indeed.

I'd rather not go via this path, but since we don't save the artifacts cache, I can't find a better way to not block all the other developers using GitLab CI.

But I prefer we disable the tests until something better emerge, rather that having the developers stopping running these tests. So this is a quick band-aid patch.

Quoted 3 lines

Thanks!

Phil.

Acked-by
#4 lore raw
Quoted 15 lines · Philippe Mathieu-Daudé
Quoted 14 lines · Thomas Huth
Quoted 12 lines · Philippe Mathieu-Daudé

Armbian servers are not very reliable and confused the GitLab CI users a few times this month (path updated, archives moved, and now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired" error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add them to the cache).

---
 tests/acceptance/boot_linux_console.py | 10 ++++++++++
 tests/acceptance/replay_kernel.py      |  2 ++
 2 files changed, 12 insertions(+)
Quoted 5 lines
Quoted 1 line

These failures in the Gitlab-CI are quite annoying, indeed.

I'd rather not go via this path, but since we don't save the artifacts cache, I can't find a better way to not block all the other developers using GitLab CI.

Well, we would need to cache them somewhere, either on qemu.org, or maybe have a download job in ci which publishes them on qemu.gitlab.io (gitlab pages).

Quoted 3 lines

But I prefer we disable the tests until something better emerge, rather that having the developers stopping running these tests. So this is a quick band-aid patch.

Yep, lets disable them until we have a solution.

take care,
Gerd

#5 lore raw
In reply to Philippe Mathieu-Daudé
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed build test on FreeBSD host. Please find the details below.

The full log is available at
http://patchew.org/logs/[email protected]/testing.FreeBSD/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

Reviewed-by on v1 1/1
#6 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 77 lines · Philippe Mathieu-Daudé

Armbian servers are not very reliable and confused the GitLab CI users a few times this month (path updated, archives moved, and now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired" error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add them to the cache).

---
 tests/acceptance/boot_linux_console.py | 10 ++++++++++
 tests/acceptance/replay_kernel.py      |  2 ++
 2 files changed, 12 insertions(+)
tests/acceptance/boot_linux_console.py +10 -0
index 4a366ce93e4..68534693109 100644
@@ -485,6 +485,8 @@ def test_arm_exynos4210_initrd(self):
         self.wait_for_console_pattern('Boot successful.')
         # TODO user command, for now the uart is stuck
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_initrd(self):
         """
         :avocado: tags=arch:arm
@@ -525,6 +527,8 @@ def test_arm_cubieboard_initrd(self):
                                                 'system-control@1c00000')
         # cubieboard's reboot is not functioning; omit reboot test.
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_sata(self):
         """
         :avocado: tags=arch:arm
@@ -651,6 +655,8 @@ def test_arm_quanta_gsj_initrd(self):
         self.wait_for_console_pattern(
                 'Give root password for system maintenance')
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_orangepi(self):
         """
         :avocado: tags=arch:arm
@@ -676,6 +682,8 @@ def test_arm_orangepi(self):
         console_pattern = 'Kernel command line: %s' % kernel_command_line
         self.wait_for_console_pattern(console_pattern)
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_orangepi_initrd(self):
         """
         :avocado: tags=arch:arm
@@ -718,6 +726,8 @@ def test_arm_orangepi_initrd(self):
         # Wait for VM to shut down gracefully
         self.vm.wait()
 
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_orangepi_sd(self):
         """
         :avocado: tags=arch:arm
tests/acceptance/replay_kernel.py +2 -1
index b79fc8daf8e..7b4310fccf3 100644
@@ -127,6 +127,8 @@ def test_arm_virt(self):
         self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
 
     @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_initrd(self):
         """
         :avocado: tags=arch:arm
--
Quoted 3 lines

2.26.2

#7 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 9 lines · Philippe Mathieu-Daudé

Armbian servers are not very reliable and confused the GitLab CI users a few times this month (path updated, archives moved, and now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired" error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add them to the cache).

---

[...]

Quoted 6 lines
tests/acceptance/replay_kernel.py +0 -0
index b79fc8daf8e..7b4310fccf3 100644

tests/acceptance/replay_kernel.py def test_arm_virt(self): · line 134

Quoted 8 lines
tests/acceptance/replay_kernel.py +2 -0
@@ -127,6 +127,8 @@ def test_arm_virt(self):
         self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
 
     @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
+    @skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+                'Test artifacts fetched from unreliable apt.armbian.com')
     def test_arm_cubieboard_initrd(self):
         """
         :avocado: tags=arch:arm

Oops for this one I forgot the "from avocado import skipUnless" line.

#8 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 7 lines · Philippe Mathieu-Daudé

Armbian servers are not very reliable and confused the GitLab CI users a few times this month (path updated, archives moved, and now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired" error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add them to the cache).

Quoted 5 lines
---
 tests/acceptance/boot_linux_console.py | 10 ++++++++++
 tests/acceptance/replay_kernel.py      |  2 ++
 2 files changed, 12 insertions(+)

Thanks, applied to my acceptance-testing tree.