Skip to contentSkip to conversation

Fedora images: use URLs from stable "archives.fedoraproject.org"

PATCH In review Cleber Rosa 7 participants 13 messages Sep 4, 2019 – Sep 11, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Fedora images: use URLs from stable "archives.fedoraproject.org"

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

Documentation and the "vm tests" fedora definition were also updated.

---
 qemu-doc.texi                          |  6 +++---
 tests/acceptance/boot_linux_console.py | 25 +++++++++++++++----------
 tests/acceptance/linux_initrd.py       |  5 +++--
 tests/vm/fedora                        |  2 +-
 4 files changed, 22 insertions(+), 16 deletions(-)
qemu-doc.texi +3 -3
index 577d1e8376..37795f86fb 100644
@@ -440,15 +440,15 @@ of <protocol>.
 
 Example: boot from a remote Fedora 20 live ISO image
 @example
-qemu-system-x86_64 --drive media=cdrom,file=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file=https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
 
-qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
 @end example
 
 Example: boot from a remote Fedora 20 cloud image using a local overlay for
 writes, copy-on-read, and a readahead of 64k
 @example
-qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
+qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
 
 qemu-system-x86_64 -drive file=/tmp/Fedora-x86_64-20-20131211.1-sda.qcow2,copy-on-read=on
 @end example
tests/acceptance/boot_linux_console.py +15 -10
index 2504ef0150..8a9a314ab4 100644
@@ -76,8 +76,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:x86_64
         :avocado: tags=machine:pc
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -250,8 +251,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:aarch64
         :avocado: tags=machine:virt
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -271,8 +273,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:arm
         :avocado: tags=machine:virt
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/armhfp/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -318,8 +321,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:s390x
         :avocado: tags=machine:s390_ccw_virtio
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/s390x/os/images/kernel.img')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/s390x/os/images'
+                      '/kernel.img')
         kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
         kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
tests/acceptance/linux_initrd.py +3 -2
index 23be5a63aa..c61d9826a4 100644
@@ -54,8 +54,9 @@ class LinuxInitrd(Test):
         QEMU has supported up to 4 GiB initrd for recent kernel
         Expect guest can reach 'Unpacking initramfs...'
         """
-        kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
-                      'Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/28/Everything/x86_64/os/images/pxeboot/'
+                      'vmlinuz')
         kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
         max_size = 2 * (1024 ** 3) + 1
tests/vm/fedora +1 -1
index e8fa5bf0d2..7fec1479fb 100755
@@ -23,7 +23,7 @@ class FedoraVM(basevm.BaseVM):
     name = "fedora"
     arch = "x86_64"
 
-    base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/"
+    base = "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/30/"
     link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
     repo = base + "Server/x86_64/os/"
     full = base + "Everything/x86_64/os/"
Signature · 1 line
2.21.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 132 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

Documentation and the "vm tests" fedora definition were also updated.

---
 qemu-doc.texi                          |  6 +++---
 tests/acceptance/boot_linux_console.py | 25 +++++++++++++++----------
 tests/acceptance/linux_initrd.py       |  5 +++--
 tests/vm/fedora                        |  2 +-
 4 files changed, 22 insertions(+), 16 deletions(-)
qemu-doc.texi +3 -3
index 577d1e8376..37795f86fb 100644
@@ -440,15 +440,15 @@ of <protocol>.
 
 Example: boot from a remote Fedora 20 live ISO image
 @example
-qemu-system-x86_64 --drive media=cdrom,file=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file=https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
 
-qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
 @end example
 
 Example: boot from a remote Fedora 20 cloud image using a local overlay for
 writes, copy-on-read, and a readahead of 64k
 @example
-qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
+qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
 
 qemu-system-x86_64 -drive file=/tmp/Fedora-x86_64-20-20131211.1-sda.qcow2,copy-on-read=on
 @end example
tests/acceptance/boot_linux_console.py +15 -10
index 2504ef0150..8a9a314ab4 100644
@@ -76,8 +76,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:x86_64
         :avocado: tags=machine:pc
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -250,8 +251,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:aarch64
         :avocado: tags=machine:virt
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -271,8 +273,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:arm
         :avocado: tags=machine:virt
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/armhfp/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -318,8 +321,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:s390x
         :avocado: tags=machine:s390_ccw_virtio
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/s390x/os/images/kernel.img')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/s390x/os/images'
+                      '/kernel.img')
         kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
         kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
tests/acceptance/linux_initrd.py +3 -2
index 23be5a63aa..c61d9826a4 100644
@@ -54,8 +54,9 @@ class LinuxInitrd(Test):
         QEMU has supported up to 4 GiB initrd for recent kernel
         Expect guest can reach 'Unpacking initramfs...'
         """
-        kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
-                      'Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/28/Everything/x86_64/os/images/pxeboot/'
+                      'vmlinuz')
         kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
         max_size = 2 * (1024 ** 3) + 1
tests/vm/fedora +0 -0
index e8fa5bf0d2..7fec1479fb 100755

tests/vm/fedora class FedoraVM(basevm.BaseVM): · line 29

Quoted 8 lines
tests/vm/fedora +1 -1
@@ -23,7 +23,7 @@ class FedoraVM(basevm.BaseVM):
     name = "fedora"
     arch = "x86_64"
 
-    base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/"
+    base = "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/30/"
     link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
     repo = base + "Server/x86_64/os/"
     full = base + "Everything/x86_64/os/"

Thanks for the fix Cleber.

Reviewed-by on v1 1/1
#2 lore raw
In reply to Cleber Rosa

Hi Cleber,

Quoted 132 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

Documentation and the "vm tests" fedora definition were also updated.

---
  qemu-doc.texi                          |  6 +++---
  tests/acceptance/boot_linux_console.py | 25 +++++++++++++++----------
  tests/acceptance/linux_initrd.py       |  5 +++--
  tests/vm/fedora                        |  2 +-
  4 files changed, 22 insertions(+), 16 deletions(-)
qemu-doc.texi +3 -3
index 577d1e8376..37795f86fb 100644
@@ -440,15 +440,15 @@ of <protocol>.
 
  Example: boot from a remote Fedora 20 live ISO image
  @example
-qemu-system-x86_64 --drive media=cdrom,file=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file=https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
 
-qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
  @end example
 
  Example: boot from a remote Fedora 20 cloud image using a local overlay for
  writes, copy-on-read, and a readahead of 64k
  @example
-qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
+qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
 
  qemu-system-x86_64 -drive file=/tmp/Fedora-x86_64-20-20131211.1-sda.qcow2,copy-on-read=on
  @end example
tests/acceptance/boot_linux_console.py +15 -10
index 2504ef0150..8a9a314ab4 100644
@@ -76,8 +76,9 @@ class BootLinuxConsole(Test):
          :avocado: tags=arch:x86_64
          :avocado: tags=machine:pc
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
+                      '/vmlinuz')
          kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -250,8 +251,9 @@ class BootLinuxConsole(Test):
          :avocado: tags=arch:aarch64
          :avocado: tags=machine:virt
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
+                      '/vmlinuz')
          kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -271,8 +273,9 @@ class BootLinuxConsole(Test):
          :avocado: tags=arch:arm
          :avocado: tags=machine:virt
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/armhfp/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
+                      '/vmlinuz')
          kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -318,8 +321,9 @@ class BootLinuxConsole(Test):
          :avocado: tags=arch:s390x
          :avocado: tags=machine:s390_ccw_virtio
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/s390x/os/images/kernel.img')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/s390x/os/images'
+                      '/kernel.img')
          kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
          :avocado: tags=arch:ppc64
          :avocado: tags=machine:pseries
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
          kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
tests/acceptance/linux_initrd.py +3 -2
index 23be5a63aa..c61d9826a4 100644
@@ -54,8 +54,9 @@ class LinuxInitrd(Test):
          QEMU has supported up to 4 GiB initrd for recent kernel
          Expect guest can reach 'Unpacking initramfs...'
          """
-        kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
-                      'Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/28/Everything/x86_64/os/images/pxeboot/'
+                      'vmlinuz')
          kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
          kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
          max_size = 2 * (1024 ** 3) + 1
tests/vm/fedora +0 -0
index e8fa5bf0d2..7fec1479fb 100755

tests/vm/fedora class FedoraVM(basevm.BaseVM): · line 29

Quoted 8 lines
tests/vm/fedora +1 -1
@@ -23,7 +23,7 @@ class FedoraVM(basevm.BaseVM):
      name = "fedora"
      arch = "x86_64"
 
-    base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/"
+    base = "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/30/"
      link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
      repo = base + "Server/x86_64/os/"
      full = base + "Everything/x86_64/os/"

Currently it has just a few test cases that boot a Linux kernel. At this point maintain the urls to fetch the files across a few source file is manageable but it can get complex as it increases in number of tests. While ago I started to work on the introduction of a "kernel provider" mechanism to replace this "fetch a kernel" boilerplate. Unfortunately I didn't time to complete its implementation, now If you think it is valuable then I can manage to resume the work. Find the draft implementation here: https://github.com/wainersm/qemu/tree/acceptance_boot_linux

An alternate approach would be to extend Avocado by mimic the avocado.utils.vmimage module [1], but rather have kernel providers.

[1] https://avocado-framework.readthedocs.io/en/71.0/api/utils/avocado.utils.html#module-avocado.utils.vmimage

Anyway, the changes proposed on this patch looks good to me.

Thanks,

Wainer

#3 lore raw
Quoted 176 lines · Wainer dos Santos Moschetta

Hi Cleber,

Quoted 173 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file.  The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org".  As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs.  The good news is that the currently supported versions are also hosted on the later.  This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

Documentation and the "vm tests" fedora definition were also updated.

---
  qemu-doc.texi                          |  6 +++--- tests/acceptance/boot_linux_console.py | 25 +++++++++++++++---------- tests/acceptance/linux_initrd.py       |  5 +++-- tests/vm/fedora                        |  2 +-

  4 files changed, 22 insertions(+), 16 deletions(-)
qemu-doc.texi +0 -0
index 577d1e8376..37795f86fb 100644
@@ -440,15 +440,15 @@ of <protocol>.

    Example: boot from a remote Fedora 20 live ISO image
  @example
-qemu-system-x86_64 --drive media=cdrom,file=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly

+qemu-system-x86_64 --drive media=cdrom,file=https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly

  -qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly

+qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly

  @end example
    Example: boot from a remote Fedora 20 cloud image using a local
overlay for
  writes, copy-on-read, and a readahead of 64k
  @example
-qemu-img create -f qcow2 -o
backing_file='json:@{"file.driver":"http",, "file.url":"https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
+qemu-img create -f qcow2 -o
backing_file='json:@{"file.driver":"http",, "file.url":"http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
    qemu-system-x86_64 -drive
file=/tmp/Fedora-x86_64-20-20131211.1-sda.qcow2,copy-on-read=on
  @end example

tests/acceptance/boot_linux_console.py +0 -0
index 2504ef0150..8a9a314ab4 100644
@@ -76,8 +76,9 @@ class BootLinuxConsole(Test):

          :avocado: tags=arch:x86_64
          :avocado: tags=machine:pc
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-
'releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+
'/linux/releases/29/Everything/x86_64/os/images/pxeboot'
+                      '/vmlinuz')
          kernel_hash = '23bebd2680757891cf7adedb033532163a792495' kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
  @@ -250,8 +251,9 @@ class BootLinuxConsole(Test):
          :avocado: tags=arch:aarch64
          :avocado: tags=machine:virt
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
- 'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz') +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+
'/linux/releases/29/Everything/aarch64/os/images/pxeboot'
+                      '/vmlinuz')
          kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493' kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
  @@ -271,8 +273,9 @@ class BootLinuxConsole(Test):
          :avocado: tags=arch:arm
          :avocado: tags=machine:virt
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-
'releases/29/Everything/armhfp/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+
'/linux/releases/29/Everything/armhfp/os/images/pxeboot'
+                      '/vmlinuz')
          kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4' kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
  @@ -318,8 +321,9 @@ class BootLinuxConsole(Test):
          :avocado: tags=arch:s390x
          :avocado: tags=machine:s390_ccw_virtio
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/' -
'releases/29/Everything/s390x/os/images/kernel.img')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive' +
'/fedora-secondary/releases/29/Everything/s390x/os/images'
+                      '/kernel.img')
          kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313' kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
  @@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
          :avocado: tags=arch:ppc64
          :avocado: tags=machine:pseries
          """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/' -
'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive' +
'/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
          kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77' kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
  diff --git a/tests/acceptance/linux_initrd.py b/tests/acceptance/linux_initrd.py
index 23be5a63aa..c61d9826a4 100644

tests/acceptance/linux_initrd.py +0 -0
@@ -54,8 +54,9 @@ class LinuxInitrd(Test):

          QEMU has supported up to 4 GiB initrd for recent kernel Expect guest can reach 'Unpacking initramfs...' """
-        kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/' -                      'Everything/x86_64/os/images/pxeboot/vmlinuz') +        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+
'/linux/releases/28/Everything/x86_64/os/images/pxeboot/'
+                      'vmlinuz')
          kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a' kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
          max_size = 2 * (1024 ** 3) + 1

tests/vm/fedora +0 -0
index e8fa5bf0d2..7fec1479fb 100755
@@ -23,7 +23,7 @@ class FedoraVM(basevm.BaseVM):

      name = "fedora"
      arch = "x86_64"
  -    base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/" +    base = "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/30/"

Quoted 14 lines
Quoted 4 lines

      link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso" repo = base + "Server/x86_64/os/"
      full = base + "Everything/x86_64/os/"

Currently it has just a few test cases that boot a Linux kernel. At this point maintain the urls to fetch the files across a few source file is manageable but it can get complex as it increases in number of tests. While ago I started to work on the introduction of a "kernel provider" mechanism to replace this "fetch a kernel" boilerplate. Unfortunately I didn't time to complete its implementation, now If you think it is valuable then I can manage to resume the work. Find the draft implementation here:
https://github.com/wainersm/qemu/tree/acceptance_boot_linux

I remember and liked it, maybe you should start with a post asking the community if the effort is worthwhile and would be useful, and what combinations (distrib{name/version} kernel{version} arch profile{debug,lpae,...} ...) are expected.

Quoted 11 lines

An alternate approach would be to extend Avocado by mimic the avocado.utils.vmimage module [1], but rather have kernel providers.

[1]
https://avocado-framework.readthedocs.io/en/71.0/api/utils/avocado.utils.html#module-avocado.utils.vmimage

Anyway, the changes proposed on this patch looks good to me.

Ditto:

#4 lore raw
In reply to Cleber Rosa
Quoted 15 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

Quoted 4 lines

Documentation and the "vm tests" fedora definition were also updated.

---

I was planning to apply this to master to fix the Travis builds, but it doesn't apply -- conflicts in qemu-doc.texi. Could you respin?

thanks
-- PMM

#5 lore raw
Quoted 17 lines · Peter Maydell
Quoted 15 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

Quoted 7 lines
Quoted 4 lines

Documentation and the "vm tests" fedora definition were also updated.

---

I was planning to apply this to master to fix the Travis builds, but it doesn't apply -- conflicts in qemu-doc.texi. Could you respin?

Ah, never mind -- Alex tells me he's already picked this patch up in a pending pull request and has done the rebase and fixup there.

thanks
-- PMM

#6 lore raw
Quoted 23 lines · Peter Maydell
Quoted 22 lines · Peter Maydell
Quoted 19 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

Documentation and the "vm tests" fedora definition were also updated.

---

Quoted 8 lines
Quoted 2 lines

I was planning to apply this to master to fix the Travis builds, but it doesn't apply -- conflicts in qemu-doc.texi. Could you respin?

Ah, never mind -- Alex tells me he's already picked this patch up in a pending pull request and has done the rebase and fixup there.

thanks
-- PMM

Oh, OK, good!

Thanks!
- Cleber.

#7 lore raw
In reply to Peter Maydell
Quoted 18 lines · Peter Maydell
Quoted 17 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

Documentation and the "vm tests" fedora definition were also updated.

Quoted 9 lines
Quoted 2 lines

---

I was planning to apply this to master to fix the Travis builds, but it doesn't apply -- conflicts in qemu-doc.texi. Could you respin?

thanks
-- PMM

Sure!

- Cleber.

#8 lore raw
In reply to Cleber Rosa
Quoted 8 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

Quoted 5 lines

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

The download.fedoraproject.org site we're (mostly) currently using is serviced by the Fedora mirrors which is very desirable as it spreads the load.

The archive.fedoraproject.org site is the master Fedora hosting server(s). dl.fedoraproject.org is the same master hosting service that is intended for use by the mirror sites to sync their content from. Projects really shouldn't use either of these URLs for getting any content which is available via the mirror service as it places uncessary load on the Fedora master servers.

Quoted 23 lines

Documentation and the "vm tests" fedora definition were also updated.

---
 qemu-doc.texi                          |  6 +++---
 tests/acceptance/boot_linux_console.py | 25 +++++++++++++++----------
 tests/acceptance/linux_initrd.py       |  5 +++--
 tests/vm/fedora                        |  2 +-
 4 files changed, 22 insertions(+), 16 deletions(-)
qemu-doc.texi +2 -2
index 577d1e8376..37795f86fb 100644
@@ -440,15 +440,15 @@ of <protocol>.
 
 Example: boot from a remote Fedora 20 live ISO image
 @example
-qemu-system-x86_64 --drive media=cdrom,file=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file=https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
 
-qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly

qemu-doc.texi of <protocol>. · line 454

Quoted 10 lines
qemu-doc.texi +1 -1
 @end example
 
 Example: boot from a remote Fedora 20 cloud image using a local overlay for
 writes, copy-on-read, and a readahead of 64k
 @example
-qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
+qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
 
 qemu-system-x86_64 -drive file=/tmp/Fedora-x86_64-20-20131211.1-sda.qcow2,copy-on-read=on
 @end example

Nothing quite says "cutting edge" like examples pointing to 5+ years old software. Or to put it another way, we should update this to point to a modern OS release. ie Fedora 30.

These should also be changed to download instead of dl.

Quoted 53 lines
tests/acceptance/boot_linux_console.py +12 -8
index 2504ef0150..8a9a314ab4 100644
@@ -76,8 +76,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:x86_64
         :avocado: tags=machine:pc
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -250,8 +251,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:aarch64
         :avocado: tags=machine:virt
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -271,8 +273,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:arm
         :avocado: tags=machine:virt
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/armhfp/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -318,8 +321,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:s390x
         :avocado: tags=machine:s390_ccw_virtio
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/s390x/os/images/kernel.img')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/s390x/os/images'
+                      '/kernel.img')
         kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 371

Quoted 10 lines
tests/acceptance/boot_linux_console.py +3 -2
@@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
         kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)

Don't change these URLs. We can update to Fedora 30 though if you want to.

Quoted 22 lines
tests/acceptance/linux_initrd.py +3 -2
index 23be5a63aa..c61d9826a4 100644
@@ -54,8 +54,9 @@ class LinuxInitrd(Test):
         QEMU has supported up to 4 GiB initrd for recent kernel
         Expect guest can reach 'Unpacking initramfs...'
         """
-        kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
-                      'Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/28/Everything/x86_64/os/images/pxeboot/'
+                      'vmlinuz')
         kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
         max_size = 2 * (1024 ** 3) + 1
tests/vm/fedora +0 -0
index e8fa5bf0d2..7fec1479fb 100755

tests/vm/fedora class FedoraVM(basevm.BaseVM): · line 29

Quoted 8 lines
tests/vm/fedora +1 -1
@@ -23,7 +23,7 @@ class FedoraVM(basevm.BaseVM):
     name = "fedora"
     arch = "x86_64"
 
-    base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/"
+    base = "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/30/"
     link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
     repo = base + "Server/x86_64/os/"
     full = base + "Everything/x86_64/os/"

This should be changed to download instead of dl.

Quoted 4 lines
Signature · 1 line
2.21.0

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#9 lore raw
Quoted 57 lines · Daniel P. Berrangé
Quoted 8 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

Quoted 5 lines

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

The download.fedoraproject.org site we're (mostly) currently using is serviced by the Fedora mirrors which is very desirable as it spreads the load.

The archive.fedoraproject.org site is the master Fedora hosting server(s). dl.fedoraproject.org is the same master hosting service that is intended for use by the mirror sites to sync their content from. Projects really shouldn't use either of these URLs for getting any content which is available via the mirror service as it places uncessary load on the Fedora master servers.

Quoted 31 lines

Documentation and the "vm tests" fedora definition were also updated.

---
 qemu-doc.texi                          |  6 +++---
 tests/acceptance/boot_linux_console.py | 25 +++++++++++++++----------
 tests/acceptance/linux_initrd.py       |  5 +++--
 tests/vm/fedora                        |  2 +-
 4 files changed, 22 insertions(+), 16 deletions(-)
qemu-doc.texi +3 -3
index 577d1e8376..37795f86fb 100644
@@ -440,15 +440,15 @@ of <protocol>.
 
 Example: boot from a remote Fedora 20 live ISO image
 @example
-qemu-system-x86_64 --drive media=cdrom,file=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file=https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
 
-qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://dl.fedoraproject.org/pub/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
+qemu-system-x86_64 --drive media=cdrom,file.driver=http,file.url=http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Live/x86_64/Fedora-Live-Desktop-x86_64-20-1.iso,readonly
 @end example
 
 Example: boot from a remote Fedora 20 cloud image using a local overlay for
 writes, copy-on-read, and a readahead of 64k
 @example
-qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
+qemu-img create -f qcow2 -o backing_file='json:@{"file.driver":"http",, "file.url":"http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2",, "file.readahead":"64k"@}' /tmp/Fedora-x86_64-20-20131211.1-sda.qcow2
 
Quoted 7 lines

qemu-doc.texi of <protocol>. · line 454

Quoted 2 lines
qemu-doc.texi +0 -0
 qemu-system-x86_64 -drive file=/tmp/Fedora-x86_64-20-20131211.1-sda.qcow2,copy-on-read=on
 @end example

Nothing quite says "cutting edge" like examples pointing to 5+ years old software. Or to put it another way, we should update this to point to a modern OS release. ie Fedora 30.

Hi Daniel,

You're right, we should, and that's an obvious next step. I tried to keep this patch within the context of fixing the test regressions, and making the overall experience stable.

Quoted 2 lines

These should also be changed to download instead of dl.

I disagree, specially here, and believe that we shouldn't use the "archive" site. Having documentation that is accurate and whose examples can be reproduced without caveats is a nice usability touch.

I understand your worries about the eventual load on the Fedora server. While I'm certainly not qualified to speak for the Fedora project, it'd be natural that the project would eventually cope with the additional load, if that's actually significant.

Quoted 53 lines
Quoted 53 lines
tests/acceptance/boot_linux_console.py +12 -8
index 2504ef0150..8a9a314ab4 100644
@@ -76,8 +76,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:x86_64
         :avocado: tags=machine:pc
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -250,8 +251,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:aarch64
         :avocado: tags=machine:virt
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/aarch64/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = '8c73e469fc6ea06a58dc83a628fc695b693b8493'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -271,8 +273,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:arm
         :avocado: tags=machine:virt
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora/linux/'
-                      'releases/29/Everything/armhfp/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/29/Everything/armhfp/os/images/pxeboot'
+                      '/vmlinuz')
         kernel_hash = 'e9826d741b4fb04cadba8d4824d1ed3b7fb8b4d4'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
@@ -318,8 +321,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:s390x
         :avocado: tags=machine:s390_ccw_virtio
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/s390x/os/images/kernel.img')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/s390x/os/images'
+                      '/kernel.img')
         kernel_hash = 'e8e8439103ef8053418ef062644ffd46a7919313'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
Quoted 15 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 371

Quoted 11 lines
tests/acceptance/boot_linux_console.py +3 -2
@@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
         kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)

Don't change these URLs. We can update to Fedora 30 though if you want to.

I believe your suggestion to not change the URLs is based solely on your perspective on the load on the "archives" server, right?

You should know that there's a number of counter points. One is that these tests are used on environments, which are maintained for a much longer time than the lifespan of the currently maintained Fedora versions. We really need to make it stable and reliable, and IMO it should be done upstream, for the benefit of all.

Best,
- Cleber.

Quoted 44 lines
Quoted 22 lines
tests/acceptance/linux_initrd.py +3 -2
index 23be5a63aa..c61d9826a4 100644
@@ -54,8 +54,9 @@ class LinuxInitrd(Test):
         QEMU has supported up to 4 GiB initrd for recent kernel
         Expect guest can reach 'Unpacking initramfs...'
         """
-        kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/'
-                      'Everything/x86_64/os/images/pxeboot/vmlinuz')
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
+                      '/linux/releases/28/Everything/x86_64/os/images/pxeboot/'
+                      'vmlinuz')
         kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
         max_size = 2 * (1024 ** 3) + 1
tests/vm/fedora +0 -0
index e8fa5bf0d2..7fec1479fb 100755

tests/vm/fedora class FedoraVM(basevm.BaseVM): · line 29

Quoted 8 lines
tests/vm/fedora +1 -1
@@ -23,7 +23,7 @@ class FedoraVM(basevm.BaseVM):
     name = "fedora"
     arch = "x86_64"
 
-    base = "http://dl.fedoraproject.org/pub/fedora/linux/releases/30/"
+    base = "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/30/"
     link = base + "Server/x86_64/iso/Fedora-Server-netinst-x86_64-30-1.2.iso"
     repo = base + "Server/x86_64/os/"
     full = base + "Everything/x86_64/os/"

This should be changed to download instead of dl.

Quoted 4 lines
Signature · 1 line
2.21.0

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#10 lore raw
Quoted 27 lines · Cleber Rosa
Quoted 26 lines · Daniel P. Berrangé
Quoted 8 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

Quoted 5 lines

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

The download.fedoraproject.org site we're (mostly) currently using is serviced by the Fedora mirrors which is very desirable as it spreads the load.

The archive.fedoraproject.org site is the master Fedora hosting server(s). dl.fedoraproject.org is the same master hosting service that is intended for use by the mirror sites to sync their content from. Projects really shouldn't use either of these URLs for getting any content which is available via the mirror service as it places uncessary load on the Fedora master servers.

[...]

Quoted 16 lines
Quoted 6 lines
Quoted 6 lines
tests/acceptance/boot_linux_console.py +0 -2
@@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
Quoted 9 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 371

Quoted 5 lines
tests/acceptance/boot_linux_console.py +3 -0
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
         kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)

Don't change these URLs. We can update to Fedora 30 though if you want to.

Quoted 8 lines

I believe your suggestion to not change the URLs is based solely on your perspective on the load on the "archives" server, right?

You should know that there's a number of counter points. One is that these tests are used on environments, which are maintained for a much longer time than the lifespan of the currently maintained Fedora versions. We really need to make it stable and reliable, and IMO it should be done upstream, for the benefit of all.

We're trying to offload the costs of long term hosting for our test cases to a third party (Fedora Project), but we need to know if that's really acceptable usage of archives.fedoraproject.org.

Signature · 1 line
Eduardo
#11 lore raw
Quoted 54 lines · Eduardo Habkost
Quoted 27 lines · Cleber Rosa
Quoted 26 lines · Daniel P. Berrangé
Quoted 8 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

Quoted 5 lines

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

The download.fedoraproject.org site we're (mostly) currently using is serviced by the Fedora mirrors which is very desirable as it spreads the load.

The archive.fedoraproject.org site is the master Fedora hosting server(s). dl.fedoraproject.org is the same master hosting service that is intended for use by the mirror sites to sync their content from. Projects really shouldn't use either of these URLs for getting any content which is available via the mirror service as it places uncessary load on the Fedora master servers.

[...]

Quoted 16 lines
Quoted 6 lines
Quoted 6 lines
tests/acceptance/boot_linux_console.py +0 -2
@@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
Quoted 9 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 371

Quoted 5 lines
tests/acceptance/boot_linux_console.py +3 -0
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
         kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)

Don't change these URLs. We can update to Fedora 30 though if you want to.

Quoted 8 lines

I believe your suggestion to not change the URLs is based solely on your perspective on the load on the "archives" server, right?

You should know that there's a number of counter points. One is that these tests are used on environments, which are maintained for a much longer time than the lifespan of the currently maintained Fedora versions. We really need to make it stable and reliable, and IMO it should be done upstream, for the benefit of all.

Quoted 6 lines

We're trying to offload the costs of long term hosting for our test cases to a third party (Fedora Project), but we need to know if that's really acceptable usage of archives.fedoraproject.org.

Signature · 1 line
Eduardo

I'd argue that we're boosting archives.fedoraproject.org morale. As a public Internet service, it begs to better be used! :)

Now, more seriously, I agree that we should coordinate with them. I have the feeling that it shouldn't be a problem of resources per se, given that we're redirecting traffic. But, it may be a matter of letting them better allocate these resources.

Daniel,

You seem to have some much better than average knowledge about the mirror structure. Does that mean you know who to contact about this?

Thanks,
- Cleber.

#12 lore raw
Quoted 59 lines · Cleber Rosa
Quoted 58 lines · Eduardo Habkost
Quoted 27 lines · Cleber Rosa
Quoted 26 lines · Daniel P. Berrangé
Quoted 8 lines · Cleber Rosa

The LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 test, from tests/acceptance/linux_initrd.py, is currently failing to fetch the "vmlinuz" file. The reason for the failure is that the Fedora project retires older versions from the "dl.fedoraproject.org" URL, and keeps them in "archives.fedoraproject.org". As an added note, that test uses a Fedora 28 image, because of the specific Linux kernel version requirements of the test.

Quoted 5 lines

For the sake of stability, let's use URLs from the archived and supposedely ever stable URLs. The good news is that the currently supported versions are also hosted on the later. This change limits itself to change the URLs, while keeping the fetched files the same (as can be evidenced by the unchanged hashes).

The download.fedoraproject.org site we're (mostly) currently using is serviced by the Fedora mirrors which is very desirable as it spreads the load.

The archive.fedoraproject.org site is the master Fedora hosting server(s). dl.fedoraproject.org is the same master hosting service that is intended for use by the mirror sites to sync their content from. Projects really shouldn't use either of these URLs for getting any content which is available via the mirror service as it places uncessary load on the Fedora master servers.

[...]

Quoted 16 lines
Quoted 6 lines
Quoted 6 lines
tests/acceptance/boot_linux_console.py +0 -2
@@ -360,8 +364,9 @@ class BootLinuxConsole(Test):
         :avocado: tags=arch:ppc64
         :avocado: tags=machine:pseries
         """
-        kernel_url = ('https://download.fedoraproject.org/pub/fedora-secondary/'
-                      'releases/29/Everything/ppc64le/os/ppc/ppc64/vmlinuz')
Quoted 9 lines

tests/acceptance/boot_linux_console.py class BootLinuxConsole(Test): · line 371

Quoted 5 lines
tests/acceptance/boot_linux_console.py +3 -0
+        kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+                      '/fedora-secondary/releases/29/Everything/ppc64le/os'
+                      '/ppc/ppc64/vmlinuz')
         kernel_hash = '3fe04abfc852b66653b8c3c897a59a689270bc77'
         kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)

Don't change these URLs. We can update to Fedora 30 though if you want to.

Quoted 8 lines

I believe your suggestion to not change the URLs is based solely on your perspective on the load on the "archives" server, right?

You should know that there's a number of counter points. One is that these tests are used on environments, which are maintained for a much longer time than the lifespan of the currently maintained Fedora versions. We really need to make it stable and reliable, and IMO it should be done upstream, for the benefit of all.

We're trying to offload the costs of long term hosting for our test cases to a third party (Fedora Project), but we need to know if that's really acceptable usage of archives.fedoraproject.org.

Quoted 5 lines
Quoted 2 lines
Signature · 1 line
Eduardo

I'd argue that we're boosting archives.fedoraproject.org morale. As a public Internet service, it begs to better be used! :)

Not really, in general we want users to keep updated with latest supported Fedora releases and not use EOL versions from archive. The critical reason for archives.fp.org to exist is license, beyond that its largely just a historical record, but we digress off topic here...

Quoted 9 lines

Now, more seriously, I agree that we should coordinate with them. I have the feeling that it shouldn't be a problem of resources per se, given that we're redirecting traffic. But, it may be a matter of letting them better allocate these resources.

Daniel,

You seem to have some much better than average knowledge about the mirror structure. Does that mean you know who to contact about this?

I spoke with some folks I know who didn't have an answer on the impact or policy around using archive.fp.org, but did point out that there's a way to avoid breaking

The Fedora mirror manager software provides a URL to query what sites currently host a given version

eg to find Fedora 16 for x86_64 fetch this

https://mirrors.fedoraproject.org/mirrorlist?repo=fedora-16&arch=x86_64

whereupon it reports something like this:

# repo = fedora-16 arch = x86_64 country = DK country = DE country = UA country = global 
https://mirrors.dotsrc.org/fedora-buffet/archive/fedora/linux/releases/16/Everything/x86_64/os/
https://fedora-archive.ip-connect.info/fedora/linux/releases/16/Everything/x86_64/os/
https://ftp-stud.hs-esslingen.de/pub/Mirrors/archive.fedoraproject.org/fedora/linux/releases/16/Everything/x86_64/os/
https://fedora-archive.ip-connect.vn.ua/fedora/linux/releases/16/Everything/x86_64/os/
http://ftp.cuhk.edu.hk/pub/linux/fedora-archive/fedora/linux/releases/16/Everything/x86_64/os/
http://fedora-archive.mirror.liquidtelecom.com/archive/fedora/linux/releases/16/Everything/x86_64/os/
http://mirrors.kernel.org/fedora-buffet/archive/fedora/linux/releases/16/Everything/x86_64/os/
http://mirror.math.princeton.edu/pub/fedora-archive/fedora/linux/releases/16/Everything/x86_64/os/
https://pubmirror2.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/16/Everything/x86_64/os/
https://pubmirror1.math.uh.edu/fedora-buffet/archive/fedora/linux/releases/16/Everything/x86_64/os/
https://dl.fedoraproject.org/pub/archive/fedora/linux/releases/16/Everything/x86_64/os/

any one of those returned URLs should serve the content we want. It is interesting to see that there's several mirrors still carrying ancient software, despite 'download.fp.org' returning 404

Admittedly this extra redirection is a bit more tedious for the test suite, but we could hide that behind a common helper function.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|