Skip to contentSkip to conversation

tests/vm/freebsd: Get up-to-date package list from lcitool

PATCH In review Philippe Mathieu-Daudé 6 participants 22 messages Jul 11, 2023 – Jan 3, 2024

lore
Share & tools
lore
#0 lore raw

v3 cover tests/vm/freebsd: Get up-to-date package list from lcitool

Inspired by this patch from Thomas:
https://lore.kernel.org/qemu-devel/[email protected]/

Instead of updating the package list manually, use lcitool vars file.

Since v2:
- Commit generated json (Daniel)

Since v1:
- Addressed Erik & Daniel comments (generate in JSON)

Philippe Mathieu-Daudé (4):
tests/lcitool: Generate distribution packages list in JSON format tests/lcitool: Refresh generated files
tests/vm: Introduce get_qemu_packages_from_lcitool_json() helper tests/vm/freebsd: Get up-to-date package list from lcitool vars file

 tests/docker/dockerfiles/debian-amd64.docker |  2 -
 tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
 tests/docker/dockerfiles/ubuntu2204.docker   |  2 -
 tests/lcitool/refresh                        | 11 +++
 tests/vm/basevm.py                           | 11 +++
 tests/vm/freebsd                             | 42 +----------
 tests/vm/generated/README                    |  5 ++
 tests/vm/generated/freebsd.json              | 77 ++++++++++++++++++++
 8 files changed, 106 insertions(+), 46 deletions(-)
 create mode 100644 tests/vm/generated/README
 create mode 100644 tests/vm/generated/freebsd.json
Signature · 1 line
2.38.1
#1 lore raw

PATCH v3 1/4 tests/lcitool: Generate distribution packages list in JSON format

Add the generate_pkglist() helper to generate a list of packages required by a distribution to build QEMU.

Since we can not add a "THIS FILE WAS AUTO-GENERATED" comment in JSON, create the files under tests/vm/generated/ sub-directory; add a README mentioning the files are generated.

---
 tests/lcitool/refresh     | 11 +++++++++++
 tests/vm/generated/README |  5 +++++
 2 files changed, 16 insertions(+)
 create mode 100644 tests/vm/generated/README
tests/lcitool/refresh +11 -0
index b54566edcc..4584870ea1 100755
@@ -84,6 +84,12 @@ def generate_cirrus(target, trailer=None):
     generate(filename, cmd, trailer)
 
 
+def generate_pkglist(vm, target):
+    filename = Path(src_dir, "tests", "vm", "generated", vm + ".json")
+    cmd = lcitool_cmd + ["variables", "--format", "json", target, "qemu"]
+    generate(filename, cmd, None)
+
+
 # Netmap still needs to be manually built as it is yet to be packaged
 # into a distro. We also add cscope and gtags which are used in the CI
 # test
@@ -191,6 +197,11 @@ try:
     generate_cirrus("freebsd-13")
     generate_cirrus("macos-12")
 
+    #
+    # VM packages lists
+    #
+    generate_pkglist("freebsd", "freebsd-13")
+
     sys.exit(0)
 except Exception as ex:
     print(str(ex), file=sys.stderr)
tests/vm/generated/README new file +5 -0
new file mode 100644
index 0000000000..7ccc6ffd3d
@@ -0,0 +1,5 @@
+# FILES IN THIS FOLDER WERE AUTO-GENERATED
+#
+#  $ make lcitool-refresh
+#
+# https://gitlab.com/libvirt/libvirt-ci
Signature · 1 line
2.38.1
Reviewed-by on v3 1/4
#2 lore raw

Re: [PATCH v3 1/4] tests/lcitool: Generate distribution packages list in JSON format

Quoted 7 lines · Philippe Mathieu-Daudé

Add the generate_pkglist() helper to generate a list of packages required by a distribution to build QEMU.

Since we can not add a "THIS FILE WAS AUTO-GENERATED" comment in JSON, create the files under tests/vm/generated/ sub-directory; add a README mentioning the files are generated.

Quoted 5 lines
---
 tests/lcitool/refresh     | 11 +++++++++++
 tests/vm/generated/README |  5 +++++

Or 's/generated/packages/' perhaps ?

Quoted 2 lines
 2 files changed, 16 insertions(+)
 create mode 100644 tests/vm/generated/README

Either way, at your discretion

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Reviewed-by on v3 1/4
#3 lore raw

Re: [PATCH v3 1/4] tests/lcitool: Generate distribution packages list in JSON format

In reply to Philippe Mathieu-Daudé
Quoted 10 lines · Philippe Mathieu-Daudé

Add the generate_pkglist() helper to generate a list of packages required by a distribution to build QEMU.

Since we can not add a "THIS FILE WAS AUTO-GENERATED" comment in JSON, create the files under tests/vm/generated/ sub-directory; add a README mentioning the files are generated.

also, FreeBSD 14 branches next month... do I just grep for FreeBSD-13 to find all the places to update for 14.0?

Warner

---

Quoted 49 lines
 tests/lcitool/refresh     | 11 +++++++++++
 tests/vm/generated/README |  5 +++++
 2 files changed, 16 insertions(+)
 create mode 100644 tests/vm/generated/README
tests/lcitool/refresh +11 -0
index b54566edcc..4584870ea1 100755
@@ -84,6 +84,12 @@ def generate_cirrus(target, trailer=None):
     generate(filename, cmd, trailer)
 
 
+def generate_pkglist(vm, target):
+    filename = Path(src_dir, "tests", "vm", "generated", vm + ".json")
+    cmd = lcitool_cmd + ["variables", "--format", "json", target, "qemu"]
+    generate(filename, cmd, None)
+
+
 # Netmap still needs to be manually built as it is yet to be packaged
 # into a distro. We also add cscope and gtags which are used in the CI
 # test
@@ -191,6 +197,11 @@ try:
     generate_cirrus("freebsd-13")
     generate_cirrus("macos-12")
 
+    #
+    # VM packages lists
+    #
+    generate_pkglist("freebsd", "freebsd-13")
+
     sys.exit(0)
 except Exception as ex:
     print(str(ex), file=sys.stderr)
tests/vm/generated/README new file +5 -1
new file mode 100644
index 0000000000..7ccc6ffd3d
@@ -0,0 +1,5 @@
+# FILES IN THIS FOLDER WERE AUTO-GENERATED
+#
+#  $ make lcitool-refresh
+#
+# https://gitlab.com/libvirt/libvirt-ci
--

2.38.1

#4 lore raw

Re: [PATCH v3 1/4] tests/lcitool: Generate distribution packages list in JSON format

Quoted 10 lines · Warner Losh

On Tue, Jul 11, 2023 at 8:49 AM Philippe Mathieu-Daudé <[email protected] <mailto:[email protected]>> wrote:

Add the generate_pkglist() helper to generate a list of packages required by a distribution to build QEMU.

Since we can not add a "THIS FILE WAS AUTO-GENERATED" comment in JSON, create the files under tests/vm/generated/ sub-directory; add a README mentioning the files are generated.

Quoted 8 lines

<mailto:[email protected]>>

<mailto:[email protected]>>

Reviewed-by Warner Losh <[email protected] <mailto:[email protected]>>

also, FreeBSD 14 branches next month... do I just grep for FreeBSD-13 to find all the places to update for 14.0?

Per docs/about/build-platforms.rst:

Linux OS, macOS, FreeBSD, NetBSD, OpenBSD
-----------------------------------------

The project aims to support the most recent major version at all times for up to five years after its initial release. Support for the previous major version will be dropped 2 years after the new major version is released or when the vendor itself drops support, whichever comes first. [...]

We want to be able to test the oldest/newest releases.

Maybe we can add a pair of definitions, so we'd have to only change 2 lines in a single place when releases occur?

Quoted 8 lines

@@ -191,6 +197,11 @@ try:
     generate_cirrus("freebsd-13")
     generate_cirrus("macos-12")

+    #
+    # VM packages lists
+    #
+    generate_pkglist("freebsd", "freebsd-13")

#5 lore raw

Re: [PATCH v3 1/4] tests/lcitool: Generate distribution packages list in JSON format

Quoted 33 lines · Philippe Mathieu-Daudé
Quoted 12 lines · Warner Losh

On Tue, Jul 11, 2023 at 8:49 AM Philippe Mathieu-Daudé <[email protected] <mailto:[email protected]>> wrote:

Add the generate_pkglist() helper to generate a list of packages required by a distribution to build QEMU.

Since we can not add a "THIS FILE WAS AUTO-GENERATED" comment in JSON, create the files under tests/vm/generated/ sub-directory; add a README mentioning the files are generated.

<mailto:[email protected]>>

Quoted 8 lines

<mailto:[email protected]>>

Reviewed-by Warner Losh <[email protected] <mailto:[email protected]>>

also, FreeBSD 14 branches next month... do I just grep for FreeBSD-13 to find all the places to update for 14.0?

Per docs/about/build-platforms.rst:

Linux OS, macOS, FreeBSD, NetBSD, OpenBSD
-----------------------------------------

The project aims to support the most recent major version at all times for up to five years after its initial release. Support for the previous major version will be dropped 2 years after the new major version is released or when the vendor itself drops support, whichever comes first. [...]

Quoted 5 lines

We want to be able to test the oldest/newest releases.

Maybe we can add a pair of definitions, so we'd have to only change 2 lines in a single place when releases occur?

I'd love that. There's two bits of information per release: name and checksum
of release artifacts.

Warner

Quoted 11 lines
Quoted 9 lines

@@ -191,6 +197,11 @@ try:
generate_cirrus("freebsd-13")
generate_cirrus("macos-12")

+ #
+ # VM packages lists
+ #
+ generate_pkglist("freebsd", "freebsd-13")

#6 lore raw

PATCH v3 2/4 tests/lcitool: Refresh generated files

In reply to Philippe Mathieu-Daudé

Refresh the generated files by running:

  $ make lcitool-refresh
---
 tests/docker/dockerfiles/debian-amd64.docker |  2 -
 tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
 tests/docker/dockerfiles/ubuntu2204.docker   |  2 -
 tests/vm/generated/freebsd.json              | 77 ++++++++++++++++++++
 4 files changed, 77 insertions(+), 6 deletions(-)
 create mode 100644 tests/vm/generated/freebsd.json
tests/docker/dockerfiles/debian-amd64.docker +0 -2
index e39871c7bb..8f7521fdc2 100644
@@ -70,7 +70,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libpam0g-dev \
                       libpcre2-dev \
                       libpixman-1-dev \
-                      libpmem-dev \
                       libpng-dev \
                       libpulse-dev \
                       librbd-dev \
@@ -96,7 +95,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libvdeplug-dev \
                       libvirglrenderer-dev \
                       libvte-2.91-dev \
-                      libxen-dev \
                       libzstd-dev \
                       llvm \
                       locales \
tests/docker/dockerfiles/ubuntu2004.docker +0 -2
index 8f864d19e6..7f60143cbb 100644
@@ -69,7 +69,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libpam0g-dev \
                       libpcre2-dev \
                       libpixman-1-dev \
-                      libpmem-dev \
                       libpng-dev \
                       libpulse-dev \
                       librbd-dev \
@@ -94,7 +93,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libvdeplug-dev \
                       libvirglrenderer-dev \
                       libvte-2.91-dev \
-                      libxen-dev \
                       libzstd-dev \
                       llvm \
                       locales \
tests/docker/dockerfiles/ubuntu2204.docker +0 -2
index 1d442cdfe6..5162927016 100644
@@ -70,7 +70,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libpam0g-dev \
                       libpcre2-dev \
                       libpixman-1-dev \
-                      libpmem-dev \
                       libpng-dev \
                       libpulse-dev \
                       librbd-dev \
@@ -96,7 +95,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libvdeplug-dev \
                       libvirglrenderer-dev \
                       libvte-2.91-dev \
-                      libxen-dev \
                       libzstd-dev \
                       llvm \
                       locales \
tests/vm/generated/freebsd.json new file +77 -0

77 lines not on this page · raw message

Signature · 1 line
2.38.1
Reviewed-by on v3 2/4
#7 lore raw

Re: [PATCH v3 2/4] tests/lcitool: Refresh generated files

Quoted 9 lines · Philippe Mathieu-Daudé

Refresh the generated files by running:

  $ make lcitool-refresh
---
 tests/docker/dockerfiles/debian-amd64.docker |  2 -
 tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
 tests/docker/dockerfiles/ubuntu2204.docker   |  2 -

I don't know why this would be removing xen/pmem from these files. If I run 'lcitool-refresh' on current git master that doesn't happen

Do you have any other local changes on top ?

Quoted 3 lines
 tests/vm/generated/freebsd.json              | 77 ++++++++++++++++++++
 4 files changed, 77 insertions(+), 6 deletions(-)
 create mode 100644 tests/vm/generated/freebsd.json
Quoted 148 lines
tests/docker/dockerfiles/debian-amd64.docker +0 -2
index e39871c7bb..8f7521fdc2 100644
@@ -70,7 +70,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libpam0g-dev \
                       libpcre2-dev \
                       libpixman-1-dev \
-                      libpmem-dev \
                       libpng-dev \
                       libpulse-dev \
                       librbd-dev \
@@ -96,7 +95,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libvdeplug-dev \
                       libvirglrenderer-dev \
                       libvte-2.91-dev \
-                      libxen-dev \
                       libzstd-dev \
                       llvm \
                       locales \
tests/docker/dockerfiles/ubuntu2004.docker +0 -2
index 8f864d19e6..7f60143cbb 100644
@@ -69,7 +69,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libpam0g-dev \
                       libpcre2-dev \
                       libpixman-1-dev \
-                      libpmem-dev \
                       libpng-dev \
                       libpulse-dev \
                       librbd-dev \
@@ -94,7 +93,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libvdeplug-dev \
                       libvirglrenderer-dev \
                       libvte-2.91-dev \
-                      libxen-dev \
                       libzstd-dev \
                       llvm \
                       locales \
tests/docker/dockerfiles/ubuntu2204.docker +0 -2
index 1d442cdfe6..5162927016 100644
@@ -70,7 +70,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libpam0g-dev \
                       libpcre2-dev \
                       libpixman-1-dev \
-                      libpmem-dev \
                       libpng-dev \
                       libpulse-dev \
                       librbd-dev \
@@ -96,7 +95,6 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
                       libvdeplug-dev \
                       libvirglrenderer-dev \
                       libvte-2.91-dev \
-                      libxen-dev \
                       libzstd-dev \
                       llvm \
                       locales \
tests/vm/generated/freebsd.json new file +77 -0
new file mode 100644
index 0000000000..7c435cf23e
@@ -0,0 +1,77 @@
+{
+  "ccache": "/usr/local/bin/ccache",
+  "cpan_pkgs": [],
+  "cross_pkgs": [],
+  "make": "/usr/local/bin/gmake",
+  "ninja": "/usr/local/bin/ninja",
+  "packaging_command": "pkg",
+  "pip3": "/usr/local/bin/pip-3.8",
+  "pkgs": [
+    "alsa-lib",
+    "bash",
+    "bison",
+    "bzip2",
+    "ca_root_nss",
+    "capstone4",
+    "ccache",
+    "cmocka",
+    "ctags",
+    "curl",
+    "cyrus-sasl",
+    "dbus",
+    "diffutils",
+    "dtc",
+    "flex",
+    "fusefs-libs3",
+    "gettext",
+    "git",
+    "glib",
+    "gmake",
+    "gnutls",
+    "gsed",
+    "gtk3",
+    "json-c",
+    "libepoxy",
+    "libffi",
+    "libgcrypt",
+    "libjpeg-turbo",
+    "libnfs",
+    "libslirp",
+    "libspice-server",
+    "libssh",
+    "libtasn1",
+    "llvm",
+    "lzo2",
+    "meson",
+    "mtools",
+    "ncurses",
+    "nettle",
+    "ninja",
+    "opencv",
+    "pixman",
+    "pkgconf",
+    "png",
+    "py39-numpy",
+    "py39-pillow",
+    "py39-pip",
+    "py39-sphinx",
+    "py39-sphinx_rtd_theme",
+    "py39-yaml",
+    "python3",
+    "rpm2cpio",
+    "sdl2",
+    "sdl2_image",
+    "snappy",
+    "sndio",
+    "socat",
+    "spice-protocol",
+    "tesseract",
+    "usbredir",
+    "virglrenderer",
+    "vte3",
+    "xorriso",
+    "zstd"
+  ],
+  "pypi_pkgs": [],
+  "python": "/usr/local/bin/python3"
+}
Signature · 1 line
2.38.1

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#8 lore raw

Re: [PATCH v3 2/4] tests/lcitool: Refresh generated files

Quoted 11 lines · Daniel P. Berrangé
Quoted 9 lines · Philippe Mathieu-Daudé

Refresh the generated files by running:

   $ make lcitool-refresh
---
  tests/docker/dockerfiles/debian-amd64.docker |  2 -
  tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
  tests/docker/dockerfiles/ubuntu2204.docker   |  2 -
Quoted 4 lines

I don't know why this would be removing xen/pmem from these files. If I run 'lcitool-refresh' on current git master that doesn't happen

Do you have any other local changes on top ?

No...

$ git checkout origin/master
((887cba855b...))$ git submodule sync
Synchronizing submodule url for 'roms/QemuMacDrivers'
Synchronizing submodule url for 'roms/SLOF'
Synchronizing submodule url for 'roms/edk2'
Synchronizing submodule url for 'roms/ipxe'
Synchronizing submodule url for 'roms/openbios'
Synchronizing submodule url for 'roms/opensbi'
Synchronizing submodule url for 'roms/qboot'
Synchronizing submodule url for 'roms/qemu-palcode'
Synchronizing submodule url for 'roms/seabios'
Synchronizing submodule url for 'roms/seabios-hppa'
Synchronizing submodule url for 'roms/skiboot'
Synchronizing submodule url for 'roms/u-boot'
Synchronizing submodule url for 'roms/u-boot-sam460ex'
Synchronizing submodule url for 'roms/vbootrom'
Synchronizing submodule url for 'tests/lcitool/libvirt-ci'
((887cba855b...))$ git submodule update
((887cba855b...))$ git submodule status
 90c488d5f4a407342247b9ea869df1c2d9c8e266 roms/QemuMacDrivers 
(heads/master)
 6b6c16b4b40763507cf1f518096f3c3883c5cf2d roms/SLOF 
(qemu-slof-20190703-109-g6b6c16b)
 f80f052277c88a67c55e107b550f504eeea947d3 roms/edk2 
(edk2-stable201903-5245-gf80f052277)
 4bd064de239dab2426b31c9789a1f4d78087dc63 roms/ipxe (v1.20.1-70-g4bd064de)
 af97fd7af5e7c18f591a7b987291d3db4ffb28b5 roms/openbios 
(remotes/origin/HEAD)
 2552799a1df30a3dcd2321a8b75d61d06f5fb9fc roms/opensbi (v1.3)
 8ca302e86d685fa05b16e2b208888243da319941 roms/qboot (remotes/origin/HEAD)
 99d9b4dcf27d7fbcbadab71bdc88ef6531baf6bf roms/qemu-palcode (heads/master)
 ea1b7a0733906b8425d948ae94fba63c32b1d425 roms/seabios (rel-1.16.2)
 673d2595d4f773cc266cbf8dbaf2f475a6adb949 roms/seabios-hppa 
(seabios-hppa-v8)
 24a7eb35966d93455520bc2debdd7954314b638b roms/skiboot (v7.0)
 840658b093976390e9537724f802281c9c8439f5 roms/u-boot (v2021.07)
 60b3916f33e617a815973c5a6df77055b2e3a588 roms/u-boot-sam460ex 
(heads/master)
 0c37a43527f0ee2b9584e7fb2fdc805e902635ac roms/vbootrom (heads/master)
 b0f44f929a81c0a604fb7fbf8afc34d37ab0eae9 tests/lcitool/libvirt-ci 
(heads/master-555-gb0f44f92)
((887cba855b...))$ make -C build lcitool-refresh
Generate tests/docker/dockerfiles/alpine.docker
Generate tests/docker/dockerfiles/centos8.docker
Generate tests/docker/dockerfiles/debian-amd64.docker
Generate tests/docker/dockerfiles/fedora.docker
Generate tests/docker/dockerfiles/opensuse-leap.docker
Generate tests/docker/dockerfiles/ubuntu2004.docker
Generate tests/docker/dockerfiles/ubuntu2204.docker
Generate tests/docker/dockerfiles/debian-amd64-cross.docker
Generate tests/docker/dockerfiles/debian-arm64-cross.docker
Generate tests/docker/dockerfiles/debian-armel-cross.docker
Generate tests/docker/dockerfiles/debian-armhf-cross.docker
Generate tests/docker/dockerfiles/debian-mips64el-cross.docker
Generate tests/docker/dockerfiles/debian-mipsel-cross.docker
Generate tests/docker/dockerfiles/debian-ppc64el-cross.docker
Generate tests/docker/dockerfiles/debian-riscv64-cross.docker
Generate tests/docker/dockerfiles/debian-s390x-cross.docker
Generate tests/docker/dockerfiles/fedora-win32-cross.docker
Generate tests/docker/dockerfiles/fedora-win64-cross.docker
Generate .gitlab-ci.d/cirrus/freebsd-13.vars
Generate .gitlab-ci.d/cirrus/macos-12.vars
((887cba855b...))$ git diff -U0
tests/docker/dockerfiles/debian-amd64.docker +0 -2
index e39871c7bb..8f7521fdc2 100644
@@ -73 +72,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -99 +97,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \
tests/docker/dockerfiles/ubuntu2004.docker +0 -2
index 8f864d19e6..7f60143cbb 100644
@@ -72 +71,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -97 +95,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \
tests/docker/dockerfiles/ubuntu2204.docker +0 -2
index 1d442cdfe6..5162927016 100644
@@ -73 +72,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -99 +97,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \

((887cba855b...))$

🤷

#9 lore raw

Re: [PATCH v3 2/4] tests/lcitool: Refresh generated files

Quoted 67 lines · Philippe Mathieu-Daudé
Quoted 11 lines · Daniel P. Berrangé
Quoted 9 lines · Philippe Mathieu-Daudé

Refresh the generated files by running:

   $ make lcitool-refresh
---
  tests/docker/dockerfiles/debian-amd64.docker |  2 -
  tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
  tests/docker/dockerfiles/ubuntu2204.docker   |  2 -
Quoted 4 lines

I don't know why this would be removing xen/pmem from these files. If I run 'lcitool-refresh' on current git master that doesn't happen

Do you have any other local changes on top ?

((887cba855b...))$ make -C build lcitool-refresh Generate tests/docker/dockerfiles/alpine.docker Generate tests/docker/dockerfiles/centos8.docker Generate tests/docker/dockerfiles/debian-amd64.docker Generate tests/docker/dockerfiles/fedora.docker Generate tests/docker/dockerfiles/opensuse-leap.docker Generate tests/docker/dockerfiles/ubuntu2004.docker Generate tests/docker/dockerfiles/ubuntu2204.docker Generate tests/docker/dockerfiles/debian-amd64-cross.docker Generate tests/docker/dockerfiles/debian-arm64-cross.docker Generate tests/docker/dockerfiles/debian-armel-cross.docker Generate tests/docker/dockerfiles/debian-armhf-cross.docker Generate tests/docker/dockerfiles/debian-mips64el-cross.docker Generate tests/docker/dockerfiles/debian-mipsel-cross.docker Generate tests/docker/dockerfiles/debian-ppc64el-cross.docker Generate tests/docker/dockerfiles/debian-riscv64-cross.docker Generate tests/docker/dockerfiles/debian-s390x-cross.docker Generate tests/docker/dockerfiles/fedora-win32-cross.docker Generate tests/docker/dockerfiles/fedora-win64-cross.docker Generate .gitlab-ci.d/cirrus/freebsd-13.vars Generate .gitlab-ci.d/cirrus/macos-12.vars ((887cba855b...))$ git diff -U0

tests/docker/dockerfiles/debian-amd64.docker +0 -2
index e39871c7bb..8f7521fdc2 100644
@@ -73 +72,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -99 +97,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \
tests/docker/dockerfiles/ubuntu2004.docker +0 -2
index 8f864d19e6..7f60143cbb 100644
@@ -72 +71,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -97 +95,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \
tests/docker/dockerfiles/ubuntu2204.docker +0 -1
index 1d442cdfe6..5162927016 100644
@@ -73 +72,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \

tests/docker/dockerfiles/ubuntu2204.docker RUN export DEBIAN_FRONTEND=noninteractive && \ · line 99

Quoted 1 line
tests/docker/dockerfiles/ubuntu2204.docker +0 -1
@@ -99 +97,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \

What architecture are you running from ? I'm suspecting it is a non x86_64 architecture as these pacakges are both arch conditionalized.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#10 lore raw

Re: [PATCH v3 2/4] tests/lcitool: Refresh generated files

Quoted 13 lines · Daniel P. Berrangé
Quoted 12 lines · Philippe Mathieu-Daudé
Quoted 11 lines · Daniel P. Berrangé
Quoted 9 lines · Philippe Mathieu-Daudé

Refresh the generated files by running:

    $ make lcitool-refresh
---
   tests/docker/dockerfiles/debian-amd64.docker |  2 -
   tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
   tests/docker/dockerfiles/ubuntu2204.docker   |  2 -
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I don't know why this would be removing xen/pmem from these files. If I run 'lcitool-refresh' on current git master that doesn't happen

Do you have any other local changes on top ?

(I just noticed manually updating the libvirt-ci submodule is
pointless because the 'lcitool-refresh' rule does that)

Quoted 4 lines
Quoted 4 lines
tests/docker/dockerfiles/ubuntu2204.docker +0 -1
index 1d442cdfe6..5162927016 100644
--- a/tests/docker/dockerfiles/ubuntu2204.docker
Quoted 8 lines

tests/docker/dockerfiles/ubuntu2204.docker RUN export DEBIAN_FRONTEND=noninteractive && \ · line 99

Quoted 5 lines

+++ b/tests/docker/dockerfiles/ubuntu2204.docker

tests/docker/dockerfiles/ubuntu2204.docker +0 -2
@@ -73 +72,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -99 +97,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \

What architecture are you running from ? I'm suspecting it is a non x86_64 architecture as these pacakges are both arch conditionalized.

My host is Darwin aarch64, but how is this relevant to what we generate for guests? Are we missing specifying the target arch somewhere? (This is not the '--cross-arch' argument, because we don't want to cross-build). These dockerfiles always targeted x86_64, in particular the debian-amd64.docker one...

#11 lore raw

Re: [PATCH v3 2/4] tests/lcitool: Refresh generated files

Quoted 35 lines · Philippe Mathieu-Daudé
Quoted 13 lines · Daniel P. Berrangé
Quoted 12 lines · Philippe Mathieu-Daudé
Quoted 11 lines · Daniel P. Berrangé
Quoted 9 lines · Philippe Mathieu-Daudé

Refresh the generated files by running:

    $ make lcitool-refresh
---
   tests/docker/dockerfiles/debian-amd64.docker |  2 -
   tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
   tests/docker/dockerfiles/ubuntu2204.docker   |  2 -
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I don't know why this would be removing xen/pmem from these files. If I run 'lcitool-refresh' on current git master that doesn't happen

Do you have any other local changes on top ?

(I just noticed manually updating the libvirt-ci submodule is
pointless because the 'lcitool-refresh' rule does that)

Quoted 4 lines
Quoted 4 lines
tests/docker/dockerfiles/ubuntu2204.docker +0 -1
index 1d442cdfe6..5162927016 100644
--- a/tests/docker/dockerfiles/ubuntu2204.docker
Quoted 8 lines

tests/docker/dockerfiles/ubuntu2204.docker RUN export DEBIAN_FRONTEND=noninteractive && \ · line 99

Quoted 5 lines

+++ b/tests/docker/dockerfiles/ubuntu2204.docker

tests/docker/dockerfiles/ubuntu2204.docker +0 -2
@@ -73 +72,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -99 +97,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \

What architecture are you running from ? I'm suspecting it is a non x86_64 architecture as these pacakges are both arch conditionalized.

Quoted 5 lines

My host is Darwin aarch64, but how is this relevant to what we generate for guests? Are we missing specifying the target arch somewhere? (This is not the '--cross-arch' argument, because we don't want to cross-build). These dockerfiles always targeted x86_64, in particular the debian-amd64.docker one...

This is an unfortunate design choice of lcitool.

If you give '--cross-arch' it will always spit out the dockerfile suitable for cross-compiling to that arch.

If you omit '--cross-arch' it will always spit out the dockerfile suitable for compiling on the *current* host arch.

When we're committing the dockerfiles to qemu (or any libvirt project using lcitool), we've got an implicit assumption that the non-cross dockerfiles were for x86_64, and hence an implicit assumption that the person who ran 'lcitool' was also on x86_64.

This is clearly a bad design choice mistake in retrospect as people are increasingly likely to be using aarch64 for day-to-day dev work.

So I guess we need to come up with a more explicit way to say give me a native container for x86_64.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#12 lore raw

Re: [PATCH v3 2/4] tests/lcitool: Refresh generated files

Quoted 9 lines · Daniel P. Berrangé
Quoted 8 lines · Philippe Mathieu-Daudé
Quoted 7 lines · Daniel P. Berrangé
Quoted 6 lines · Philippe Mathieu-Daudé
Quoted 5 lines · Daniel P. Berrangé
Quoted 4 lines · Philippe Mathieu-Daudé

Refresh the generated files by running:

     $ make lcitool-refresh
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
---
    tests/docker/dockerfiles/debian-amd64.docker |  2 -
    tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
    tests/docker/dockerfiles/ubuntu2204.docker   |  2 -

I don't know why this would be removing xen/pmem from these files. If I run 'lcitool-refresh' on current git master that doesn't happen

FTR since commit cb039ef3d9 libxdp-devel is also being changed on my host, similarly to libpmem-devel, so I suppose it also has some host specific restriction.

Quoted 45 lines
Quoted 18 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

Do you have any other local changes on top ?

(I just noticed manually updating the libvirt-ci submodule is
pointless because the 'lcitool-refresh' rule does that)

Quoted 4 lines
Quoted 4 lines
tests/docker/dockerfiles/ubuntu2204.docker +0 -1
index 1d442cdfe6..5162927016 100644
--- a/tests/docker/dockerfiles/ubuntu2204.docker
Quoted 8 lines

tests/docker/dockerfiles/ubuntu2204.docker RUN export DEBIAN_FRONTEND=noninteractive && \ · line 99

Quoted 5 lines

+++ b/tests/docker/dockerfiles/ubuntu2204.docker

tests/docker/dockerfiles/ubuntu2204.docker +0 -2
@@ -73 +72,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -99 +97,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \

What architecture are you running from ? I'm suspecting it is a non x86_64 architecture as these pacakges are both arch conditionalized.

Quoted 5 lines

My host is Darwin aarch64, but how is this relevant to what we generate for guests? Are we missing specifying the target arch somewhere? (This is not the '--cross-arch' argument, because we don't want to cross-build). These dockerfiles always targeted x86_64, in particular the debian-amd64.docker one...

This is an unfortunate design choice of lcitool.

If you give '--cross-arch' it will always spit out the dockerfile suitable for cross-compiling to that arch.

If you omit '--cross-arch' it will always spit out the dockerfile suitable for compiling on the *current* host arch.

When we're committing the dockerfiles to qemu (or any libvirt project using lcitool), we've got an implicit assumption that the non-cross dockerfiles were for x86_64, and hence an implicit assumption that the person who ran 'lcitool' was also on x86_64.

This is clearly a bad design choice mistake in retrospect as people are increasingly likely to be using aarch64 for day-to-day dev work.

So I guess we need to come up with a more explicit way to say give me a native container for x86_64.

With regards,
Daniel

#13 lore raw

Re: [PATCH v3 2/4] tests/lcitool: Refresh generated files

Quoted 16 lines · Philippe Mathieu-Daudé
Quoted 15 lines · Daniel P. Berrangé
Quoted 14 lines · Philippe Mathieu-Daudé
Quoted 13 lines · Daniel P. Berrangé
Quoted 12 lines · Philippe Mathieu-Daudé
Quoted 11 lines · Daniel P. Berrangé
Quoted 9 lines · Philippe Mathieu-Daudé

Refresh the generated files by running:

     $ make lcitool-refresh
---
    tests/docker/dockerfiles/debian-amd64.docker |  2 -
    tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
    tests/docker/dockerfiles/ubuntu2204.docker   |  2 -
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I don't know why this would be removing xen/pmem from these files. If I run 'lcitool-refresh' on current git master that doesn't happen

FTR since commit cb039ef3d9 libxdp-devel is also being changed on my host, similarly to libpmem-devel, so I suppose it also has some host specific restriction.

Yeah, many distributions are not building libxdp for non-x86 architectures today. There are no real reasons not to, but they just do not, because the package is relatively new. It will likely become available in the future.

I see this thread is a bit old. Was a solution found for the pmem/xen? i.e. do I need to do something specific for libxdp at this point?

Best regards, Ilya Maximets.

Quoted 47 lines
Quoted 46 lines
Quoted 19 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Do you have any other local changes on top ?

(I just noticed manually updating the libvirt-ci submodule is
pointless because the 'lcitool-refresh' rule does that)

Quoted 4 lines
Quoted 4 lines
tests/docker/dockerfiles/ubuntu2204.docker +0 -1
index 1d442cdfe6..5162927016 100644
--- a/tests/docker/dockerfiles/ubuntu2204.docker
Quoted 8 lines

tests/docker/dockerfiles/ubuntu2204.docker RUN export DEBIAN_FRONTEND=noninteractive && \ · line 99

Quoted 5 lines

+++ b/tests/docker/dockerfiles/ubuntu2204.docker

tests/docker/dockerfiles/ubuntu2204.docker +0 -2
@@ -73 +72,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -99 +97,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \

What architecture are you running from ? I'm suspecting it is a non x86_64 architecture as these pacakges are both arch conditionalized.

Quoted 5 lines

My host is Darwin aarch64, but how is this relevant to what we generate for guests? Are we missing specifying the target arch somewhere? (This is not the '--cross-arch' argument, because we don't want to cross-build). These dockerfiles always targeted x86_64, in particular the debian-amd64.docker one...

This is an unfortunate design choice of lcitool.

If you give '--cross-arch' it will always spit out the dockerfile suitable for cross-compiling to that arch.

If you omit '--cross-arch' it will always spit out the dockerfile suitable for compiling on the *current* host arch.

When we're committing the dockerfiles to qemu (or any libvirt project using lcitool), we've got an implicit assumption that the non-cross dockerfiles were for x86_64, and hence an implicit assumption that the person who ran 'lcitool' was also on x86_64.

This is clearly a bad design choice mistake in retrospect as people are increasingly likely to be using aarch64 for day-to-day dev work.

So I guess we need to come up with a more explicit way to say give me a native container for x86_64.

With regards,
Daniel

#14 lore raw

Re: [PATCH v3 2/4] tests/lcitool: Refresh generated files

Quoted 24 lines · Ilya Maximets
Quoted 16 lines · Philippe Mathieu-Daudé
Quoted 15 lines · Daniel P. Berrangé
Quoted 14 lines · Philippe Mathieu-Daudé
Quoted 13 lines · Daniel P. Berrangé
Quoted 12 lines · Philippe Mathieu-Daudé
Quoted 11 lines · Daniel P. Berrangé
Quoted 9 lines · Philippe Mathieu-Daudé

Refresh the generated files by running:

      $ make lcitool-refresh
---
     tests/docker/dockerfiles/debian-amd64.docker |  2 -
     tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
     tests/docker/dockerfiles/ubuntu2204.docker   |  2 -
Quoted 6 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

I don't know why this would be removing xen/pmem from these files. If I run 'lcitool-refresh' on current git master that doesn't happen

FTR since commit cb039ef3d9 libxdp-devel is also being changed on my host, similarly to libpmem-devel, so I suppose it also has some host specific restriction.

Quoted 8 lines

Yeah, many distributions are not building libxdp for non-x86 architectures today. There are no real reasons not to, but they just do not, because the package is relatively new. It will likely become available in the future.

I see this thread is a bit old. Was a solution found for the pmem/xen? i.e. do I need to do something specific for libxdp at this point?

Maybe this is tracked in this issue?
https://gitlab.com/libvirt/libvirt-ci/-/issues/30

Otherwise libvirt/lcitool maintainers are Cc'ed :)

Quoted 50 lines

Best regards, Ilya Maximets.

Quoted 47 lines
Quoted 46 lines
Quoted 19 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Do you have any other local changes on top ?

(I just noticed manually updating the libvirt-ci submodule is
pointless because the 'lcitool-refresh' rule does that)

Quoted 4 lines
Quoted 4 lines
tests/docker/dockerfiles/ubuntu2204.docker +0 -1
index 1d442cdfe6..5162927016 100644
--- a/tests/docker/dockerfiles/ubuntu2204.docker
Quoted 8 lines

tests/docker/dockerfiles/ubuntu2204.docker RUN export DEBIAN_FRONTEND=noninteractive && \ · line 99

Quoted 5 lines

+++ b/tests/docker/dockerfiles/ubuntu2204.docker

tests/docker/dockerfiles/ubuntu2204.docker +0 -2
@@ -73 +72,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libpmem-dev \
@@ -99 +97,0 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
-                      libxen-dev \

What architecture are you running from ? I'm suspecting it is a non x86_64 architecture as these pacakges are both arch conditionalized.

Quoted 5 lines

My host is Darwin aarch64, but how is this relevant to what we generate for guests? Are we missing specifying the target arch somewhere? (This is not the '--cross-arch' argument, because we don't want to cross-build). These dockerfiles always targeted x86_64, in particular the debian-amd64.docker one...

This is an unfortunate design choice of lcitool.

If you give '--cross-arch' it will always spit out the dockerfile suitable for cross-compiling to that arch.

If you omit '--cross-arch' it will always spit out the dockerfile suitable for compiling on the *current* host arch.

When we're committing the dockerfiles to qemu (or any libvirt project using lcitool), we've got an implicit assumption that the non-cross dockerfiles were for x86_64, and hence an implicit assumption that the person who ran 'lcitool' was also on x86_64.

This is clearly a bad design choice mistake in retrospect as people are increasingly likely to be using aarch64 for day-to-day dev work.

So I guess we need to come up with a more explicit way to say give me a native container for x86_64.

With regards,
Daniel

#15 lore raw

PATCH v3 3/4 tests/vm: Introduce get_qemu_packages_from_lcitool_json() helper

In reply to Philippe Mathieu-Daudé

Add the get_qemu_packages_from_lcitool_json() helper which return such package list from a lcitool env var file in JSON format.

---
 tests/vm/basevm.py | 11 +++++++++++
 1 file changed, 11 insertions(+)
tests/vm/basevm.py +11 -0
index 23229e23d1..a97e23b0ce 100644
@@ -27,6 +27,7 @@
 import multiprocessing
 import traceback
 import shlex
+import json
 
 from qemu.machine import QEMUMachine
 from qemu.utils import get_info_usernet_hostfwd_port, kvm_available
@@ -501,6 +502,16 @@ def gen_cloud_init_iso(self):
                               stderr=self._stdout)
         return os.path.join(cidir, "cloud-init.iso")
 
+    def get_qemu_packages_from_lcitool_json(self, json_path=None):
+        """Parse a lcitool variables json file and return the PKGS list."""
+        if json_path is None:
+            json_path = os.path.join(
+                os.path.dirname(__file__), "generated", self.name + ".json"
+            )
+        with open(json_path, "r") as fh:
+            return json.load(fh)["pkgs"]
+
+
 def get_qemu_path(arch, build_path=None):
     """Fetch the path to the qemu binary."""
     # If QEMU environment variable set, it takes precedence
Signature · 1 line
2.38.1
Reviewed-by on v3 3/4
#16 lore raw

Re: [PATCH v3 3/4] tests/vm: Introduce get_qemu_packages_from_lcitool_json() helper

Quoted 8 lines · Philippe Mathieu-Daudé

Add the get_qemu_packages_from_lcitool_json() helper which return such package list from a lcitool env var file in JSON format.

---
 tests/vm/basevm.py | 11 +++++++++++
 1 file changed, 11 insertions(+)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#17 lore raw

PATCH v3 4/4 tests/vm/freebsd: Get up-to-date package list from lcitool vars file

In reply to Philippe Mathieu-Daudé

Get an up-to-date package list from lcitool, that way we don't need to manually keep this array in sync.

---
 tests/vm/freebsd | 42 ++----------------------------------------
 1 file changed, 2 insertions(+), 40 deletions(-)
tests/vm/freebsd +2 -40
index 11de6473f4..ac51376c82 100755
@@ -31,45 +31,6 @@ class FreeBSDVM(basevm.BaseVM):
     link = "https://download.freebsd.org/releases/CI-IMAGES/13.2-RELEASE/amd64/Latest/FreeBSD-13.2-RELEASE-amd64-BASIC-CI.raw.xz"
     csum = "a4fb3b6c7b75dd4d58fb0d75e4caf72844bffe0ca00e66459c028b198ffb3c0e"
     size = "20G"
-    pkgs = [
-        # build tools
-        "git",
-        "pkgconf",
-        "bzip2",
-        "python39",
-        "ninja",
-
-        # gnu tools
-        "bash",
-        "gmake",
-        "gsed",
-        "gettext",
-
-        # libs: crypto
-        "gnutls",
-
-        # libs: images
-        "jpeg-turbo",
-        "png",
-
-        # libs: ui
-        "sdl2",
-        "gtk3",
-        "libxkbcommon",
-
-        # libs: opengl
-        "libepoxy",
-        "mesa-libs",
-
-        # libs: migration
-        "zstd",
-
-        # libs: networking
-        "libslirp",
-
-        # libs: sndio
-        "sndio",
-    ]
 
     BUILD_SCRIPT = """
         set -e;
@@ -151,8 +112,9 @@ class FreeBSDVM(basevm.BaseVM):
         self.console_wait(prompt)
         self.console_send("echo 'chmod 666 /dev/vtbd1' >> /etc/rc.local\n")
 
+        pkgs = self.get_qemu_packages_from_lcitool_json()
         self.print_step("Installing packages")
-        self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs))
+        self.ssh_root_check("pkg install -y %s\n" % " ".join(pkgs))
 
         # shutdown
         self.ssh_root(self.poweroff)
Signature · 1 line
2.38.1
Reviewed-by on v3 4/4
#18 lore raw

Re: [PATCH v3 4/4] tests/vm/freebsd: Get up-to-date package list from lcitool vars file

Quoted 8 lines · Philippe Mathieu-Daudé

Get an up-to-date package list from lcitool, that way we don't need to manually keep this array in sync.

---
 tests/vm/freebsd | 42 ++----------------------------------------
 1 file changed, 2 insertions(+), 40 deletions(-)

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#19 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 17 lines · Philippe Mathieu-Daudé

Inspired by this patch from Thomas:
https://lore.kernel.org/qemu-devel/[email protected]/

Instead of updating the package list manually, use lcitool vars file.

Since v2:
- Commit generated json (Daniel)

Since v1:
- Addressed Erik & Daniel comments (generate in JSON)

Philippe Mathieu-Daudé (4):
tests/lcitool: Generate distribution packages list in JSON format tests/lcitool: Refresh generated files
tests/vm: Introduce get_qemu_packages_from_lcitool_json() helper tests/vm/freebsd: Get up-to-date package list from lcitool vars file

Quoted 3 lines
  tests/docker/dockerfiles/debian-amd64.docker |  2 -
  tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
  tests/docker/dockerfiles/ubuntu2204.docker   |  2 -

Thanks, I'm queuing your patches for my next pull request - but I'm dropping the hunks that change libpmem-dev and libxen-dev in the docker files - otherwise this would just get reverted once somebody using a x86 host runs the refresh step again. I hope this issue can be tackled by a future lcitool update instead.

Thomas

#20 lore raw
Quoted 22 lines · Thomas Huth
Quoted 17 lines · Philippe Mathieu-Daudé

Inspired by this patch from Thomas:
https://lore.kernel.org/qemu-devel/[email protected]/

Instead of updating the package list manually, use lcitool vars file.

Since v2:
- Commit generated json (Daniel)

Since v1:
- Addressed Erik & Daniel comments (generate in JSON)

Philippe Mathieu-Daudé (4):
tests/lcitool: Generate distribution packages list in JSON format tests/lcitool: Refresh generated files
tests/vm: Introduce get_qemu_packages_from_lcitool_json() helper tests/vm/freebsd: Get up-to-date package list from lcitool vars file

Quoted 3 lines
  tests/docker/dockerfiles/debian-amd64.docker |  2 -
  tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
  tests/docker/dockerfiles/ubuntu2204.docker   |  2 -
Quoted 8 lines

Thanks, I'm queuing your patches for my next pull request - but I'm dropping the hunks that change libpmem-dev and libxen-dev in the docker files - otherwise this would just get reverted once somebody using a x86 host runs the refresh step again. I hope this issue can be tackled by a future lcitool update instead.

Thomas

Can you guys please create an issue at
https://gitlab.com/libvirt/libvirt-ci.git to have this officially tracked?

Thanks,
Erik

#21 lore raw
Quoted 29 lines · Erik Skultety
Quoted 28 lines · Thomas Huth
Quoted 17 lines · Philippe Mathieu-Daudé

Inspired by this patch from Thomas:
https://lore.kernel.org/qemu-devel/[email protected]/

Instead of updating the package list manually, use lcitool vars file.

Since v2:
- Commit generated json (Daniel)

Since v1:
- Addressed Erik & Daniel comments (generate in JSON)

Philippe Mathieu-Daudé (4):
tests/lcitool: Generate distribution packages list in JSON format tests/lcitool: Refresh generated files
tests/vm: Introduce get_qemu_packages_from_lcitool_json() helper tests/vm/freebsd: Get up-to-date package list from lcitool vars file

Quoted 3 lines
  tests/docker/dockerfiles/debian-amd64.docker |  2 -
  tests/docker/dockerfiles/ubuntu2004.docker   |  2 -
  tests/docker/dockerfiles/ubuntu2204.docker   |  2 -

Thanks, I'm queuing your patches for my next pull request - but I'm dropping the hunks that change libpmem-dev and libxen-dev in the docker files - otherwise this would just get reverted once somebody using a x86 host runs the refresh step again. I hope this issue can be tackled by a future lcitool update instead.

Quoted 5 lines
Quoted 2 lines

Thomas

Can you guys please create an issue at
https://gitlab.com/libvirt/libvirt-ci.git to have this officially tracked?

Done:

https://gitlab.com/libvirt/libvirt-ci/-/issues/26

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|