Skip to contentSkip to conversation

osstest: Fix libvirt build

PATCH In review Anthony PERARD 2 participants 6 messages Aug 2, 2022

lore
Share & tools
lore
#0 lore raw

cover osstest: Fix libvirt build

libvirt now use meson+ninja instead of autoconf, so teach osstest to use that.

I ran a flight with that, here is the result:
"tolerable all pass"
http://logs.test-lab.xenproject.org/osstest/logs/172068/

Thanks,

Anthony PERARD (2):
TestSupport: Add support for installing from backport repo
ts-libvirt-build: Rework to build with meson

 Osstest/TestSupport.pm | 27 ++++++++++++++--
 ts-libvirt-build       | 70 +++++++++++++++++++++++++++++-------------
 ts-xen-build-prep      |  8 ++++-
 3 files changed, 81 insertions(+), 24 deletions(-)
Signature · 1 line
Anthony PERARD
#1 lore raw

PATCH 1/2 TestSupport: Add support for installing from backport repo

We are going to need to install package from the debian backport repository in order to do a build.

---
 Osstest/TestSupport.pm | 27 +++++++++++++++++++++++++--
 1 file changed, 25 insertions(+), 2 deletions(-)
Osstest/TestSupport.pm +25 -2
index 156cac799d58..91b0a7ab23a1 100644
@@ -73,6 +73,7 @@ BEGIN {
                       target_run_pkgmanager_install
                       target_install_packages target_install_packages_norec
 		      target_install_packages_nonfree_nonconcurrent
+                      target_install_packages_backport
                       target_jobdir target_extract_jobdistpath_subdir
                       target_extract_jobdistpath target_extract_distpart
 		      target_tftp_prefix
@@ -650,8 +651,8 @@ sub target_putfile_root ($$$$;$) {
     tputfileex('root', @_);
 }
 
-sub target_run_pkgmanager_install ($$;$$) {
-    my ($ho, $packagelist, $norec, $force) = @_;
+sub target_run_pkgmanager_install ($$;$$$) {
+    my ($ho, $packagelist, $norec, $force, $backport) = @_;
     my @cmd;
     if ($ho->{OS} eq "freebsd") {
         push @cmd, qw(lockf /var/run/osstest-pkg-lock pkg-static install);
@@ -660,6 +661,7 @@ sub target_run_pkgmanager_install ($$;$$) {
                       with-lock-ex -w /var/lock/osstest-apt apt-get);
 	push @cmd, qw(-f) if $force;
 	push @cmd, qw(--no-install-recommends) if $norec;
+	push @cmd, "-t", "$ho->{Suite}-backports" if $backport;
 	push @cmd, qw(-y install);
     }
     push @cmd, @$packagelist;
@@ -688,6 +690,27 @@ END
     apt-get update
 END
 }
+sub target_install_packages_backport ($@) {
+    my ($ho, @packages) = @_;
+    my $had_backport_repo = 0;
+    target_editfile_root($ho, '/etc/apt/sources.list', sub {
+        my $suite = $ho->{Suite};
+        my $bp_url = Osstest::Debian::debian_mirror_url($ho);
+        while (<::EI>) {
+            if (m/^# $suite backports/) {
+                $had_backport_repo = 1;
+            }
+            print ::EO;
+        }
+        print ::EO <<EOF unless $had_backport_repo;
+
+# $suite backports
+deb $bp_url $suite-backports main
+EOF
+        });
+    target_cmd_root($ho, "apt-get update", 300) unless $had_backport_repo;
+    target_run_pkgmanager_install($ho,\@packages,0,0,1);
+}
 
 sub tpfcs_core {
     my ($tputfilef,$ho,$timeout,$filedata, $rdest,$lleaf) = @_;
Signature · 1 line
Anthony PERARD
Acked-by on v1 1/2
#2 lore raw

Re: [OSSTEST PATCH 1/2] TestSupport: Add support for installing from backport repo

Anthony PERARD writes ("[OSSTEST PATCH 1/2] TestSupport: Add support for installing from backport repo"):

Quoted 2 lines

We are going to need to install package from the debian backport repository in order to do a build.

LGTM. I was kind of surprised we didn't have this already, but now I come to think of it, I think that was only for kernels ? (I haven't UTSL to check.)

Quoted 8 lines
Osstest/TestSupport.pm +8 -0
+sub target_install_packages_backport ($@) {
+    my ($ho, @packages) = @_;
+    my $had_backport_repo = 0;
+    target_editfile_root($ho, '/etc/apt/sources.list', sub {
+        my $suite = $ho->{Suite};
+        my $bp_url = Osstest::Debian::debian_mirror_url($ho);
+        while (<::EI>) {
+            if (m/^# $suite backports/) {

Osstest/TestSupport.pm END · line 710

Quoted 10 lines
Osstest/TestSupport.pm +10 -0
+                $had_backport_repo = 1;
+            }
+            print ::EO;
+        }
+        print ::EO <<EOF unless $had_backport_repo;
+
+# $suite backports
+deb $bp_url $suite-backports main
+EOF
+        });

Nit: this last line looks misindented to me.

Regardless,

Thanks,
Ian.

Signature · 4 lines
Ian Jackson <[email protected]>   These opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.
#3 lore raw

Re: [OSSTEST PATCH 1/2] TestSupport: Add support for installing from backport repo

Quoted 7 lines · Ian Jackson

Anthony PERARD writes ("[OSSTEST PATCH 1/2] TestSupport: Add support for installing from backport repo"):

Quoted 2 lines

We are going to need to install package from the debian backport repository in order to do a build.

LGTM. I was kind of surprised we didn't have this already, but now I come to think of it, I think that was only for kernels ? (I haven't UTSL to check.)

Yes, there's preseed_backports_packages() to install from backports repo at host/guest installation. But nothing once the OS is already installed. I thought it would not be a good idea to install meson on every host.

Quoted 12 lines
Quoted 12 lines
Osstest/TestSupport.pm +12 -0
+sub target_install_packages_backport ($@) {
+    my ($ho, @packages) = @_;
+    my $had_backport_repo = 0;
+    target_editfile_root($ho, '/etc/apt/sources.list', sub {
+        my $suite = $ho->{Suite};
+        my $bp_url = Osstest::Debian::debian_mirror_url($ho);
+        while (<::EI>) {
+            if (m/^# $suite backports/) {
+                $had_backport_repo = 1;
+            }
+            print ::EO;
+        }
Quoted 8 lines

Osstest/TestSupport.pm END · line 710

Quoted 6 lines
Osstest/TestSupport.pm +6 -0
+        print ::EO <<EOF unless $had_backport_repo;
+
+# $suite backports
+deb $bp_url $suite-backports main
+EOF
+        });

Nit: this last line looks misindented to me.

Would be nice to fix on commit, I guess.

Quoted 1 line

Thanks,

Signature · 1 line
Anthony PERARD
#4 lore raw

PATCH 2/2 ts-libvirt-build: Rework to build with meson

In reply to Anthony PERARD

libvirt have switch build system to Meson + Ninja, there is no more autogen.sh script.

Add missing dependency "libtirpc-dev" and "meson".

libvirt needs at least meson 0.54, the version avaiable in Debian Buster is too old, but we can install it from backports.

In order to find out if Meson is used or not, we'll check if the "autogen.sh" script is missing. And in the build dir, we will look for a "Makefile".

---
 ts-libvirt-build  | 70 +++++++++++++++++++++++++++++++++--------------
 ts-xen-build-prep |  8 +++++-
 2 files changed, 56 insertions(+), 22 deletions(-)
ts-libvirt-build +49 -21
index e799f00375f7..16b45cfd02bf 100755
@@ -50,32 +50,56 @@ sub config() {
     }
     die "no xen prefix" unless $xenprefix;
 
-    # Uses --no-git because otherwise autogen.sh will undo
-    # submodulefixup's attempts to honour
-    # revision_libvirt_gnulib. This in turn requires that we specify
-    # --gnulib-srcdir, but ./autogen.sh doesn't propagate
-    # --gnulib-srcdir to ./bootstap so we use GNULIB_SRCDIR directly.
-    my $gnulib = submodule_find($submodules, "gnulib");
-    target_cmd_build($ho, 3600, $builddir, <<END);
-        cd libvirt
-	mkdir build
-	cd build
-        CFLAGS="-g -I$xenprefix/include/" \\
-        LDFLAGS="-g -L$xenprefix/lib/ -Wl,-rpath-link=$xenprefix/lib/" \\
-        PKG_CONFIG_PATH="$xenprefix/lib/pkgconfig/" \\
-        GNULIB_SRCDIR=$builddir/libvirt/$gnulib->{Path} \\
-            ../autogen.sh --no-git \\
-                         --with-libxl --without-xen --without-xenapi --without-selinux \\
-                         --without-lxc --without-vbox --without-uml \\
-                         --without-qemu --without-openvz --without-vmware \\
-                         --sysconfdir=/etc --localstatedir=/var #/
+    my $cflags = "-g -I$xenprefix/include/";
+    my $ldflags = "-g -L$xenprefix/lib/ -Wl,-rpath-link=$xenprefix/lib/";
+    my $pkg_config_path = "$xenprefix/lib/pkgconfig/";
+
+    if (target_file_exists($ho, "$builddir/libvirt/autogen.sh")) {
+        # Uses --no-git because otherwise autogen.sh will undo
+        # submodulefixup's attempts to honour
+        # revision_libvirt_gnulib. This in turn requires that we specify
+        # --gnulib-srcdir, but ./autogen.sh doesn't propagate
+        # --gnulib-srcdir to ./bootstap so we use GNULIB_SRCDIR directly.
+        my $gnulib = submodule_find($submodules, "gnulib");
+        target_cmd_build($ho, 3600, $builddir, <<END);
+            cd libvirt
+            mkdir build
+            cd build
+            CFLAGS="$cflags" \\
+            LDFLAGS="$ldflags" \\
+            PKG_CONFIG_PATH="$pkg_config_path" \\
+            GNULIB_SRCDIR=$builddir/libvirt/$gnulib->{Path} \\
+                ../autogen.sh --no-git \\
+                             --with-libxl --without-xen --without-xenapi --without-selinux \\
+                             --without-lxc --without-vbox --without-uml \\
+                             --without-qemu --without-openvz --without-vmware \\
+                             --sysconfdir=/etc --localstatedir=/var #/
+END
+    } else {
+        target_cmd_build($ho, 3600, $builddir, <<END);
+            cd libvirt
+            meson setup \\
+                -Dc_args='$cflags' \\
+                -Dc_link_args='$ldflags' \\
+                --pkg-config-path='$pkg_config_path' \\
+                --auto-features=disabled \\
+                -Ddriver_libxl=enabled \\
+                -Ddriver_libvirtd=enabled \\
+                -Ddriver_remote=enabled \\
+                --sysconfdir=/etc --localstatedir=/var \\
+                build
 END
+    }
 }
 
 sub build() {
     target_cmd_build($ho, 3600, $builddir, <<END);
         cd libvirt/build
-        (make $makeflags 2>&1 && touch ../../build-ok-stamp) |tee ../log
+        if [ -e Makefile ]; then
+            (make $makeflags 2>&1 && touch ../../build-ok-stamp) |tee ../log
+        else
+            (ninja 2>&1 && touch ../../build-ok-stamp) |tee ../log
+        fi
         test -f ../../build-ok-stamp #/
         echo ok.
 END
@@ -85,7 +109,11 @@ sub install() {
     target_cmd_build($ho, 300, $builddir, <<END);
         mkdir -p dist
         cd libvirt/build
-        make $makeflags install DESTDIR=$builddir/dist
+        if [ -e Makefile ]; then
+            make $makeflags install DESTDIR=$builddir/dist
+        else
+            DESTDIR=$builddir/dist ninja install
+        fi
         mkdir -p $builddir/dist/etc/init.d
 END
     target_putfilecontents_stash($ho, 60,
ts-xen-build-prep +7 -1
index e76c0254bfd0..3ae8f215440b 100755
@@ -199,7 +199,7 @@ END
 sub prep () {
     my @packages = qw(mercurial rsync figlet
                       build-essential bin86 bcc iasl bc
-                      flex bison cmake ninja-build
+                      flex bison cmake ninja-build meson
                       libpci-dev libncurses5-dev libssl-dev python-dev
                       libx11-dev git-core uuid-dev gettext gawk
                       libsdl-dev libyajl-dev libaio-dev libpixman-1-dev
@@ -209,6 +209,7 @@ sub prep () {
                       libdevmapper-dev libxml-xpath-perl libelf-dev
                       ccache nasm checkpolicy ebtables
 		      python3-docutils python3-dev
+                      libtirpc-dev
                       libgnutls28-dev);
 
     if ($ho->{Suite} =~ m/squeeze|wheezy|jessie/) {
@@ -251,6 +252,11 @@ END
         target_install_packages($ho, 'libc6-dev-i386');
     }
 
+    if ($ho->{Suite} =~ m/buster/) {
+        # libvirt needs a newer version of "meson"
+        target_install_packages_backport($ho, 'meson');
+    }
+
     some_extradebs($ho, [ 'DebianExtraPackages', 'build', $ho->{Suite} ]);
 }
 
Signature · 1 line
Anthony PERARD
Acked-by on v1 2/2
#5 lore raw

Re: [OSSTEST PATCH 2/2] ts-libvirt-build: Rework to build with meson

Anthony PERARD writes ("[OSSTEST PATCH 2/2] ts-libvirt-build: Rework to build with meson"):

Quoted 7 lines

libvirt have switch build system to Meson + Ninja, there is no more autogen.sh script.

Add missing dependency "libtirpc-dev" and "meson".

libvirt needs at least meson 0.54, the version avaiable in Debian Buster is too old, but we can install it from backports.

How annoying.

Quoted 3 lines

In order to find out if Meson is used or not, we'll check if the "autogen.sh" script is missing. And in the build dir, we will look for a "Makefile".

That seems right to me.

ts-xen-build-prep END · line 202

Quoted 3 lines

build-essential bin86 bcc iasl bc

ts-xen-build-prep +1 -1
-                      flex bison cmake ninja-build
+                      flex bison cmake ninja-build meson

*Some* version of meson is in even stretch so this is fine.

Signature · 4 lines
Ian Jackson <[email protected]>   These opinions are my own.  

Pronouns: they/he.  If I emailed you from @fyvzl.net or @evade.org.uk,
that is a private address which bypasses my fierce spamfilter.