Skip to contentSkip to conversation

tests/vm: Allow to set path to qemu-img

PATCH In review Wainer dos Santos Moschetta 3 participants 4 messages Nov 14, 2019 – Dec 3, 2019

lore
Share & tools
lore
#0 lore raw

cover tests/vm: Allow to set path to qemu-img

Usually I build QEMU in a system which doesn't have any qemu package installed. When I need to make vm-build-something it fails due system-qemu-<arch> and qemu-img not found system-wide (and I want to use the built binaries). The formed can be set with QEMU env var, but the later can't. So this patch allows to set the path to qemu-img using yet another env var.

Git:
- Tree: https://github.com/wainersm/qemu
- Branch: https://github.com/wainersm/qemu/tree/test_vm_qemu-img

Travis:
- (FAIL) https://travis-ci.org/wainersm/qemu/builds/611563267
Two build jobs reached timeout. Not related with this change though.

Bug found: I realized that when I export QEMU it is always overwritten
with <BUILD>/qemu/xtensa-linux-user/qemu-xtensa
See the error below.
---
[root@localhost qemu]# export QEMU_IMG=$PWD/qemu-img
[root@localhost qemu]# export QEMU=$PWD/x86_64-softmmu/qemu-system-x86_64
[root@localhost qemu]# make vm-build-fedora
VM-IMAGE fedora
### Downloading install iso ...
### Preparing iso and disk image ...
Formatting '/root/.cache/qemu-vm/images/fedora.img.tmp', fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off refcount_bits=16
### Booting installer ...
ERROR:root:Failed to launch QEMU, command line:
ERROR:root:/root/build/qemu/xtensa-linux-user/qemu-xtensa -nodefaults -m 4G -cpu max -netdev user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 8 -enable-kvm -device VGA -drive file=/root/.cache/qemu-vm/images/fedora.img.tmp,if=none,id=drive0,cache=writeback -device virtio-blk,drive=drive0,bootindex=0 -bios pc-bios/bios-256k.bin -machine graphics=off -cdrom /root/.cache/qemu-vm/images/fedora.img.install.iso ERROR:root:Log:
ERROR:root:qemu: unknown option 'chardev'
---

Wainer dos Santos Moschetta (1):
tests/vm: Allow to set qemu-img path

 docs/devel/testing.rst    | 6 ++++--
 tests/vm/Makefile.include | 1 +
 tests/vm/basevm.py        | 5 +++++
 tests/vm/centos           | 2 +-
 tests/vm/fedora           | 4 +---
 tests/vm/freebsd          | 3 +--
 tests/vm/netbsd           | 3 +--
 tests/vm/openbsd          | 3 +--
 tests/vm/ubuntu.i386      | 2 +-
 9 files changed, 16 insertions(+), 13 deletions(-)
Signature · 1 line
2.18.1
#1 lore raw

PATCH 1/1 tests/vm: Allow to set qemu-img path

By default VM build test use qemu-img from system's PATH to create the image disk. Due the lack of qemu-img on the system or the desire to simply use a version built with QEMU, it would be nice to allow one to set its path. So this patch makes that possible by reading the path to qemu-img from QEMU_IMG if set, otherwise it fallback to default behavior.

---
 docs/devel/testing.rst    | 6 ++++--
 tests/vm/Makefile.include | 1 +
 tests/vm/basevm.py        | 5 +++++
 tests/vm/centos           | 2 +-
 tests/vm/fedora           | 4 +---
 tests/vm/freebsd          | 3 +--
 tests/vm/netbsd           | 3 +--
 tests/vm/openbsd          | 3 +--
 tests/vm/ubuntu.i386      | 2 +-
 9 files changed, 16 insertions(+), 13 deletions(-)
docs/devel/testing.rst +4 -2
index 8e981e062d..9be6cd4410 100644
@@ -418,13 +418,15 @@ access, so they SHOULD NOT be exposed to external interfaces if you are
 concerned about attackers taking control of the guest and potentially
 exploiting a QEMU security bug to compromise the host.
 
-QEMU binary
------------
+QEMU binaries
+-------------
 
 By default, qemu-system-x86_64 is searched in $PATH to run the guest. If there
 isn't one, or if it is older than 2.10, the test won't work. In this case,
 provide the QEMU binary in env var: ``QEMU=/path/to/qemu-2.10+``.
 
+Likewise the path to qemu-img can be set in QEMU_IMG environment variable.
+
 Make jobs
 ---------
 
tests/vm/Makefile.include +1 -0
index fea348e845..9e7c46a473 100644
@@ -34,6 +34,7 @@ vm-help vm-test:
 	@echo "    DEBUG=1              	 - Enable verbose output on host and interactive debugging"
 	@echo "    V=1				 - Enable verbose ouput on host and guest commands"
 	@echo "    QEMU=/path/to/qemu		 - Change path to QEMU binary"
+	@echo "    QEMU_IMG=/path/to/qemu-img	 - Change path to qemu-img tool"
 
 vm-build-all: $(addprefix vm-build-, $(IMAGES))
 
tests/vm/basevm.py +5 -0
index 91a9226026..d1efeb3646 100755
@@ -152,6 +152,11 @@ class BaseVM(object):
     def build_image(self, img):
         raise NotImplementedError
 
+    def exec_qemu_img(self, *args):
+        cmd = [os.environ.get("QEMU_IMG", "qemu-img")]
+        cmd.extend(list(args))
+        subprocess.check_call(cmd)
+
     def add_source_dir(self, src_dir):
         name = "data-" + hashlib.sha1(src_dir.encode("utf-8")).hexdigest()[:5]
         tarfile = os.path.join(self._tmpdir, name + ".tar")
tests/vm/centos +1 -1
index 53976f1c4c..eac07dacd6 100755
@@ -68,7 +68,7 @@ class CentosVM(basevm.BaseVM):
         sys.stderr.write("Extracting the image...\n")
         subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
         subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
-        subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+        self.exec_qemu_img("resize", img_tmp, "50G")
         self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
         self.wait_ssh()
         self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
tests/vm/fedora +1 -3
index 7fec1479fb..8e270fc0f0 100755
@@ -74,9 +74,7 @@ class FedoraVM(basevm.BaseVM):
 
         self.print_step("Preparing iso and disk image")
         subprocess.check_call(["cp", "-f", cimg, iso])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
-
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
         self.print_step("Booting installer")
         self.boot(img_tmp, extra_args = [
             "-bios", "pc-bios/bios-256k.bin",
tests/vm/freebsd +1 -2
index 2a19461a90..1825cc5821 100755
@@ -82,8 +82,7 @@ class FreeBSDVM(basevm.BaseVM):
         self.print_step("Preparing iso and disk image")
         subprocess.check_call(["cp", "-f", cimg, iso_xz])
         subprocess.check_call(["xz", "-dvf", iso_xz])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
 
         self.print_step("Booting installer")
         self.boot(img_tmp, extra_args = [
tests/vm/netbsd +1 -2
index 611e6cc5b5..ec6f3563b2 100755
@@ -77,8 +77,7 @@ class NetBSDVM(basevm.BaseVM):
 
         self.print_step("Preparing iso and disk image")
         subprocess.check_call(["ln", "-f", cimg, iso])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
 
         self.print_step("Booting installer")
         self.boot(img_tmp, extra_args = [
tests/vm/openbsd +1 -2
index b92c39f89a..6df5162dbf 100755
@@ -73,8 +73,7 @@ class OpenBSDVM(basevm.BaseVM):
 
         self.print_step("Preparing iso and disk image")
         subprocess.check_call(["cp", "-f", cimg, iso])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
 
         self.print_step("Booting installer")
         self.boot(img_tmp, extra_args = [
tests/vm/ubuntu.i386 +1 -1
index 38f740eabf..39bbe9cc21 100755
@@ -70,7 +70,7 @@ class UbuntuX86VM(basevm.BaseVM):
             sha256sum="e30091144c73483822b7c27193e9d47346dd1064229da577c3fedcf943f7cfcc")
         img_tmp = img + ".tmp"
         subprocess.check_call(["cp", "-f", cimg, img_tmp])
-        subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+        self.exec_qemu_img("resize", img_tmp, "50G")
         self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
         self.wait_ssh()
         self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
Signature · 1 line
2.18.1
#2 lore raw

Re: [PATCH 1/1] tests/vm: Allow to set qemu-img path

Quoted 8 lines · Wainer dos Santos Moschetta

By default VM build test use qemu-img from system's PATH to create the image disk. Due the lack of qemu-img on the system or the desire to simply use a version built with QEMU, it would be nice to allow one to set its path. So this patch makes that possible by reading the path to qemu-img from QEMU_IMG if set, otherwise it fallback to default behavior.

Queued to testing/next, thanks.

Quoted 145 lines
---
 docs/devel/testing.rst    | 6 ++++--
 tests/vm/Makefile.include | 1 +
 tests/vm/basevm.py        | 5 +++++
 tests/vm/centos           | 2 +-
 tests/vm/fedora           | 4 +---
 tests/vm/freebsd          | 3 +--
 tests/vm/netbsd           | 3 +--
 tests/vm/openbsd          | 3 +--
 tests/vm/ubuntu.i386      | 2 +-
 9 files changed, 16 insertions(+), 13 deletions(-)
docs/devel/testing.rst +4 -2
index 8e981e062d..9be6cd4410 100644
@@ -418,13 +418,15 @@ access, so they SHOULD NOT be exposed to external interfaces if you are
 concerned about attackers taking control of the guest and potentially
 exploiting a QEMU security bug to compromise the host.
 
-QEMU binary
------------
+QEMU binaries
+-------------
 
 By default, qemu-system-x86_64 is searched in $PATH to run the guest. If there
 isn't one, or if it is older than 2.10, the test won't work. In this case,
 provide the QEMU binary in env var: ``QEMU=/path/to/qemu-2.10+``.
 
+Likewise the path to qemu-img can be set in QEMU_IMG environment variable.
+
 Make jobs
 ---------
 
tests/vm/Makefile.include +1 -0
index fea348e845..9e7c46a473 100644
@@ -34,6 +34,7 @@ vm-help vm-test:
 	@echo "    DEBUG=1              	 - Enable verbose output on host and interactive debugging"
 	@echo "    V=1				 - Enable verbose ouput on host and guest commands"
 	@echo "    QEMU=/path/to/qemu		 - Change path to QEMU binary"
+	@echo "    QEMU_IMG=/path/to/qemu-img	 - Change path to qemu-img tool"
 
 vm-build-all: $(addprefix vm-build-, $(IMAGES))
 
tests/vm/basevm.py +5 -0
index 91a9226026..d1efeb3646 100755
@@ -152,6 +152,11 @@ class BaseVM(object):
     def build_image(self, img):
         raise NotImplementedError
 
+    def exec_qemu_img(self, *args):
+        cmd = [os.environ.get("QEMU_IMG", "qemu-img")]
+        cmd.extend(list(args))
+        subprocess.check_call(cmd)
+
     def add_source_dir(self, src_dir):
         name = "data-" + hashlib.sha1(src_dir.encode("utf-8")).hexdigest()[:5]
         tarfile = os.path.join(self._tmpdir, name + ".tar")
tests/vm/centos +1 -1
index 53976f1c4c..eac07dacd6 100755
@@ -68,7 +68,7 @@ class CentosVM(basevm.BaseVM):
         sys.stderr.write("Extracting the image...\n")
         subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
         subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
-        subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+        self.exec_qemu_img("resize", img_tmp, "50G")
         self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
         self.wait_ssh()
         self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
tests/vm/fedora +1 -3
index 7fec1479fb..8e270fc0f0 100755
@@ -74,9 +74,7 @@ class FedoraVM(basevm.BaseVM):
 
         self.print_step("Preparing iso and disk image")
         subprocess.check_call(["cp", "-f", cimg, iso])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
-
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
         self.print_step("Booting installer")
         self.boot(img_tmp, extra_args = [
             "-bios", "pc-bios/bios-256k.bin",
tests/vm/freebsd +1 -2
index 2a19461a90..1825cc5821 100755
@@ -82,8 +82,7 @@ class FreeBSDVM(basevm.BaseVM):
         self.print_step("Preparing iso and disk image")
         subprocess.check_call(["cp", "-f", cimg, iso_xz])
         subprocess.check_call(["xz", "-dvf", iso_xz])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
 
         self.print_step("Booting installer")
         self.boot(img_tmp, extra_args = [
tests/vm/netbsd +1 -2
index 611e6cc5b5..ec6f3563b2 100755
@@ -77,8 +77,7 @@ class NetBSDVM(basevm.BaseVM):
 
         self.print_step("Preparing iso and disk image")
         subprocess.check_call(["ln", "-f", cimg, iso])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
 
         self.print_step("Booting installer")
         self.boot(img_tmp, extra_args = [
tests/vm/openbsd +1 -2
index b92c39f89a..6df5162dbf 100755
@@ -73,8 +73,7 @@ class OpenBSDVM(basevm.BaseVM):
 
         self.print_step("Preparing iso and disk image")
         subprocess.check_call(["cp", "-f", cimg, iso])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
 
         self.print_step("Booting installer")
         self.boot(img_tmp, extra_args = [
tests/vm/ubuntu.i386 +1 -1
index 38f740eabf..39bbe9cc21 100755
@@ -70,7 +70,7 @@ class UbuntuX86VM(basevm.BaseVM):
             sha256sum="e30091144c73483822b7c27193e9d47346dd1064229da577c3fedcf943f7cfcc")
         img_tmp = img + ".tmp"
         subprocess.check_call(["cp", "-f", cimg, img_tmp])
-        subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+        self.exec_qemu_img("resize", img_tmp, "50G")
         self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
         self.wait_ssh()
         self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
Signature · 1 line
Alex Bennée
Reviewed-by on v1 1/1
#3 lore raw

Re: [PATCH 1/1] tests/vm: Allow to set qemu-img path

In reply to Wainer dos Santos Moschetta
Quoted 8 lines · Wainer dos Santos Moschetta

By default VM build test use qemu-img from system's PATH to create the image disk. Due the lack of qemu-img on the system or the desire to simply use a version built with QEMU, it would be nice to allow one to set its path. So this patch makes that possible by reading the path to qemu-img from QEMU_IMG if set, otherwise it fallback to default behavior.

Quoted 146 lines
---
  docs/devel/testing.rst    | 6 ++++--
  tests/vm/Makefile.include | 1 +
  tests/vm/basevm.py        | 5 +++++
  tests/vm/centos           | 2 +-
  tests/vm/fedora           | 4 +---
  tests/vm/freebsd          | 3 +--
  tests/vm/netbsd           | 3 +--
  tests/vm/openbsd          | 3 +--
  tests/vm/ubuntu.i386      | 2 +-
  9 files changed, 16 insertions(+), 13 deletions(-)
docs/devel/testing.rst +4 -2
index 8e981e062d..9be6cd4410 100644
@@ -418,13 +418,15 @@ access, so they SHOULD NOT be exposed to external interfaces if you are
  concerned about attackers taking control of the guest and potentially
  exploiting a QEMU security bug to compromise the host.
 
-QEMU binary
------------
+QEMU binaries
+-------------
 
  By default, qemu-system-x86_64 is searched in $PATH to run the guest. If there
  isn't one, or if it is older than 2.10, the test won't work. In this case,
  provide the QEMU binary in env var: ``QEMU=/path/to/qemu-2.10+``.
 
+Likewise the path to qemu-img can be set in QEMU_IMG environment variable.
+
  Make jobs
  ---------
 
tests/vm/Makefile.include +1 -0
index fea348e845..9e7c46a473 100644
@@ -34,6 +34,7 @@ vm-help vm-test:
  	@echo "    DEBUG=1              	 - Enable verbose output on host and interactive debugging"
  	@echo "    V=1				 - Enable verbose ouput on host and guest commands"
  	@echo "    QEMU=/path/to/qemu		 - Change path to QEMU binary"
+	@echo "    QEMU_IMG=/path/to/qemu-img	 - Change path to qemu-img tool"
 
  vm-build-all: $(addprefix vm-build-, $(IMAGES))
 
tests/vm/basevm.py +5 -0
index 91a9226026..d1efeb3646 100755
@@ -152,6 +152,11 @@ class BaseVM(object):
      def build_image(self, img):
          raise NotImplementedError
 
+    def exec_qemu_img(self, *args):
+        cmd = [os.environ.get("QEMU_IMG", "qemu-img")]
+        cmd.extend(list(args))
+        subprocess.check_call(cmd)
+
      def add_source_dir(self, src_dir):
          name = "data-" + hashlib.sha1(src_dir.encode("utf-8")).hexdigest()[:5]
          tarfile = os.path.join(self._tmpdir, name + ".tar")
tests/vm/centos +1 -1
index 53976f1c4c..eac07dacd6 100755
@@ -68,7 +68,7 @@ class CentosVM(basevm.BaseVM):
          sys.stderr.write("Extracting the image...\n")
          subprocess.check_call(["ln", "-f", cimg, img_tmp + ".xz"])
          subprocess.check_call(["xz", "--keep", "-dvf", img_tmp + ".xz"])
-        subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+        self.exec_qemu_img("resize", img_tmp, "50G")
          self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
          self.wait_ssh()
          self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")
tests/vm/fedora +1 -3
index 7fec1479fb..8e270fc0f0 100755
@@ -74,9 +74,7 @@ class FedoraVM(basevm.BaseVM):
 
          self.print_step("Preparing iso and disk image")
          subprocess.check_call(["cp", "-f", cimg, iso])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
-
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
          self.print_step("Booting installer")
          self.boot(img_tmp, extra_args = [
              "-bios", "pc-bios/bios-256k.bin",
tests/vm/freebsd +1 -2
index 2a19461a90..1825cc5821 100755
@@ -82,8 +82,7 @@ class FreeBSDVM(basevm.BaseVM):
          self.print_step("Preparing iso and disk image")
          subprocess.check_call(["cp", "-f", cimg, iso_xz])
          subprocess.check_call(["xz", "-dvf", iso_xz])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
 
          self.print_step("Booting installer")
          self.boot(img_tmp, extra_args = [
tests/vm/netbsd +1 -2
index 611e6cc5b5..ec6f3563b2 100755
@@ -77,8 +77,7 @@ class NetBSDVM(basevm.BaseVM):
 
          self.print_step("Preparing iso and disk image")
          subprocess.check_call(["ln", "-f", cimg, iso])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
 
          self.print_step("Booting installer")
          self.boot(img_tmp, extra_args = [
tests/vm/openbsd +1 -2
index b92c39f89a..6df5162dbf 100755
@@ -73,8 +73,7 @@ class OpenBSDVM(basevm.BaseVM):
 
          self.print_step("Preparing iso and disk image")
          subprocess.check_call(["cp", "-f", cimg, iso])
-        subprocess.check_call(["qemu-img", "create", "-f", "qcow2",
-                               img_tmp, self.size])
+        self.exec_qemu_img("create", "-f", "qcow2", img_tmp, self.size)
 
          self.print_step("Booting installer")
          self.boot(img_tmp, extra_args = [
tests/vm/ubuntu.i386 +1 -1
index 38f740eabf..39bbe9cc21 100755
@@ -70,7 +70,7 @@ class UbuntuX86VM(basevm.BaseVM):
              sha256sum="e30091144c73483822b7c27193e9d47346dd1064229da577c3fedcf943f7cfcc")
          img_tmp = img + ".tmp"
          subprocess.check_call(["cp", "-f", cimg, img_tmp])
-        subprocess.check_call(["qemu-img", "resize", img_tmp, "50G"])
+        self.exec_qemu_img("resize", img_tmp, "50G")
          self.boot(img_tmp, extra_args = ["-cdrom", self._gen_cloud_init_iso()])
          self.wait_ssh()
          self.ssh_root_check("touch /etc/cloud/cloud-init.disabled")