tests/vm: serial console autoinstall, misc fixes.
v4 cover tests/vm: serial console autoinstall, misc fixes.
This patch series changes the way virtual machines for test builds are managed. They are created locally on the developer machine now. The installer is booted on the serial console and the scripts walks through the dialogs to install and configure the guest.
That takes the download.patchew.org server out of the loop and makes it alot easier to tweak the guest images (adding build dependencies for example).
The install scripts take care to apply host proxy settings (from *_proxy environment variables) to the guest, so any package downloads will be routed through the proxy and can be cached that way. This also makes them work behind strict firewalls.
There are also a bunch of smaller tweaks for tests/vm to fix issues I was struggling with. See commit messages of individual patches for details.
v4:
- rebase, drop patches merged.
- use in-tree seabios binary, for serial console support.
- some minor tweaks.
v3:
- python3 fixes.
- openbsd: configure memory limits.
- freebsd: configure autoboot delay.
Gerd Hoffmann (11):
tests/vm: send proxy environment variables over ssh tests/vm: use ssh with pty unconditionally tests/vm: run test builds on snapshot tests/vm: proper guest shutdown
tests/vm: add vm-boot-{ssh,serial}-<guest> targets tests/vm: serial console support helpers tests/vm: openbsd autoinstall, using serial console tests/vm: freebsd autoinstall, using serial console tests/vm: netbsd autoinstall, using serial console tests/vm: fedora autoinstall, using serial console tests/vm: ubuntu.i386: apt proxy setup
tests/vm/basevm.py | 135 +++++++++++++++++++++++---- tests/vm/Makefile.include | 23 ++++- tests/vm/fedora | 189 ++++++++++++++++++++++++++++++++++++++ tests/vm/freebsd | 180 ++++++++++++++++++++++++++++++++++-- tests/vm/netbsd | 187 +++++++++++++++++++++++++++++++++++-- tests/vm/openbsd | 159 +++++++++++++++++++++++++++++--- tests/vm/ubuntu.i386 | 4 + 7 files changed, 825 insertions(+), 52 deletions(-) create mode 100755 tests/vm/fedora
Signature · 1 line
2.18.1
PATCH v4 1/11 tests/vm: send proxy environment variables over ssh
Packages are fetched via proxy that way, if configured on the host. That might be required to pass firewalls, and it allows to route package downloads through a caching proxy server.
Needs AcceptEnv setup in sshd_config on the guest side to work.
--- tests/vm/basevm.py | 9 +++++++++ 1 file changed, 9 insertions(+)
tests/vm/basevm.py +9 -0
@@ -38,6 +38,13 @@ class BaseVM(object): GUEST_PASS = "qemupass" ROOT_PASS = "qemupass" + envvars = [ + "https_proxy", + "http_proxy", + "ftp_proxy", + "no_proxy", + ] + # The script to run in the guest that builds QEMU BUILD_SCRIPT = "" # The guest name, to be overridden by subclasses
@@ -106,6 +113,8 @@ class BaseVM(object): "-o", "UserKnownHostsFile=" + os.devnull, "-o", "ConnectTimeout=1", "-p", self.ssh_port, "-i", self._ssh_key_file] + for var in self.envvars: + ssh_cmd += ['-o', "SendEnv=%s" % var ] if interactive: ssh_cmd += ['-t'] assert not isinstance(cmd, str)
Signature · 1 line
2.18.1
PATCH v4 2/11 tests/vm: use ssh with pty unconditionally
In reply to Gerd HoffmannAllways ask ssh to run with a pseudo terminal. Not having a terminal causes problems now and then.
--- tests/vm/basevm.py | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-)
tests/vm/basevm.py +4 -9
@@ -107,16 +107,14 @@ class BaseVM(object): os.rename(fname + ".download", fname) return fname - def _ssh_do(self, user, cmd, check, interactive=False): - ssh_cmd = ["ssh", "-q", + def _ssh_do(self, user, cmd, check): + ssh_cmd = ["ssh", "-q", "-t", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=" + os.devnull, "-o", "ConnectTimeout=1", "-p", self.ssh_port, "-i", self._ssh_key_file] for var in self.envvars: ssh_cmd += ['-o', "SendEnv=%s" % var ] - if interactive: - ssh_cmd += ['-t'] assert not isinstance(cmd, str) ssh_cmd += ["%[email protected]" % user] + list(cmd) logging.debug("ssh_cmd: %s", " ".join(ssh_cmd))
@@ -128,9 +126,6 @@ class BaseVM(object): def ssh(self, *cmd): return self._ssh_do(self.GUEST_USER, cmd, False) - def ssh_interactive(self, *cmd): - return self._ssh_do(self.GUEST_USER, cmd, False, True) - def ssh_root(self, *cmd): return self._ssh_do("root", cmd, False)
@@ -284,9 +279,9 @@ def main(vmcls): return 2 if args.interactive: - if vm.ssh_interactive(*cmd) == 0: + if vm.ssh(*cmd) == 0: return 0 - vm.ssh_interactive() + vm.ssh() return 3 else: if vm.ssh(*cmd) != 0:
Signature · 1 line
2.18.1
Re: [Qemu-devel] [PATCH v4 02/11] tests/vm: use ssh with pty unconditionally
Quoted 5 lines · Gerd Hoffmann
Allways ask ssh to run with a pseudo terminal. Not having a terminal causes problems now and then.
Signed-off-by Gerd Hoffmann <[email protected]>Tested-by Thomas Huth <[email protected]>
Tags from v3:
PATCH v4 3/11 tests/vm: run test builds on snapshot
In reply to Gerd HoffmannThe build script doesn't shutdown the guest VMs properly, which results in filesystem corruption and guest boot failures sooner or later.
Use the --snapshot to run builds on a snapshot, That way killing the VM doesn't corrupt the base image.
--- tests/vm/Makefile.include | 1 + 1 file changed, 1 insertion(+)
tests/vm/Makefile.include +1 -0
@@ -57,6 +57,7 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(V),--verbose) \ --image "$<" \ $(if $(BUILD_TARGET),--build-target $(BUILD_TARGET)) \ + --snapshot \ --build-qemu $(SRC_PATH) -- \ $(if $(TARGET_LIST),--target-list=$(TARGET_LIST)) \ $(if $(EXTRA_CONFIGURE_OPTS),$(EXTRA_CONFIGURE_OPTS)), \
Signature · 1 line
2.18.1
Re: [Qemu-devel] [PATCH v4 03/11] tests/vm: run test builds on snapshot
Quoted 9 lines · Gerd Hoffmann
The build script doesn't shutdown the guest VMs properly, which results in filesystem corruption and guest boot failures sooner or later.
Use the --snapshot to run builds on a snapshot, That way killing the VM doesn't corrupt the base image.
Signed-off-by Gerd Hoffmann <[email protected]>Tested-by Thomas Huth <[email protected]>
From v3:
PATCH v4 4/11 tests/vm: proper guest shutdown
In reply to Gerd HoffmannWhen not running in snapshot mode ask the guest to poweroff and wait for this to finish instead of simply quitting qemu, so the guest can flush pending updates to disk.
--- tests/vm/basevm.py | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-)
tests/vm/basevm.py +15 -7
@@ -51,6 +51,8 @@ class BaseVM(object): name = "#base" # The guest architecture, to be overridden by subclasses arch = "#arch" + # command to halt the guest, can be overridden by subclasses + poweroff = "poweroff" def __init__(self, debug=False, vcpus=None): self._guest = None self._tmpdir = os.path.realpath(tempfile.mkdtemp(prefix="vm-test-",
@@ -202,6 +204,10 @@ class BaseVM(object): def wait(self): self._guest.wait() + def graceful_shutdown(self): + self.ssh_root(self.poweroff) + self._guest.wait() + def qmp(self, *args, **kwargs): return self._guest.qmp(*args, **kwargs)
@@ -278,11 +284,13 @@ def main(vmcls): traceback.print_exc() return 2 - if args.interactive: - if vm.ssh(*cmd) == 0: - return 0 + exitcode = 0 + if vm.ssh(*cmd) != 0: + exitcode = 3 + if exitcode != 0 and args.interactive: vm.ssh() - return 3 - else: - if vm.ssh(*cmd) != 0: - return 3 + + if not args.snapshot: + vm.graceful_shutdown() + + return exitcode
Signature · 1 line
2.18.1
Re: [Qemu-devel] [PATCH v4 04/11] tests/vm: proper guest shutdown
Quoted 6 lines · Gerd Hoffmann
When not running in snapshot mode ask the guest to poweroff and wait for this to finish instead of simply quitting qemu, so the guest can flush pending updates to disk.
Signed-off-by Gerd Hoffmann <[email protected]>Tested-by Thomas Huth <[email protected]>
Tags from v3:
PATCH v4 5/11 tests/vm: add vm-boot-{ssh, serial}-<guest> targets
In reply to Gerd HoffmannFor testing/troubleshooting convinience.
make vm-boot-serial-<guest>
Boot guest, with the serial console on stdio.
make vm-boot-ssh-<guest>
Boot guest, login via ssh.
--- tests/vm/Makefile.include | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
tests/vm/Makefile.include +19 -0
@@ -20,6 +20,10 @@ vm-test: @echo " vm-build-all - Build QEMU in all VMs" @echo " vm-clean-all - Clean up VM images" @echo + @echo "For trouble-shooting:" + @echo " vm-boot-serial-<guest> - Boot guest, serial console on stdio" + @echo " vm-boot-ssh-<guest> - Boot guest and login via ssh" + @echo @echo "Special variables:" @echo " BUILD_TARGET=foo - Override the build target" @echo " TARGET_LIST=a,b,c - Override target list in builds"
@@ -63,3 +67,18 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(EXTRA_CONFIGURE_OPTS),$(EXTRA_CONFIGURE_OPTS)), \ " VM-BUILD $*") +vm-boot-serial-%: $(IMAGES_DIR)/%.img + qemu-system-x86_64 -enable-kvm -m 4G -smp 2 -nographic \ + -drive if=none,id=vblk,cache=writeback,file="$<" \ + -netdev user,id=vnet \ + -device virtio-blk-pci,drive=vblk \ + -device virtio-net-pci,netdev=vnet \ + || true + +vm-boot-ssh-%: $(IMAGES_DIR)/%.img + $(call quiet-command, \ + $(SRC_PATH)/tests/vm/$* \ + --image "$<" \ + --interactive \ + false, \ + " VM-BOOT-SSH $*")
Signature · 1 line
2.18.1
Re: [Qemu-devel] [PATCH v4 05/11] tests/vm: add vm-boot-{ssh, serial}-<guest> targets
Quoted 1 line · Gerd Hoffmann
For testing/troubleshooting convinience.
"convenience" (v3)
Quoted 36 lines
make vm-boot-serial-<guest>
Boot guest, with the serial console on stdio.make vm-boot-ssh-<guest>
Boot guest, login via ssh.Signed-off-by Gerd Hoffmann <[email protected]>Tested-by Thomas Huth <[email protected]>--- tests/vm/Makefile.include | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)tests/vm/Makefile.include +9
-0@@ -20,6 +20,10 @@ vm-test: @echo " vm-build-all - Build QEMU in all VMs" @echo " vm-clean-all - Clean up VM images" @echo + @echo "For trouble-shooting:" + @echo " vm-boot-serial-<guest> - Boot guest, serial console on stdio" + @echo " vm-boot-ssh-<guest> - Boot guest and login via ssh" + @echo @echo "Special variables:" @echo " BUILD_TARGET=foo - Override the build target" @echo " TARGET_LIST=a,b,c - Override target list in builds"@@ -63,3 +67,18 @@ vm-build-%: $(IMAGES_DIR)/%.img $(if $(EXTRA_CONFIGURE_OPTS),$(EXTRA_CONFIGURE_OPTS)), \ " VM-BUILD $*") +vm-boot-serial-%: $(IMAGES_DIR)/%.img + qemu-system-x86_64 -enable-kvm -m 4G -smp 2 -nographic \ + -drive if=none,id=vblk,cache=writeback,file="$<" \ + -netdev user,id=vnet \ + -device virtio-blk-pci,drive=vblk \
tests/vm/Makefile.include vm-build-%: $(IMAGES_DIR)/%.img · line 84
Quoted 10 lines
tests/vm/Makefile.include +10
-0+ -device virtio-net-pci,netdev=vnet \ + || true + +vm-boot-ssh-%: $(IMAGES_DIR)/%.img + $(call quiet-command, \ + $(SRC_PATH)/tests/vm/$* \ + --image "$<" \ + --interactive \ + false, \ + " VM-BOOT-SSH $*")
From v3:
We might want the '|| true' here too:
$ exit Shutdown NOW! poweroff: [pid 769] make: *** [tests/vm/Makefile.include:78: vm-boot-ssh-freebsd] Error 3
PATCH v4 6/11 tests/vm: serial console support helpers
In reply to Gerd HoffmannAdd a bunch of helpers to talk to the guest using the serial console.
Also drop the hard-coded -serial parameter for the vm so QEMUMachine.set_console() actually works.
--- tests/vm/basevm.py | 86 ++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 83 insertions(+), 3 deletions(-)
tests/vm/basevm.py +83 -3
@@ -2,10 +2,11 @@ # # VM testing base class # -# Copyright 2017 Red Hat Inc. +# Copyright 2017-2019 Red Hat Inc. # # Authors: # Fam Zheng <[email protected]> +# Gerd Hoffmann <[email protected]> # # This code is licensed under the GPL version 2 or later. See # the COPYING file in the top-level directory.
@@ -13,7 +14,9 @@ from __future__ import print_function import os +import re import sys +import socket import logging import time import datetime
@@ -79,8 +82,7 @@ class BaseVM(object): "-cpu", "max", "-netdev", "user,id=vnet,hostfwd=:127.0.0.1:0-:22", "-device", "virtio-net-pci,netdev=vnet", - "-vnc", "127.0.0.1:0,to=20", - "-serial", "file:%s" % os.path.join(self._tmpdir, "serial.out")] + "-vnc", "127.0.0.1:0,to=20"] if vcpus and vcpus > 1: self._args += ["-smp", "%d" % vcpus] if kvm_available(self.arch):
@@ -162,6 +164,8 @@ class BaseVM(object): logging.debug("QEMU args: %s", " ".join(args)) qemu_bin = os.environ.get("QEMU", "qemu-system-" + self.arch) guest = QEMUMachine(binary=qemu_bin, args=args) + guest.set_machine('pc') + guest.set_console() try: guest.launch() except:
@@ -184,6 +188,82 @@ class BaseVM(object): raise Exception("Cannot find ssh port from 'info usernet':\n%s" % \ usernet_info) + def console_init(self, timeout = 120): + vm = self._guest + vm.console_socket.settimeout(timeout) + + def console_log(self, text): + for line in re.split("[\r\n]", text): + # filter out terminal escape sequences + line = re.sub("\x1b\[[0-9;?]*[a-zA-Z]", "", line) + line = re.sub("\x1b\([0-9;?]*[a-zA-Z]", "", line) + # replace unprintable chars + line = re.sub("\x1b", "<esc>", line) + line = re.sub("[\x00-\x1f]", ".", line) + line = re.sub("[\x80-\xff]", ".", line) + if line == "": + continue + # log console line + sys.stderr.write("con recv: %s\n" % line) + + def console_wait(self, expect): + vm = self._guest + output = "" + while True: + try: + chars = vm.console_socket.recv(1) + except socket.timeout: + sys.stderr.write("console: *** read timeout ***\n") + sys.stderr.write("console: waiting for: '%s'\n" % expect) + sys.stderr.write("console: line buffer:\n") + sys.stderr.write("\n") + self.console_log(output.rstrip()) + sys.stderr.write("\n") + raise + output += chars.decode("latin1") + if expect in output: + break + if "\r" in output or "\n" in output: + lines = re.split("[\r\n]", output) + output = lines.pop() + if self.debug: + self.console_log("\n".join(lines)) + if self.debug: + self.console_log(output) + + def console_send(self, command): + vm = self._guest + if self.debug: + logline = re.sub("\n", "<enter>", command) + logline = re.sub("[\x00-\x1f]", ".", logline) + sys.stderr.write("con send: %s\n" % logline) + for char in list(command): + vm.console_socket.send(char.encode("utf-8")) + time.sleep(0.01) + + def console_wait_send(self, wait, command): + self.console_wait(wait) + self.console_send(command) + + def console_ssh_init(self, prompt, user, pw): + sshkey_cmd = "echo '%s' > .ssh/authorized_keys\n" % SSH_PUB_KEY.rstrip() + self.console_wait_send("login:", "%s\n" % user) + self.console_wait_send("Password:", "%s\n" % pw) + self.console_wait_send(prompt, "mkdir .ssh\n") + self.console_wait_send(prompt, sshkey_cmd) + self.console_wait_send(prompt, "chmod 755 .ssh\n") + self.console_wait_send(prompt, "chmod 644 .ssh/authorized_keys\n") + + def console_sshd_config(self, prompt): + self.console_wait(prompt) + self.console_send("echo 'PermitRootLogin yes' >> /etc/ssh/sshd_config\n") + for var in self.envvars: + self.console_wait(prompt) + self.console_send("echo 'AcceptEnv %s' >> /etc/ssh/sshd_config\n" % var) + + def print_step(self, text): + sys.stderr.write("### %s ...\n" % text) + def wait_ssh(self, seconds=300): starttime = datetime.datetime.now() endtime = starttime + datetime.timedelta(seconds=seconds)
Signature · 1 line
2.18.1
Re: [Qemu-devel] [PATCH v4 06/11] tests/vm: serial console support helpers
Quoted 8 lines · Gerd Hoffmann
Add a bunch of helpers to talk to the guest using the serial console.
Also drop the hard-coded -serial parameter for the vm so
QEMUMachine.set_console()actually works.Signed-off-by Gerd Hoffmann <[email protected]>Tested-by Thomas Huth <[email protected]>
From v3:
PATCH v4 7/11 tests/vm: openbsd autoinstall, using serial console
In reply to Gerd HoffmannInstead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
--- tests/vm/openbsd | 159 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 146 insertions(+), 13 deletions(-)
tests/vm/openbsd +146 -13
@@ -2,10 +2,11 @@ # # OpenBSD VM image # -# Copyright 2017 Red Hat Inc. +# Copyright 2017-2019 Red Hat Inc. # # Authors: # Fam Zheng <[email protected]> +# Gerd Hoffmann <[email protected]> # # This code is licensed under the GPL version 2 or later. See # the COPYING file in the top-level directory.
@@ -13,34 +14,166 @@ import os import sys +import socket import subprocess import basevm class OpenBSDVM(basevm.BaseVM): name = "openbsd" arch = "x86_64" + + link = "https://cdn.openbsd.org/pub/OpenBSD/6.5/amd64/install65.iso" + csum = "38d1f8cadd502f1c27bf05c5abde6cc505dd28f3f34f8a941048ff9a54f9f608" + size = "20G" + pkgs = [ + # tools + "git", + "pkgconf", + "bzip2", "xz", + + # gnu tools + "bash", + "gmake", + "gsed", + "bison", + + # libs: usb + "libusb1", + + # libs: crypto + "gnutls", + + # libs: images + "jpeg", + "png", + + # libs: ui + "sdl2", + "gtk+3", + "libxkbcommon", + ] + BUILD_SCRIPT = """ set -e; - rm -rf /var/tmp/qemu-test.* - cd $(mktemp -d /var/tmp/qemu-test.XXXXXX); + rm -rf /home/qemu/qemu-test.* + cd $(mktemp -d /home/qemu/qemu-test.XXXXXX); + mkdir src build; cd src; tar -xf /dev/rsd1c; - ./configure --cc=x86_64-unknown-openbsd6.1-gcc-4.9.4 --python=python2.7 {configure_opts}; - gmake --output-sync -j{jobs} {verbose}; - # XXX: "gmake check" seems to always hang or fail - #gmake --output-sync -j{jobs} check {verbose}; + cd ../build + ../src/configure --cc=cc --python=python3 {configure_opts}; + gmake --output-sync -j{jobs} {target} {verbose}; """ + poweroff = "halt -p" def build_image(self, img): - cimg = self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz", - sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf') - img_tmp_xz = img + ".tmp.xz" + self.print_step("Downloading install iso") + cimg = self._download_with_cache(self.link, sha256sum=self.csum) img_tmp = img + ".tmp" - sys.stderr.write("Extracting the image...\n") - subprocess.check_call(["cp", "-f", cimg, img_tmp_xz]) - subprocess.check_call(["xz", "-dvf", img_tmp_xz]) + iso = img + ".install.iso" + + self.print_step("Preparing iso and disk image") + subprocess.check_call(["cp", "-f", cimg, iso]) + subprocess.check_call(["qemu-img", "create", "-f", "qcow2", + img_tmp, self.size]) + + self.print_step("Booting installer") + self.boot(img_tmp, extra_args = [ + "-bios", "pc-bios/bios-256k.bin", + "-machine", "graphics=off", + "-cdrom", iso + ]) + self.console_init() + self.console_wait_send("boot>", "set tty com0\n") + self.console_wait_send("boot>", "\n") + + # pre-install configuration + self.console_wait_send("(I)nstall", "i\n") + self.console_wait_send("Terminal type", "xterm\n") + self.console_wait_send("System hostname", "openbsd\n") + self.console_wait_send("Which network interface", "vio0\n") + self.console_wait_send("IPv4 address", "dhcp\n") + self.console_wait_send("IPv6 address", "none\n") + self.console_wait_send("Which network interface", "done\n") + self.console_wait_send("DNS domain name", "localnet\n") + self.console_wait("Password for root account") + self.console_send("%s\n" % self.ROOT_PASS) + self.console_wait("Password for root account") + self.console_send("%s\n" % self.ROOT_PASS) + self.console_wait_send("Start sshd(8)", "yes\n") + self.console_wait_send("X Window System", "\n") + self.console_wait_send("xenodm", "\n") + self.console_wait_send("console to com0", "\n") + self.console_wait_send("Which speed", "\n") + + self.console_wait("Setup a user") + self.console_send("%s\n" % self.GUEST_USER) + self.console_wait("Full name") + self.console_send("%s\n" % self.GUEST_USER) + self.console_wait("Password") + self.console_send("%s\n" % self.GUEST_PASS) + self.console_wait("Password") + self.console_send("%s\n" % self.GUEST_PASS) + + self.console_wait_send("Allow root ssh login", "yes\n") + self.console_wait_send("timezone", "UTC\n") + self.console_wait_send("root disk", "\n") + self.console_wait_send("(W)hole disk", "\n") + self.console_wait_send("(A)uto layout", "\n") + self.console_wait_send("Location of sets", "cd0\n") + self.console_wait_send("Pathname to the sets", "\n") + self.console_wait_send("Set name(s)", "\n") + self.console_wait_send("without verification", "yes\n") + + self.print_step("Installation started now, this will take a while") + self.console_wait_send("Location of sets", "done\n") + + self.console_wait("successfully completed") + self.print_step("Installation finished, rebooting") + self.console_wait_send("(R)eboot", "reboot\n") + + # setup qemu user + prompt = "$" + self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_wait_send(prompt, "exit\n") + + # setup root user + prompt = "openbsd#" + self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_sshd_config(prompt) + + # setup virtio-blk #1 (tarfile) + self.console_wait(prompt) + self.console_send("echo 'chmod 666 /dev/rsd1c' >> /etc/rc.local\n") + + # enable w+x for /home + self.console_wait(prompt) + self.console_send("sed -i -e '/home/s/rw,/rw,wxallowed,/' /etc/fstab\n") + + # tweak datasize limit + self.console_wait(prompt) + self.console_send("sed -i -e 's/\\(datasize[^=]*\\)=[^:]*/\\1=infinity/' /etc/login.conf\n") + + # use http (be proxy cache friendly) + self.console_wait(prompt) + self.console_send("sed -i -e 's/https/http/' /etc/installurl\n") + + self.print_step("Configuration finished, rebooting") + self.console_wait_send(prompt, "reboot\n") + self.console_wait("login:") + self.wait_ssh() + + self.print_step("Installing packages") + self.ssh_root_check("pkg_add %s\n" % " ".join(self.pkgs)) + + # shutdown + self.ssh_root(self.poweroff) + self.wait() + if os.path.exists(img): os.remove(img) os.rename(img_tmp, img) + os.remove(iso) + self.print_step("All done") if __name__ == "__main__": sys.exit(basevm.main(OpenBSDVM))
Signature · 1 line
2.18.1
Re: [Qemu-devel] [PATCH v4 07/11] tests/vm: openbsd autoinstall, using serial console
Quoted 6 lines · Gerd Hoffmann
Instead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
Signed-off-by Gerd Hoffmann <[email protected]>
From v3:
PATCH v4 8/11 tests/vm: freebsd autoinstall, using serial console
In reply to Gerd HoffmannInstead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
Note that freebsd package downloads are delivered as non-cachable content, so I had to configure squid with "ignore-no-store ignore-private ignore-reload" for pkgmir.geo.freebsd.org to make the caching actually work.
--- tests/vm/freebsd | 180 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 170 insertions(+), 10 deletions(-)
tests/vm/freebsd +170 -10
@@ -2,43 +2,203 @@ # # FreeBSD VM image # -# Copyright 2017 Red Hat Inc. +# Copyright 2017-2019 Red Hat Inc. # # Authors: # Fam Zheng <[email protected]> +# Gerd Hoffmann <[email protected]> # # This code is licensed under the GPL version 2 or later. See # the COPYING file in the top-level directory. # import os +import re import sys +import time +import socket import subprocess import basevm class FreeBSDVM(basevm.BaseVM): name = "freebsd" arch = "x86_64" + + link = "https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.0/FreeBSD-12.0-RELEASE-amd64-disc1.iso.xz" + csum = "1d40015bea89d05b8bd13e2ed80c40b522a9ec1abd8e7c8b80954fb485fb99db" + size = "20G" + pkgs = [ + # build tools + "git", + "pkgconf", + "bzip2", + + # gnu tools + "bash", + "gmake", + "gsed", + "flex", "bison", + + # libs: crypto + "gnutls", + + # libs: images + "jpeg-turbo", + "png", + + # libs: ui + "sdl2", + "gtk3", + "libxkbcommon", + + # libs: opengl + "libepoxy", + "mesa-libs", + ] + BUILD_SCRIPT = """ set -e; - rm -rf /var/tmp/qemu-test.* - cd $(mktemp -d /var/tmp/qemu-test.XXXXXX); + rm -rf /home/qemu/qemu-test.* + cd $(mktemp -d /home/qemu/qemu-test.XXXXXX); + mkdir src build; cd src; tar -xf /dev/vtbd1; - ./configure {configure_opts}; + cd ../build + ../src/configure --python=python3.6 {configure_opts}; gmake --output-sync -j{jobs} {target} {verbose}; """ + def console_boot_serial(self): + self.console_wait_send("Autoboot", "3") + self.console_wait_send("OK", "set console=comconsole\n") + self.console_wait_send("OK", "boot\n") + def build_image(self, img): - cimg = self._download_with_cache("http://download.patchew.org/freebsd-11.1-amd64.img.xz", - sha256sum='adcb771549b37bc63826c501f05121a206ed3d9f55f49145908f7e1432d65891') - img_tmp_xz = img + ".tmp.xz" + self.print_step("Downloading install iso") + cimg = self._download_with_cache(self.link, sha256sum=self.csum) img_tmp = img + ".tmp" - sys.stderr.write("Extracting the image...\n") - subprocess.check_call(["cp", "-f", cimg, img_tmp_xz]) - subprocess.check_call(["xz", "-dvf", img_tmp_xz]) + iso = img + ".install.iso" + iso_xz = iso + ".xz" + + self.print_step("Preparing iso and disk image") + subprocess.check_call(["cp", "-f", cimg, iso_xz]) + subprocess.check_call(["xz", "-dvf", iso_xz]) + subprocess.check_call(["qemu-img", "create", "-f", "qcow2", + img_tmp, self.size]) + + self.print_step("Booting installer") + self.boot(img_tmp, extra_args = [ + "-bios", "pc-bios/bios-256k.bin", + "-machine", "graphics=off", + "-cdrom", iso + ]) + self.console_init() + self.console_boot_serial() + self.console_wait_send("Console type", "xterm\n") + + # pre-install configuration + self.console_wait_send("Welcome", "\n") + self.console_wait_send("Keymap Selection", "\n") + self.console_wait_send("Set Hostname", "freebsd\n") + self.console_wait_send("Distribution Select", "\n") + self.console_wait_send("Partitioning", "\n") + self.console_wait_send("Partition", "\n") + self.console_wait_send("Scheme", "\n") + self.console_wait_send("Editor", "f") + self.console_wait_send("Confirmation", "c") + + self.print_step("Installation started now, this will take a while") + + # post-install configuration + self.console_wait("New Password:") + self.console_send("%s\n" % self.ROOT_PASS) + self.console_wait("Retype New Password:") + self.console_send("%s\n" % self.ROOT_PASS) + + self.console_wait_send("Network Configuration", "\n") + self.console_wait_send("IPv4", "y") + self.console_wait_send("DHCP", "y") + self.console_wait_send("IPv6", "n") + self.console_wait_send("Resolver", "\n") + + self.console_wait_send("Time Zone Selector", "a\n") + self.console_wait_send("Confirmation", "y") + self.console_wait_send("Time & Date", "\n") + self.console_wait_send("Time & Date", "\n") + + self.console_wait_send("System Configuration", "\n") + self.console_wait_send("System Hardening", "\n") + + # qemu user + self.console_wait_send("Add User Accounts", "y") + self.console_wait("Username") + self.console_send("%s\n" % self.GUEST_USER) + self.console_wait("Full name") + self.console_send("%s\n" % self.GUEST_USER) + self.console_wait_send("Uid", "\n") + self.console_wait_send("Login group", "\n") + self.console_wait_send("Login group", "\n") + self.console_wait_send("Login class", "\n") + self.console_wait_send("Shell", "\n") + self.console_wait_send("Home directory", "\n") + self.console_wait_send("Home directory perm", "\n") + self.console_wait_send("Use password", "\n") + self.console_wait_send("Use an empty password", "\n") + self.console_wait_send("Use a random password", "\n") + self.console_wait("Enter password:") + self.console_send("%s\n" % self.GUEST_PASS) + self.console_wait("Enter password again:") + self.console_send("%s\n" % self.GUEST_PASS) + self.console_wait_send("Lock out", "\n") + self.console_wait_send("OK", "yes\n") + self.console_wait_send("Add another user", "no\n") + + self.console_wait_send("Final Configuration", "\n") + self.console_wait_send("Manual Configuration", "\n") + self.console_wait_send("Complete", "\n") + + self.print_step("Installation finished, rebooting") + self.console_boot_serial() + + # setup qemu user + prompt = "$" + self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_wait_send(prompt, "exit\n") + + # setup root user + prompt = "root@freebsd:~ #" + self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_sshd_config(prompt) + + # setup serial console + self.console_wait(prompt) + self.console_send("echo 'console=comconsole' >> /boot/loader.conf\n") + + # setup boot delay + self.console_wait(prompt) + self.console_send("echo 'autoboot_delay=1' >> /boot/loader.conf\n") + + # setup virtio-blk #1 (tarfile) + self.console_wait(prompt) + self.console_send("echo 'chmod 666 /dev/vtbd1' >> /etc/rc.local\n") + + self.print_step("Configuration finished, rebooting") + self.console_wait_send(prompt, "reboot\n") + self.console_wait("login:") + self.wait_ssh() + + self.print_step("Installing packages") + self.ssh_root_check("pkg install -y %s\n" % " ".join(self.pkgs)) + + # shutdown + self.ssh_root(self.poweroff) + self.console_wait("Uptime:") + self.wait() + if os.path.exists(img): os.remove(img) os.rename(img_tmp, img) + os.remove(iso) + self.print_step("All done") if __name__ == "__main__": sys.exit(basevm.main(FreeBSDVM))
Signature · 1 line
2.18.1
PATCH v4 9/11 tests/vm: netbsd autoinstall, using serial console
In reply to Gerd HoffmannInstead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
--- tests/vm/netbsd | 187 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 177 insertions(+), 10 deletions(-)
tests/vm/netbsd +177 -10
@@ -2,10 +2,11 @@ # # NetBSD VM image # -# Copyright 2017 Red Hat Inc. +# Copyright 2017-2019 Red Hat Inc. # # Authors: # Fam Zheng <[email protected]> +# Gerd Hoffmann <[email protected]> # # This code is licensed under the GPL version 2 or later. See # the COPYING file in the top-level directory.
@@ -13,32 +14,198 @@ import os import sys +import time import subprocess import basevm class NetBSDVM(basevm.BaseVM): name = "netbsd" arch = "x86_64" + + link = "https://cdn.netbsd.org/pub/NetBSD/NetBSD-8.0/images/NetBSD-8.0-amd64.iso" + size = "20G" + pkgs = [ + # tools + "git-base", + "pkgconf", + "xz", + "python37", + + # gnu tools + "bash", + "gmake", + "gsed", + "flex", "bison", + + # libs: crypto + "gnutls", + + # libs: images + "jpeg", + "png", + + # libs: ui + "SDL2", + "gtk3+", + "libxkbcommon", + ] + BUILD_SCRIPT = """ set -e; - rm -rf /var/tmp/qemu-test.* - cd $(mktemp -d /var/tmp/qemu-test.XXXXXX); + rm -rf /home/qemu/qemu-test.* + cd $(mktemp -d /home/qemu/qemu-test.XXXXXX); + mkdir src build; cd src; tar -xf /dev/rld1a; - ./configure --python=python2.7 {configure_opts}; + cd ../build + ../src/configure --python=python3.7 --disable-opengl {configure_opts}; gmake --output-sync -j{jobs} {target} {verbose}; """ + poweroff = "/sbin/poweroff" def build_image(self, img): - cimg = self._download_with_cache("http://download.patchew.org/netbsd-7.1-amd64.img.xz", - sha256sum='b633d565b0eac3d02015cd0c81440bd8a7a8df8512615ac1ee05d318be015732') - img_tmp_xz = img + ".tmp.xz" + cimg = self._download_with_cache(self.link) img_tmp = img + ".tmp" - sys.stderr.write("Extracting the image...\n") - subprocess.check_call(["cp", "-f", cimg, img_tmp_xz]) - subprocess.check_call(["xz", "-dvf", img_tmp_xz]) + iso = img + ".install.iso" + + self.print_step("Preparing iso and disk image") + subprocess.check_call(["cp", "-f", cimg, iso]) + subprocess.check_call(["qemu-img", "create", "-f", "qcow2", + img_tmp, self.size]) + + self.print_step("Booting installer") + self.boot(img_tmp, extra_args = [ + "-bios", "pc-bios/bios-256k.bin", + "-machine", "graphics=off", + "-cdrom", iso + ]) + self.console_init() + self.console_wait("Primary Bootstrap") + + # serial console boot menu output doesn't work for some + # reason, so we have to fly blind ... + for char in list("5consdev com0\n"): + time.sleep(0.2) + self.console_send(char) + self.console_wait("") + self.console_wait_send("> ", "boot\n") + + self.console_wait_send("Terminal type", "xterm\n") + self.console_wait_send("a: Installation messages", "a\n") + self.console_wait_send("b: US-English", "b\n") + self.console_wait_send("a: Install NetBSD", "a\n") + self.console_wait("Shall we continue?") + self.console_wait_send("b: Yes", "b\n") + + self.console_wait_send("a: ld0", "a\n") + self.console_wait_send("a: This is the correct", "a\n") + self.console_wait_send("b: Use the entire disk", "b\n") + self.console_wait("NetBSD bootcode") + self.console_wait_send("a: Yes", "a\n") + self.console_wait_send("b: Use existing part", "b\n") + self.console_wait_send("x: Partition sizes ok", "x\n") + self.console_wait_send("for your NetBSD disk", "\n") + self.console_wait("Shall we continue?") + self.console_wait_send("b: Yes", "b\n") + + self.console_wait_send("b: Use serial port com0", "b\n") + self.console_wait_send("f: Set serial baud rate", "f\n") + self.console_wait_send("a: 9600", "a\n") + self.console_wait_send("x: Exit", "x\n") + + self.console_wait_send("a: Full installation", "a\n") + self.console_wait_send("a: CD-ROM", "a\n") + + self.print_step("Installation started now, this will take a while") + self.console_wait_send("Hit enter to continue", "\n") + + self.console_wait_send("d: Change root password", "d\n") + self.console_wait_send("a: Yes", "a\n") + self.console_wait("New password:") + self.console_send("%s\n" % self.ROOT_PASS) + self.console_wait("New password:") + self.console_send("%s\n" % self.ROOT_PASS) + self.console_wait("Retype new password:") + self.console_send("%s\n" % self.ROOT_PASS) + + self.console_wait_send("o: Add a user", "o\n") + self.console_wait("username") + self.console_send("%s\n" % self.GUEST_USER) + self.console_wait("to group wheel") + self.console_wait_send("a: Yes", "a\n") + self.console_wait_send("a: /bin/sh", "a\n") + self.console_wait("New password:") + self.console_send("%s\n" % self.GUEST_PASS) + self.console_wait("New password:") + self.console_send("%s\n" % self.GUEST_PASS) + self.console_wait("Retype new password:") + self.console_send("%s\n" % self.GUEST_PASS) + + self.console_wait_send("a: Configure network", "a\n") + self.console_wait_send("a: vioif0", "a\n") + self.console_wait_send("Network media type", "\n") + self.console_wait("autoconfiguration") + self.console_wait_send("a: Yes", "a\n") + self.console_wait_send("DNS domain", "localnet\n") + self.console_wait("Are they OK?") + self.console_wait_send("a: Yes", "a\n") + self.console_wait("installed in /etc") + self.console_wait_send("a: Yes", "a\n") + + self.console_wait_send("e: Enable install", "e\n") + proxy = os.environ.get("http_proxy") + if not proxy is None: + self.console_wait_send("f: Proxy", "f\n") + self.console_wait("Proxy") + self.console_send("%s\n" % proxy) + self.console_wait_send("x: Install pkgin", "x\n") + self.console_init(1200) + self.console_wait_send("Hit enter to continue", "\n") + self.console_init() + + self.console_wait_send("g: Enable sshd", "g\n") + self.console_wait_send("x: Finished conf", "x\n") + self.console_wait_send("Hit enter to continue", "\n") + + self.print_step("Installation finished, rebooting") + self.console_wait_send("d: Reboot the computer", "d\n") + + # setup qemu user + prompt = "localhost$" + self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_wait_send(prompt, "exit\n") + + # setup root user + prompt = "localhost#" + self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_sshd_config(prompt) + + # setup virtio-blk #1 (tarfile) + self.console_wait(prompt) + self.console_send("echo 'chmod 666 /dev/rld1a' >> /etc/rc.local\n") + + # turn off mprotect (conflicts with tcg) + self.console_wait(prompt) + self.console_send("echo security.pax.mprotect.enabled=0 >> /etc/sysctl.conf\n") + + self.print_step("Configuration finished, rebooting") + self.console_wait_send(prompt, "reboot\n") + self.console_wait("login:") + self.wait_ssh() + + self.print_step("Installing packages") + self.ssh_root_check("pkgin update\n") + self.ssh_root_check("pkgin -y install %s\n" % " ".join(self.pkgs)) + + # shutdown + self.ssh_root(self.poweroff) + self.console_wait("entering state S5") + self.wait() + if os.path.exists(img): os.remove(img) os.rename(img_tmp, img) + os.remove(iso) + self.print_step("All done") if __name__ == "__main__": sys.exit(basevm.main(NetBSDVM))
Signature · 1 line
2.18.1
Re: [Qemu-devel] [PATCH v4 09/11] tests/vm: netbsd autoinstall, using serial console
Quoted 8 lines · Gerd Hoffmann
Instead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
Signed-off-by Gerd Hoffmann <[email protected]>Reviewed-by Kamil Rytarowski <[email protected]>Tested-by Thomas Huth <[email protected]>
From v3:
Re: [Qemu-devel] [PATCH v4 09/11] tests/vm: netbsd autoinstall, using serial console
In reply to Gerd HoffmannQuoted 4 lines · Gerd Hoffmann
Instead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
I've had to drop this from my v3 PR as Peter was seeing a very slow/hanging install when running his merge tests. I've tried to reproduce and I see it stall while installing packages but nowhere near the delay Peter has seen.
Any pointers on how to debug gratefully received.
Quoted 223 lines
Signed-off-by Gerd Hoffmann <[email protected]>Reviewed-by Kamil Rytarowski <[email protected]>Tested-by Thomas Huth <[email protected]>--- tests/vm/netbsd | 187 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 177 insertions(+), 10 deletions(-)tests/vm/netbsd +173
-10@@ -2,10 +2,11 @@ # # NetBSD VM image # -# Copyright 2017 Red Hat Inc. +# Copyright 2017-2019 Red Hat Inc. # # Authors: # Fam Zheng <[email protected]> +# Gerd Hoffmann <[email protected]> # # This code is licensed under the GPL version 2 or later. See # the COPYING file in the top-level directory.@@ -13,32 +14,198 @@ import os import sys +import time import subprocess import basevm class NetBSDVM(basevm.BaseVM): name = "netbsd" arch = "x86_64" + + link = "https://cdn.netbsd.org/pub/NetBSD/NetBSD-8.0/images/NetBSD-8.0-amd64.iso" + size = "20G" + pkgs = [ + # tools + "git-base", + "pkgconf", + "xz", + "python37", + + # gnu tools + "bash", + "gmake", + "gsed", + "flex", "bison", + + # libs: crypto + "gnutls", + + # libs: images + "jpeg", + "png", + + # libs: ui + "SDL2", + "gtk3+", + "libxkbcommon", + ] + BUILD_SCRIPT = """ set -e; - rm -rf /var/tmp/qemu-test.* - cd $(mktemp -d /var/tmp/qemu-test.XXXXXX); + rm -rf /home/qemu/qemu-test.* + cd $(mktemp -d /home/qemu/qemu-test.XXXXXX); + mkdir src build; cd src; tar -xf /dev/rld1a; - ./configure --python=python2.7 {configure_opts}; + cd ../build + ../src/configure --python=python3.7 --disable-opengl {configure_opts}; gmake --output-sync -j{jobs} {target} {verbose}; """ + poweroff = "/sbin/poweroff" def build_image(self, img): - cimg = self._download_with_cache("http://download.patchew.org/netbsd-7.1-amd64.img.xz", - sha256sum='b633d565b0eac3d02015cd0c81440bd8a7a8df8512615ac1ee05d318be015732') - img_tmp_xz = img + ".tmp.xz" + cimg = self._download_with_cache(self.link) img_tmp = img + ".tmp" - sys.stderr.write("Extracting the image...\n") - subprocess.check_call(["cp", "-f", cimg, img_tmp_xz]) - subprocess.check_call(["xz", "-dvf", img_tmp_xz]) + iso = img + ".install.iso" + + self.print_step("Preparing iso and disk image") + subprocess.check_call(["cp", "-f", cimg, iso]) + subprocess.check_call(["qemu-img", "create", "-f", "qcow2", + img_tmp, self.size]) + + self.print_step("Booting installer") + self.boot(img_tmp, extra_args = [ + "-bios", "pc-bios/bios-256k.bin", + "-machine", "graphics=off", + "-cdrom", iso + ]) + self.console_init() + self.console_wait("Primary Bootstrap") + + # serial console boot menu output doesn't work for some + # reason, so we have to fly blind ... + for char in list("5consdev com0\n"): + time.sleep(0.2) + self.console_send(char) + self.console_wait("") + self.console_wait_send("> ", "boot\n") + + self.console_wait_send("Terminal type", "xterm\n") + self.console_wait_send("a: Installation messages", "a\n") + self.console_wait_send("b: US-English", "b\n") + self.console_wait_send("a: Install NetBSD", "a\n") + self.console_wait("Shall we continue?") + self.console_wait_send("b: Yes", "b\n") + + self.console_wait_send("a: ld0", "a\n") + self.console_wait_send("a: This is the correct", "a\n") + self.console_wait_send("b: Use the entire disk", "b\n") + self.console_wait("NetBSD bootcode") + self.console_wait_send("a: Yes", "a\n") + self.console_wait_send("b: Use existing part", "b\n") + self.console_wait_send("x: Partition sizes ok", "x\n") + self.console_wait_send("for your NetBSD disk", "\n") + self.console_wait("Shall we continue?") + self.console_wait_send("b: Yes", "b\n") + + self.console_wait_send("b: Use serial port com0", "b\n") + self.console_wait_send("f: Set serial baud rate", "f\n") + self.console_wait_send("a: 9600", "a\n") + self.console_wait_send("x: Exit", "x\n") + + self.console_wait_send("a: Full installation", "a\n") + self.console_wait_send("a: CD-ROM", "a\n") + + self.print_step("Installation started now, this will take a while") + self.console_wait_send("Hit enter to continue", "\n") + + self.console_wait_send("d: Change root password", "d\n") + self.console_wait_send("a: Yes", "a\n") + self.console_wait("New password:") + self.console_send("%s\n" % self.ROOT_PASS) + self.console_wait("New password:") + self.console_send("%s\n" % self.ROOT_PASS) + self.console_wait("Retype new password:") + self.console_send("%s\n" % self.ROOT_PASS) + + self.console_wait_send("o: Add a user", "o\n") + self.console_wait("username") + self.console_send("%s\n" % self.GUEST_USER) + self.console_wait("to group wheel") + self.console_wait_send("a: Yes", "a\n") + self.console_wait_send("a: /bin/sh", "a\n") + self.console_wait("New password:") + self.console_send("%s\n" % self.GUEST_PASS) + self.console_wait("New password:") + self.console_send("%s\n" % self.GUEST_PASS) + self.console_wait("Retype new password:") + self.console_send("%s\n" % self.GUEST_PASS) + + self.console_wait_send("a: Configure network", "a\n") + self.console_wait_send("a: vioif0", "a\n") + self.console_wait_send("Network media type", "\n") + self.console_wait("autoconfiguration") + self.console_wait_send("a: Yes", "a\n") + self.console_wait_send("DNS domain", "localnet\n") + self.console_wait("Are they OK?") + self.console_wait_send("a: Yes", "a\n") + self.console_wait("installed in /etc") + self.console_wait_send("a: Yes", "a\n") + + self.console_wait_send("e: Enable install", "e\n") + proxy = os.environ.get("http_proxy") + if not proxy is None: + self.console_wait_send("f: Proxy", "f\n") + self.console_wait("Proxy") + self.console_send("%s\n" % proxy) + self.console_wait_send("x: Install pkgin", "x\n") + self.console_init(1200) + self.console_wait_send("Hit enter to continue", "\n") + self.console_init() + + self.console_wait_send("g: Enable sshd", "g\n") + self.console_wait_send("x: Finished conf", "x\n") + self.console_wait_send("Hit enter to continue", "\n") + + self.print_step("Installation finished, rebooting") + self.console_wait_send("d: Reboot the computer", "d\n") + + # setup qemu user + prompt = "localhost$" + self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_wait_send(prompt, "exit\n") + + # setup root user + prompt = "localhost#" + self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_sshd_config(prompt) + + # setup virtio-blk #1 (tarfile) + self.console_wait(prompt) + self.console_send("echo 'chmod 666 /dev/rld1a' >> /etc/rc.local\n") + + # turn off mprotect (conflicts with tcg) + self.console_wait(prompt) + self.console_send("echo security.pax.mprotect.enabled=0 >> /etc/sysctl.conf\n") + + self.print_step("Configuration finished, rebooting") + self.console_wait_send(prompt, "reboot\n") + self.console_wait("login:") + self.wait_ssh() + + self.print_step("Installing packages") + self.ssh_root_check("pkgin update\n") + self.ssh_root_check("pkgin -y install %s\n" % " ".join(self.pkgs)) + + # shutdown + self.ssh_root(self.poweroff) + self.console_wait("entering state S5")
tests/vm/netbsd line 211
Quoted 10 lines
tests/vm/netbsd +4
-0+ self.wait() + if os.path.exists(img): os.remove(img) os.rename(img_tmp, img) + os.remove(iso) + self.print_step("All done") if __name__ == "__main__": sys.exit(basevm.main(NetBSDVM))
--
Alex Bennée
Re: [Qemu-devel] [PATCH v4 09/11] tests/vm: netbsd autoinstall, using serial console
Quoted 8 lines · Alex Bennée
Quoted 4 lines · Gerd Hoffmann
Instead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
Quoted 7 lines
I've had to drop this from my v3 PR as Peter was seeing a very slow/hanging install when running his merge tests. I've tried to reproduce and I see it stall while installing packages but nowhere near the delay Peter has seen.
Any pointers on how to debug gratefully received.
Does it use virtio? There were performance issues with virio disk device.
Quoted 238 lines
Quoted 223 lines
Signed-off-by Gerd Hoffmann <[email protected]>Reviewed-by Kamil Rytarowski <[email protected]>Tested-by Thomas Huth <[email protected]>--- tests/vm/netbsd | 187 +++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 177 insertions(+), 10 deletions(-)tests/vm/netbsd +173
-10@@ -2,10 +2,11 @@ # # NetBSD VM image # -# Copyright 2017 Red Hat Inc. +# Copyright 2017-2019 Red Hat Inc. # # Authors: # Fam Zheng <[email protected]> +# Gerd Hoffmann <[email protected]> # # This code is licensed under the GPL version 2 or later. See # the COPYING file in the top-level directory.@@ -13,32 +14,198 @@ import os import sys +import time import subprocess import basevm class NetBSDVM(basevm.BaseVM): name = "netbsd" arch = "x86_64" + + link = "https://cdn.netbsd.org/pub/NetBSD/NetBSD-8.0/images/NetBSD-8.0-amd64.iso" + size = "20G" + pkgs = [ + # tools + "git-base", + "pkgconf", + "xz", + "python37", + + # gnu tools + "bash", + "gmake", + "gsed", + "flex", "bison", + + # libs: crypto + "gnutls", + + # libs: images + "jpeg", + "png", + + # libs: ui + "SDL2", + "gtk3+", + "libxkbcommon", + ] + BUILD_SCRIPT = """ set -e; - rm -rf /var/tmp/qemu-test.* - cd $(mktemp -d /var/tmp/qemu-test.XXXXXX); + rm -rf /home/qemu/qemu-test.* + cd $(mktemp -d /home/qemu/qemu-test.XXXXXX); + mkdir src build; cd src; tar -xf /dev/rld1a; - ./configure --python=python2.7 {configure_opts}; + cd ../build + ../src/configure --python=python3.7 --disable-opengl {configure_opts}; gmake --output-sync -j{jobs} {target} {verbose}; """ + poweroff = "/sbin/poweroff" def build_image(self, img): - cimg = self._download_with_cache("http://download.patchew.org/netbsd-7.1-amd64.img.xz", - sha256sum='b633d565b0eac3d02015cd0c81440bd8a7a8df8512615ac1ee05d318be015732') - img_tmp_xz = img + ".tmp.xz" + cimg = self._download_with_cache(self.link) img_tmp = img + ".tmp" - sys.stderr.write("Extracting the image...\n") - subprocess.check_call(["cp", "-f", cimg, img_tmp_xz]) - subprocess.check_call(["xz", "-dvf", img_tmp_xz]) + iso = img + ".install.iso" + + self.print_step("Preparing iso and disk image") + subprocess.check_call(["cp", "-f", cimg, iso]) + subprocess.check_call(["qemu-img", "create", "-f", "qcow2", + img_tmp, self.size]) + + self.print_step("Booting installer") + self.boot(img_tmp, extra_args = [ + "-bios", "pc-bios/bios-256k.bin", + "-machine", "graphics=off", + "-cdrom", iso + ]) + self.console_init() + self.console_wait("Primary Bootstrap") + + # serial console boot menu output doesn't work for some + # reason, so we have to fly blind ... + for char in list("5consdev com0\n"): + time.sleep(0.2) + self.console_send(char) + self.console_wait("") + self.console_wait_send("> ", "boot\n") + + self.console_wait_send("Terminal type", "xterm\n") + self.console_wait_send("a: Installation messages", "a\n") + self.console_wait_send("b: US-English", "b\n") + self.console_wait_send("a: Install NetBSD", "a\n") + self.console_wait("Shall we continue?") + self.console_wait_send("b: Yes", "b\n") + + self.console_wait_send("a: ld0", "a\n") + self.console_wait_send("a: This is the correct", "a\n") + self.console_wait_send("b: Use the entire disk", "b\n") + self.console_wait("NetBSD bootcode") + self.console_wait_send("a: Yes", "a\n") + self.console_wait_send("b: Use existing part", "b\n") + self.console_wait_send("x: Partition sizes ok", "x\n") + self.console_wait_send("for your NetBSD disk", "\n") + self.console_wait("Shall we continue?") + self.console_wait_send("b: Yes", "b\n") + + self.console_wait_send("b: Use serial port com0", "b\n") + self.console_wait_send("f: Set serial baud rate", "f\n") + self.console_wait_send("a: 9600", "a\n") + self.console_wait_send("x: Exit", "x\n") + + self.console_wait_send("a: Full installation", "a\n") + self.console_wait_send("a: CD-ROM", "a\n") + + self.print_step("Installation started now, this will take a while") + self.console_wait_send("Hit enter to continue", "\n") + + self.console_wait_send("d: Change root password", "d\n") + self.console_wait_send("a: Yes", "a\n") + self.console_wait("New password:") + self.console_send("%s\n" % self.ROOT_PASS) + self.console_wait("New password:") + self.console_send("%s\n" % self.ROOT_PASS) + self.console_wait("Retype new password:") + self.console_send("%s\n" % self.ROOT_PASS) + + self.console_wait_send("o: Add a user", "o\n") + self.console_wait("username") + self.console_send("%s\n" % self.GUEST_USER) + self.console_wait("to group wheel") + self.console_wait_send("a: Yes", "a\n") + self.console_wait_send("a: /bin/sh", "a\n") + self.console_wait("New password:") + self.console_send("%s\n" % self.GUEST_PASS) + self.console_wait("New password:") + self.console_send("%s\n" % self.GUEST_PASS) + self.console_wait("Retype new password:") + self.console_send("%s\n" % self.GUEST_PASS) + + self.console_wait_send("a: Configure network", "a\n") + self.console_wait_send("a: vioif0", "a\n") + self.console_wait_send("Network media type", "\n") + self.console_wait("autoconfiguration") + self.console_wait_send("a: Yes", "a\n") + self.console_wait_send("DNS domain", "localnet\n") + self.console_wait("Are they OK?") + self.console_wait_send("a: Yes", "a\n") + self.console_wait("installed in /etc") + self.console_wait_send("a: Yes", "a\n") + + self.console_wait_send("e: Enable install", "e\n") + proxy = os.environ.get("http_proxy") + if not proxy is None: + self.console_wait_send("f: Proxy", "f\n") + self.console_wait("Proxy") + self.console_send("%s\n" % proxy) + self.console_wait_send("x: Install pkgin", "x\n") + self.console_init(1200) + self.console_wait_send("Hit enter to continue", "\n") + self.console_init() + + self.console_wait_send("g: Enable sshd", "g\n") + self.console_wait_send("x: Finished conf", "x\n") + self.console_wait_send("Hit enter to continue", "\n") + + self.print_step("Installation finished, rebooting") + self.console_wait_send("d: Reboot the computer", "d\n") + + # setup qemu user + prompt = "localhost$" + self.console_ssh_init(prompt, self.GUEST_USER, self.GUEST_PASS) + self.console_wait_send(prompt, "exit\n") + + # setup root user + prompt = "localhost#" + self.console_ssh_init(prompt, "root", self.ROOT_PASS) + self.console_sshd_config(prompt) + + # setup virtio-blk #1 (tarfile) + self.console_wait(prompt) + self.console_send("echo 'chmod 666 /dev/rld1a' >> /etc/rc.local\n") + + # turn off mprotect (conflicts with tcg) + self.console_wait(prompt) + self.console_send("echo security.pax.mprotect.enabled=0 >> /etc/sysctl.conf\n") + + self.print_step("Configuration finished, rebooting") + self.console_wait_send(prompt, "reboot\n") + self.console_wait("login:") + self.wait_ssh() + + self.print_step("Installing packages") + self.ssh_root_check("pkgin update\n") + self.ssh_root_check("pkgin -y install %s\n" % " ".join(self.pkgs)) + + # shutdown + self.ssh_root(self.poweroff) + self.console_wait("entering state S5")tests/vm/netbsd line 211
Quoted 10 lines
tests/vm/netbsd +4
-0+ self.wait() + if os.path.exists(img): os.remove(img) os.rename(img_tmp, img) + os.remove(iso) + self.print_step("All done") if __name__ == "__main__": sys.exit(basevm.main(NetBSDVM))--
Alex Bennée
Re: [Qemu-devel] [PATCH v4 09/11] tests/vm: netbsd autoinstall, using serial console
Quoted 14 lines · Kamil Rytarowski
Quoted 13 lines · Alex Bennée
Quoted 4 lines · Gerd Hoffmann
Instead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
I've had to drop this from my v3 PR as Peter was seeing a very slow/hanging install when running his merge tests. I've tried to reproduce and I see it stall while installing packages but nowhere near the delay Peter has seen.
Quoted 4 lines
Quoted 2 lines
Any pointers on how to debug gratefully received.
Does it use virtio? There were performance issues with virio disk device.
Yes, virtio-blk, you can see how the VM is started looking at tests/vm/basevm.py:
"-drive", "file=%s,if=none,id=%s,cache=writeback,format=raw" % (tarfile, name),
"-device", "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)
Re: [Qemu-devel] [PATCH v4 09/11] tests/vm: netbsd autoinstall, using serial console
Quoted 20 lines · Philippe Mathieu-Daudé
Quoted 14 lines · Kamil Rytarowski
Quoted 13 lines · Alex Bennée
Quoted 4 lines · Gerd Hoffmann
Instead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
I've had to drop this from my v3 PR as Peter was seeing a very slow/hanging install when running his merge tests. I've tried to reproduce and I see it stall while installing packages but nowhere near the delay Peter has seen.
Quoted 4 lines
Quoted 2 lines
Any pointers on how to debug gratefully received.
Does it use virtio? There were performance issues with virio disk device.
Quoted 9 lines
Yes, virtio-blk, you can see how the VM is started looking at
tests/vm/basevm.py:"-drive", "file=%s,if=none,id=%s,cache=writeback,format=raw" % (tarfile, name),
"-device", "virtio-blk,drive=%s,serial=%s,bootindex=1" % (name, name)
Try to switch to a non-virtio option and retry.
Re: [Qemu-devel] [PATCH v4 09/11] tests/vm: netbsd autoinstall, using serial console
In reply to Kamil RytarowskiQuoted 15 lines · Kamil Rytarowski
Quoted 13 lines · Alex Bennée
Quoted 4 lines · Gerd Hoffmann
Instead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
I've had to drop this from my v3 PR as Peter was seeing a very slow/hanging install when running his merge tests. I've tried to reproduce and I see it stall while installing packages but nowhere near the delay Peter has seen.
Quoted 4 lines
Quoted 2 lines
Any pointers on how to debug gratefully received.
Does it use virtio? There were performance issues with virio disk device.
The symptoms I see with this patch are that the VM appears to completely stall, rather than progressing but slowly. (I do also see on the current NetBSD tests/vm setup that sometimes 'make check' runs pretty slowly, but in that case it does make progress, it just completes much slower than either the openbsd or freebsd VMs. 'make check' is not particularly IO intensive, certainly not compared to the actual compile phase, though. And sometimes the initial "wait for VM to boot and ssh in" phase just hits our 6 minute timeout.)
thanks
-- PMM
Re: [Qemu-devel] [PATCH v4 09/11] tests/vm: netbsd autoinstall, using serial console
Quoted 21 lines · Peter Maydell
Quoted 15 lines · Kamil Rytarowski
Quoted 13 lines · Alex Bennée
Quoted 4 lines · Gerd Hoffmann
Instead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
I've had to drop this from my v3 PR as Peter was seeing a very slow/hanging install when running his merge tests. I've tried to reproduce and I see it stall while installing packages but nowhere near the delay Peter has seen.
Quoted 4 lines
Quoted 2 lines
Any pointers on how to debug gratefully received.
Does it use virtio? There were performance issues with virio disk device.
Quoted 9 lines
The symptoms I see with this patch are that the VM appears to completely stall, rather than progressing but slowly. (I do also see on the current NetBSD tests/vm setup that sometimes 'make check' runs pretty slowly, but in that case it does make progress, it just completes much slower than either the openbsd or freebsd VMs. 'make check' is not particularly IO intensive, certainly not compared to the actual compile phase, though. And sometimes the initial "wait for VM to boot and ssh in" phase just hits our 6 minute timeout.)
I wonder if there's some undiagnosed timer or interrupt problems we're hitting in NetBSD. We had someone just yesterdayt on IRC reporting they needed kernel irqchip disabled for OpenBSD under some circumstances for example. So could be worth trying one or more of the various tweak args like:
-machine kernel_irqchip=off
-global kvm-pit.lost_tick_policy=discard
to see if it affects stability to any useful extent.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [Qemu-devel] [PATCH v4 09/11] tests/vm: netbsd autoinstall, using serial console
In reply to Peter MaydellQuoted 16 lines · Peter Maydell
Quoted 15 lines · Kamil Rytarowski
Quoted 13 lines · Alex Bennée
Quoted 4 lines · Gerd Hoffmann
Instead of fetching the prebuilt image from patchew download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
I've had to drop this from my v3 PR as Peter was seeing a very slow/hanging install when running his merge tests. I've tried to reproduce and I see it stall while installing packages but nowhere near the delay Peter has seen.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Any pointers on how to debug gratefully received.
Does it use virtio? There were performance issues with virio disk device.
The symptoms I see with this patch are that the VM appears to completely stall, rather than progressing but slowly.
which is odd given we are both testing on the same machine (hackbox2?). I wonder if there is a environmental issue? Possibly the host file-system?
Quoted 10 lines
(I do also see on the current NetBSD tests/vm setup that sometimes 'make check' runs pretty slowly, but in that case it does make progress, it just completes much slower than either the openbsd or freebsd VMs. 'make check' is not particularly IO intensive, certainly not compared to the actual compile phase, though. And sometimes the initial "wait for VM to boot and ssh in" phase just hits our 6 minute timeout.)
thanks
-- PMM
--
Alex Bennée
PATCH v4 10/11 tests/vm: fedora autoinstall, using serial console
In reply to Gerd HoffmannDownload the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
Yes, we have docker images for fedora. But for trouble-shooting it might be helpful to have a vm too. When vm builds fail you can use it to figure whenever the vm setup or the guest os is the problem.
--- tests/vm/basevm.py | 9 +- tests/vm/Makefile.include | 3 +- tests/vm/fedora | 189 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 199 insertions(+), 2 deletions(-) create mode 100755 tests/vm/fedora
tests/vm/basevm.py +8 -1
@@ -206,7 +206,7 @@ class BaseVM(object): # log console line sys.stderr.write("con recv: %s\n" % line) - def console_wait(self, expect): + def console_wait(self, expect, expectalt = None): vm = self._guest output = "" while True:
@@ -215,6 +215,8 @@ class BaseVM(object): except socket.timeout: sys.stderr.write("console: *** read timeout ***\n") sys.stderr.write("console: waiting for: '%s'\n" % expect) + if not expectalt is None: + sys.stderr.write("console: waiting for: '%s' (alt)\n" % expectalt) sys.stderr.write("console: line buffer:\n") sys.stderr.write("\n") self.console_log(output.rstrip())
@@ -223,6 +225,8 @@ class BaseVM(object): output += chars.decode("latin1") if expect in output: break + if not expectalt is None and expectalt in output: + break if "\r" in output or "\n" in output: lines = re.split("[\r\n]", output) output = lines.pop()
@@ -230,6 +234,9 @@ class BaseVM(object): self.console_log("\n".join(lines)) if self.debug: self.console_log(output) + if not expectalt is None and expectalt in output: + return False + return True def console_send(self, command): vm = self._guest
tests/vm/Makefile.include +2 -1
@@ -2,7 +2,7 @@ .PHONY: vm-build-all vm-clean-all -IMAGES := ubuntu.i386 freebsd netbsd openbsd centos +IMAGES := ubuntu.i386 freebsd netbsd openbsd centos fedora IMAGES_DIR := $(HOME)/.cache/qemu-vm/images IMAGE_FILES := $(patsubst %, $(IMAGES_DIR)/%.img, $(IMAGES))
@@ -16,6 +16,7 @@ vm-test: @echo " vm-build-netbsd - Build QEMU in NetBSD VM" @echo " vm-build-openbsd - Build QEMU in OpenBSD VM" @echo " vm-build-centos - Build QEMU in CentOS VM, with Docker" + @echo " vm-build-fedora - Build QEMU in Fedora VM" @echo "" @echo " vm-build-all - Build QEMU in all VMs" @echo " vm-clean-all - Clean up VM images"
tests/vm/fedora new file +189 -0
189 lines not on this page · raw message
Signature · 1 line
2.18.1
Re: [Qemu-devel] [PATCH v4 10/11] tests/vm: fedora autoinstall, using serial console
Quoted 10 lines · Gerd Hoffmann
Download the install iso and prepare the image locally. Install to disk, using the serial console. Create qemu user, configure ssh login. Install packages needed for qemu builds.
Yes, we have docker images for fedora. But for trouble-shooting it might be helpful to have a vm too. When vm builds fail you can use it to figure whenever the vm setup or the guest os is the problem.
Signed-off-by Gerd Hoffmann <[email protected]>Tested-by Thomas Huth <[email protected]>
From v3:
PATCH v4 11/11 tests/vm: ubuntu.i386: apt proxy setup
In reply to Gerd HoffmannConfigure apt proxy so package downloads
can be cached and can pass firewalls.
--- tests/vm/ubuntu.i386 | 4 ++++ 1 file changed, 4 insertions(+)
tests/vm/ubuntu.i386 +4 -0
@@ -51,6 +51,10 @@ class UbuntuX86VM(basevm.BaseVM): " ssh-authorized-keys:\n", " - %s\n" % basevm.SSH_PUB_KEY, "locale: en_US.UTF-8\n"]) + proxy = os.environ.get("http_proxy") + if not proxy is None: + udata.writelines(["apt:\n", + " proxy: %s" % proxy]) udata.close() subprocess.check_call(["genisoimage", "-output", "cloud-init.iso", "-volid", "cidata", "-joliet", "-rock",
Signature · 1 line
2.18.1
Quoted 27 lines · Gerd Hoffmann
This patch series changes the way virtual machines for test builds are managed. They are created locally on the developer machine now. The installer is booted on the serial console and the scripts walks through the dialogs to install and configure the guest.
That takes the download.patchew.org server out of the loop and makes it alot easier to tweak the guest images (adding build dependencies for example).
The install scripts take care to apply host proxy settings (from
*_proxyenvironment variables) to the guest, so any package downloads will be routed through the proxy and can be cached that way. This also makes them work behind strict firewalls.There are also a bunch of smaller tweaks for tests/vm to fix issues I was struggling with. See commit messages of individual patches for details.
v4:
- rebase, drop patches merged.
- use in-tree seabios binary, for serial console support.
- some minor tweaks.
v3:
- python3 fixes.
- openbsd: configure memory limits.
- freebsd: configure autoboot delay.
<snip>
Quoted 4 lines
tests/vm: openbsd autoinstall, using serial console tests/vm: freebsd autoinstall, using serial console tests/vm: netbsd autoinstall, using serial console tests/vm: fedora autoinstall, using serial console
For reference there are some minor conflicts with Cleber's copy cleanups currently in my testing/next. The easiest fix was to apply to a clean origin/master and then rebase it so I could deal with the conflicts in git.
Currently testing it out:
https://github.com/stsquad/qemu/tree/review/test-vm-serial-v4
--
Alex Bennée
Quoted 4 lines · Gerd Hoffmann
This patch series changes the way virtual machines for test builds are managed. They are created locally on the developer machine now. The installer is booted on the serial console and the scripts walks through the dialogs to install and configure the guest.
I've tested this on the previously failing box and it works fine.
Queued to testing/next, thanks.
Quoted 36 lines
That takes the download.patchew.org server out of the loop and makes it alot easier to tweak the guest images (adding build dependencies for example).
The install scripts take care to apply host proxy settings (from
*_proxyenvironment variables) to the guest, so any package downloads will be routed through the proxy and can be cached that way. This also makes them work behind strict firewalls.There are also a bunch of smaller tweaks for tests/vm to fix issues I was struggling with. See commit messages of individual patches for details.
v4:
- rebase, drop patches merged.
- use in-tree seabios binary, for serial console support.
- some minor tweaks.
v3:
- python3 fixes.
- openbsd: configure memory limits.
- freebsd: configure autoboot delay.Gerd Hoffmann (11):
tests/vm: send proxy environment variables over ssh tests/vm: use ssh with pty unconditionally tests/vm: run test builds on snapshot tests/vm: proper guest shutdown
tests/vm: add vm-boot-{ssh,serial}-<guest> targets tests/vm: serial console support helpers tests/vm: openbsd autoinstall, using serial console tests/vm: freebsd autoinstall, using serial console tests/vm: netbsd autoinstall, using serial console tests/vm: fedora autoinstall, using serial console tests/vm: ubuntu.i386: apt proxy setup
Quoted 9 lines
tests/vm/basevm.py | 135 +++++++++++++++++++++++---- tests/vm/Makefile.include | 23 ++++- tests/vm/fedora | 189 ++++++++++++++++++++++++++++++++++++++ tests/vm/freebsd | 180 ++++++++++++++++++++++++++++++++++-- tests/vm/netbsd | 187 +++++++++++++++++++++++++++++++++++-- tests/vm/openbsd | 159 +++++++++++++++++++++++++++++--- tests/vm/ubuntu.i386 | 4 + 7 files changed, 825 insertions(+), 52 deletions(-) create mode 100755 tests/vm/fedora
--
Alex Bennée