runqemu: refactor it and remove machine knowledge
v5 cover runqemu: refactor it and remove machine knowledge
* V5
- Fix race issues when building multi image recipes at the same time, they might write qemuboo.py at the same time in the past. Now each image has a unique ${IMAGE_LINK_NAME}.runqemu.conf.
- qemuboot.py -> ${IMAGE_LINK_NAME}.runqemu.conf, I used "import qemuboot.py", but now we can't "import ${IMAGE_LINK_NAME}.runqemu.py" since IMAGE_LINK_NAME contains dash "-", so rename it to .conf and use configparser to read/write it.
* V4
- Now the conf files can use @TAP@, @ROOTFS@ and @PORT@ when set vars for runqemu, so all the arch info are gone. And also update comments in qemuboot.bbclass.
- Add var QB_SLIRP_OPT for slirp mode as Nathan suggested, also add QB_TAP_OPT whichi is for tap mode.
- Print "Try 'runqemu help' on how to use it" when failed to parse args as Nathan suggested.
- Fix device locking problems.
- Fix nativesdk-qemu-helper.
- Fix comments for qemurunner.py/qemutinyrunner.py
* V3
- Move the qemu boot settings out of tune files (e.g., arch-x86.inc) to avoid mixing BSP settings with the tunings as Mark suggested. and put them to <machine>.conf or <qemuboot>.inc file.
* V2
- Fixes for RP's comments
a) Remove a kvm commit message since qemu-kvm can only run target when its arch is the same as the host arch:
The kvm support is not limited on x86 or x86_64
b) Fix a typo in commit message: qemu-boot.bbclass -> qemuboot.bbclass, You may wonder why I use qemuboot.bbclass but not qemu-boot.bbclass, this is because the bbclass will save vars into qemuboot.conf, the qemuboot.conf can't be qemu-boot.py here, otherwise, "import qemu-boot.py" doesn't work since "-" is not a validate identifier, so use "qemuboot" to make them unify.
c) Reorder the patches to make sure they don't break the bisect build.
- Update commit message:
"Fix SLIRP mode" -> "Fix SLIRP mode, add help message, avoid mixing
with tap"
* V1
Previously, runqemu had hard coded machine knowledge, which limited its usage, for example, qemu can boot genericx86, but runqemu can't, we need edit runqemu/runqemu-internal a lot if we want to boot genericx86.
Now bsp conf files can set vars to make it can be boot by runqemu, and qemuboot.bbclass will save these info to DEPLOY_DIR_IMAGE/qemuboot.conf. Please see qemuboot.bbclass' comments on how to set the vars.
- Re-write it in python3, which can reduce lines from 1239 to about 750 lines
- All the machine knowledges are gone
- All of the TUN_ARCH knowledge are gone (FIXED in V4)
- All the previous options are preserved, and there is a new way to run
runqemu: (it doesn't need run "bitake -e" in such a case) $ runqemu tmp/deploy/images/qemux86
or:
$ runqemu tmp/deploy/images/qemuarm/<image>.ext4
or:
$ runqemu tmp/deploy/images/qemuarm/qemuboot.conf
- Fixed audio support, not limited on x86 or x86_64
- The kvm support is not limited on x86 or x86_64 (REMOVED by V2)
- Fix SLIRP mode (FIXED by V2)
- Fix NFS boot, it will extract <image>.tar.bz2 or tar.gz to DEPLOY_DIR_IMAGE/<image>-nfsroot when no NFS_DIR, and remove it after stop.
- More bsps can be boot, such as genericx86 and genericx86-64.
- The patch for qemuzynq, qemuzynqmp, qemumicroblaze has been sent to meta-xilinx' mailing list.
- I can't find any qemush4 bsp or how to build it, so it is not considered atm.
- Tested the following machines, all of them work well. genericx86 genericx86-64
qemuarm qemuarm64
qemumips qemumips64 qemuppc
qemux86 qemux86-64
qemuzynq qemuzynqmp
Tested ext4, nfs, slirp, vmdk, iso and cpio.gz when they are available.
And also test "bitbake core-image-sato -ctestimage" which uses runqemu, it works, well.
// Robert
The following changes since commit a7f1435c4c26237cdb55066c9f5408b4fdf016aa:
combo-layer: python3: fix UnicodeDecodeError (2016-08-24 13:58:24 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib rbt/runqemu
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/runqemu
Robert Yang (10):
qemuboot.bbclass: add it for runqemu qemuarm.conf: set vars for runqemu qemuarm64.conf: set vars for runqemu qemux86.conf/qemux86-64.conf: set vars for runqemu qemumips/qemumips64.conf: set vars for runqemu qemuppc.conf: set vars for runqemu
qemu.inc: inherit qemuboot.bbclass
runqemu: refactor it and remove machine knowledge
nativesdk-qemu-helper: fix for new runqemu qemurunner.py/qemutinyrunner.py: remove runqemu-internal
meta/classes/qemuboot.bbclass | 75 ++ meta/conf/machine/include/qemu.inc | 2 + meta/conf/machine/include/qemuboot-mips.inc | 7 + meta/conf/machine/include/qemuboot-x86.inc | 14 + meta/conf/machine/qemuarm.conf | 5 + meta/conf/machine/qemuarm64.conf | 13 + meta/conf/machine/qemumips.conf | 1 + meta/conf/machine/qemumips64.conf | 1 + meta/conf/machine/qemuppc.conf | 7 + meta/conf/machine/qemux86-64.conf | 1 + meta/conf/machine/qemux86.conf | 1 + meta/lib/oeqa/utils/qemurunner.py | 2 +- meta/lib/oeqa/utils/qemutinyrunner.py | 2 +- .../qemu/nativesdk-qemu-helper_1.0.bb | 9 +- scripts/runqemu | 1332 ++++++++++++-------- scripts/runqemu-internal | 739 ----------- 16 files changed, 941 insertions(+), 1270 deletions(-) create mode 100644 meta/classes/qemuboot.bbclass create mode 100644 meta/conf/machine/include/qemuboot-mips.inc create mode 100644 meta/conf/machine/include/qemuboot-x86.inc delete mode 100755 scripts/runqemu-internal
Signature · 1 line
2.9.0
PATCH v5 1/10 qemuboot.bbclass: add it for runqemu
It saves vars in ${DEPLOY_DIR_IMAGE}/<image>.qemuboot.conf, and runqemu will read it.
The bsp which can be boot by runqemu will inherit it.
--- meta/classes/qemuboot.bbclass | 75 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 meta/classes/qemuboot.bbclass
meta/classes/qemuboot.bbclass new file +75 -0
@@ -0,0 +1,75 @@ +# Help runqemu boot target board, "QB" means Qemu Boot, the following +# vars can be set in conf files, such as <bsp.conf> to make it can be +# boot by runqemu: +# +# QB_SYSTEM_NAME: qemu name, e.g., "qemu-system-i386" +# QB_OPT_APPEND: options to append to qemu, e.g., "-show-cursor" +# QB_DEFAULT_KERNEL: default kernel to boot, e.g., "bzImage" +# QB_DEFAULT_FSTYPE: default FSTYPE to boot, e.g., "ext4" +# QB_MEM: memory, e.g., "-m 512" +# QB_MACHINE: qemu machine, e.g., "-machine virt" +# QB_CPU: qemu cpu, e.g., "-cpu qemu32" +# QB_CPU_KVM: the similar to QB_CPU, but used when kvm, e.g., '-cpu kvm64', +# set it when support kvm. +# QB_KERNEL_CMDLINE_APPEND: options to append to kernel's -append +# option, e.g., "console=ttyS0 console=tty" +# QB_DTB: qemu dtb name +# QB_AUDIO_DRV: qemu audio driver, e.g., "alsa", set it when support audio +# QB_AUDIO_OPT: qemu audio option, e.g., "-soundhw ac97,es1370", used +# when QB_AUDIO_DRV is set. +# QB_KERNEL_ROOT: kernel's root, e.g., /dev/vda +# QB_TAP_OPT: netowrk option for 'tap' mode, e.g., +# "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no -device virtio-net-device,netdev=net0" +# Note, runqemu will replace "@TAP@" with the one which is used, such as tap0, tap1 ... +# QB_SLIRP_OPT: network option for SLIRP mode, e.g., +# "-netdev user,id=net0 -device virtio-net-device,netdev=net0" +# QB_ROOTFS_OPT: used as rootfs, e.g., +# "-drive id=disk0,file=@ROOTFS@,if=none,format=raw -device virtio-blk-device,drive=disk0" +# Note, runqemu will replace "@ROOTFS@" with the one which is used, such as core-image-minimal-qemuarm64.ext4. +# QB_SERIAL_OPT: serial port, e.g., "-serial mon:stdio" +# QB_TCPSERIAL_OPT: tcp serial port option, e.g., +# " -device virtio-serial-device -chardev socket,id=virtcon,port=@PORT@,host=127.0.0.1 -device virtconsole,chardev=virtcon" +# Note, runqemu will replace "@PORT@" with the port number which is used. +# +# Usage: +# IMAGE_CLASSES += "qemuboot" +# See "runqemu help" for more info + +QB_MEM ?= "-m 256" +QB_SERIAL_OPT ?= "-serial mon:stdio -serial null" +QB_DEFAULT_KERNEL ?= "${KERNEL_IMAGETYPE}" +QB_DEFAULT_FSTYPE ?= "ext4" +QB_OPT_APPEND ?= "-show-cursor" + +# Create qemuboot.conf +ROOTFS_POSTPROCESS_COMMAND += "write_qemuboot_conf; " + +python write_qemuboot_conf() { + import configparser + + build_vars = ['MACHINE', 'TUNE_ARCH', 'DEPLOY_DIR_IMAGE', \ + 'IMAGE_NAME', 'IMAGE_LINK_NAME', 'STAGING_DIR_NATIVE', \ + 'STAGING_BINDIR_NATIVE', 'STAGING_DIR_HOST'] + + # Vars from bsp + qb_vars = [] + for k in d.keys(): + if k.startswith('QB_'): + qb_vars.append(k) + + qemuboot = "%s/%s.qemuboot.conf" % (d.getVar('DEPLOY_DIR_IMAGE', True), d.getVar('IMAGE_NAME', True)) + qemuboot_link = "%s/%s.qemuboot.conf" % (d.getVar('DEPLOY_DIR_IMAGE', True), d.getVar('IMAGE_LINK_NAME', True)) + cf = configparser.ConfigParser() + cf.add_section('config_bsp') + for k in build_vars + qb_vars: + cf.set('config_bsp', k, '%s' % d.getVar(k, True)) + with open(qemuboot, 'w') as f: + cf.write(f) + + if d.getVar('RM_OLD_IMAGE', True) == "1" and os.path.exists(qemuboot_link): + os.remove(os.path.realpath(qemuboot_link)) + + if os.path.lexists(qemuboot_link): + os.remove(qemuboot_link) + os.symlink(os.path.basename(qemuboot), qemuboot_link) +}
Signature · 1 line
2.9.0
PATCH v5 2/10 qemuarm.conf: set vars for runqemu
In reply to Robert YangThese info are from old runqemu.
--- meta/conf/machine/qemuarm.conf | 5 +++++ 1 file changed, 5 insertions(+)
meta/conf/machine/qemuarm.conf +5 -0
@@ -10,3 +10,8 @@ KERNEL_IMAGETYPE = "zImage" SERIAL_CONSOLES = "115200;ttyAMA0 115200;ttyAMA1" +# For runqemu +QB_SYSTEM_NAME = "qemu-system-arm" +QB_MACHINE = "-machine versatilepb" +QB_KERNEL_CMDLINE_APPEND = "console=ttyAMA0,115200 console=tty" +QB_OPT_APPEND = "-show-cursor -usb -usbdevice tablet"
Signature · 1 line
2.9.0
PATCH v5 3/10 qemuarm64.conf: set vars for runqemu
In reply to Robert Yang--- meta/conf/machine/qemuarm64.conf | 13 +++++++++++++ 1 file changed, 13 insertions(+)
meta/conf/machine/qemuarm64.conf +13 -0
@@ -8,3 +8,16 @@ require conf/machine/include/qemu.inc KERNEL_IMAGETYPE = "Image" SERIAL_CONSOLES = "38400;ttyAMA0 38400;hvc0" + +# For runqemu +QB_SYSTEM_NAME = "qemu-system-aarch64" +QB_MEM = "-m 512" +QB_MACHINE = "-machine virt" +QB_CPU = "-cpu cortex-a57" +QB_KERNEL_CMDLINE_APPEND = "console=ttyAMA0,38400" +QB_OPT_APPEND = "-show-cursor" +QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no -device virtio-net-device,netdev=net0" +QB_SLIRP_OPT = "-netdev user,id=net0 -device virtio-net-device,netdev=net0" +QB_ROOTFS_OPT = "-drive id=disk0,file=@ROOTFS@,if=none,format=raw -device virtio-blk-device,drive=disk0" +QB_SERIAL_OPT = "-device virtio-serial-device -chardev null,id=virtcon -device virtconsole,chardev=virtcon" +QB_TCPSERIAL_OPT = " -device virtio-serial-device -chardev socket,id=virtcon,port=@PORT@,host=127.0.0.1 -device virtconsole,chardev=virtcon"
Signature · 1 line
2.9.0
PATCH v5 4/10 qemux86.conf/qemux86-64.conf: set vars for runqemu
In reply to Robert YangAdd qemuboot-x86.inc to reduce duplicated code, the x86/x86_64 bsps which can be boot by runqemu can require qemuboot-x86.inc.
--- meta/conf/machine/include/qemuboot-x86.inc | 14 ++++++++++++++ meta/conf/machine/qemux86-64.conf | 1 + meta/conf/machine/qemux86.conf | 1 + 3 files changed, 16 insertions(+) create mode 100644 meta/conf/machine/include/qemuboot-x86.inc
meta/conf/machine/include/qemuboot-x86.inc new file +14 -0
@@ -0,0 +1,14 @@ +# For runqemu +IMAGE_CLASSES += "qemuboot" +QB_SYSTEM_NAME_x86 = "qemu-system-i386" +QB_CPU_x86 = "-cpu qemu32" +QB_CPU_KVM_x86 = "-cpu kvm32" + +QB_SYSTEM_NAME_x86-64 = "qemu-system-x86_64" +QB_CPU_x86-64 = "-cpu core2duo" +QB_CPU_KVM_x86-64 = "-cpu kvm64" + +QB_AUDIO_DRV = "alsa" +QB_AUDIO_OPT = "-soundhw ac97,es1370" +QB_KERNEL_CMDLINE_APPEND = "vga=0 uvesafb.mode_option=640x480-32 oprofile.timer=1" +QB_OPT_APPEND = "-vga vmware -show-cursor -usb -usbdevice tablet"
meta/conf/machine/qemux86-64.conf +1 -0
@@ -10,6 +10,7 @@ PREFERRED_PROVIDER_virtual/libgles2 ?= "mesa" require conf/machine/include/qemu.inc DEFAULTTUNE ?= "core2-64" require conf/machine/include/tune-core2.inc +require conf/machine/include/qemuboot-x86.inc KERNEL_IMAGETYPE = "bzImage"
meta/conf/machine/qemux86.conf +1 -0
@@ -9,6 +9,7 @@ PREFERRED_PROVIDER_virtual/libgles2 ?= "mesa" require conf/machine/include/qemu.inc require conf/machine/include/tune-i586.inc +require conf/machine/include/qemuboot-x86.inc KERNEL_IMAGETYPE = "bzImage"
Signature · 1 line
2.9.0
PATCH v5 5/10 qemumips/qemumips64.conf: set vars for runqemu
In reply to Robert YangAdd qemuboot-mips.inc to reduce duplicated code, the various mips bsps which can be boot by runqemu can require qemuboot-mips.inc
--- meta/conf/machine/include/qemuboot-mips.inc | 7 +++++++ meta/conf/machine/qemumips.conf | 1 + meta/conf/machine/qemumips64.conf | 1 + 3 files changed, 9 insertions(+) create mode 100644 meta/conf/machine/include/qemuboot-mips.inc
meta/conf/machine/include/qemuboot-mips.inc new file +7 -0
@@ -0,0 +1,7 @@ +# For runqemu +IMAGE_CLASSES += "qemuboot" +QB_MEM = "-m 256" +QB_MACHINE = "-machine malta" +QB_KERNEL_CMDLINE_APPEND = "console=ttyS0 console=tty" +QB_OPT_APPEND = "-vga cirrus -show-cursor -usb -usbdevice tablet" +QB_SYSTEM_NAME = "qemu-system-${TUNE_ARCH}"
meta/conf/machine/qemumips.conf +1 -0
@@ -4,6 +4,7 @@ require conf/machine/include/qemu.inc require conf/machine/include/tune-mips32r2.inc +require conf/machine/include/qemuboot-mips.inc KERNEL_IMAGETYPE = "vmlinux" KERNEL_ALT_IMAGETYPE = "vmlinux.bin"
meta/conf/machine/qemumips64.conf +1 -0
@@ -4,6 +4,7 @@ require conf/machine/include/qemu.inc require conf/machine/include/tune-mips64.inc +require conf/machine/include/qemuboot-mips.inc KERNEL_IMAGETYPE = "vmlinux" KERNEL_ALT_IMAGETYPE = "vmlinux.bin"
Signature · 1 line
2.9.0
PATCH v5 6/10 qemuppc.conf: set vars for runqemu
In reply to Robert Yang--- meta/conf/machine/qemuppc.conf | 7 +++++++ 1 file changed, 7 insertions(+)
meta/conf/machine/qemuppc.conf +7 -0
@@ -11,3 +11,10 @@ KERNEL_IMAGETYPE = "vmlinux" SERIAL_CONSOLES = "115200;ttyS0 115200;ttyS1" +# For runqemu +QB_SYSTEM_NAME = "qemu-system-ppc" +QB_MACHINE = "-machine mac99" +QB_CPU = "-cpu G4" +QB_KERNEL_CMDLINE_APPEND = "console=ttyS0 console=tty" +QB_OPT_APPEND = "-show-cursor -usb -usbdevice tablet" +QB_TAP_OPT = "-net tap,vlan=0,ifname=@TAP@,script=no,downscript=no -net nic,model=pcnet"
Signature · 1 line
2.9.0
PATCH v5 7/10 qemu.inc: inherit qemuboot.bbclass
In reply to Robert YangAll qemu boards should be able to boot by runqemu.
--- meta/conf/machine/include/qemu.inc | 2 ++ 1 file changed, 2 insertions(+)
meta/conf/machine/include/qemu.inc +2 -0
@@ -29,3 +29,5 @@ EXTRA_IMAGEDEPENDS += "qemu-native qemu-helper-native" KERNEL_FEATURES_append_pn-linux-yocto = " features/nfsd/nfsd-enable.scc" MACHINE_EXTRA_RRECOMMENDS += "rng-tools" + +IMAGE_CLASSES += "qemuboot"
Signature · 1 line
2.9.0
PATCH v5 8/10 runqemu: refactor it and remove machine knowledge
In reply to Robert YangPreviously, runqemu had hard coded machine knowledge, which limited its usage, for example, qemu can boot genericx86, but runqemu can't, we need edit runqemu/runqemu-internal a lot if we want to boot genericx86.
Now bsp conf files can set vars to make it can be boot by runqemu, and qemuboot.bbclass will save these info to DEPLOY_DIR_IMAGE/qemuboot.conf. Please see qemuboot.bbclass' comments on how to set the vars.
* Re-write it in python3, which can reduce lines from 1239 to about 750 lines
* All the machine knowledges are gone
* All of the TUN_ARCH knowledge are gone
* All the previous options are preserved, and there is a new way to run
runqemu: (it doesn't need run "bitake -e" in such a case)
$ runqemu tmp/deploy/images/qemux86 or: $ runqemu tmp/deploy/images/qemuarm/<image>.ext4 or: $ runqemu tmp/deploy/images/qemuarm/qemuboot.conf * Fixed audio support, not limited on x86 or x86_64 * Fix SLIRP mode, add help message, avoid mixing with tap * Fix NFS boot, it will extract <image>.tar.bz2 or tar.gz to DEPLOY_DIR_IMAGE/<image>-nfsroot when no NFS_DIR, and remove it after stop. * More bsps can be boot, such as genericx86 and genericx86-64. * The patch for qemuzynq, qemuzynqmp, qemumicroblaze has been sent to meta-xilinx' mailing list. * I can't find any qemush4 bsp or how to build it, so it is not considered atm.
[YOCTO #1018]
[YOCTO #4827]
[YOCTO #7459]
[YOCTO #7887]
--- scripts/runqemu | 1332 ++++++++++++++++++++++++++++------------------ scripts/runqemu-internal | 739 ------------------------- 2 files changed, 807 insertions(+), 1264 deletions(-) delete mode 100755 scripts/runqemu-internal
scripts/runqemu +807 -525
@@ -1,8 +1,9 @@ -#!/bin/bash -# +#!/usr/bin/env python3 + # Handle running OE images standalone with QEMU # # Copyright (C) 2006-2011 Linux Foundation +# Copyright (c) 2016 Wind River Systems, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License version 2 as
@@ -17,536 +18,817 @@ # with this program; if not, write to the Free Software Foundation, Inc., # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -usage() { - MYNAME=`basename $0` -cat <<_EOF +import os +import sys +import logging +import subprocess +import re +import fcntl +import shutil +import glob +import configparser + +def create_logger(): + logger = logging.getLogger('runqemu') + logger.setLevel(logging.INFO) + + # create console handler and set level to debug + ch = logging.StreamHandler() + ch.setLevel(logging.INFO) + + # create formatter + formatter = logging.Formatter('%(name)s - %(levelname)s - %(message)s') + + # add formatter to ch + ch.setFormatter(formatter) + + # add ch to logger + logger.addHandler(ch) + + return logger + +logger = create_logger() +def print_usage(): + print(""" Usage: you can run this script with any valid combination of the following environment variables (in any order): KERNEL - the kernel image file to use ROOTFS - the rootfs image file or nfsroot directory to use MACHINE - the machine name (optional, autodetected from KERNEL filename if unspecified) Simplified QEMU command-line options can be passed with: - nographic - disables video console - serial - enables a serial console on /dev/ttyS0 - kvm - enables KVM when running qemux86/qemux86-64 (VT-capable CPU required) - kvm-vhost - enables KVM with vhost support when running qemux86/qemux86-64 (VT-capable CPU required) + nographic - disable video console + serial - enable a serial console on /dev/ttyS0 + slirp - enable user networking, no root privileges is required + kvm - enable KVM when running x86/x86_64 (VT-capable CPU required) + kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU required) publicvnc - enable a VNC server open to all hosts - qemuparams="xyz" - specify custom parameters to QEMU - bootparams="xyz" - specify custom kernel parameters during boot + audio - enable audio + tcpserial=<port> - specify tcp serial port number + biosdir=<dir> - specify custom bios dir + biosfilename=<filename> - specify bios filename + qemuparams=<xyz> - specify custom parameters to QEMU + bootparams=<xyz> - specify custom kernel parameters during boot + help: print this text Examples: - $MYNAME qemuarm - $MYNAME qemux86-64 core-image-sato ext4 - $MYNAME qemux86-64 wic-image-minimal wic - $MYNAME path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial - $MYNAME qemux86 iso/hddimg/vmdk/qcow2/vdi/ramfs/cpio.gz... - $MYNAME qemux86 qemuparams="-m 256" - $MYNAME qemux86 bootparams="psplash=false" - $MYNAME path/to/<image>-<machine>.vmdk - $MYNAME path/to/<image>-<machine>.wic -_EOF - exit 1 -} - -if [ "x$1" = "x" ]; then - usage -fi - -error() { - echo "Error: "$* - usage -} - -MACHINE=${MACHINE:=""} -KERNEL=${KERNEL:=""} -ROOTFS=${ROOTFS:=""} -FSTYPE=${FSTYPE:=""} -LAZY_ROOTFS="" -SCRIPT_QEMU_OPT="" -SCRIPT_QEMU_EXTRA_OPT="" -SCRIPT_KERNEL_OPT="" -SERIALSTDIO="" -TCPSERIAL_PORTNUM="" -KVM_ENABLED="no" -KVM_ACTIVE="no" -VHOST_ENABLED="no" -VHOST_ACTIVE="no" -IS_VM="false" - -# Determine whether the file is a kernel or QEMU image, and set the -# appropriate variables -process_filename() { - filename=$1 - - # Extract the filename extension - EXT=`echo $filename | awk -F . '{ print \$NF }'` - case /$EXT/ in - /bin/) - # A file ending in .bin is a kernel - [ -z "$KERNEL" ] && KERNEL=$filename || \ - error "conflicting KERNEL args [$KERNEL] and [$filename]" - ;; - /ext[234]/|/jffs2/|/btrfs/) - # A file ending in a supportted fs type is a rootfs image - if [ -z "$FSTYPE" -o "$FSTYPE" = "$EXT" ]; then - FSTYPE=$EXT - ROOTFS=$filename - else - error "conflicting FSTYPE types [$FSTYPE] and [$EXT]" - fi - ;; - /hddimg/|/hdddirect/|/vmdk/|/wic/|/qcow2/|/vdi/) - FSTYPE=$EXT - VM=$filename - ROOTFS=$filename - IS_VM="true" - ;; - *) - error "unknown file arg [$filename]" - ;; - esac -} - -check_fstype_conflicts() { - if [ -z "$FSTYPE" -o "$FSTYPE" = "$1" ]; then - FSTYPE=$1 - else - error "conflicting FSTYPE types [$FSTYPE] and [$1]" - fi -} -# Parse command line args without requiring specific ordering. It's a -# bit more complex, but offers a great user experience. -while true; do - arg=${1} - case "$arg" in - "qemux86" | "qemux86-64" | "qemuarm" | "qemuarm64" | "qemumips" | "qemumipsel" | \ - "qemumips64" | "qemush4" | "qemuppc" | "qemumicroblaze" | "qemuzynq" | "qemuzynqmp") - [ -z "$MACHINE" -o "$MACHINE" = "$arg" ] && MACHINE=$arg || \ - error "conflicting MACHINE types [$MACHINE] and [$arg]" - ;; - "ext"[234] | "jffs2" | "nfs" | "btrfs") - check_fstype_conflicts $arg - ;; - "hddimg" | "hdddirect" | "wic" | "vmdk" | "qcow2" | "vdi" | "iso") - check_fstype_conflicts $arg - IS_VM="true" - ;; - "ramfs" | "cpio.gz") - FSTYPE=cpio.gz - ;; - "nographic") - SCRIPT_QEMU_OPT="$SCRIPT_QEMU_OPT -nographic" - SCRIPT_KERNEL_OPT="$SCRIPT_KERNEL_OPT console=ttyS0" - ;; - "serial") - SCRIPT_QEMU_OPT="$SCRIPT_QEMU_OPT -serial stdio" - SCRIPT_KERNEL_OPT="$SCRIPT_KERNEL_OPT console=ttyS0" - SERIALSTDIO="1" - ;; - "tcpserial="*) - TCPSERIAL_PORTNUM=${arg##tcpserial=} - ;; - "biosdir="*) - CUSTOMBIOSDIR="${arg##biosdir=}" - ;; - "biosfilename="*) - SCRIPT_QEMU_OPT="$SCRIPT_QEMU_OPT -bios ${arg##biosfilename=}" - ;; - "qemuparams="*) - SCRIPT_QEMU_EXTRA_OPT="${arg##qemuparams=}" - - # Warn user if they try to specify serial or kvm options - # to use simplified options instead - serial_option=`expr "$SCRIPT_QEMU_EXTRA_OPT" : '.*\(-serial\)'` - kvm_option=`expr "$SCRIPT_QEMU_EXTRA_OPT" : '.*\(-enable-kvm\)'` - vga_option=`expr "$SCRIPT_QEMU_EXTRA_OPT" : '.*\(-vga\)'` - [ ! -z "$serial_option" -o ! -z "$kvm_option" ] && \ - echo "Please use simplified serial or kvm options instead" - ;; - "bootparams="*) - SCRIPT_KERNEL_OPT="$SCRIPT_KERNEL_OPT ${arg##bootparams=}" - ;; - "audio") - if [ "x$MACHINE" = "xqemux86" -o "x$MACHINE" = "xqemux86-64" ]; then - echo "Enabling audio in qemu." - echo "Please install snd_intel8x0 or snd_ens1370 driver in linux guest." - QEMU_AUDIO_DRV="alsa" - SCRIPT_QEMU_OPT="$SCRIPT_QEMU_OPT -soundhw ac97,es1370" - fi - ;; - "kvm") - KVM_ENABLED="yes" - KVM_CAPABLE=`grep -q 'vmx\|svm' /proc/cpuinfo && echo 1` - ;; - "kvm-vhost") - KVM_ENABLED="yes" - KVM_CAPABLE=`grep -q 'vmx\|svm' /proc/cpuinfo && echo 1` - VHOST_ENABLED="yes" - ;; - "slirp") - SLIRP_ENABLED="yes" - ;; - "publicvnc") - SCRIPT_QEMU_OPT="$SCRIPT_QEMU_OPT -vnc :0" - ;; - *-image*) - [ -z "$ROOTFS" ] || \ - error "conflicting ROOTFS args [$ROOTFS] and [$arg]" - if [ -f "$arg" ]; then - process_filename $arg - elif [ -d "$arg" ]; then - # Handle the case where the nfsroot dir has -image- - # in the pathname - echo "Assuming $arg is an nfs rootfs" - FSTYPE=nfs - ROOTFS=$arg - else - ROOTFS=$arg - LAZY_ROOTFS="true" - fi - ;; - "") break ;; - *) - # A directory name is an nfs rootfs - if [ -d "$arg" ]; then - echo "Assuming $arg is an nfs rootfs" - if [ -z "$FSTYPE" -o "$FSTYPE" = "nfs" ]; then - FSTYPE=nfs - else - error "conflicting FSTYPE types [$arg] and nfs" - fi - - if [ -z "$ROOTFS" ]; then - ROOTFS=$arg - else - error "conflicting ROOTFS args [$ROOTFS] and [$arg]" - fi - elif [ -f "$arg" ]; then - process_filename $arg - else - error "unable to classify arg [$arg]" - fi - ;; - esac - shift -done - -if [ ! -c /dev/net/tun ] ; then - echo "TUN control device /dev/net/tun is unavailable; you may need to enable TUN (e.g. sudo modprobe tun)" - exit 1 -elif [ ! -w /dev/net/tun ] ; then - echo "TUN control device /dev/net/tun is not writable, please fix (e.g. sudo chmod 666 /dev/net/tun)" - exit 1 -fi - -# Report errors for missing combinations of options -if [ -z "$MACHINE" -a -z "$KERNEL" -a -z "$VM" -a "$FSTYPE" != "wic" ]; then - error "you must specify at least a MACHINE or KERNEL argument" -fi -if [ "$FSTYPE" = "nfs" -a -z "$ROOTFS" ]; then - error "NFS booting without an explicit ROOTFS path is not yet supported" -fi - -if [ -z "$MACHINE" ]; then - if [ "$IS_VM" = "true" ]; then - [ "x$FSTYPE" = "xwic" ] && filename=$ROOTFS || filename=$VM - MACHINE=`basename $filename | sed -n 's/.*\(qemux86-64\|qemux86\|qemuarm64\|qemuarm\|qemumips64\|qemumips\|qemuppc\|qemush4\).*/\1/p'` - if [ -z "$MACHINE" ]; then - error "Unable to set MACHINE from image filename [$VM]" - fi - echo "Set MACHINE to [$MACHINE] based on image [$VM]" - else - MACHINE=`basename $KERNEL | sed -n 's/.*\(qemux86-64\|qemux86\|qemuarm64\|qemuarm\|qemumips64\|qemumips\|qemuppc\|qemush4\).*/\1/p'` - if [ -z "$MACHINE" ]; then - error "Unable to set MACHINE from kernel filename [$KERNEL]" - fi - echo "Set MACHINE to [$MACHINE] based on kernel [$KERNEL]" - fi -fi - -YOCTO_KVM_WIKI="https://wiki.yoctoproject.org/wiki/How_to_enable_KVM_for_Poky_qemu" -YOCTO_PARAVIRT_KVM_WIKI="https://wiki.yoctoproject.org/wiki/Running_an_x86_Yocto_Linux_image_under_QEMU_KVM" -# Detect KVM configuration -if [ "x$KVM_ENABLED" = "xyes" ]; then - if [ -z "$KVM_CAPABLE" ]; then - echo "You are trying to enable KVM on a cpu without VT support." - echo "Remove kvm from the command-line, or refer" - echo "$YOCTO_KVM_WIKI"; - exit 1; - fi - if [ "x$MACHINE" != "xqemux86" -a "x$MACHINE" != "xqemux86-64" ]; then - echo "KVM only support x86 & x86-64. Remove kvm from the command-line"; - exit 1; - fi - if [ ! -e /dev/kvm ]; then - echo "Missing KVM device. Have you inserted kvm modules?" - echo "For further help see:" - echo "$YOCTO_KVM_WIKI"; - exit 1; - fi - if [ -w /dev/kvm -a -r /dev/kvm ]; then - SCRIPT_QEMU_OPT="$SCRIPT_QEMU_OPT -enable-kvm" - KVM_ACTIVE="yes" - else - echo "You have no rights on /dev/kvm." - echo "Please change the ownership of this file as described at:" - echo "$YOCTO_KVM_WIKI"; - exit 1; - fi - if [ "x$VHOST_ENABLED" = "xyes" ]; then - if [ ! -e /dev/vhost-net ]; then - echo "Missing virtio net device. Have you inserted vhost-net module?" - echo "For further help see:" - echo "$YOCTO_PARAVIRT_KVM_WIKI"; - exit 1; - fi - - if [ -w /dev/vhost-net -a -r /dev/vhost-net ]; then - VHOST_ACTIVE="yes" - else - echo "You have no rights on /dev/vhost-net." - echo "Please change the ownership of this file as described at:" - echo "$YOCTO_KVM_WIKI"; - exit 1; - fi - fi -fi - -machine2=`echo $MACHINE | tr 'a-z' 'A-Z' | sed 's/-/_/'` -# MACHINE is now set for all cases - -# Defaults used when these vars need to be inferred -QEMUX86_DEFAULT_KERNEL=bzImage-qemux86.bin -QEMUX86_DEFAULT_FSTYPE=ext4 - -QEMUX86_64_DEFAULT_KERNEL=bzImage-qemux86-64.bin -QEMUX86_64_DEFAULT_FSTYPE=ext4 - -QEMUARM_DEFAULT_KERNEL=zImage-qemuarm.bin -QEMUARM_DEFAULT_FSTYPE=ext4 - -QEMUARM64_DEFAULT_KERNEL=Image-qemuarm64.bin -QEMUARM64_DEFAULT_FSTYPE=ext4 - -QEMUMIPS_DEFAULT_KERNEL=vmlinux-qemumips.bin -QEMUMIPS_DEFAULT_FSTYPE=ext4 - -QEMUMIPSEL_DEFAULT_KERNEL=vmlinux-qemumipsel.bin -QEMUMIPSEL_DEFAULT_FSTYPE=ext4 - -QEMUMIPS64_DEFAULT_KERNEL=vmlinux-qemumips64.bin -QEMUMIPS64_DEFAULT_FSTYPE=ext4 - -QEMUSH4_DEFAULT_KERNEL=vmlinux-qemumips.bin -QEMUSH4_DEFAULT_FSTYPE=ext4 - -QEMUPPC_DEFAULT_KERNEL=vmlinux-qemuppc.bin -QEMUPPC_DEFAULT_FSTYPE=ext4 - -QEMUMICROBLAZE_DEFAULT_KERNEL=linux.bin.ub -QEMUMICROBLAZE_DEFAULT_FSTYPE=cpio - -QEMUZYNQ_DEFAULT_KERNEL=uImage -QEMUZYNQ_DEFAULT_FSTYPE=cpio - -QEMUZYNQMP_DEFAULT_KERNEL=Image -QEMUZYNQMP_DEFAULT_FSTYPE=cpio - -setup_path_vars() { - if [ -z "$OE_TMPDIR" ] ; then - PATHS_REQUIRED=true - elif [ "$1" = "1" -a -z "$DEPLOY_DIR_IMAGE" ] ; then - PATHS_REQUIRED=true - else - PATHS_REQUIRED=false - fi - - if [ "$PATHS_REQUIRED" = "true" ]; then - # Try to get the variable values from bitbake - type -P bitbake &>/dev/null || { - echo "In order for this script to dynamically infer paths"; - echo "to kernels or filesystem images, you either need"; - echo "bitbake in your PATH or to source oe-init-build-env"; - echo "before running this script" >&2; - exit 1; } - - # We have bitbake in PATH, get the variable values from bitbake - BITBAKE_ENV_TMPFILE=`mktemp --tmpdir runqemu.XXXXXXXXXX` - if [ "$?" != "0" ] ; then - echo "Error: mktemp failed for bitbake environment output" - exit 1 - fi - - MACHINE=$MACHINE bitbake -e > $BITBAKE_ENV_TMPFILE - if [ -z "$OE_TMPDIR" ] ; then - OE_TMPDIR=`sed -n 's/^TMPDIR=\"\(.*\)\"/\1/p' $BITBAKE_ENV_TMPFILE` - fi - if [ -z "$DEPLOY_DIR_IMAGE" ] ; then - DEPLOY_DIR_IMAGE=`sed -n 's/^DEPLOY_DIR_IMAGE=\"\(.*\)\"/\1/p' $BITBAKE_ENV_TMPFILE` - fi - if [ -z "$QEMU_DTB" ] ; then - QEMU_DTB=`sed -n 's/^QEMU_DTB=\"\(.*\)\"/\1/p' $BITBAKE_ENV_TMPFILE` - fi - if [ -z "$OE_TMPDIR" ]; then - # Check for errors from bitbake that the user needs to know about - BITBAKE_OUTPUT=`cat $BITBAKE_ENV_TMPFILE | wc -l` - if [ "$BITBAKE_OUTPUT" -eq "0" ]; then - echo "Error: this script needs to be run from your build directory, or you need" - echo "to explicitly set OE_TMPDIR and DEPLOY_DIR_IMAGE in your environment" - else - echo "There was an error running bitbake to determine TMPDIR" - echo "Here is the output from 'bitbake -e':" - cat $BITBAKE_ENV_TMPFILE - fi - rm $BITBAKE_ENV_TMPFILE - exit 1 - fi - rm $BITBAKE_ENV_TMPFILE - fi -} - -setup_sysroot() { - # Toolchain installs set up $OECORE_NATIVE_SYSROOT in their - # environment script. If that variable isn't set, we're - # either in an in-tree build scenario or the environment - # script wasn't source'd. - if [ -z "$OECORE_NATIVE_SYSROOT" ]; then - setup_path_vars - BUILD_ARCH=`uname -m` - BUILD_OS=`uname | tr '[A-Z]' '[a-z]'` - BUILD_SYS="$BUILD_ARCH-$BUILD_OS" - - OECORE_NATIVE_SYSROOT=$OE_TMPDIR/sysroots/$BUILD_SYS - fi - - # Some recipes store the BIOS under $OE_TMPDIR/sysroots/$MACHINE, - # now defined as OECORE_MACHINE_SYSROOT. The latter is used when searching - # BIOS, VGA BIOS and keymaps. - if [ -z "$OECORE_MACHINE_SYSROOT" ]; then - OECORE_MACHINE_SYSROOT=$OE_TMPDIR/sysroots/$MACHINE - fi -} - -# Locate a rootfs image to boot which matches our expected -# machine and fstype. -findimage() { - where=$1 - machine=$2 - extension=$3 - - # Sort rootfs candidates by modification time - the most - # recently created one is the one we most likely want to boot. - filename=`ls -t1 $where/*-image*$machine.$extension 2>/dev/null | head -n1` - if [ "x$filename" != "x" ]; then - ROOTFS=$filename - return - fi - - echo "Couldn't find a $machine rootfs image in $where." - exit 1 -} - -if [ -e "$ROOTFS" -a -z "$FSTYPE" ]; then - # Extract the filename extension - EXT=`echo $ROOTFS | awk -F . '{ print \$NF }'` - if [ "x$EXT" = "xext2" -o "x$EXT" = "xext3" -o \ - "x$EXT" = "xjffs2" -o "x$EXT" = "xbtrfs" -o \ - "x$EXT" = "xext4" ]; then - FSTYPE=$EXT - else - echo "Note: Unable to determine filesystem extension for $ROOTFS" - echo "We will use the default FSTYPE for $MACHINE" - # ...which is done further below... - fi -fi - -if [ -z "$KERNEL" -a "$IS_VM" = "false" ]; then \ - setup_path_vars 1 - eval kernel_file=\$${machine2}_DEFAULT_KERNEL - KERNEL=$DEPLOY_DIR_IMAGE/$kernel_file - - if [ -z "$KERNEL" ]; then - error "Unable to determine default kernel for MACHINE [$MACHINE]" - fi -fi -# KERNEL is now set for all cases - -if [ -z "$FSTYPE" ]; then - eval FSTYPE=\$${machine2}_DEFAULT_FSTYPE - - if [ -z "$FSTYPE" ]; then - error "Unable to determine default fstype for MACHINE [$MACHINE]" - fi -fi - -# FSTYPE is now set for all cases - -# Handle cases where a ROOTFS type is given instead of a filename, e.g. -# core-image-sato -if [ "$LAZY_ROOTFS" = "true" ]; then - setup_path_vars 1 - echo "Assuming $ROOTFS really means $DEPLOY_DIR_IMAGE/$ROOTFS-$MACHINE.$FSTYPE" - if [ "$IS_VM" = "true" ]; then - VM=$DEPLOY_DIR_IMAGE/$ROOTFS-$MACHINE.$FSTYPE - else - ROOTFS=$DEPLOY_DIR_IMAGE/$ROOTFS-$MACHINE.$FSTYPE - fi -fi - -if [ -z "$ROOTFS" ]; then - setup_path_vars 1 - T=$DEPLOY_DIR_IMAGE - eval rootfs_list=\$${machine2}_DEFAULT_ROOTFS - findimage $T $MACHINE $FSTYPE - - if [ -z "$ROOTFS" ]; then - error "Unable to determine default rootfs for MACHINE [$MACHINE]" - elif [ "$IS_VM" = "true" ]; then - VM=$ROOTFS - fi -fi -# ROOTFS is now set for all cases, now expand it to be an absolute path, it should exist at this point - -ROOTFS=`readlink -f $ROOTFS` - -echo "" -echo "Continuing with the following parameters:" -if [ "$IS_VM" = "false" ]; then - echo "KERNEL: [$KERNEL]" - echo "ROOTFS: [$ROOTFS]" -else - echo "VM: [$VM]" -fi -echo "FSTYPE: [$FSTYPE]" - -setup_sysroot -# OECORE_NATIVE_SYSROOT and OECORE_MACHINE_SYSROOT are now set for all cases - -INTERNAL_SCRIPT="$0-internal" -if [ ! -f "$INTERNAL_SCRIPT" -o ! -r "$INTERNAL_SCRIPT" ]; then -INTERNAL_SCRIPT=`which runqemu-internal` -fi - -# Specify directory for BIOS, VGA BIOS and keymaps -if [ ! -z "$CUSTOMBIOSDIR" ]; then - if [ -d "$OECORE_NATIVE_SYSROOT/$CUSTOMBIOSDIR" ]; then - echo "Assuming biosdir is $OECORE_NATIVE_SYSROOT/$CUSTOMBIOSDIR" - SCRIPT_QEMU_OPT="$SCRIPT_QEMU_OPT -L $OECORE_NATIVE_SYSROOT/$CUSTOMBIOSDIR" - elif [ -d "$OECORE_MACHINE_SYSROOT/$CUSTOMBIOSDIR" ]; then - echo "Assuming biosdir is $OECORE_MACHINE_SYSROOT/$CUSTOMBIOSDIR" - SCRIPT_QEMU_OPT="$SCRIPT_QEMU_OPT -L $OECORE_MACHINE_SYSROOT/$CUSTOMBIOSDIR" - else - if [ ! -d "$CUSTOMBIOSDIR" ]; then - echo "Custom BIOS directory not found. Tried: $CUSTOMBIOSDIR" - echo "and $OECORE_NATIVE_SYSROOT/$CUSTOMBIOSDIR" - echo "and $OECORE_MACHINE_SYSROOT/$CUSTOMBIOSDIR" - exit 1; - fi - echo "Assuming biosdir is $CUSTOMBIOSDIR" - SCRIPT_QEMU_OPT="$SCRIPT_QEMU_OPT -L $CUSTOMBIOSDIR" - fi -fi - -. $INTERNAL_SCRIPT -exit $? + runqemu qemuarm + runqemu tmp/deploy/images/qemuarm + runqemu tmp/deploy/images/qemux86/.qemuboot.conf + runqemu qemux86-64 core-image-sato ext4 + runqemu qemux86-64 wic-image-minimal wic + runqemu path/to/bzImage-qemux86.bin path/to/nfsrootdir/ serial + runqemu qemux86 iso/hddimg/vmdk/qcow2/vdi/ramfs/cpio.gz... + runqemu qemux86 qemuparams="-m 256" + runqemu qemux86 bootparams="psplash=false" + runqemu path/to/<image>-<machine>.vmdk + runqemu path/to/<image>-<machine>.wic +""") + +def check_tun(): + """Check /dev/net/run""" + dev_tun = '/dev/net/tun' + if not os.path.exists(dev_tun): + raise Exception("TUN control device %s is unavailable; you may need to enable TUN (e.g. sudo modprobe tun)" % dev_tun) + + if not os.access(dev_tun, os.W_OK): + raise Exception("TUN control device %s is not writable, please fix (e.g. sudo chmod 666 %s)" % (dev_tun, dev_tun)) + +def check_libgl(qemu_bin): + cmd = 'ldd %s' % qemu_bin + logger.info('Running %s...' % cmd) + need_gl = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8') + if re.search('libGLU', need_gl): + # We can't run without a libGL.so + libgl = False + check_files = (('/usr/lib/libGL.so', '/usr/lib/libGLU.so'), \ + ('/usr/lib64/libGL.so', '/usr/lib64/libGLU.so'), \ + ('/usr/lib/*-linux-gnu/libGL.so', '/usr/lib/*-linux-gnu/libGLU.so')) + + for (f1, f2) in check_files: + if re.search('\*', f1): + for g1 in glob.glob(f1): + if libgl: + break + if os.path.exists(g1): + for g2 in glob.glob(f2): + if os.path.exists(g2): + libgl = True + break + if libgl: + break + else: + if os.path.exists(f1) and os.path.exists(f2): + libgl = True + break + if not libgl: + logger.error("You need libGL.so and libGLU.so to exist in your library path to run the QEMU emulator.") + logger.error("Ubuntu package names are: libgl1-mesa-dev and libglu1-mesa-dev.") + logger.error("Fedora package names are: mesa-libGL-devel mesa-libGLU-devel.") + raise Exception('%s requires libGLU, but not found' % qemu_bin) + +class BaseConfig(object): + def __init__(self): + # Vars can be merged with .qemuboot.conf, use a dict to manage them. + self.d = { + 'MACHINE': '', + 'DEPLOY_DIR_IMAGE': '', + 'QB_KERNEL_ROOT': '/dev/vda', + } + + self.qemu_opt = '' + self.qemu_opt_script = '' + self.nfs_dir = '' + self.clean_nfs_dir = False + self.nfs_server = '' + self.rootfs = '' + self.qemuboot = '' + self.kernel = '' + self.kernel_cmdline = '' + self.kernel_cmdline_script = '' + self.fstype = '' + self.kvm_enabled = False + self.vhost_enabled = False + self.slirp_enabled = False + self.nfs_instance = 0 + self.nfs_running = False + self.serialstdio = False + self.cleantap = False + self.saved_stty = '' + self.audio_enabled = False + self.tcpserial_portnum = '' + self.custombiosdir = '' + self.lock = '' + self.lock_descriptor = '' + self. bitbake_e = '' + self.fstypes = ('ext2', 'ext3', 'ext4', 'jffs2', 'nfs', 'btrfs', 'cpio.gz', 'cpio', 'ramfs') + self.vmtypes = ('hddimg', 'hdddirect', 'wic', 'vmdk', 'qcow2', 'vdi', 'iso') + + def acquire_lock(self): + logger.info("Acquiring lockfile %s..." % self.lock) + lock_descriptor = open(self.lock, 'w') + try: + fcntl.flock(lock_descriptor, fcntl.LOCK_EX|fcntl.LOCK_NB) + except Exception as e: + logger.info("Acquiring lockfile %s failed: %s" % (self.lock, e)) + lock_descriptor.close() + return False + self.lock_descriptor = lock_descriptor + return True + + def release_lock(self): + fcntl.flock(self.lock_descriptor, fcntl.LOCK_UN) + self.lock_descriptor.close() + os.remove(self.lock) + + def get(self, key): + if key in self.d: + return self.d.get(key) + else: + return '' + + def set(self, key, value): + self.d[key] = value + + def is_deploy_dir_image(self, p): + if os.path.isdir(p): + if not re.search('.qemuboot.conf$', '\n'.join(os.listdir(p)), re.M): + logger.info("Can't find required qemuboot.conf in %s" % p) + return False + if not re.search('-image-', '\n'.join(os.listdir(p))): + logger.info("Can't find *-image-* in %s" % p) + return False + return True + else: + return False + + def check_arg_fstype(self, fst): + """Check and set FSTYPE""" + if fst not in self.fstypes + self.vmtypes: + logger.warn("Maybe unsupported FSTYPE: %s" % fst) + if not self.fstype or self.fstype == fst: + if fst == 'ramfs': + fst = 'cpio.gz' + self.fstype = fst + else: + raise Exception("Conflicting: FSTYPE %s and %s" % (self.fstype, fst)) + + def set_machine_deploy_dir(self, machine, deploy_dir_image): + """Set MACHINE and DEPLOY_DIR_IMAGE""" + logger.info('MACHINE: %s' % machine) + self.set("MACHINE", machine) + logger.info('DEPLOY_DIR_IMAGE: %s' % deploy_dir_image) + self.set("DEPLOY_DIR_IMAGE", deploy_dir_image) + + def check_arg_nfs(self, p): + if os.path.isdir(p): + self.nfs_dir = p + else: + m = re.match('(.*):(.*)', p) + self.nfs_server = m.group(1) + self.nfs_dir = m.group(2) + self.rootfs = "" + self.check_arg_fstype('nfs') + + def check_arg_path(self, p): + """ + - Check whether it is <image>.qemuboot.conf or contains <image>.qemuboot.conf + - Check whether is a kernel file + - Check whether is a image file + - Check whether it is a nfs dir + """ + if p.endswith('.qemuboot.conf'): + self.qemuboot = p + elif re.search('\.bin$', p) or re.search('bzImage', p) or \ + re.search('zImage', p) or re.search('vmlinux', p) or \ + re.search('fitImage', p) or re.search('uImage', p): + self.kernel = p + elif os.path.exists(p) and (not os.path.isdir(p)) and re.search('-image-', os.path.basename(p)): + self.rootfs = p + dirpath = os.path.dirname(p) + m = re.search('(.*)\.(.*)$', p) + if m: + qb = '%s%s' % (re.sub('\.rootfs$', '', m.group(1)), '.qemuboot.conf') + if os.path.exists(qb): + self.qemuboot = qb + else: + logger.warn("%s doesn't exist" % qb) + fst = m.group(2) + self.check_arg_fstype(fst) + else: + raise Exception("Can't find FSTYPE from: %s" % p) + elif os.path.isdir(p) or re.search(':', arg) and re.search('/', arg): + if self.is_deploy_dir_image(p): + logger.info('DEPLOY_DIR_IMAGE: %s' % p) + self.set("DEPLOY_DIR_IMAGE", p) + else: + logger.info("Assuming %s is an nfs rootfs" % p) + self.check_arg_nfs(p) + else: + raise Exception("Unknown path arg %s" % p) + + def check_arg_machine(self, arg): + """Check whether it is a machine""" + if self.get('MACHINE') and self.get('MACHINE') != arg or re.search('/', arg): + raise Exception("Unknown arg: %s" % arg) + elif self.get('MACHINE') == arg: + return + logger.info('Assuming MACHINE = %s' % arg) + cmd = 'MACHINE=%s bitbake -e' % arg + logger.info('Running %s...' % cmd) + self.bitbake_e = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8') + # bitbake -e doesn't report invalid MACHINE as an error, so + # let's check DEPLOY_DIR_IMAGE to make sure that it is a valid + # MACHINE. + s = re.search('^DEPLOY_DIR_IMAGE="(.*)"', self.bitbake_e, re.M) + if s: + deploy_dir_image = s.group(1) + else: + raise Exception("bitbake -e %s" % self.bitbake_e) + if self.is_deploy_dir_image(deploy_dir_image): + self.set_machine_deploy_dir(arg, deploy_dir_image) + else: + loger.error("%s not a directory valid DEPLOY_DIR_IMAGE" % deploy_dir_image) + raise Exception("Falied to set MACHINE to %s. Unknown arg: %s" % (arg, arg)) + + def check_args(self): + unknown_arg = "" + for arg in sys.argv[1:]: + if arg in self.fstypes + self.vmtypes: + self.check_arg_fstype(arg) + elif arg == 'nographic': + self.qemu_opt_script += ' -nographic' + self.kernel_cmdline_script += ' console=ttyS0' + elif arg == 'serial': + self.kernel_cmdline_script += ' console=ttyS0' + self.serialstdio = True + elif arg == 'audio': + logger.info("Enabling audio in qemu") + logger.info("Please install sound drivers in linux host") + self.audio_enabled = True + elif arg == 'kvm': + self.kvm_enabled = True + elif arg == 'kvm-vhost': + self.vhost_enabled = True + elif arg == 'slirp': + self.slirp_enabled = True + elif arg == 'publicvnc': + self.qemu_opt_script += ' -vnc :0' + elif arg.startswith('tcpserial='): + self.tcpserial_portnum = arg[len('tcpserial='):] + elif arg.startswith('biosdir='): + self.custombiosdir = arg[len('biosdir='):] + elif arg.startswith('biosfilename='): + self.qemu_opt_script += ' -bios %s' % arg[len('biosfilename='):] + elif arg.startswith('qemuparams='): + self.qemu_opt_script += ' %s' % arg[len('qemuparams='):] + elif arg.startswith('bootparams='): + self.kernel_cmdline_script += ' %s' % arg[len('bootparams='):] + elif os.path.exists(arg) or (re.search(':', arg) and re.search('/', arg)): + self.check_arg_path(os.path.abspath(arg)) + elif re.search('-image-', arg): + # Lazy rootfs + self.rootfs = arg + else: + # At last, assume is it the MACHINE + if (not unknown_arg) or unknown_arg == arg: + unknown_arg = arg + else: + raise Exception("Can't handle two unknown args: %s %s" % (unknown_arg, arg)) + # Check to make sure it is a valid machine + if unknown_arg: + if self.get('MACHINE') == unknown_arg: + return + if not self.get('DEPLOY_DIR_IMAGE'): + # Trying to get DEPLOY_DIR_IMAGE from env. + p = os.getenv('DEPLOY_DIR_IMAGE') + if p and self.is_deploy_dir_image(p): + machine = os.path.basename(p) + if unknown_arg == machine: + self.set_machine_deploy_dir(machine, p) + return + else: + logger.info('DEPLOY_DIR_IMAGE: %s' % p) + self.set("DEPLOY_DIR_IMAGE", p) + self.check_arg_machine(unknown_arg) + + def check_kvm(self): + """Check kvm and kvm-host""" + if not (self.kvm_enabled or self.vhost_enabled): + self.qemu_opt_script += ' %s %s' % (self.get('QB_MACHINE'), self.get('QB_CPU')) + return + + if not self.get('QB_CPU_KVM'): + raise Exception("QB_CPU_KVM is NULL, this board doesn't support kvm") + + self.qemu_opt_script += ' %s %s' % (self.get('QB_MACHINE'), self.get('QB_CPU_KVM')) + yocto_kvm_wiki = "https://wiki.yoctoproject.org/wiki/How_to_enable_KVM_for_Poky_qemu" + yocto_paravirt_kvm_wiki = "https://wiki.yoctoproject.org/wiki/Running_an_x86_Yocto_Linux_image_under_QEMU_KVM" + dev_kvm = '/dev/kvm' + dev_vhost = '/dev/vhost-net' + with open('/proc/cpuinfo', 'r') as f: + kvm_cap = re.search('vmx|svm', "".join(f.readlines())) + if not kvm_cap: + logger.error("You are trying to enable KVM on a cpu without VT support.") + logger.error("Remove kvm from the command-line, or refer:") + raise Exception(yocto_kvm_wiki) + + if not os.path.exists(dev_kvm): + logger.error("Missing KVM device. Have you inserted kvm modules?") + logger.error("For further help see:") + raise Exception(yocto_kvm_wiki) + + if os.access(dev_kvm, os.W_OK|os.R_OK): + self.qemu_opt_script += ' -enable-kvm' + else: + logger.error("You have no read or write permission on /dev/kvm.") + logger.error("Please change the ownership of this file as described at:") + raise Exception(yocto_kvm_wiki) + + if self.vhost_enabled: + if not os.path.exists(dev_vhost): + logger.error("Missing virtio net device. Have you inserted vhost-net module?") + logger.error("For further help see:") + raise Exception(yocto_paravirt_kvm_wiki) + + if not os.access(dev_kvm, os.W_OK|os.R_OK): + logger.error("You have no read or write permission on /dev/vhost-net.") + logger.error("Please change the ownership of this file as described at:") + raise Exception(yocto_kvm_wiki) + + def check_fstype(self): + """Check and setup FSTYPE""" + if not self.fstype: + fstype = self.get('QB_DEFAULT_FSTYPE') + if fstype: + self.fstype = fstype + else: + raise Exception("FSTYPE is NULL!") + + def check_rootfs(self): + """Check and set rootfs""" + + if self.fstype == 'nfs': + return + + if self.rootfs and not os.path.exists(self.rootfs): + # Lazy rootfs + self.rootfs = "%s/%s-%s.%s" % (self.get('DEPLOY_DIR_IMAGE'), + self.rootfs, self.get('MACHINE'), + self.fstype) + elif not self.rootfs: + cmd = '%s/%s*.%s' % (self.get('DEPLOY_DIR_IMAGE'), self.get('IMAGE_NAME'), self.fstype) + all_files = glob.glob(cmd) + if all_files: + self.rootfs = all_files[0] + else: + raise Exception("Failed to find rootfs: %s" % cmd) + + if not os.path.exists(self.rootfs): + raise Exception("Can't find rootfs: %s" % self.rootfs) + + def check_kernel(self): + """Check and set kernel, dtb""" + # The vm image doesn't need a kernel + if self.fstype in self.vmtypes: + return + kernel = self.kernel + if not kernel: + kernel = "%s/%s" % (self.get('DEPLOY_DIR_IMAGE'), self.get('QB_DEFAULT_KERNEL')) + + if os.path.exists(kernel): + self.kernel = kernel + else: + raise Exception("KERNEL %s not found" % kernel) + + dtb = self.get('QB_DTB') + if dtb: + dtb = "%s/%s" % (self.get('DEPLOY_DIR_IMAGE'), dtb) + if os.path.exists(dtb): + self.set('QB_DTB', '-dtb %s' % dtb) + else: + raise Exception("DTB %s not found" % dtb) + + + def check_biosdir(self): + """Check custombiosdir""" + if not self.custombiosdir: + return + + biosdir = "" + biosdir_native = "%s/%s" % (self.get('STAGING_DIR_NATIVE'), self.custombiosdir) + biosdir_host = "%s/%s" % (self.get('STAGING_DIR_HOST'), self.custombiosdir) + for i in (self.custombiosdir, biosdir_native, biosdir_host): + if os.path.isdir(i): + biosdir = i + break + + if biosdir: + logger.info("Assuming biosdir is: %s" % biosdir) + self.qemu_opt_script += ' -L %s' % biosdir + else: + logger.error("Custom BIOS directory not found. Tried: %s, %s, and %s" % (self.custombiosdir, biosdir_native, biosdir_host)) + raise Exception("Invalid custombiosdir: %s" % self.custombiosdir) + + def check_mem(self): + s = re.search('-m +([0-9]+)', self.qemu_opt_script) + if s: + self.set('QB_MEM', '-m %s' % s.group(1)) + elif not self.get('QB_MEM'): + logger.info('QB_MEM is not set, use 512M by default') + self.set('QB_MEM', '-m 512') + + self.kernel_cmdline_script += ' mem=%s' % self.get('QB_MEM').replace('-m','').strip() + 'M' + self.qemu_opt_script += ' %s' % self.get('QB_MEM') + + def check_tcpserial(self): + if self.tcpserial_portnum: + if self.get('QB_TCPSERIAL_OPT'): + self.qemu_opt_script += self.get('QB_TCPSERIAL_OPT').replace('@PORT@', self.tcpserial_portnum) + else: + self.qemu_opt_script += ' -serial tcp:127.0.0.1:%s' % self.tcpserial_portnum + + def check_and_set(self): + """Check configs sanity and set when needed""" + check_tun() + # Check audio + if self.audio_enabled: + if not self.get('QB_AUDIO_DRV'): + raise Exception("QB_AUDIO_DRV is NULL, this board doesn't support audio") + if not self.get('QB_AUDIO_OPT'): + logger.warn('QB_AUDIO_OPT is NULL, you may need define it to make audio work') + else: + self.qemu_opt_script += ' %s' % self.get('QB_AUDIO_OPT') + os.putenv('QEMU_AUDIO_DRV', self.get('QB_AUDIO_DRV')) + else: + os.putenv('QEMU_AUDIO_DRV', 'none') + + self.check_kvm() + self.check_fstype() + self.check_rootfs() + self.check_kernel() + self.check_biosdir() + self.check_mem() + self.check_tcpserial() + + def read_qemuboot(self): + if not self.qemuboot: + if self.get('DEPLOY_DIR_IMAGE'): + deploy_dir_image = self.get('DEPLOY_DIR_IMAGE') + elif os.getenv('DEPLOY_DIR_IMAGE'): + deploy_dir_image = os.getenv('DEPLOY_DIR_IMAGE') + else: + raise Exception("DEPLOY_DIR_IMAGE is NULL!") + + if self.rootfs and not os.path.exists(self.rootfs): + # Lazy rootfs + machine = self.get('MACHINE') + if not machine: + machine = os.path.basename(deploy_dir_image) + self.qemuboot = "%s/%s-%s.qemuboot.conf" % (deploy_dir_image, + self.rootfs, machine) + else: + cmd = 'ls -t %s/*.qemuboot.conf' % deploy_dir_image + logger.info('Running %s...' % cmd) + qbs = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8') + if qbs: + self.qemuboot = qbs.split()[0] + + if not os.path.exists(self.qemuboot): + raise Exception("Failed to find <image>.qemuboot.conf!") + + logger.info('CONFFILE: %s' % self.qemuboot) + + cf = configparser.ConfigParser() + cf.read(self.qemuboot) + for k, v in cf.items('config_bsp'): + k_upper = k.upper() + self.set(k_upper, v) + + def print_config(self): + logger.info('Continuing with the following parameters:\n') + if not self.fstype in self.vmtypes: + print('KERNEL: [%s]' % self.kernel) + print('MACHINE: [%s]' % self.get('MACHINE')) + print('FSTYPE: [%s]' % self.fstype) + if self.fstype == 'nfs': + print('NFS_DIR: [%s]' % self.nfs_dir) + else: + print('ROOTFS: [%s]' % self.rootfs) + print('CONFFILE: [%s]' % self.qemuboot) + print('') + + def setup_nfs(self): + if not self.nfs_server: + if self.slirp_enabled: + self.nfs_server = '10.0.2.2' + else: + self.nfs_server = '192.168.7.1' + + nfs_instance = int(self.nfs_instance) + + mountd_rpcport = 21111 + nfs_instance + nfsd_rpcport = 11111 + nfs_instance + nfsd_port = 3049 + 2 * nfs_instance + mountd_port = 3048 + 2 * nfs_instance + unfs_opts="nfsvers=3,port=%s,mountprog=%s,nfsprog=%s,udp,mountport=%s" % (nfsd_port, mountd_rpcport, nfsd_rpcport, mountd_port) + self.unfs_opts = unfs_opts + + p = '%s/.runqemu-sdk/pseudo' % os.getenv('HOME') + os.putenv('PSEUDO_LOCALSTATEDIR', p) + + # Extract .tar.bz2 or .tar.bz if no self.nfs_dir + if not self.nfs_dir: + src_prefix = '%s/%s' % (self.get('DEPLOY_DIR_IMAGE'), self.get('IMAGE_LINK_NAME')) + dest = "%s-nfsroot" % src_prefix + if os.path.exists('%s.pseudo_state' % dest): + logger.info('Use %s as NFS_DIR' % dest) + self.nfs_dir = dest + else: + src = "" + src1 = '%s.tar.bz2' % src_prefix + src2 = '%s.tar.gz' % src_prefix + if os.path.exists(src1): + src = src1 + elif os.path.exists(src2): + src = src2 + if not src: + raise Exception("No NFS_DIR is set but can't find %s or %s to extract" % (src1, src2)) + logger.info('NFS_DIR not found, extracting %s to %s' % (src, dest)) + cmd = 'runqemu-extract-sdk %s %s' % (src, dest) + logger.info('Running %s...' % cmd) + if subprocess.call(cmd, shell=True) != 0: + raise Exception('Failed to run %s' % cmd) + self.clean_nfs_dir = True + self.nfs_dir = dest + + # Start the userspace NFS server + cmd = 'runqemu-export-rootfs restart %s' % self.nfs_dir + logger.info('Running %s...' % cmd) + if subprocess.call(cmd, shell=True) != 0: + raise Exception('Failed to run %s' % cmd) + + self.nfs_running = True + + + def setup_slirp(self): + if self.fstype == 'nfs': + self.setup_nfs() + self.kernel_cmdline_script += ' ip=dhcp' + self.set('NETWORK_CMD', self.get('QB_SLIRP_OPT')) + + def setup_tap(self): + """Setup tap""" + + # This file is created when runqemu-gen-tapdevs creates a bank of tap + # devices, indicating that the user should not bring up new ones using + # sudo. + nosudo_flag = '/etc/runqemu-nosudo' + self.qemuifup = shutil.which('runqemu-ifup') + self.qemuifdown = shutil.which('runqemu-ifdown') + ip = shutil.which('ip') + lockdir = "/tmp/qemu-tap-locks" + + if not (self.qemuifup and self.qemuifdown and ip): + raise Exception("runqemu-ifup, runqemu-ifdown or ip not found") + + if not os.path.exists(lockdir): + os.mkdir(lockdir) + + cmd = '%s link' % ip + logger.info('Running %s...' % cmd) + ip_link = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8') + # Matches line like: 6: tap0: <foo> + possibles = re.findall('^[1-9]+: +(tap[0-9]+): <.*', ip_link, re.M) + tap = "" + for p in possibles: + lockfile = os.path.join(lockdir, p) + if os.path.exists('%s.skip' % lockfile): + logger.info('Found %s.skip, skipping %s' % (lockfile, p)) + continue + self.lock = lockfile + '.lock' + if self.acquire_lock(): + tap = p + logger.info("Using preconfigured tap device %s" % tap) + logger.info("If this is not intended, touch %s.skip to make runqemu skip %s." %(lockfile, tap)) + break + + if not tap: + if os.path.exists(nosudo_flag): + logger.error("Error: There are no available tap devices to use for networking,") + logger.error("and I see %s exists, so I am not going to try creating" % nosudo_flag) + raise Exception("a new one with sudo.") + + gid = os.getgid() + uid = os.getuid() + logger.info("Setting up tap interface under sudo") + cmd = 'sudo %s %s %s %s' % (self.qemuifup, uid, gid, self.get('STAGING_DIR_NATIVE')) + tap = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8').rstrip('\n') + lockfile = os.path.join(lockdir, tap) + self.lock = lockfile + '.lock' + self.acquire_lock() + self.cleantap = True + logger.info('Created tap: %s' % tap) + + self.tap = tap + n0 = tap[3:] + n1 = int(n0) * 2 + 1 + n2 = n1 + 1 + self.nfs_instance = n0 + if self.fstype == 'nfs': + self.setup_nfs() + self.kernel_cmdline_script += " ip=192.168.7.%s::192.168.7.%s:255.255.255.0" % (n2, n1) + qb_tap_opt = self.get('QB_TAP_OPT') + if qb_tap_opt: + qemu_tap_opt = qb_tap_opt.replace('@TAP@', tap) + else: + qemu_tap_opt = "-net nic,model=virtio -net tap,vlan=0,ifname=%s,script=no,downscript=no" % self.tap + + if self.vhost_enabled: + qemu_tap_opt += ',vhost=on' + + self.set('NETWORK_CMD', qemu_tap_opt) + + def setup_network(self): + cmd = "stty -g" + self.saved_stty = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8') + if self.slirp_enabled: + self.setup_slirp() + else: + self.setup_tap() + + qb_rootfs_opt = self.get('QB_ROOTFS_OPT') + if qb_rootfs_opt: + self.rootfs_options = qb_rootfs_opt.replace('@ROOTFS@', self.rootfs) + else: + self.rootfs_options = '-drive file=%s,if=virtio,format=raw' % self.rootfs + + if self.fstype in ('cpio.gz', 'cpio'): + self.set('NETWORK_CMD', '') + self.kernel_cmdline = 'root=/dev/ram0 rw debugshell' + self.rootfs_options = '-initrd %s' % self.rootfs + else: + if self.fstype in self.vmtypes: + if self.fstype == 'iso': + vm_drive = '-cdrom %s' % self.rootfs + else: + cmd1 = "grep -q 'root=/dev/sd' %s" % self.rootfs + cmd2 = "grep -q 'root=/dev/hd' %s" % self.rootfs + if subprocess.call(cmd1, shell=True) == 0: + logger.info('Using scsi drive') + vm_drive = '-drive if=none,id=hd,file=%s -device virtio-scsi-pci,id=scsi -device scsi-hd,drive=hd' % self.rootfs + elif subprocess.call(cmd2, shell=True) == 0: + logger.info('Using scsi drive') + vm_drive = self.rootfs + else: + logger.warn("Can't detect drive type %s" % self.rootfs) + logger.warn('Tring to use virtio block drive') + vm_drive = '-drive if=virtio,file=%s' % self.rootfs + self.rootfs_options = '%s -no-reboot' % vm_drive + self.kernel_cmdline = 'root=%s rw highres=off' % (self.get('QB_KERNEL_ROOT')) + + if self.fstype == 'nfs': + self.rootfs_options = '' + k_root = '/dev/nfs nfsroot=%s:%s,%s' % (self.nfs_server, self.nfs_dir, self.unfs_opts) + self.kernel_cmdline = 'root=%s rw highres=off' % k_root + + self.set('ROOTFS_OPTIONS', self.rootfs_options) + + def setup_final(self): + qemu_system = self.get('QB_SYSTEM_NAME') + if not qemu_system: + raise Exception("Failed to boot, QB_SYSTEM_NAME is NULL!") + + qemu_bin = '%s/%s' % (self.get('STAGING_BINDIR_NATIVE'), qemu_system) + if not os.access(qemu_bin, os.X_OK): + raise Exception("No QEMU binary '%s' could be found" % qemu_bin) + + check_libgl(qemu_bin) + + self.qemu_opt = "%s %s %s %s %s %s" % (qemu_bin, self.get('NETWORK_CMD'), self.qemu_opt_script, self.get('ROOTFS_OPTIONS'), self.get('QB_DTB'), self.get('QB_OPT_APPEND')) + + if self.serialstdio: + logger.info("Interrupt character is '^]'") + cmd = "stty intr ^]" + subprocess.call(cmd, shell=True) + + first_serial = "" + if not re.search("-nographic", self.qemu_opt): + first_serial = "-serial mon:vc" + # We always want a ttyS1. Since qemu by default adds a serial + # port when nodefaults is not specified, it seems that all that + # would be needed is to make sure a "-serial" is there. However, + # it appears that when "-serial" is specified, it ignores the + # default serial port that is normally added. So here we make + # sure to add two -serial if there are none. And only one if + # there is one -serial already. + serial_num = len(re.findall("-serial", self.qemu_opt)) + if serial_num == 0: + self.qemu_opt += " %s %s" % (first_serial, self.get("QB_SERIAL_OPT")) + elif serial_num == 1: + self.qemu_opt += " %s" % self.get("QB_SERIAL_OPT") + + def start_qemu(self): + if self.kernel: + kernel_opts = "-kernel %s -append '%s %s %s'" % (self.kernel, self.kernel_cmdline, self.kernel_cmdline_script, self.get('QB_KERNEL_CMDLINE_APPEND')) + else: + kernel_opts = "" + cmd = "%s %s" % (self.qemu_opt, kernel_opts) + logger.info('Running %s' % cmd) + if subprocess.call(cmd, shell=True) != 0: + raise Exception('Failed to run %s' % cmd) + + def cleanup(self): + if self.cleantap: + cmd = 'sudo %s %s %s' % (self.qemuifdown, self.tap, self.get('STAGING_DIR_NATIVE')) + logger.info('Running %s' % cmd) + subprocess.call(cmd, shell=True) + if self.lock_descriptor: + logger.info("Releasing lockfile for tap device '%s'" % self.tap) + self.release_lock() + + if self.nfs_running: + logger.info("Shutting down the userspace NFS server...") + cmd = "runqemu-export-rootfs stop %s" % self.nfs_dir + logger.info('Running %s' % cmd) + subprocess.call(cmd, shell=True) + + if self.saved_stty: + cmd = "stty %s" % self.saved_stty + subprocess.call(cmd, shell=True) + + if self.clean_nfs_dir: + logger.info('Removing %s' % self.nfs_dir) + shutil.rmtree(self.nfs_dir) + shutil.rmtree('%s.pseudo_state' % self.nfs_dir) + +def main(): + if len(sys.argv) == 1 or "help" in sys.argv: + print_usage() + return 0 + config = BaseConfig() + try: + config.check_args() + except Exception as esc: + logger.error(esc) + logger.error("Try 'runqemu help' on how to use it") + return 1 + config.read_qemuboot() + config.check_and_set() + config.print_config() + try: + config.setup_network() + config.setup_final() + config.start_qemu() + finally: + config.cleanup() + return 0 + +if __name__ == "__main__": + try: + ret = main() + except Exception as esc: + ret = 1 + import traceback + traceback.print_exc() + sys.exit(ret)
scripts/runqemu-internal deleted +0 -739
739 lines not on this page · raw message
Signature · 1 line
2.9.0
PATCH v5 9/10 nativesdk-qemu-helper: fix for new runqemu
In reply to Robert YangThere is no runqemu-internal anymore, and it is a python script now which requires several python modules.
--- meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
meta/recipes-devtools/qemu/nativesdk-qemu-helper_1.0.bb +5 -4
@@ -1,15 +1,16 @@ SUMMARY = "Qemu helper scripts" LICENSE = "GPLv2" -RDEPENDS_${PN} = "nativesdk-qemu" -PR = "r9" +RDEPENDS_${PN} = "nativesdk-qemu nativesdk-python3-subprocess \ + nativesdk-python3-shell nativesdk-python3-fcntl \ + " +PR = "r9" LIC_FILES_CHKSUM = "file://${WORKDIR}/tunctl.c;endline=4;md5=ff3a09996bc5fff6bc5d4e0b4c28f999 \ - file://${COREBASE}/scripts/runqemu;endline=18;md5=77fbe442a88b1bcdc29c3ba67733b21b" + file://${COREBASE}/scripts/runqemu;endline=19;md5=a8ad1905c709a2deaa057608ebaf705a" SRC_URI = "file://${COREBASE}/scripts/runqemu \ - file://${COREBASE}/scripts/runqemu-internal \ file://${COREBASE}/scripts/runqemu-addptable2image \ file://${COREBASE}/scripts/runqemu-gen-tapdevs \ file://${COREBASE}/scripts/runqemu-ifup \
Signature · 1 line
2.9.0
PATCH v5 10/10 qemurunner.py/qemutinyrunner.py: remove runqemu-internal
In reply to Robert YangThere is no runqemu-internal any more.
--- meta/lib/oeqa/utils/qemurunner.py | 2 +- meta/lib/oeqa/utils/qemutinyrunner.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +1 -1
@@ -375,7 +375,7 @@ class QemuRunner: newparents = next #print("Children matching %s:" % str(parents)) for p in parents: - # Need to be careful here since runqemu-internal runs "ldd qemu-system-xxxx" + # Need to be careful here since runqemu runs "ldd qemu-system-xxxx" # Also, old versions of ldd (2.11) run "LD_XXXX qemu-system-xxxx" basecmd = commands[p].split()[0] basecmd = os.path.basename(basecmd)
meta/lib/oeqa/utils/qemutinyrunner.py +1 -1
@@ -162,7 +162,7 @@ class QemuTinyRunner(QemuRunner): newparents = next #print("Children matching %s:" % str(parents)) for p in parents: - # Need to be careful here since runqemu-internal runs "ldd qemu-system-xxxx" + # Need to be careful here since runqemu runs "ldd qemu-system-xxxx" # Also, old versions of ldd (2.11) run "LD_XXXX qemu-system-xxxx" basecmd = commands[p].split()[0] basecmd = os.path.basename(basecmd)
Signature · 1 line
2.9.0
Hi Robert,
So close! qemuarm64 is breaking on the AB:
I suspect we're passing some options to runqemu that you didn't cover in your testing.
Ross
Quoted 150 lines · Robert Yang
* V5
- Fix race issues when building multi image recipes at the same time,
they
might writeqemuboo.pyat the same time in the past. Now each image has a unique ${IMAGE_LINK_NAME}.runqemu.conf.-
qemuboot.py-> ${IMAGE_LINK_NAME}.runqemu.conf, I used "importqemuboot.py",
but now we can't "import ${IMAGE_LINK_NAME}.runqemu.py" sinceIMAGE_LINK_NAMEcontains dash "-", so rename it to .conf and use
configparser
to read/write it.* V4
- Now the conf files can use @TAP@, @ROOTFS@ and @PORT@ when set vars for runqemu, so all the arch info are gone. And also update comments in qemuboot.bbclass.
- Add varQB_SLIRP_OPTfor slirp mode as Nathan suggested, also addQB_TAP_OPTwhichi is for tap mode.
- Print "Try 'runqemu help' on how to use it" when failed to parse args as Nathan suggested.
- Fix device locking problems.
- Fix nativesdk-qemu-helper.
- Fix comments forqemurunner.py/qemutinyrunner.py* V3
- Move the qemu boot settings out of tune files (e.g., arch-x86.inc) to avoid mixing BSP settings with the tunings as Mark suggested. and put them to <machine>.conf or <qemuboot>.inc file.* V2
- Fixes for RP's comments
a) Remove a kvm commit message since qemu-kvm can only run target when its arch is the same as the host arch:
The kvm support is not limited on x86 orx86_64b) Fix a typo in commit message: qemu-boot.bbclass -> qemuboot.bbclass, You may wonder why I use qemuboot.bbclass but not qemu-boot.bbclass, this is because the bbclass will save vars into qemuboot.conf, the qemuboot.conf can't be
qemu-boot.pyhere, otherwise, "importqemu-boot.py" doesn't work since "-" is not a validate identifier, so use "qemuboot" to make them unify.c) Reorder the patches to make sure they don't break the bisect build.
- Update commit message:
"Fix SLIRP mode" -> "Fix SLIRP mode, add help message, avoid mixingwith tap"
* V1
Previously, runqemu had hard coded machine knowledge, which limited its usage, for example, qemu can boot genericx86, but runqemu can't, we need edit runqemu/runqemu-internal a lot if we want to boot genericx86.Now bsp conf files can set vars to make it can be boot by runqemu, and qemuboot.bbclass will save these info to
DEPLOY_DIR_IMAGE/qemuboot.conf. Please see qemuboot.bbclass' comments on how to set the vars.- Re-write it in python3, which can reduce lines from 1239 to about 750 lines
- All the machine knowledges are gone
- All of theTUN_ARCHknowledge are gone (FIXED in V4)
- All the previous options are preserved, and there is a new way to run
runqemu: (it doesn't need run "bitake -e" in such a case) $ runqemu tmp/deploy/images/qemux86
or:
$ runqemu tmp/deploy/images/qemuarm/<image>.ext4
or:
$ runqemutmp/deploy/images/qemuarm/qemuboot.conf
- Fixed audio support, not limited on x86 orx86_64
- The kvm support is not limited on x86 orx86_64(REMOVED by V2)
- Fix SLIRP mode (FIXED by V2)
- Fix NFS boot, it will extract <image>.tar.bz2 or tar.gz to DEPLOY_DIR_IMAGE/<image>-nfsroot when noNFS_DIR, and remove it after stop.
- More bsps can be boot, such as genericx86 and genericx86-64.
- The patch for qemuzynq, qemuzynqmp, qemumicroblaze has been sent to meta-xilinx' mailing list.
- I can't find any qemush4 bsp or how to build it, so it is not considered atm.- Tested the following machines, all of them work well. genericx86 genericx86-64
qemuarm qemuarm64
qemumips qemumips64 qemuppc
qemux86 qemux86-64
qemuzynq qemuzynqmpTested ext4, nfs, slirp, vmdk, iso and cpio.gz when they are available.
And also test "bitbake core-image-sato -ctestimage" which uses runqemu, it works, well.
// Robert
The following changes since commit
a7f1435c4c26237cdb55066c9f5408b4fdf016aa:combo-layer: python3: fix UnicodeDecodeError (2016-08-24 13:58:24 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contribrbt/runqemu
http://cgit.openembedded.org/cgit.cgi/openembedded-core-
contrib/log/?h=rbt/runqemuRobert Yang (10):
qemuboot.bbclass: add it for runqemu qemuarm.conf: set vars for runqemu qemuarm64.conf: set vars for runqemuqemux86.conf/qemux86-64.conf: set vars for runqemuqemumips/qemumips64.conf: set vars for runqemu qemuppc.conf: set vars for runqemu
qemu.inc: inherit qemuboot.bbclass
runqemu: refactor it and remove machine knowledge
nativesdk-qemu-helper: fix for new runqemuqemurunner.py/qemutinyrunner.py: remove runqemu-internalmeta/classes/qemuboot.bbclass | 75 ++ meta/conf/machine/include/qemu.inc | 2 + meta/conf/machine/include/qemuboot-mips.inc | 7 + meta/conf/machine/include/qemuboot-x86.inc | 14 + meta/conf/machine/qemuarm.conf | 5 + meta/conf/machine/qemuarm64.conf | 13 + meta/conf/machine/qemumips.conf | 1 + meta/conf/machine/qemumips64.conf | 1 + meta/conf/machine/qemuppc.conf | 7 + meta/conf/machine/qemux86-64.conf | 1 + meta/conf/machine/qemux86.conf | 1 + meta/lib/oeqa/utils/qemurunner.py | 2 +- meta/lib/oeqa/utils/qemutinyrunner.py | 2 +- .../qemu/nativesdk-qemu-helper_1.0.bb | 9 +- scripts/runqemu | 1332 ++++++++++++-------- scripts/runqemu-internal | 739 ----------- 16 files changed, 941 insertions(+), 1270 deletions(-) create mode 100644 meta/classes/qemuboot.bbclass create mode 100644 meta/conf/machine/include/qemuboot-mips.inc create mode 100644 meta/conf/machine/include/qemuboot-x86.inc delete mode 100755 scripts/runqemu-internal--
2.9.0--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
On Thu, 2016-08-25 at 22:45 +0100, Burton, Ross wrote:
Hi Robert,
So close! qemuarm64 is breaking on the AB:
I suspect we're passing some options to runqemu that you didn't cover in your testing.
Ross
It looks like it's just a missing space between the -m arg and -device:
-m 512-device
-Bill
On 25 August 2016 at 15:40, Robert Yang <[email protected]<mailto:[email protected]>> wrote:
* V5
- Fix race issues when building multi image recipes at the same time, they
might write qemuboo.py at the same time in the past. Now each image has
a unique ${IMAGE_LINK_NAME}.runqemu.conf.
- qemuboot.py -> ${IMAGE_LINK_NAME}.runqemu.conf, I used "import qemuboot.py", but now we can't "import ${IMAGE_LINK_NAME}.runqemu.py" since IMAGE_LINK_NAME contains dash "-", so rename it to .conf and use configparser to read/write it.
* V4
- Now the conf files can use @TAP@, @ROOTFS@ and @PORT@ when set vars
for runqemu, so all the arch info are gone. And also update comments
in qemuboot.bbclass.
- Add var QB_SLIRP_OPT for slirp mode as Nathan suggested, also add QB_TAP_OPT whichi is for tap mode.
- Print "Try 'runqemu help' on how to use it" when failed to parse args
as Nathan suggested.
- Fix device locking problems.
- Fix nativesdk-qemu-helper.
- Fix comments for qemurunner.py/qemutinyrunner.py<http://qemurunner.py/qemutinyrunner.py>
* V3
- Move the qemu boot settings out of tune files (e.g., arch-x86.inc) to avoid mixing BSP settings with the tunings as Mark suggested. and put them to <machine>.conf or <qemuboot>.inc file.
* V2
- Fixes for RP's comments
a) Remove a kvm commit message since qemu-kvm can only run target when its arch is the same as the host arch:
The kvm support is not limited on x86 or x86_64
b) Fix a typo in commit message: qemu-boot.bbclass -> qemuboot.bbclass, You may wonder why I use qemuboot.bbclass but not qemu-boot.bbclass, this is because the bbclass will save vars into qemuboot.conf, the qemuboot.conf can't be qemu-boot.py here, otherwise, "import qemu-boot.py" doesn't work since "-" is not a validate identifier, so use "qemuboot" to make them unify.
c) Reorder the patches to make sure they don't break the bisect build.
- Update commit message:
"Fix SLIRP mode" -> "Fix SLIRP mode, add help message, avoid mixing
with tap"
* V1
Previously, runqemu had hard coded machine knowledge, which limited its usage, for example, qemu can boot genericx86, but runqemu can't, we need edit runqemu/runqemu-internal a lot if we want to boot genericx86.
Now bsp conf files can set vars to make it can be boot by runqemu, and qemuboot.bbclass will save these info to DEPLOY_DIR_IMAGE/qemuboot.conf. Please see qemuboot.bbclass' comments on how to set the vars.
- Re-write it in python3, which can reduce lines from 1239 to about 750 lines
- All the machine knowledges are gone
- All of the TUN_ARCH knowledge are gone (FIXED in V4)
- All the previous options are preserved, and there is a new way to run
runqemu: (it doesn't need run "bitake -e" in such a case) $ runqemu tmp/deploy/images/qemux86
or:
$ runqemu tmp/deploy/images/qemuarm/<image>.ext4
or:
$ runqemu tmp/deploy/images/qemuarm/qemuboot.conf
- Fixed audio support, not limited on x86 or x86_64
- The kvm support is not limited on x86 or x86_64 (REMOVED by V2)
- Fix SLIRP mode (FIXED by V2)
- Fix NFS boot, it will extract <image>.tar.bz2 or tar.gz to DEPLOY_DIR_IMAGE/<image>-nfsroot when no NFS_DIR, and remove it after stop.
- More bsps can be boot, such as genericx86 and genericx86-64.
- The patch for qemuzynq, qemuzynqmp, qemumicroblaze has been sent to meta-xilinx' mailing list.
- I can't find any qemush4 bsp or how to build it, so it is not considered atm.
- Tested the following machines, all of them work well. genericx86 genericx86-64
qemuarm qemuarm64
qemumips qemumips64 qemuppc
qemux86 qemux86-64
qemuzynq qemuzynqmp
Tested ext4, nfs, slirp, vmdk, iso and cpio.gz when they are available.
And also test "bitbake core-image-sato -ctestimage" which uses runqemu, it works, well.
// Robert
The following changes since commit a7f1435c4c26237cdb55066c9f5408b4fdf016aa:
combo-layer: python3: fix UnicodeDecodeError (2016-08-24 13:58:24 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib<http://git.openembedded.org/openembedded-core-contrib> rbt/runqemu
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/runqemu
Robert Yang (10):
qemuboot.bbclass: add it for runqemu
qemuarm.conf: set vars for runqemu
qemuarm64.conf: set vars for runqemu
qemux86.conf/qemux86-64.conf: set vars for runqemu
qemumips/qemumips64.conf: set vars for runqemu
qemuppc.conf: set vars for runqemu
qemu.inc: inherit qemuboot.bbclass
runqemu: refactor it and remove machine knowledge
nativesdk-qemu-helper: fix for new runqemu qemurunner.py/qemutinyrunner.py<http://qemurunner.py/qemutinyrunner.py>: remove runqemu-internal
meta/classes/qemuboot.bbclass | 75 ++ meta/conf/machine/include/qemu.inc | 2 + meta/conf/machine/include/qemuboot-mips.inc | 7 + meta/conf/machine/include/qemuboot-x86.inc | 14 + meta/conf/machine/qemuarm.conf | 5 + meta/conf/machine/qemuarm64.conf | 13 + meta/conf/machine/qemumips.conf | 1 + meta/conf/machine/qemumips64.conf | 1 + meta/conf/machine/qemuppc.conf | 7 + meta/conf/machine/qemux86-64.conf | 1 + meta/conf/machine/qemux86.conf | 1 + meta/lib/oeqa/utils/qemurunner.py | 2 +- meta/lib/oeqa/utils/qemutinyrunner.py | 2 +- .../qemu/nativesdk-qemu-helper_1.0.bb<http://nativesdk-qemu-helper_1.0.bb> | 9 +- scripts/runqemu | 1332 ++++++++++++-------- scripts/runqemu-internal | 739 ----------- 16 files changed, 941 insertions(+), 1270 deletions(-) create mode 100644 meta/classes/qemuboot.bbclass create mode 100644 meta/conf/machine/include/qemuboot-mips.inc create mode 100644 meta/conf/machine/include/qemuboot-x86.inc delete mode 100755 scripts/runqemu-internal
--
2.9.0
--
_______________________________________________
Openembedded-core mailing list [email protected]<mailto:[email protected]>
http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
_______________________________________________
Openembedded-core mailing list [email protected]<mailto:[email protected]>
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Quoted 8 lines · William C
Quoted 7 lines · Ross
Hi Robert,
So close! qemuarm64 is breaking on the AB:
https://autobuilder.yoctoproject.org/main/builders/nightly-arm64/bu
ilds/536/steps/Running%20Sanity%20Tests/logs/stdio
Quoted 8 lines
Quoted 5 lines
I suspect we're passing some options to runqemu that you didn't cover in your testing.
Ross
It looks like it's just a missing space between the -m arg and -device:
-m 512-device
I added an attempt at a fix for this into -next FWIW
Cheers,
Richard
Quoted 12 lines · Richard Purdie
Quoted 11 lines · William C
Quoted 10 lines · Ross
Hi Robert,
So close! qemuarm64 is breaking on the AB:
https://autobuilder.yoctoproject.org/main/builders/nightly-arm64/bu
ilds/536/steps/Running%20Sanity%20Tests/logs/stdioI suspect we're passing some options to runqemu that you didn't cover in your testing.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Ross
It looks like it's just a missing space between the -m arg and -device:
-m 512-deviceI added an attempt at a fix for this into -next FWIW
Thanks, I had fixed this in the repo, and the fix is the same as RP's, I should reply to this thread, too, but sorry I replied to another thread:
Re: [OE-core] [PATCH V4 00/11] runqemu: refactor it and remove machine knowledge
// Robert
Quoted 5 lines
Cheers,
Richard