Skip to contentSkip to conversation

runqemu: 5 fixes

PATCH In review Robert Yang 1 participant 8 messages Jul 18, 2017 – Jul 26, 2017

lore
Share & tools
lore
#0 lore raw

cover runqemu: 5 fixes

The following changes since commit ef68005a8c527e9b1d05b7769f0ec8ebe9ec3f91:

webkitgtk: Upgrade to 2.16.5 (2017-07-17 13:49:04 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/runqemu
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/runqemu

Robert Yang (5):
runqemu: add --debug and --quiet
runqemu: check qbconfload before running bitbake
runqemu: check tar.bz2 and .tar.gz
runqemu: validate combos
runqemu: chmod 0o777 for lockdir

 scripts/runqemu | 64 ++++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 43 insertions(+), 21 deletions(-)
Signature · 1 line
2.11.0
#1 lore raw

PATCH 1/5 runqemu: add --debug and --quiet

And move some debug info into logger.debug(), this can make it easy to read key messages like errors or warnings.

I checked meta/lib/oeqa/ they don't depend on these messages. And I have run "oe-selftest -a", it doesn't break anything.

[YOCTO #10474]

---
 scripts/runqemu | 50 +++++++++++++++++++++++++++++++-------------------
 1 file changed, 31 insertions(+), 19 deletions(-)
scripts/runqemu +31 -19
index d44afc7e7a1..7188bd78830 100755
@@ -48,7 +48,7 @@ def create_logger():
 
     # create console handler and set level to debug
     ch = logging.StreamHandler()
-    ch.setLevel(logging.INFO)
+    ch.setLevel(logging.DEBUG)
 
     # create formatter
     formatter = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
@@ -85,6 +85,8 @@ of the following environment variables (in any order):
   qemuparams=<xyz> - specify custom parameters to QEMU
   bootparams=<xyz> - specify custom kernel parameters during boot
   help, -h, --help: print this text
+  -d, --debug: Enable debug output
+  -q, --quite: Hide most output except error messages
 
 Examples:
   runqemu
@@ -112,7 +114,7 @@ def check_tun():
 
 def check_libgl(qemu_bin):
     cmd = 'ldd %s' % qemu_bin
-    logger.info('Running %s...' % cmd)
+    logger.debug('Running %s...' % cmd)
     need_gl = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8')
     if re.search('libGLU', need_gl):
         # We can't run without a libGL.so
@@ -229,12 +231,12 @@ class BaseConfig(object):
         self.mac_slirp = "52:54:00:12:35:"
 
     def acquire_lock(self):
-        logger.info("Acquiring lockfile %s..." % self.lock)
+        logger.debug("Acquiring lockfile %s..." % self.lock)
         try:
             self.lock_descriptor = open(self.lock, 'w')
             fcntl.flock(self.lock_descriptor, fcntl.LOCK_EX|fcntl.LOCK_NB)
         except Exception as e:
-            logger.info("Acquiring lockfile %s failed: %s" % (self.lock, e))
+            logger.error("Acquiring lockfile %s failed: %s" % (self.lock, e))
             if self.lock_descriptor:
                 self.lock_descriptor.close()
             return False
@@ -259,10 +261,10 @@ class BaseConfig(object):
     def is_deploy_dir_image(self, p):
         if os.path.isdir(p):
             if not re.search('.qemuboot.conf$', '\n'.join(os.listdir(p)), re.M):
-                logger.info("Can't find required *.qemuboot.conf in %s" % p)
+                logger.debug("Can't find required *.qemuboot.conf in %s" % p)
                 return False
             if not any(map(lambda name: '-image-' in name, os.listdir(p))):
-                logger.info("Can't find *-image-* in %s" % p)
+                logger.debug("Can't find *-image-* in %s" % p)
                 return False
             return True
         else:
@@ -281,9 +283,9 @@ class BaseConfig(object):
 
     def set_machine_deploy_dir(self, machine, deploy_dir_image):
         """Set MACHINE and DEPLOY_DIR_IMAGE"""
-        logger.info('MACHINE: %s' % machine)
+        logger.debug('MACHINE: %s' % machine)
         self.set("MACHINE", machine)
-        logger.info('DEPLOY_DIR_IMAGE: %s' % deploy_dir_image)
+        logger.debug('DEPLOY_DIR_IMAGE: %s' % deploy_dir_image)
         self.set("DEPLOY_DIR_IMAGE", deploy_dir_image)
 
     def check_arg_nfs(self, p):
@@ -337,10 +339,10 @@ class BaseConfig(object):
 
         elif os.path.isdir(p) or re.search(':', p) and re.search('/', p):
             if self.is_deploy_dir_image(p):
-                logger.info('DEPLOY_DIR_IMAGE: %s' % p)
+                logger.debug('DEPLOY_DIR_IMAGE: %s' % p)
                 self.set("DEPLOY_DIR_IMAGE", p)
             else:
-                logger.info("Assuming %s is an nfs rootfs" % p)
+                logger.debug("Assuming %s is an nfs rootfs" % p)
                 self.check_arg_nfs(p)
         elif os.path.basename(p).startswith('ovmf'):
             self.ovmf_bios.append(p)
@@ -356,7 +358,7 @@ class BaseConfig(object):
         elif re.search('/', arg):
             raise RunQemuError("Unknown arg: %s" % arg)
 
-        logger.info('Assuming MACHINE = %s' % arg)
+        logger.debug('Assuming MACHINE = %s' % arg)
 
         # if we're running under testimage, or similarly as a child
         # of an existing bitbake invocation, we can't invoke bitbake
@@ -393,6 +395,16 @@ class BaseConfig(object):
             self.set("MACHINE", arg)
 
     def check_args(self):
+        for debug in ("-d", "--debug"):
+            if debug in sys.argv:
+                logger.setLevel(logging.DEBUG)
+                sys.argv.remove(debug)
+
+        for quiet in ("-q", "--quiet"):
+            if quiet in sys.argv:
+                logger.setLevel(logging.ERROR)
+                sys.argv.remove(quiet)
+
         unknown_arg = ""
         for arg in sys.argv[1:]:
             if arg in self.fstypes + self.vmtypes:
@@ -608,7 +620,7 @@ class BaseConfig(object):
                 break
 
         if biosdir:
-            logger.info("Assuming biosdir is: %s" % biosdir)
+            logger.debug("Assuming biosdir is: %s" % biosdir)
             self.qemu_opt_script += ' -L %s' % biosdir
         else:
             logger.error("Custom BIOS directory not found. Tried: %s, %s, and %s" % (self.custombiosdir, biosdir_native, biosdir_host))
@@ -663,7 +675,7 @@ class BaseConfig(object):
             if self.get('DEPLOY_DIR_IMAGE'):
                 deploy_dir_image = self.get('DEPLOY_DIR_IMAGE')
             else:
-                logger.info("Can't find qemuboot conf file, DEPLOY_DIR_IMAGE is NULL!")
+                logger.warn("Can't find qemuboot conf file, DEPLOY_DIR_IMAGE is NULL!")
                 return
 
             if self.rootfs and not os.path.exists(self.rootfs):
@@ -675,7 +687,7 @@ class BaseConfig(object):
                         self.rootfs, machine)
             else:
                 cmd = 'ls -t %s/*.qemuboot.conf' %  deploy_dir_image
-                logger.info('Running %s...' % cmd)
+                logger.debug('Running %s...' % cmd)
                 try:
                     qbs = subprocess.check_output(cmd, shell=True).decode('utf-8')
                 except subprocess.CalledProcessError as err:
@@ -700,7 +712,7 @@ class BaseConfig(object):
         if not os.path.exists(self.qemuboot):
             raise RunQemuError("Failed to find %s (wrong image name or BSP does not support running under qemu?)." % self.qemuboot)
 
-        logger.info('CONFFILE: %s' % self.qemuboot)
+        logger.debug('CONFFILE: %s' % self.qemuboot)
 
         cf = configparser.ConfigParser()
         cf.read(self.qemuboot)
@@ -899,7 +911,7 @@ class BaseConfig(object):
                 pass
 
         cmd = '%s link' % ip
-        logger.info('Running %s...' % cmd)
+        logger.debug('Running %s...' % cmd)
         ip_link = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE).stdout.read().decode('utf-8')
         # Matches line like: 6: tap0: <foo>
         possibles = re.findall('^[1-9]+: +(tap[0-9]+): <.*', ip_link, re.M)
@@ -931,7 +943,7 @@ class BaseConfig(object):
             self.lock = lockfile + '.lock'
             self.acquire_lock()
             self.cleantap = True
-            logger.info('Created tap: %s' % tap)
+            logger.debug('Created tap: %s' % tap)
 
         if not tap:
             logger.error("Failed to setup tap device. Run runqemu-gen-tapdevs to manually create.")
@@ -1150,7 +1162,7 @@ class BaseConfig(object):
     def cleanup(self):
         if self.cleantap:
             cmd = 'sudo %s %s %s' % (self.qemuifdown, self.tap, self.bindir_native)
-            logger.info('Running %s' % cmd)
+            logger.debug('Running %s' % cmd)
             subprocess.check_call(cmd, shell=True)
         if self.lock_descriptor:
             logger.info("Releasing lockfile for tap device '%s'" % self.tap)
@@ -1159,7 +1171,7 @@ class BaseConfig(object):
         if self.nfs_running:
             logger.info("Shutting down the userspace NFS server...")
             cmd = "runqemu-export-rootfs stop %s" % self.rootfs
-            logger.info('Running %s' % cmd)
+            logger.debug('Running %s' % cmd)
             subprocess.check_call(cmd, shell=True)
 
         if self.saved_stty:
Signature · 1 line
2.11.0
#2 lore raw

PATCH 2/5 runqemu: check qbconfload before running bitbake

In reply to Robert Yang

If qbconfload (.qemuboot.conf is found) is present, we can get DEPLOY_DIR_IMAGE from it rather than "bitbake -e".

---
 scripts/runqemu | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/runqemu +1 -1
index 7188bd78830..781814ae651 100755
@@ -466,7 +466,7 @@ class BaseConfig(object):
 
             self.check_arg_machine(unknown_arg)
 
-        if not self.get('DEPLOY_DIR_IMAGE'):
+        if not (self.get('DEPLOY_DIR_IMAGE') or self.qbconfload):
             self.load_bitbake_env()
             s = re.search('^DEPLOY_DIR_IMAGE="(.*)"', self.bitbake_e, re.M)
             if s:
Signature · 1 line
2.11.0
#3 lore raw

PATCH 3/5 runqemu: check tar.bz2 and .tar.gz

In reply to Robert Yang

Handle them as nfs, so that cmd like the following can be boot:

$ runqemu tmp/deploy/images/qemux86/core-image-minimal-qemux86.tar.bz2

[YOCTO #11286]

---
 scripts/runqemu | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
scripts/runqemu +4 -1
index 781814ae651..bdb559f82ff 100755
@@ -218,7 +218,8 @@ class BaseConfig(object):
         self.lock_descriptor = ''
         self.bitbake_e = ''
         self.snapshot = False
-        self.fstypes = ('ext2', 'ext3', 'ext4', 'jffs2', 'nfs', 'btrfs', 'cpio.gz', 'cpio', 'ramfs')
+        self.fstypes = ('ext2', 'ext3', 'ext4', 'jffs2', 'nfs', 'btrfs',
+                        'cpio.gz', 'cpio', 'ramfs', 'tar.bz2', 'tar.gz')
         self.vmtypes = ('hddimg', 'hdddirect', 'wic', 'vmdk', 'qcow2', 'vdi', 'iso')
         self.network_device = "-device e1000,netdev=net0,mac=@MAC@"
         # Use different mac section for tap and slirp to avoid
@@ -277,6 +278,8 @@ class BaseConfig(object):
         if not self.fstype or self.fstype == fst:
             if fst == 'ramfs':
                 fst = 'cpio.gz'
+            if fst in ('tar.bz2', 'tar.gz'):
+                fst = 'nfs'
             self.fstype = fst
         else:
             raise RunQemuError("Conflicting: FSTYPE %s and %s" % (self.fstype, fst))
Signature · 1 line
2.11.0
#4 lore raw

PATCH 4/5 runqemu: validate combos

In reply to Robert Yang

Error out ealier if the combos is invalid, e.g.:

$ runqemu tmp/deploy/images/qemux86/bzImage-qemux86.bin tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic

This will fail at kernel panic, no we check and error out early. We can add other checkings in the future.

[YOCTO #11286]

---
 scripts/runqemu | 6 ++++++
 1 file changed, 6 insertions(+)
scripts/runqemu +6 -0
index bdb559f82ff..e9ed890bb27 100755
@@ -1209,6 +1209,10 @@ class BaseConfig(object):
             self.bitbake_e = ''
             logger.warn("Couldn't run 'bitbake -e' to gather environment information:\n%s" % err.output.decode('utf-8'))
 
+    def validate_combos(self):
+        if (self.fstype in self.vmtypes) and self.kernel:
+            raise Exception("%s doesn't need kernel %s!" % (self.fstype, self.kernel))
+
     @property
     def bindir_native(self):
         result = self.get('STAGING_BINDIR_NATIVE')
@@ -1240,6 +1244,8 @@ def main():
         config.check_args()
         config.read_qemuboot()
         config.check_and_set()
+        # Check whether the combos is valid or not
+        config.validate_combos()
         config.print_config()
         config.setup_network()
         config.setup_rootfs()
Signature · 1 line
2.11.0
#5 lore raw

Re: [PATCH 4/5] runqemu: validate combos

Quoted 18 lines · Robert Yang

Error out ealier if the combos is invalid, e.g.:

$ runqemu tmp/deploy/images/qemux86/bzImage-qemux86.bin tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic

This will fail at kernel panic, no we check and error out early. We can add other checkings in the future.

[YOCTO #11286]

---
 scripts/runqemu | 6 ++++++
 1 file changed, 6 insertions(+)
scripts/runqemu +0 -0
index bdb559f82ff..e9ed890bb27 100755

scripts/runqemu class BaseConfig(object): · line 1214

Quoted 6 lines
scripts/runqemu +3 -0
@@ -1209,6 +1209,10 @@ class BaseConfig(object):
             self.bitbake_e = ''
             logger.warn("Couldn't run 'bitbake -e' to gather environment information:\n%s" % err.output.decode('utf-8'))
 
+    def validate_combos(self):
+        if (self.fstype in self.vmtypes) and self.kernel:
+            raise Exception("%s doesn't need kernel %s!" % (self.fstype, self.kernel))

I just found that here should be raise RunQemuError(), updated it in the repo.

// Robert

Quoted 14 lines
scripts/runqemu +3 -0
+
     @property
     def bindir_native(self):
         result = self.get('STAGING_BINDIR_NATIVE')
@@ -1240,6 +1244,8 @@ def main():
         config.check_args()
         config.read_qemuboot()
         config.check_and_set()
+        # Check whether the combos is valid or not
+        config.validate_combos()
         config.print_config()
         config.setup_network()
         config.setup_rootfs()
#6 lore raw

PATCH 5/5 runqemu: chmod 0o777 for lockdir

In reply to Robert Yang

Multi-users may run qemu on the same host, all of them should be able to create or remove lock in lockdir, so set lockdir's mode to 0o777.

Note, os.mkdir()'s mode is default to 0o777, but the current umask value is first masked out, so use os.chmod() to set it.

---
 scripts/runqemu | 1 +
 1 file changed, 1 insertion(+)
scripts/runqemu +1 -0
index e9ed890bb27..2be27804841 100755
@@ -910,6 +910,7 @@ class BaseConfig(object):
             # running at the same time.
             try:
                 os.mkdir(lockdir)
+                os.chmod(lockdir, 0o777)
             except FileExistsError:
                 pass
 
Signature · 1 line
2.11.0
#7 lore raw
In reply to Robert Yang

Ping.

// Robert

Quoted 19 lines · Robert Yang

The following changes since commit ef68005a8c527e9b1d05b7769f0ec8ebe9ec3f91:

webkitgtk: Upgrade to 2.16.5 (2017-07-17 13:49:04 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/runqemu
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/runqemu

Robert Yang (5):
runqemu: add --debug and --quiet
runqemu: check qbconfload before running bitbake
runqemu: check tar.bz2 and .tar.gz
runqemu: validate combos
runqemu: chmod 0o777 for lockdir

 scripts/runqemu | 64 ++++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 43 insertions(+), 21 deletions(-)