oeqa related: 3 fixes

PATCH In review Robert Yang 2 participants 10 messages Dec 27, 2018 – Jan 2, 2019

lore

Share & tools

lore

Robert Yang

author

2018-12-27 09:00 UTC #0 lore raw

cover oeqa related: 3 fixes

The following changes since commit 7bbc453ac2b8d63680855789948a145fc448017f:

  glibc: Enable --with-default-link (2018-12-26 11:19:47 +0000)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/oeqa
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=rbt/oeqa

Robert Yang (3):
oeqa: make it work for multiple users
runqemu-gen-tapdevs: Allow run --help without sudo oeqa/utils/qemurunner: Print output when failed to login

 meta/classes/testimage.bbclass    |  2 +-
 meta/lib/oeqa/runtime/context.py  |  7 +++----
 meta/lib/oeqa/utils/dump.py       |  5 ++++-
 meta/lib/oeqa/utils/qemurunner.py |  1 +
 scripts/runqemu-gen-tapdevs       | 12 ++++++------
 5 files changed, 15 insertions(+), 12 deletions(-)

Signature · 1 line

2.10.2

Robert Yang

author

2018-12-27 09:00 UTC #1 lore raw

PATCH 1/3 oeqa: make it work for multiple users

There are failures when multiple users run oe-selftest on the same host:

PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-tests/201812250324_qemu'

This is because /tmp/oe-saved-tests was created by user A, while user B tries to write data in it, then the error will happen. This patch can fix the problem.

Signed-off-by Robert Yang <[email protected]>

---
 meta/classes/testimage.bbclass   | 2 +-
 meta/lib/oeqa/runtime/context.py | 7 +++----
 meta/lib/oeqa/utils/dump.py      | 5 ++++-
 3 files changed, 8 insertions(+), 6 deletions(-)

meta/classes/testimage.bbclass +1 -1

index 8a15c30..78b7b5e 100644
@@ -77,7 +77,7 @@ TESTIMAGEDEPENDS += "${@bb.utils.contains('IMAGE_PKGTYPE', 'deb', 'apt-native:do
 TESTIMAGELOCK = "${TMPDIR}/testimage.lock"
 TESTIMAGELOCK_qemuall = ""
 
-TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests/"
+TESTIMAGE_DUMP_DIR ?= "/tmp/oe-saved-tests.${USER}/"
 
 TESTIMAGE_UPDATE_VARS ?= "DL_DIR WORKDIR DEPLOY_DIR"

meta/lib/oeqa/runtime/context.py +3 -4

index 943e29b..96c231d 100644
@@ -6,7 +6,7 @@ import os
 from oeqa.core.context import OETestContext, OETestContextExecutor
 from oeqa.core.target.ssh import OESSHTarget
 from oeqa.core.target.qemu import OEQemuTarget
-from oeqa.utils.dump import HostDumper
+from oeqa.utils.dump import HostDumper, default_host_dumper_dir
 
 from oeqa.runtime.loader import OERuntimeTestLoader

@@ -49,7 +49,6 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
     default_manifest = 'data/manifest'
     default_server_ip = '192.168.7.1'
     default_target_ip = '192.168.7.2'
-    default_host_dumper_dir = '/tmp/oe-saved-tests'
     default_extract_dir = 'packages/extracted'
 
     def register_commands(self, logger, subparsers):

@@ -71,9 +70,9 @@ class OERuntimeTestContextExecutor(OETestContextExecutor):
                 % self.default_server_ip)
 
         runtime_group.add_argument('--host-dumper-dir', action='store',
-                default=self.default_host_dumper_dir,
+                default=default_host_dumper_dir,
                 help="Directory where host status is dumped, if tests fails, default: %s" \
-                % self.default_host_dumper_dir)
+                % default_host_dumper_dir)
 
         runtime_group.add_argument('--packages-manifest', action='store',
                 default=self.default_manifest,

meta/lib/oeqa/utils/dump.py +4 -1

index 5a7edc1..3ce8dd1 100644
@@ -3,8 +3,11 @@ import sys
 import errno
 import datetime
 import itertools
+import getpass
 from .commands import runCmd
 
+default_host_dumper_dir = "/tmp/oe-saved-tests.%s" % getpass.getuser()
+
 class BaseDumper(object):
     """ Base class to dump commands from host/target """

@@ -12,7 +15,7 @@ class BaseDumper(object):
         self.cmds = []
         # Some testing doesn't inherit testimage, so it is needed
         # to set some defaults.
-        self.parent_dir = parent_dir or "/tmp/oe-saved-tests"
+        self.parent_dir = parent_dir or default_host_dumper_dir
         dft_cmds = """  top -bn1
                         iostat -x -z -N -d -p ALL 20 2
                         ps -ef

Signature · 1 line

2.10.2

Robert Yang

author

2018-12-27 09:00 UTC #7 lore raw

PATCH 2/3 runqemu-gen-tapdevs: Allow run --help without sudo

In reply to Robert Yang

Then we can get user's UID and GID rather than hardcode to 1000, e.g.:

$ sudo ../poky/scripts/runqemu-gen-tapdevs 15220 100 4 tmp/sysroots-components/x86_64/qemu-helper-native/usr/bin

Signed-off-by Robert Yang <[email protected]>

---
 scripts/runqemu-gen-tapdevs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

scripts/runqemu-gen-tapdevs +6 -6

index 11de318..67c4fcc 100755
@@ -29,15 +29,10 @@ usage() {
 	echo "<num> is the number of tap devices to create (0 to remove all)"
 	echo "<native-sysroot-basedir> is the path to the build system's native sysroot"
 	echo "e.g. $ bitbake qemu-helper-native"
-	echo "$ sudo $0 1000 1000 4 tmp/sysroots-components/x86_64/qemu-helper-native/usr/bin"
+	echo "$ sudo $0 `id -u` `id -g` 4 tmp/sysroots-components/x86_64/qemu-helper-native/usr/bin"
 	exit 1
 }
 
-if [ $EUID -ne 0 ]; then
-	echo "Error: This script must be run with root privileges"
-	exit
-fi
-
 if [ $# -ne 4 ]; then
 	echo "Error: Incorrect number of arguments"
 	usage

@@ -54,6 +49,11 @@ if [[ ! -x "$TUNCTL" || -d "$TUNCTL" ]]; then
 	usage
 fi
 
+if [ $EUID -ne 0 ]; then
+	echo "Error: This script must be run with root privileges"
+	exit
+fi
+
 SCRIPT_DIR=`dirname $0`
 RUNQEMU_IFUP="$SCRIPT_DIR/runqemu-ifup"
 if [ ! -x "$RUNQEMU_IFUP" ]; then

Signature · 1 line

2.10.2

Robert Yang

author

2018-12-27 09:01 UTC #9 lore raw

PATCH 3/3 oeqa/utils/qemurunner: Print output when failed to login

In reply to Robert Yang

This is useful for debugging.

Signed-off-by Robert Yang <[email protected]>

---
 meta/lib/oeqa/utils/qemurunner.py | 1 +
 1 file changed, 1 insertion(+)

meta/lib/oeqa/utils/qemurunner.py +1 -0

index 3a9ba02..cce95e1 100644
@@ -354,6 +354,7 @@ class QemuRunner:
             else:
                 self.logger.debug("Couldn't login into serial console"
                             " as root using blank password")
+                self.logger.debug("The output:\n%s" % output)
         except:
             self.logger.debug("Serial console failed while trying to login")
         return True

Signature · 1 line

2.10.2

Richard Purdie

2018-12-27 11:32 UTC #8 lore raw

Re: [PATCH 2/3] runqemu-gen-tapdevs: Allow run --help without sudo

In reply to Robert Yang

Quoted 10 lines · Robert Yang

Then we can get user's UID and GID rather than hardcode to 1000, e.g.:
$ sudo ../poky/scripts/runqemu-gen-tapdevs 15220 100 4 tmp/sysroots-
components/x86_64/qemu-helper-native/usr/bin
Signed-off-by Robert Yang <[email protected]>
---
 scripts/runqemu-gen-tapdevs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

This fixes the help output without sudo but breaks the help output under sudo. How about we check SUDO_UID and SUDO_GID? Something like:

uid=id -u
if [ -n "$SUDO_UID" ]; then
uid=$SUDO_UID
fi

Cheers,

Richard

Richard Purdie

2018-12-27 11:37 UTC #2 lore raw

Re: [PATCH 1/3] oeqa: make it work for multiple users

In reply to Robert Yang

Quoted 11 lines · Robert Yang

There are failures when multiple users run oe-selftest on the same host:
PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu'
This is because /tmp/oe-saved-tests was created by user A, while user B tries
to write data in it, then the error will happen. This patch can fix the problem.

Quoted 6 lines

Signed-off-by Robert Yang <[email protected]>

---
 meta/classes/testimage.bbclass   | 2 +-
 meta/lib/oeqa/runtime/context.py | 7 +++----
 meta/lib/oeqa/utils/dump.py      | 5 ++++-
 3 files changed, 8 insertions(+), 6 deletions(-)

Putting this into /tmp isn't good. Why don't we move this to LOG_DIR/runtime-hostdump/?

Cheers,

Richard

Robert Yang

author

2018-12-28 01:58 UTC #3 lore raw

Re: [PATCH 1/3] oeqa: make it work for multiple users

Quoted 19 lines · Richard Purdie

Quoted 11 lines · Robert Yang
There are failures when multiple users run oe-selftest on the same host:
PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu'
This is because /tmp/oe-saved-tests was created by user A, while user B tries
to write data in it, then the error will happen. This patch can fix the problem.
Quoted 6 lines
Signed-off-by Robert Yang <[email protected]>
---
  meta/classes/testimage.bbclass   | 2 +-
  meta/lib/oeqa/runtime/context.py | 7 +++----
  meta/lib/oeqa/utils/dump.py      | 5 ++++-
  3 files changed, 8 insertions(+), 6 deletions(-)

Quoted 2 lines

Putting this into /tmp isn't good. Why don't we move this to LOG_DIR/runtime-hostdump/?

Thanks, make sense, I will send a V2 for patch 1 and 2.

// Robert

Quoted 6 lines

Cheers,
Richard

Robert Yang

author

2018-12-28 07:53 UTC #4 lore raw

Re: [PATCH 1/3] oeqa: make it work for multiple users

In reply to Richard Purdie

Quoted 19 lines · Richard Purdie

Quoted 11 lines · Robert Yang
There are failures when multiple users run oe-selftest on the same host:
PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu'
This is because /tmp/oe-saved-tests was created by user A, while user B tries
to write data in it, then the error will happen. This patch can fix the problem.
Quoted 6 lines
Signed-off-by Robert Yang <[email protected]>
---
  meta/classes/testimage.bbclass   | 2 +-
  meta/lib/oeqa/runtime/context.py | 7 +++----
  meta/lib/oeqa/utils/dump.py      | 5 ++++-
  3 files changed, 8 insertions(+), 6 deletions(-)

Quoted 2 lines

Putting this into /tmp isn't good. Why don't we move this to LOG_DIR/runtime-hostdump/?

I'm afraid that we can't do this, the problem is that we need call get_bb_vars() to get LOG_DIR (d.getVar('LOG_DIR') doesn't work), but get_bb_vars() runs "bitbake -e" which doesn't work when "bitbake <image> -ctestimage" since there is a bitbake server running. We can only get BUILDDIR if we don't call get_bb_vars(), so I think that we have the following 2 solutions:

1) Save it to /tmp/runtime-hostdump

Or:

2) Save it to BUILDDIR/runtime-hostdump

What's your opinion, please ?

// Robert

Quoted 6 lines

Cheers,
Richard

[email protected]

2018-12-28 23:09 UTC #5 lore raw

Re: [PATCH 1/3] oeqa: make it work for multiple users

Quoted 37 lines · Robert Yang

Quoted 22 lines · Richard Purdie
Quoted 14 lines · Robert Yang
There are failures when multiple users run oe-selftest on the same
host:
PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu'
This is because /tmp/oe-saved-tests was created by user A, while user
B tries
to write data in it, then the error will happen. This patch can fix
the problem.
Quoted 6 lines
Signed-off-by Robert Yang <[email protected]>
---
  meta/classes/testimage.bbclass   | 2 +-
  meta/lib/oeqa/runtime/context.py | 7 +++----
  meta/lib/oeqa/utils/dump.py      | 5 ++++-
  3 files changed, 8 insertions(+), 6 deletions(-)
Quoted 2 lines
Putting this into /tmp isn't good. Why don't we move this to LOG_DIR/runtime-hostdump/?
I'm afraid that we can't do this, the problem is that we need call get_bb_vars()
to get LOG_DIR (d.getVar('LOG_DIR') doesn't work), but get_bb_vars() runs
"bitbake -e" which doesn't work when "bitbake <image> -ctestimage" since there
is a bitbake server running. We can only get BUILDDIR if we don't call
get_bb_vars(), so I think that we have the following 2 solutions:

Quoted 7 lines

1) Save it to /tmp/runtime-hostdump
Or:
2) Save it to BUILDDIR/runtime-hostdump
What's your opinion, please ?

I think it should be possible to use LOG_DIR for most of the usecases. It won't work for the "on target" mode but it should be possible elsewhere even if we have to tweak the code a bit to ensure the right variables are passed around?

Cheers,

Richard

Robert Yang

author

2019-01-02 02:04 UTC #6 lore raw

Re: [PATCH 1/3] oeqa: make it work for multiple users

Quoted 42 lines · On 12/29/18 7:09 AM, [email protected] wrote:

Quoted 41 lines · Robert Yang
Quoted 22 lines · Richard Purdie
Quoted 14 lines · Robert Yang
There are failures when multiple users run oe-selftest on the same
host:
PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved- tests/201812250324_qemu'
This is because /tmp/oe-saved-tests was created by user A, while user
B tries
to write data in it, then the error will happen. This patch can fix
the problem.
Quoted 6 lines
Signed-off-by Robert Yang <[email protected]>
---
   meta/classes/testimage.bbclass   | 2 +-
   meta/lib/oeqa/runtime/context.py | 7 +++----
   meta/lib/oeqa/utils/dump.py      | 5 ++++-
   3 files changed, 8 insertions(+), 6 deletions(-)
Quoted 2 lines
Putting this into /tmp isn't good. Why don't we move this to LOG_DIR/runtime-hostdump/?
I'm afraid that we can't do this, the problem is that we need call get_bb_vars()
to get LOG_DIR (d.getVar('LOG_DIR') doesn't work), but get_bb_vars() runs
"bitbake -e" which doesn't work when "bitbake <image> -ctestimage" since there
is a bitbake server running. We can only get BUILDDIR if we don't call
get_bb_vars(), so I think that we have the following 2 solutions:
1) Save it to /tmp/runtime-hostdump
Or:

Quoted 8 lines

Quoted 3 lines
2) Save it to BUILDDIR/runtime-hostdump
What's your opinion, please ?
I think it should be possible to use LOG_DIR for most of the usecases. It won't work for the "on target" mode but it should be possible elsewhere even if we have to tweak the code a bit to ensure the right variables are passed around?

Yes, I agree with that, I will try it.

// Robert

Quoted 6 lines

Cheers,
Richard