support OEQA inside container
cover support OEQA inside container
Hi,
With these patches it is possible to run bitbake inside a docker container and still successfully execute the OEQA tests that require the target to talk to a temporary http server spawned on the machine running bitake, e.g. using crops.
This wasn't possible so far because:
* we need to map a port from outside the container into the container for the target to be able to reach the http server
* we need to pass the IP address of the docker host to the target, e.g. in the OPKG repository URL, but we can't bind the http server to that IP address, as the http server is running inside the container with a different IP address - the bind would fail
To make this work, TEST_SERVER_IP needs to be set to the IP address of the docker host, and the port that is mapped into the container can now also be specified in TEST_SERVER_IP, similar to TEST_TARGET_IP
Cheers,
Andre'
PATCH 1/4 oeqa/utils/httpserver: allow to pass in listening port
Being able to specify the listening port is useful when running OEQA from within a docker container, e.g. crops or any other solution.
In that case, a port on the outside must be mapped to a specific port inside the container. If no port is specified for the http server module in this case, the http server would choose a random port, which is unlikely to be mapped and thus won't be reachable from the outside.
--- meta/lib/oeqa/utils/httpserver.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
meta/lib/oeqa/utils/httpserver.py +2 -2
@@ -22,10 +22,10 @@ class HTTPRequestHandler(http.server.SimpleHTTPRequestHandler): class HTTPService(object): - def __init__(self, root_dir, host='', logger=None): + def __init__(self, root_dir, host='', port=0, logger=None): self.root_dir = root_dir self.host = host - self.port = 0 + self.port = port self.logger = logger def start(self):
Signature · 1 line
2.24.0
PATCH 2/4 oeqa/runtime/context.py: support listening port in TEST_SERVER_IP
In reply to André DraszikSimilar to the existing possibility to specify a port in TEST_TARGET_IP, allow TEST_SERVER_IP to also contain a port.
The intention is for this port to be passed into e.g. the http server from the apt / dnf / opkg tests, or any other (custom) tests that might need the target to connect to a service spawned by bitbake / oeqa, where bitbake is actually running inside a docker container.
--- meta/lib/oeqa/runtime/context.py | 6 ++++++ 1 file changed, 6 insertions(+)
meta/lib/oeqa/runtime/context.py +6 -0
@@ -98,6 +98,12 @@ class OERuntimeTestContextExecutor(OETestContextExecutor): target_ip = target_ip_port[0] kwargs['port'] = target_ip_port[1] + if server_ip: + server_ip_port = server_ip.split(':') + if len(server_ip_port) == 2: + server_ip = server_ip_port[0] + kwargs['server_port'] = int(server_ip_port[1]) + if target_type == 'simpleremote': target = OESSHTarget(logger, target_ip, server_ip, **kwargs) elif target_type == 'qemu':
Signature · 1 line
2.24.0
PATCH 3/4 oeqa/target/ssh oeqa/target/qemu: expose server listening port to tests
In reply to André DraszikAllow tests to access the listening port as just introduced. Note that when using qemu this infrastructure shouldn't be needed, but we still need to set the port to 0 so that a listening port is determined automatically (e.g. by the python http server).
--- meta/lib/oeqa/core/target/qemu.py | 1 + meta/lib/oeqa/core/target/ssh.py | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)
meta/lib/oeqa/core/target/qemu.py +1 -0
@@ -24,6 +24,7 @@ class OEQemuTarget(OESSHTarget): user, port) self.server_ip = server_ip + self.server_port = 0 self.machine = machine self.rootfs = rootfs self.kernel = kernel
meta/lib/oeqa/core/target/ssh.py +2 -1
@@ -15,7 +15,7 @@ from . import OETarget class OESSHTarget(OETarget): def __init__(self, logger, ip, server_ip, timeout=300, user='root', - port=None, **kwargs): + port=None, server_port=0, **kwargs): if not logger: logger = logging.getLogger('target') logger.setLevel(logging.INFO)
@@ -30,6 +30,7 @@ class OESSHTarget(OETarget): super(OESSHTarget, self).__init__(logger) self.ip = ip self.server_ip = server_ip + self.server_port = server_port self.timeout = timeout self.user = user ssh_options = [
Signature · 1 line
2.24.0
PATCH 4/4 oeqa/runtime/apt dnf opkg: support running from within docker container
In reply to André DraszikIf the user specified an http port to use for serving files, use that instead of a random one. At the same time, have the http server bind to all interfaces.
Binding to the server_ip might not always be possible, e.g. in the case of running bitbake / oeqa from within a docker container. In this case, the ip address is valid outside the container, but not inside, and hence can't be bound to. So switch to simply binding to all interfaces.
--- meta/lib/oeqa/runtime/cases/apt.py | 4 +++- meta/lib/oeqa/runtime/cases/dnf.py | 3 ++- meta/lib/oeqa/runtime/cases/opkg.py | 4 +++- 3 files changed, 8 insertions(+), 3 deletions(-)
meta/lib/oeqa/runtime/cases/apt.py +3 -1
@@ -22,7 +22,9 @@ class AptRepoTest(AptTest): @classmethod def setUpClass(cls): service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all') - cls.repo_server = HTTPService(service_repo, cls.tc.target.server_ip, logger=cls.tc.logger) + cls.repo_server = HTTPService(service_repo, + '0.0.0.0', port=cls.tc.target.server_port, + logger=cls.tc.logger) cls.repo_server.start() @classmethod
meta/lib/oeqa/runtime/cases/dnf.py +2 -1
@@ -53,7 +53,8 @@ class DnfRepoTest(DnfTest): @classmethod def setUpClass(cls): cls.repo_server = HTTPService(os.path.join(cls.tc.td['WORKDIR'], 'oe-testimage-repo'), - cls.tc.target.server_ip, logger=cls.tc.logger) + '0.0.0.0', port=cls.tc.target.server_port, + logger=cls.tc.logger) cls.repo_server.start() @classmethod
meta/lib/oeqa/runtime/cases/opkg.py +3 -1
@@ -25,7 +25,9 @@ class OpkgRepoTest(OpkgTest): if cls.tc.td["MULTILIB_VARIANTS"]: allarchfeed = cls.tc.td["TUNE_PKGARCH"] service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_IPK'], allarchfeed) - cls.repo_server = HTTPService(service_repo, cls.tc.target.server_ip, logger=cls.tc.logger) + cls.repo_server = HTTPService(service_repo, + '0.0.0.0', port=cls.tc.target.server_port, + logger=cls.tc.logger) cls.repo_server.start() @classmethod
Signature · 1 line
2.24.0