Autostart bitbake server

PATCH New Ed Bartosh 1 participant 4 messages Jul 18, 2016

Share & tools

Ed Bartosh

author

2016-07-18 16:32 UTC #1 lore raw

PATCH 1/3 bitbake: xmlrpc: implement check of connection to server

Implemented check_connection function. The purpose of this function is to check if bitbake server is accessible and functional. To check this this function tries to connect to bitbake server and run getVariable command.

This API is going to be used to implement autoloading of bitbake server.

[YOCTO #5534]

Signed-off-by Ed Bartosh <[email protected]>

---
 bitbake/lib/bb/server/xmlrpc.py | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

bitbake/lib/bb/server/xmlrpc.py +17 -0

index 57c59a8..4131b52 100644
@@ -85,6 +85,23 @@ def _create_server(host, port, timeout = 60):
     s = xmlrpc.client.ServerProxy("http://%s:%d/" % (host, port), transport=t, allow_none=True, use_builtin_types=True)
     return s, t
 
+def check_connection(remote, timeout):
+    try:
+        host, port = remote.split(":")
+        port = int(port)
+    except Exception as e:
+        bb.warn("Failed to read remote definition (%s)" % str(e))
+        raise e
+
+    server, _transport = _create_server(host, port, timeout)
+    try:
+        ret, err =  server.runCommand(['getVariable', 'TOPDIR'])
+        if err or not ret:
+            return False
+    except ConnectionError:
+        return False
+    return True
+
 class BitBakeServerCommands():
 
     def __init__(self, server):

Signature · 1 line

2.1.4

Ed Bartosh

author

2016-07-18 16:32 UTC #2 lore raw

PATCH 2/3 bitbake: utils: add BBSERVER to the list of preserved variables

In reply to Ed Bartosh

All environment variables that are not in the list returned by preserved_envvars_exported are cleaned by bb.utils.clean_environment.

Added BBSERVER to the list as we need to access it in bb/main.py after the call of bb.utils.clean_environment.

Signed-off-by Ed Bartosh <[email protected]>

---
 bitbake/lib/bb/utils.py | 1 +
 1 file changed, 1 insertion(+)

bitbake/lib/bb/utils.py +1 -0

index 0a1bf68..00d6474 100644
@@ -574,6 +574,7 @@ def preserved_envvars_exported():
         'TERM',
         'USER',
         'LC_ALL',
+        'BBSERVER',
     ]
 
 def preserved_envvars():

Signature · 1 line

2.1.4

Ed Bartosh

author

2016-07-18 16:32 UTC #3 lore raw

PATCH 3/3 bitbake: main: implement server autostart feature

In reply to Ed Bartosh

If environment variable BBSERVER == 'autostart' bitbake will automatically load server if it's not running yet.

If host and port are in bitbake.lock then bitbake tries to check if server is running and responses to commands and starts new server only if this check fails.

[YOCTO #5534]

Signed-off-by Ed Bartosh <[email protected]>

---
 bitbake/lib/bb/main.py | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

bitbake/lib/bb/main.py +16 -5

index 3fc3ff5..1c32e93 100755
@@ -303,8 +303,10 @@ class BitBakeConfigParameters(cookerdata.ConfigParameters):
 
         # if BBSERVER says to autodetect, let's do that
         if options.remote_server:
-            [host, port] = options.remote_server.split(":", 2)
-            port = int(port)
+            port = -1
+            if options.remote_server != 'autostart':
+                host, port = options.remote_server.split(":", 2)
+                port = int(port)
             # use automatic port if port set to -1, means read it from
             # the bitbake.lock file; this is a bit tricky, but we always expect
             # to be in the base of the build directory if we need to have a

@@ -321,17 +323,18 @@ class BitBakeConfigParameters(cookerdata.ConfigParameters):
                     lf.close()
                     options.remote_server = remotedef
                 except Exception as e:
-                    raise BBMainException("Failed to read bitbake.lock (%s), invalid port" % str(e))
+                    if options.remote_server != 'autostart':
+                        raise BBMainException("Failed to read bitbake.lock (%s), invalid port" % str(e))
 
         return options, targets[1:]
 
 
 def start_server(servermodule, configParams, configuration, features):
     server = servermodule.BitBakeServer()
-    single_use = not configParams.server_only
+    single_use = not configParams.server_only and os.getenv('BBSERVER') != 'autostart'
     if configParams.bind:
         (host, port) = configParams.bind.split(':')
-        server.initServer((host, int(port)), single_use)
+        server.initServer((host, int(port)), single_use=single_use)
         configuration.interface = [server.serverImpl.host, server.serverImpl.port]
     else:
         server.initServer(single_use=single_use)

@@ -445,6 +448,14 @@ def bitbake_main(configParams, configuration):
         server = start_server(servermodule, configParams, configuration, featureset)
         bb.event.ui_queue = []
     else:
+        if os.getenv('BBSERVER') == 'autostart':
+            if configParams.remote_server == 'autostart' or \
+               not servermodule.check_connection(configParams.remote_server, timeout=2):
+                configParams.bind = 'localhost:0'
+                srv = start_server(servermodule, configParams, configuration, featureset)
+                configParams.remote_server = '%s:%d' % tuple(configuration.interface)
+                bb.event.ui_queue = []
+
         # we start a stub server that is actually a XMLRPClient that connects to a real server
         server = servermodule.BitBakeXMLRPCClient(configParams.observe_only,
                                                   configParams.xmlrpctoken)

Signature · 1 line

2.1.4