Skip to contentSkip to conversation

scripts/autobuild-run: add support for running parallel builds

PATCH New Thomas Petazzoni 1 participant 2 messages Dec 3, 2019 – Dec 4, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 scripts/autobuild-run: add support for running parallel builds

When BR2_PER_PACKAGE_DIRECTORIES=y in the generated configuration, we then use top-level parallel build on 50% of the builds. This will allow to give some testing exposure to this new functionality.

---
 scripts/autobuild-run | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
scripts/autobuild-run +14 -1
index 7e3c7a0..5921edd 100755
@@ -266,6 +266,7 @@ class Builder:
         self.upload = upload
         self.buildpid = buildpid
         self.debug = debug
+        self.build_parallel = False
 
         # frequently needed directories
         self.idir = "instance-%d" % self.instance
@@ -480,6 +481,13 @@ class Builder:
                "-C", self.srcdir, "BR2_DL_DIR=%s" % self.dldir,
                "BR2_JLEVEL=%s" % self.njobs] \
             + self.make_opts.split()
+
+        # when building in parallel, we use both -jX and BR2_JLEVEL=X,
+        # trusting the make job server to properly throttle the total
+        # number of processes.
+        if self.build_parallel:
+            cmd.append("-j%s" % self.njobs)
+
         sub = subprocess.Popen(cmd, stdout=f, stderr=f)
 
         # Setup hung build monitoring thread
@@ -741,7 +749,12 @@ class Builder:
 
             # Check if the build test is supposed to be a reproducible test
             with open(os.path.join(self.outputdir, ".config"), "r") as fconf:
-                reproducible = "BR2_REPRODUCIBLE=y\n" in fconf.read()
+                conf = fconf.read()
+                reproducible = "BR2_REPRODUCIBLE=y\n" in conf
+                per_package = "BR2_PER_PACKAGE_DIRECTORIES=y\n" in conf
+
+            self.build_parallel = per_package and randint(0, 1) == 0
+
             if reproducible:
                 ret = self.do_reproducible_build()
             else:
Signature · 1 line
2.23.0
#1 lore raw

PATCH 1/1 scripts/autobuild-run: add support for running parallel builds

Quoted 8 lines · On Tue, 3 Dec 2019 17:54:50 +0100 Thomas Petazzoni…

When BR2_PER_PACKAGE_DIRECTORIES=y in the generated configuration, we then use top-level parallel build on 50% of the builds. This will allow to give some testing exposure to this new functionality.

---
 scripts/autobuild-run | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

Applied to buildroot-test. Of course, for now, it won't do anything until utils/genrandconfig in Buildroot is patched to use BR2_PER_PACKAGE_DIRECTORIES=y.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com