Skip to contentSkip to conversation

autobuild: add config option to set niceness

PATCH New Yann E. MORIN 1 participant 2 messages Nov 24, 2015 – Nov 25, 2015

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 autobuild: add config option to set niceness

When the machine running the autobuilder is shared, it is important that the autobuilder instances do not hog all of the CPU.

Add an option to the configuration, so that the user can set the niceness to run the instances with. By default, the niceness is 0, which means the current niceness.

Properly doing so would require setting up a container with proper CPU cgroup, but it's a bit complicated.

---
 scripts/autobuild-run | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
scripts/autobuild-run +9 -1
index 76b7201..fb48950 100755
@@ -64,6 +64,7 @@ defaults = {
     '--njobs': '1',
     '--submitter': 'N/A',
     '--make-opts': '',
+    '--nice': 0,
 }
 
 doc = """autobuild-run - run Buildroot autobuilder
@@ -77,6 +78,8 @@ Options:
                                  Defaults to %(--ninstances)s.
   -j, --njobs NJOBS              number of parallel jobs
                                  Defaults to %(--njobs)s.
+  --nice N                       Niceness, positive number
+                                 No default.
   -s, --submitter SUBMITTER      name/machine of submitter
                                  Defaults to %(--submitter)s.
   --http-login LOGIN             username to send results with
@@ -101,6 +104,7 @@ Format of the configuration file:
    [main]
    ninstances = <value>
    njobs = <value>
+   nice = <value>
    http-login = <value>
    http-password = <value>
    submitter = <value>
@@ -540,6 +544,7 @@ def do_build(**kwargs):
 
     idir = "instance-%d" % kwargs['instance']
     log = kwargs['log']
+    nice = kwargs['nice']
 
     # We need the absolute path to use with O=, because the relative
     # path to the output directory here is not relative to the
@@ -551,7 +556,9 @@ def do_build(**kwargs):
     f = open(os.path.join(outputdir, "logfile"), "w+")
     log_write(log, "INFO: build started")
 
-    cmd = ["timeout", str(MAX_DURATION), "make", "O=%s" % outputdir,
+    cmd = ["timeout", str(MAX_DURATION),
+            "nice", "-n", nice,
+            "make", "O=%s" % outputdir,
             "-C", srcdir, "BR2_DL_DIR=%s" % dldir,
             "BR2_JLEVEL=%s" % kwargs['njobs']] \
           + kwargs['make_opts'].split()
@@ -851,6 +858,7 @@ def main():
                 http_password = args['--http-password'],
                 submitter = args['--submitter'],
                 make_opts = (args['--make-opts'] or ''),
+                nice = (args['--nice'] or 0),
                 upload = upload,
                 buildpid = buildpid
             ))
Signature · 1 line
1.9.1
#1 lore raw

PATCH 1/1 autobuild: add config option to set niceness

Thomas, All,

On 2015-11-24 23:26 +0100, Yann E. MORIN spake thusly:

Quoted 11 lines

When the machine running the autobuilder is shared, it is important that the autobuilder instances do not hog all of the CPU.

Add an option to the configuration, so that the user can set the niceness to run the instances with. By default, the niceness is 0, which means the current niceness.

Properly doing so would require setting up a container with proper CPU cgroup, but it's a bit complicated.

Forget about that one, I'll submit an updated version later tonight, with the docopt help fixed with the real default value, like isdone for the other default values.

Regards,
Yann E. MORIN.

Quoted 63 lines
---
 scripts/autobuild-run | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
scripts/autobuild-run +9 -1
index 76b7201..fb48950 100755
@@ -64,6 +64,7 @@ defaults = {
     '--njobs': '1',
     '--submitter': 'N/A',
     '--make-opts': '',
+    '--nice': 0,
 }
 
 doc = """autobuild-run - run Buildroot autobuilder
@@ -77,6 +78,8 @@ Options:
                                  Defaults to %(--ninstances)s.
   -j, --njobs NJOBS              number of parallel jobs
                                  Defaults to %(--njobs)s.
+  --nice N                       Niceness, positive number
+                                 No default.
   -s, --submitter SUBMITTER      name/machine of submitter
                                  Defaults to %(--submitter)s.
   --http-login LOGIN             username to send results with
@@ -101,6 +104,7 @@ Format of the configuration file:
    [main]
    ninstances = <value>
    njobs = <value>
+   nice = <value>
    http-login = <value>
    http-password = <value>
    submitter = <value>
@@ -540,6 +544,7 @@ def do_build(**kwargs):
 
     idir = "instance-%d" % kwargs['instance']
     log = kwargs['log']
+    nice = kwargs['nice']
 
     # We need the absolute path to use with O=, because the relative
     # path to the output directory here is not relative to the
@@ -551,7 +556,9 @@ def do_build(**kwargs):
     f = open(os.path.join(outputdir, "logfile"), "w+")
     log_write(log, "INFO: build started")
 
-    cmd = ["timeout", str(MAX_DURATION), "make", "O=%s" % outputdir,
+    cmd = ["timeout", str(MAX_DURATION),
+            "nice", "-n", nice,
+            "make", "O=%s" % outputdir,
             "-C", srcdir, "BR2_DL_DIR=%s" % dldir,
             "BR2_JLEVEL=%s" % kwargs['njobs']] \
           + kwargs['make_opts'].split()
@@ -851,6 +858,7 @@ def main():
                 http_password = args['--http-password'],
                 submitter = args['--submitter'],
                 make_opts = (args['--make-opts'] or ''),
+                nice = (args['--nice'] or 0),
                 upload = upload,
                 buildpid = buildpid
             ))
Signature · 1 line
1.9.1
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'