kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
v3 cover kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
Make use of DEB_BUILD_OPTIONS' parallel option because that is the only parallel build option that is set by default in current dpkg-buildpackage versions.
v2:
* Clarify that this is for current dpkg-buildpackage versions
* Evaluate DEB_BUILD_OPTIONS in debian/rules.
v3:
* Use the Policy snippet which sets MAKEFLAGS and uses patsubst over subst
* Mention the dpkg commit that changed the -j behaviour.
Bastian Germann (1):
kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
scripts/package/mkdebian | 5 +++++ 1 file changed, 5 insertions(+)
Signature · 1 line
2.39.2
PATCH v3 1/1 kbuild: deb-pkg: Build parallely with current dpkg-buildpackage
With dpkg-buildpackage v1.21.10 or later, the only way to build the deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS environment variable or using the --jobs-force option, see dpkg commit 1d0ea9b2ba3f ("dpkg-buildpackage: Change -j, --jobs semantics to non-force mode"). The package ignores the usual parallel build option that is described in Debian Policy.
Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable that ends up being set by dpkg-buildpackage -j<N>. The snippet is copied from Debian Policy.
--- scripts/package/mkdebian | 5 +++++ 1 file changed, 5 insertions(+)
scripts/package/mkdebian +5 -0
@@ -238,6 +238,11 @@ fi cat <<EOF > debian/rules #!$(command -v $MAKE) -f +ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) + NUMJOBS = \$(patsubst parallel=%,%,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) + MAKEFLAGS += -j\$(NUMJOBS) +endif + srctree ?= . build-indep:
Signature · 1 line
2.39.2
Quoted 8 lines · Bastian Germann
With dpkg-buildpackage v1.21.10 or later, the only way to build the deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS environment variable or using the --jobs-force option, see dpkg commit
1d0ea9b2ba3f("dpkg-buildpackage: Change -j, --jobs semantics to non-force mode"). The package ignores the usual parallel build option that is described in Debian Policy.
Quoted 7 lines
Derive make's -j parameter from the
DEB_BUILD_OPTIONSenvironment variable that ends up being set by dpkg-buildpackage -j<N>. The snippet is copied from Debian Policy.Signed-off-by Bastian Germann <[email protected]>---
I was about to pick this up, then I stopped it.
If I apply this patch,
"make -j2 bindeb-pkg" shows a new warning.
make[3]: warning: -j16 forced in makefile: resetting jobserver mode.
Please note nproc returns 16 on my machine.
"make bindeb-pkg" also runs in 16 threads because $(DEB_BUILD_OPTIONS) always contains "parallel=16" even if -j<N> option is given at all.
In other words, this patch will take away user's freedom to choose the number of threads.
Quoted 22 lines
scripts/package/mkdebian | 5 +++++ 1 file changed, 5 insertions(+)scripts/package/mkdebian +5
-1@@ -238,6 +238,11 @@ fi cat <<EOF > debian/rules #!$(command -v $MAKE) -f +ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) + NUMJOBS = \$(patsubst parallel=%,%,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS))) + MAKEFLAGS += -j\$(NUMJOBS) +endif + srctree ?= . build-indep: --2.39.2
Signature · 2 lines
Best Regards Masahiro Yamada
Am 21.03.23 um 11:18 schrieb Masahiro Yamada:
Quoted 6 lines
"make bindeb-pkg" also runs in 16 threads because $(DEB_BUILD_OPTIONS) always contains "parallel=16" even if -j<N> option is given at all.
In other words, this patch will take away user's freedom to choose the number of threads.
Okay. To solve this, I would forward make's -j option to the dpkg-buildpackage calls, which is now called with the defaults.
I think you should see the warning with older dpkg-buildpackage versions without the patch applied as well because their default is dpkg-buildpackage -jnproc and is in effect currently.
Quoted 11 lines · Bastian Germann
Am 21.03.23 um 11:18 schrieb Masahiro Yamada:
Quoted 6 lines
"make bindeb-pkg" also runs in 16 threads because $(DEB_BUILD_OPTIONS) always contains "parallel=16" even if -j<N> option is given at all.
In other words, this patch will take away user's freedom to choose the number of threads.
Okay. To solve this, I would forward make's -j option to the dpkg-buildpackage calls, which is now called with the defaults.
Yes. Agreed.
Quoted 3 lines
I think you should see the warning with older dpkg-buildpackage versions without the patch applied as well because their default is dpkg-buildpackage -j
nprocand is in effect currently.
Signature · 2 lines
Best Regards Masahiro Yamada