Skip to contentSkip to conversation

kbuild: deb-pkg: Build parallely with current dpkg-buildpackage

PATCH Superseded Bastian Germann 3 participants 4 messages Mar 15, 2023 Kbuild

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover kbuild: deb-pkg: Build parallely with current dpkg-buildpackage

Make use of DEB_BUILD_OPTIONS' parallel option because that is the only parallel build option that is set by default in current dpkg-buildpackage versions.

v2:
* Clarify that this is for current dpkg-buildpackage versions
* Evaluate DEB_BUILD_OPTIONS in debian/rules.

Bastian Germann (1):
kbuild: deb-pkg: Build parallely with current dpkg-buildpackage

 scripts/package/mkdebian | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
Signature · 1 line
2.39.2
#1 lore raw

PATCH v2 1/1 kbuild: deb-pkg: Build parallely with current dpkg-buildpackage

With dpkg-buildpackage v1.21.10 or later, the only way to build the deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS environment variable or using the --jobs-force option. The package ignores the usual parallel build option that is described in Debian Policy §4.9.1.

Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable that ends up being set by dpkg-buildpackage -j<N>.

---
 scripts/package/mkdebian | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
scripts/package/mkdebian +7 -3
index f74380036bb5..0217790e7a26 100755
@@ -238,11 +238,15 @@ fi
 cat <<EOF > debian/rules
 #!$(command -v $MAKE) -f
 
+ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS)))
+	JOBS = -j\$(subst parallel=,,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS)))
+endif
+
 srctree ?= .
 
 build-indep:
 build-arch:
-	\$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \
+	\$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \$(JOBS) \
 	\$(shell \$(srctree)/scripts/package/deb-build-option) \
 	olddefconfig all
 
@@ -250,10 +254,10 @@ build: build-arch
 
 binary-indep:
 binary-arch: build-arch
-	\$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} intdeb-pkg
+	\$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \$(JOBS) intdeb-pkg
 clean:
 	rm -rf debian/files debian/linux-*
-	\$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} clean
+	\$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \$(JOBS) clean
 
 binary: binary-arch
 EOF
Signature · 1 line
2.39.2
#2 lore raw
Quoted 2 lines · Bastian Germann

With dpkg-buildpackage v1.21.10 or later, the only way to build the

Please include the commit hash.

I guess

Commit 1d0ea9b2ba3f ("dpkg-buildpackage: Change -j, --jobs semantics to non-force mode")

is the one.

Quoted 3 lines

deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS environment variable or using the --jobs-force option. The package ignores the usual parallel build option that is described in Debian Policy §4.9.1.

If a new section is inserted before, the section number '4.9.1' may not be true in the future.

I think you can drop the section number
if you add the Link: tag close to the relevant section.

Quoted 5 lines

Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable that ends up being set by dpkg-buildpackage -j<N>.

might be better - it points to the relevant section.

Quoted 22 lines

build-indep:
build-arch:

scripts/package/mkdebian +3 -4
-       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \
+       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \$(JOBS) \
        \$(shell \$(srctree)/scripts/package/deb-build-option) \
        olddefconfig all
@@ -250,10 +254,10 @@ build: build-arch
 
 binary-indep:
 binary-arch: build-arch
-       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} intdeb-pkg
+       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \$(JOBS) intdeb-pkg
 clean:
        rm -rf debian/files debian/linux-*
-       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} clean
+       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \$(JOBS) clean
 
 binary: binary-arch
 EOF
--
Quoted 2 lines

2.39.2

I prefer appending it to MAKEFLAGS instead of touching every target. (or is there any reason why MAKEFLAGS does not work here?)

The reference code exists in the link you mentioned.

ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))

    NUMJOBS = $(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
    MAKEFLAGS += -j$(NUMJOBS)

endif

Signature · 2 lines
Best Regards
Masahiro Yamada
#3 lore raw
In reply to Bastian Germann
Quoted 10 lines · Bastian Germann

With dpkg-buildpackage v1.21.10 or later, the only way to build the deb-pkg generated package parallely is adding -j<N> to the MAKEFLAGS environment variable or using the --jobs-force option. The package ignores the usual parallel build option that is described in Debian Policy §4.9.1.

Derive make's -j parameter from the DEB_BUILD_OPTIONS environment variable that ends up being set by dpkg-buildpackage -j<N>.

Thanks for the patch v2 and clarifications.

Substitute above link? Or add as a 2nd link?

-Sedat-

Quoted 42 lines
---
 scripts/package/mkdebian | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
scripts/package/mkdebian +7 -4
index f74380036bb5..0217790e7a26 100755
@@ -238,11 +238,15 @@ fi
 cat <<EOF > debian/rules
 #!$(command -v $MAKE) -f
 
+ifneq (,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS)))
+       JOBS = -j\$(subst parallel=,,\$(filter parallel=%,\$(DEB_BUILD_OPTIONS)))
+endif
+
 srctree ?= .
 
 build-indep:
 build-arch:
-       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \
+       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \$(JOBS) \
        \$(shell \$(srctree)/scripts/package/deb-build-option) \
        olddefconfig all
 
@@ -250,10 +254,10 @@ build: build-arch
 
 binary-indep:
 binary-arch: build-arch
-       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} intdeb-pkg
+       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \$(JOBS) intdeb-pkg
 clean:
        rm -rf debian/files debian/linux-*
-       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} clean
+       \$(MAKE) -f \$(srctree)/Makefile ARCH=${ARCH} \$(JOBS) clean
 
 binary: binary-arch
 EOF
--

2.39.2